26 KiB
x2 — the three exit-gate obligations the joint gate found missing
Branch p5/x2, e61d0012..579118a1 (3 commits), off p5/joint@e61d00123c41d74f98cd06b63768be12d9b6348a.
Worktree ~/w7/p5-x2, left clean with submodule links off. Identity
Swung0x48 <swung0x48@outlook.com>; no attribution trailers. No LFS fetch, no adb.
All three obligations are closed, each with an executed red-once. The headline numbers:
| reading | value |
|---|---|
| maximum record bytes on the reduced path and on OpenRA | 784 bytes, row SetVertexAttribDefaults, against a cap of 4 194 304 (RingProducer::MaxRecordBytes(), half of the 8 MiB SEG_CMD) — 0.019 % |
| E2's clear-drop control | armed, read, 29 Clear records dropped — and ssim=1.000000, mismatchPixels=0. It cannot redden OpenRA |
| E2's new draw-drop control | 758 DrawVbo records dropped, ssim=0.000036, mismatchPixels=295296 — red for the knob's own reason |
| E3(e)'s small ring | the 1 MiB lane wrote ≈40 KiB and never wrapped; it now drives 1 314 880 bytes and reports ringwraps=1 (ringpads=0, ringwaits=1) |
1. E2's negative control — why it stayed green, and what replaced it
The finding
MOBILEGL_IPC_E2_DROP_CLEAR was not broken. All four of the joint's hypotheses were tested and
the answer is the first one.
- Is the knob read on that path? Yes.
BuildRemoteEmitTable()reads it and the WARN is in the library's own log. Added here: the count. With the knob armed the library now prints, once per Present,MGPipe: E2 control armed - drop-draw=0 drop-clear=1, 29 records dropped on the wire (draw=0 clear=29), frame 29. 29 Clear records were dropped — the knob is on the live path and the record never reached the ring. - Does it drop before or after the frame's first draw? Before, and it does not matter: it drops every clear in every frame.
- Does the harness compare a frame the clear never affects? Effectively yes, and this is the
cause.
apitrace dump --calls=0-31249 openra.traceover the replayed range counts 30glClear, 30glXSwapBuffers, 788glDrawArrays, 752glBufferSubDataand ZEROglDrawElements(which is also why OpenRA raises noFatal{UnmigratedVerb, "DrawElements"}). The snapshot frame issues its clear at call 30197 and then covers the surface four times over with a terrain layer —glDrawArrays(GL_TRIANGLES, first=56064, count=16128)×4 under a scissor of-24,-24 688×528over a 640×480 surface — before the target call 31249. A frame that overdraws every pixel it clears has a picture that does not depend on the clear.
So the knob's observable is invisible to this trace. That is R-16's exact defect — a gate that cannot go red for its own reason — and it is a property of the workload, not of the emitter.
What shipped
MOBILEGL_IPC_E2_DROP_DRAW=1 (MG_Remote/Client/EmitTables.cpp, beside the clear drop; same
shape — BeforeDrawVerb() still runs, only the record is skipped). The golden is made of that
geometry, so it is the honest "the wire carried the frame" control.
MOBILEGL_IPC_E2_DROP_CLEAR is kept, not retired: it drops a real record, it now publishes the
count it dropped, and it is observable wherever the picture depends on the clear
(ClearThenReadPixelsScenario is the reduced path's target A). What it is no longer allowed to be
is E2's retrace control. Both knobs are still getenv and not MG_Config::Ipc — Config.h is
c0's; the request to adopt them into IpcTable stands (c1-v2 §10 debt 6, unchanged).
scripts/ci/retrace_drop_draw_control.sh is the lane's control (new file; the body of a new step in
.github/workflows/test.yml's retrace-split job, beside the existing pull-library control).
It never accepts a bare threshold failure. Three guards, each a different way for "the picture
went red" to be somebody else's red:
- a non-empty selection, counted before the run (ID-46 finding 8(b)'s hole);
- the SSIM actually fell —
ssim=andssimThreshold=are parsed out of the retrace's own summary and compared withawk, so a loader failure, a missing fixture, a timeout or aFatal{(all non-zero, all not this red) are refused; - the library said it dropped something — its own
MGPipe: E2 control armed … N records dropped on the wireline from the replay's log, with N > 0. Without this, a knob that was never read would redden the picture for some other reason and pass.
The three arms, measured (evidence ~/w7/p5-x2-e2-{baseline,dropclear,dropdraw,stats}/)
| arm | result |
|---|---|
| baseline | passed 1/1, ssim=1.000000, mismatchPixels=0 |
MOBILEGL_IPC_E2_DROP_CLEAR=1 |
passed 1/1, ssim=1.000000, mismatchPixels=0, 29 clears dropped |
MOBILEGL_IPC_E2_DROP_DRAW=1 |
passed 0/1, ssim=0.000036, mismatchPixels=295296, 758 draws dropped |
The real control script was then driven over a real retrace (~/w7/p5-x2-e2-control-live.sh; this
worktree does not configure tools/trace_replay, so a five-line shim answers the control's two
ctest calls with ~/w7/retrace_gate.py — everything the control reads is produced by the library
under test). Verdict:
the dropped draws turned the split retrace red for their own reason (ctest exit 1): 1 selected case(s), ssim 0.000036 < 0.990000, and the library dropped 758 record(s) on the wire
Red once
- The lane: red once by setting
MOBILEGL_IPC_E2_DROP_DRAW=1—ssim 1.000000 → 0.000036,mismatchPixels 0 → 295296. - The control: red once by disabling the drop in the source with the knob still set and still
announced — one token at
EmitTables.cpp'sEmitDrawArrayssite only (line 238,if (g_dropDrawEmission)→if (false && g_dropDrawEmission); the arming WARN 120 lines above was deliberately left alone), rebuilt, control re-run. It refused:::error::the split retrace PASSED with every DrawVbo record dropped on the wire…. Source restored byte-for-byte —sha256daff5fdec035e1acca71before and after — and rebuilt. Driver:~/w7/p5-x2-redcheck.sh e2. - Seven stubbed modes of the control run in
scripts/ci/control_smoke_test.shagainst the real script (16 cases total, was 9): empty selection, the lane green, a red with no ssim summary, a red whose ssim is above threshold, no armed line, an armed knob that dropped zero, and the real thing.
2. R-10's proof obligation — the maximum record bytes, published
PipeWireEncoder::MaxRecordBytesSeen() existed with no consumer outside the codec unit cases, so
BRIEF §8 item 3 had no measurement (joint-v1 §5, "NO MEASUREMENT"). Three channels now carry it:
- the
MGPipe stats:line — a new push-onlyPipeStats::Gaugefamily, published by the client at the frame boundary:maxrec=<bytes> maxcap=<cap> ringwraps=<n> ringpads=<n> ringwaits=<n>, inside the existing#if MOBILEGL_PIPE_PUSHblock (G1 is intact, see §5). They are run totals on a windowed line, deliberately and documented: a windowed maximum reads 0 in every window that did not contain the biggest record, which is a proof obligation that cannot fail. They are published before the Present record is emitted — placed after it, the first summary line of every run readmaxrec=0 maxcap=0, and a zero meaning "not published yet" is printed in the same shape as a zero meaning "nothing crossed the wire". - every split private log, at
ClientSession::Stop, with noMOBILEGL_PIPE_STATSneeded — only 2 of the 21 split entries set it, and R-10 is about the phase, not about two counting lanes:MG_Remote client: wire ledger: maxrec=784 maxrecop=SetVertexAttribDefaults cap=4194304 (0.0% of RingProducer::MaxRecordBytes, half of a 8388608 byte SEG_CMD) cmdbytes=4040 ringwraps=0 ringpads=0 ringwaits=0 emitseq=48. - the row name, because R-10 makes the integrator choose between early chunking and a bigger
ring and that is a decision about a record family. A trace replay never reaches
ClientSession::Stop(measured: the OpenRA lane's library log simply ends), so a stats-enabled run also logsMGPipe: wire ledger: new maximum record - maxrec=… maxrecop=… cap=…whenever the maximum grows — bounded by the number of distinct maxima, one in a whole OpenRA replay.
The numbers (BRIEF §8 item 3)
| workload | maxrec | row | cap | % of cap | cmd bytes | records |
|---|---|---|---|---|---|---|
TriangleScenario.TheSameVboAndVaoRedrawAcrossAFrameBoundary (split, inproc) |
784 | SetVertexAttribDefaults |
4 194 304 | 0.019 % | 4 040 | 48 |
PersistentCoherentMapScenario.AWriteAfterAFrameBoundaryReachesTheNextFramesDraw |
784 | SetVertexAttribDefaults |
4 194 304 | 0.019 % | 4 904 | 59 |
OpenRA DirectGLES retrace, inproc, 25 frames |
784 | SetVertexAttribDefaults |
4 194 304 | 0.019 % | — | — |
DirectGLES.Split.SmallRing.TriangleScenario.… (1 MiB ring) |
784 | SetVertexAttribDefaults |
524 288 | 0.15 % | 1 314 880 | 24 323 |
Probes ~/w7/p5-x2-probe-{triangle,pmap}.{log,out}; OpenRA in ~/w7/p5-x2-e2-stats/. The same two
probes reproduce the joint's other readings unchanged: Triangle pmap=0.00 mpr=0 rsp=35, persistent
map pmap=600.00 mpr=1 rsp=35; OpenRA rsp=2690 over a 5-frame window.
c1-v2 §10 guessed wrong about which row to watch. It named SetGlobalConstants
("per program per frame, unbounded length"); the measured maximum on the reduced path and on OpenRA
is SetVertexAttribDefaults, whose var-tail is bounded by GL_MAX_VERTEX_ATTRIBS. R-10 needs no
chunking in P5 and nothing is near the cap: 784 bytes against 4 MiB is a factor of 5 349.
Assertions
- the existing codec case
MaxRecordBytesSeenStaysFarBelowHalfTheRingkeepsEXPECT_LT(largest, Cmd().MaxRecordBytes())and gainsMaxRecordBytesCap() == Cmd().MaxRecordBytes() == kCmdBytes / 2— everything outsideMG_Remotecompares against the encoder's published cap rather than againstMOBILEGL_IPC_RING_MB / 2, and that number moved twice in this phase without the environment variable changing; - one integration entry under inproc:
Harness/WireLedgerChecks.h'sExpectMaxRecordBytesUnderCap, called from both split arms ofTriangleScenario.TheSameVboAndVaoRedrawAcrossAFrameBoundary. It assertsmaxrec > 0as well asmaxrec < cap— a maximum of zero satisfies< capperfectly and is the shape of an emit table that resolved the transport and then fell through to the driver. Both values go into the JUnit properties.
Red once
PipeWireCodecTest.ARecordLargerThanHalfTheRingIsFatalRingOverrun (new, forked — never
EXPECT_DEATH, per that file's rule). A real DrawVbo with 2 736 MGPDrawRange tails over the
fixture's 64 KiB SEG_CMD exceeds the 32 KiB cap and dies:
MGPipe: Fatal{RingOverrun, "DrawVbo"} a <N> byte record exceeds RingProducer::MaxRecordBytes() == 32768 (half of a 65536 byte SEG_CMD); P5 does not chunk (R-10) - report it to the integrator, who decides between early chunking and a bigger default ring
The case asserts all three of Fatal{RingOverrun,, exceeds RingProducer::MaxRecordBytes() and
does not chunk (R-10) — the last because the person reading that line has to make a decision that
is not a local fix. The knob route is not available: MOBILEGL_IPC_RING_MB floors at 1 (cap
512 KiB) and the largest record on the reduced path is 784 bytes, so no environment value can
shrink the ring under a real record.
3. E3(e) — the SmallRing lane proved nothing, and now it does
The finding, measured before anything was asserted
A whole split scenario writes about 4 KiB into SEG_CMD (emitseq 22–71, cmdbytes 4 040–4 904,
maxrec 784). The lane's MOBILEGL_IPC_RING_MB=1 is 250× the traffic. The head never came near
its wrap boundary, so DirectGLES.Split.SmallRing.* and DirectGLES.Split.* were the same run under
two names — exactly what ID-53's own comment and t1-v1's debt warned about
("WHAT THIS LANE DOES NOT YET ASSERT").
A second finding, which changed what is asserted
The first cut asserted the kRecPad count — R-9's "a pad record does not advance seq, both sides must skip it and count again". Driving 1 310 824 bytes through the 1 MiB ring produced one and a half trips round the ring and ZERO pads. The reason is arithmetic, not a defect: a workload whose records repeat at a uniform stride that divides a power-of-two capacity lands on the boundary exactly, every time. A gate on the pad count would have been a gate on the sizes in the record catalogue, red today and green the day one of them changed.
So three counters are kept and only one is load-bearing:
ringwraps=— the head crossed a multiple of the capacity (the ring went round). Asserted: it is guaranteed once the workload writes more bytes than the ring holds.ringpads=— a record straddled the boundary and got a filler. Recorded, not asserted.ringwaits=— SEG_STAGE allocations that only fitted after the encoder reclaimed what the server had retired: the producer's progress depended onretiredSeq. This is P5's only real back-pressure event, and the command ring has none by construction — the verb barrier makesEmitAndWaitwait forappliedSeqafter every record (R-1), so at most one record is ever in flight on SEG_CMD and a full command ring is not a wait butFatal{RingOverrun}. Publishing a "command-ring waits" counter that can only ever read zero-or-dead would be a decoration.
What shipped
TriangleScenario.TheSameVboAndVaoRedrawAcrossAFrameBoundary, in the SmallRing lane only, drives
ordinary GL (clear + VBO-backed draw, no readback — a readback is a SEG_REPLY round trip per
iteration) until the producer has written more than 1.25 MiB, then asserts. Nothing touches the ring
directly: the loop's only input is the producer's own head cursor through
Harness/SplitRuntimePeek, and its only output is GL the scenario already issues. The assertion
prints the denominator with the count, because "0 wraps" is a defect after 1.25 MiB through a
1 MiB ring and a tautology after 4 KiB.
Measured, green: cmdbytes=1314880 ringwraps=1 ringpads=0 ringwaits=1 emitseq=24323, cap 524 288.
Cost: that one entry goes from 0.31 s to 0.68 s.
Red once
~/w7/p5-x2-e3e-redcheck.sh 8 re-runs that entry's own command and environment with
MOBILEGL_IPC_RING_MB raised from the lane's floor of 1 to the default 8, nothing else changed.
The assertion names the missing wrap:
exit gate E3(e) - 1310824 bytes were written into a 8388608 byte SEG_CMD and the producer's head NEVER WENT ROUND: no wrap, so this entry exercised exactly what the default lane exercises and the word SmallRing in its name asserts nothing…
followed by its own explanation (the denominator). …redcheck.sh 1 is green on the same binary.
The order is deliberate: the missing wrap is the gate, the byte count is the reason, and an ASSERT
on the denominator would have printed only the reason — which is how the first version of this
red-check read, and why it is written this way round.
4. E3(a)'s named diagnostic
PersistentMapTracker::PushBlocksFor returned silently at blockBytes == 0, so E3(a)'s red could
only ever be a pixel assertion (joint-v1 §3; ID-65 assigned the line here). It now says so, once
per process (the function runs at every validate point of every member; an unlatched log would be
one line per draw per mapping, and a control that greps a log cannot tell a message that fired from
a message that flooded):
MGPipe: persistent-map push disabled - MOBILEGL_IPC_PERSISTENT_BLOCK_KB=0 is exit gate E3(a)'s NEGATIVE CONTROL, not 'unlimited': a live persistent WRITE mapping's dirty blocks are NOT being pushed, so the server draws whatever bytes last crossed by some other route. A lane that stays green with this set is not getting its pixels from the push
MGLOG_W, not a Fatal: 0 is a legal configured value whose whole purpose is to keep running with
the push off, and aborting would turn every E3(a) entry into a subprocess abort and take the pixel
evidence with it.
scripts/ci/split_negative_controls.sh's run_control gains a fourth argument, a private-log
evidence regex, and E3(a) now requires both halves: the scenario's own assertion in ctest's
output and this line in the entry's private file. split_log_paths.py evidence gains an optional
label so its failure message names the control (E1's wording is unchanged byte-for-byte — the
private-log smoke test greps for that sentence).
Measured on the real lane: baseline 19 passed / 0 failed / 2 skipped; E1 turned 14 entries red with
the barrier Fatal in a fresh private file; E3(a) turned 6 red and its private-log evidence came
from DirectGLES.Split.PersistentCoherentMapScenario.TwoWritesThroughTheCoherentPointerEachReachTheirOwnDraw.
Red once
Two ways, both executed.
- Source: one token removed the line (
if (!m_blockZeroAnnounced)→if (false && !m_blockZeroAnnounced)), rebuilt, control re-run: E1 still passed and E3(a) refused —SplitLogPaths FAILED: negative control E3(a) (MOBILEGL_IPC_PERSISTENT_BLOCK_KB=0) FAILED: no selected private log carries /MGPipe: persistent-map push disabled - MOBILEGL_IPC_PERSISTENT_BLOCK_KB=0/. Restored byte-for-byte,sha256f20d5a02a17dc92a28d4before and after. Driver~/w7/p5-x2-redcheck.sh e3a. - Stub: a new
e3-no-privatemode inscripts/ci/testdata/stub_ctest.shdrives the real control with the pixel assertion present and the library line absent;split_private_log_smoke.shis 6 cases now (was 5) and requires that refusal by name.
5. An incidental finding: the control-smoke red-check never perturbed the split control
On this base, scripts/ci/redcheck_control_smoke_test.sh looked for an evidence check on a line
starting with if ! . The split control's has been elif ! tr -s … | grep -qE "${evidence}" since it
was written, so the perturbation found zero checks in it, the Python SystemExit was not checked
by the caller, and the run went on to smoke-test the unperturbed controls and print
"RED-CHECK FAILED: the controls accept an unrelated failure again and the smoke test still passed" —
a true sentence about a perturbation that never happened. It failed safe, so nothing was ever
silently proved, but the control it was most about was never exercised. Reproduced on
p5/joint@e61d0012 before the fix.
Fixed here, and extended: the perturbation now accepts elif, keeps the keyword when it rewrites the
line (turning an elif into an if splits the chain and leaves a dangling fi), aborts loudly when
it does not apply, and reverts seven checks across three controls — including E1's private-log
read and E3(a)'s new one, without which the smoke test's "unrelated failure" case never flips,
because the other guards catch it. P5_T1_CONTROL_SMOKE_REDCHECK_OK: 11 passed / 5 failed, red on
exactly the five cases the evidence checks pin.
This file may also have been touched by p5/j0's rounds 2–3 (cc20de37), which the joint did not
merge. Merge care is needed there.
6. Acceptance, all measured on 579118a1
Runner ~/w7/p5-x2-accept.sh, transcript ~/w7/p5-x2-accept.log.
| gate | result |
|---|---|
G1 symbol_report.py --threshold 0 vs ~/w7/p5-before-libMobileGL.so |
.text 10806611 → 10806611 (+0, +0.000%), 27814 → 27814 defined symbols: 0 added, 0 removed, 0 resized, 0 renamed |
G1-split nm |
pull 0 MG_Remote symbols, split 623 |
| unit pull / push / split | 1816 / 1816 / 2040 (2038 + 2 new codec cases), 0 failed |
| G2 pull-vs-push name diff | 0 |
G14 vs p5-before-ctest-names.txt (comm) |
0 removed, 42 added — the joint's 42, unchanged |
push integration-gpu |
1128 / 1128 |
split integration-gpu (monolith) |
1149 / 1149 |
integration-split (inproc) |
21 total, 19 ran, 2 skipped by design, 0 failed |
| E2 OpenRA retrace, inproc | baseline 1/1 pass, ssim=1.000000; RED under the control, ssim=0.000036 |
G5 p3a_untouched_regions.sh / p4a_untouched_regions.sh vs ff2994d9 |
rc=0, byte-identical, both |
gen_pipe.py --check / dirty-surface / field-ownership |
up to date, all three |
control_smoke_test.sh |
16 / 16 (was 9) |
testdata/split_private_log_smoke.sh |
6 / 6 (was 5) |
redcheck_control_smoke_test.sh |
P5_T1_CONTROL_SMOKE_REDCHECK_OK |
All of this is push/split-only, and G1 proves it: every new counter, gauge, name-table entry and
format-line field sits inside #if MOBILEGL_PIPE_PUSH in PipeStats.{h,cpp} — the only file this
package touches that is compiled into the pull build — and everything else is under MG_Remote/,
MG_IntegrationTest/, MG_Test/Wire/ or scripts/. .text +0 and 0/0/0/0 are the measurement, not
the argument.
7. Files touched, and why
| file | why |
|---|---|
MG_Remote/Wire/PipeWireCodec.{h,cpp} |
w1's. MaxRecordBytesCap(), MaxRecordOpName(), CmdWraps(), CmdWrapPads(), CmdBytesWritten(), StageReclaimWaits(); both wrap readings derived from the producer's own cursor rather than from a flag Reserve does not return, so no Transport/ file was touched |
MG_Remote/Client/EmitTables.{h,cpp} |
c1's. MOBILEGL_IPC_E2_DROP_DRAW, the dropped-record count and its per-frame line, the gauge publication at the frame boundary |
MG_Remote/Client/ClientSession.{h,cpp} |
c1's. LogWireLedger() at teardown |
MG_Remote/Client/PersistentMapTracker.{h,cpp} |
c1's directory (b1's logic). E3(a)'s named line, latched once |
MG_Util/Metrics/PipeStats.{h,cpp} |
not in the named grant — but "add it to the library's MGPipe stats: line" is where that line lives. Push-guarded throughout; G1 .text +0 |
MG_Test/Wire/PipeWireCodecTest.cpp |
the codec test the brief names, +2 cases |
MG_IntegrationTest/Harness/{SplitRuntimePeek.*,WireLedgerChecks.h}, Scenarios/TriangleScenario.cpp |
the lanes |
MG_IntegrationTest/Harness/split_log_paths.py |
j0's control helper (granted file): the optional label |
scripts/ci/{retrace_drop_draw_control.sh,split_negative_controls.sh,control_smoke_test.sh,redcheck_control_smoke_test.sh,testdata/*} |
the controls |
.github/workflows/test.yml |
one new step in retrace-split, the E2 step the brief points at |
No new integration-split or integration-gpu ctest entry was registered — the joint pinned
19 ran / 2 skipped and 1128 / 1149, and the two readings were added to an existing case rather
than to a new one. tools/trace_replay was not touched: the control needs no replay flag.
8. For the integrator
MOBILEGL_IPC_E2_DROP_CLEARis kept but is no longer E2's retrace control. If you would rather retire it, the one thing that would be lost is a cheap perturbation for scenarios whose picture does depend on the clear; nothing in CI depends on it today.- Both E2 knobs are still
getenv. Adopting them intoMG_Config::Ipcis aConfig.hedit, i.e. c0's. c1 asked once (c1-v2 §10 item 6); x2 asks again, now that there are two of them. ringwaitson SEG_CMD is structurally zero while R-1's barrier is armed. The publishedringwaitsis the SEG_STAGE reclaim wait, and it is non-zero on real workloads (OpenRA: 11). When the barrier retires family by family in P6, a command-ring wait becomes reachable and wants its own counter.- The SmallRing lane's ring floor is
MOBILEGL_IPC_RING_MB=1and that is 250× the reduced path's traffic. x2 closed the gap by making the workload bigger, which is what was available; the alternative — a KiB-granularity ring knob — is aConfig.h/ConfigLoader.cppchange and was not taken unilaterally. Worth a ruling before P6 adds lanes. - R-10 is closed and comfortable: 784 bytes against a 4 MiB cap, row
SetVertexAttribDefaults. The MEASUREMENTS row for BRIEF §8 item 3 can be written from §2's table. redcheck_control_smoke_test.shcollides withp5/j0@cc20de37if j0's later rounds touched it. Take x2's version's matcher (theelif/keyword fix and the abort-on-no-op) whichever way the merge goes — without it that script has never perturbed the split control.- The local gate is not updated.
~/w7/notes/tools/wsl_p5_gate*.shlive outside the repo and are shared; the drop-draw control is wired into CI only. One line beside the existingretrace_pull_library_control.shinvocation would add it, and the gate's E2 section is the right place. - A retrace process never reaches
ClientSession::Stop— measured, the OpenRA lane's library log just ends — so the teardown ledger is absent from retrace logs by design, and the stats-enabled "new maximum record" line is what covers that lane.
9. Retained evidence and reproducers
~/w7/p5-x2-accept.{sh,log},~/w7/p5-x2-symbols.md,~/w7/p5-x2-isplit.{log,xml},~/w7/p5-x2-igpu-{push,split-mono}.log,~/w7/p5-x2-names-{linux,push}.txt.~/w7/p5-x2-e2.shand~/w7/p5-x2-e2-{baseline,dropclear,dropdraw,stats}/(+ the four.out).~/w7/p5-x2-e2-control-live.shand~/w7/p5-x2-e2-live/— the real control over a real retrace.~/w7/p5-x2-redcheck.sh(e2,e3a) and~/w7/p5-x2-redcheck-{e2,e3a}.orig,~/w7/p5-x2-controls.out,~/w7/p5-x2-controls-red.out,~/w7/p5-x2-controls{,-red}/.~/w7/p5-x2-e3e-redcheck.shand~/w7/p5-x2-e3e-redcheck-ring{1,8}.log.~/w7/p5-x2-probe.shand~/w7/p5-x2-probe-{triangle,pmap}.{log,out}.~/w7/p5-x2-trace/dump.txt— theapitrace dumpcensus ofopenra.traceover calls 0–31249.~/w7/p5-x2-{setup,submodlinks,build}.sh,~/w7/p5-x2-build-*.log.
Every uncited ~/w7/p5-x2-* temporary was deleted individually; no glob rm was used while other
jobs were writing logs (a concurrent p5-v1/build-split was running throughout).