Two things land together: the plan and contract draft P6 needs before it starts, and the
working notes that until now existed on exactly one machine.
1. P6's plan and contract draft (neither started; not one line of code)
docs/Disaggregated/P6-SPAWN-PLAN.md the package plan
docs/Disaggregated/P6-CONTRACT-DRAFT.md the contract draft (moves to
MG_Remote/CONTRACT-P6.md when c6 lands)
The three findings that mattered, all written into the documents:
a. P6 is SMALLER than its roadmap cell suggests. The transport primitives are already in
the tree and already tested: SocketDoorbell including hangup detection and the two
lost-wakeup fences (Doorbell.cpp:211-360, four cases in FdPassingTest), SCM_RIGHTS fd
passing, ShmSegment::Adopt, and the SurfaceOp / WindowKind schema already defined in
protocol.fbs. inproc uses a real second shm mapping rather than an alias, and
SessionRings.h:22-28 says that was done so today is not the first time it runs. P6 is
assembly plus a control plane plus a process, not "write a socket transport".
b. The evidence for "P6 is only a transport swap" has EXPIRED. It was bought by P5c, whose
audit found 59 direct memory accesses in a tree that already looked clean. P5e changed
the tree again. So the first package, a6, is a read-only audit that writes no code, and
every contract row that depends on it is marked PENDING a6 rather than decided in
advance.
c. P6 does not get a game running on the phone. ANativeWindow* is meaningless in the child,
and protocol.fbs:186-196's own comment says Android transfers the window out of band -
that band is P12. P6 lands pbuffer / surfaceless / offscreen, which is what HeadlessGL,
the split-equivalent lane and trace replay need, and which is exactly what P6's own exit
gate measures. A real window reaching the server is a named refusal. This gets its own
section because "P6 is done" would otherwise read as "the game runs now".
The contract adds rule G: a role may not name a process-local handle. It extends P5c's
rule E (a role may not name the other role's memory) from memory to identity - EGLDisplay /
EGLSurface / EGLContext / ANativeWindow* / fd number / pid / mapped address either cross as
a token this contract defines or they do not cross, and a descriptor crosses only through
SCM_RIGHTS, where the kernel renumbers it.
What P5e left for P6 gets its own section. The latch semantics are already designed
(ARCHITECTURE.md:504) and are copied unchanged. What is new is that after run-ahead the
client no longer waits while the peer process can DIE: the client holds records that were
published and will never be applied, and Android's freezer suspends the child without
closing anything. So slow and dead must be distinguishable WITHOUT a timeout threshold.
That is P6's only new design; everything else is carriage. It shares a predicate with E1's
outstanding re-specification (ID-122), and the two should be designed together.
Two gates carried over from P5e's lessons: G2 requires integration-spawn's test-name set to
match integration-split's name for name, because a differing set means the lane is quietly
running less; and the arm-proof gate requires every entry to record the child pid and
transport=spawn in its OWN private log - ID-124 established that this kind of false green
really happens, and a spawn lane that actually ran monolith would pass every other gate.
S2's shape must be settled before it is written, and the document says why: E1 is the
cautionary case.
2. The working notes move into the repository (docs/Disaggregated/notes/, 415 .md files)
ROADMAP / CURRENT_STAGE_PROGRESS / MEASUREMENTS cite ruling numbers, audit line numbers and
the provenance of gate figures from ~/w7/notes/, a directory that exists on one machine and
is under no version control. A record cited hundreds of times that lives only locally is not
a record, so it moves in whole, with its directory structure intact.
Only .md moves. Scripts, run logs, census and gate result JSON, patches, APKs and nm dumps
stay where they are: they are evidence DATA rather than documents - large, regenerated per
run, and mostly carrying absolute paths and device serials that mean something only on the
machine that produced them. The mapping and this split are written in
docs/Disaggregated/notes/README.md.
Two exceptions are recorded there too: P6's BRIEF has been promoted to a first-class
document and keeps no copy under notes/, so the two cannot drift apart; and recovered/ holds
material salvaged from failed workflow runs, which may duplicate the like-named phase
directories and is kept because a few of those were the only complete version at the time.
CI's citation lint takes docs/Disaggregated/*.md only (non-recursive), and notes/ is
deliberately outside it: those notes' file:line citations were written against their own
code heads, and resolving them against today's HEAD would fail wholesale - which is not an
error, it is their correct state as a historical record. To check one, use the revision it
declares (--rev).
3. Inconsistencies fixed in passing
- README.md's status line still said P5e was in progress while the stage table already said
closed.
- CURRENT_STAGE_PROGRESS's next-step item 1 still listed the exit gate as pending, though
(a) and (b) were run in the previous commit; rewritten to closed plus the one item that
actually remains (E1).
- ROADMAP's P6 row still had the pre-phase plan in three of its cells; it now carries the
package order, the scope boundary and the two new gates.
Checked with check_doc_citations.py: 125 citations across docs/Disaggregated/*.md, 0 problems.
39 KiB
P2 measurement & gating: what exists, what is missing, what it costs
Scout pass over C:/Users/geekerwan/AndroidStudioProjects/FoldCraftLauncher/MobileGL-disagg
(branch feat/disaggregated @ e7a6a72f). Read-only. Every claim below carries a
file:line I actually opened. Paths are repo-relative to that worktree unless absolute.
0. What P2 is obliged to produce (the target the infra has to hit)
- P2 acceptance row:
docs/Disaggregated/ROADMAP.md:18— 「集成 × 2 后端 × {pull, push} 逐名相同; 40 trace push 下 SSIM ≥ 0.99 双后端;verify 零分歧;HandleRecycleScenario绿且重键前红; G7 测试绿且拿掉一个字段能红;两台设备配对逐线程 CPU p50/p99 不差且 tracker 绝对 ns 在上限内; Blaze3D blend-toggle 微基准;CSO 内容寻址关闭的负面对照」. - Day-43 GO/NO-GO checklist:
docs/Disaggregated/ROADMAP.md:42-58. The seven[ ]items are at:46-52; the pass/fail wording at:56is 「两台设备 p50 与 p99 逐线程 CPU 增量都不为负; tracker 绝对 ns 在上限内;Track H 单位成本不超出估计的 50%」, and:57says NO-GO does not roll back. - The five-part validation gate this replaces byte-identity with:
docs/Disaggregated/ARCHITECTURE.md:497-507; part 4 (performance) isARCHITECTURE.md:504: 「两台设备 reboot-clean、同热窗口、配对 A/B,tools/bench.sh+ trace replay--benchmark逐帧 JSON; 指标是逐线程 CPU 时间,p50 与 p99;绝对阈值 … Blaze3D blend-toggle 微基准单列; 关掉 CSO 内容寻址的负面对照」.- Doc bug worth fixing in the brief:
ARCHITECTURE.md:504namestools/bench.sh. The script is attools/device_bench/bench.sh(tools/device_bench/README.md:1, usage at:36-38). There is notools/bench.sh.
- Doc bug worth fixing in the brief:
- Milestone 「第 43 天(P2 出口):GO/NO-GO」 —
docs/Disaggregated/ROADMAP.md:39. - Open question #1 (P2's headline number) —
docs/Disaggregated/ROADMAP.md:74: 「client 侧 dirty 走查的 真实每 draw CPU 代价 … 逐线程 CPU + 绝对 ns,两台设备」. - The absolute-ns ceiling's source: steady-state pull cost is 6.5–9.3 accessor calls + memo probes
per draw (
docs/Disaggregated/MEASUREMENTS.md:64, table at:50-58). A relative/noise threshold would pass trivially — that is exactly whyARCHITECTURE.md:504demands an absolute ns cap.
1. Per-thread CPU measurement infrastructure
1.1 The blunt finding: there is no per-thread CPU-time instrumentation in the tree
Greps over MobileGL/, tools/, android-plugin/, scripts/ for
CLOCK_THREAD_CPUTIME_ID | CLOCK_PROCESS_CPUTIME_ID | getrusage | RUSAGE_THREAD | /proc/self/task
return zero hits (verified: grep -rnE ... --include=*.cpp --include=*.h --include=*.py --include=*.sh MobileGL tools android-plugin scripts → empty). There is likewise no
pthread_setname_np / prctl(PR_SET_NAME) anywhere under MobileGL/, so MobileGL's own worker
threads (ShaderCompilePool) carry generic names in any external profiler.
Consequence: the day-43 metric that the whole GO/NO-GO hangs on (ROADMAP.md:56) has no
first-party collector today. Everything below is either wall-clock or a sampling profiler.
1.2 What does exist
(a) PipeStats — the MGPipe boundary counters (P0, landed).
- Header/contract:
MobileGL/MG_Util/Metrics/PipeStats.h(whole file). Byte classesPipeStats.h:46-77, call classes:81-102, the six memo gates:107-122, payload histogram:129, default summary period 120 frames:132, theEnabled()latch:139-141. - Site inventory (the contract for what is and is not counted):
MG_Util/Metrics/PipeStats.cpp:16-100.accessor-callsis a static tally at ~10 hot entry points, a LOWER BOUND, not a wrapper over all sites (PipeStats.cpp:76-88); readingacc/drawoutside a draw-dominated window is called out atPipeStats.cpp:93-99. - Counters are
std::atomic<Uint64>with relaxed adds (PipeStats.cpp:111-122); off-path cost is one global load + a predicted branch (PipeStats.h:25-36). - Lifecycle:
PipeStats::Init()atMobileGL/Init.cpp:114(right after config load);PipeStats::Shutdown()atMobileGL/Init.cpp:50(first thing in teardown; emits final line + JSON). - Frame hook
OnPresent()PipeStats.cpp:305-349, called fromMG_Backend/DirectGLES/DirectGLES.cpp:10596andMG_Backend/DirectVulkan/DirectVulkan.cpp:1339. - 63 counting call sites across 8 backend files (
grep -c "PipeStats::Add|PipeStats::CountGate" MobileGL/MG_Backend= 63; files:DirectGLES/{DirectGLES,Managers,MultiDraw}.cpp,DirectVulkan/{DirectVulkan,Renderer/UniformManager,Renderer/VkBufferManager, Renderer/VkTextureManager,Renderer/VulkanRenderer}.cpp). - Unit coverage:
MobileGL/MG_Test/Util/PipeStatsTest.cpp(16TEST_Fs, 287 lines) — includingSummaryLineCarriesEveryClassAndGate:137,SummaryLinesReportDisjointWindows:174,CounterNamesAreStable:260,SummaryPeriodIsOneHundredAndTwentyFrames:280. Any new counter class P2 adds must extend these (CounterNamesAreStablepins the strings). PipeStatsmeasures counts, never time. There is no ns/CPU counter in it today.
(b) Tracy. MOBILEGL_ENABLE_TRACY option CMakeLists.txt:13; forced into TRACY_ENABLE
CMakeLists.txt:227-234 (on-demand, delayed init, manual lifetime, no crash handler);
definition added to both targets CMakeLists.txt:682-686; header pulled in MobileGL/Includes.h:140-145
with three zone colours. Backend zones are dense — e.g. MG_Backend/DirectGLES/DirectGLES.cpp:354,425, 473,517,543,685,1203,1220,1241,1303,1530,1727,1835,1895,2025,2725,2857,2899,2932,2952,3001 ("BindCurrentVAO"),3027 ("ResolveAndBindUnitTextures"),3303,3377,3400,3462.
PipeStats::OnPresent() also plots every counter per frame via TracyPlot
(PipeStats.cpp:320-344; per-gate hit/miss plot names PipeStats.cpp:184-191). Tracy gives
per-thread zone timelines, not aggregated per-thread CPU seconds, and needs a connected profiler —
usable for attribution, not as the paired A/B number.
(c) simpleperf — the only real per-thread CPU attribution available.
tools/device_bench/profile.sh (70 lines):
- record:
simpleperf record --app <PKG> -e cpu-clock -f 800 -g --duration N(profile.sh:52-53); DWARF unwinding is mandatory, frame-pointer graphs are broken (profile.sh:15-16). - symbolize:
binary_cache_builder.pyagainst the unstrippedMobileGL/build/intermediates/merged_native_libs/fordebug/mergeFordebugNativeLibs/out/lib/arm64-v8a(profile.sh:58-62). - per-thread summary is exactly
simpleperf report --sort comm -n(profile.sh:63-65), then--comms <renderthread> --sort symbolto drill in (profile.sh:66-67). - Its target is hardcoded to the FCL fordebug game package
com.tungsten.fcl.mgdebug.debug(profile.sh:25), not the trace APK, and it needs Git Bash not PowerShell (profile.sh:18). - Caveat carried in the user's memory
mobilegl-profiling-pipeline.md:13-15: MC's render thread is a JVM thread namedThread-1x(varies per run), holds 50-75 % of samples;--call-graph fpuseless. simpleperf --apprequires a debuggable app. The trace APK's release build type isisDebuggable = debuggableRelease(android-plugin/app/build.gradle.kts:183), fed by the gradle propertymobilegl.debuggableRelease, default"false"(build.gradle.kts:62). So profiling the retrace harness needs a-Pmobilegl.debuggableRelease=truebuild — flag this to the implementers; and notefordebug -O0history (memoryfordebug-o0-trap) does not apply here (that was FCL's native lib, fixed at34b7cc77).
(d) tools/device_bench/bench.sh — the in-game FPS harness (wall clock, not CPU).
- What it measures: FCL's overlay counts
eglSwapBuffersnatively and logs oneFCLFPS: <n>logcat line per second (tools/device_bench/README.md:8-16);bench.sh:178tailslogcat -v raw -s FCLFPS:I,:180-187collects--samples(default 30,bench.sh:36) after--warmup(default 180 s,bench.sh:37, andREADME.md:48-50explains the 3-minute ART JIT ramp). - Output: one JSON line per run into
results/results.jsonlwithfps_mean/median/min/max/sd, gpu_busy_mean, temp_start/mid/end_mc, big_cur, little_cur, gpu_cur_khz, pinned, warmup_s(bench.sh:210-214). - Protocol discipline that P2 must repeat verbatim: thermal gate, 180 s warmup, pin-integrity check
(
big_cur/little_cur/gpu_cur_khzsampled at window end, discard if off-pin), paired back-to-back A/B in the same session, F3 off, FPS overlay on, root required —README.md:45-58. devices/odinlite.envis the only device profile present (tools/device_bench/devices/); there is no profile for35d0befaor3B159D009VZ00000— P2 must author two.bench.shreports no CPU time at all. Pairing it withprofile.shis the only way to get the per-thread number out of the game path today.
1.3 Device protocol (from the P0 brief and the P0 run scripts)
- Devices:
35d0befa= Xiaomi 24129PN74C, Adreno 830, Android 16;3B159D009VZ00000= Oppo PLG110, Mali, Android 16 (ColorOS) —docs/Disaggregated/MEASUREMENTS.md:3, and.../eb1a694b-.../scratchpad/wf2/BRIEF-P0.md:25-27. - Lock protocol (
BRIEF-P0.md:27): atomicmkdirlock before any adb use atC:/Users/GEEKER~1/AppData/Local/Temp/claude/C--Users-geekerwan-.../8ab6b3cb-c783-4826-9fd7-39ff3498af5b/scratchpad/w7/locks/<serial>,echo "<owner> $(date)" > "$LOCK/owner",rm -rfwhen done, hold ≤ 25 min per acquisition; a lock held > 60 min is not to be broken — report "blocked by device lock". Always-s <serial>. Wake+unlock first (input keyevent KEYCODE_WAKEUP; input keyevent 82). Device logs from/sdcard/MG/latest.log, not logcat. - The working P0 loop that already implements this end to end is
.../scratchpad/wf2/stats_baseline.sh:16-29— lock acquire with a 1500 s timeout and 20 s poll (:18), one hold per (case, backend) run, results copied out of.trace-work/android-retrace-result/*-$b/, andgrep 'MGPipe stats'onmobilegl.log(:28). Copy this shape for P2's paired A/B rather than reinventing it. - Serialization rule:
run_android_retrace_local.pyshares one.trace-work/android-retrace-resultroot per tree andrmtrees it per invocation (run_android_retrace_local.py:14,:354-356), so two devices must be driven serially from one tree —MEASUREMENTS.md:93. For P2's paired A/B across two devices, either accept the serialization or use two worktrees. - ColorOS traps: first install of a not-yet-installed package blocks on the
com.oplus.appdetail InstallGuideActivitydialog (MEASUREMENTS.md:18); a foreign-signed trace APK must be uninstalled first (INSTALL_FAILED_UPDATE_INCOMPATIBLE). - MSYS trap: an embedded
/data/...inside an--envvalue gets path-converted; run withMSYS_NO_PATHCONV=1(MEASUREMENTS.md:94, andstats_baseline.sh:2).
1.4 The gap, and the cheapest way to close it
The GO/NO-GO wants per-thread CPU p50/p99, paired, reboot-clean, on two devices. Options, in order of cost:
- (recommended) Add thread CPU time to the retrace benchmark itself. Retrace runs
--singlethread(android-plugin/app/src/trace/cpp/trace_replay_core.cpp:459), andtrace_benchmark.hpp:12-13states that Begin/OnFrameBoundary/End are only ever reached from that one retrace thread — so that thread's CPU time is the client-side CPU cost, with no sampling, no root, no debuggable build and no profiler. The change is small and lands in shared code (desktop CLI and Android APK compile the same files —tools/trace_replay/CMakeLists.txt:229,254-255):android-plugin/app/src/trace/cpp/trace_benchmark.cpp:75-77— alongside theClock::now()wall delta, takeclock_gettime(CLOCK_THREAD_CPUTIME_ID, …)and push a second per-frame series intoReport(trace_benchmark.hpp:32-39).trace_replay_core.cpp:832-865(SummarizeBenchmark) — the mean/median/nearest-rank-p95 code is already there and can be reused verbatim for the CPU series.trace_replay_core.cpp:867-899(WriteBenchmarkJson) — addmeanFrameCpuMs / medianFrameCpuMs / p95FrameCpuMsand aframeCpuTimesMs[]array next to the existingframeTimesMs[]at:889-896.tools/trace_replay/run_android_retrace_local.py:228-237(format_benchmark) — print them.- p99 needs no device change even today:
frameTimesMs[]is dumped in full (trace_replay_core.cpp:889-896), so any percentile is computable host-side frombenchmark.json. Only the headline fields stop at p95.
- simpleperf
--sort comm(tools/device_bench/profile.sh:63-65) for the game path and for attribution ("where did the ns go"), accepting the debuggable-build requirement above. - A
/proc/<pid>/task/*/statsampler. Nothing like it exists in the tree; whether a non-rootadb shellcan read another app's task stats on these Android 16 ROMs is unverified — do not put it on the critical path without a probe.
2. Trace-replay benchmark mode, and how MOBILEGL_PIPE_STATS numbers are collected on device
2.1 tools/trace_replay/run_android_retrace_local.py --benchmark
- Flags and defaults (
run_android_retrace_local.py:310-339):--benchmark— replay each case end to end as a frame-timing benchmark instead of comparing one frame to its golden (:310-315).--benchmark-repeats, default 3; the run with the lowest mean frame time is reported as "best" (:316-321, selection at:281-291; a run that recorded no frames reports-1and is forced toinfso it cannot win —:281-284).--benchmark-tail-frames, default 200 — statistics are computed over the trailing N frames only (:322-327; same default in C++ attrace_replay_core.hpp:18).--benchmark-no-finish— drops the per-frameglFinish, so frame times measure CPU submission only (:328-333). The semantics are spelled out attrace_benchmark.hpp:16-24: with finish ON the numbers cover GPU completion and are pessimistic but deterministic and comparable between backends/revisions; with it OFF you measure the CPU side. For P2 (a CPU-cost question) the--benchmark-no-finisharm is the primary one, and the finish-ON arm is the sanity check.--benchmark-timeout-seconds, default 900 per run — a benchmark replays the whole trace, not just up totarget_call(:334-339).--env KEY=VALUE(repeatable) — generic passthrough applied immediately beforelibMobileGL.soloads; a KEY with no=unsets it (:301-309). It reaches the device as--es mobilegl_env "'K=V;K=V'"(android-plugin/trace-replay-ci.sh:411-416) and lands inApplyEnvOverrides→setenv(trace_replay_core.cpp:143-152).
- Repeat economics: only run 1 installs the APK and pushes the trace; repeats pass
--reuse-fixture(run_android_retrace_local.py:257-258, honoured attrace-replay-ci.sh:522-525). The previous repeat'sbenchmark.jsonis deleted first so it cannot be re-read as this run's result (:259-262). - Printed per run and then a "best of N" line (
:276-291), format fromformat_benchmark(:228-237):frames=… total=…s tail=… mean=…ms median=…ms p95=…ms fps=…. - Benchmark mode renders no SSIM summary page (
:360-364); "passed" only means the replay reached the end of the trace without an error (trace_replay_core.cpp:1037-1049).
2.2 android-plugin/trace-replay-ci.sh (what the runner shells out to)
- Usage/flags:
trace-replay-ci.sh:36-40(--benchmark,--benchmark-tail-frames,--benchmark-finish,--reuse-fixture), semantics at:58-63; defaultsbenchmark=0,benchmark_tail_frames=200,benchmark_finish=1at:129-131. - It turns those into Activity extras:
--ez benchmark true,--ei benchmark_tail_frames N,--es benchmark_result_path <appdir>/output/benchmark.json,--ez benchmark_finish …(:417-426). - Completion is polled once a second for
--timeout-secondsby looking foroutput/result.jsonthroughrun-as(:443-454), with an "app exited early" branch (:447-452) and an infrastructure-retry exit code 75 (:6,:475-478). - Artifacts copied back on every run:
result.json(:481),retrace.log(:488) andmobilegl.log(:489);benchmark.jsonadditionally in benchmark mode (:490-492). A benchmark run copies noactual/diffPNG (:483-487). - This is why one run gives you both numbers:
mobilegl.log(whereMGPipe stats:lines land) is pulled even in--benchmarkmode.
2.3 What the device writes
trace_benchmark.cppis the timer:Begin(finishEachFrame):49-58,OnFrameBoundary():60-78(optionalglFinishresolved lazily throughdlsymon the already-loadedlibMobileGL.so:29-45),End():80-91. Frame boundary hook is called from the platform glws swapBuffers override —android-plugin/app/src/trace/cpp/apitrace_glws_android.cpp:191-192.- Statistics:
SummarizeBenchmarktrace_replay_core.cpp:832-865— mean/median over the trailingtailFrames, nearest-rank p95 (:858-863),fps = 1000/mean(:864). - JSON:
WriteBenchmarkJsontrace_replay_core.cpp:867-899—tracePath, backend, benchmarkFinish, width, height, totalFrames, tailFrames, totalSeconds, meanFrameMs, medianFrameMs, p95FrameMs, fps, frameTimesMs[].result.jsoncarries the same headline numbers underbenchmark*keys (:954-968). - Retrace argv:
-b --singlethread --no-context-check, and in benchmark mode the-s/-Ssnapshot pair is omitted entirely so the readback and PNG encode are not in the timed loop (trace_replay_core.cpp:455-471, rationale at:461-464). - The same
--benchmarkfamily exists on the desktop CLI (tools/trace_replay/trace_replay_cli.cpp:32-40, parsing:160-170), sharing the identical core (tools/trace_replay/CMakeLists.txt:229,254-255). So a WSL/lavapipe pre-flight of any benchmark change is free.
2.4 MOBILEGL_PIPE_STATS on device
-
Knobs (all
MOBILEGL_-prefixed, so no allow-list edit is ever needed —MobileGL/ConfigLoader.cpp:242-244, mechanism atConfigLoader.cpp:31-61+IsAcceptedPrefix):MOBILEGL_PIPE_STATS→Features.PipeStats(Config.h:353-355,ConfigLoader.cpp:254).MOBILEGL_PIPE_STATS_PERIOD→Features.PipeStatsPeriod, default 120, range [1, 1000000] (Config.h:370-374,ConfigLoader.cpp:262); latched inPipeStats::Init()(PipeStats.cpp:254-267), never 0 (:258-260).MOBILEGL_PIPE_STATS_FILE→Features.PipeStatsFile, empty = no dump (Config.h:375-378,ConfigLoader.cpp:263).
-
Summary line format — built by
FormatWindowLine()PipeStats.cpp:366-432, emitted byEmitSummaryLine():223-232viaMGLOG_I(deliberately INFO, justified at:225-229), fired everyg_summaryPeriodframes fromOnPresent():345-348:MGPipe stats: frames=<total> window=<n> draws=<n> draws/f=<x.xx> acc=<n> acc/draw=<x.xx> bytes/f[buf= tex= ubog= ubon= vtxc= idxc= icmd= pmap= resid=] tex[emit= box= rect= jobs=] gates[ers=h/m etl=h/m eub=h/m mfp=h/m mpm=h/m mdt=h/m]Short names:
PipeStats.cpp:193-195(buf tex ubog ubon vtxc idxc icmd pmap resid;ers etl eub mfp mpm mdt). Two decimals viaFormatFixed2, and a zero denominator printsn/a, never0.00(:159-166). If a window contains no Present the label degrades frombytes/f[tobytes[and carries window totals (:404-406, reason at:371-375). Windows are disjoint:AdvanceSummaryWindow():434-446rebases;FormatWindowLine()is pure and can be called twice (PipeStats.h:175-183). -
JSON dump limitation (the trap):
WriteJsonDump()runs only fromShutdown()(PipeStats.cpp:271-278), which is called fromMobileGL::Destroy(MobileGL/Init.cpp:50). The trace app never reaches that teardown, soMOBILEGL_PIPE_STATS_FILEnever produces a file on device — only the periodicmobilegl.loglines exist, and a trace shorter than one period yields nothing at all. Documented atdocs/Disaggregated/MEASUREMENTS.md:92; that is precisely whyMOBILEGL_PIPE_STATS_PERIODwas added (Config.h:370-373). It does work on desktop, where teardown runs. -
The reproducible device recipe (
MEASUREMENTS.md:70-76):ANDROID_SERIAL=<serial> MSYS_NO_PATHCONV=1 \ python3 tools/trace_replay/run_android_retrace_local.py \ --case minecraft-1.21.4-in-world --backend DirectGLES \ --env MOBILEGL_PIPE_STATS=1 --env MOBILEGL_PIPE_STATS_PERIOD=120 # numbers: grep 'MGPipe stats:' <result dir>/mobilegl.log, take the LAST complete windowMOBILEGL_LOG_FILE_PATHis set by the harness to<outputDir>/mobilegl.log(trace_replay_core.cpp:984,setenvat:1011). -
The P0 baseline that P2 compares against is
MEASUREMENTS.md:50-58(four traces × two backends × two devices), and the reading at:62-66.
3. What a Blaze3D blend-toggle microbenchmark attaches to
It already exists. MobileGL/MG_Benchmark/Driver/DriverBenchCases.inc:558-570:
/* Blaze3D toggles blend around batches: 46 glEnable/glDisable pairs and 28
* glBlendFuncSeparate per vanilla frame. a = toggle pairs. */
static void case_mc_state_toggle(int frame, long a, long b) { … glEnable(GL_BLEND);
glBlendFuncSeparate(...); glDrawElements(...); glDisable(GL_BLEND); glDrawElements(...); }
registered in the case table as {"mc_state_toggle", case_mc_state_toggle, 46, 0, 46}
(DriverBenchCases.inc:619-639, table type at :611-617; a=toggle pairs, opsPerFrame is the
normaliser). Sibling MC-shaped cases that P2 will want in the same table:
mc_vanilla_draw (5495 draws/f), mc_pass_switch (:543-556), mc_tex_param (:575-586),
mc_use_program (:590-604), mc_ubo_range, mc_sampler_churn.
How it runs.
MobileGL/MG_Benchmark/Driver/DriverBench.cis a headless EGL client that is not linked against MobileGL: itdlopens exactly one EGL provider from$DRIVERBENCH_EGL_LIB— the systemlibEGL.so.1for the native driver, or alibMobileGL.sopath for either backend selected withMOBILEGL_BACKEND_TYPE(DriverBench.c:9-18, CMake commentDriver/CMakeLists.txt:3-6).- Loop: 30 warmup frames then 120 timed frames, each closed with a real fence wait, not
glFinish— because MobileGL implementsglFinish/glFlushas no-ops and a finish-paced loop would time only CPU submit on MobileGL while timing submit+GPU natively (DriverBench.c:245-266, run loop:268-292). Knobsg_warmup=30, g_frames=120(DriverBench.c:254), env overridesDRIVERBENCH_DRAWS / DRIVERBENCH_FRAMES / DRIVERBENCH_SPRITES(:475-477). - Output is one CSV line per case:
case,frames,ops_per_frame,median_frame_ms,ns_per_op,fps(DriverBench.c:31-32, header:482, print:288-289).ns_per_opis exactly the shape the day-43 "tracker absolute ns per draw" number needs. - Driver script
MobileGL/MG_Benchmark/Driver/run_driver_bench.sh: modesnative | espryt <so> | magma <so>(:1-9), pins the EGL vendor and Vulkan ICD explicitly so a barelibEGL.so.1does not silently fall to llvmpipe under a benchmark (:10-15,:19-20,:26-40).
How it is built.
MOBILEGL_BUILD_BENCHMARKoption, default ON (CMakeLists.txt:6), documentedREADME.md:140; subdirectory added atCMakeLists.txt:782-784(afterMG_Testso googletest is already available,:776-777).MobileGL/MG_Benchmark/CMakeLists.txtFetchContents google/benchmark v1.9.4 (:18-25), linksMobileGL_s(:14-16), and addsProgram Buffer Driver Container ShaderCache Transpile(:43-48). Every bench is a ctest underLABELS benchmark(e.g.Driver/CMakeLists.txt:14-15,Buffer/CMakeLists.txt:19-20), so CI runs them withctest -L benchmark(.github/workflows/test.yml:745).
Two hard constraints P2 must plan around.
DriverBenchis Linux-desktop only:Driver/CMakeLists.txt:7-9—if (NOT UNIX OR APPLE OR ANDROID) return().- The whole benchmark tree is force-disabled on Android:
CMakeLists.txt:33-35setsMOBILEGL_BUILD_BENCHMARK OFF … FORCEwhenANDROID. → The blend-toggle microbench can be run today on WSL/lavapipe+llvmpipe (and against a real desktop GPU) but not on35d0befa/3B159D009VZ00000without new work. Since P2's question is client-side CPU per draw, the desktop run is legitimate evidence for the tracker ns number, but the checklist's two-device requirement (ROADMAP.md:49) applies to the per-thread CPU deltas, which come from the retrace benchmark (§2), not from DriverBench. If an on-device blend-toggle is wanted, the cheapest route is a new Android-only target modelled onDriverBench.c(it already dlopens EGL and needs no window — surfaceless fallback atDriverBench.c:294-300, 417), not enabling the google-benchmark tree on Android.
- The other benches (
Buffer/BufferBench.cpp:23-40,Program/ProgramBench.cpp) drive frontend entry points againstMobileGL_swith no driver; they are the wrong attachment point for a draw-path microbench.
4. The CSO negative control (a switch that disables content addressing)
Design intent, already written down:
docs/Disaggregated/ARCHITECTURE.md:367(§9.6): 「MOBILEGL_PIPE_PUSH子系统位图(含一位关闭 CSO 内容寻址,负面对照)在阶段 B 是真正的旧-vs-新 A/B」.- What content addressing is:
ARCHITECTURE.md:63— per-classska::flat_hash_map<xxHash, MGPipeHandle>, caps render-state 64 / vertex-elements 1024 / sampler 256 / sampler-view 4096, LRU evict emitsdelete_*. - Why the identity is a subset of the render-state blob and not the whole thing:
ARCHITECTURE.md:184-189— whole-blob addressing would mint a new CSO on everyglViewport/glScissor/glBlendColor/glClearColorand flush the server's pipeline memo (a regression recorded inRenderState.h); the client order is pipeline version unchanged → reuse handle, zero hashing; changed → xxHash the ~25-30-word pipeline subset → CSO map probe → hit sends a 12 B bind, miss sends create+bind;m_versionchanged but subset unchanged →set_dynamic_state(~200 B). The negative control has to disable the map probe/reuse, not the CSO records — otherwise it measures a different thing. - Why the push happens at validate time and not in the GL setter:
ARCHITECTURE.md:151(Blaze3D wraps every batch inglEnable/glDisable(GL_BLEND); per-setter push would be strictly slower than today). This is the same workload the microbench of §3 models.
Where the knob lives today.
MG_Config::FeaturesTable::PipePush—MobileGL/Config.h:319-326. The comment at:320-325already promises: "per-subsystem bitmask selecting which state the frontend PUSHES … One bit of it also turns OFF client-side content addressing of CSOs, which is the negative control the CSO design is measured against. Accepts decimal or 0x-prefixed hex." Default0.- Parsed at
MobileGL/ConfigLoader.cpp:245withQueryEnvUint64("MOBILEGL_PIPE_PUSH", 0), whose contract is atConfigLoader.cpp:162-192: decimal by default, explicit0xfor hex, neverstrtoullbase 0 (a leading zero would read010as 8), and any-is rejected rather than wrapped to all-bits-set. - Status:
Features.PipePushhas no reader.grep -rn "PipePush" MobileGL/hits onlyConfig.h:326andConfigLoader.cpp:245. There is no bit enum anywhere (grepforMGPipeSubsystem | kPipePush | ContentAddress→ nothing). So P2 mints both the bit constants and the first consumer. - Do not confuse it with the CMake option of the same spelling:
MOBILEGL_PIPE_PUSHis also a compile-time option (CMakeLists.txt:28, define at:548-549) that selects theMGB_CTXarm (MobileGL/MG_Pipe/PipeInputsSwitch.h:17-27). The runtime bitmask is a separate, additive thing that only matters in a push build.
Naming and wiring conventions to follow (MOBILEGL_PIPE_* family).
- All eight live knobs are declared in
Config.h:319-378and parsed in one block atConfigLoader.cpp:242-263. Helper choice is by type:QueryEnvFlag(default-off boolean, truthy rule atConfigLoader.cpp:72-87),QueryEnvQuirkOverride(tri-state, used when the default is ON so only an explicitly falsy value turns it off —ConfigLoader.cpp:89-98, e.g.PipeLegacyMemosat:258-259andPipeVerifyFatalat:249-250),QueryEnvUint32(key, def, min, max)(:141-160),QueryEnvUint64(bitmask only,:167-192),QueryEnvVariable(string,:63-70). - Knobs that must not exist in a shipping build are
#if MOBILEGL_PIPE_PUSH-guarded so the pull build'sFeaturesTabledoes not change size (Config.h:333-352, mirrored atConfigLoader.cpp:247-253). - No allow-list edit is ever needed:
InitializeAcceptedEnvVariablesaccepts everyMOBILEGL_/LIBGL_-prefixed variable found in the environment (ConfigLoader.cpp:31-61, noted at:242-244). - Precedent for a knob that is explicitly the negative control and how to word it:
Config.h:341-345(MOBILEGL_PIPE_VERIFY_CORRUPT, negative control A) and:346-351(MOBILEGL_PIPE_POISON_OMIT, negative control B). Both are wired as always-on CI steps, not as manual procedures —MobileGL/MG_IntegrationTest/CMakeLists.txt:700-711(A) and:719-725(B). P2's CSO negative control should get the same treatment: a ctest entry that must be red when the switch is on and green when it is off, so a dead switch cannot pass silently. - Recommendation for the brief: name the bit in the same file/format
(
Config.h, near:326), reserve a high bit (e.g.0x8000'0000'0000'0000— "not a subsystem, a behaviour") so subsystem bits stay a dense low range, and document the decimal/hex parse in the knob comment because operators will pass it as hex.
5. CI cost picture — what exists and what P2 can afford to add
Workflow .github/workflows/test.yml (1541 lines). Triggers: push to dev,
Feat/Backend-Direct-GLES, Feat/Backend-Direct-Vulkan, plus workflow_dispatch with a
baseline_sha input (test.yml:3-17). feat/disaggregated is not a push-triggered branch —
every run on it so far has been a manual dispatch (gh run list: 34027009840 / 34009031893 /
34008829271 all workflow_dispatch).
5.1 Job inventory (name → what it runs → last observed duration)
Durations are from the last completed dispatch on feat/disaggregated,
run 34009031893 (head 087685d1, 2026-09-06 03:27–04:13Z, 127 jobs, ~46 min wall).
| job (line) | what it does | last run |
|---|---|---|
build-linux (:20) |
clang-20 + ccache configure/build, uploads mobilegl-linux-runtime.tgz; benchmarks ON here (:95 -DMOBILEGL_BUILD_BENCHMARK=ON) |
11m |
test (:157) |
ctest -L unit --no-tests=error on the downloaded runtime (:194-203) |
2m |
integration (:213) |
ctest -L integration-gpu on llvmpipe/lavapipe plus a second filtered pass with MOBILEGL_ESPRYT_DISABLE_INVALIDATE_FLUSH=1 -R 'Buffer|Readback|Atomic|Ssbo|Arena' (:279-296); MOBILEGL_ITEST_REQUIRE_GPU=1 so a driverless runner fails instead of skipping (:256-259, :274-278) |
6m |
benchmark (:699) |
ctest -V -L benchmark --no-tests=error (:740-745), needs libegl-mesa0 explicitly (:710-717), uploads core dumps on failure |
3m (2m on the previous run) |
flatc-check (:649) |
regenerates the flatbuffers header and diffs | 1m |
include-graph-check (:677) |
include-closure purity gate | 0m |
pipe-gates (:1477) |
source-level only, deliberately independent of build-linux (:1480-1481): gen_pipe.py + git diff --exit-code (:1492-1495), gen_pipe.py --self-test (:1501-1502), symbol_report.py --self-test (:1505-1506), the stdio-instrumentation grep gate over MG_Backend+MG_State (:1515-1522), the informational gen_pipe_dirty_surface.py --summary (:1527-1528), and the citation lint, warning-only for now (:1533-1541) |
10 s |
build-retrace (:755) |
builds the desktop replay runner; needs build-linux, test, benchmark (:757-760) |
2m |
trace-cases (:900) |
emits three matrices from trace_cases.json: full, names, and the verify subset (:915-922) |
0m |
trace-fixtures (:924) |
48 fixture jobs, max-parallel: 4 (:928-932), LFS fetch + cache |
~0m each |
retrace (:989) |
2 backends × ~39 CI cases, max-parallel: 4 (:998-1001) |
0–3m per job, ~78 jobs; dominates the tail of the 46 min |
retrace-summary (:1095) |
renders the overview page | 0m |
remove-artifact-clutter (:1417) |
artifact hygiene | 0m |
build-linux-verify (:311) |
second build with -DMOBILEGL_PIPE_VERIFY=ON, timeout-minutes: 120, own ccache (:311-323) |
never yet completed |
integration-verify (:473) |
ctest -L integration-verify under the comparator, timeout-minutes: 180 (:473-539) + an arming-log assertion (:554-560) |
never yet completed |
retrace-verify (:1141) |
verify build × the verify: true subset, max-parallel: 4, timeout-minutes: 240, --timeout 10800 per case because comparator arms are 5-10× slower (:1141-1154, :1220-1230); refuses to run against a comparator-free library via an nm --defined-only … MGPipeVerifyInputs check (:1207-1218) |
never yet completed |
monolith-symbol-report (:1298) |
workflow_dispatch only, timeout-minutes: 180 |
in progress at time of writing |
5.2 Facts the brief should state plainly
- The three verify lanes have never produced a green CI run. They were added by
0fe7bf82("the third CI mode …") and hardened by97b997d5, both dated 2026-09-06; the only run whose head contains them is 34027009840 (heade7a6a72f), stillin_progresswithbuild-linux-verifyandmonolith symbol reportrunning at 10:28Z. The P1 verify evidence inMEASUREMENTS.md:129-142is therefore local WSL evidence (~/w7/pipe), not CI evidence. P2's plan must not assume a known CI cost for these lanes; the only cost datum isMEASUREMENTS.md:142: integration-verify's 818 entries at 4-way parallelism are about the same order as integration-gpu; 79 retrace cases at 4-way take ~20 minutes (locally, 28-core WSL). - Verify scope is deliberately narrower than
integration:test.yml:520-526states the second,MOBILEGL_ESPRYT_DISABLE_INVALIDATE_FLUSH=1filtered pass (186 entries) is not affordable under the comparator and is explicitly deferred — "P2 can take it once the comparator's cost is known." That is a P2 decision item the brief should surface. - Trace corpus:
tools/trace_replay/trace_cases.jsonholds 40 cases, 1 marked"ci": false(minecraft-1.21.4-rd12-odinlite-in-world, line 75), and 8 marked"verify": true(lines 16, 31, 39, 48, 56, 64, 79, 287 →OpenRA,minecraft-1.21.4-startup,minecraft-1.21.4-main-menu,minecraft-1.21.11-main-menu,minecraft-1.17-main-menu-854,minecraft-1.21.4-in-world,minecraft-1.21.4-fabric-sodium-in-world,improved-transparency-minecraft-26.3). The verify matrix is a strict subset soretrace-verifyneeds no fixtures of its own (test.yml:920-922). - Where P2's new CI cost would land, cheapest first:
pipe-gates(test.yml:1477) — 10 seconds, no build dependency. Thegen_pipe_dirty_surface.pystep is already flagged attest.yml:1524-1526as "becomes a gate in P2, when the mapping file exists to diff against". Turning it from informational into--exit-codeis P2's cheapest gate and costs nothing measurable. The G7 setter-consistency test and theResidualValueBlockoffsetofasserts (ARCHITECTURE.md:505) belong intest(unit), also ~free.test/integration— a{pull, push}A/B of the integration suite doubles the 6mintegrationjob (~+6m of a lane that is not on the critical path, sinceretraceis).ARCHITECTURE.md:503requires name-for-name identity betweenDirectGLES.andDirectGLES.Pipe., so this is a second ctest pass, not a second build.benchmark(3m) — adding one or two DriverBench cases tokBenchCases(DriverBenchCases.inc:619-639) costs ~1.2 s of wall per case (120 timed frames + 30 warmup at MC frame rates) and lands in an already-existing 3m job. The blend-toggle case is already in the table, so if CI is not currently running it as a separate assertion, exposing it costs nothing:add_test(NAME DriverBench COMMAND DriverBench draw_tiny)(Driver/CMakeLists.txt:14) currently runs onlydraw_tiny.- What P2 must not put in CI: the two-device paired A/B. Both devices are shared with the
GL4.6 Wave-7 campaign under the mkdir lock (§1.3); the numbers require reboot-clean, thermally
matched, back-to-back sessions (
tools/device_bench/README.md:45-58) that no hosted runner can provide. Device evidence is a recorded artifact inMEASUREMENTS.md, exactly as P0's baseline was, not a lane.
6. Suggested shape of the P2 measurement plan (assembled from the above)
- Land a CPU-time series in the shared benchmark core (§1.4 option 1) — 4 small edits, all
cited above, verifiable on WSL first through the desktop CLI, and it is the only change that turns
the existing harness into a per-thread-CPU instrument. Recompute p50/p99 host-side from
frameTimesMs[]/the new CPU array; the device already dumps the full per-frame arrays. - Paired A/B protocol per device, one lock hold per arm,
stats_baseline.sh:16-29as the template: for each of {pull build, push build} × {--benchmark-no-finish, finish} run--benchmark --benchmark-repeats 3 --benchmark-tail-frames 200, and in the same run collectmobilegl.logfor theMGPipe stats:window (both artifacts come out of one invocation,trace-replay-ci.sh:489,491). Cases: the four P0 baseline traces (MEASUREMENTS.md:50-58) so the numbers are directly comparable;create-indirectstays excluded (pre-existing failure on both devices,MEASUREMENTS.md:96). - Tracker absolute ns from DriverBench
ns_per_oponmc_state_toggle+mc_vanilla_draw(DriverBenchCases.inc:560,620,627), desktop, withrun_driver_bench.sh espryt|magmaand anativecontrol; ceiling derived from the 6.5–9.3 accessor/draw pull baseline (MEASUREMENTS.md:64). - CSO negative control as a
MOBILEGL_PIPE_PUSHbit (Config.h:326,ConfigLoader.cpp:245), plus an always-on ctest entry in the style ofMG_IntegrationTest/CMakeLists.txt:700-725that is red with the bit set and green without, so the switch cannot rot. - Free gates: promote
gen_pipe_dirty_surface.pyto--exit-codeinpipe-gates(test.yml:1524-1528); add G7 +ResidualValueBlockasserts to the unit lane; extendPipeStatsTest.cpp(CounterNamesAreStable:260) for any new counter. - Fix the doc pointer
ARCHITECTURE.md:504→tools/device_bench/bench.sh, and author the two missing device profiles undertools/device_bench/devices/(onlyodinlite.envexists).