From 8bb6309a1ce47b277758354ef98745f7bd5ec7b6 Mon Sep 17 00:00:00 2001 From: Swung0x48 Date: Tue, 22 Sep 2026 14:23:47 -0400 Subject: [PATCH] [scripts, CI, docs] (P7): the 184-symbol link-closure ratchet lands as a gate CONTRACT-P6.md 12.1 has carried "a ratchet recomputes the 184 in CI and asserts it only ever falls" as an obligation since P6. The roadmap handed it to P6.5; the P6.5 line never received it, and scripts/ci/ has never had anything that computes this number. ID-P7-7 gives it to P7 wave 0. This is it. scripts/link_ratchet.py mechanises a6's hand experiment: partition the per-object output of the single MobileGL target into SERVER / FRONTEND / SHARED, then compute (SERVER undefined) - (SERVER defined) - (SHARED defined) intersect (FRONTEND defined), which is the undefined-symbol list a real link of a server-only image would print. The partition is an explicit table with a rationale per row, because it IS the module boundary the build system does not have (SOURCE_FILES is one flat list; boundaries are P13's work). An object matching no row is a hard failure, as is a build whose CMakeCache lacks the three disaggregated options, or an empty role set - every way this gate could go green without having measured anything is an explicit exit 2. Measured on 78b7d6be: 186 symbols. The three backend buckets reproduce a6's hand-made 101 / 14 / 60 exactly, as do its per-object referrer and definer distributions (UniformManager 90, VulkanRenderer 60, Managers 53, DirectGLES 51; TextureObject 59, SamplerObject 30, BufferObject 18, FramebufferObject 16, PipeFill 9). Nothing from a6's 184 has been cleared; the +2 are both P6.5's: Client::ClientSession::StartSpawned() - fork-per-session GLState::ProgramArtifactsSchemaFingerprint() - archive v2 caps The baseline stores the LIST, not a count (ID-P7-7): a count-only ratchet goes green on the day one symbol is cleared and another added. --assert-monotone fails only on symbols absent from the baseline and names each one's defining object and every referring object; symbols that disappeared are reported with an instruction to re-baseline in the same commit, and never fail. The CI step goes in build-linux-split, not build-linux: build-linux takes the three options from MOBILEGL_CI_DISAGGREGATED, which is ON only on feat/disaggregated and codex/fix-github-ci, so on any other branch it builds a monolith and the gate would measure a SERVER set with no transport half. The JSON is uploaded even when the gate fires. Two findings for the integrator, both written up in the note: - a6's four published bucket totals sum to 181 against its own headline of 184. These buckets are a strict partition and always sum to the headline; the difference is in a6's core bucket, which it listed selectively. - 5 of the 14 "P3b/P4b" symbols are MG_Pipe slot-allocator and apply-thread plumbing that sits under MG_Impl/Pipe/ by file location alone. P3b/P4b cannot clear them by migrating Espryt functionality; they need P13's module move. That bucket's floor is 5, not 0, so the plan's 0.5-day Tier 1 estimate for "D12's 14 symbols" should be re-cut against 9. All 7 symbols from that object carry a `# P13` annotation. red-once (R-16) is recorded verbatim in the note. Worth knowing: the plan's own example (SamplerObject::GetMinFilter) does NOT turn this gate red, because it is already in the baseline - the ratchet reddens on new symbols, so a negative control has to pick one that is not in the 186. A reach to GetBorderColorForm() from DirectGLES/Utils.cpp does, naming the symbol, its definer and its referrer, exit 1; reverted and green again. --- .github/workflows/test.yml | 42 + MobileGL/MG_Remote/CONTRACT-P6.md | 8 +- docs/Disaggregated/notes/p7/link-ratchet.md | 230 ++++++ scripts/data/link_ratchet_baseline.txt | 200 +++++ scripts/link_ratchet.py | 861 ++++++++++++++++++++ 5 files changed, 1340 insertions(+), 1 deletion(-) create mode 100644 docs/Disaggregated/notes/p7/link-ratchet.md create mode 100644 scripts/data/link_ratchet_baseline.txt create mode 100644 scripts/link_ratchet.py diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index f618f54e..2d55c92a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -987,6 +987,48 @@ jobs: fi echo "libMobileGL.so defines ${remote} MG_Remote symbol(s) and the MGPipe applier (${defined} defined symbols)" + # The link-closure ratchet (CONTRACT-P6.md 12.1, ID-P7-7): the set of frontend symbols a + # server-only image would still have to link, recomputed from the per-object partition in + # scripts/link_ratchet.py and compared against the stored LIST. + # + # THIS job and not `build-linux`: build-linux passes the same three options from + # ${MOBILEGL_CI_DISAGGREGATED}, which is ON only on feat/disaggregated and + # codex/fix-github-ci, so on every other branch it builds a monolith - MG_Remote would not + # be compiled at all, the SERVER half of the partition would be empty, and the gate would + # go green having measured nothing. This job turns the three options on unconditionally, + # and the nm step immediately above has just proved they took. The script re-reads them + # from CMakeCache.txt anyway and refuses the run if they are not all ON. + # + # Only NEW symbols fail. A symbol that has DISAPPEARED is the ratchet turning and is + # printed with an instruction to re-baseline in the same commit - the baseline stores the + # list rather than a count precisely so that clearing one symbol while adding another does + # not walk through at an unchanged total (ID-P7-7). + # + # It reads the per-object output of the MobileGL target, so it must run before anything + # that cleans the build tree; the packaging step below excludes *.o from the tarball but + # does not delete them. + - name: The server image reaches no further into the frontend than the baseline + run: | + mkdir -p ci-artifacts + python3 scripts/link_ratchet.py --self-test + python3 scripts/link_ratchet.py \ + --build-dir "${BUILD_DIR}" \ + --baseline scripts/data/link_ratchet_baseline.txt \ + --assert-monotone \ + --bucket \ + --json ci-artifacts/link-ratchet.json + + # Uploaded even when the gate fired: a red ratchet is exactly the moment someone needs the + # per-symbol definer/referrer attribution the JSON carries, and a job that failed before + # uploading its evidence is a job nobody can act on. + - name: Upload link ratchet + if: always() + uses: actions/upload-artifact@v7 + with: + name: link-ratchet + path: ci-artifacts/link-ratchet.json + if-no-files-found: warn + - name: Show ccache stats if: always() run: ccache --show-stats diff --git a/MobileGL/MG_Remote/CONTRACT-P6.md b/MobileGL/MG_Remote/CONTRACT-P6.md index 778002c8..7ef5552a 100644 --- a/MobileGL/MG_Remote/CONTRACT-P6.md +++ b/MobileGL/MG_Remote/CONTRACT-P6.md @@ -699,7 +699,13 @@ owed to P13. Method and full symbol list: `Client::AdoptTierIsEmulate()`; `MG_Pipe/PipeApply.cpp` needs `Client::ClientSession::Active()` and `AdoptTierIsEmulate()`. -**Gate**: a ratchet recomputes the 184 in CI and asserts it only ever falls. +**Gate (landed, P7 wave 0 — ID-P7-7)**: `scripts/link_ratchet.py` recomputes this set from an +explicit per-object partition on every `build-linux-split` run and fails on any symbol absent from +`scripts/data/link_ratchet_baseline.txt` — which stands at **186** on `78b7d6be`, the 184 plus +P6.5's `Client::ClientSession::StartSpawned()` and `MG_State::GLState::ProgramArtifactsSchemaFingerprint()`, +with the three backend buckets reproducing a6's 101 / 14 / 60 exactly — while a symbol that +disappears is reported rather than failed, so progress never turns the build red +([`link-ratchet.md`](../../docs/Disaggregated/notes/p7/link-ratchet.md)). ### 12.2 D11 — five unbounded peer-driven allocations, capped, with no `try`/`catch` diff --git a/docs/Disaggregated/notes/p7/link-ratchet.md b/docs/Disaggregated/notes/p7/link-ratchet.md new file mode 100644 index 00000000..2e03d57b --- /dev/null +++ b/docs/Disaggregated/notes/p7/link-ratchet.md @@ -0,0 +1,230 @@ +# 链接闭包棘轮（P7 wave 0，ID-P7-7） + +> 脚本 `scripts/link_ratchet.py`，基线 `scripts/data/link_ratchet_baseline.txt`， +> CI 步骤在 `.github/workflows/test.yml` 的 `build-linux-split` 作业里。 +> 被量的那件事见 [`a6-link-experiment.md`](../p6/a6-link-experiment.md) +> 与 [`a6-link-experiment-data.md`](../p6/a6-link-experiment-data.md)。 + +## 0 这道门归谁 + +契约 `CONTRACT-P6.md` §12.1 把 184 记成债并写下一句义务：「一道棘轮在 CI 上重算这 184 并断言它只减不增」。 +路线图把它写成**「移交 P6.5」——P6.5 行从未接收**：P6.5 落地的是传输栈两轴、归档 v2、发布边界、半关闭与 +`AcceptPair`/角色修复，没有一行是这道门。`scripts/ci/` 下也确实没有任何脚本算这个数 +（`link_seam_purity.py` 量的是源码文本里的所有权/派发纪律，`p65_link_metrics.py` 量的是帧日志）。 + +**ID-P7-7 把它归 P7 wave 0**，并定了两条：基线存**符号列表**而不是计数；`--assert-monotone` +只对**新增**符号变红，符号消失只提示重基线。第二条是这道门的全部意义—— +裸计数的棘轮在「清掉一个、又加一个」的那天是绿的，而那天正是它该红的那天。 + +## 1 怎么量 + +没有模块 target 可链（`SOURCE_FILES` 是一张平的源文件列表，同时喂 `MobileGL` 与 `MobileGL_s`； +建立模块边界本身是 P13 的具名工作），所以 a6 的办法是对**单个 `MobileGL` target 的逐个 object** +施加一个纸面上的分区，然后做集合运算： + +``` +(SERVER 未定义) − (SERVER 已定义) − (SHARED 已定义) ∩ (FRONTEND 已定义) +``` + +这就是一次真链接会打印的那份未定义符号表。脚本照做，**唯一的区别是那张分区表现在写在脚本里** +（`link_ratchet.py` 的 `PARTITION`），一行一条路径前缀、一个角色、一句理由，最长前缀匹配。 +它就是构建系统没有的那道模块边界，只不过写下来了。 + +两条纪律： + +- **任何 object 匹配不到 `PARTITION` 的任何一行 = 硬失败**，不是静默忽略。 + 树会长，新目录会出现；一个悄悄被漏掉的 object 会让这个数字悄悄停止度量它声称度量的东西。 +- **`CMakeCache.txt` 里三个选项不全为 ON = 硬失败**（`--no-require-flags` 可关）。 + 选项为 OFF 时 `MG_Remote` 根本不编译，SERVER 集少掉整个传输半边，脚本会给出一个 + 很小、很绿、毫无意义的数。 + +`nm` 带 `-C`，所以集合运算的单位是**解构后的名字**——a6 发表的就是它，人也只读得懂它。 +代价写在脚本的 docstring 里：基线树上 FRONTEND 集有 14974 个不同的 mangling 落在 14880 个不同的 +解构拼写上，那 94 处重合几乎全是 C1/C2 构造与 D1/D2 析构对（按定义解构成同一串文本）。 +**解构棘轮是链接的一个可读的近似，不是链接。** 要在 mangling 上较真的工具是 `symbol_report.py`。 + +只有 `U` 算引用；弱未定义（`w`）在无人定义时解析为 0，不构成对任何人的义务。 +基线树上没有任何 `w` 触及前端定义，所以这条过滤今天不花钱。 + +## 2 今天的数字 + +基座 `78b7d6be`，`build-split`（Release，`-DMOBILEGL_BUILD_DISAGGREGATED=ON +-DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON -DMOBILEGL_PIPE_PUSH=ON`），211 个 object： + +| | | +|---|---:| +| 分区 | SERVER 52 / FRONTEND 71 / SHARED 88 | +| SERVER 集未定义符号总数 | 1579 | +| 其中 SERVER + SHARED 自己满足不了的 | 593 | +| **其中由 FRONTEND 定义的** | **186** | + +分桶（`--bucket`），与 a6 手做的那份逐个对照： + +| 桶 | 符号数 | a6 | delta | 含义 | +|---|---:|---:|---:|---| +| `p6-core` | 11 | 6 | +5 | 没有任何后端 object 引用它：传输/applier 自己的账 | +| `p7-magma` | 101 | **101** | **+0** | 只有 `DirectVulkan` 引用：P7 的账 | +| `p3b-p4b-espryt` | 14 | **14** | **+0** | 只有 `DirectGLES` 引用：P3b/P4b 的账 | +| `both-backends` | 60 | **60** | **+0** | 两个后端都引用：两边都迁完才掉 | +| **合计** | **186** | **184** | **+2** | | + +**三个后端桶逐一精确复现 a6 手做的 101 / 14 / 60。** 引用方与定义方的分布也逐一对上： +`UniformManager.cpp` 90、`VulkanRenderer.cpp` 60、`DirectGLES/Managers.cpp` 53、`DirectGLES.cpp` 51； +`TextureObject.cpp` 59、`SamplerObject.cpp` 30、`BufferObject.cpp` 18、`FramebufferObject.cpp` 16、 +`PipeFill.cpp` 9。a6 的 §3.1 / §3.2 一个数都没差。 + +### 2.1 +2 的来源：两个符号，都是 P6.5 的 + +对 a6 发表的 184 逐符号取差集：**新增 2 个，消失 0 个**。 + +| 新增符号 | 引用方 | 是什么 | +|---|---|---| +| `MG_Remote::Client::ClientSession::StartSpawned()` | `MG_Backend/Init.cpp.o` | P6.5 的 fork-per-session 启动路径 | +| `MG_State::GLState::ProgramArtifactsSchemaFingerprint()` | `MG_Remote/CapsCodec.cpp.o` | P6.5 的归档 v2，caps 里带上 schema 指纹 | + +SERVER 集本身也长了：a6 时 45 个 object，现在 52 个（`Transport` +5、`Server` +2，全是 P6.5）。 +Magma 各 wave 改了 `DirectVulkan` 的内容但没有改它要的前端符号集合——这一点本身值得记一笔。 + +### 2.2 a6 自己的四个总数加不齐 + +a6 §3.2 / §5 发表的四个数 **101 + 14 + 60 + 6 = 181**，而它的头条是 **184**，差 3。 +本脚本的四个桶是**严格划分**，恒等于头条。差额落在 `p6-core`：a6 §5 的表只列了它觉得有意思的 +6 个，实际「没有任何后端引用」的是 9 个（184 − 175）。今天是 11 个（9 + §2.1 那 2 个）。 + +对得上的算法是 **9 = 6 − 1 + 4**：a6 §5 列的 6 个里有 1 个（`MGPipeApplierIsUnbarrieredApply()`） +今天已经不在 core 桶（见下）；漏列的 4 个是 `BackendObject_Remote::BackendObject_Remote()`、 +`ClientSession::Start(...)`、`ClientSession::Stop()`、`ClientSessionInstance()`—— +四个全由 `MG_Backend/Init.cpp.o` 引用，即**后端工厂在挑 remote 后端时调 client 的会话对象**。 +a6 §5 的叙述（「server 不该问 client 要答案」）本来就该把它们算进去，只是表里没写。 + +另外 a6 §5 说 `Server/PipeApplier.cpp` 需要 `MGPipeApplierIsUnbarrieredApply()`。 +今天它落在 `p3b-p4b-espryt` 桶里：引用它的只剩 `DirectGLES/DirectGLES.cpp.o`， +`MG_Remote/Server` 已经不引用了。这是 P6.5 之后的真实变化，不是分类差异。 + +## 3 怎么读这张表 + +- **`p7-magma` 的 101 是出口门 4 的分母。** 它只会因为 Magma 真的把功能迁到 server 侧而下降。 +- **`p6-core` 的 11 不是后端的债**，推给 P7/P3b/P4b 是错的。它的形状很清楚： + **9 个是 server 侧在问 client 要答案**（`MG_Backend/Init.cpp.o` 5 个、 + `MG_Pipe/PipeApply.cpp` + `Wire/PipeWireCodec.cpp` 2 个、`Server/PipeApplier.cpp` 2 个）， + 1 个是需要搬到正确模块的解码助手（`DecodeProgramArchive`）， + 1 个是 P6.5 的 schema 指纹（`ProgramArtifactsSchemaFingerprint`）。 +- **`both-backends` 的 60 是最硬的一块**：两个后端都要，所以任何一边单独迁完它都不动。 + +### 3.1 给集成者的一条：`p3b-p4b-espryt` 的 14 个里有 5 个 P3b/P4b 清不掉 + +基线里带 `# P13` 标注的是**由 `MG_Impl/Pipe/SlotAllocator.cpp.o` 定义的全部 7 个符号**。 +规则是机械的、可复核的：要清掉其中任何一个，都得让 slot allocator 不再住在 `MG_Impl/` 下面—— +那是一次模块边界搬迁，契约 §12.1 与 a6 §6.4 都把它记在 P13 头上，**任何后端迁移都够不到它**。 + +这 7 个的落桶是 **`p3b-p4b-espryt` 5 个 + `both-backends` 2 个**： + +| 符号 | 桶 | +|---|---| +| `MG_Pipe::MGPipeApplierIsUnbarrieredApply()` | `p3b-p4b-espryt` | +| `MG_Pipe::MGPipeRefuseAllocatorFromApplyThread(char const*)` | `p3b-p4b-espryt` | +| `MG_Pipe::MGPipeRefuseFrontendKeyedRegistryFromApplyThread(char const*)` | `p3b-p4b-espryt` | +| `MG_Pipe::MGPipeSlotAllocator::Acquire(...)` | `p3b-p4b-espryt` | +| `MG_Pipe::MGPipeSlotAllocator::Free(...)` | `p3b-p4b-espryt` | +| `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId(...) const` | `both-backends` | +| `MG_Pipe::MGPipeSlots()` | `both-backends` | + +也就是说：**`p3b-p4b-espryt` 这个桶的地板是 5，不是 0。** +计划正文把「D12 的 14 个符号」写成 Tier 1（0.5 天）在棘轮落地后做—— +按今天的数据，其中 5 个不在 P3b/P4b 的能力范围内，那一行的估算应按 9 个重做。 +标注只是给人看的，解析器忽略它，门的行为不受影响。 + +## 4 red-once（R-16） + +在 `~/w7/p7-ratchet`（`78b7d6be`，`build-split`）里往一个**被分类为 SERVER 的** object +加一次前端触及，只重编那一个 object，跑门，然后还原。 + +**注意选哪个符号。** 计划书里举的例子 `SamplerObject s(0); s.GetMinFilter();` **不会让门变红**—— +`GetMinFilter()` 已经在基线里（两个后端都在调它）。棘轮只对**新增**符号红， +所以负控必须挑一个今天不在那 186 个里的符号。挑的是 +`SamplerObject::GetBorderColorForm() const`（在 `SamplerObject.cpp.o` 里是 `T`，外部调用， +不会被内联掉；今天没有任何 server object 引用它）。 + +加在 `MG_Backend/DirectGLES/Utils.cpp`（SERVER，`espryt` 族）： + +```cpp + unsigned P7RatchetRedOnceProbe() { + MG_State::GLState::SamplerObject s(0); + return static_cast(s.GetBorderColorForm()); + } +``` + +``` +$ ninja -C build-split CMakeFiles/MobileGL.dir/MobileGL/MG_Backend/DirectGLES/Utils.cpp.o +$ python3 scripts/link_ratchet.py --build-dir build-split \ + --baseline scripts/data/link_ratchet_baseline.txt --assert-monotone +``` + +**实测红线，逐字**： + +``` +link-ratchet: OF WHICH defined by FRONTEND: 187 (a6 measured 184) +link-ratchet: baseline scripts/data/link_ratchet_baseline.txt: 186 symbol(s), 7 annotated +link-ratchet: 1 symbol(s) are NEW frontend reach: a server object now needs a frontend definition the baseline did not record. This is what the gate exists for - either stop reaching, or land the reach with the reason in the commit and re-baseline deliberately: +link-ratchet: NEW: MobileGL::MG_State::GLState::SamplerObject::GetBorderColorForm() const +link-ratchet: defined by: MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o +link-ratchet: referred by: MobileGL/MG_Backend/DirectGLES/Utils.cpp.o +link-ratchet: FAIL --assert-monotone: 1 new frontend symbol(s) (named above) +$ echo $? +1 +``` + +三项计数同步动了：1579 → 1580 未定义、593 → 594 未满足、186 → 187 前端定义。 +还原并重编那一个 object 之后门重新为绿（`ratchet: unchanged at 186 symbol(s)`，退出 0）。 + +**这条红线的价值在第三行和第四行**：它不是「数字变了」，它点名了**哪个符号**、 +**谁定义的**、**谁在够它**。红的那一刻就知道该去看哪个文件。 + +## 5 日常用法 + +```bash +# 看今天的数和分桶 +python3 scripts/link_ratchet.py --build-dir build-split --bucket + +# 门（CI 就是这一行） +python3 scripts/link_ratchet.py --build-dir build-split \ + --baseline scripts/data/link_ratchet_baseline.txt --assert-monotone \ + --json link-ratchet.json + +# 清掉符号之后，在同一个提交里重基线（标注会被带过去） +python3 scripts/link_ratchet.py --build-dir build-split \ + --baseline scripts/data/link_ratchet_baseline.txt \ + --write-baseline scripts/data/link_ratchet_baseline.txt + +# 不需要构建 +python3 scripts/link_ratchet.py --self-test +``` + +**符号消失不是失败**，脚本会把它们逐个列出来并要求在同一个提交里重基线。 +**符号新增是失败**，脚本会对每一个打印它的定义方 object 与全部引用方 object， +所以红出来的那一刻就知道是谁在够谁。 + +## 6 没有验证的（读之前先读这一条） + +**基线是在参考构建 `~/w7/pipe/build-split` 上生成的，不是在 CI 的机器上。** +两边都是 clang + **libstdc++**（CI 装了 `libc++-20-dev` 但 `CMakeLists.txt` 与工作流都没有传 +`-stdlib=libc++`，所以 Linux 上落的是 libstdc++；基线里的 `std::__cxx11::basic_string` 拼写 +就是这么来的），配置三选项与 `CMAKE_BUILD_TYPE=Release` 也一致——**但 clang 版本不同** +（参考构建是 Arch 的 clang，CI 是 `clang++-20`）。 + +内联决策随版本变：某个方法在一边被内联、在另一边留成外部调用，就会让那个符号在 reach 集里 +出现或消失。**所以这道门在 CI 上的第一次运行有可能是红的，且那种红不是代码变坏。** +处理办法是明确的，不要绕过门：看第一次运行上传的 `link-ratchet.json` artifact， +逐个核对 `new_since_baseline` / `cleared_since_baseline`，确认它们全都是内联差异 +（同一个类、同一族方法、没有新的引用方 object），然后用一个单独的提交重基线并在提交信息里 +写明是工具链差异。**如果里面混着一个新的引用方 object，那就不是内联差异。** + +其余： + +- Android NDK 构建没试过，而且**会**不一样：NDK 用 libc++，解构拼写变成 `std::__1::…`， + 整份基线都会算作新增。这道门今天只对 Linux 主机构建有意义。 +- `MobileGL_s`（同源的静态库 target）没量；`--target` 可以指过去，但没跑过。 +- 解构名重合（§1 那 94 处 C1/C2）今天不影响结果，但脚本没有为此加断言—— + 要加的话应当是「reach 集里的每个解构名在 FRONTEND 侧只对应一个 mangling」。 +- `check_flags` 的「选项为 OFF」分支是手工用一个假的 `CMakeCache.txt` 验的， + 不在 `--self-test` 覆盖范围内。 diff --git a/scripts/data/link_ratchet_baseline.txt b/scripts/data/link_ratchet_baseline.txt new file mode 100644 index 00000000..7c8e9126 --- /dev/null +++ b/scripts/data/link_ratchet_baseline.txt @@ -0,0 +1,200 @@ +# MobileGL link-closure ratchet baseline (CONTRACT-P6.md 12.1, ID-P7-7). +# Regenerate: python3 scripts/link_ratchet.py --build-dir \ +# --baseline scripts/data/link_ratchet_baseline.txt \ +# --write-baseline scripts/data/link_ratchet_baseline.txt +# base commit : 78b7d6beacbebb89a5cf8aa412dd3739be5f1247 +# configure : -DMOBILEGL_BUILD_DISAGGREGATED=ON -DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON -DMOBILEGL_PIPE_PUSH=ON -DCMAKE_BUILD_TYPE=Release +# build dir : build-split +# count : 186 +# +# One demangled symbol per line. A trailing ` # note` is an annotation the parser +# ignores; `# P13` marks a symbol that cannot fall before P13 establishes module +# target boundaries. The LIST is the baseline, not the count: a ratchet that +# stored only a number would go green on the day one symbol is cleared and another +# added (ID-P7-7). +MobileGL::MG_Impl::GLImpl::CopyTextureImageToClientOrPBO_State(std::shared_ptr const&, MobileGL::TextureUploadTarget, int, unsigned int, unsigned int, int, void*, char const*) +MobileGL::MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo +MobileGL::MG_Pipe::MGPipeApplierIsUnbarrieredApply() # P13 +MobileGL::MG_Pipe::MGPipeNoteAggregate(MobileGL::MG_Pipe::MGPipeAggregate) +MobileGL::MG_Pipe::MGPipeRefuseAllocatorFromApplyThread(char const*) # P13 +MobileGL::MG_Pipe::MGPipeRefuseFrontendKeyedRegistryFromApplyThread(char const*) # P13 +MobileGL::MG_Pipe::MGPipeSlotAllocator::Acquire(MobileGL::MG_Pipe::MGPipeKind, unsigned long) # P13 +MobileGL::MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId(MobileGL::MG_Pipe::MGPipeKind, unsigned long) const # P13 +MobileGL::MG_Pipe::MGPipeSlotAllocator::Free(MobileGL::MG_Pipe::MGPipeKind, MobileGL::MG_Pipe::MGPipeHandle) # P13 +MobileGL::MG_Pipe::MGPipeSlots() # P13 +MobileGL::MG_Pipe::PipeInputs::GetBufferBindingPointCount(MobileGL::BufferTarget) const +MobileGL::MG_Pipe::PipeInputs::GetProgramObject(unsigned int) +MobileGL::MG_Pipe::PipeInputs::GetTextureObject(unsigned int) +MobileGL::MG_Pipe::PipeInputs::HasOpenTransformFeedbackSpan(unsigned long) const +MobileGL::MG_Pipe::PipeInputs::InvalidateCompileEnv() +MobileGL::MG_Pipe::PipeInputs::IsLive() const +MobileGL::MG_Pipe::PipeInputs::RecordError(MobileGL::ErrorCode, std::unique_ptr >) +MobileGL::MG_Pipe::PipeInputs::ValidateProgramName(unsigned int) const +MobileGL::MG_Remote::Client::AdoptTierIsEmulate() +MobileGL::MG_Remote::Client::BackendObject_Remote::BackendObject_Remote() +MobileGL::MG_Remote::Client::ClientSession::Active() +MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier() +MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadLeftApplier() +MobileGL::MG_Remote::Client::ClientSession::Start(MobileGL::MG_Config::TransportMode, std::__cxx11::basic_string, std::allocator > const&) +MobileGL::MG_Remote::Client::ClientSession::StartSpawned() +MobileGL::MG_Remote::Client::ClientSession::Stop() +MobileGL::MG_Remote::Client::ClientSessionInstance() +MobileGL::MG_Remote::Client::EmitObjectDeathRecord(MobileGL::MG_Pipe::MGPipeKind, unsigned long) +MobileGL::MG_State::GLState::BufferObject::DownloadSubData(void*, unsigned long, unsigned long) const +MobileGL::MG_State::GLState::BufferObject::EnsureGpuResidentStorage() +MobileGL::MG_State::GLState::BufferObject::GetBackendResource() const +MobileGL::MG_State::GLState::BufferObject::GetChangeSerial() const +MobileGL::MG_State::GLState::BufferObject::GetExternalIndex() const +MobileGL::MG_State::GLState::BufferObject::GetSize() const +MobileGL::MG_State::GLState::BufferObject::GetUsage() const +MobileGL::MG_State::GLState::BufferObject::HasDefinedContent() const +MobileGL::MG_State::GLState::BufferObject::IsBackendPersistentMapped() const +MobileGL::MG_State::GLState::BufferObject::IsMapped() const +MobileGL::MG_State::GLState::BufferObject::MappedData() const +MobileGL::MG_State::GLState::BufferObject::MarkGpuWritten() +MobileGL::MG_State::GLState::BufferObject::SetBackendResource(std::shared_ptr) +MobileGL::MG_State::GLState::BufferObject::SyncGpuWrites() +MobileGL::MG_State::GLState::BufferObject::SyncPersistentMappedRange() +MobileGL::MG_State::GLState::BufferObject::WritebackFromBackend(MobileGL::DataPtr, unsigned long) +MobileGL::MG_State::GLState::ClassifyVertexAttribType(unsigned int) +MobileGL::MG_State::GLState::DecodeProgramArchive(unsigned char const*, unsigned long, MobileGL::MG_State::GLState::ProgramArchive&) +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetRenderbuffer() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetSize() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTexture() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureLayer() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureLevel() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureUploadTarget() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsComplete() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsEmpty() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsLayered() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsRenderbuffer() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsTexture() const +MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsValid() const +MobileGL::MG_State::GLState::FramebufferObject::GetAllAttachmentObjects() const +MobileGL::MG_State::GLState::FramebufferObject::GetAttachment(MobileGL::FramebufferAttachmentType) const +MobileGL::MG_State::GLState::FramebufferObject::GetDrawBuffers() const +MobileGL::MG_State::GLState::FramebufferObject::GetExternalIndex() const +MobileGL::MG_State::GLState::GetBufferBackendOps() +MobileGL::MG_State::GLState::MGPipeTextureLegacyArmScope::MGPipeTextureLegacyArmScope() +MobileGL::MG_State::GLState::MGPipeTextureLegacyArmScope::~MGPipeTextureLegacyArmScope() +MobileGL::MG_State::GLState::ProgramArtifactsSchemaFingerprint() +MobileGL::MG_State::GLState::ProgramObject::AllocateLifetimeId() +MobileGL::MG_State::GLState::ProgramObject::AttachShader(std::shared_ptr const&) +MobileGL::MG_State::GLState::ProgramObject::JoinPendingLink() const +MobileGL::MG_State::GLState::ProgramObject::JoinPendingSpirv() const +MobileGL::MG_State::GLState::ProgramObject::Link(bool) +MobileGL::MG_State::GLState::ProgramObject::~ProgramObject() +MobileGL::MG_State::GLState::RenderbufferObject::GetExternalIndex() const +MobileGL::MG_State::GLState::RenderbufferObject::GetHeight() const +MobileGL::MG_State::GLState::RenderbufferObject::GetInternalFormat() const +MobileGL::MG_State::GLState::RenderbufferObject::GetSamples() const +MobileGL::MG_State::GLState::RenderbufferObject::GetWidth() const +MobileGL::MG_State::GLState::RenderbufferObject::IsAllocated() const +MobileGL::MG_State::GLState::SamplerObject::GetAllSamplerParameters() const +MobileGL::MG_State::GLState::SamplerObject::GetBorderColor() const +MobileGL::MG_State::GLState::SamplerObject::GetBorderColorI() const +MobileGL::MG_State::GLState::SamplerObject::GetBorderColorUI() const +MobileGL::MG_State::GLState::SamplerObject::GetCompareMode() const +MobileGL::MG_State::GLState::SamplerObject::GetExternalIndex() const +MobileGL::MG_State::GLState::SamplerObject::GetLifetimeId() const +MobileGL::MG_State::GLState::SamplerObject::GetLodBias() const +MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const +MobileGL::MG_State::GLState::SamplerObject::GetMaxAnisotropy() const +MobileGL::MG_State::GLState::SamplerObject::GetMaxLod() const +MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const +MobileGL::MG_State::GLState::SamplerObject::GetMinLod() const +MobileGL::MG_State::GLState::SamplerObject::GetMipmapMode() const +MobileGL::MG_State::GLState::SamplerObject::GetSamplerCompareFunc() const +MobileGL::MG_State::GLState::SamplerObject::GetVersion() const +MobileGL::MG_State::GLState::SamplerObject::GetWrapR() const +MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const +MobileGL::MG_State::GLState::SamplerObject::GetWrapT() const +MobileGL::MG_State::GLState::SamplerObject::SamplerObject(unsigned int) +MobileGL::MG_State::GLState::SamplerObject::SetCompareMode(MobileGL::SamplerCompareMode) +MobileGL::MG_State::GLState::SamplerObject::SetLodRange(float, float) +MobileGL::MG_State::GLState::SamplerObject::SetMagFilter(MobileGL::SamplerFilterMode) +MobileGL::MG_State::GLState::SamplerObject::SetMinFilter(MobileGL::SamplerFilterMode) +MobileGL::MG_State::GLState::SamplerObject::SetMipmapMode(MobileGL::SamplerMipmapMode) +MobileGL::MG_State::GLState::SamplerObject::SetSamplerCompareFunc(MobileGL::SamplerCompareFunc) +MobileGL::MG_State::GLState::SamplerObject::SetWrapR(MobileGL::SamplerWrapMode) +MobileGL::MG_State::GLState::SamplerObject::SetWrapS(MobileGL::SamplerWrapMode) +MobileGL::MG_State::GLState::SamplerObject::SetWrapT(MobileGL::SamplerWrapMode) +MobileGL::MG_State::GLState::SamplerObject::~SamplerObject() +MobileGL::MG_State::GLState::SamplesAsIncompleteTexture(MobileGL::MG_State::GLState::ITextureObject const*, MobileGL::MG_State::GLState::SamplerObject const*) +MobileGL::MG_State::GLState::SetBufferBackendOps(MobileGL::MG_State::GLState::BufferBackendOps const*) +MobileGL::MG_State::GLState::ShaderObject::Compile() +MobileGL::MG_State::GLState::ShaderObject::DropCompileNode() const +MobileGL::MG_State::GLState::ShaderObject::JoinPendingCompile() const +MobileGL::MG_State::GLState::ShaderObject::ReleaseCompileNode() +MobileGL::MG_State::GLState::ShaderObject::SetShaderSource(std::__cxx11::basic_string, std::allocator >&&) +MobileGL::MG_State::GLState::TextureObject1D::TextureObject1D(unsigned int) +MobileGL::MG_State::GLState::TextureObject2D::TextureObject2D(unsigned int) +MobileGL::MG_State::GLState::TextureObject2DCube::TextureObject2DCube(unsigned int) +MobileGL::MG_State::GLState::TextureObject3D::TextureObject3D(unsigned int) +MobileGL::MG_State::GLState::TextureObjectBase::GetAllSwizzleParams() const +MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColor() const +MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorForm() const +MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorI() const +MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorUI() const +MobileGL::MG_State::GLState::TextureObjectBase::GetContentVersion() const +MobileGL::MG_State::GLState::TextureObjectBase::GetExternalIndex() const +MobileGL::MG_State::GLState::TextureObjectBase::GetFormat() const +MobileGL::MG_State::GLState::TextureObjectBase::GetImmutableLevels() const +MobileGL::MG_State::GLState::TextureObjectBase::GetLevelRange() const +MobileGL::MG_State::GLState::TextureObjectBase::GetLifetimeId() const +MobileGL::MG_State::GLState::TextureObjectBase::GetSamplerObject() const +MobileGL::MG_State::GLState::TextureObjectBase::GetSamples() const +MobileGL::MG_State::GLState::TextureObjectBase::GetShapeVersion() const +MobileGL::MG_State::GLState::TextureObjectBase::GetSwizzleParam(MobileGL::TextureSwizzleParam) const +MobileGL::MG_State::GLState::TextureObjectBase::GetTarget() const +MobileGL::MG_State::GLState::TextureObjectBase::GetTextureParamsVersion() const +MobileGL::MG_State::GLState::TextureObjectBase::GetViewStorageOwner() const +MobileGL::MG_State::GLState::TextureObjectBase::HasFixedSampleLocations() const +MobileGL::MG_State::GLState::TextureObjectBase::IsImmutable() const +MobileGL::MG_State::GLState::TextureObjectBase::IsMipmapCompleteForFilterCached(bool) const +MobileGL::MG_State::GLState::TextureObjectBase::PipePublishParams() +MobileGL::MG_State::GLState::TextureObjectBase::SetBaseLevel(unsigned int) +MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColor(MobileGL::Vec4 const&) +MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColorI(MobileGL::Vec4 const&) +MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColorUI(MobileGL::Vec4 const&) +MobileGL::MG_State::GLState::TextureObjectBase::SetFixedSampleLocations(bool) +MobileGL::MG_State::GLState::TextureObjectBase::SetImmutableLevels(unsigned int) +MobileGL::MG_State::GLState::TextureObjectBase::SetInternalFormat(MobileGL::TextureInternalFormat) +MobileGL::MG_State::GLState::TextureObjectBase::SetMaxLevel(unsigned int) +MobileGL::MG_State::GLState::TextureObjectBase::SetSamples(int) +MobileGL::MG_State::GLState::TextureObjectBase::SetSwizzleParam(MobileGL::TextureSwizzleParam, MobileGL::TextureSwizzleParam) +MobileGL::MG_State::GLState::TextureObjectBase::SetSwizzleParamRGBA(MobileGL::Vec4 const&) +MobileGL::MG_State::GLState::TextureObjectBase::TextureObjectBase(MobileGL::TextureTarget, unsigned int) +MobileGL::MG_State::GLState::TextureObjectBase::~TextureObjectBase() +MobileGL::MG_State::GLState::TextureObjectBuffer::GetBufferBindingSlot(MobileGL::TextureUploadTarget) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::AllocateStorage(MobileGL::TextureUploadTarget, unsigned int, MobileGL::MG_State::GLState::MipmapInput) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetBaseSize() const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapByteSize(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapCompressedByteSize(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapCompressedFormat(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapLevelCount() const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapRequestedCompressedFormat(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapTexelSize(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetStorageDirtyRects(MobileGL::TextureUploadTarget, unsigned int, MobileGL::MG_State::GLState::MipmapDirtyRegion*, unsigned long) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetStorageDirtyRegion(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::IsComplete() const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::IsStorageDirty(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MapMipmapCompressedImage(MobileGL::TextureUploadTarget, unsigned int) const +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MapMipmapData(MobileGL::TextureUploadTarget, unsigned int) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MarkStorageDirty(MobileGL::TextureUploadTarget, unsigned int, bool) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MarkStorageDirtyRegion(MobileGL::TextureUploadTarget, unsigned int, MobileGL::Vec3, MobileGL::Vec3) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::SetMipmapCompressedImage(MobileGL::TextureUploadTarget, unsigned int, unsigned int, void const*, unsigned long) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::SetMipmapRequestedCompressedFormat(MobileGL::TextureUploadTarget, unsigned int, unsigned int) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::TruncateMipmapLevels(MobileGL::TextureUploadTarget, unsigned int) +MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::UpdateMipmapSubData(MobileGL::TextureUploadTarget, unsigned int, MobileGL::DataPtr) +MobileGL::MG_State::GLState::TextureUnit::GetAllBindingSlots() +MobileGL::MG_State::GLState::TextureUnit::GetBindingSlot(MobileGL::TextureTarget) +MobileGL::MG_State::GLState::TextureUnit::GetSamplerObject() const +MobileGL::MG_State::GLState::VertexArrayObject::GetAllAttributeVersions() const +MobileGL::MG_State::GLState::VertexArrayObject::GetAllAttributes() const +MobileGL::MG_State::GLState::VertexArrayObject::GetAttribute(unsigned int) const +MobileGL::MG_State::GLState::VertexArrayObject::GetIndexBufferBindingSlot() +MobileGL::MG_State::GLState::VertexArrayObject::GetIndexBufferBindingSlot() const +typeinfo for MobileGL::MG_State::GLState::TextureObjectBase +typeinfo for MobileGL::MG_State::GLState::TextureObjectWithOneMipmap +vtable for MobileGL::MG_State::GLState::TextureObjectWithOneMipmap diff --git a/scripts/link_ratchet.py b/scripts/link_ratchet.py new file mode 100644 index 00000000..e0892c20 --- /dev/null +++ b/scripts/link_ratchet.py @@ -0,0 +1,861 @@ +#!/usr/bin/env python3 +# MobileGL - scripts/link_ratchet.py +# Copyright (c) 2025-2026 MobileGL-Dev +# Licensed under the GNU Lesser General Public License v3.0: +# https://www.gnu.org/licenses/gpl-3.0.txt +# https://www.gnu.org/licenses/lgpl-3.0.txt +# SPDX-License-Identifier: LGPL-3.0-only +# End of Source File Header +"""The link-closure ratchet: which frontend symbols would a server-only image still need? + +CONTRACT-P6.md 12.1 records the debt as 184 symbols and says "a ratchet recomputes the 184 +in CI and asserts it only ever falls". This is that ratchet. The measurement is a6's +(docs/Disaggregated/notes/p6/a6-link-experiment.md), mechanised: + + (SERVER undefined) - (SERVER defined) - (SHARED defined) intersect (FRONTEND defined) + +i.e. the undefined-symbol list a real link of a server-only image would print today. There is +no module target to link - `SOURCE_FILES` is one flat list feeding `MobileGL` and `MobileGL_s` +(CMakeLists.txt) and establishing module boundaries is P13's named work - so the partition is +applied to the per-object output of the single `MobileGL` target instead. That is exactly why +the partition lives in PARTITION below as an explicit table with a rationale per row rather +than as a regex someone has to reverse-engineer: it IS the module boundary, written down. + + python3 scripts/link_ratchet.py --build-dir build-split + python3 scripts/link_ratchet.py --build-dir build-split --bucket + python3 scripts/link_ratchet.py --build-dir build-split \ + --baseline scripts/data/link_ratchet_baseline.txt --assert-monotone \ + --json link-ratchet.json + python3 scripts/link_ratchet.py --self-test + +Exit codes: 0 green, 1 the ratchet itself went red (new frontend reach), 2 the run could not +be trusted to mean anything (an object no PARTITION row claims, or a build configured without +the three disaggregated options). The two are separate on purpose - a 2 is never "the code got +worse", it is "fix the measurement first". + +WHAT THE GATE IS FOR (ID-P7-7). The baseline stores the symbol LIST, not a count. A count-only +ratchet passes on the day one symbol is cleared and another added, which is precisely the +regression this gate exists to catch: new backend code reaching into a frontend object. So +--assert-monotone fails on symbols present now and ABSENT from the baseline, and merely reports +symbols that disappeared, with an instruction to re-baseline in the same commit. Progress is +never a red build; new reach always is. + +GUARD RAILS. The build must be a disaggregated one +(-DMOBILEGL_BUILD_DISAGGREGATED=ON -DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON +-DMOBILEGL_PIPE_PUSH=ON): with the option OFF, MG_Remote is not compiled at all, the SERVER set +is missing its transport half and the script would report a cheerfully small number that means +nothing. The run reads CMakeCache.txt and refuses such a build by name unless it is given +--no-require-flags. Two more refusals for the same reason - every way this gate can go green +without having measured anything is an explicit exit 2: + * an object that matches no PARTITION row (the tree moved and the partition did not), and + * an empty SERVER, FRONTEND or SHARED set (the target was never built in that directory, + which otherwise classifies perfectly into three empty sets and reports a beautiful zero). + +SYMBOL IDENTITY, and the one place this is looser than a linker. nm is run with -C, so the unit +of the set arithmetic is the DEMANGLED name - what a6 published and what a reviewer can read +and grep for. That is deliberately not the linker's unit. Measured on the base tree, the +FRONTEND set defines 14974 distinct manglings under 14880 distinct demangled spellings: the 94 +collisions are overwhelmingly the C1/C2 (complete/base) constructor and D1/D2 destructor pairs, +which demangle to the same text by construction, plus `[abi:cxx11]` tags and `.cold` parts. The +consequence to know about: if a server object ever referenced the C1 form of a constructor that +the frontend defined only as C2, this script would call it satisfied and a real link would not. +Both forms are emitted together in practice, so no such case exists today - but a demangled +ratchet is a readable approximation of a link, not a link. `scripts/symbol_report.py` is the +tool that works in manglings when the mangling is the point. + +Only `U` counts as a reference. A weak undefined (`w`) resolves to zero when nothing defines +it, so it obliges nobody; on the base tree no `w` reaches a frontend definition, so this +filter costs nothing today and keeps the number honest the day it does. + +THE BASELINE IS TOOLCHAIN-SENSITIVE, and this is the one way this gate goes red without any +code having got worse. Whether a small method is inlined or left as an out-of-line call is a +compiler-version decision, and it moves symbols into and out of the reach set. The stored +baseline was generated with a different clang than CI's `clang++-20` (same libstdc++, same +Release, same three options). If the first CI run is red, read the uploaded JSON before +touching anything: if every entry in `new_since_baseline` is a sibling method of a class +already in the list and brings NO new referring object, it is an inlining difference and the +answer is one deliberate re-baseline commit that says so. A new REFERRING OBJECT is never an +inlining difference - that is the real thing this gate is looking for. An Android NDK build is +a different question again: the NDK is libc++, every `std::` spelling changes, and the whole +baseline would read as new. This gate is for Linux host builds. +""" + +import argparse +import json +import os +import re +import subprocess +import sys + +PREFIX = "link-ratchet: " + +SERVER = "SERVER" +FRONTEND = "FRONTEND" +SHARED = "SHARED" + +# --------------------------------------------------------------------------- +# The partition. This table IS the module boundary that the build system does not have. +# +# Rows are (object path prefix, role, why). Paths are relative to +# /CMakeFiles/.dir/ and a row ending in `.o` is an exact object, not a +# prefix. Matching is LONGEST PREFIX WINS, so `MG_Remote/Client/` (FRONTEND) overrides nothing +# here only because no shorter `MG_Remote/` row exists - if one is ever added, the longer row +# still wins and the intent stays readable. +# +# Provenance: a6-link-experiment-data.md lines 6-8 name the three sets. Every row below that is +# not in a6's three lines carries "(post-a6)" in its rationale and is called out in the report, +# because a partition that silently absorbed a new directory would silently change the number. +# --------------------------------------------------------------------------- +PARTITION = ( + # --- SERVER: what a server-only image would be built from --------------------------- + ("MobileGL/MG_Backend/", SERVER, + "backend translation - GL/Vulkan command submission is the server's entire job"), + ("MobileGL/MG_Pipe/", SERVER, + "the applier and its routing decode into the backend inside the server process"), + ("MobileGL/MG_Remote/Transport/", SERVER, + "the transport stack; the server end of every link lives here"), + ("MobileGL/MG_Remote/Protocol/", SERVER, + "wire schema helpers used by the server's decode path"), + ("MobileGL/MG_Remote/Wire/", SERVER, + "the wire codec the server decodes with"), + ("MobileGL/MG_Remote/Server/", SERVER, + "the server loop, session, applier and spawn"), + ("MobileGL/MG_Remote/CapsCodec.cpp.o", SERVER, + "the server encodes the caps blob the client mirrors; a6 names this object by hand"), + + # --- FRONTEND: what a server-only image is supposed to be able to drop --------------- + ("MobileGL/MG_Impl/", FRONTEND, + "the GL/EGL/GLX entry points and the client-side pipe fill - guest-process code"), + ("MobileGL/MG_State/", FRONTEND, + "the GL object model; the server is meant to see records on the wire, not these objects"), + ("MobileGL/MG_Remote/Client/", FRONTEND, + "the client half of the session - by definition not in a server image"), + + # --- SHARED: linked into both images, so it satisfies references without being reach -- + ("MobileGL/MG_Util/", SHARED, + "role-neutral utilities: converters, transpiler, metrics, math"), + ("MobileGL/MG_Remote/FatalFunnel.cpp.o", SHARED, + "(post-a6, landed in 5968863f) every session death funnels through it from BOTH roles, " + "so it is role-neutral infrastructure. a6 predates the file and names no rule for it"), + ("MobileGL/Init.cpp.o", SHARED, + "library init, runs in whichever process loaded the library"), + ("MobileGL/GlobalObjects.cpp.o", SHARED, + "the process-global singletons both roles reach"), + ("MobileGL/ConfigLoader.cpp.o", SHARED, + "environment/config parsing, read by both roles"), +) + +# --------------------------------------------------------------------------- +# Referrer families, for --bucket. a6 attributed the debt to P7 (Magma), P3b/P4b (Espryt), +# both, and "P6's own" - which it derived by hand from who REFERS to each symbol. Same table +# shape, same longest-prefix rule; every SERVER object falls in exactly one family. +# --------------------------------------------------------------------------- +FAMILIES = ( + ("MobileGL/MG_Backend/DirectVulkan/", "magma", + "Magma / DirectVulkan - P7's full migration"), + ("MobileGL/MG_Backend/DirectGLES/", "espryt", + "Espryt / DirectGLES - P3b/P4b's deepening"), + ("", "core", + "server-side code that is neither backend: MG_Remote transport/wire/server, the MG_Pipe " + "applier, and the backend-neutral MG_Backend glue. a6 calls this bucket 'P6's own'"), +) + +BUCKETS = ( + ("p6-core", "referred to by NO backend object: the transport/applier's own account " + "(a6: 'P6 self' = 6)"), + ("p7-magma", "referred to only by DirectVulkan: P7's account (a6: 101)"), + ("p3b-p4b-espryt", "referred to only by DirectGLES: P3b/P4b's account (a6: 14)"), + ("both-backends", "referred to by both backends: falls only when both migrate (a6: 60)"), +) + +A6_BUCKET_TOTALS = {"p6-core": 6, "p7-magma": 101, "p3b-p4b-espryt": 14, "both-backends": 60} +A6_TOTAL = 184 + +# `nm -C -o` prints ": ". The demangled name +# contains spaces, `<`, `,` and `::`, so the type letter is located by position after the +# known object path rather than by splitting the whole line. +NM_BODY_RE = re.compile(r"^\s*([0-9a-fA-F]*)\s+([A-Za-z?])\s(.+)$") + +# A baseline line may carry a trailing ` # note` annotation (ID-P7-7 asks for `# P13` on the +# symbols that cannot fall before P13). No demangled C++ name contains " #", so splitting on +# the first occurrence is unambiguous. +BASELINE_ANNOTATION = " #" + +REQUIRED_FLAGS = ( + ("MOBILEGL_BUILD_DISAGGREGATED", "ON"), + ("MOBILEGL_BUILD_DISAGGREGATED_INPROC", "ON"), + ("MOBILEGL_PIPE_PUSH", "ON"), +) + + +def say(message): + print(PREFIX + message) + + +def classify(obj_path, table=PARTITION): + """Longest-prefix match against the explicit table. None means: no row claims this object.""" + best = None + for prefix, role, _why in table: + if obj_path == prefix or (prefix.endswith("/") and obj_path.startswith(prefix)): + if best is None or len(prefix) > len(best[0]): + best = (prefix, role) + return best[1] if best else None + + +def family_of(obj_path, table=FAMILIES): + """Which referrer family a SERVER object belongs to. The empty prefix row is the default.""" + best = ("", None) + for prefix, name, _why in table: + if prefix == "" or obj_path.startswith(prefix): + if name is not None and len(prefix) >= len(best[0]): + best = (prefix, name) + return best[1] + + +def find_objects(build_dir, target): + """Every .o under /CMakeFiles/.dir, keyed by path relative to that dir.""" + root = os.path.join(build_dir, "CMakeFiles", target + ".dir") + if not os.path.isdir(root): + raise SystemExit(PREFIX + "no such object directory: {} (is --build-dir a configured " + "CMake build, and is --target right? the disaggregated build " + "has MobileGL.dir, MobileGL_s.dir and MobileGLServer.dir)" + .format(root)) + objects = [] + for dirpath, _dirnames, filenames in os.walk(root): + for name in filenames: + if name.endswith(".o"): + full = os.path.join(dirpath, name) + objects.append(os.path.relpath(full, root).replace(os.sep, "/")) + return root, sorted(objects) + + +def check_flags(build_dir): + """Refuse a build whose options make the measurement meaningless. Returns the flag lines.""" + cache = os.path.join(build_dir, "CMakeCache.txt") + if not os.path.isfile(cache): + return None, ["no CMakeCache.txt in {} - cannot confirm the build is disaggregated" + .format(build_dir)] + values = {} + with open(cache, "r", encoding="utf-8", errors="replace") as handle: + for line in handle: + match = re.match(r"^([A-Za-z0-9_]+):[A-Z]+=(.*)$", line.strip()) + if match: + values[match.group(1)] = match.group(2) + problems = [] + for name, wanted in REQUIRED_FLAGS: + got = values.get(name) + if got != wanted: + problems.append( + "{}={} (need {}). With the option off, MG_Remote is not compiled at all, the " + "SERVER set has no transport half, and every number below would be a small, " + "green, meaningless one.".format(name, got if got is not None else "", + wanted)) + return values, problems + + +def run_nm(nm_tool, mode, paths, cwd, chunk=150): + """`nm -C -o ---only` over paths, returned as raw text. Chunked for argv limits.""" + out = [] + for i in range(0, len(paths), chunk): + batch = paths[i:i + chunk] + completed = subprocess.run([nm_tool, "-C", "-o", "--" + mode + "-only"] + batch, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + text=True, cwd=cwd) + # nm exits non-zero on a file it cannot read; an object it merely has no symbols for is + # not an error. Anything on stderr is worth surfacing - a silently skipped object is a + # silently shrunk SERVER set. + if completed.returncode != 0 and completed.stderr.strip(): + raise SystemExit(PREFIX + "nm failed: " + completed.stderr.strip()) + out.append(completed.stdout) + return "".join(out) + + +def parse_nm(text, known_objects): + """`nm -C -o` transcript -> {object: {name: type-letter}}. Keys come from known_objects.""" + table = {obj: {} for obj in known_objects} + ordered = sorted(known_objects, key=len, reverse=True) + for line in text.splitlines(): + if not line.strip(): + continue + obj = None + for candidate in ordered: + if line.startswith(candidate + ":"): + obj = candidate + break + if obj is None: + continue + match = NM_BODY_RE.match(line[len(obj) + 1:]) + if not match: + continue + table[obj][match.group(3)] = match.group(2) + return table + + +def compute(roles, undefined, defined): + """The set arithmetic, on plain dicts so the self-test can drive it without a build. + + roles : {object: SERVER|FRONTEND|SHARED} + undefined : {object: {name: type}} - only SERVER objects need to be present + defined : {object: {name: type}} - every object + + Returns the sorted symbol list plus the per-symbol referrer/definer attribution. + """ + server = [o for o, r in roles.items() if r == SERVER] + frontend = [o for o, r in roles.items() if r == FRONTEND] + shared = [o for o, r in roles.items() if r == SHARED] + + referrers = {} + for obj in server: + for name, kind in undefined.get(obj, {}).items(): + # `U` only. A weak undefined (`w`) resolves to zero when nothing defines it, so it + # is not a reference that obliges anybody. + if kind != "U": + continue + referrers.setdefault(name, set()).add(obj) + + server_defined = set() + for obj in server: + server_defined.update(defined.get(obj, {})) + shared_defined = set() + for obj in shared: + shared_defined.update(defined.get(obj, {})) + + definers = {} + for obj in frontend: + for name in defined.get(obj, {}): + definers.setdefault(name, set()).add(obj) + + server_undefined = set(referrers) + unsatisfied = server_undefined - server_defined - shared_defined + reach = sorted(unsatisfied & set(definers)) + + return { + "symbols": reach, + "referrers": {name: sorted(referrers[name]) for name in reach}, + "definers": {name: sorted(definers[name]) for name in reach}, + "counts": { + "server_objects": len(server), + "frontend_objects": len(frontend), + "shared_objects": len(shared), + "server_undefined": len(server_undefined), + "unsatisfied_by_server_or_shared": len(unsatisfied), + "defined_by_frontend": len(reach), + }, + } + + +def bucketise(result): + """Group the reach by referrer family, reproducing a6's four hand-made buckets.""" + buckets = {name: [] for name, _why in BUCKETS} + also_core = [] + for name in result["symbols"]: + families = {family_of(obj) for obj in result["referrers"][name]} + backends = families & {"magma", "espryt"} + if not backends: + buckets["p6-core"].append(name) + else: + if backends == {"magma"}: + buckets["p7-magma"].append(name) + elif backends == {"espryt"}: + buckets["p3b-p4b-espryt"].append(name) + else: + buckets["both-backends"].append(name) + if "core" in families: + also_core.append(name) + return buckets, also_core + + +def read_baseline(path): + """Baseline file -> (symbols, {symbol: annotation}). `#` at line start is a comment.""" + symbols = [] + notes = {} + with open(path, "r", encoding="utf-8") as handle: + for line in handle: + line = line.rstrip("\n").rstrip("\r") + if not line.strip() or line.lstrip().startswith("#"): + continue + if BASELINE_ANNOTATION in line: + symbol, annotation = line.split(BASELINE_ANNOTATION, 1) + symbol = symbol.rstrip() + notes[symbol] = annotation.strip() + else: + symbol = line.rstrip() + symbols.append(symbol) + return symbols, notes + + +def monotone(current, baseline): + """(new, gone). `new` is the red: reach that was not in the baseline.""" + current_set = set(current) + baseline_set = set(baseline) + return sorted(current_set - baseline_set), sorted(baseline_set - current_set) + + +# --------------------------------------------------------------------------- +# Self-test: the classification and the set arithmetic on synthetic nm output, no build. +# --------------------------------------------------------------------------- +CANNED_OBJECTS = { + # SERVER + "MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o": SERVER, + "MobileGL/MG_Backend/DirectGLES/Managers.cpp.o": SERVER, + "MobileGL/MG_Backend/BackendObject.cpp.o": SERVER, + "MobileGL/MG_Remote/Server/PipeApplier.cpp.o": SERVER, + "MobileGL/MG_Pipe/PipeApply.cpp.o": SERVER, + "MobileGL/MG_Remote/CapsCodec.cpp.o": SERVER, + # FRONTEND + "MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o": FRONTEND, + "MobileGL/MG_Impl/Pipe/SlotAllocator.cpp.o": FRONTEND, + "MobileGL/MG_Remote/Client/ClientSession.cpp.o": FRONTEND, + # SHARED + "MobileGL/MG_Util/Math/Vec.cpp.o": SHARED, + "MobileGL/MG_Remote/FatalFunnel.cpp.o": SHARED, + "MobileGL/ConfigLoader.cpp.o": SHARED, +} + +CANNED_UNDEFINED = { + "MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o": { + "MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const": "U", + "MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "U", + "MobileGL::Shared::Helper()": "U", # satisfied by SHARED + "MobileGL::MG_Backend::BackendObject::Tick()": "U", # satisfied by SERVER + "MobileGL::Nowhere::Absent()": "U", # satisfied by nobody: not our business + "MobileGL::MG_State::GLState::WeakOnly()": "w", # weak undef: not a reference + }, + "MobileGL/MG_Backend/DirectGLES/Managers.cpp.o": { + "MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "U", + "MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const": "U", + }, + "MobileGL/MG_Remote/Server/PipeApplier.cpp.o": { + "MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()": "U", + }, + "MobileGL/MG_Pipe/PipeApply.cpp.o": { + "MobileGL::MG_Pipe::MGPipeSlots()": "U", + }, + "MobileGL/MG_Backend/BackendObject.cpp.o": {}, + "MobileGL/MG_Remote/CapsCodec.cpp.o": {}, +} + +CANNED_DEFINED = { + "MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o": { + "MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const": "T", + "MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "T", + "MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const": "T", + "MobileGL::MG_State::GLState::WeakOnly()": "W", + }, + "MobileGL/MG_Impl/Pipe/SlotAllocator.cpp.o": { + "MobileGL::MG_Pipe::MGPipeSlots()": "T", + }, + "MobileGL/MG_Remote/Client/ClientSession.cpp.o": { + "MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()": "T", + }, + "MobileGL/MG_Util/Math/Vec.cpp.o": { + "MobileGL::Shared::Helper()": "T", + }, + "MobileGL/MG_Backend/BackendObject.cpp.o": { + "MobileGL::MG_Backend::BackendObject::Tick()": "T", + }, + "MobileGL/MG_Remote/FatalFunnel.cpp.o": {}, + "MobileGL/ConfigLoader.cpp.o": {}, +} + +CANNED_NM_TRANSCRIPT = """\ +MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o: U MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const +MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o: w MobileGL::MG_State::GLState::WeakOnly() +MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o:0000000000000010 T MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const +MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o:0000000000000020 W MobileGL::MG_State::GLState::WeakOnly() +this line belongs to no known object and must be ignored +""" + + +def self_test(): + problems = [] + + # --- the partition table --------------------------------------------------------- + expected_roles = [ + ("MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.cpp.o", SERVER), + ("MobileGL/MG_Pipe/PipeRoute.cpp.o", SERVER), + ("MobileGL/MG_Remote/Transport/ShmLink.cpp.o", SERVER), + ("MobileGL/MG_Remote/Server/ServerLoop.cpp.o", SERVER), + ("MobileGL/MG_Remote/CapsCodec.cpp.o", SERVER), + ("MobileGL/MG_Remote/Client/ClientSession.cpp.o", FRONTEND), + ("MobileGL/MG_State/GLState/TextureState/TextureObject.cpp.o", FRONTEND), + ("MobileGL/MG_Impl/GLImpl/Texture/GL_Texture.cpp.o", FRONTEND), + ("MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/Pass.cpp.o", SHARED), + ("MobileGL/MG_Remote/FatalFunnel.cpp.o", SHARED), + ("MobileGL/ConfigLoader.cpp.o", SHARED), + ("MobileGL/MG_Nowhere/Invented.cpp.o", None), + ] + for path, want in expected_roles: + got = classify(path) + if got != want: + problems.append("classify({}) = {}, expected {}".format(path, got, want)) + # Longest prefix wins even when a shorter row would also match. + longer = PARTITION + (("MobileGL/", SHARED, "hypothetical catch-all"),) + if classify("MobileGL/MG_Remote/Server/ServerLoop.cpp.o", longer) != SERVER: + problems.append("longest-prefix match lost to a shorter catch-all row") + + for path, want in (("MobileGL/MG_Backend/DirectVulkan/DirectVulkan.cpp.o", "magma"), + ("MobileGL/MG_Backend/DirectGLES/Utils.cpp.o", "espryt"), + ("MobileGL/MG_Backend/BackendObject.cpp.o", "core"), + ("MobileGL/MG_Remote/Server/PipeApplier.cpp.o", "core"), + ("MobileGL/MG_Pipe/PipeApply.cpp.o", "core")): + got = family_of(path) + if got != want: + problems.append("family_of({}) = {}, expected {}".format(path, got, want)) + + # --- the nm parser --------------------------------------------------------------- + parsed = parse_nm(CANNED_NM_TRANSCRIPT, [ + "MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o", + "MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o"]) + uni = parsed["MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o"] + sam = parsed["MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o"] + if uni.get("MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const") != "U": + problems.append("nm parse: undefined line with a spaced demangled name: {}".format(uni)) + if uni.get("MobileGL::MG_State::GLState::WeakOnly()") != "w": + problems.append("nm parse: weak-undefined type letter lost: {}".format(uni)) + if sam.get("MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const") != "T": + problems.append("nm parse: defined line with an address: {}".format(sam)) + if len(uni) != 2 or len(sam) != 2: + problems.append("nm parse: unowned line was not ignored: {} / {}".format(uni, sam)) + + # --- the set arithmetic ---------------------------------------------------------- + result = compute(CANNED_OBJECTS, CANNED_UNDEFINED, CANNED_DEFINED) + want_symbols = [ + "MobileGL::MG_Pipe::MGPipeSlots()", + "MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()", + "MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const", + "MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const", + "MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const", + ] + if result["symbols"] != want_symbols: + problems.append("reach set: {}".format(result["symbols"])) + # Each exclusion for its own reason, so a regression names which subtraction broke. + if "MobileGL::Shared::Helper()" in result["symbols"]: + problems.append("a SHARED-defined symbol survived the subtraction") + if "MobileGL::MG_Backend::BackendObject::Tick()" in result["symbols"]: + problems.append("a SERVER-defined symbol survived the subtraction") + if "MobileGL::Nowhere::Absent()" in result["symbols"]: + problems.append("a symbol nobody defines was reported as frontend reach") + if "MobileGL::MG_State::GLState::WeakOnly()" in result["symbols"]: + problems.append("a weak undefined (`w`) was counted as a reference") + # 8 distinct `U` names across the four referring server objects (the `w` is not one of + # them, and GetMinFilter is referred to twice but counted once); minus Tick (SERVER-defined) + # and Helper (SHARED-defined) leaves 6; of those, Absent is defined by nobody, leaving 5. + if result["counts"]["server_undefined"] != 8: + problems.append("server undefined count: {}".format(result["counts"])) + if result["counts"]["unsatisfied_by_server_or_shared"] != 6: + problems.append("unsatisfied count: {}".format(result["counts"])) + if result["counts"]["defined_by_frontend"] != 5: + problems.append("frontend-defined count: {}".format(result["counts"])) + if (result["counts"]["server_objects"], result["counts"]["frontend_objects"], + result["counts"]["shared_objects"]) != (6, 3, 3): + problems.append("partition counts: {}".format(result["counts"])) + + # --- the buckets ------------------------------------------------------------------ + buckets, also_core = bucketise(result) + want_buckets = { + "p6-core": ["MobileGL::MG_Pipe::MGPipeSlots()", + "MobileGL::MG_Remote::Client::ClientSession::" + "NoteApplyThreadEnteredApplier()"], + "p7-magma": ["MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const"], + "p3b-p4b-espryt": ["MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const"], + "both-backends": ["MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const"], + } + if buckets != want_buckets: + problems.append("buckets: {}".format(buckets)) + if sum(len(v) for v in buckets.values()) != len(result["symbols"]): + problems.append("buckets do not partition the reach set") + if also_core: + problems.append("unexpected backend+core overlap: {}".format(also_core)) + + # --- the ratchet ------------------------------------------------------------------ + base = ["a", "b", "c"] + new, gone = monotone(["b", "c", "d"], base) + if new != ["d"] or gone != ["a"]: + problems.append("monotone(): new={} gone={}".format(new, gone)) + if monotone(["a", "b"], base) != ([], ["c"]): + problems.append("a shrunk set must be green with a re-baseline note") + if monotone(base, base) != ([], []): + problems.append("an identical set must be silent") + # ID-P7-7's whole point: one cleared, one added, same count -> still red. + new, gone = monotone(["a", "b", "z"], base) + if new != ["z"] or gone != ["c"]: + problems.append("clear-one-add-one at equal count did not name the new symbol") + + # --- the baseline parser ---------------------------------------------------------- + import tempfile + with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False, + encoding="utf-8", newline="\n") as handle: + handle.write("# a header comment\n" + "# base: deadbeef\n" + "\n" + "MobileGL::MG_Pipe::MGPipeSlots() # P13\n" + "MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const\n" + "MobileGL::Vec4::Vec4(int, int, int, int) # P13 allocator-shaped\n") + tmp = handle.name + try: + symbols, notes = read_baseline(tmp) + if symbols != ["MobileGL::MG_Pipe::MGPipeSlots()", + "MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const", + "MobileGL::Vec4::Vec4(int, int, int, int)"]: + problems.append("baseline parse dropped or mangled a symbol: {}".format(symbols)) + if notes.get("MobileGL::MG_Pipe::MGPipeSlots()") != "P13": + problems.append("baseline annotation not read: {}".format(notes)) + if len(notes) != 2: + problems.append("baseline annotations: {}".format(notes)) + finally: + os.unlink(tmp) + + for problem in problems: + say("self-test: " + problem) + say("self-test: " + ("OK (partition, families, nm parser, set arithmetic, buckets, " + "ratchet, baseline parser)" if not problems else "FAILED")) + return 0 if not problems else 1 + + +def render_buckets(result, buckets, also_core, verbose): + lines = [] + lines.append("") + lines.append("| bucket | symbols | a6 | delta | meaning |") + lines.append("|---|---:|---:|---:|---|") + for name, why in BUCKETS: + count = len(buckets[name]) + a6 = A6_BUCKET_TOTALS[name] + lines.append("| `{}` | {} | {} | {:+d} | {} |".format(name, count, a6, count - a6, why)) + total = sum(len(v) for v in buckets.values()) + lines.append("| **total** | **{}** | **{}** | **{:+d}** | |".format( + total, A6_TOTAL, total - A6_TOTAL)) + lines.append("") + lines.append("a6's four published totals sum to {} against its own headline of {}; the " + "buckets above are a strict partition, so they always sum to the headline." + .format(sum(A6_BUCKET_TOTALS.values()), A6_TOTAL)) + if also_core: + lines.append("") + lines.append("{} backend-bucket symbol(s) are ALSO referred to by a core object, so " + "they do not fall when the backend that owns them migrates:".format( + len(also_core))) + for name in also_core: + lines.append(" - " + name) + if verbose: + for name, _why in BUCKETS: + lines.append("") + lines.append("### {} ({})".format(name, len(buckets[name]))) + for symbol in buckets[name]: + lines.append(" {}".format(symbol)) + lines.append(" defined by: " + ", ".join(result["definers"][symbol])) + lines.append(" referred by: " + ", ".join(result["referrers"][symbol])) + return lines + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--build-dir", help="a configured, BUILT disaggregated CMake build dir") + parser.add_argument("--target", default="MobileGL", + help="the CMake target whose objects carry the partition " + "(default: MobileGL; MobileGL_s is the same sources as a static " + "lib, MobileGLServer is the P0 spike stub)") + parser.add_argument("--nm", default="nm") + parser.add_argument("--baseline", help="the stored symbol list") + parser.add_argument("--assert-monotone", action="store_true", + help="exit non-zero for every symbol present now and absent from the " + "baseline. Symbols that vanished are reported and are never a " + "failure - progress does not turn the build red.") + parser.add_argument("--bucket", action="store_true", + help="group the reach by referring object family and print the " + "P7 / P3b-P4b / both / core totals against a6's") + parser.add_argument("--verbose-buckets", action="store_true", + help="with --bucket, list every symbol with its definer and referrers") + parser.add_argument("--list", action="store_true", + help="print the symbol list itself (implied when no baseline is given)") + parser.add_argument("--json", help="write the machine-readable result here") + parser.add_argument("--write-baseline", metavar="FILE", + help="write the current list out in baseline format, annotations from " + "--baseline carried over") + parser.add_argument("--no-require-flags", dest="require_flags", action="store_false", + help="do not refuse a build whose CMakeCache lacks the three " + "disaggregated options (for inspecting an odd build by hand)") + parser.add_argument("--self-test", action="store_true") + args = parser.parse_args() + + if args.self_test: + return self_test() + if not args.build_dir: + parser.error("--build-dir is required (or use --self-test)") + if args.assert_monotone and not args.baseline: + parser.error("--assert-monotone needs --baseline") + if args.baseline and not os.path.isfile(args.baseline): + # Named, not a traceback: in CI this is a path typo or a missing checkout, and the two + # look identical in a stack trace. + say("FAIL no baseline file at {} (the ratchet has nothing to compare against; " + "scripts/data/link_ratchet_baseline.txt is the one CI uses)".format(args.baseline)) + return 2 + + cache_values, flag_problems = check_flags(args.build_dir) + if flag_problems: + for problem in flag_problems: + say(("FAIL " if args.require_flags else "warning: ") + problem) + if args.require_flags: + return 2 + + root, objects = find_objects(args.build_dir, args.target) + roles = {} + unclassified = [] + for obj in objects: + role = classify(obj) + if role is None: + unclassified.append(obj) + else: + roles[obj] = role + if unclassified: + say("FAIL {} object(s) match no PARTITION row. The tree moved and this measurement " + "no longer measures what it claims to; add a row (with its reason) rather than a " + "catch-all:".format(len(unclassified))) + for obj in unclassified: + say(" unclassified: " + obj) + return 2 + + # An unbuilt (or half-built) directory classifies cleanly into three empty sets and would + # otherwise report a beautiful zero. The ratchet's happiest possible output must not also be + # its output for "there was nothing to look at". + empty = [name for name, members in ((SERVER, [o for o, r in roles.items() if r == SERVER]), + (FRONTEND, [o for o, r in roles.items() if r == FRONTEND]), + (SHARED, [o for o, r in roles.items() if r == SHARED])) + if not members] + if empty: + say("FAIL {} set(s) are empty ({}) in {} - the target was not built, or was built " + "somewhere else. A ratchet that reports 0 because it found no objects is worse " + "than no ratchet.".format(len(empty), ", ".join(empty), root)) + return 2 + + server = sorted(o for o, r in roles.items() if r == SERVER) + undefined = parse_nm(run_nm(args.nm, "undefined", server, root), server) + defined = parse_nm(run_nm(args.nm, "defined", objects, root), objects) + result = compute(roles, undefined, defined) + + counts = result["counts"] + say("build-dir {} (target {}, {} objects)".format(args.build_dir, args.target, len(objects))) + if cache_values: + say("configure: " + ", ".join("{}={}".format(n, cache_values.get(n, "")) + for n, _ in REQUIRED_FLAGS) + + ", CMAKE_BUILD_TYPE=" + cache_values.get("CMAKE_BUILD_TYPE", "")) + say("partition: SERVER {} / FRONTEND {} / SHARED {} objects".format( + counts["server_objects"], counts["frontend_objects"], counts["shared_objects"])) + say("SERVER undefined symbols: {}".format(counts["server_undefined"])) + say(" not satisfied by SERVER or SHARED: {}".format( + counts["unsatisfied_by_server_or_shared"])) + say(" OF WHICH defined by FRONTEND: {} (a6 measured {})".format( + counts["defined_by_frontend"], A6_TOTAL)) + + buckets, also_core = bucketise(result) + if args.bucket: + print("\n".join(render_buckets(result, buckets, also_core, args.verbose_buckets))) + + if args.list or not args.baseline: + print("") + for symbol in result["symbols"]: + print(symbol) + + exit_code = 0 + new, gone = [], [] + baseline_symbols, baseline_notes = [], {} + if args.baseline: + baseline_symbols, baseline_notes = read_baseline(args.baseline) + new, gone = monotone(result["symbols"], baseline_symbols) + say("baseline {}: {} symbol(s), {} annotated".format( + args.baseline, len(baseline_symbols), len(baseline_notes))) + if gone: + say("{} symbol(s) in the baseline are GONE. That is the ratchet turning - not a " + "failure. Re-baseline in the same commit that cleared them " + "(--write-baseline):".format(len(gone))) + for symbol in gone: + say(" cleared: " + symbol + + ((" [" + baseline_notes[symbol] + "]") if symbol in baseline_notes + else "")) + if new: + say("{} symbol(s) are NEW frontend reach: a server object now needs a frontend " + "definition the baseline did not record. This is what the gate exists for - " + "either stop reaching, or land the reach with the reason in the commit and " + "re-baseline deliberately:".format(len(new))) + for symbol in new: + say(" NEW: " + symbol) + say(" defined by: " + ", ".join(result["definers"][symbol])) + say(" referred by: " + ", ".join(result["referrers"][symbol])) + if args.assert_monotone and new: + exit_code = 1 + if not new and not gone: + say("ratchet: unchanged at {} symbol(s)".format(len(result["symbols"]))) + + if args.write_baseline: + if not args.baseline: + say("warning: --write-baseline without --baseline: the ` # P13` annotations of the " + "old file cannot be carried over and will be lost. Pass --baseline too.") + write_baseline(args.write_baseline, result["symbols"], baseline_notes, cache_values, + args.build_dir) + say("baseline written to {} ({} symbol(s), {} annotation(s) carried over)".format( + args.write_baseline, len(result["symbols"]), + sum(1 for s in result["symbols"] if s in baseline_notes))) + + if args.json: + payload = { + "build_dir": args.build_dir, + "target": args.target, + "configure": {name: (cache_values or {}).get(name) for name, _ in REQUIRED_FLAGS}, + "counts": counts, + # The partition itself travels with the artifact: a reviewer reading the JSON six + # months from now should not have to guess which objects were on which side. + "roles": roles, + "a6": {"total": A6_TOTAL, "buckets": A6_BUCKET_TOTALS}, + "symbols": result["symbols"], + "buckets": {name: buckets[name] for name, _ in BUCKETS}, + "bucket_counts": {name: len(buckets[name]) for name, _ in BUCKETS}, + "backend_symbols_also_referred_by_core": also_core, + "definers": result["definers"], + "referrers": result["referrers"], + "baseline": args.baseline, + "baseline_count": len(baseline_symbols) if args.baseline else None, + "new_since_baseline": new, + "cleared_since_baseline": gone, + "assert_monotone": args.assert_monotone, + "exit_code": exit_code, + } + with open(args.json, "w", encoding="utf-8", newline="\n") as handle: + json.dump(payload, handle, indent=2, ensure_ascii=False) + handle.write("\n") + say("json written to " + args.json) + + if exit_code: + say("FAIL --assert-monotone: {} new frontend symbol(s) (named above)".format(len(new))) + return exit_code + + +def write_baseline(path, symbols, notes, cache_values, build_dir): + head = subprocess.run(["git", "rev-parse", "HEAD"], stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, text=True).stdout.strip() or "" + flags = " ".join("-D{}={}".format(n, (cache_values or {}).get(n, "?")) + for n, _ in REQUIRED_FLAGS) + with open(path, "w", encoding="utf-8", newline="\n") as handle: + handle.write( + "# MobileGL link-closure ratchet baseline (CONTRACT-P6.md 12.1, ID-P7-7).\n" + "# Regenerate: python3 scripts/link_ratchet.py --build-dir \\\n" + "# --baseline scripts/data/link_ratchet_baseline.txt \\\n" + "# --write-baseline scripts/data/link_ratchet_baseline.txt\n" + "# base commit : {}\n" + "# configure : {} -DCMAKE_BUILD_TYPE={}\n" + "# build dir : {}\n" + "# count : {}\n" + "#\n" + "# One demangled symbol per line. A trailing ` # note` is an annotation the parser\n" + "# ignores; `# P13` marks a symbol that cannot fall before P13 establishes module\n" + "# target boundaries. The LIST is the baseline, not the count: a ratchet that\n" + "# stored only a number would go green on the day one symbol is cleared and another\n" + "# added (ID-P7-7).\n".format( + head, flags, (cache_values or {}).get("CMAKE_BUILD_TYPE", "?"), build_dir, + len(symbols))) + for symbol in symbols: + note = notes.get(symbol) + handle.write(symbol + (" # " + note if note else "") + "\n") + + +if __name__ == "__main__": + sys.exit(main())