[scripts, CI, docs] (P7): the 184-symbol link-closure ratchet lands as a gate

CONTRACT-P6.md 12.1 has carried "a ratchet recomputes the 184 in CI and asserts it only
ever falls" as an obligation since P6. The roadmap handed it to P6.5; the P6.5 line never
received it, and scripts/ci/ has never had anything that computes this number. ID-P7-7
gives it to P7 wave 0. This is it.

scripts/link_ratchet.py mechanises a6's hand experiment: partition the per-object output
of the single MobileGL target into SERVER / FRONTEND / SHARED, then compute
(SERVER undefined) - (SERVER defined) - (SHARED defined) intersect (FRONTEND defined),
which is the undefined-symbol list a real link of a server-only image would print. The
partition is an explicit table with a rationale per row, because it IS the module boundary
the build system does not have (SOURCE_FILES is one flat list; boundaries are P13's work).
An object matching no row is a hard failure, as is a build whose CMakeCache lacks the three
disaggregated options, or an empty role set - every way this gate could go green without
having measured anything is an explicit exit 2.

Measured on 78b7d6be: 186 symbols. The three backend buckets reproduce a6's hand-made
101 / 14 / 60 exactly, as do its per-object referrer and definer distributions
(UniformManager 90, VulkanRenderer 60, Managers 53, DirectGLES 51; TextureObject 59,
SamplerObject 30, BufferObject 18, FramebufferObject 16, PipeFill 9). Nothing from a6's
184 has been cleared; the +2 are both P6.5's:
  Client::ClientSession::StartSpawned()                 - fork-per-session
  GLState::ProgramArtifactsSchemaFingerprint()          - archive v2 caps

The baseline stores the LIST, not a count (ID-P7-7): a count-only ratchet goes green on the
day one symbol is cleared and another added. --assert-monotone fails only on symbols absent
from the baseline and names each one's defining object and every referring object; symbols
that disappeared are reported with an instruction to re-baseline in the same commit, and
never fail.

The CI step goes in build-linux-split, not build-linux: build-linux takes the three options
from MOBILEGL_CI_DISAGGREGATED, which is ON only on feat/disaggregated and
codex/fix-github-ci, so on any other branch it builds a monolith and the gate would measure
a SERVER set with no transport half. The JSON is uploaded even when the gate fires.

Two findings for the integrator, both written up in the note:
  - a6's four published bucket totals sum to 181 against its own headline of 184. These
    buckets are a strict partition and always sum to the headline; the difference is in
    a6's core bucket, which it listed selectively.
  - 5 of the 14 "P3b/P4b" symbols are MG_Pipe slot-allocator and apply-thread plumbing
    that sits under MG_Impl/Pipe/ by file location alone. P3b/P4b cannot clear them by
    migrating Espryt functionality; they need P13's module move. That bucket's floor is
    5, not 0, so the plan's 0.5-day Tier 1 estimate for "D12's 14 symbols" should be
    re-cut against 9. All 7 symbols from that object carry a `# P13` annotation.

red-once (R-16) is recorded verbatim in the note. Worth knowing: the plan's own example
(SamplerObject::GetMinFilter) does NOT turn this gate red, because it is already in the
baseline - the ratchet reddens on new symbols, so a negative control has to pick one that
is not in the 186. A reach to GetBorderColorForm() from DirectGLES/Utils.cpp does, naming
the symbol, its definer and its referrer, exit 1; reverted and green again.
This commit is contained in:
2026-09-22 14:26:04 -04:00
parent 1de8f02ebe
commit 8bb6309a1c
5 changed files with 1340 additions and 1 deletions
+42
View File
@@ -987,6 +987,48 @@ jobs:
fi
echo "libMobileGL.so defines ${remote} MG_Remote symbol(s) and the MGPipe applier (${defined} defined symbols)"
# The link-closure ratchet (CONTRACT-P6.md 12.1, ID-P7-7): the set of frontend symbols a
# server-only image would still have to link, recomputed from the per-object partition in
# scripts/link_ratchet.py and compared against the stored LIST.
#
# THIS job and not `build-linux`: build-linux passes the same three options from
# ${MOBILEGL_CI_DISAGGREGATED}, which is ON only on feat/disaggregated and
# codex/fix-github-ci, so on every other branch it builds a monolith - MG_Remote would not
# be compiled at all, the SERVER half of the partition would be empty, and the gate would
# go green having measured nothing. This job turns the three options on unconditionally,
# and the nm step immediately above has just proved they took. The script re-reads them
# from CMakeCache.txt anyway and refuses the run if they are not all ON.
#
# Only NEW symbols fail. A symbol that has DISAPPEARED is the ratchet turning and is
# printed with an instruction to re-baseline in the same commit - the baseline stores the
# list rather than a count precisely so that clearing one symbol while adding another does
# not walk through at an unchanged total (ID-P7-7).
#
# It reads the per-object output of the MobileGL target, so it must run before anything
# that cleans the build tree; the packaging step below excludes *.o from the tarball but
# does not delete them.
- name: The server image reaches no further into the frontend than the baseline
run: |
mkdir -p ci-artifacts
python3 scripts/link_ratchet.py --self-test
python3 scripts/link_ratchet.py \
--build-dir "${BUILD_DIR}" \
--baseline scripts/data/link_ratchet_baseline.txt \
--assert-monotone \
--bucket \
--json ci-artifacts/link-ratchet.json
# Uploaded even when the gate fired: a red ratchet is exactly the moment someone needs the
# per-symbol definer/referrer attribution the JSON carries, and a job that failed before
# uploading its evidence is a job nobody can act on.
- name: Upload link ratchet
if: always()
uses: actions/upload-artifact@v7
with:
name: link-ratchet
path: ci-artifacts/link-ratchet.json
if-no-files-found: warn
- name: Show ccache stats
if: always()
run: ccache --show-stats
+7 -1
View File
@@ -699,7 +699,13 @@ owed to P13. Method and full symbol list:
`Client::AdoptTierIsEmulate()`; `MG_Pipe/PipeApply.cpp` needs `Client::ClientSession::Active()` and
`AdoptTierIsEmulate()`.
**Gate**: a ratchet recomputes the 184 in CI and asserts it only ever falls.
**Gate (landed, P7 wave 0 — ID-P7-7)**: `scripts/link_ratchet.py` recomputes this set from an
explicit per-object partition on every `build-linux-split` run and fails on any symbol absent from
`scripts/data/link_ratchet_baseline.txt` — which stands at **186** on `78b7d6be`, the 184 plus
P6.5's `Client::ClientSession::StartSpawned()` and `MG_State::GLState::ProgramArtifactsSchemaFingerprint()`,
with the three backend buckets reproducing a6's 101 / 14 / 60 exactly — while a symbol that
disappears is reported rather than failed, so progress never turns the build red
([`link-ratchet.md`](../../docs/Disaggregated/notes/p7/link-ratchet.md)).
### 12.2 D11 — five unbounded peer-driven allocations, capped, with no `try`/`catch`
+230
View File
@@ -0,0 +1,230 @@
# 链接闭包棘轮(P7 wave 0,ID-P7-7)
> 脚本 `scripts/link_ratchet.py`,基线 `scripts/data/link_ratchet_baseline.txt`,
> CI 步骤在 `.github/workflows/test.yml` 的 `build-linux-split` 作业里。
> 被量的那件事见 [`a6-link-experiment.md`](../p6/a6-link-experiment.md)
> 与 [`a6-link-experiment-data.md`](../p6/a6-link-experiment-data.md)。
## 0 这道门归谁
契约 `CONTRACT-P6.md` §12.1 把 184 记成债并写下一句义务:「一道棘轮在 CI 上重算这 184 并断言它只减不增」。
路线图把它写成**「移交 P6.5」——P6.5 行从未接收**:P6.5 落地的是传输栈两轴、归档 v2、发布边界、半关闭与
`AcceptPair`/角色修复,没有一行是这道门。`scripts/ci/` 下也确实没有任何脚本算这个数
(`link_seam_purity.py` 量的是源码文本里的所有权/派发纪律,`p65_link_metrics.py` 量的是帧日志)。
**ID-P7-7 把它归 P7 wave 0**,并定了两条:基线存**符号列表**而不是计数;`--assert-monotone`
只对**新增**符号变红,符号消失只提示重基线。第二条是这道门的全部意义——
裸计数的棘轮在「清掉一个、又加一个」的那天是绿的,而那天正是它该红的那天。
## 1 怎么量
没有模块 target 可链(`SOURCE_FILES` 是一张平的源文件列表,同时喂 `MobileGL` 与 `MobileGL_s`;
建立模块边界本身是 P13 的具名工作),所以 a6 的办法是对**单个 `MobileGL` target 的逐个 object**
施加一个纸面上的分区,然后做集合运算:
```
(SERVER 未定义) − (SERVER 已定义) − (SHARED 已定义) ∩ (FRONTEND 已定义)
```
这就是一次真链接会打印的那份未定义符号表。脚本照做,**唯一的区别是那张分区表现在写在脚本里**
(`link_ratchet.py` 的 `PARTITION`),一行一条路径前缀、一个角色、一句理由,最长前缀匹配。
它就是构建系统没有的那道模块边界,只不过写下来了。
两条纪律:
- **任何 object 匹配不到 `PARTITION` 的任何一行 = 硬失败**,不是静默忽略。
树会长,新目录会出现;一个悄悄被漏掉的 object 会让这个数字悄悄停止度量它声称度量的东西。
- **`CMakeCache.txt` 里三个选项不全为 ON = 硬失败**(`--no-require-flags` 可关)。
选项为 OFF 时 `MG_Remote` 根本不编译,SERVER 集少掉整个传输半边,脚本会给出一个
很小、很绿、毫无意义的数。
`nm` 带 `-C`,所以集合运算的单位是**解构后的名字**——a6 发表的就是它,人也只读得懂它。
代价写在脚本的 docstring 里:基线树上 FRONTEND 集有 14974 个不同的 mangling 落在 14880 个不同的
解构拼写上,那 94 处重合几乎全是 C1/C2 构造与 D1/D2 析构对(按定义解构成同一串文本)。
**解构棘轮是链接的一个可读的近似,不是链接。** 要在 mangling 上较真的工具是 `symbol_report.py`。
只有 `U` 算引用;弱未定义(`w`)在无人定义时解析为 0,不构成对任何人的义务。
基线树上没有任何 `w` 触及前端定义,所以这条过滤今天不花钱。
## 2 今天的数字
基座 `78b7d6be`,`build-split`(Release,`-DMOBILEGL_BUILD_DISAGGREGATED=ON
-DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON -DMOBILEGL_PIPE_PUSH=ON`),211 个 object:
| | |
|---|---:|
| 分区 | SERVER 52 / FRONTEND 71 / SHARED 88 |
| SERVER 集未定义符号总数 | 1579 |
| 其中 SERVER + SHARED 自己满足不了的 | 593 |
| **其中由 FRONTEND 定义的** | **186** |
分桶(`--bucket`),与 a6 手做的那份逐个对照:
| 桶 | 符号数 | a6 | delta | 含义 |
|---|---:|---:|---:|---|
| `p6-core` | 11 | 6 | +5 | 没有任何后端 object 引用它:传输/applier 自己的账 |
| `p7-magma` | 101 | **101** | **+0** | 只有 `DirectVulkan` 引用:P7 的账 |
| `p3b-p4b-espryt` | 14 | **14** | **+0** | 只有 `DirectGLES` 引用:P3b/P4b 的账 |
| `both-backends` | 60 | **60** | **+0** | 两个后端都引用:两边都迁完才掉 |
| **合计** | **186** | **184** | **+2** | |
**三个后端桶逐一精确复现 a6 手做的 101 / 14 / 60。** 引用方与定义方的分布也逐一对上:
`UniformManager.cpp` 90、`VulkanRenderer.cpp` 60、`DirectGLES/Managers.cpp` 53、`DirectGLES.cpp` 51;
`TextureObject.cpp` 59、`SamplerObject.cpp` 30、`BufferObject.cpp` 18、`FramebufferObject.cpp` 16、
`PipeFill.cpp` 9。a6 的 §3.1 / §3.2 一个数都没差。
### 2.1 +2 的来源:两个符号,都是 P6.5 的
对 a6 发表的 184 逐符号取差集:**新增 2 个,消失 0 个**。
| 新增符号 | 引用方 | 是什么 |
|---|---|---|
| `MG_Remote::Client::ClientSession::StartSpawned()` | `MG_Backend/Init.cpp.o` | P6.5 的 fork-per-session 启动路径 |
| `MG_State::GLState::ProgramArtifactsSchemaFingerprint()` | `MG_Remote/CapsCodec.cpp.o` | P6.5 的归档 v2,caps 里带上 schema 指纹 |
SERVER 集本身也长了:a6 时 45 个 object,现在 52 个(`Transport` +5、`Server` +2,全是 P6.5)。
Magma 各 wave 改了 `DirectVulkan` 的内容但没有改它要的前端符号集合——这一点本身值得记一笔。
### 2.2 a6 自己的四个总数加不齐
a6 §3.2 / §5 发表的四个数 **101 + 14 + 60 + 6 = 181**,而它的头条是 **184**,差 3。
本脚本的四个桶是**严格划分**,恒等于头条。差额落在 `p6-core`:a6 §5 的表只列了它觉得有意思的
6 个,实际「没有任何后端引用」的是 9 个(184 − 175)。今天是 11 个(9 + §2.1 那 2 个)。
对得上的算法是 **9 = 6 − 1 + 4**:a6 §5 列的 6 个里有 1 个(`MGPipeApplierIsUnbarrieredApply()`)
今天已经不在 core 桶(见下);漏列的 4 个是 `BackendObject_Remote::BackendObject_Remote()`、
`ClientSession::Start(...)`、`ClientSession::Stop()`、`ClientSessionInstance()`——
四个全由 `MG_Backend/Init.cpp.o` 引用,即**后端工厂在挑 remote 后端时调 client 的会话对象**。
a6 §5 的叙述(「server 不该问 client 要答案」)本来就该把它们算进去,只是表里没写。
另外 a6 §5 说 `Server/PipeApplier.cpp` 需要 `MGPipeApplierIsUnbarrieredApply()`。
今天它落在 `p3b-p4b-espryt` 桶里:引用它的只剩 `DirectGLES/DirectGLES.cpp.o`,
`MG_Remote/Server` 已经不引用了。这是 P6.5 之后的真实变化,不是分类差异。
## 3 怎么读这张表
- **`p7-magma` 的 101 是出口门 4 的分母。** 它只会因为 Magma 真的把功能迁到 server 侧而下降。
- **`p6-core` 的 11 不是后端的债**,推给 P7/P3b/P4b 是错的。它的形状很清楚:
**9 个是 server 侧在问 client 要答案**(`MG_Backend/Init.cpp.o` 5 个、
`MG_Pipe/PipeApply.cpp` + `Wire/PipeWireCodec.cpp` 2 个、`Server/PipeApplier.cpp` 2 个),
1 个是需要搬到正确模块的解码助手(`DecodeProgramArchive`),
1 个是 P6.5 的 schema 指纹(`ProgramArtifactsSchemaFingerprint`)。
- **`both-backends` 的 60 是最硬的一块**:两个后端都要,所以任何一边单独迁完它都不动。
### 3.1 给集成者的一条:`p3b-p4b-espryt` 的 14 个里有 5 个 P3b/P4b 清不掉
基线里带 `# P13` 标注的是**由 `MG_Impl/Pipe/SlotAllocator.cpp.o` 定义的全部 7 个符号**。
规则是机械的、可复核的:要清掉其中任何一个,都得让 slot allocator 不再住在 `MG_Impl/` 下面——
那是一次模块边界搬迁,契约 §12.1 与 a6 §6.4 都把它记在 P13 头上,**任何后端迁移都够不到它**。
这 7 个的落桶是 **`p3b-p4b-espryt` 5 个 + `both-backends` 2 个**:
| 符号 | 桶 |
|---|---|
| `MG_Pipe::MGPipeApplierIsUnbarrieredApply()` | `p3b-p4b-espryt` |
| `MG_Pipe::MGPipeRefuseAllocatorFromApplyThread(char const*)` | `p3b-p4b-espryt` |
| `MG_Pipe::MGPipeRefuseFrontendKeyedRegistryFromApplyThread(char const*)` | `p3b-p4b-espryt` |
| `MG_Pipe::MGPipeSlotAllocator::Acquire(...)` | `p3b-p4b-espryt` |
| `MG_Pipe::MGPipeSlotAllocator::Free(...)` | `p3b-p4b-espryt` |
| `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId(...) const` | `both-backends` |
| `MG_Pipe::MGPipeSlots()` | `both-backends` |
也就是说:**`p3b-p4b-espryt` 这个桶的地板是 5,不是 0。**
计划正文把「D12 的 14 个符号」写成 Tier 1(0.5 天)在棘轮落地后做——
按今天的数据,其中 5 个不在 P3b/P4b 的能力范围内,那一行的估算应按 9 个重做。
标注只是给人看的,解析器忽略它,门的行为不受影响。
## 4 red-once(R-16)
在 `~/w7/p7-ratchet`(`78b7d6be`,`build-split`)里往一个**被分类为 SERVER 的** object
加一次前端触及,只重编那一个 object,跑门,然后还原。
**注意选哪个符号。** 计划书里举的例子 `SamplerObject s(0); s.GetMinFilter();` **不会让门变红**——
`GetMinFilter()` 已经在基线里(两个后端都在调它)。棘轮只对**新增**符号红,
所以负控必须挑一个今天不在那 186 个里的符号。挑的是
`SamplerObject::GetBorderColorForm() const`(在 `SamplerObject.cpp.o` 里是 `T`,外部调用,
不会被内联掉;今天没有任何 server object 引用它)。
加在 `MG_Backend/DirectGLES/Utils.cpp`(SERVER,`espryt` 族):
```cpp
unsigned P7RatchetRedOnceProbe() {
MG_State::GLState::SamplerObject s(0);
return static_cast<unsigned>(s.GetBorderColorForm());
}
```
```
$ ninja -C build-split CMakeFiles/MobileGL.dir/MobileGL/MG_Backend/DirectGLES/Utils.cpp.o
$ python3 scripts/link_ratchet.py --build-dir build-split \
--baseline scripts/data/link_ratchet_baseline.txt --assert-monotone
```
**实测红线,逐字**:
```
link-ratchet: OF WHICH defined by FRONTEND: 187 (a6 measured 184)
link-ratchet: baseline scripts/data/link_ratchet_baseline.txt: 186 symbol(s), 7 annotated
link-ratchet: 1 symbol(s) are NEW frontend reach: a server object now needs a frontend definition the baseline did not record. This is what the gate exists for - either stop reaching, or land the reach with the reason in the commit and re-baseline deliberately:
link-ratchet: NEW: MobileGL::MG_State::GLState::SamplerObject::GetBorderColorForm() const
link-ratchet: defined by: MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o
link-ratchet: referred by: MobileGL/MG_Backend/DirectGLES/Utils.cpp.o
link-ratchet: FAIL --assert-monotone: 1 new frontend symbol(s) (named above)
$ echo $?
1
```
三项计数同步动了:1579 → 1580 未定义、593 → 594 未满足、186 → 187 前端定义。
还原并重编那一个 object 之后门重新为绿(`ratchet: unchanged at 186 symbol(s)`,退出 0)。
**这条红线的价值在第三行和第四行**:它不是「数字变了」,它点名了**哪个符号**、
**谁定义的**、**谁在够它**。红的那一刻就知道该去看哪个文件。
## 5 日常用法
```bash
# 看今天的数和分桶
python3 scripts/link_ratchet.py --build-dir build-split --bucket
# 门(CI 就是这一行)
python3 scripts/link_ratchet.py --build-dir build-split \
--baseline scripts/data/link_ratchet_baseline.txt --assert-monotone \
--json link-ratchet.json
# 清掉符号之后,在同一个提交里重基线(标注会被带过去)
python3 scripts/link_ratchet.py --build-dir build-split \
--baseline scripts/data/link_ratchet_baseline.txt \
--write-baseline scripts/data/link_ratchet_baseline.txt
# 不需要构建
python3 scripts/link_ratchet.py --self-test
```
**符号消失不是失败**,脚本会把它们逐个列出来并要求在同一个提交里重基线。
**符号新增是失败**,脚本会对每一个打印它的定义方 object 与全部引用方 object,
所以红出来的那一刻就知道是谁在够谁。
## 6 没有验证的(读之前先读这一条)
**基线是在参考构建 `~/w7/pipe/build-split` 上生成的,不是在 CI 的机器上。**
两边都是 clang + **libstdc++**(CI 装了 `libc++-20-dev` 但 `CMakeLists.txt` 与工作流都没有传
`-stdlib=libc++`,所以 Linux 上落的是 libstdc++;基线里的 `std::__cxx11::basic_string` 拼写
就是这么来的),配置三选项与 `CMAKE_BUILD_TYPE=Release` 也一致——**但 clang 版本不同**
(参考构建是 Arch 的 clang,CI 是 `clang++-20`)。
内联决策随版本变:某个方法在一边被内联、在另一边留成外部调用,就会让那个符号在 reach 集里
出现或消失。**所以这道门在 CI 上的第一次运行有可能是红的,且那种红不是代码变坏。**
处理办法是明确的,不要绕过门:看第一次运行上传的 `link-ratchet.json` artifact,
逐个核对 `new_since_baseline` / `cleared_since_baseline`,确认它们全都是内联差异
(同一个类、同一族方法、没有新的引用方 object),然后用一个单独的提交重基线并在提交信息里
写明是工具链差异。**如果里面混着一个新的引用方 object,那就不是内联差异。**
其余:
- Android NDK 构建没试过,而且**会**不一样:NDK 用 libc++,解构拼写变成 `std::__1::…`,
整份基线都会算作新增。这道门今天只对 Linux 主机构建有意义。
- `MobileGL_s`(同源的静态库 target)没量;`--target` 可以指过去,但没跑过。
- 解构名重合(§1 那 94 处 C1/C2)今天不影响结果,但脚本没有为此加断言——
要加的话应当是「reach 集里的每个解构名在 FRONTEND 侧只对应一个 mangling」。
- `check_flags` 的「选项为 OFF」分支是手工用一个假的 `CMakeCache.txt` 验的,
不在 `--self-test` 覆盖范围内。
+200
View File
@@ -0,0 +1,200 @@
# MobileGL link-closure ratchet baseline (CONTRACT-P6.md 12.1, ID-P7-7).
# Regenerate: python3 scripts/link_ratchet.py --build-dir <dir> \
# --baseline scripts/data/link_ratchet_baseline.txt \
# --write-baseline scripts/data/link_ratchet_baseline.txt
# base commit : 78b7d6beacbebb89a5cf8aa412dd3739be5f1247
# configure : -DMOBILEGL_BUILD_DISAGGREGATED=ON -DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON -DMOBILEGL_PIPE_PUSH=ON -DCMAKE_BUILD_TYPE=Release
# build dir : build-split
# count : 186
#
# One demangled symbol per line. A trailing ` # note` is an annotation the parser
# ignores; `# P13` marks a symbol that cannot fall before P13 establishes module
# target boundaries. The LIST is the baseline, not the count: a ratchet that
# stored only a number would go green on the day one symbol is cleared and another
# added (ID-P7-7).
MobileGL::MG_Impl::GLImpl::CopyTextureImageToClientOrPBO_State(std::shared_ptr<MobileGL::MG_State::GLState::ITextureObject> const&, MobileGL::TextureUploadTarget, int, unsigned int, unsigned int, int, void*, char const*)
MobileGL::MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo
MobileGL::MG_Pipe::MGPipeApplierIsUnbarrieredApply() # P13
MobileGL::MG_Pipe::MGPipeNoteAggregate(MobileGL::MG_Pipe::MGPipeAggregate)
MobileGL::MG_Pipe::MGPipeRefuseAllocatorFromApplyThread(char const*) # P13
MobileGL::MG_Pipe::MGPipeRefuseFrontendKeyedRegistryFromApplyThread(char const*) # P13
MobileGL::MG_Pipe::MGPipeSlotAllocator::Acquire(MobileGL::MG_Pipe::MGPipeKind, unsigned long) # P13
MobileGL::MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId(MobileGL::MG_Pipe::MGPipeKind, unsigned long) const # P13
MobileGL::MG_Pipe::MGPipeSlotAllocator::Free(MobileGL::MG_Pipe::MGPipeKind, MobileGL::MG_Pipe::MGPipeHandle) # P13
MobileGL::MG_Pipe::MGPipeSlots() # P13
MobileGL::MG_Pipe::PipeInputs::GetBufferBindingPointCount(MobileGL::BufferTarget) const
MobileGL::MG_Pipe::PipeInputs::GetProgramObject(unsigned int)
MobileGL::MG_Pipe::PipeInputs::GetTextureObject(unsigned int)
MobileGL::MG_Pipe::PipeInputs::HasOpenTransformFeedbackSpan(unsigned long) const
MobileGL::MG_Pipe::PipeInputs::InvalidateCompileEnv()
MobileGL::MG_Pipe::PipeInputs::IsLive() const
MobileGL::MG_Pipe::PipeInputs::RecordError(MobileGL::ErrorCode, std::unique_ptr<MobileGL::ErrorInfo, std::default_delete<MobileGL::ErrorInfo> >)
MobileGL::MG_Pipe::PipeInputs::ValidateProgramName(unsigned int) const
MobileGL::MG_Remote::Client::AdoptTierIsEmulate()
MobileGL::MG_Remote::Client::BackendObject_Remote::BackendObject_Remote()
MobileGL::MG_Remote::Client::ClientSession::Active()
MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()
MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadLeftApplier()
MobileGL::MG_Remote::Client::ClientSession::Start(MobileGL::MG_Config::TransportMode, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
MobileGL::MG_Remote::Client::ClientSession::StartSpawned()
MobileGL::MG_Remote::Client::ClientSession::Stop()
MobileGL::MG_Remote::Client::ClientSessionInstance()
MobileGL::MG_Remote::Client::EmitObjectDeathRecord(MobileGL::MG_Pipe::MGPipeKind, unsigned long)
MobileGL::MG_State::GLState::BufferObject::DownloadSubData(void*, unsigned long, unsigned long) const
MobileGL::MG_State::GLState::BufferObject::EnsureGpuResidentStorage()
MobileGL::MG_State::GLState::BufferObject::GetBackendResource() const
MobileGL::MG_State::GLState::BufferObject::GetChangeSerial() const
MobileGL::MG_State::GLState::BufferObject::GetExternalIndex() const
MobileGL::MG_State::GLState::BufferObject::GetSize() const
MobileGL::MG_State::GLState::BufferObject::GetUsage() const
MobileGL::MG_State::GLState::BufferObject::HasDefinedContent() const
MobileGL::MG_State::GLState::BufferObject::IsBackendPersistentMapped() const
MobileGL::MG_State::GLState::BufferObject::IsMapped() const
MobileGL::MG_State::GLState::BufferObject::MappedData() const
MobileGL::MG_State::GLState::BufferObject::MarkGpuWritten()
MobileGL::MG_State::GLState::BufferObject::SetBackendResource(std::shared_ptr<MobileGL::MG_State::GLState::BackendBufferResource>)
MobileGL::MG_State::GLState::BufferObject::SyncGpuWrites()
MobileGL::MG_State::GLState::BufferObject::SyncPersistentMappedRange()
MobileGL::MG_State::GLState::BufferObject::WritebackFromBackend(MobileGL::DataPtr, unsigned long)
MobileGL::MG_State::GLState::ClassifyVertexAttribType(unsigned int)
MobileGL::MG_State::GLState::DecodeProgramArchive(unsigned char const*, unsigned long, MobileGL::MG_State::GLState::ProgramArchive&)
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetRenderbuffer() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetSize() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTexture() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureLayer() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureLevel() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::GetTextureUploadTarget() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsComplete() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsEmpty() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsLayered() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsRenderbuffer() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsTexture() const
MobileGL::MG_State::GLState::FramebufferAttachmentObject::IsValid() const
MobileGL::MG_State::GLState::FramebufferObject::GetAllAttachmentObjects() const
MobileGL::MG_State::GLState::FramebufferObject::GetAttachment(MobileGL::FramebufferAttachmentType) const
MobileGL::MG_State::GLState::FramebufferObject::GetDrawBuffers() const
MobileGL::MG_State::GLState::FramebufferObject::GetExternalIndex() const
MobileGL::MG_State::GLState::GetBufferBackendOps()
MobileGL::MG_State::GLState::MGPipeTextureLegacyArmScope::MGPipeTextureLegacyArmScope()
MobileGL::MG_State::GLState::MGPipeTextureLegacyArmScope::~MGPipeTextureLegacyArmScope()
MobileGL::MG_State::GLState::ProgramArtifactsSchemaFingerprint()
MobileGL::MG_State::GLState::ProgramObject::AllocateLifetimeId()
MobileGL::MG_State::GLState::ProgramObject::AttachShader(std::shared_ptr<MobileGL::MG_State::GLState::ShaderObject> const&)
MobileGL::MG_State::GLState::ProgramObject::JoinPendingLink() const
MobileGL::MG_State::GLState::ProgramObject::JoinPendingSpirv() const
MobileGL::MG_State::GLState::ProgramObject::Link(bool)
MobileGL::MG_State::GLState::ProgramObject::~ProgramObject()
MobileGL::MG_State::GLState::RenderbufferObject::GetExternalIndex() const
MobileGL::MG_State::GLState::RenderbufferObject::GetHeight() const
MobileGL::MG_State::GLState::RenderbufferObject::GetInternalFormat() const
MobileGL::MG_State::GLState::RenderbufferObject::GetSamples() const
MobileGL::MG_State::GLState::RenderbufferObject::GetWidth() const
MobileGL::MG_State::GLState::RenderbufferObject::IsAllocated() const
MobileGL::MG_State::GLState::SamplerObject::GetAllSamplerParameters() const
MobileGL::MG_State::GLState::SamplerObject::GetBorderColor() const
MobileGL::MG_State::GLState::SamplerObject::GetBorderColorI() const
MobileGL::MG_State::GLState::SamplerObject::GetBorderColorUI() const
MobileGL::MG_State::GLState::SamplerObject::GetCompareMode() const
MobileGL::MG_State::GLState::SamplerObject::GetExternalIndex() const
MobileGL::MG_State::GLState::SamplerObject::GetLifetimeId() const
MobileGL::MG_State::GLState::SamplerObject::GetLodBias() const
MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const
MobileGL::MG_State::GLState::SamplerObject::GetMaxAnisotropy() const
MobileGL::MG_State::GLState::SamplerObject::GetMaxLod() const
MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const
MobileGL::MG_State::GLState::SamplerObject::GetMinLod() const
MobileGL::MG_State::GLState::SamplerObject::GetMipmapMode() const
MobileGL::MG_State::GLState::SamplerObject::GetSamplerCompareFunc() const
MobileGL::MG_State::GLState::SamplerObject::GetVersion() const
MobileGL::MG_State::GLState::SamplerObject::GetWrapR() const
MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const
MobileGL::MG_State::GLState::SamplerObject::GetWrapT() const
MobileGL::MG_State::GLState::SamplerObject::SamplerObject(unsigned int)
MobileGL::MG_State::GLState::SamplerObject::SetCompareMode(MobileGL::SamplerCompareMode)
MobileGL::MG_State::GLState::SamplerObject::SetLodRange(float, float)
MobileGL::MG_State::GLState::SamplerObject::SetMagFilter(MobileGL::SamplerFilterMode)
MobileGL::MG_State::GLState::SamplerObject::SetMinFilter(MobileGL::SamplerFilterMode)
MobileGL::MG_State::GLState::SamplerObject::SetMipmapMode(MobileGL::SamplerMipmapMode)
MobileGL::MG_State::GLState::SamplerObject::SetSamplerCompareFunc(MobileGL::SamplerCompareFunc)
MobileGL::MG_State::GLState::SamplerObject::SetWrapR(MobileGL::SamplerWrapMode)
MobileGL::MG_State::GLState::SamplerObject::SetWrapS(MobileGL::SamplerWrapMode)
MobileGL::MG_State::GLState::SamplerObject::SetWrapT(MobileGL::SamplerWrapMode)
MobileGL::MG_State::GLState::SamplerObject::~SamplerObject()
MobileGL::MG_State::GLState::SamplesAsIncompleteTexture(MobileGL::MG_State::GLState::ITextureObject const*, MobileGL::MG_State::GLState::SamplerObject const*)
MobileGL::MG_State::GLState::SetBufferBackendOps(MobileGL::MG_State::GLState::BufferBackendOps const*)
MobileGL::MG_State::GLState::ShaderObject::Compile()
MobileGL::MG_State::GLState::ShaderObject::DropCompileNode() const
MobileGL::MG_State::GLState::ShaderObject::JoinPendingCompile() const
MobileGL::MG_State::GLState::ShaderObject::ReleaseCompileNode()
MobileGL::MG_State::GLState::ShaderObject::SetShaderSource(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)
MobileGL::MG_State::GLState::TextureObject1D::TextureObject1D(unsigned int)
MobileGL::MG_State::GLState::TextureObject2D::TextureObject2D(unsigned int)
MobileGL::MG_State::GLState::TextureObject2DCube::TextureObject2DCube(unsigned int)
MobileGL::MG_State::GLState::TextureObject3D::TextureObject3D(unsigned int)
MobileGL::MG_State::GLState::TextureObjectBase::GetAllSwizzleParams() const
MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColor() const
MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorForm() const
MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorI() const
MobileGL::MG_State::GLState::TextureObjectBase::GetBorderColorUI() const
MobileGL::MG_State::GLState::TextureObjectBase::GetContentVersion() const
MobileGL::MG_State::GLState::TextureObjectBase::GetExternalIndex() const
MobileGL::MG_State::GLState::TextureObjectBase::GetFormat() const
MobileGL::MG_State::GLState::TextureObjectBase::GetImmutableLevels() const
MobileGL::MG_State::GLState::TextureObjectBase::GetLevelRange() const
MobileGL::MG_State::GLState::TextureObjectBase::GetLifetimeId() const
MobileGL::MG_State::GLState::TextureObjectBase::GetSamplerObject() const
MobileGL::MG_State::GLState::TextureObjectBase::GetSamples() const
MobileGL::MG_State::GLState::TextureObjectBase::GetShapeVersion() const
MobileGL::MG_State::GLState::TextureObjectBase::GetSwizzleParam(MobileGL::TextureSwizzleParam) const
MobileGL::MG_State::GLState::TextureObjectBase::GetTarget() const
MobileGL::MG_State::GLState::TextureObjectBase::GetTextureParamsVersion() const
MobileGL::MG_State::GLState::TextureObjectBase::GetViewStorageOwner() const
MobileGL::MG_State::GLState::TextureObjectBase::HasFixedSampleLocations() const
MobileGL::MG_State::GLState::TextureObjectBase::IsImmutable() const
MobileGL::MG_State::GLState::TextureObjectBase::IsMipmapCompleteForFilterCached(bool) const
MobileGL::MG_State::GLState::TextureObjectBase::PipePublishParams()
MobileGL::MG_State::GLState::TextureObjectBase::SetBaseLevel(unsigned int)
MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColor(MobileGL::Vec4<float> const&)
MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColorI(MobileGL::Vec4<int> const&)
MobileGL::MG_State::GLState::TextureObjectBase::SetBorderColorUI(MobileGL::Vec4<unsigned int> const&)
MobileGL::MG_State::GLState::TextureObjectBase::SetFixedSampleLocations(bool)
MobileGL::MG_State::GLState::TextureObjectBase::SetImmutableLevels(unsigned int)
MobileGL::MG_State::GLState::TextureObjectBase::SetInternalFormat(MobileGL::TextureInternalFormat)
MobileGL::MG_State::GLState::TextureObjectBase::SetMaxLevel(unsigned int)
MobileGL::MG_State::GLState::TextureObjectBase::SetSamples(int)
MobileGL::MG_State::GLState::TextureObjectBase::SetSwizzleParam(MobileGL::TextureSwizzleParam, MobileGL::TextureSwizzleParam)
MobileGL::MG_State::GLState::TextureObjectBase::SetSwizzleParamRGBA(MobileGL::Vec4<MobileGL::TextureSwizzleParam> const&)
MobileGL::MG_State::GLState::TextureObjectBase::TextureObjectBase(MobileGL::TextureTarget, unsigned int)
MobileGL::MG_State::GLState::TextureObjectBase::~TextureObjectBase()
MobileGL::MG_State::GLState::TextureObjectBuffer::GetBufferBindingSlot(MobileGL::TextureUploadTarget)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::AllocateStorage(MobileGL::TextureUploadTarget, unsigned int, MobileGL::MG_State::GLState::MipmapInput)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetBaseSize() const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapByteSize(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapCompressedByteSize(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapCompressedFormat(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapLevelCount() const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapRequestedCompressedFormat(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetMipmapTexelSize(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetStorageDirtyRects(MobileGL::TextureUploadTarget, unsigned int, MobileGL::MG_State::GLState::MipmapDirtyRegion*, unsigned long) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::GetStorageDirtyRegion(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::IsComplete() const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::IsStorageDirty(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MapMipmapCompressedImage(MobileGL::TextureUploadTarget, unsigned int) const
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MapMipmapData(MobileGL::TextureUploadTarget, unsigned int)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MarkStorageDirty(MobileGL::TextureUploadTarget, unsigned int, bool)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::MarkStorageDirtyRegion(MobileGL::TextureUploadTarget, unsigned int, MobileGL::Vec3<int>, MobileGL::Vec3<int>)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::SetMipmapCompressedImage(MobileGL::TextureUploadTarget, unsigned int, unsigned int, void const*, unsigned long)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::SetMipmapRequestedCompressedFormat(MobileGL::TextureUploadTarget, unsigned int, unsigned int)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::TruncateMipmapLevels(MobileGL::TextureUploadTarget, unsigned int)
MobileGL::MG_State::GLState::TextureObjectWithOneMipmap::UpdateMipmapSubData(MobileGL::TextureUploadTarget, unsigned int, MobileGL::DataPtr)
MobileGL::MG_State::GLState::TextureUnit::GetAllBindingSlots()
MobileGL::MG_State::GLState::TextureUnit::GetBindingSlot(MobileGL::TextureTarget)
MobileGL::MG_State::GLState::TextureUnit::GetSamplerObject() const
MobileGL::MG_State::GLState::VertexArrayObject::GetAllAttributeVersions() const
MobileGL::MG_State::GLState::VertexArrayObject::GetAllAttributes() const
MobileGL::MG_State::GLState::VertexArrayObject::GetAttribute(unsigned int) const
MobileGL::MG_State::GLState::VertexArrayObject::GetIndexBufferBindingSlot()
MobileGL::MG_State::GLState::VertexArrayObject::GetIndexBufferBindingSlot() const
typeinfo for MobileGL::MG_State::GLState::TextureObjectBase
typeinfo for MobileGL::MG_State::GLState::TextureObjectWithOneMipmap
vtable for MobileGL::MG_State::GLState::TextureObjectWithOneMipmap
+861
View File
@@ -0,0 +1,861 @@
#!/usr/bin/env python3
# MobileGL - scripts/link_ratchet.py
# Copyright (c) 2025-2026 MobileGL-Dev
# Licensed under the GNU Lesser General Public License v3.0:
# https://www.gnu.org/licenses/gpl-3.0.txt
# https://www.gnu.org/licenses/lgpl-3.0.txt
# SPDX-License-Identifier: LGPL-3.0-only
# End of Source File Header
"""The link-closure ratchet: which frontend symbols would a server-only image still need?
CONTRACT-P6.md 12.1 records the debt as 184 symbols and says "a ratchet recomputes the 184
in CI and asserts it only ever falls". This is that ratchet. The measurement is a6's
(docs/Disaggregated/notes/p6/a6-link-experiment.md), mechanised:
(SERVER undefined) - (SERVER defined) - (SHARED defined) intersect (FRONTEND defined)
i.e. the undefined-symbol list a real link of a server-only image would print today. There is
no module target to link - `SOURCE_FILES` is one flat list feeding `MobileGL` and `MobileGL_s`
(CMakeLists.txt) and establishing module boundaries is P13's named work - so the partition is
applied to the per-object output of the single `MobileGL` target instead. That is exactly why
the partition lives in PARTITION below as an explicit table with a rationale per row rather
than as a regex someone has to reverse-engineer: it IS the module boundary, written down.
python3 scripts/link_ratchet.py --build-dir build-split
python3 scripts/link_ratchet.py --build-dir build-split --bucket
python3 scripts/link_ratchet.py --build-dir build-split \
--baseline scripts/data/link_ratchet_baseline.txt --assert-monotone \
--json link-ratchet.json
python3 scripts/link_ratchet.py --self-test
Exit codes: 0 green, 1 the ratchet itself went red (new frontend reach), 2 the run could not
be trusted to mean anything (an object no PARTITION row claims, or a build configured without
the three disaggregated options). The two are separate on purpose - a 2 is never "the code got
worse", it is "fix the measurement first".
WHAT THE GATE IS FOR (ID-P7-7). The baseline stores the symbol LIST, not a count. A count-only
ratchet passes on the day one symbol is cleared and another added, which is precisely the
regression this gate exists to catch: new backend code reaching into a frontend object. So
--assert-monotone fails on symbols present now and ABSENT from the baseline, and merely reports
symbols that disappeared, with an instruction to re-baseline in the same commit. Progress is
never a red build; new reach always is.
GUARD RAILS. The build must be a disaggregated one
(-DMOBILEGL_BUILD_DISAGGREGATED=ON -DMOBILEGL_BUILD_DISAGGREGATED_INPROC=ON
-DMOBILEGL_PIPE_PUSH=ON): with the option OFF, MG_Remote is not compiled at all, the SERVER set
is missing its transport half and the script would report a cheerfully small number that means
nothing. The run reads CMakeCache.txt and refuses such a build by name unless it is given
--no-require-flags. Two more refusals for the same reason - every way this gate can go green
without having measured anything is an explicit exit 2:
* an object that matches no PARTITION row (the tree moved and the partition did not), and
* an empty SERVER, FRONTEND or SHARED set (the target was never built in that directory,
which otherwise classifies perfectly into three empty sets and reports a beautiful zero).
SYMBOL IDENTITY, and the one place this is looser than a linker. nm is run with -C, so the unit
of the set arithmetic is the DEMANGLED name - what a6 published and what a reviewer can read
and grep for. That is deliberately not the linker's unit. Measured on the base tree, the
FRONTEND set defines 14974 distinct manglings under 14880 distinct demangled spellings: the 94
collisions are overwhelmingly the C1/C2 (complete/base) constructor and D1/D2 destructor pairs,
which demangle to the same text by construction, plus `[abi:cxx11]` tags and `.cold` parts. The
consequence to know about: if a server object ever referenced the C1 form of a constructor that
the frontend defined only as C2, this script would call it satisfied and a real link would not.
Both forms are emitted together in practice, so no such case exists today - but a demangled
ratchet is a readable approximation of a link, not a link. `scripts/symbol_report.py` is the
tool that works in manglings when the mangling is the point.
Only `U` counts as a reference. A weak undefined (`w`) resolves to zero when nothing defines
it, so it obliges nobody; on the base tree no `w` reaches a frontend definition, so this
filter costs nothing today and keeps the number honest the day it does.
THE BASELINE IS TOOLCHAIN-SENSITIVE, and this is the one way this gate goes red without any
code having got worse. Whether a small method is inlined or left as an out-of-line call is a
compiler-version decision, and it moves symbols into and out of the reach set. The stored
baseline was generated with a different clang than CI's `clang++-20` (same libstdc++, same
Release, same three options). If the first CI run is red, read the uploaded JSON before
touching anything: if every entry in `new_since_baseline` is a sibling method of a class
already in the list and brings NO new referring object, it is an inlining difference and the
answer is one deliberate re-baseline commit that says so. A new REFERRING OBJECT is never an
inlining difference - that is the real thing this gate is looking for. An Android NDK build is
a different question again: the NDK is libc++, every `std::` spelling changes, and the whole
baseline would read as new. This gate is for Linux host builds.
"""
import argparse
import json
import os
import re
import subprocess
import sys
PREFIX = "link-ratchet: "
SERVER = "SERVER"
FRONTEND = "FRONTEND"
SHARED = "SHARED"
# ---------------------------------------------------------------------------
# The partition. This table IS the module boundary that the build system does not have.
#
# Rows are (object path prefix, role, why). Paths are relative to
# <build-dir>/CMakeFiles/<target>.dir/ and a row ending in `.o` is an exact object, not a
# prefix. Matching is LONGEST PREFIX WINS, so `MG_Remote/Client/` (FRONTEND) overrides nothing
# here only because no shorter `MG_Remote/` row exists - if one is ever added, the longer row
# still wins and the intent stays readable.
#
# Provenance: a6-link-experiment-data.md lines 6-8 name the three sets. Every row below that is
# not in a6's three lines carries "(post-a6)" in its rationale and is called out in the report,
# because a partition that silently absorbed a new directory would silently change the number.
# ---------------------------------------------------------------------------
PARTITION = (
# --- SERVER: what a server-only image would be built from ---------------------------
("MobileGL/MG_Backend/", SERVER,
"backend translation - GL/Vulkan command submission is the server's entire job"),
("MobileGL/MG_Pipe/", SERVER,
"the applier and its routing decode into the backend inside the server process"),
("MobileGL/MG_Remote/Transport/", SERVER,
"the transport stack; the server end of every link lives here"),
("MobileGL/MG_Remote/Protocol/", SERVER,
"wire schema helpers used by the server's decode path"),
("MobileGL/MG_Remote/Wire/", SERVER,
"the wire codec the server decodes with"),
("MobileGL/MG_Remote/Server/", SERVER,
"the server loop, session, applier and spawn"),
("MobileGL/MG_Remote/CapsCodec.cpp.o", SERVER,
"the server encodes the caps blob the client mirrors; a6 names this object by hand"),
# --- FRONTEND: what a server-only image is supposed to be able to drop ---------------
("MobileGL/MG_Impl/", FRONTEND,
"the GL/EGL/GLX entry points and the client-side pipe fill - guest-process code"),
("MobileGL/MG_State/", FRONTEND,
"the GL object model; the server is meant to see records on the wire, not these objects"),
("MobileGL/MG_Remote/Client/", FRONTEND,
"the client half of the session - by definition not in a server image"),
# --- SHARED: linked into both images, so it satisfies references without being reach --
("MobileGL/MG_Util/", SHARED,
"role-neutral utilities: converters, transpiler, metrics, math"),
("MobileGL/MG_Remote/FatalFunnel.cpp.o", SHARED,
"(post-a6, landed in 5968863f) every session death funnels through it from BOTH roles, "
"so it is role-neutral infrastructure. a6 predates the file and names no rule for it"),
("MobileGL/Init.cpp.o", SHARED,
"library init, runs in whichever process loaded the library"),
("MobileGL/GlobalObjects.cpp.o", SHARED,
"the process-global singletons both roles reach"),
("MobileGL/ConfigLoader.cpp.o", SHARED,
"environment/config parsing, read by both roles"),
)
# ---------------------------------------------------------------------------
# Referrer families, for --bucket. a6 attributed the debt to P7 (Magma), P3b/P4b (Espryt),
# both, and "P6's own" - which it derived by hand from who REFERS to each symbol. Same table
# shape, same longest-prefix rule; every SERVER object falls in exactly one family.
# ---------------------------------------------------------------------------
FAMILIES = (
("MobileGL/MG_Backend/DirectVulkan/", "magma",
"Magma / DirectVulkan - P7's full migration"),
("MobileGL/MG_Backend/DirectGLES/", "espryt",
"Espryt / DirectGLES - P3b/P4b's deepening"),
("", "core",
"server-side code that is neither backend: MG_Remote transport/wire/server, the MG_Pipe "
"applier, and the backend-neutral MG_Backend glue. a6 calls this bucket 'P6's own'"),
)
BUCKETS = (
("p6-core", "referred to by NO backend object: the transport/applier's own account "
"(a6: 'P6 self' = 6)"),
("p7-magma", "referred to only by DirectVulkan: P7's account (a6: 101)"),
("p3b-p4b-espryt", "referred to only by DirectGLES: P3b/P4b's account (a6: 14)"),
("both-backends", "referred to by both backends: falls only when both migrate (a6: 60)"),
)
A6_BUCKET_TOTALS = {"p6-core": 6, "p7-magma": 101, "p3b-p4b-espryt": 14, "both-backends": 60}
A6_TOTAL = 184
# `nm -C -o` prints "<path>:<addr-or-blanks> <type> <demangled name>". The demangled name
# contains spaces, `<`, `,` and `::`, so the type letter is located by position after the
# known object path rather than by splitting the whole line.
NM_BODY_RE = re.compile(r"^\s*([0-9a-fA-F]*)\s+([A-Za-z?])\s(.+)$")
# A baseline line may carry a trailing ` # note` annotation (ID-P7-7 asks for `# P13` on the
# symbols that cannot fall before P13). No demangled C++ name contains " #", so splitting on
# the first occurrence is unambiguous.
BASELINE_ANNOTATION = " #"
REQUIRED_FLAGS = (
("MOBILEGL_BUILD_DISAGGREGATED", "ON"),
("MOBILEGL_BUILD_DISAGGREGATED_INPROC", "ON"),
("MOBILEGL_PIPE_PUSH", "ON"),
)
def say(message):
print(PREFIX + message)
def classify(obj_path, table=PARTITION):
"""Longest-prefix match against the explicit table. None means: no row claims this object."""
best = None
for prefix, role, _why in table:
if obj_path == prefix or (prefix.endswith("/") and obj_path.startswith(prefix)):
if best is None or len(prefix) > len(best[0]):
best = (prefix, role)
return best[1] if best else None
def family_of(obj_path, table=FAMILIES):
"""Which referrer family a SERVER object belongs to. The empty prefix row is the default."""
best = ("", None)
for prefix, name, _why in table:
if prefix == "" or obj_path.startswith(prefix):
if name is not None and len(prefix) >= len(best[0]):
best = (prefix, name)
return best[1]
def find_objects(build_dir, target):
"""Every .o under <build-dir>/CMakeFiles/<target>.dir, keyed by path relative to that dir."""
root = os.path.join(build_dir, "CMakeFiles", target + ".dir")
if not os.path.isdir(root):
raise SystemExit(PREFIX + "no such object directory: {} (is --build-dir a configured "
"CMake build, and is --target right? the disaggregated build "
"has MobileGL.dir, MobileGL_s.dir and MobileGLServer.dir)"
.format(root))
objects = []
for dirpath, _dirnames, filenames in os.walk(root):
for name in filenames:
if name.endswith(".o"):
full = os.path.join(dirpath, name)
objects.append(os.path.relpath(full, root).replace(os.sep, "/"))
return root, sorted(objects)
def check_flags(build_dir):
"""Refuse a build whose options make the measurement meaningless. Returns the flag lines."""
cache = os.path.join(build_dir, "CMakeCache.txt")
if not os.path.isfile(cache):
return None, ["no CMakeCache.txt in {} - cannot confirm the build is disaggregated"
.format(build_dir)]
values = {}
with open(cache, "r", encoding="utf-8", errors="replace") as handle:
for line in handle:
match = re.match(r"^([A-Za-z0-9_]+):[A-Z]+=(.*)$", line.strip())
if match:
values[match.group(1)] = match.group(2)
problems = []
for name, wanted in REQUIRED_FLAGS:
got = values.get(name)
if got != wanted:
problems.append(
"{}={} (need {}). With the option off, MG_Remote is not compiled at all, the "
"SERVER set has no transport half, and every number below would be a small, "
"green, meaningless one.".format(name, got if got is not None else "<unset>",
wanted))
return values, problems
def run_nm(nm_tool, mode, paths, cwd, chunk=150):
"""`nm -C -o --<mode>-only` over paths, returned as raw text. Chunked for argv limits."""
out = []
for i in range(0, len(paths), chunk):
batch = paths[i:i + chunk]
completed = subprocess.run([nm_tool, "-C", "-o", "--" + mode + "-only"] + batch,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, cwd=cwd)
# nm exits non-zero on a file it cannot read; an object it merely has no symbols for is
# not an error. Anything on stderr is worth surfacing - a silently skipped object is a
# silently shrunk SERVER set.
if completed.returncode != 0 and completed.stderr.strip():
raise SystemExit(PREFIX + "nm failed: " + completed.stderr.strip())
out.append(completed.stdout)
return "".join(out)
def parse_nm(text, known_objects):
"""`nm -C -o` transcript -> {object: {name: type-letter}}. Keys come from known_objects."""
table = {obj: {} for obj in known_objects}
ordered = sorted(known_objects, key=len, reverse=True)
for line in text.splitlines():
if not line.strip():
continue
obj = None
for candidate in ordered:
if line.startswith(candidate + ":"):
obj = candidate
break
if obj is None:
continue
match = NM_BODY_RE.match(line[len(obj) + 1:])
if not match:
continue
table[obj][match.group(3)] = match.group(2)
return table
def compute(roles, undefined, defined):
"""The set arithmetic, on plain dicts so the self-test can drive it without a build.
roles : {object: SERVER|FRONTEND|SHARED}
undefined : {object: {name: type}} - only SERVER objects need to be present
defined : {object: {name: type}} - every object
Returns the sorted symbol list plus the per-symbol referrer/definer attribution.
"""
server = [o for o, r in roles.items() if r == SERVER]
frontend = [o for o, r in roles.items() if r == FRONTEND]
shared = [o for o, r in roles.items() if r == SHARED]
referrers = {}
for obj in server:
for name, kind in undefined.get(obj, {}).items():
# `U` only. A weak undefined (`w`) resolves to zero when nothing defines it, so it
# is not a reference that obliges anybody.
if kind != "U":
continue
referrers.setdefault(name, set()).add(obj)
server_defined = set()
for obj in server:
server_defined.update(defined.get(obj, {}))
shared_defined = set()
for obj in shared:
shared_defined.update(defined.get(obj, {}))
definers = {}
for obj in frontend:
for name in defined.get(obj, {}):
definers.setdefault(name, set()).add(obj)
server_undefined = set(referrers)
unsatisfied = server_undefined - server_defined - shared_defined
reach = sorted(unsatisfied & set(definers))
return {
"symbols": reach,
"referrers": {name: sorted(referrers[name]) for name in reach},
"definers": {name: sorted(definers[name]) for name in reach},
"counts": {
"server_objects": len(server),
"frontend_objects": len(frontend),
"shared_objects": len(shared),
"server_undefined": len(server_undefined),
"unsatisfied_by_server_or_shared": len(unsatisfied),
"defined_by_frontend": len(reach),
},
}
def bucketise(result):
"""Group the reach by referrer family, reproducing a6's four hand-made buckets."""
buckets = {name: [] for name, _why in BUCKETS}
also_core = []
for name in result["symbols"]:
families = {family_of(obj) for obj in result["referrers"][name]}
backends = families & {"magma", "espryt"}
if not backends:
buckets["p6-core"].append(name)
else:
if backends == {"magma"}:
buckets["p7-magma"].append(name)
elif backends == {"espryt"}:
buckets["p3b-p4b-espryt"].append(name)
else:
buckets["both-backends"].append(name)
if "core" in families:
also_core.append(name)
return buckets, also_core
def read_baseline(path):
"""Baseline file -> (symbols, {symbol: annotation}). `#` at line start is a comment."""
symbols = []
notes = {}
with open(path, "r", encoding="utf-8") as handle:
for line in handle:
line = line.rstrip("\n").rstrip("\r")
if not line.strip() or line.lstrip().startswith("#"):
continue
if BASELINE_ANNOTATION in line:
symbol, annotation = line.split(BASELINE_ANNOTATION, 1)
symbol = symbol.rstrip()
notes[symbol] = annotation.strip()
else:
symbol = line.rstrip()
symbols.append(symbol)
return symbols, notes
def monotone(current, baseline):
"""(new, gone). `new` is the red: reach that was not in the baseline."""
current_set = set(current)
baseline_set = set(baseline)
return sorted(current_set - baseline_set), sorted(baseline_set - current_set)
# ---------------------------------------------------------------------------
# Self-test: the classification and the set arithmetic on synthetic nm output, no build.
# ---------------------------------------------------------------------------
CANNED_OBJECTS = {
# SERVER
"MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o": SERVER,
"MobileGL/MG_Backend/DirectGLES/Managers.cpp.o": SERVER,
"MobileGL/MG_Backend/BackendObject.cpp.o": SERVER,
"MobileGL/MG_Remote/Server/PipeApplier.cpp.o": SERVER,
"MobileGL/MG_Pipe/PipeApply.cpp.o": SERVER,
"MobileGL/MG_Remote/CapsCodec.cpp.o": SERVER,
# FRONTEND
"MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o": FRONTEND,
"MobileGL/MG_Impl/Pipe/SlotAllocator.cpp.o": FRONTEND,
"MobileGL/MG_Remote/Client/ClientSession.cpp.o": FRONTEND,
# SHARED
"MobileGL/MG_Util/Math/Vec.cpp.o": SHARED,
"MobileGL/MG_Remote/FatalFunnel.cpp.o": SHARED,
"MobileGL/ConfigLoader.cpp.o": SHARED,
}
CANNED_UNDEFINED = {
"MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o": {
"MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const": "U",
"MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "U",
"MobileGL::Shared::Helper()": "U", # satisfied by SHARED
"MobileGL::MG_Backend::BackendObject::Tick()": "U", # satisfied by SERVER
"MobileGL::Nowhere::Absent()": "U", # satisfied by nobody: not our business
"MobileGL::MG_State::GLState::WeakOnly()": "w", # weak undef: not a reference
},
"MobileGL/MG_Backend/DirectGLES/Managers.cpp.o": {
"MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "U",
"MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const": "U",
},
"MobileGL/MG_Remote/Server/PipeApplier.cpp.o": {
"MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()": "U",
},
"MobileGL/MG_Pipe/PipeApply.cpp.o": {
"MobileGL::MG_Pipe::MGPipeSlots()": "U",
},
"MobileGL/MG_Backend/BackendObject.cpp.o": {},
"MobileGL/MG_Remote/CapsCodec.cpp.o": {},
}
CANNED_DEFINED = {
"MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o": {
"MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const": "T",
"MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const": "T",
"MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const": "T",
"MobileGL::MG_State::GLState::WeakOnly()": "W",
},
"MobileGL/MG_Impl/Pipe/SlotAllocator.cpp.o": {
"MobileGL::MG_Pipe::MGPipeSlots()": "T",
},
"MobileGL/MG_Remote/Client/ClientSession.cpp.o": {
"MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()": "T",
},
"MobileGL/MG_Util/Math/Vec.cpp.o": {
"MobileGL::Shared::Helper()": "T",
},
"MobileGL/MG_Backend/BackendObject.cpp.o": {
"MobileGL::MG_Backend::BackendObject::Tick()": "T",
},
"MobileGL/MG_Remote/FatalFunnel.cpp.o": {},
"MobileGL/ConfigLoader.cpp.o": {},
}
CANNED_NM_TRANSCRIPT = """\
MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o: U MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const
MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o: w MobileGL::MG_State::GLState::WeakOnly()
MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o:0000000000000010 T MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const
MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o:0000000000000020 W MobileGL::MG_State::GLState::WeakOnly()
this line belongs to no known object and must be ignored
"""
def self_test():
problems = []
# --- the partition table ---------------------------------------------------------
expected_roles = [
("MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.cpp.o", SERVER),
("MobileGL/MG_Pipe/PipeRoute.cpp.o", SERVER),
("MobileGL/MG_Remote/Transport/ShmLink.cpp.o", SERVER),
("MobileGL/MG_Remote/Server/ServerLoop.cpp.o", SERVER),
("MobileGL/MG_Remote/CapsCodec.cpp.o", SERVER),
("MobileGL/MG_Remote/Client/ClientSession.cpp.o", FRONTEND),
("MobileGL/MG_State/GLState/TextureState/TextureObject.cpp.o", FRONTEND),
("MobileGL/MG_Impl/GLImpl/Texture/GL_Texture.cpp.o", FRONTEND),
("MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/Pass.cpp.o", SHARED),
("MobileGL/MG_Remote/FatalFunnel.cpp.o", SHARED),
("MobileGL/ConfigLoader.cpp.o", SHARED),
("MobileGL/MG_Nowhere/Invented.cpp.o", None),
]
for path, want in expected_roles:
got = classify(path)
if got != want:
problems.append("classify({}) = {}, expected {}".format(path, got, want))
# Longest prefix wins even when a shorter row would also match.
longer = PARTITION + (("MobileGL/", SHARED, "hypothetical catch-all"),)
if classify("MobileGL/MG_Remote/Server/ServerLoop.cpp.o", longer) != SERVER:
problems.append("longest-prefix match lost to a shorter catch-all row")
for path, want in (("MobileGL/MG_Backend/DirectVulkan/DirectVulkan.cpp.o", "magma"),
("MobileGL/MG_Backend/DirectGLES/Utils.cpp.o", "espryt"),
("MobileGL/MG_Backend/BackendObject.cpp.o", "core"),
("MobileGL/MG_Remote/Server/PipeApplier.cpp.o", "core"),
("MobileGL/MG_Pipe/PipeApply.cpp.o", "core")):
got = family_of(path)
if got != want:
problems.append("family_of({}) = {}, expected {}".format(path, got, want))
# --- the nm parser ---------------------------------------------------------------
parsed = parse_nm(CANNED_NM_TRANSCRIPT, [
"MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o",
"MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o"])
uni = parsed["MobileGL/MG_Backend/DirectVulkan/Renderer/UniformManager.cpp.o"]
sam = parsed["MobileGL/MG_State/GLState/SamplerState/SamplerObject.cpp.o"]
if uni.get("MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const") != "U":
problems.append("nm parse: undefined line with a spaced demangled name: {}".format(uni))
if uni.get("MobileGL::MG_State::GLState::WeakOnly()") != "w":
problems.append("nm parse: weak-undefined type letter lost: {}".format(uni))
if sam.get("MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const") != "T":
problems.append("nm parse: defined line with an address: {}".format(sam))
if len(uni) != 2 or len(sam) != 2:
problems.append("nm parse: unowned line was not ignored: {} / {}".format(uni, sam))
# --- the set arithmetic ----------------------------------------------------------
result = compute(CANNED_OBJECTS, CANNED_UNDEFINED, CANNED_DEFINED)
want_symbols = [
"MobileGL::MG_Pipe::MGPipeSlots()",
"MobileGL::MG_Remote::Client::ClientSession::NoteApplyThreadEnteredApplier()",
"MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const",
"MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const",
"MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const",
]
if result["symbols"] != want_symbols:
problems.append("reach set: {}".format(result["symbols"]))
# Each exclusion for its own reason, so a regression names which subtraction broke.
if "MobileGL::Shared::Helper()" in result["symbols"]:
problems.append("a SHARED-defined symbol survived the subtraction")
if "MobileGL::MG_Backend::BackendObject::Tick()" in result["symbols"]:
problems.append("a SERVER-defined symbol survived the subtraction")
if "MobileGL::Nowhere::Absent()" in result["symbols"]:
problems.append("a symbol nobody defines was reported as frontend reach")
if "MobileGL::MG_State::GLState::WeakOnly()" in result["symbols"]:
problems.append("a weak undefined (`w`) was counted as a reference")
# 8 distinct `U` names across the four referring server objects (the `w` is not one of
# them, and GetMinFilter is referred to twice but counted once); minus Tick (SERVER-defined)
# and Helper (SHARED-defined) leaves 6; of those, Absent is defined by nobody, leaving 5.
if result["counts"]["server_undefined"] != 8:
problems.append("server undefined count: {}".format(result["counts"]))
if result["counts"]["unsatisfied_by_server_or_shared"] != 6:
problems.append("unsatisfied count: {}".format(result["counts"]))
if result["counts"]["defined_by_frontend"] != 5:
problems.append("frontend-defined count: {}".format(result["counts"]))
if (result["counts"]["server_objects"], result["counts"]["frontend_objects"],
result["counts"]["shared_objects"]) != (6, 3, 3):
problems.append("partition counts: {}".format(result["counts"]))
# --- the buckets ------------------------------------------------------------------
buckets, also_core = bucketise(result)
want_buckets = {
"p6-core": ["MobileGL::MG_Pipe::MGPipeSlots()",
"MobileGL::MG_Remote::Client::ClientSession::"
"NoteApplyThreadEnteredApplier()"],
"p7-magma": ["MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const"],
"p3b-p4b-espryt": ["MobileGL::MG_State::GLState::SamplerObject::GetWrapS() const"],
"both-backends": ["MobileGL::MG_State::GLState::SamplerObject::GetMinFilter() const"],
}
if buckets != want_buckets:
problems.append("buckets: {}".format(buckets))
if sum(len(v) for v in buckets.values()) != len(result["symbols"]):
problems.append("buckets do not partition the reach set")
if also_core:
problems.append("unexpected backend+core overlap: {}".format(also_core))
# --- the ratchet ------------------------------------------------------------------
base = ["a", "b", "c"]
new, gone = monotone(["b", "c", "d"], base)
if new != ["d"] or gone != ["a"]:
problems.append("monotone(): new={} gone={}".format(new, gone))
if monotone(["a", "b"], base) != ([], ["c"]):
problems.append("a shrunk set must be green with a re-baseline note")
if monotone(base, base) != ([], []):
problems.append("an identical set must be silent")
# ID-P7-7's whole point: one cleared, one added, same count -> still red.
new, gone = monotone(["a", "b", "z"], base)
if new != ["z"] or gone != ["c"]:
problems.append("clear-one-add-one at equal count did not name the new symbol")
# --- the baseline parser ----------------------------------------------------------
import tempfile
with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False,
encoding="utf-8", newline="\n") as handle:
handle.write("# a header comment\n"
"# base: deadbeef\n"
"\n"
"MobileGL::MG_Pipe::MGPipeSlots() # P13\n"
"MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const\n"
"MobileGL::Vec4<int>::Vec4(int, int, int, int) # P13 allocator-shaped\n")
tmp = handle.name
try:
symbols, notes = read_baseline(tmp)
if symbols != ["MobileGL::MG_Pipe::MGPipeSlots()",
"MobileGL::MG_State::GLState::SamplerObject::GetMagFilter() const",
"MobileGL::Vec4<int>::Vec4(int, int, int, int)"]:
problems.append("baseline parse dropped or mangled a symbol: {}".format(symbols))
if notes.get("MobileGL::MG_Pipe::MGPipeSlots()") != "P13":
problems.append("baseline annotation not read: {}".format(notes))
if len(notes) != 2:
problems.append("baseline annotations: {}".format(notes))
finally:
os.unlink(tmp)
for problem in problems:
say("self-test: " + problem)
say("self-test: " + ("OK (partition, families, nm parser, set arithmetic, buckets, "
"ratchet, baseline parser)" if not problems else "FAILED"))
return 0 if not problems else 1
def render_buckets(result, buckets, also_core, verbose):
lines = []
lines.append("")
lines.append("| bucket | symbols | a6 | delta | meaning |")
lines.append("|---|---:|---:|---:|---|")
for name, why in BUCKETS:
count = len(buckets[name])
a6 = A6_BUCKET_TOTALS[name]
lines.append("| `{}` | {} | {} | {:+d} | {} |".format(name, count, a6, count - a6, why))
total = sum(len(v) for v in buckets.values())
lines.append("| **total** | **{}** | **{}** | **{:+d}** | |".format(
total, A6_TOTAL, total - A6_TOTAL))
lines.append("")
lines.append("a6's four published totals sum to {} against its own headline of {}; the "
"buckets above are a strict partition, so they always sum to the headline."
.format(sum(A6_BUCKET_TOTALS.values()), A6_TOTAL))
if also_core:
lines.append("")
lines.append("{} backend-bucket symbol(s) are ALSO referred to by a core object, so "
"they do not fall when the backend that owns them migrates:".format(
len(also_core)))
for name in also_core:
lines.append(" - " + name)
if verbose:
for name, _why in BUCKETS:
lines.append("")
lines.append("### {} ({})".format(name, len(buckets[name])))
for symbol in buckets[name]:
lines.append(" {}".format(symbol))
lines.append(" defined by: " + ", ".join(result["definers"][symbol]))
lines.append(" referred by: " + ", ".join(result["referrers"][symbol]))
return lines
def main():
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--build-dir", help="a configured, BUILT disaggregated CMake build dir")
parser.add_argument("--target", default="MobileGL",
help="the CMake target whose objects carry the partition "
"(default: MobileGL; MobileGL_s is the same sources as a static "
"lib, MobileGLServer is the P0 spike stub)")
parser.add_argument("--nm", default="nm")
parser.add_argument("--baseline", help="the stored symbol list")
parser.add_argument("--assert-monotone", action="store_true",
help="exit non-zero for every symbol present now and absent from the "
"baseline. Symbols that vanished are reported and are never a "
"failure - progress does not turn the build red.")
parser.add_argument("--bucket", action="store_true",
help="group the reach by referring object family and print the "
"P7 / P3b-P4b / both / core totals against a6's")
parser.add_argument("--verbose-buckets", action="store_true",
help="with --bucket, list every symbol with its definer and referrers")
parser.add_argument("--list", action="store_true",
help="print the symbol list itself (implied when no baseline is given)")
parser.add_argument("--json", help="write the machine-readable result here")
parser.add_argument("--write-baseline", metavar="FILE",
help="write the current list out in baseline format, annotations from "
"--baseline carried over")
parser.add_argument("--no-require-flags", dest="require_flags", action="store_false",
help="do not refuse a build whose CMakeCache lacks the three "
"disaggregated options (for inspecting an odd build by hand)")
parser.add_argument("--self-test", action="store_true")
args = parser.parse_args()
if args.self_test:
return self_test()
if not args.build_dir:
parser.error("--build-dir is required (or use --self-test)")
if args.assert_monotone and not args.baseline:
parser.error("--assert-monotone needs --baseline")
if args.baseline and not os.path.isfile(args.baseline):
# Named, not a traceback: in CI this is a path typo or a missing checkout, and the two
# look identical in a stack trace.
say("FAIL no baseline file at {} (the ratchet has nothing to compare against; "
"scripts/data/link_ratchet_baseline.txt is the one CI uses)".format(args.baseline))
return 2
cache_values, flag_problems = check_flags(args.build_dir)
if flag_problems:
for problem in flag_problems:
say(("FAIL " if args.require_flags else "warning: ") + problem)
if args.require_flags:
return 2
root, objects = find_objects(args.build_dir, args.target)
roles = {}
unclassified = []
for obj in objects:
role = classify(obj)
if role is None:
unclassified.append(obj)
else:
roles[obj] = role
if unclassified:
say("FAIL {} object(s) match no PARTITION row. The tree moved and this measurement "
"no longer measures what it claims to; add a row (with its reason) rather than a "
"catch-all:".format(len(unclassified)))
for obj in unclassified:
say(" unclassified: " + obj)
return 2
# An unbuilt (or half-built) directory classifies cleanly into three empty sets and would
# otherwise report a beautiful zero. The ratchet's happiest possible output must not also be
# its output for "there was nothing to look at".
empty = [name for name, members in ((SERVER, [o for o, r in roles.items() if r == SERVER]),
(FRONTEND, [o for o, r in roles.items() if r == FRONTEND]),
(SHARED, [o for o, r in roles.items() if r == SHARED]))
if not members]
if empty:
say("FAIL {} set(s) are empty ({}) in {} - the target was not built, or was built "
"somewhere else. A ratchet that reports 0 because it found no objects is worse "
"than no ratchet.".format(len(empty), ", ".join(empty), root))
return 2
server = sorted(o for o, r in roles.items() if r == SERVER)
undefined = parse_nm(run_nm(args.nm, "undefined", server, root), server)
defined = parse_nm(run_nm(args.nm, "defined", objects, root), objects)
result = compute(roles, undefined, defined)
counts = result["counts"]
say("build-dir {} (target {}, {} objects)".format(args.build_dir, args.target, len(objects)))
if cache_values:
say("configure: " + ", ".join("{}={}".format(n, cache_values.get(n, "<unset>"))
for n, _ in REQUIRED_FLAGS)
+ ", CMAKE_BUILD_TYPE=" + cache_values.get("CMAKE_BUILD_TYPE", "<unset>"))
say("partition: SERVER {} / FRONTEND {} / SHARED {} objects".format(
counts["server_objects"], counts["frontend_objects"], counts["shared_objects"]))
say("SERVER undefined symbols: {}".format(counts["server_undefined"]))
say(" not satisfied by SERVER or SHARED: {}".format(
counts["unsatisfied_by_server_or_shared"]))
say(" OF WHICH defined by FRONTEND: {} (a6 measured {})".format(
counts["defined_by_frontend"], A6_TOTAL))
buckets, also_core = bucketise(result)
if args.bucket:
print("\n".join(render_buckets(result, buckets, also_core, args.verbose_buckets)))
if args.list or not args.baseline:
print("")
for symbol in result["symbols"]:
print(symbol)
exit_code = 0
new, gone = [], []
baseline_symbols, baseline_notes = [], {}
if args.baseline:
baseline_symbols, baseline_notes = read_baseline(args.baseline)
new, gone = monotone(result["symbols"], baseline_symbols)
say("baseline {}: {} symbol(s), {} annotated".format(
args.baseline, len(baseline_symbols), len(baseline_notes)))
if gone:
say("{} symbol(s) in the baseline are GONE. That is the ratchet turning - not a "
"failure. Re-baseline in the same commit that cleared them "
"(--write-baseline):".format(len(gone)))
for symbol in gone:
say(" cleared: " + symbol
+ ((" [" + baseline_notes[symbol] + "]") if symbol in baseline_notes
else ""))
if new:
say("{} symbol(s) are NEW frontend reach: a server object now needs a frontend "
"definition the baseline did not record. This is what the gate exists for - "
"either stop reaching, or land the reach with the reason in the commit and "
"re-baseline deliberately:".format(len(new)))
for symbol in new:
say(" NEW: " + symbol)
say(" defined by: " + ", ".join(result["definers"][symbol]))
say(" referred by: " + ", ".join(result["referrers"][symbol]))
if args.assert_monotone and new:
exit_code = 1
if not new and not gone:
say("ratchet: unchanged at {} symbol(s)".format(len(result["symbols"])))
if args.write_baseline:
if not args.baseline:
say("warning: --write-baseline without --baseline: the ` # P13` annotations of the "
"old file cannot be carried over and will be lost. Pass --baseline too.")
write_baseline(args.write_baseline, result["symbols"], baseline_notes, cache_values,
args.build_dir)
say("baseline written to {} ({} symbol(s), {} annotation(s) carried over)".format(
args.write_baseline, len(result["symbols"]),
sum(1 for s in result["symbols"] if s in baseline_notes)))
if args.json:
payload = {
"build_dir": args.build_dir,
"target": args.target,
"configure": {name: (cache_values or {}).get(name) for name, _ in REQUIRED_FLAGS},
"counts": counts,
# The partition itself travels with the artifact: a reviewer reading the JSON six
# months from now should not have to guess which objects were on which side.
"roles": roles,
"a6": {"total": A6_TOTAL, "buckets": A6_BUCKET_TOTALS},
"symbols": result["symbols"],
"buckets": {name: buckets[name] for name, _ in BUCKETS},
"bucket_counts": {name: len(buckets[name]) for name, _ in BUCKETS},
"backend_symbols_also_referred_by_core": also_core,
"definers": result["definers"],
"referrers": result["referrers"],
"baseline": args.baseline,
"baseline_count": len(baseline_symbols) if args.baseline else None,
"new_since_baseline": new,
"cleared_since_baseline": gone,
"assert_monotone": args.assert_monotone,
"exit_code": exit_code,
}
with open(args.json, "w", encoding="utf-8", newline="\n") as handle:
json.dump(payload, handle, indent=2, ensure_ascii=False)
handle.write("\n")
say("json written to " + args.json)
if exit_code:
say("FAIL --assert-monotone: {} new frontend symbol(s) (named above)".format(len(new)))
return exit_code
def write_baseline(path, symbols, notes, cache_values, build_dir):
head = subprocess.run(["git", "rev-parse", "HEAD"], stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True).stdout.strip() or "<unknown>"
flags = " ".join("-D{}={}".format(n, (cache_values or {}).get(n, "?"))
for n, _ in REQUIRED_FLAGS)
with open(path, "w", encoding="utf-8", newline="\n") as handle:
handle.write(
"# MobileGL link-closure ratchet baseline (CONTRACT-P6.md 12.1, ID-P7-7).\n"
"# Regenerate: python3 scripts/link_ratchet.py --build-dir <dir> \\\n"
"# --baseline scripts/data/link_ratchet_baseline.txt \\\n"
"# --write-baseline scripts/data/link_ratchet_baseline.txt\n"
"# base commit : {}\n"
"# configure : {} -DCMAKE_BUILD_TYPE={}\n"
"# build dir : {}\n"
"# count : {}\n"
"#\n"
"# One demangled symbol per line. A trailing ` # note` is an annotation the parser\n"
"# ignores; `# P13` marks a symbol that cannot fall before P13 establishes module\n"
"# target boundaries. The LIST is the baseline, not the count: a ratchet that\n"
"# stored only a number would go green on the day one symbol is cleared and another\n"
"# added (ID-P7-7).\n".format(
head, flags, (cache_values or {}).get("CMAKE_BUILD_TYPE", "?"), build_dir,
len(symbols)))
for symbol in symbols:
note = notes.get(symbol)
handle.write(symbol + (" # " + note if note else "") + "\n")
if __name__ == "__main__":
sys.exit(main())