[Build] (MG_Remote, Protocol): pin the flatbuffers submodule, add the control-plane schema and commit its generated header

- 3rdparty/flatbuffers submodule pinned to the latest release tag v25.12.19 (7e163021). The runtime is header-only, so only 3rdparty/flatbuffers/include is ever used and no library is linked; CMake never calls add_subdirectory on it and flatc is not in the build graph (plan B section 8.1, inheriting the earlier plan's section 7.1).
- MobileGL/MG_Remote/Protocol/protocol.fbs carries the CONTROL PLANE only: SegmentRef, Hello, Welcome, CapsSnapshot, DefaultFramebufferInfo, SurfaceOp, SurfaceReply, ResyncRequest, ResyncDone, AuxRequest, Fatal, LogLine, union CtrlMsg and the CtrlEnvelope root with a file_identifier. Hot-path records are FlatBuffers structs generated from MG_Pipe/PipeCalls.def in a later package and are deliberately absent here, so record numbering never churns.
- Two deviations from the earlier plan's section 7.1 sketch, both deliberate: (a) ProgramReflection is not a union member - plan B ships program artifacts inside the create_shader_state CSO blob (section 8.2), and union tags are wire values that may only ever be appended, so reserving a tag for a message that may never exist is worse than appending one later; (b) maxComputeWorkGroupCount/Size are vectors, not [int:3] - fixed-size arrays are legal only in FlatBuffers structs, never in tables.
- scripts/gen_protocol.py resolves flatc as MOBILEGL_FLATC_EXECUTABLE, otherwise builds the pinned flatc ONCE into <repo>/../flatc-build (override with MOBILEGL_FLATC_BUILD_DIR), outside the project build graph. A flatc found on PATH is deliberately refused and a version mismatch against the pinned runtime is a hard error: the generated header static_asserts FLATBUFFERS_VERSION, so a stray flatc either fails to compile or churns the committed file on every machine. The earlier branch did the opposite - Protocol/CMakeLists.txt:22-38 add_subdirectory'd the FlatBuffers tree with FLATBUFFERS_BUILD_FLATC=ON whenever MOBILEGL_FLATC_EXECUTABLE was unset, which is exactly the NDK trap it claimed to avoid (cross-compile an arm64 flatc, then run it on the host).
- protocol_generated.h is committed with the project source header prepended by the generator, so regeneration is byte-identical: verified by running gen_protocol.py twice and by perturbing the file and regenerating it back.
- Reuse from Feat/CS-Delta-IPC: MobileGL/Protocol/mg_protocol_base.h, kept as the shared C vocabulary (result codes, byte spans, shm region, id typedefs) and keeping the structSize-first versioning discipline that section 14.2 calls the answer to risk B-R10. Dropped from it: MobileGLObjectKind / MobileGLObjectScope / MobileGLObjectHandle - plan B never puts GL object identity on the wire (the frontend allocates {slot, generation} handles in MG_Pipe, section 4.2.1), so a second identity vocabulary would be a drift surface with no reader. Added MOBILEGL_ERR_BUFFER_TOO_SMALL as an append-only code for the receive contract.
This commit is contained in:
2026-09-05 20:16:49 -04:00
parent bd2b4158e0
commit a1e22c26ab
6 changed files with 2312 additions and 0 deletions
+3
View File
@@ -34,3 +34,6 @@
[submodule "include/ska"]
path = include/ska
url = https://github.com/MobileGL-Dev/flat_hash_map.git
[submodule "3rdparty/flatbuffers"]
path = 3rdparty/flatbuffers
url = https://github.com/google/flatbuffers.git
Vendored Submodule
+1
Submodule 3rdparty/flatbuffers added at 7e163021e5
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,120 @@
// MobileGL - MobileGL/MG_Remote/Protocol/mg_protocol_base.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// Shared vocabulary of the MG_Remote wire contracts (transport, framing, ring,
// shm). Inherited from the earlier `Feat/CS-Delta-IPC` branch
// (MobileGL/Protocol/mg_protocol_base.h) and cut down to what plan B's
// transport actually needs: result codes, byte spans, a shm region reference
// and the id typedefs.
//
// Deliberately NOT inherited: MobileGLObjectKind / MobileGLObjectScope /
// MobileGLObjectHandle. Plan B does not put GL object identity on the wire at
// all - the frontend allocates {slot, generation} handles in MG_Pipe
// (PLAN-B.md section 4.2.1) and those are the only identity the backend ever
// sees, so a second object-identity vocabulary here would be a drift surface
// with no reader.
//
// This header must stay:
// - pure C (compilable from C and C++, no MG C++ types, no exceptions/RTTI),
// - dependency-free (only <stdbool.h>/<stddef.h>/<stdint.h>),
// - append-only within an ABI major (see versioning rules below).
//
// Versioning rules (contract-wide):
// - Every versioned struct starts with uint32_t structSize.
// - Appending fields at the tail is a MINOR bump; receivers must ignore
// bytes beyond the structSize they know.
// - Changing/removing/reordering existing fields is a MAJOR bump.
// - A major mismatch is a hard, structured failure, never an exception.
// (Plan B keeps the structSize-first discipline as the answer to risk B-R10,
// PLAN-B.md section 14.2.)
#ifndef MOBILEGL_REMOTE_PROTOCOL_BASE_H
#define MOBILEGL_REMOTE_PROTOCOL_BASE_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
// ---------------------------------------------------------------------------
// ABI versions
// ---------------------------------------------------------------------------
#define MOBILEGL_PROTOCOL_ABI_MAJOR 1
#define MOBILEGL_PROTOCOL_ABI_MINOR 0
#define MOBILEGL_ABI_VERSION(major, minor) (((uint32_t)(major) << 16) | (uint32_t)(minor))
#define MOBILEGL_ABI_MAJOR_OF(version) ((uint32_t)(version) >> 16)
#define MOBILEGL_ABI_MINOR_OF(version) ((uint32_t)(version) & 0xFFFFu)
// ---------------------------------------------------------------------------
// Ids
// ---------------------------------------------------------------------------
typedef uint64_t MobileGLSessionId; // one client GL context flow
typedef uint64_t MobileGLRequestSeq; // matches a request to its reply
typedef uint32_t MobileGLSegmentId; // shm segment id within a connection
// ---------------------------------------------------------------------------
// Spans / regions
// ---------------------------------------------------------------------------
// Borrowed, read-only byte span. The pointee is owned by the producing side
// and is only valid for the duration documented at the consuming call site.
typedef struct MobileGLByteSpan {
const void* data;
uint64_t size;
} MobileGLByteSpan;
typedef struct MobileGLMutableByteSpan {
void* data;
uint64_t size;
} MobileGLMutableByteSpan;
// A byte range inside an already-established shm segment. Segments are
// announced out of band (the SegmentRef table on the control channel, with the
// fd itself passed by SCM_RIGHTS) and stay stable for their declared lifetime;
// offsets are segment-relative.
typedef struct MobileGLShmRegion {
MobileGLSegmentId segmentId;
uint32_t reserved;
uint64_t offset;
uint64_t size;
} MobileGLShmRegion;
// ---------------------------------------------------------------------------
// Result codes (structured errors across every contract boundary)
// ---------------------------------------------------------------------------
typedef enum MobileGLResult {
MOBILEGL_OK = 0,
MOBILEGL_ERR_NOT_INITIALIZED = 1,
MOBILEGL_ERR_INVALID_ARGUMENT = 2,
MOBILEGL_ERR_UNSUPPORTED = 3,
MOBILEGL_ERR_OUT_OF_MEMORY = 4,
MOBILEGL_ERR_PROTOCOL_MISMATCH = 5, // ABI/wire major mismatch, bad framing
MOBILEGL_ERR_TRANSPORT_CLOSED = 6, // peer gone / EOF
MOBILEGL_ERR_TIMEOUT = 7, // nothing arrived within the deadline
MOBILEGL_ERR_SHM_EXHAUSTED = 8,
MOBILEGL_ERR_SESSION_UNKNOWN = 9,
MOBILEGL_ERR_HANDLE_UNKNOWN = 10,
// The caller's buffer is smaller than the pending message. The message is
// NOT consumed and the required size is reported back; see
// ITransport::ReceiveFrame.
MOBILEGL_ERR_BUFFER_TOO_SMALL = 11,
MOBILEGL_ERR_FORCE_U32 = 0x7FFFFFFF
} MobileGLResult;
#ifdef __cplusplus
} // extern "C"
#endif
#endif // MOBILEGL_REMOTE_PROTOCOL_BASE_H
+235
View File
@@ -0,0 +1,235 @@
// MobileGL - MobileGL/MG_Remote/Protocol/protocol.fbs
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// MobileGL disaggregated wire protocol - CONTROL PLANE ONLY.
//
// Plan B (docs plan "MGPipe") section 8.1 inherits the transport design of the
// earlier plan verbatim, and its section 7.1 splits the schema in two:
//
// - rare / variable-length / must-evolve messages -> FlatBuffers *tables*,
// carried as complete framed messages over the control channel. That is
// everything in this file.
// - the hot path -> FlatBuffers *structs* (fixed layout, no vtable, no
// offset indirection) written straight into the SEG_CMD ring. Those
// records are generated from MG_Pipe/PipeCalls.def and are deliberately
// NOT in this schema yet: the call catalogue is a separate P0 deliverable
// and record numbering must never churn.
//
// Regeneration: scripts/gen_protocol.py (flatc is NOT part of the default
// build graph). generated/protocol_generated.h is committed and CI's
// flatc-check regenerates it and runs `git diff --exit-code`.
namespace MobileGL.Wire;
// ---------------------------------------------------------------------------
// Segments
// ---------------------------------------------------------------------------
// Segment layout is inherited unchanged (earlier plan section 6.1):
// SEG_CMD 8MiB / SEG_STAGE 32MiB+ / SEG_REPLY 8MiB / SEG_EVENT 256KiB /
// SEG_SHADOW[n] / SEG_ADOPT[n].
enum SegmentKind : ubyte {
None = 0,
Cmd = 1, // client-owned command ring (RingControl + records)
Stage = 2, // client-owned bulk staging
Reply = 3, // server-owned reply pool
Event = 4, // server-owned event ring
Shadow = 5, // client-owned per-object shadow (P4.5+)
Adopt = 6, // server-owned adopted store, client RW (>= 16MiB)
}
// The fd itself never travels in a message: POSIX passes it with SCM_RIGHTS on
// the aux socket (ITransport::ShareFd), Windows resolves `name`.
table SegmentRef {
id: uint;
kind: SegmentKind;
sizeBytes: ulong;
name: string;
}
// ---------------------------------------------------------------------------
// Handshake
// ---------------------------------------------------------------------------
table Hello {
abiMajor: uint;
abiMinor: uint;
buildFingerprint: string;
backendType: uint;
pid: uint;
configBlob: [ubyte];
}
table Welcome {
abiMajor: uint;
abiMinor: uint;
serverPid: uint;
cmdRing: SegmentRef;
stageRing: SegmentRef;
replyPool: SegmentRef;
eventRing: SegmentRef;
}
// ---------------------------------------------------------------------------
// Capabilities
// ---------------------------------------------------------------------------
// Replaces the 40 `pActiveBackendObject->` reads plus the 89 caps read sites
// (plan B appendix A, `get_caps`). The three blobs are byte-for-byte images of
// the corresponding POD structs; they are versioned by structSize-first
// discipline, not by this schema.
table CapsSnapshot {
dynamicParameters: [ubyte];
rendererInfo: [ubyte];
formatCaps: [ubyte];
extensions: [string];
apiVersion: string;
maxComputeWorkGroupCount: [int]; // 3 entries
maxComputeWorkGroupSize: [int]; // 3 entries
tableSlotMask: ulong; // which GLFunctionsTable slots the peer registered
prefersCpuXfbPrimitiveAccounting: bool;
}
table DefaultFramebufferInfo {
width: int;
height: int;
colorFormat: uint;
depthFormat: uint;
stencilFormat: uint;
}
// ---------------------------------------------------------------------------
// Surface / EGL lifecycle
// ---------------------------------------------------------------------------
enum SurfaceOpKind : ubyte {
None = 0,
InitializeDisplay = 1,
CreateWindowSurface = 2,
CreatePbufferSurface = 3,
ResizeWindowSurface = 4,
ReleaseSurface = 5,
MakeCurrent = 6,
ReleaseCurrent = 7,
}
enum WindowKind : ubyte {
None = 0,
AndroidNativeWindow = 1,
X11 = 2,
Win32Hwnd = 3,
Surfaceless = 4,
Pbuffer = 5,
}
table SurfaceOp {
seq: ulong;
kind: SurfaceOpKind;
display: ulong;
surface: ulong;
windowKind: WindowKind;
nativeToken: ulong; // X11 XID / HWND; Android transfers the window out of band
width: int;
height: int;
swapInterval: int;
}
table SurfaceReply {
seq: ulong;
ok: bool;
eglMajor: int;
eglMinor: int;
defaultFb: DefaultFramebufferInfo;
}
// ---------------------------------------------------------------------------
// Resync / aux / diagnostics
// ---------------------------------------------------------------------------
// Sent by the client after it observes a serverEpoch bump (context lost or
// server restart): every cached ring offset and every server-side object is
// gone and the whole pushed state has to be replayed.
table ResyncRequest {
serverEpoch: uint;
}
table ResyncDone {}
enum AuxRequestKind : ubyte {
None = 0,
FenceClientWait = 1,
QueryResult = 2,
ScalarGet = 3,
}
// Requests issued from a thread that is not the ring producer (foreign-thread
// sync / query polling), so they cannot take the SPSC ring.
table AuxRequest {
seq: ulong;
kind: AuxRequestKind;
payload: [ubyte];
}
enum FatalCode : uint {
None = 0,
ProtocolCorruption = 1, // record bounds / self-describing length violated
RingOverrun = 2,
SegmentMismatch = 3,
DeviceLost = 4,
ServerCrashed = 5,
AbiMismatch = 6,
}
table Fatal {
code: FatalCode;
message: string;
}
// Severity-graded per plan B section 8.2: <= Warn is lossy, >= Error is
// lossless and rate limited.
enum LogLevel : ubyte {
Debug = 0,
Info = 1,
Warn = 2,
Error = 3,
Fatal = 4,
}
table LogLine {
level: LogLevel;
text: string;
}
// ---------------------------------------------------------------------------
// Envelope
// ---------------------------------------------------------------------------
// Union tags are wire values: only ever APPEND to this list.
// ProgramReflection from the earlier plan's section 7.1 is intentionally
// absent - plan B ships program artifacts inside the create_shader_state CSO
// blob, so if a control-plane reflection message is ever needed it appends
// here rather than reserving a tag today.
union CtrlMsg {
Hello,
Welcome,
CapsSnapshot,
SurfaceOp,
SurfaceReply,
ResyncRequest,
ResyncDone,
AuxRequest,
Fatal,
LogLine,
}
table CtrlEnvelope {
msg: CtrlMsg;
}
root_type CtrlEnvelope;
file_identifier "MGLC";
+175
View File
@@ -0,0 +1,175 @@
#!/usr/bin/env python3
"""Regenerate MobileGL/MG_Remote/Protocol/generated/protocol_generated.h from protocol.fbs.
flatc is a developer/CI tool ONLY: it is never part of the default build graph
(the earlier branch's Protocol/CMakeLists.txt:22-38 did add_subdirectory the
FlatBuffers tree and turned FLATBUFFERS_BUILD_FLATC ON when
MOBILEGL_FLATC_EXECUTABLE was unset, which is exactly the NDK trap it claimed
to avoid: cross-compiling an arm64 flatc and then trying to run it on the
host). The FlatBuffers runtime is header-only, so a build only needs
3rdparty/flatbuffers/include on the include path.
flatc resolution order:
1. --flatc / MOBILEGL_FLATC_EXECUTABLE
2. a flatc built once from the pinned 3rdparty/flatbuffers submodule into a
directory OUTSIDE the repository (default: <repo>/../flatc-build,
override with MOBILEGL_FLATC_BUILD_DIR)
A flatc found on PATH is deliberately NOT used: the generated header carries a
FLATBUFFERS_VERSION static_assert against the runtime headers, so a stray flatc
of another version produces a header that either fails to compile or churns the
committed file on every machine.
"""
from __future__ import annotations
import argparse
import os
import re
import shutil
import subprocess
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
SCHEMA = REPO_ROOT / "MobileGL" / "MG_Remote" / "Protocol" / "protocol.fbs"
OUT_DIR = REPO_ROOT / "MobileGL" / "MG_Remote" / "Protocol" / "generated"
OUT_FILE = OUT_DIR / "protocol_generated.h"
SUBMODULE = REPO_ROOT / "3rdparty" / "flatbuffers"
LICENSE_HEADER = """\
// MobileGL - MobileGL/MG_Remote/Protocol/generated/protocol_generated.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// GENERATED FILE - DO NOT EDIT.
// Regenerate with `python3 scripts/gen_protocol.py` after changing
// MobileGL/MG_Remote/Protocol/protocol.fbs. CI's flatc-check step regenerates
// this file and fails on `git diff --exit-code`.
"""
def submodule_version() -> str | None:
base = SUBMODULE / "include" / "flatbuffers" / "base.h"
if not base.is_file():
return None
text = base.read_text(encoding="utf-8", errors="replace")
parts = []
for macro in ("FLATBUFFERS_VERSION_MAJOR", "FLATBUFFERS_VERSION_MINOR",
"FLATBUFFERS_VERSION_REVISION"):
match = re.search(r"#\s*define\s+" + macro + r"\s+(\d+)", text)
if not match:
return None
parts.append(match.group(1))
return ".".join(parts)
def flatc_version(flatc: Path) -> str | None:
try:
out = subprocess.run([str(flatc), "--version"], check=True,
capture_output=True, text=True).stdout
except (OSError, subprocess.CalledProcessError):
return None
match = re.search(r"(\d+\.\d+\.\d+)", out)
return match.group(1) if match else None
def build_flatc(build_dir: Path, jobs: int) -> Path:
if not (SUBMODULE / "CMakeLists.txt").is_file():
sys.exit(f"error: {SUBMODULE} is empty - run "
f"`git submodule update --init 3rdparty/flatbuffers`")
exe_name = "flatc.exe" if os.name == "nt" else "flatc"
for candidate in (build_dir / exe_name, build_dir / "Release" / exe_name):
if candidate.is_file():
return candidate
build_dir.mkdir(parents=True, exist_ok=True)
cmake = shutil.which("cmake")
if cmake is None:
sys.exit("error: cmake not found; needed to build flatc from the submodule")
configure = [
cmake, "-S", str(SUBMODULE), "-B", str(build_dir),
"-DCMAKE_BUILD_TYPE=Release",
"-DFLATBUFFERS_BUILD_FLATC=ON",
"-DFLATBUFFERS_BUILD_FLATLIB=OFF",
"-DFLATBUFFERS_BUILD_FLATHASH=OFF",
"-DFLATBUFFERS_BUILD_TESTS=OFF",
"-DFLATBUFFERS_INSTALL=OFF",
]
if shutil.which("ninja"):
configure += ["-G", "Ninja"]
print("[gen_protocol] configuring flatc:", " ".join(configure), flush=True)
subprocess.run(configure, check=True)
print("[gen_protocol] building flatc", flush=True)
subprocess.run([cmake, "--build", str(build_dir), "--target", "flatc",
"--config", "Release", "--parallel", str(jobs)], check=True)
for candidate in (build_dir / exe_name, build_dir / "Release" / exe_name):
if candidate.is_file():
return candidate
sys.exit(f"error: flatc was not produced under {build_dir}")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--flatc", default=os.environ.get("MOBILEGL_FLATC_EXECUTABLE", ""),
help="path to a flatc binary (default: $MOBILEGL_FLATC_EXECUTABLE)")
parser.add_argument("--build-dir",
default=os.environ.get("MOBILEGL_FLATC_BUILD_DIR", ""),
help="where to build flatc from the submodule "
"(default: <repo>/../flatc-build, kept out of the repo)")
parser.add_argument("--jobs", type=int, default=os.cpu_count() or 4)
parser.add_argument("--check", action="store_true",
help="fail if the committed header is not what flatc produces")
parser.add_argument("--allow-version-mismatch", action="store_true",
help="proceed when flatc's version differs from the submodule's")
args = parser.parse_args()
if not SCHEMA.is_file():
sys.exit(f"error: schema not found: {SCHEMA}")
if args.flatc:
flatc = Path(args.flatc)
if not flatc.is_file():
sys.exit(f"error: --flatc/{'MOBILEGL_FLATC_EXECUTABLE'} points at a "
f"missing file: {flatc}")
else:
build_dir = Path(args.build_dir) if args.build_dir else REPO_ROOT.parent / "flatc-build"
flatc = build_flatc(build_dir.resolve(), args.jobs)
have, want = flatc_version(flatc), submodule_version()
print(f"[gen_protocol] flatc={flatc} version={have} submodule={want}", flush=True)
if have and want and have != want and not args.allow_version_mismatch:
sys.exit(f"error: flatc {have} does not match the pinned FlatBuffers runtime "
f"{want}; the generated header's version static_assert would fail. "
f"Unset MOBILEGL_FLATC_EXECUTABLE to build the pinned flatc, or pass "
f"--allow-version-mismatch.")
OUT_DIR.mkdir(parents=True, exist_ok=True)
previous = OUT_FILE.read_bytes() if OUT_FILE.is_file() else None
cmd = [str(flatc), "--cpp", "--cpp-std", "c++17", "-o", str(OUT_DIR), str(SCHEMA)]
print("[gen_protocol]", " ".join(cmd), flush=True)
subprocess.run(cmd, check=True, cwd=str(REPO_ROOT))
if not OUT_FILE.is_file():
sys.exit(f"error: flatc did not produce {OUT_FILE}")
body = OUT_FILE.read_text(encoding="utf-8")
OUT_FILE.write_text(LICENSE_HEADER + "\n" + body, encoding="utf-8", newline="\n")
if args.check and previous is not None and previous != OUT_FILE.read_bytes():
sys.exit("error: committed protocol_generated.h is stale; rerun "
"scripts/gen_protocol.py and commit the result")
print(f"[gen_protocol] wrote {OUT_FILE.relative_to(REPO_ROOT)}")
return 0
if __name__ == "__main__":
sys.exit(main())