[Docs] (docs/Disaggregated, MG_Remote, MG_Remote/Client, MG_Remote/Protocol, MG_Remote/Server, MG_Remote/Transport, MG_Remote/Wire): shorten long comments and move their design arguments into docs

This commit is contained in:
2026-10-15 21:53:39 -04:00
parent 37ecba200b
commit 23ee8bdb49
63 changed files with 930 additions and 3343 deletions
+8 -18
View File
@@ -6,24 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P11 B2 (docs/archive/contracts/CONTRACT-P11.md B2): T0, the zero-copy adoption of a persistently mapped
// store. The CLIENT allocates the store as an AHardwareBuffer (AHB), keeps a CPU lock on it for
// the store's life and uses that pointer as the adopted mapping; the SERVER imports the same
// pages as the store's resident twin (Espryt glBufferStorageExternalEXT, Magma
// VK_ANDROID_external_memory_android_hardware_buffer).
//
// HOW THE HANDLE CROSSES, AND WHY IT IS ORDERED WITH THE RECORDS THAT USE THE STORE. The adoption
// rides the data-ring map_persistent record (not a control op, which the apply thread could
// overtake under run-ahead). Its AHB travels over the session's aux socket as ONE descriptor: a
// fresh socketpair end (the "hop") on which AHardwareBuffer_sendHandleToUnixSocket already wrote
// the handle, with an Offer in the sideband naming the record's seq. The hop keeps the aux channel
// uniform (one fd + sideband per message, FdPassing's shape on every route) and the AHB's own
// message format off it. Step 1 of B2 proved both this and the direct form on the Redmi across
// every route (shell, another app, same app, spawned child; 64 KiB to 128 MiB).
//
// Every T0 map_persistent is preceded by exactly one Offer for its seq - with the handle, or with
// flags 0 when this client could not allocate one - so the server always has something to wait
// for and never waits for nothing.
// T0: zero-copy adoption of a persistently mapped buffer store (ARCHITECTURE.md §12). The client
// allocates the store as an AHardwareBuffer and keeps it CPU-locked for the store's life as its
// mapping; the server imports the same pages as the store's resident copy. The handle travels
// with the data-ring map_persistent record, not as a control op, which the apply thread could
// overtake under run-ahead: one aux-socket descriptor (a socketpair end the handle was written
// on) plus an Offer naming the record's seq. Every T0 map_persistent is preceded by exactly one
// Offer for its seq, with the handle or with flags 0 when the client could not allocate one, so
// the server never waits for an Offer that is not coming.
#pragma once
+7 -27
View File
@@ -6,33 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P11 A1 + B2 (docs/archive/contracts/CONTRACT-P11.md §1 and B2): THE PERSISTENT-MAP ADOPTION TIER IS ASKED
// AT THE HANDSHAKE, BY THE SIDE THAT KNOWS THE DATA PLANE, BEFORE THE FIRST map_persistent.
//
// Tier names follow docs/Disaggregated/design/07: T0 = the client allocates an AHardwareBuffer
// and the server imports it; T1 = the server exports an opaque fd (CLOSED, ID-P11-1); T2 = the
// resource owner declines and the client keeps the shadow and pushes.
//
// MOBILEGL_IPC_ADOPT_TIER IS THE SWITCH (ruling ID-P11-14). UNSET - the default since 2026-09-29,
// the user's decision - asks for T0 and falls back to T2 QUIETLY (MGLOG_D only, still counted),
// so a tcp session or a host session is not one line longer; 0 asks for T0 BY NAME (the table
// below); 1 is refused by name and runs T2; 2 is T2 byte for byte. T0 that this session cannot use
// is never a Fatal - it is T2 plus one named line (knob 0) or one D line (unset):
//
// data plane side knob 0 knob 1
// Stream (tcp) either `Refuse{AdoptTierOnStream, "T0"}`, T2 `Refuse{AdoptTierOnStream, "T1"}`, T2
// (A1, unchanged: a stream shares no memory, so no tier but T2 exists)
// SharedSegments client asks T0 in its Hello (LinkTerms. `Refuse{AdoptTierClosed, "T1"}`, asks T2
// adoptTier = 0); whether it runs T0 is
// read at the first map_persistent from
// kCapAdoptT0 (ClientSession::AdoptT0)
// SharedSegments server its own knob is not consulted: the client's ask decides, and the
// server grants it at its first native bind (ServerSession::
// SettleAdoptT0AtBind) or names why not
//
// In MG_Remote/Transport (the link-ratchet's SERVER partition) because both roles call it:
// ServerSession::Accept and ClientSession's two Start halves. The at-use check the server's
// applier and codec ask on the T2 path (Client::AdoptTierIsEmulate) reads the settlement back.
// The persistent-map adoption tier is settled at the handshake, before the first map_persistent,
// by the side that knows the data plane. T0: the client allocates an AHardwareBuffer and the
// server imports it; T1: a server-exported fd (closed); T2: decline, the client keeps the shadow
// and pushes. A T0 the session cannot use is never Fatal: it is T2 plus one line. The rules per
// data plane and knob value are in ARCHITECTURE.md §12.1. Lives in Transport because both roles
// call it (ServerSession::Accept, ClientSession::Start); Client::AdoptTierIsEmulate reads the
// settlement back on the T2 path.
#pragma once
+6 -30
View File
@@ -6,36 +6,12 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// PH-7 (1)(2)(3), ID-P7-3. THE ONE PLACE THAT KNOWS WHAT `MOBILEGL_IPC_TOKEN` MEANS.
//
// It was three places, and the three did not agree:
//
// SocketTransport.cpp's ListenTcp asked only "is it non-empty", and a one-byte token
// therefore unlocked a `tcp://0.0.0.0` listen.
// ServerSession.cpp compared with `std::strcmp` under "the role is not InProcess AND a token
// is configured".
// ServerMain.cpp compared with `std::string::operator==` under "the control connection is
// TCP", which additionally demanded that a peer present an EMPTY token to a server that had
// configured none - a rule the other site did not have and no document states.
//
// Two of the three compared byte-at-a-time and returned at the first differing byte. That is the
// textbook remote timing oracle: an attacker who can retry learns the token one byte at a time
// instead of guessing 2^128. Whether it is exploitable over a LAN at these timings is beside the
// point; the comparison costs the same either way, and a security property nobody has to
// re-derive is worth more than a measurement.
//
// WHY A MINIMUM LENGTH IS A LISTEN-TIME REFUSAL AND NOT A WARNING. Ph's threat model (ID-P7-3)
// is "anybody who can reach the port has a remote code path". The answer to it is the token, so
// a token an attacker can enumerate is not an answer - it is the appearance of one, which is
// worse than no token at all, because `no token` is already handled: it confines the listener to
// loopback. A four-byte token, before this file, did not confine anything. 16 bytes is the
// floor, it is checked where the listener is opened, and a short token FAILS THE LISTEN rather
// than silently degrading to loopback-only: an operator who set a token and got a loopback
// server would have no way to tell that from success, which is the exact silence Ph exists to
// remove.
//
// The token is never logged, at any level, on any path. The refusals name the policy, not the
// secret (CONTRACT-P65: "错令牌返回 Refuse,不打印令牌值").
// The one definition of what MOBILEGL_IPC_TOKEN means; listeners and handshake checks use these
// helpers (ARCHITECTURE.md §15.1). No token confines the listener to loopback. A configured
// token shorter than kMinimumAuthTokenBytes fails the listen rather than falling back to
// loopback, since an operator who set a token could not tell that fallback from success.
// Comparisons are constant time: a byte-at-a-time compare lets a peer that can retry learn the
// token one byte at a time. The token is never logged; refusals name the policy, not the secret.
#pragma once
+38 -162
View File
@@ -6,50 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The bidirectional doorbell: spin briefly, then park.
//
// Both directions exist, and that is the point (inherited design, earlier plan
// section 6.2a):
// - client -> server: the consumer spins, sets consumerParked, then blocks;
// the producer rings only when consumerParked is set.
// - server -> client: the client spins MOBILEGL_IPC_SPIN_US (default 50us),
// sets producerParked, then blocks; the server rings after advancing any
// watermark, only when producerParked is set.
// Without the second direction every client wait - present credit, a blocking
// kNeedsAck request, a full ring - degenerates into a cross-process spin on
// one shared cache line: up to a whole frame of a big core at full clock on a
// phone, fighting the GPU and the game's JVM for it. MobileGL has no affinity
// control anywhere in the tree, so it cannot even be pushed to a little core.
//
// Two implementations, no platform-specific wakeup primitive (no futex, no
// eventfd, no named event):
// - CondVarDoorbell for `inproc` (one process, two threads),
// - SocketDoorbell for `spawn` (one byte on a socket; POSIX only).
//
// The lost-wakeup window is closed by two seq_cst FENCES, not by the ordering
// of the park flag's own load and store:
// - the waiter sets the flag, executes std::atomic_thread_fence(seq_cst),
// and THEN re-tests the condition (Doorbell::Wait);
// - the notifier publishes its watermark, executes the same fence, and THEN
// reads the flag (NotifyIfParked).
// Both fences sit in the single seq_cst total order, so one precedes the
// other, and [atomics.order] then forces at least one side to observe the
// other's store. The flag's own accesses may be relaxed: they are not what
// closes the window.
//
// A seq_cst store paired with a seq_cst load would NOT be enough, which is
// why the fences are here and why neither may be removed. That Dekker
// argument needs all FOUR accesses in the total order, and the other two are
// not: the watermark publish is a release store (RingProducer::Publish) and
// the condition re-test is an acquire load. On x86 the gap is concrete rather
// than theoretical - a release store is a plain MOV that can still sit in the
// store buffer while the load of the park flag, also a plain MOV, reads 0, so
// the notifier skips the ring and the waiter parks on a stale watermark
// forever. (ARMv8 survives it only because STLR->LDAR is RCsc, i.e. by luck.)
//
// The other half of the contract is ordering between the caller and the
// fence: NotifyIfParked must be called AFTER the watermark is published. A
// fence only orders what precedes it.
// The bidirectional doorbell: a waiter spins, sets its park flag, then blocks, and the other side
// rings only when that flag is set (client -> server on consumerParked; server -> client on
// producerParked, after any watermark advance). Without the server -> client direction every client
// wait would be a long cross-process spin on one shared cache line. No platform wakeup primitive
// (futex, eventfd, named event): CondVarDoorbell in process, SocketDoorbell across processes.
// Two seq_cst fences close the lost-wakeup window: Doorbell::Wait's (after setting the flag, before
// re-testing) and NotifyIfParked's (after publishing, before reading the flag). Neither may be
// removed, and NotifyIfParked must follow the publish; why: ARCHITECTURE.md §11.3.1.
#pragma once
@@ -83,45 +47,14 @@ namespace MobileGL::MG_Remote::Transport {
#endif
}
// -----------------------------------------------------------------------
// THE SPIN BUDGET IS COUNTED IN ITERATIONS, NOT IN CLOCK READS.
//
// WHY IT CHANGED (P5d round 3, package T item 3). Wait() used to spin until
// steady_clock passed `start + spinUs`, reading the clock once per 64 spins.
// On the workload device a steady_clock read is a real syscall (no vDSO), and
// simpleperf at head 56a77348 measured the GL thread paying
// steady_clock::now() 2.34 self / 5.91 inclusive plus __kernel_clock_gettime
// 2.37 and clock_gettime 1.94 - about 6.6% of the thread spent asking what
// time it is, at 900+ waits per frame, for a duration that is a HINT. The
// common case (WaitForApplied, server not yet there) paid at least two of
// those reads per wait even when the spin succeeded on its second iteration.
//
// WHAT REPLACES IT. Once per process the loop measures itself: how many
// `load(acquire)` + CpuRelax() iterations fit in a microsecond. Wait() then
// spins `itersPerUs * spinUs` iterations and reads no clock at all; only the
// park phase - which is about to block, so a syscall there is noise - computes
// a deadline.
//
// THE OVERSHOOT BOUND, STATED HONESTLY. The old loop overshot by at most one
// batch of 64 yields. This one overshoots by whatever the calibration was
// wrong by, and the calibration is a single reading taken on one core at one
// frequency:
// - a later DVFS drop or a migration to a little core stretches the spin by
// the frequency ratio (on the workload device's big/little spread that is
// bounded by roughly 3-4x, so a 50 us budget can become ~200 us);
// - a preemption inside the spin adds one scheduler slice, exactly as before;
// - the calibration probe does not run the caller's `ready()`, which is one
// or two acquire loads, so a real iteration is dearer than a probed one and
// the spin ends EARLY rather than late by that factor;
// - the clamps below cap the pathological directions: a probe that was
// preempted cannot produce a budget smaller than kMinSpinItersPerUs, and a
// probe that saw a broken clock cannot produce one larger than
// kMaxSpinItersPerUs.
// None of it is a correctness bound: `ready()` is re-tested every iteration
// and the deadline, when there is one, is still measured against the real
// clock in the park phase. The cost of being wrong is CPU burnt before a park
// (too long) or a park that a few more spins would have avoided (too short).
// -----------------------------------------------------------------------
// The spin budget is counted in iterations, not clock reads: CalibrateSpinItersPerUs measures
// once per process how many acquire-load + CpuRelax() iterations fit in a microsecond, and Wait
// spins itersPerUs * spinUs without reading a clock (a steady_clock read can be a syscall, and
// Wait runs hundreds of times per frame). Only the park phase computes a deadline. The budget
// is a hint, not a correctness bound: `ready()` is re-tested every iteration; a DVFS drop or a
// move to a little core stretches the spin by the frequency ratio (roughly 3-4x); the probe
// omits `ready()`, so real spins end early; the clamps in CalibrateSpinItersPerUs bound bad
// readings.
namespace Detail {
// 0 until the first Wait calibrates it. Plain relaxed atomic rather than a
// function-local static: a static's guard variable is a load and a branch on
@@ -182,29 +115,12 @@ namespace MobileGL::MG_Remote::Transport {
// peer process, which is what makes the whole latch a no-op under inproc and monolith.
virtual bool PeerHungUp() const { return false; }
// How many Wait() calls on this bell exhausted their spin budget and
// really blocked. THE WAIT LEDGER'S RAW READING: the pair (waits, parks)
// is what says whether the spin budget is sized for the workload, and it
// is the one number a wait counter cannot give - a wait that spun and a
// wait that blocked cost three orders of magnitude apart.
//
// ONCE PER Wait, NOT ONCE PER Park: a wait that blocks, wakes on a
// remembered Notify, re-tests false and blocks again is ONE wait that
// paid the park, and a `chunkMs == 0` Park is a poll that does not block
// at all. Both are counted out at the site below, so parks <= waits holds
// for a single waiter.
//
// PER BELL, THOUGH, NOT PER WAIT PATH. A bell belongs to an ENDPOINT and
// every waiter on that endpoint shares it: the client's self bell carries
// SessionProducer's barrier waits AND the encoder's SEG_STAGE retirement
// wait (PipeWireCodec's m_stageRetirementBell, which is the same object).
// A ledger that prints parks as the subset of ITS OWN waits must pass a
// `parkTally` to Wait rather than read this; this counter answers "parks
// on this bell", which is what a test on a private bell wants.
//
// Read from any thread; the counter is monotone and relaxed, so a reader
// may see a value one behind, which is exactly what a per-frame gauge
// wants.
// How many Wait() calls on this bell exhausted their spin budget and really blocked,
// counted once per Wait (re-parks and zero-length polls are not counted), so parks <= waits
// for one waiter. The count is per bell, and a bell is shared by every waiter on its
// endpoint (the client's self bell also carries PipeWireCodec's SEG_STAGE retirement wait);
// a caller that needs parks as a subset of its own waits passes `parkTally` to Wait
// instead. Monotone and relaxed: a reader on another thread may see a value one behind.
std::uint64_t ParkEntries() const { return m_parkEntries.load(std::memory_order_relaxed); }
// Spin `spinUs`, then park until `ready()` or the deadline.
@@ -221,24 +137,9 @@ namespace MobileGL::MG_Remote::Transport {
return true;
}
// THE SPIN PHASE READS NO CLOCK AT ALL. It used to read one per 64
// iterations plus one before the loop - see the SpinItersPerUs block
// above for the measurement that made even that too dear, and for the
// overshoot bound the iteration budget buys instead. The duration is
// still a hint; what changed is that asking for it is free.
//
// spinUs == 0 NOW MEANS ZERO ITERATIONS, AND THAT IS A DELIBERATE CHANGE
// rather than a side effect. The old batched loop tested
// `(++spinBatch & 63) != 0 || now < spinEnd`, so a caller that asked for
// no spin still ran 63 iterations before the first clock read could end
// the loop. That was an artifact of the batch size, never a promise, and
// multiplying a budget the caller deliberately set to zero would be the
// wrong way to keep it. Two callers pass 0: PipeWireCodec's SEG_STAGE
// retirement wait (rare, already slow, and it wants the park) and any
// lane that sets MOBILEGL_IPC_SPIN_US=0 as a no-spin control, which now
// really gets no spin - which is the point of such a control. The
// short-circuit also keeps a zero-spin caller from paying the one-time
// calibration probe for a reading it will never multiply by.
// The spin phase reads no clock (see SpinItersPerUs). spinUs == 0 means no spin at all
// and skips the calibration probe: PipeWireCodec's SEG_STAGE retirement wait passes 0
// because it wants the park, and MOBILEGL_IPC_SPIN_US=0 is a no-spin control.
const std::uint64_t budget =
spinUs == 0 ? 0ull
: static_cast<std::uint64_t>(SpinItersPerUs()) *
@@ -294,27 +195,12 @@ namespace MobileGL::MG_Remote::Transport {
std::chrono::duration_cast<std::chrono::milliseconds>(deadline - now).count();
chunkMs = remaining <= 0 ? 0 : static_cast<std::uint32_t>(remaining);
}
// COUNTED HERE, at the one place a waiter stops spinning and
// really blocks (P5d round 3, package T item 4). Unconditional
// rather than behind the stats latch: it is one relaxed add on a
// path that is about to enter a condition variable or a poll(),
// and a counter that only exists when MOBILEGL_PIPE_STATS is set
// could not be read by a test that wants to prove the spin budget
// avoided the park.
//
// TWO THINGS ARE COUNTED OUT, because this number is published as
// the SUBSET of the waits that blocked, and a subset that can
// exceed its own denominator is worse than no number at all:
// - RE-PARKS. One Wait can traverse this loop many times: a
// CondVarDoorbell remembers a Notify delivered while nobody
// was waiting, so Park returns at once, `ready()` is still
// false (the peer had already drained what it published) and
// the wait parks again. That is one wait that paid a park, not
// two, so only the first is counted.
// - POLLS. `chunkMs == 0` is the last sub-millisecond of a
// finite timeout rounding down; Park(0) does not block, and
// counting it would charge a park per iteration of a tight
// re-test loop that never left the CPU.
// Counted here, where a waiter really blocks, and unconditionally (one relaxed
// add before a condvar wait or poll), so a test can read it without
// MOBILEGL_PIPE_STATS. Only the first blocking Park of a Wait counts: a
// CondVarDoorbell can return at once on a remembered Notify and re-park, and
// chunkMs == 0 is a non-blocking poll at the end of a finite timeout. That keeps
// parks a subset of waits.
if (chunkMs != 0 && !countedPark) {
countedPark = true;
m_parkEntries.fetch_add(1, std::memory_order_relaxed);
@@ -438,23 +324,13 @@ namespace MobileGL::MG_Remote::Transport {
int Fd() const { return m_fd; }
// P6 `dl` (CONTRACT-P6 D5c). THE DESCRIPTOR THAT ANSWERS "IS THE PEER STILL THERE",
// which for this object can never be the one it parks on.
//
// The client parks on clientBell[0] and rings ITSELF through clientBell[1], so it holds
// BOTH ends of that socketpair. EOF arrives only when every writer closes and the client
// is one of them, so that descriptor cannot hang up no matter what happens to the server.
// A bell that can wake itself is a bell that cannot hear a death. Measured: a server that
// died on its first Clear left the client waiting out the full 120 s barrier and
// reporting Fatal{BarrierTimeout} - the wrong diagnosis, two minutes late.
//
// The witness is a descriptor whose FAR END ONLY THE PEER HOLDS - in practice the control
// socket. It is NOT OWNED here and is NEVER READ FROM: Park adds it to the poll set with
// `events` asking for hangup ALONE, so a control reply sitting unread in its queue cannot
// wake the bell and cannot be consumed from under SocketTransport's reassembler. What it
// contributes is exactly one fact, and only once: the peer is gone.
//
// -1 disables it, which is what every bell that has no peer socket uses.
// The death witness: a descriptor whose far end only the peer holds (in practice the
// control socket). The park descriptor cannot report the peer's death when this side also
// holds the notify end of the same socketpair (the client rings itself through
// clientBell[1]), so without a witness a dead server surfaces only as a barrier timeout.
// Not owned and never read: Park polls it for hangup only, so unread control replies
// neither wake the bell nor get taken from SocketTransport's reassembler. A hangup sets
// PeerHungUp(). -1 disables it.
void SetDeathWitness(int fd) { m_witnessFd = fd; }
private:
+7 -24
View File
@@ -6,30 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// SEG_EVENT: the server -> client reverse channel. Owner: package s1.
//
// WHAT P5 OWES HERE AND NOTHING MORE (BRIEF R-12, and s1's brief): it must be
// able to CARRY OnBufferWriteback, OnGpuWritten and OnSurfaceChanged. The other
// seven MGPipeCallbacks members are off P5's reduced path, and THE OVERFLOW
// POLICY IS P9's - what is here is the mechanism (a full ring latches
// eventRingFull, a lossy post that is dropped counts in eventDropped) and not a
// policy that decides between them.
//
// IT IS A SECOND RingControl, NOT A THIRD CURSOR SET. RingControl carries two
// cursor triples (SEG_CMD and SEG_STAGE) and adding a third would resize the
// shared page that Ring.h static_asserts at exactly 4096 bytes. So SEG_EVENT
// gets its OWN control page at its own head and drives it with the Cmd cursor
// set - the same RingProducer/RingConsumer code, in the opposite direction. The
// two EVENT FLAGS still live in the SEG_CMD page, because that is where Ring.h
// declares them and where the client's own waits already look.
//
// THE PAYLOAD SHAPES ARE FIXED-WIDTH AND LIVE HERE, not in MG_Pipe's headers.
// Nothing under Transport/ may reach MobileGL/Includes.h (WireLog.h:9-24, and
// the purity gate's `wire-header` probe), so MGPipeHandle / MGPRange /
// MGPSurfaceInfo cannot be named in this file. The wire shapes below mirror them
// field for field and the SESSIONS assert the two agree, which is the same
// discipline the rest of the wire is held to: a wire struct is fixed-width, and
// the conversion happens where the frontend types are legal.
// SEG_EVENT: the server-to-client reverse channel for MGPipeCallbacks events; the overflow rules
// (eventRingFull, eventDropped) are ARCHITECTURE.md §11.7. It has its own RingControl page,
// driven with the Cmd cursor set by the same producer/consumer code in the opposite direction,
// because a third cursor set would grow RingControl past its fixed 4 KiB page. The two event
// flags stay in the SEG_CMD page, where Ring.h declares them. Nothing under Transport/ may
// include MobileGL/Includes.h, so the payloads below are fixed-width mirrors of MGPipeHandle,
// MGPRange and MGPSurfaceInfo, converted by the sessions where the frontend types are legal.
#pragma once
+6 -18
View File
@@ -6,24 +6,12 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// SCM_RIGHTS descriptor passing over an AF_UNIX socket pair. POSIX only.
//
// This is the FIRST transport commit, deliberately (inherited design, plan
// section 8.1, "SCM_RIGHTS must be implemented in the first transport
// commit"). The earlier branch pushed it to a later phase and hardcoded
// `out->fd = -1` in its offer poll, so on the only platform that matters its
// data plane could never move a byte: every segment announcement resolved to
// "no descriptor". A transport whose shm cannot cross the process boundary is
// not a transport.
//
// Channel shape: a dedicated AF_UNIX SOCK_DGRAM socketpair, NOT the control
// byte stream. Two reasons:
// - SOCK_DGRAM preserves message boundaries on every POSIX (SOCK_SEQPACKET
// does not exist on macOS), so one sendmsg is exactly one recvmsg and the
// ancillary data can never be split away from its payload;
// - ancillary data attached to a byte stream binds to whichever ordinary
// byte happens to be at the front of the reader's buffer, which is
// unmanageable once frames are being reassembled.
// SCM_RIGHTS descriptor passing over an AF_UNIX socket pair (POSIX only); without it the shared
// segments cannot cross the process boundary. The channel is a dedicated SOCK_DGRAM socketpair,
// not the control byte stream: datagrams keep message boundaries on every POSIX (macOS has no
// SOCK_SEQPACKET), so one sendmsg is one recvmsg and ancillary data stays with its payload; on a
// byte stream it would bind to whichever byte is at the front of the reader's buffer, which the
// frame reassembler cannot manage.
#pragma once
+7 -20
View File
@@ -6,26 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// Control-channel wire framing: [u32 magic 'MGLF'][u32 payloadLength][payload].
// Length excludes the 8-byte header and is capped at 64 MiB.
//
// Two defects of the earlier branch's codec are fixed here, and both are the
// reason this file is not a copy of it:
//
// 1. Its Feed() unconditionally returned OK and its header peek merely
// returned false on a bad magic or an oversized length. A corrupt or
// desynchronized stream therefore turned into a silent, permanent hang -
// the reader kept waiting for a message that could never be parsed, with
// no error anywhere. Here a violation latches a failed state, is logged at
// ERROR, and every later call returns MOBILEGL_ERR_PROTOCOL_MISMATCH.
//
// 2. Its receive path failed the call and consumed the message when the
// caller's buffer was too small, wedging the stream. Here
// MOBILEGL_ERR_BUFFER_TOO_SMALL reports the required size and KEEPS the
// message queued.
//
// The reader is a plain byte-stream reassembler: it never assumes a read()
// returned a whole frame.
// Control-channel wire framing: [u32 magic 'MGLF'][u32 payloadLength][payload]. The length
// excludes the 8-byte header and is capped at 64 MiB.
// A bad magic or an oversized length latches the reader failed, is logged at ERROR, and makes
// every later call return MOBILEGL_ERR_PROTOCOL_MISMATCH, so a desynchronized stream fails rather
// than hangs. A buffer too small for the next message gets MOBILEGL_ERR_BUFFER_TOO_SMALL with the
// required size and the message stays queued. The reader is a byte-stream reassembler: it never
// assumes a read() returned a whole frame.
#pragma once
+14 -96
View File
@@ -6,64 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The DATA-plane seam. Declared by package c6, implemented by package lk.
// Authority: CONTRACT-P6.md §8.
//
// WHY THIS EXISTS AND WHY IT IS NOT ITransport. ITransport.h:16-20 says in its
// own header that everything on the hot path bypasses it: it carries the
// handshake, surface ops, resync, aux requests and fatals, and that scope is
// already correct for a socket. Below it there was NO interface of any kind -
// Ring.h, ReplySlot.h and EventRing.h each take a raw `void* base` plus a
// capacity, so "swap the transport" had nothing to swap. This header is that
// missing thing, and ITransport is neither extended nor changed by it.
//
// NOTHING INCLUDES THIS FILE IN c6. That is the whole of c6's behaviour
// guarantee: the seam is declared, StreamLink.h is declared against it (which
// is the adequacy proof - see below), and not one call site moves. Package lk
// implements ShmLink by relocating today's SessionSegments / Ring /
// ReplySlotPool / EventRing behind it, line for line, and only then rewrites
// the call sites.
//
// THE SEAM IS SETUP-TIME, NOT HOT-PATH. No method here is called per record,
// per publish, or inside a spin predicate. The producer takes a LinkArena once
// at attach and reserves through it non-virtually; every wait predicate reads
// LinkProgress through a raw pointer taken once. Three grounds, the second
// decisive (CONTRACT-P6.md §8.2):
//
// 1. Two virtuals per record at the 4000 records/frame SEG_CMD is sized
// against is ~8000 indirect calls/frame, about 0.5% of the measured
// 15.72 ms client frame - affordable, but the only lane that could
// measure it has +/-4% dispersion, EIGHT TIMES the effect. A gate that
// can only ever print "within noise" is ID-124's failure mode one level
// down.
// 2. Doorbell::Wait is a non-virtual template whose spin loop calls ready()
// up to SpinItersPerUs() * spinUs times; at 916 waits/frame with 99.8%
// resolved inside the spin, the predicate runs 1e5-1e6 times per frame
// and reads the watermarks straight off the shared page. That is the
// 31.58%-of-the-GL-thread path. A virtual there is not a rounding
// question.
// 3. The encode path already depends on four inline accessors around its one
// Reserve; virtualising Reserve alone forces either four more virtuals or
// moving R-10's wrap counters behind the seam, and those are deliberately
// encoder-owned.
//
// So lk's gate is an objdump call-count of the encode path, not a wall clock:
// after lk that path has no more out-of-line calls than before. That is
// falsifiable; "within noise" is not.
//
// WHAT THIS HEADER DELIBERATELY DOES NOT DO.
// - It does not define Watermark::Reached or any other predicate that
// SessionRings.h already defines. One definition, one owner.
// - It does not relocate the bodies of RingProducer::Reserve or
// RingConsumer::Pop. Those log (MGLOG_E / MGLOG_D) and use Align8 and
// <cstring>; no header under Transport/ pulls the logger in, and this one
// does not start. They stay out of line and move to ShmLink.cpp in lk.
// - It declares no function it does not define. Every entry point here is
// pure virtual or a POD accessor.
// - It does not regroup RingControl. That regroup is real and necessary
// (CONTRACT-P6.md §8.3) but its rename blast radius is 109 production
// sites, five of them behind a non-Android #if, so it is lk's with a
// published per-file itemisation - not a side effect of declaring a seam.
// The data-plane seam: record reservation and commit, watermarks, replies, events and span
// resolution, implemented by ShmLink and StreamLink (ARCHITECTURE.md §11.9). ITransport carries
// only the control plane and is not part of it.
// Reserve, pop and the wait predicates never dispatch through it: callers cache the ring
// endpoints (CommandsOut/CommandsIn) and the LinkProgress pointer once at attach. The spin
// predicate runs up to 1e5-1e6 times a frame, where a virtual call is a measurable cost.
// Watermark predicates such as Watermark::Reached stay in SessionRings.h.
#pragma once
@@ -94,44 +43,13 @@ namespace MobileGL::MG_Remote::Transport {
};
// ---- the progress block -------------------------------------------------
//
// The watermarks every wait predicate reads. On ShmLink this aliases the
// shared control page and the loads are cross-process acquire loads; on
// StreamLink it is process-local and a reader thread stores into it from
// decoded progress messages.
//
// THE RULES SURVIVE THE MECHANISM VERBATIM, and that is the point of naming
// the block rather than the page: late-never-early (Ring.h's "BATCHING MAY ONLY MAKE A WATERMARK LATE"), `>=`
// never `==` (SessionRings.h's Watermark::Reached), monotone with a
// backwards move Fatal (Ring.cpp's AdvanceMonotonic). A stream link changes
// only HOW a value arrives, never what it means.
//
// WHY THESE FOUR AND NOT THE WHOLE PAGE, AND WHY THE EVENT FLAGS ARE NOT
// HERE. RingControl's watermark cache line HELD, until lk regrouped it, {appliedSeq,
// submittedSeq, retiredSeq, completedFrameSerial, presentAckSerial}, and
// submittedSeq is PRODUCER-written (Ring.h's "THE FIVE WATERMARKS" block,
// "Advanced by the PRODUCER after it publishes") while the other four are consumer-written - so the
// line has MIXED WRITERS as it stands and no single-writer block can be
// aliased over it. lk's regroup moves submittedSeq out to the producer's
// own line, beside cmdHead, which is the other thing the producer writes;
// what is left IS this struct, and it IS a named member of RingControl,
// pinned there by per-field offsetof static_asserts (Ring.h).
//
// THE TWO EVENT FLAGS ARE DELIBERATELY EXCLUDED, and an earlier draft of
// this header had them in and was wrong. eventRingFull has TWO writers:
// the server latches it with store(1) when SEG_EVENT fills
// (EventRing.h:140) and the CLIENT clears it with an exchange(0) on EVERY
// DRAIN (EventRing.h:200). Promoting it here would put a per-drain RMW by
// the client on the same cache line as appliedSeq - which the server
// release-stores every 64 records and the client's spin predicate reads
// 1e5-1e6 times per frame. That is new false sharing on the hottest line
// in the system, and it would land in exactly the numbers lk's gate has to
// hold constant. They stay in the doorbell/generation group, where mixed
// writers already live beside consumerParked/producerParked, and cross the
// seam through EventFlags() instead.
//
// It was free to regroup while both peers are the same binary; a wire break the day a second build exists,
// which is why it is booked now rather than later.
// The watermarks every wait predicate reads: on ShmLink an alias of the shared control page,
// on StreamLink process-local, stored by a reader thread from progress messages. Both keep
// the same rules: a watermark may arrive late, never early (Ring.h), compare with `>=`
// (Watermark::Reached), and a backwards move is Fatal (AdvanceMonotonic). It holds only
// consumer-written fields, so it is one RingControl member on its own line (ARCHITECTURE.md
// §11.2). The event flags stay out: the client clears eventRingFull on every drain, an RMW
// that would share the cache line the client's spin predicate reads for appliedSeq.
struct LinkProgress {
std::atomic<std::uint64_t> appliedSeq; // records applied
std::atomic<std::uint64_t> retiredSeq; // GPU finished
+14 -38
View File
@@ -6,27 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The control-plane transport interface.
//
// It is deliberately dumb: complete messages in, complete messages out, plus
// the one thing shared memory cannot do without help - handing a file
// descriptor to the peer. No session routing, no seq accounting, no
// serialization; those live above, in the protocol layer.
//
// Everything on the hot path bypasses this interface entirely: records go into
// the SEG_CMD ring (Ring.h) and the peer is woken through a Doorbell
// (Doorbell.h). ITransport carries the handshake, surface ops, resync, aux
// requests and fatals - the rare, variable-length, must-evolve traffic that
// plan section 7.1 assigns to FlatBuffers tables.
//
// This header stays dependency-light on purpose (mg_protocol_base.h plus the
// standard library): it is included by both roles and by the eventual
// server-side binary, and nothing about a byte pipe needs the GL frontend's
// umbrella header.
//
// Threading: one instance is not internally synchronized for send; callers
// serialize sends. ReceiveFrame/ReceiveFd may be called from one dedicated
// reader thread concurrently with sends from another.
// The control-plane transport interface: complete messages in, complete messages out, plus
// handing a file descriptor to the peer. No session routing, seq accounting or serialization;
// those live in the protocol layer. Hot-path records bypass it (Ring.h, Doorbell.h); it carries
// the handshake, surface ops, aux requests and fatals (ARCHITECTURE.md §11.4).
// Dependency-light (mg_protocol_base.h and the standard library): both roles include it, and a
// byte pipe needs nothing from the GL frontend's umbrella header.
// Threading: callers serialize sends; one reader thread may call ReceiveFrame/ReceiveFd
// concurrently with sends from another.
#pragma once
@@ -60,23 +47,12 @@ namespace MobileGL::MG_Remote::Transport {
// Receives the next complete message.
//
// MOBILEGL_OK - copied into `buffer`, *outSize is
// the message size, message consumed.
// MOBILEGL_ERR_BUFFER_TOO_SMALL - `buffer` is too small. *outSize is
// the size required and THE MESSAGE
// STAYS QUEUED: call again with a
// buffer of at least that size and it
// is still there.
// MOBILEGL_ERR_TIMEOUT - nothing arrived within timeoutMs
// (0 = non-blocking poll).
// MOBILEGL_ERR_TRANSPORT_CLOSED - peer gone, nothing left buffered.
// MOBILEGL_ERR_PROTOCOL_MISMATCH- framing violated; the transport is
// latched failed and never recovers.
//
// The buffer-too-small half of that contract is the whole point of
// having one: the earlier branch's transport failed the call AND
// dropped the message, which wedges the stream permanently the first
// time a message is bigger than the reader's guess.
// MOBILEGL_OK - copied into `buffer`, *outSize is its size, consumed.
// MOBILEGL_ERR_BUFFER_TOO_SMALL - *outSize is the size required; the message stays
// queued for a call with a large enough buffer.
// MOBILEGL_ERR_TIMEOUT - nothing arrived within timeoutMs (0 = poll).
// MOBILEGL_ERR_TRANSPORT_CLOSED - peer gone, nothing left buffered.
// MOBILEGL_ERR_PROTOCOL_MISMATCH - framing violated; the transport stays failed.
virtual MobileGLResult ReceiveFrame(MobileGLMutableByteSpan buffer, std::uint64_t* outSize,
std::uint32_t timeoutMs) = 0;
@@ -6,30 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The `inproc` transport: two in-memory message queues and a pair of condvar
// doorbells, one connected endpoint at each end.
//
// It is not a test double. `inproc` is a delivery mode of its own - the server
// side is the monolith's own render thread, which is the single largest CPU
// lever this project has, and it is also the CI form of the split build. What
// it does NOT exercise is serialization of the byte stream, so the framing
// codec is covered separately by FramingTest.
//
// It is built by MOBILEGL_BUILD_DISAGGREGATED, the one option this skeleton
// adds, and selected at RUNTIME (plan appendix B: MOBILEGL_TRANSPORT =
// monolith / inproc / spawn / ...). The plan also reserves a separate
// MOBILEGL_BUILD_DISAGGREGATED_INPROC option for the role-isolation shim that
// a single-process CI build will need; that option does not exist yet, and
// nothing here depends on it.
//
// Messages are queued whole, so no framing bytes are involved; the size cap is
// still enforced so that a payload which would be illegal on a socket is
// illegal here too and does not pass CI only to fail after the switch to
// `spawn`.
//
// Descriptor passing is a plain dup(): both ends are the same process, so
// there is nothing to transfer, but the API stays identical so callers can be
// written once.
// The `inproc` transport: two in-memory message queues and a pair of condvar doorbells, one
// connected endpoint at each end. Not a test double: inproc is a delivery mode of its own and
// the CI form of the split build. It carries only the control plane; records use the same
// shared-segment rings as spawn. Messages are queued whole, so framing is not exercised here
// (FramingTest covers it), but the size cap is enforced so a payload illegal on a socket fails
// here too. Descriptor passing is a plain dup(), since both ends are one process, behind the
// same API as the socket transport. Selected at run time by MOBILEGL_TRANSPORT=inproc.
#pragma once
+7 -59
View File
@@ -6,65 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P9 W3. THE SERVER'S LOG LINES ON THE CONTROL PLANE, GRADED, AND NEVER A REASON TO STOP APPLYING.
//
// On TCP the server forwards every line it logs to its client as a LogLine control frame, and the
// client files it under `<base>.server.log` (ControlInbox.cpp). Until this file the forward was a
// blocking SocketTransport::SendFrame made by the LOGGING THREAD, inside the logger: a client that
// stopped reading its control connection - a GL thread parked in a debugger, a ControlInbox whose
// bounded queue filled, a peer that simply stopped - filled the socket, and the next MGLOG on the
// apply thread never returned. Every record behind it waited; a forfeit that would have ended the
// session by name (ServerSession::ForfeitReverseChannel) was itself a log line and never finished
// being said. Design/04 §8.1 had written the policy down from the start (<= WARN lossy, >= ERROR
// lossless and rate limited), and LogLine.level has been on the wire for it since P6; nothing
// used either, and every line went out as Info.
//
// THE SHAPE. A LogForwardChannel owns a queue and ONE sender thread; the forwarder the logger
// calls (Offer) only enqueues, so the logging thread never touches the socket. The sender is the
// only thread that writes log frames, and it writes them in the order they were offered.
//
// THE POLICY, per severity (MOBILEGL_LOG_LEVEL_*):
// * DEBUG / INFO / WARN are LOSSY. A line that would take the queue's lossy bytes past
// LogForwardLimits::lossyQueueBytes is dropped and COUNTED. The drop is not silent to the
// client either: when the gap closes - the next line at or below WARN that fits, a LogFlush
// ack, or Close - the channel puts a WARN line of its own in the stream, `LogForward{Dropped} -
// N line(s) at or below WARN (B bytes) were not forwarded, from <time> until here`, so the gap
// in `<base>.server.log` is marked and counted. (ERROR lines that got through in the middle of
// the gap sit before that line; the time says where the gap began.)
// * ERROR is LOSSLESS, and PACED rather than dropped: at most errorLinesPerSecond leave per
// second after a burst of errorBurstLines, and a line over that budget WAITS in the queue, in
// order (it is delayed, never discarded). Memory still has to be bounded, so the error bytes
// queued are capped too (errorQueueBytes, eight times the lossy cap): a line past it - a peer
// that has not read for that long, or a flood well past the pace for that long - is not
// queued, and is replaced in the stream by one ERROR line `LogForward{Coalesced} - N ERROR
// line(s) were not forwarded here` (queued before the next item that does get in), never by
// nothing. The server's own log file (written before any forward, Log.cpp) has every line
// either way.
// * FATAL is ERROR plus a wait: the logging thread is about to abort, and a line still in the
// queue when it does is a line the client never gets - the one it most needs. So a FATAL
// Offer waits until the sender has written everything up to and including it, for at most
// fatalFlushMs, and then returns whether or not the peer took it - and once such a wait has run
// out, later FATAL lines do not wait again until the sender has written something (MGLOG_F also
// marks survivable conditions, several in a row). (Log.cpp hands a FATAL line over outside the
// log mutex, so that wait holds nobody else's line.)
// * A LogFlush acknowledgement (OfferControlFrame) is queued BEHIND every line offered before
// it, which is what makes MGPipeSyncPeerLog's promise ("every line before the flush is in the
// file when the ack arrives") still true now that a line is not on the wire when MGLOG returns.
// While one is queued the sender does not pace: the client is waiting for it.
//
// THE BOUND. An Offer below FATAL takes the channel's mutex, copies the line and returns; the
// sender never holds that mutex across a socket write. So the time a logging thread spends
// forwarding is a lock and a copy, whatever the peer does - LogForwardCounters::maxOfferNs is that
// time measured, and the F2 fault injection (EventForfeitPeerTest) asserts on it.
//
// CLOSE. Close() stops taking lines, gives the queue closeDrainMs to go out UNPACED, and then stops
// the sender. A sender still inside a write when that budget runs out is blocked on a peer that is
// not reading; Close() then calls the `unblock` hook the owner supplied (ServerMain: shutdown(2) of
// the control socket - the session is over by then) so the thread can be joined rather than
// leaked. What was still queued is counted as abandoned.
//
// SCOPE. Only the TCP server shapes forward (ServerMain.cpp, MOBILEGL_IPC_LOG_FORWARD): a unix spawn
// server and an inproc apply thread write `<base>.server.log` themselves.
// Forwards the server's log lines to its client as LogLine control frames, graded by severity,
// so that logging below FATAL never waits on the peer: Offer() copies the line into a queue, and
// one sender thread writes frames in offer order without holding the queue's mutex. DEBUG to
// WARN are lossy, ERROR is paced but not dropped (coalesced only past errorQueueBytes), FATAL
// also waits up to fatalFlushMs, and a LogFlush ack follows every line offered before it.
// Policy, counters and Close(): ARCHITECTURE.md §8.3.1. Only TCP server shapes forward; other
// servers write `<base>.server.log` themselves.
#pragma once
+13 -65
View File
@@ -6,47 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// SEG_REPLY: the slot pool the server writes a kReplySlot answer into, and the
// client reads it back out of. Owner: package s1.
//
// THE ID IS THE RECORD SEQUENCE NUMBER (P5 R-3). There is no second id space and
// no allocator: the ten kReplySlot calls carry no MGPReplySlot in their payloads
// (that is why the id has to be DERIVED rather than carried), the wire has no
// per-record seq field (ARCHITECTURE.md §4), so the record's ordinal IS its
// reply-slot id. The slot is addressed `seq % slotCount` and the server STAMPS
// THE SEQ BACK INTO THE SLOT HEADER, which is what makes a wrong-slot read
// detectable rather than merely plausible. Seq is 1-based; 0 means "no record".
//
// THE SLOT HEADER IS CONTRACT-P5 TABLE 0's ROW, verbatim:
// { Uint64 Seq; Int32 Status; Uint32 Size; } // 16 bytes, then the payload
// Status: 0 = OK, 1 = DECLINED, 2 = ERROR
//
// DECLINED IS A REAL ANSWER, NOT A FAILURE. It is how MapPersistent says nullptr
// (R-6) and how the four Bool acceptance entry points - ResourceCreate,
// ResourceRespecify, ResourceSubData, SetTextureParams - say false (R-5). A
// client that folds DECLINED into "the call failed" re-creates ID-39's 66 lost
// uploads from the other side, and a client that folds it into OK accepts a
// pointer the server never handed out.
//
// A REPLY LARGER THAN ONE SLOT IS FATAL, NOT CHUNKED, AND THE CLIENT SAYS SO
// FIRST (ID-47). P5's only large answer is ReadPixels, and the client knows its
// size before it emits the record, so it refuses an oversize ANSWER BY NAME
// before emission (RequireReadPixelsFits: Fatal{ReplyTooLarge, "ReadPixels
// <w>x<h> <format> <bytes> > <cap>"}); Post's own refusal is the server's last
// line of defence, and reaching it means the two sides disagree about the frame
// rather than that the pool is too small. The pool still has no knob and a reply
// is still never chunked: since P7 gate 5 (g5-readback) the client splits a READ
// larger than one slot into bands, each an ordinary read_pixels record whose
// answer fits (EmitTables.h PlanReadbackBands), so the refusal is left for the
// one read no banding can answer - a single pixel larger than a slot.
//
// ORDERING. The client only looks at a slot after it has seen
// RingControl::appliedSeq >= its own seq with an ACQUIRE load, and the server
// advances appliedSeq with a RELEASE store AFTER posting the reply
// (PipeApplier::ApplyOne's order: decode -> stamp -> apply -> post -> advance).
// That pair is what publishes the slot's bytes; the fences below are the
// belt-and-braces for a caller - a unit test, or P9's async pool - that reads a
// slot without going through appliedSeq first.
// SEG_REPLY: the slot pool the server writes a kReplySlot answer into and the client reads back.
// The slot id is the record's 1-based seq (0 means "no record"), addressed `seq % slotCount`; the
// server stamps the seq into the 16-byte ReplySlotHeader so a wrong-slot read is detected.
// DECLINED is a real answer, not a failure: MapPersistent's nullptr, the Bool appliers' `false`.
// A reply never exceeds one slot and is never chunked; the client bands large reads
// (ARCHITECTURE.md §13.1.3), so Post's refusal is a last line of defence. The server posts, then
// advances appliedSeq (release); the client reads a slot only after an acquire load shows
// appliedSeq >= seq. The fences in Post and Read cover readers that bypass appliedSeq.
#pragma once
@@ -84,30 +51,11 @@ namespace MobileGL::MG_Remote::Transport {
kReplyStatusError = 2,
};
// Eight slots of a 16 MiB SEG_REPLY is 2 MiB per slot, and MaxReplyBytes() is
// 2 MiB minus the 16-byte header = 2,097,136 bytes per answer (ID-47).
//
// Why eight and not sixty-four: while the verb barrier holds (R-1) the client
// blocks at every verb boundary, so the in-flight depth is exactly ONE and
// every extra slot buys nothing but a smaller maximum answer. The trade is
// the other way round - fewer slots, bigger replies - and P5's only large
// answer is a blocking ReadPixels. P9, which is what makes the pool
// asynchronous, re-chooses this geometry with real depth to size it against.
//
// WHY 2 MiB AND NOT 1. The first version of this file said "1 MiB covers a
// 512x512 RGBA8 read", and it did not: 512*512*4 is exactly 1 MiB and the
// slot header takes 16 of those bytes, so Post refused it by sixteen. Worse,
// the E2 retrace harness's snapshot is a full-surface GL_RGBA/GL_UNSIGNED_BYTE
// read of OpenRA's 640x480 surface = 1,228,800 bytes through the interposer,
// 17% over the old cap - Post would have aborted on the first snapshot the
// day the client's ReadPixels emitter landed. Contract §2 row 23 says the
// slot is "sized from the scenario's largest read rather than guessed";
// 2 MiB is that size for every P5 exit-gate read (E2 is the largest at
// 640x480). A 2400x1080 RGBA8 device surface is ~10.4 MB and is a P6 debt
// (chunked readback or a dedicated readback carrier), recorded in the
// ROADMAP by the integrator - the geometry here is not the place it is paid.
// It was paid at P7 gate 5 by client-side banding (g5-readback), with this
// geometry unchanged.
// Eight slots of a 16 MiB SEG_REPLY: 2 MiB per slot, so MaxReplyBytes() is 2 MiB minus the
// 16-byte header (2,097,136 bytes). Few large slots, because the client never defers reading
// an answer from this pool (ShmLink does not retain replies), so extra slots would only shrink
// the largest answer. 2 MiB holds a full 640x480 RGBA8 read (1,228,800 bytes) in one answer;
// larger reads are split into bands by the client (PlanReadbackBands).
inline constexpr std::uint32_t kDefaultReplySlotCount = 8;
// Slot 0 exists and is used: seq is 1-based, so seq % slotCount hits slot 0
+26 -149
View File
@@ -6,90 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// SEG_CMD / SEG_STAGE ring control and the SPSC producer/consumer over it.
//
// RingControl is the shared page at the head of SEG_CMD, laid out exactly as
// the inherited transport design (plan section 8.1, referring the earlier
// plan's section 6.2) specifies:
//
// - TWO independent cursor triples, one for SEG_CMD and one for SEG_STAGE.
// The stage ring needs its own because "SEG_STAGE has less than a quarter
// left" is a publish trigger and that occupancy cannot be derived from the
// command ring's cursors, and because a stage slot retires on a different
// event than a command record does.
// - THREE separate sequence watermarks. Conflating them is the classic bug:
// appliedSeq releases *AppliedTail, submittedSeq releases staging,
// retiredSeq / completedFrameSerial release *RetiredTail and adopted
// stores.
// - TWO tails per ring, not one. Once the server borrows a ring slot into
// the GPU timeline instead of copying it out again, that slot can only be
// recycled after completedFrameSerial; a single tail would silently
// degrade to conservative reclaim the day borrowing lands.
// - Both park flags, because the doorbell is bidirectional: without the
// server->client direction every client wait degenerates into a
// cross-process spin on one shared cache line (a whole 16.6ms frame of a
// big core, on a phone, competing with the GPU and the game's JVM).
//
// Cursors are monotonically increasing byte counts; the ring is indexed with a
// power-of-two mask. They are never reset, so a torn read can never look like
// a valid earlier position. ringGeneration is bumped after a hard drain to
// invalidate every cached offset.
//
// Record framing inside the ring is the 8-byte header below, which is the
// layout the plan's RecHeader already fixes ({u16 kind, u16 flags, u32 size},
// size including the header and a multiple of 8). The record CATALOGUE
// (Records.def / PipeCalls.def) is a separate deliverable; the ring itself
// only needs kind/flags/size, so it can carry the real records the day they
// land without changing shape.
//
// ---------------------------------------------------------------------------
// THE FIVE WATERMARKS (P5 R-9). Four of them are Transport::LinkProgress and
// are reached as `control.Progress.<name>`; submittedSeq is the fifth and stays
// at RingControl scope, on the producer's line (lk, CONTRACT-P6 §8.3).
// One sentence each, and they are a contract:
// every one of the five was declared here at P0 and written by nobody but
// InitRingControl, so until P5 there was nothing to disagree with.
//
// submittedSeq Advanced by the PRODUCER after it publishes. NOBODY
// WAITS ON IT - it is diagnostic, the answer to "how far
// ahead of the server is the client right now". It lives
// on the producer's own cache line beside cmdHead; it is
// the ONE of the five that is not in LinkProgress, and
// that is why LinkProgress can be single-writer.
// appliedSeq Advanced by the CONSUMER for EVERY SINGLE RECORD it
// applies. The client's verb barrier and every reply wait
// read it, so it is the one watermark P5 FORBIDS BATCHING:
// the sixty-four-record batching this ring was designed
// for makes a waiter block on work that already ran, or -
// far worse - resume on work that has not.
// retiredSeq Advanced by the CONSUMER once it has finished with the
// SEG_STAGE bytes a record referenced. The staging
// allocator reclaims behind it, and nothing else may.
// completedFrameSerial Advanced by the SERVER when a present completes. What
// recycling and ageing wait on; it trails appliedSeq by
// the GPU's own depth and must never be conflated with it.
// presentAckSerial Advanced by the SERVER when it returns a present credit.
// The client's present throttle waits on it; it is the
// only back-pressure that bounds latency rather than bytes.
//
// Every wait on all five is `>=`, never `==`: a waiter that tests equality
// misses the wakeup the moment a producer or consumer moves by more than one.
//
// BATCHING MAY ONLY MAKE A WATERMARK LATE. All five except appliedSeq may be
// published lazily, because a waiter that sees an old value waits longer than
// it had to and is still correct. NONE of them may ever be published EARLY: a
// watermark that reports more than was actually done turns every waiter into a
// silent use of work that has not happened, and there is no checksum anywhere
// on this ring that would catch it.
//
// kRecPad DOES NOT ADVANCE SEQ. A wrap filler is framing, not a record: it has
// no opcode, no payload meaning and no reply slot. Both sides must skip it
// BEFORE counting. If one side counts it and the other does not, the two seq
// spaces drift by one at every wrap - and because seq IS the reply-slot id
// (P5 R-3), a drifted seq silently reads another call's answer rather than
// failing. Nothing on this ring would detect that, which is why the rule is
// stated here rather than left to each side's loop.
// ---------------------------------------------------------------------------
// RingControl, the shared page at the head of SEG_CMD, and the SPSC producer/consumer over it
// (ARCHITECTURE.md §11.2). Cursors are monotonic byte counts indexed with a power-of-two mask and
// never reset, so a torn read cannot look like an earlier position; ringGeneration is bumped after
// a hard drain. Each ring has two tails because a kRecBorrowSlot record retires only after
// completedFrameSerial; both park flags exist because the doorbell rings in both directions.
// The five watermarks (ARCHITECTURE.md §11.2.1, SessionRings.h's Watermark) are waited on with
// `>=` and may be published late, never early; appliedSeq advances once per applied record.
// kRecPad wrap fillers do not advance seq, the reply-slot id: both sides skip them before counting.
#pragma once
@@ -119,28 +43,11 @@ namespace MobileGL::MG_Remote::Transport {
// ---- SEG_STAGE cursors ------------------------------------------------
//
// DEAD IN P5, DELIBERATELY, AND NOBODY MAY WIRE THEM UP HALFWAY.
//
// SEG_STAGE is NOT a ring any more. Package w1's encoder owns staging as
// an ENCODER-LOCAL LINEAR ALLOCATOR: a staged byte run carries no
// RingRecordHeader, there is no consumer walking SEG_STAGE, and the
// allocator reclaims on `retiredSeq` - the sequence watermark below -
// rather than on these three cursors. So all three stay ZERO for the
// whole of P5, `RingCursorSet::Stage` has no producer and no consumer,
// and `SessionTest.TheStageCursorTripleStaysDeadAcrossAWholeSession`
// pins that rather than leaving it to be noticed.
//
// They are kept rather than deleted because RingCursorSet, the three
// cursor accessors in Ring.cpp and RingTest's fixture are all written
// against a two-triple page, and P8/P11's shadow and adopt segments are
// the ring-shaped users this triple was reserved for. What is NOT
// acceptable is the middle state: a producer publishing `stageHead` with
// nothing advancing the two tails makes FreeBytes() fall to zero the
// first time the head laps the capacity and never recover, which is a
// guaranteed hang rather than a slow path. Five watermarks already spent
// a whole phase declared-and-written-by-nobody; this is the sixth, and
// it is declared-and-written-by-nobody ON PURPOSE, which is only
// different if it is written down.
// Unused: staging is the encoder's linear allocator, which reclaims on retiredSeq, so these
// stay zero and RingCursorSet::Stage has no producer or consumer (pinned by
// SessionTest.TheStageCursorTripleStaysDeadAcrossAWholeSession). They keep the page layout
// for a future ring-shaped segment. Never publish stageHead without a consumer advancing
// both tails: FreeBytes() would reach zero after the first lap and the producer would hang.
alignas(64) std::atomic<std::uint64_t> stageHead;
alignas(64) std::atomic<std::uint64_t> stageAppliedTail;
std::atomic<std::uint64_t> stageRetiredTail;
@@ -264,33 +171,12 @@ namespace MobileGL::MG_Remote::Transport {
};
static_assert(sizeof(RingRecordHeader) == 8, "RecHeader is 8 bytes on the wire");
// THESE ARE RING FLAGS AND THEY ARE NOT MGPipeCallFlags, AND THREE OF THE
// BITS COLLIDE WITH A DIFFERENT MEANING. `MGPipeCallFlags` (MG_Pipe/MGPipe.h:
// 42-54) is a SEPARATE SPACE that happens to overlap this one, and an encoder
// that copies `MGPipeCallFlagsFor(op)` into RingRecordHeader::flags without
// translating puts a call's bits into a framing field:
//
// bit 0 kNeedsAck == kRecNeedsAck same meaning, harmless
// bit 1 kHasBlob == kRecHasBlob same meaning, harmless
// bit 2 kVarTail == kRecPad WORST: a var-tail record would read
// as a WRAP FILLER and be skipped
// silently by Pop, losing the record
// with nothing logged anywhere
// bit 3 kHostSpan == kRecBorrowSlot a host-span record would read as
// borrowed into the GPU timeline, and
// the consumer would stop reclaiming
// ring bytes behind it for ever
// bit 4 kReplySlot == kRecVarTail a blocking call would read as having
// a tail it does not have
// bit 5 kOptional == (unused here)
//
// Translating is the ENCODER's job. Two things on this side make the first
// two of those survivable anyway rather than trusting it: `Pop` requires a
// filler to carry BOTH kRecPad AND kind == kRingPadRecordKind, so a real
// record with bit 2 set is delivered rather than eaten (a call record always
// has a real opcode kind, the catalogue starts at 1); and SessionConsumer
// counts and NAMES every kRecBorrowSlot it sees, because P5 produces no
// borrowed slots at all and the bit arriving means the collision did.
// Ring framing flags. MGPipeCallFlags (MG_Pipe/MGPipe.h) is a separate space in the same bits:
// kVarTail == kRecPad, kHostSpan == kRecBorrowSlot, kReplySlot == kRecVarTail. The encoder
// translates call flags and never copies them (PipeWireCodec.cpp's FlagSpace asserts). As a
// backstop, Pop skips a record as a wrap filler only if it carries kRecPad and
// kind == kRingPadRecordKind, and SessionConsumer names every kRecBorrowSlot it sees, since
// nothing produces borrowed slots.
enum RingRecordFlags : std::uint16_t {
kRecNone = 0,
kRecNeedsAck = 1u << 0,
@@ -376,23 +262,14 @@ namespace MobileGL::MG_Remote::Transport {
// Bytes still writable before the consumer has to catch up.
std::uint64_t FreeBytes() const;
// Reserves room for one record and returns a pointer to its payload,
// or nullptr when the ring is full. The payload is uninitialized;
// alignment padding at its tail is NOT zeroed. Emits a pad record
// automatically when the record would straddle the wrap boundary, so
// every record is contiguous.
// Reserves room for one record and returns a pointer to its payload, or nullptr when the
// ring is full. The payload and its tail padding are uninitialized. A record that would
// straddle the wrap boundary is preceded by a pad record, so every record is contiguous.
//
// A record whose total (header + payload, rounded up to 8) exceeds
// MaxRecordBytes() == Capacity()/2 is refused outright, with an error
// log and however empty the ring is: chunking it is the emitter's job
// (plan section 8.2, the G3 chunking rule). Half is exact, not
// conservative - it is the largest record EVERY head offset can place,
// because a wrap pad costs at most total-8 bytes on top of the record
// and 2*total-8 <= capacity-8 holds exactly up to capacity/2. Above it
// a record is placeable at some offsets and not at others, and a
// producer waiting for FreeBytes() >= total stalls forever on an empty
// ring. So: nullptr with FreeBytes() >= total never means "wait"; it
// can only mean "too big, chunk".
// A record whose total (header + payload, rounded up to 8) exceeds MaxRecordBytes() ==
// Capacity()/2 is refused with an error log however empty the ring is; the emitter chunks
// it. Half is the largest total every head offset can place (a wrap pad adds at most
// total-8 bytes), so nullptr with FreeBytes() >= total means "too big", never "wait".
void* Reserve(std::uint16_t kind, std::uint16_t flags, std::uint64_t payloadBytes);
// The largest header+payload total Reserve accepts: Capacity()/2. This
+7 -25
View File
@@ -6,31 +6,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// Peak-RSS accounting for the two roles. Owner: package s1; CONSUMER: package t1,
// which puts the numbers in MEASUREMENTS.
//
// WHY BOTH HALVES ARE NEEDED, AND WHY NEITHER ALONE IS THE ANSWER.
//
// VmHWM is the kernel's own high-water mark of resident set size, in
// /proc/self/status. It is the only number that cannot be argued with - it
// counts what the process actually touched, including the pages the allocator
// never gave back. But under `inproc` BOTH ROLES ARE ONE PROCESS, so a single
// VmHWM cannot be split between them and reporting it as "the client's" would
// be a lie that only becomes visible in P6.
//
// The segment ledger is the other half: every ShmSegment this process mapped,
// by kind and by role. It is exact, it IS separable by role, and under `spawn`
// it is the part that appears in both processes at once (one mapping, two
// address spaces, one set of physical pages) - which is precisely the number a
// naive "sum the two VmHWMs" double-counts.
//
// So the pair is the measurement: VmHWM for what the process really cost, the
// ledger for how much of it is shared mapping that a second process will not pay
// for again. t1 reports both, per role, and the split's memory claim is
// (client VmHWM + server VmHWM - shared ledger), never either half on its own.
//
// A SAMPLE IS A SYSCALL AND A PARSE. Take it at phase boundaries - after the
// handshake, after the first frame, at teardown - never per record.
// Peak-RSS accounting for the two roles, logged per role at phase boundaries. It needs two
// halves: VmHWM (/proc/self/status) is what the process really touched, but under inproc both
// roles are one process and it cannot be split between them; the segment ledger is exact and
// per role, and under spawn its pages are mapped in both processes at once. The split's memory
// cost is client VmHWM + server VmHWM - the shared ledger, never either half alone.
// A sample is a syscall and a parse: take it after the handshake, after the first frame and at
// teardown, never per record.
#pragma once
+26 -112
View File
@@ -6,61 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The ring-owning half of a session: the four segments, the two ring endpoints,
// and the five watermarks. Owner: package s1.
//
// This is the object the P5 gate is really about. ARCHITECTURE.md §10.1 asks that
// `inproc` run THE SAME G3 codec as `spawn`, and InProcessTransport cannot
// deliver that no matter how it is edited: it is two deque<vector<uint8_t>> plus
// two condvar doorbells (InProcessTransport.cpp:38-97), it owns THE BELLS BUT
// NOT THE RING, and it runs no codec at all. So the rings live here, above
// ITransport, in one implementation both delivery modes use - and the transport
// supplies the control plane and the two bells, which is exactly what its own
// header says it is for (ITransport.h:16-20: "everything on the hot path
// bypasses this interface entirely").
//
// INPROC USES ShmSegment TOO, NOT new[]. In one address space a heap allocation
// would work and would be faster to write. It is refused deliberately: it is half
// of what makes "the same code path" true rather than nominal. A `new` here means
// the mapping, the alignment, the size rounding, the read-only peer view and the
// lifetime are all exercised for the first time in P6, on the day the second
// process appears - which is the shape of every "it was green in CI" failure this
// phase is trying not to repeat.
//
// ---------------------------------------------------------------------------
// THE KNOB NAMES THE RING; THE SEGMENT IS THE RING PLUS ONE CONTROL PAGE.
//
// Ring.h:11-13 puts RingControl at the HEAD of SEG_CMD and RingProducer requires
// a POWER-OF-TWO capacity (Ring.cpp:89-103 - the mask IS the indexing). Those two
// facts together mean a segment and its ring cannot both be 8 MiB, and one of the
// two numbers has to give.
//
// The one that gives is the SEGMENT: SEG_CMD is `MOBILEGL_IPC_RING_MB` MiB PLUS
// 4096, so the ring inside it is exactly MOBILEGL_IPC_RING_MB MiB and
// RingProducer::MaxRecordBytes() is exactly half of that. CONTRACT-P5 §5 and
// Config.h's MOBILEGL_IPC_RING_MB comment - "A RECORD MAY BE AT MOST HALF OF
// THIS, so 8 MiB caps one record at 4 MiB" - are then TRUE AS WRITTEN, which
// matters because that sentence is what every other package sizes against.
//
// The first version of this file did the opposite: an 8 MiB segment with a 4 MiB
// ring and a 2 MiB record cap, on the grounds that ProtocolSmokeTest.cpp:72 pinned
// the four announced sizes. That was wrong on the facts - that test builds four
// SegmentRefs from its own literals and round-trips them through the schema; it
// says nothing about what a session announces, and it never mentions
// SessionSegments at all. So the alternative was available at no cost, and the
// version that made two live documents false and left half of SEG_CMD mapped and
// unreachable was the worse of the two.
//
// SegmentRef.sizeBytes therefore announces the MAPPING size (ring + page), which
// is what a spawn peer must mmap. The four numbers a reader recognises - 8 MiB /
// 32 MiB / 16 MiB / 256 KiB (ID-47) - are the RING sizes, which is what the
// knobs name.
//
// SEG_STAGE has no control page of its own: RingControl carries TWO cursor
// triples (Ring.h:101-109) and the stage triple is the second. So SEG_STAGE is
// exactly its ring, and 32 MiB is already a power of two. SEG_REPLY is not a ring
// at all.
// ---------------------------------------------------------------------------
// The ring-owning half of a session: the four segments, the two ring endpoints, the five
// watermarks. The rings live above ITransport (control plane only), so every delivery mode runs
// the same ring and codec code, and `inproc` maps real ShmSegments rather than heap memory so the
// mapping, sizing and peer-view paths are exercised on every run.
// Sizes name the ring: SEG_CMD and SEG_EVENT are a power-of-two ring plus one RingControl page at
// the head, so MOBILEGL_IPC_RING_MB (rounded down to a power of two) is the command ring itself
// and MaxRecordBytes() is half of it; SegmentRef.sizeBytes announces the whole mapping. SEG_STAGE
// (a linear allocator) and SEG_REPLY (a slot pool) have no control page.
#pragma once
@@ -142,24 +95,12 @@ namespace MobileGL::MG_Remote::Transport {
// Private, identically addressed mirrors for the stream data plane.
MobileGLResult CreatePrivate(const SessionSegmentSizes& sizes, MemoryRole role);
// The inproc peer's view of the owner's four segments, booked under the
// OTHER role. It does NOT re-init the control pages - there is one shared
// page per ring and re-initialising it would zero the owner's cursors out
// from under whoever is already using them.
//
// ON POSIX THIS IS A REAL SECOND MAPPING, NOT AN ALIAS: each descriptor is
// dup()ed and adopted through ShmSegment::Adopt + Map, so the peer gets
// its own virtual addresses over the same memfd. That is the same reason
// inproc uses ShmSegment at all (see the header block): the ATTACH half is
// the half P6 replaces with an SCM_RIGHTS Adopt, and aliasing the owner's
// ShmSegment objects would leave it first exercised on the day the second
// process appears - which is exactly the criticism this file levels at
// new[]. It also removes a raw lifetime coupling: an aliased view holds
// pointers into the owner's members with no ownership, so the two Closes
// have to be ordered by hand.
//
// Windows has no Adopt (ShmSegment::Adopt is POSIX-only; the section name
// travels in SegmentRef instead), so there it still aliases and says so.
// The inproc peer's view of the owner's four segments, booked under the other role. It
// does not re-initialise the control pages: they are shared, and that would zero the
// owner's cursors. On POSIX each descriptor is dup()ed and adopted (ShmSegment::Adopt +
// Map), a real second mapping, so the inproc attach runs the same path as the cross-process
// one (AdoptFromDescriptors) and the two views close independently. Windows has no Adopt
// (the section name travels in SegmentRef), so there the view aliases the owner's.
MobileGLResult AttachInProcess(SessionSegments& owner, MemoryRole role);
// P6 `sm`: the spawn client's half. The four descriptors arrived over
@@ -239,26 +180,13 @@ namespace MobileGL::MG_Remote::Transport {
};
// -----------------------------------------------------------------------
// The five watermarks (R-9). Every write and every wait goes through here,
// so the rules in Ring.h's header have exactly one implementation.
// The five watermarks. Every write and every wait goes through here, so the rules in
// Ring.h's header have exactly one implementation.
// -----------------------------------------------------------------------
//
// THE ONE RULE THAT MATTERS: a watermark may be published LATE but NEVER
// EARLY. Late costs a waiter some latency; early makes every waiter a silent
// use of work that has not happened, and there is no checksum anywhere on
// this ring that would catch it. The callers are responsible for never
// calling an advance before the work is done - that is the half only a
// call-site review and R-9's unit cases can enforce.
//
// THE HALF THAT IS MECHANICALLY DETECTABLE - a watermark moving BACKWARDS -
// IS FATAL, not logged-and-ignored. Logging it and returning was the first
// version of this file and it was worse than useless: SessionConsumer keeps
// its own counter, so once the shared appliedSeq is behind, every later
// advance is a no-op for ever and every WaitForApplied(seq, kWaitForever) -
// the verb barrier and every reply wait - blocks permanently. The user sees a
// hang and the only evidence is one ERROR line. This is the same class as
// RingCursorsValid returning false, and Ring.h:181-185 already calls that "a
// Fatal{ProtocolCorruption}, never a retry".
// A watermark may be published late but never early: early lets waiters use work that has not
// happened, and nothing on the ring would catch it, so callers advance only after the work is
// done. A watermark moving backwards is Fatal{ProtocolCorruption}: a stale shared appliedSeq
// would make every later advance a no-op and every WaitForApplied(kWaitForever) hang.
namespace Watermark {
// Producer, after Publish. Nobody waits on it - it is the answer to "how
@@ -379,26 +307,12 @@ namespace MobileGL::MG_Remote::Transport {
Doorbell* SelfDoorbell() const { return m_selfBell; }
std::uint32_t SpinUs() const { return m_spinUs; }
// THE WAIT LEDGER'S CLIENT HALF (P5d round 3, package T item 4), and it is a PAIR
// because one number without the other says nothing. `Waits()` is every entry into
// Park - the R-1 barrier's wait, the present throttle and the ring-space wait, which
// simpleperf at head 56a77348 measured as WaitForApplied 27.9% self on the GL thread;
// `Parks()` is how many of them ran out of spin budget and actually blocked. A high
// wait count with near-zero parks is a thread spinning on a server that is nearly
// there; a high park count is a server that is not. The two readings are what tell
// those apart, and the split's whole frame budget turns on which one it is.
//
// Both counters are OURS, and the park one deliberately is not the bell's own
// ParkEntries(). Doorbell::Wait is the only code that knows whether its spin budget
// ran out, so it does the counting - but a bell belongs to an ENDPOINT, not to a wait
// path, and this one is shared: ClientSession hands the very same object to the
// encoder as its SEG_STAGE retirement bell (ClientSession::Start ->
// PipeWireCodec::SetStageRetirementDoorbell), whose waits never pass through
// SessionProducer::Park and so never reach m_waits. Reading ParkEntries() here would
// fold that second subsystem's parks into a number this header, PipeStats and the
// summary line all describe as the subset of OUR waits - and with the retirement
// wait's spinUs of 0 it parks every time, so the subset could print larger than the
// set it is a subset of. Wait's `parkTally` out-parameter exists for exactly this.
// The client half of the wait ledger. Waits() counts every entry into Park (the verb
// barrier, the present throttle, the ring-space wait); Parks() counts those that ran out of
// spin budget and really blocked. Many waits with few parks means the server nearly keeps
// up; many parks means it does not. Parks is fed through Wait's `parkTally`, not read from
// the self bell's ParkEntries(): that bell also carries the encoder's SEG_STAGE retirement
// wait (spinUs 0, so it always parks), which would let parks exceed waits.
std::uint64_t Waits() const { return m_waits.load(std::memory_order_relaxed); }
std::uint64_t Parks() const { return m_parks.load(std::memory_order_relaxed); }
+8 -19
View File
@@ -6,25 +6,14 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// One shared-memory segment: SEG_CMD, SEG_STAGE, SEG_REPLY, SEG_EVENT, a
// per-object SEG_SHADOW or a SEG_ADOPT store (inherited segment layout, plan
// section 8.1).
//
// Creation matrix (earlier plan section 6.1):
// - Android: ASharedMemory_create (API 26; libc's memfd_create wrapper
// only appears at API 30, which is above our floor)
// - desktop Linux: syscall(SYS_memfd_create, ...) directly, for the same
// reason - the glibc wrapper is recent and this file has to
// build against old sysroots
// - other POSIX: shm_open + immediate shm_unlink, the fd keeps it alive
// - Windows: CreateFileMappingW in the Local\ namespace
//
// Transfer is NOT done here. On POSIX the fd travels by SCM_RIGHTS
// (FdPassing.h / ITransport::ShareFd) and the name is only a debugging label;
// on Windows the section name travels inside the SegmentRef table.
//
// The Windows implementation is compile-guarded and untested at the time it
// was written: no Windows machine is a correctness gate for this project.
// One shared-memory segment: SEG_CMD, SEG_STAGE, SEG_REPLY, SEG_EVENT, a per-object SEG_SHADOW
// or a SEG_ADOPT store (ARCHITECTURE.md §11.1). Created with ASharedMemory_create on Android
// (libc's memfd_create wrapper needs API 30, above the API 26 floor), syscall(SYS_memfd_create)
// on desktop Linux (old sysroots lack the glibc wrapper), shm_open plus immediate shm_unlink on
// other POSIX, and CreateFileMappingW in Local\ on Windows (compile-guarded, untested: Windows
// is not a correctness gate). Transfer happens elsewhere: on POSIX the fd travels by SCM_RIGHTS
// (FdPassing.h, ITransport::ShareFd) and the name is a debug label; on Windows the section name
// travels in the SegmentRef table.
#pragma once
@@ -233,27 +233,12 @@ namespace MobileGL::MG_Remote::Transport {
} // namespace
namespace {
// A filesystem AF_UNIX path, bounded by sun_path. Refused BY NAME when it
// does not fit rather than silently truncated - a truncated path is a
// different socket, and both sides would then "succeed" onto two
// different names.
// A LEADING '@' MEANS THE ABSTRACT NAMESPACE, which is the convention systemd and D-Bus
// use and the only rendezvous that works on Android.
//
// A filesystem AF_UNIX endpoint needs a writable directory, and an Android app has no
// /tmp - measured, not assumed: the first device run of the spawn arm launched its server
// fine and then died on `could not connect to "/tmp/mgl-15731-...sock" ... No such file or
// directory`, because bind() in the child had nowhere to put the node. The app's own cache
// dir would work, but only if every integrator remembered to say where it is, and the
// library has no way to ask.
//
// An abstract name has no filesystem presence at all: no directory, no mode bits, no stale
// node to unlink after a crash - which is the bind() failure `Listen` currently works
// around. It is scoped to the network namespace, so the two processes of ONE app find each
// other and nothing else can, which is exactly the reach this needs.
//
// Linux-only by construction. A path without '@' still works everywhere, and that is what
// an explicit endpoint (P12's "connect to something somebody else is listening on") uses.
// An AF_UNIX endpoint, bounded by sun_path: a path that does not fit is refused by name,
// never truncated (a truncated path is a different socket). A leading '@' selects the
// abstract namespace (Linux and Android only): no filesystem node, so no writable directory
// is needed (an Android app has no /tmp) and no stale node survives a crash; the name is
// scoped to the network namespace instead of a directory. A path without '@' is a
// filesystem socket and works everywhere.
bool IsAbstractEndpoint(const std::string& path) {
return !path.empty() && path[0] == '@';
}
+13 -65
View File
@@ -6,41 +6,12 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// ITransport over a connected stream socket. Package `so` (CONTRACT-P6.md).
//
// WHY THIS IS THE FIRST P6 CODE THAT RUNS. It is the whole control plane for a
// second process, and it needs NOTHING from the second process to be correct:
// CONTRACT-P6 and the spawn plan both require it green over a plain
// socketpair() with two threads, running the WHOLE InProcessTransportTest
// suite, BEFORE a child is ever forked. Same discipline as SessionRings.h's
// "the attach half is the half P6 replaces, so exercise it now" - a transport
// first exercised on the day the process appears is a transport whose bugs all
// arrive at once.
//
// WHAT IT IS NOT. It is not the data plane. The hot path bypasses ITransport
// entirely (ITransport.h) and crosses through ILink, which is a separate seam
// with a separate implementation. This class carries the handshake, surface
// ops, resync, aux requests and fatals - the rare, variable-length,
// must-evolve traffic.
//
// THREE THINGS IT DOES NOT REINVENT, because they are already in the tree and
// already tested:
// Framing.h - AppendFrame writes [u32 'MGLF'][u32 len][payload]; FrameReader
// is a real byte-stream reassembler with the BUFFER_TOO_SMALL
// keep-the-message semantics ITransport requires. A socket
// hands you arbitrary fragments, which is precisely what
// FrameReader was written for and what InProcessTransport's
// message queue never needed.
// FdPassing.h - SCM_RIGHTS over a DEDICATED SOCK_DGRAM aux socket, so an fd
// offer is one datagram and cannot be half-consumed.
// Doorbell.h - SocketDoorbell already exists, is unit-tested, and has NO
// production construction site until now. It was written for
// this and is priced as new work here, not as already paid for.
//
// LIFETIME. The transport owns its stream fd and its aux fd and closes both in
// Shutdown/dtor. Shutdown is idempotent and tears down BOTH directions, which
// is what ITransport promises and what a socket does anyway: after either end
// calls it, the peer's reads return 0 and its sends fail.
// ITransport over a connected stream socket: the control plane (handshake, surface ops, resync,
// aux requests, fatals). The data plane bypasses ITransport and crosses through ILink.
// Framing.h's FrameReader reassembles the arbitrary fragments a socket delivers; descriptors
// travel by SCM_RIGHTS on a separate aux socket (FdPassing.h), never on the framed stream.
// The transport owns its stream and aux fds and closes both in Shutdown and the destructor.
// Shutdown is idempotent and closes both directions: the peer's reads return 0, its sends fail.
#pragma once
#include <atomic>
@@ -66,36 +37,13 @@ namespace MobileGL::MG_Remote::Transport {
std::unique_ptr<SocketTransport>& outServer);
// ---- independent processes: listen / connect -----------------------
//
// THE TWO PROCESSES ARE NOT COUPLED BY INHERITANCE. An earlier shape of
// this package forked the server and handed it fds 3..6; that can never
// be the end state, where the server is a standing application and the
// client connects to it from somewhere else entirely - possibly from
// another kernel. So the rendezvous is a NAME, and both sides are
// started by whoever starts them.
//
// TWO CONNECTIONS, not one. SCM_RIGHTS needs a socket of its own: an fd
// offer is a sendmsg whose ancillary data rides with specific BYTES, and
// interleaving those bytes with the framed control stream would make the
// frame reassembler and the descriptor receiver race for the same bytes.
// So the client connects twice to the same listening path - control, then
// aux - and each connection's first frame is a PairBind carrying one nonce
// and which half it is. One name, no extra configuration, and FdPassing
// keeps the dedicated socket it was written for.
//
// P11 PAIR: THE SERVER PAIRS THEM BY THAT NONCE, NOT BY ARRIVAL ORDER
// (Server/PairAcceptor.h). "First is control, second is aux" was a protocol
// only while nothing else could reach the listener: a readiness probe's
// lone connect() + close() became the next client's control connection, and
// two clients connecting at once could swap halves (B0-CROSS-APP.md F2).
//
// `path` is a filesystem AF_UNIX path. ARCHITECTURE.md §15.1 ruled
// against one for the FORK shape, where the fds were inherited and a
// path would have been a strictly larger attack surface for no gain.
// Independent processes have to name a rendezvous somehow, and a path
// with 0600 permissions in a private directory is the portable answer;
// the abstract namespace is Linux-only and TCP would be reachable off
// the machine.
// The server is a standing process reached by name; neither side inherits the other's
// fds. A Unix endpoint uses two connections, control then aux, because SCM_RIGHTS
// ancillary data must not interleave with the framed control stream. Each presents a
// PairBind (one shared nonce, which half) as its first frame, and the server pairs them by
// that nonce, never by arrival order (Server/PairAcceptor.h), so a probe's bare connect or
// two clients at once cannot mis-pair. `path` is a filesystem AF_UNIX path (bound 0600),
// '@' plus a Linux abstract name, or tcp://host:port.
static MobileGLResult Listen(const std::string& path, int* outListenFd);
// The client's half: connects twice to `path`, each connection presenting a PairBind
+8 -45
View File
@@ -115,51 +115,14 @@ namespace MobileGL::MG_Remote::Transport {
std::vector<LinkSpan> stage;
std::vector<std::uint8_t> readReply;
std::uint64_t replySeq = 0, sentCmd = 0, receivedCmd = 0, sentEvent = 0, receivedEvent = 0;
// ---- THE RECENT-REPLY RING (P12) -------------------------------------------------
//
// ILink::ReadReply takes a SEQ, and until P12 that was decoration: every caller read the
// answer it had just waited for, which is always the newest one, so a single slot answered
// every question and `replySeq != seq` could stand in for "no such answer". The create
// window is the first caller that reads an OLDER seq - it defers up to four answers and
// takes them in order - and against a single slot every one of those reads fails
// PROTOCOL_MISMATCH for ever. What that looked like on the device, measured: the window
// filled to 4 and STAYED there, `refusals=0` because no answer was ever read, every one of
// the frame's 4,536 creates fell through to the blocking path, and the wall clock did not
// move (29.0 s against the 30.1 s of the same binary with the window off). The deferral
// was pure overhead: four answers held unread, plus one failed drain attempt per record.
//
// ---- AND THE RING WAS NOT ENOUGH, SO THE STORE IS KEYED ON A DECLARATION (P12) -------
//
// THE MEASUREMENT THAT SETTLED IT. With the ring at 8 the drain read back NOTHING:
// `taken=63 pushed=65 readfail=8,872` on a load frame, and the miss probe showed why -
// `want=310` on every miss while the ring had moved on to [1175..1504]. Between two
// creates this client emits on the order of a thousand records and looks at the window
// not once, so a deferral has to survive an unbounded amount of other traffic, and WHAT
// DISPLACES IT IS NOT WHAT THE CLIENT EMITS BUT WHAT THE SERVER ANSWERS - it is behind,
// and its late answers to the EARLIER creates fill a bounded buffer first.
//
// Ring 8 -> 1024 as a one-line experiment, and every deferred answer came back:
// `taken=4,499/4,500 readfail=0`, wall 27.9 s -> 23.0 s, and with the residency budget
// out of the way 28.66 s -> 16.80 s at window 2 and 11.45 s at window 4. So the property
// needed is "the answer is still there when the reader comes back", and 1024 is a
// STAND-IN for it, not the design: 1024 retained answers is ~2 GiB worst case, and "deep
// enough" is a probability rather than a bound.
//
// THE BOUND COMES FROM THE CLIENT. A seq is retained only if the wire DECLARED it will be
// read (ILink::DeclareReplyRead), so the outstanding set is the window's depth plus the
// one blocking row in flight plus whatever a retired window strands - it does not depend
// on how many answers the server chooses to send. Retaining on ARRIVAL instead would grow
// ~11.3 entries per create against a server that ignores the no-reply bit (55,903 answers
// a frame where this client reads 4,501), and any cap would then be reached and would
// evict precisely the oldest WANTED answer - the same wedge, a few hundred creates later.
//
// OVERFLOW IS FATAL AND NEVER AN EVICTION. An evicted wanted answer is unrecoverable:
// sending another create to re-latch the object wipes the server's record for it
// (PipeApply.cpp:2033) along with its PendingUploads (PipeApply.h:332), and the client
// cleared its dirty flags at EMISSION (PipeApply.h:298-304), so nobody re-sends those
// texels. The capacity is sized so that cannot be reached, and reaching it is a bug that
// says its own name.
// ONE DEFINITION, IN THE HEADER, because the client's window is bounded by this number.
// Declared-answer store. ReadReply takes a seq, and the create window reads answers older
// than the newest, after on the order of a thousand other records, so a ring of recent
// answers loses them. An answer is retained only if the wire declared it will be read
// (ILink::DeclareReplyRead), so the client, not the server's answer count, bounds the
// store. Overflow is a Fatal, never an eviction: a lost wanted answer is unrecoverable,
// since a re-sent create wipes the server's pending uploads, which nobody re-sends.
// Measurements: docs/Performance/ipc-create-window.md. The cap is defined in the header
// because the client's window is bounded by it.
static constexpr std::size_t kRetainedCap = StreamLink::kRetainedAnswersMax;
struct StoredReply {
std::uint64_t seq = 0;
+5 -16
View File
@@ -6,22 +6,11 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// A one-function logging shim for the wire layer's header-only code.
//
// MG_Util/Debug/Log.h includes <Includes.h>, the GL frontend's umbrella
// header - 661 headers, measured with `clang++ -H`. That is fine inside a
// .cpp, and Ring.cpp / Doorbell.cpp / the transports all do it. It is not fine
// in a header of this layer: ITransport.h states the rule ("nothing about a
// byte pipe needs the GL frontend's umbrella header") because these headers
// are included by both roles and by the eventual server-side binary, and
// because the disaggregated build's include-graph purity gate (plan section
// 10.3, gate A) asserts on `-H` output rather than on symbols. Framing.h was
// the one header under Transport/ that broke the rule; it now calls this
// instead, and the umbrella stays inside WireLog.cpp.
//
// ERROR and FATAL only, deliberately. Everything routed here is a latched
// protocol violation, never per-frame noise; non-critical wire lines use
// MGLOG_D from a .cpp, where the INFO build compiles them out entirely.
// A logging shim for the wire layer's header-only code. MG_Util/Debug/Log.h includes the GL
// frontend's umbrella header, which headers of this layer must not (ITransport.h's rule; the
// include-graph purity gate checks `clang++ -H` output), so the umbrella stays in WireLog.cpp.
// ERROR and FATAL only: everything routed here is a latched protocol violation, never per-frame
// noise; other wire lines use MGLOG_D from a .cpp, which the INFO build compiles out.
#pragma once