swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-11-15 14:03:23 +09:00
Code Issues Packages Projects Releases Wiki Activity

[MG_Remote, MG_Impl] (P6/dl): the device-lost latch, taken from a descriptor and never from a deadline

Browse Source 
Before this, CONTRACT-P6 5.3's own sentence held: "a dead server leaves the
client silently DECLINE-ing every verb forever with glGetError clean".
FatalCode::DeviceLost had zero producers and zero consumers and
glGetGraphicsResetStatus returned GL_NO_ERROR unconditionally, so an
application had no way to learn that the process holding its GPU objects was
gone. Measured cost of that: a server killed on its first Clear left the
client waiting out the full 120 s barrier and reporting Fatal{BarrierTimeout}
- the wrong diagnosis, two minutes late. It now notices in 0.01 s.

THE MACHINERY WAS ALREADY THERE. Doorbell::Wait breaks out on Dead(),
SessionProducer::Park turns that into SessionWait::ShutDown, and ClientSession
already had the named DECLINED path. What did not exist was the death FACT:
D5c's finding is that the client parks on clientBell[0] and rings ITSELF
through clientBell[1], holding both ends, so EOF can never arrive on the one
descriptor it watches. A bell that can wake itself cannot hear a death.

SetDeathWitness(fd) fixes that with no new descriptor: the control socket's
far end is held only by the server. The subtlety is that the bell must watch a
socket it must NEVER read - the transport's reassembler owns those bytes and a
control reply is not a death. So the witness enters the poll set with
`events = POLLRDHUP` and nothing else: POLLHUP/POLLERR/POLLNVAL arrive in
revents unrequested, so the hangup is seen while POLLIN never is. Nothing is
recv'd from it and its ownership stays with SocketTransport.

Dead() AND PeerHungUp() ARE TWO FACTS. Park sets m_dead for POLLERR, POLLNVAL
and unrecognised revents too - each a fault in THIS process's descriptor - and
CondVarDoorbell::Kill() sets it on every orderly Stop(). A latch taken from
Dead() would report a lost GPU for a local fd bug and a context reset for
every clean exit, so only PeerHungUp() arms it.

GL_UNKNOWN_CONTEXT_RESET, not GUILTY or INNOCENT: those assign blame, and the
server died for a reason that never crossed the wire.

D5b, in the control-reply wait, which had the right comment and no code: EOF
on the control stream is device loss; a TIMEOUT with the socket still open is
"ALIVE BUT SILENT" and says so by name. 5.4 forbids calling a frozen or merely
slow peer dead, and under P5e run-ahead a server one frame behind is the
INTENDED steady state - a latch armed by a deadline would fire on a healthy
session under load.

MOBILEGL_IPC_RESPAWN is now a NAMED REFUSAL rather than a variable nothing
reads. The latch is one-way on purpose: every handle this client minted names
an object inside the server process, so recovery means re-pushing the world.

THREE TEST NAMES ADDED, none renamed (G14), and R-16 run both ways:
  * remove the witness -> AKilledServerLatchesDeviceLostFromTheHangupAndNot-
    ADeadline goes red, takes 2 s instead of 0.01 s, and names D5c.
  * collapse PeerHungUp() into Dead() -> ADeadBellIsNotAlwaysAHungUpPeer goes
    red naming the local-fd-bug consequence.
  * AnOrderlyStopIsNotADeviceLoss stays GREEN under that collapse, and its
    comment now says so. The first draft claimed it was the control for it;
    running the falsification proved otherwise - under spawn the self bell is a
    SocketDoorbell and Stop() never kills one. The comment was corrected rather
    than the claim quietly kept.

G1 RE-MEASURED, NOT ASSUMED: 0 symbol diff against the c6 baseline and .text
unchanged at 0xa52203. That measurement earned its keep - it caught a 16-byte
drift that had nothing to do with dl: P6's O_APPEND log-sink fix was compiling
into the PULL build, where a second process can never exist. Now guarded, and
.text is back to the baseline exactly.

Full unit suite: 2350 tests, the same two pre-existing environmental failures
and no others. All six OpenRA arms (2 backends x monolith/inproc/spawn) green.
This commit is contained in:
swung0x48
2026-09-21 18:49:56 -04:00
parent d83fc6643a
commit 8b13d8a84c
9 changed files with 405 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MobileGL/MG_Remote/Client/ClientSession.cpp
+76 -1
View File
@@ -466,6 +466,27 @@ namespace MobileGL::MG_Remote::Client {
// monolith answer. Every call that PRESUMES a session aborts instead.
ClientSession* ClientSession::Active() { return g_active; }
Bool ClientSession::DeviceLost() {
const ClientSession* session = Active();
return session != nullptr && session->m_deviceLost.load(std::memory_order_acquire);
}
void ClientSession::LatchDeviceLost(const char* why) {
// exchange, so the log line is written EXACTLY once however many waits notice the same
// hangup at the same moment - and there will be several: the barrier, the control wait
// and the event pump can all be in flight when the server goes.
if (m_deviceLost.exchange(true, std::memory_order_acq_rel)) {
return;
}
MGLOG_E("MGPipe: DEVICE LOST - %s. The server process is gone; every verb from here is "
"DECLINED and glGetGraphicsResetStatus reports GL_UNKNOWN_CONTEXT_RESET. This is "
"latched from the peer's HANGUP and not from any deadline (CONTRACT-P6 5.4): a "
"server one frame behind is P5e's intended steady state and must never read as a "
"dead one. There is no recovery - MOBILEGL_IPC_RESPAWN is the stage that would "
"add one and it is refused by name until then.",
why == nullptr ? "the peer hung up" : why);
}
ClientSession& ClientSessionInstance() {
// Leak at exit, deliberately and per ID-8, exactly as ServerSessionInstance does.
static ClientSession* instance = new ClientSession{};
@@ -835,7 +856,20 @@ namespace MobileGL::MG_Remote::Client {
// fds[5] our park end
// fds[6] ring OURSELVES
m_socketPeerBell = MakeUnique<Transport::SocketDoorbell>(-1, fds[4], 1, true);
m_socketSelfBell = MakeUnique<Transport::SocketDoorbell>(fds[5], fds[6], 1, true);
// Built into a TYPED pointer first: the witness is SocketDoorbell's, and the member
// is a Doorbell so that the inproc path can put a CondVarDoorbell in it.
auto selfBell = MakeUnique<Transport::SocketDoorbell>(fds[5], fds[6], 1, true);
// `dl`: THE DEATH WITNESS, and it has to be a descriptor this side does not also
// write to. Our own bell is a socketpair we hold BOTH ends of - slot 5 to park on,
// slot 6 to ring ourselves - so it can never hang up however dead the server is. The
// control socket's far end is held only by the server, so its hangup IS the death.
//
// Passing the fd rather than the transport keeps the ownership honest: the bell polls
// it for hangup alone and never reads it, and SocketTransport goes on owning and
// closing it. CONTRACT-P6 D5c.
selfBell->SetDeathWitness(m_socketTransport->StreamFd());
m_socketSelfBell = Move(selfBell);
}
// `cp`: from here the twelve EGL forwarders cross the socket instead of
@@ -896,6 +930,32 @@ namespace MobileGL::MG_Remote::Client {
const MobileGLResult received =
ReceiveEnvelope(*m_transport, reply, kRemoteControlTimeoutMs);
if (received != MOBILEGL_OK) {
// `dl` (D5b): THE EXPIRY IS NOT FATAL AND IT IS NOT ONE ANSWER. The transport
// already distinguishes the two states this wait can end in, and the contract
// says they must stay distinguished without a threshold:
//
// TRANSPORT_CLOSED - the control socket returned EOF. Only the server held the
// far end, so it is gone. That is the device-loss fact, taken from a
// descriptor.
// TIMEOUT - nothing arrived in the budget, and the socket is still open. The
// peer is ALIVE AND SILENT: wedged, frozen, SIGSTOP'd, or merely slower than
// the budget. 5.4 forbids calling that a death, so it gets a NAME instead.
// The bell is asked anyway, because a hangup noticed by the ring side counts
// here too.
if (received == MOBILEGL_ERR_TRANSPORT_CLOSED) {
LatchDeviceLost("the control stream reached EOF while a reply was owed");
} else if (const Transport::Doorbell* bell = m_producer.SelfDoorbell();
bell != nullptr && bell->PeerHungUp()) {
LatchDeviceLost("a control reply timed out and the peer had hung up");
} else {
MGLOG_E("MG_Remote client: the server is ALIVE BUT SILENT - no SurfaceReply "
"for %s seq %llu within %u ms and the control stream is still open. "
"This is NOT device loss (CONTRACT-P6 5.4): a frozen or merely slow "
"peer produces no hangup, and only a hangup may arm the latch. If "
"this repeats, the server is wedged rather than gone.",
Server::SurfaceControlOpName(frame.kind),
static_cast<unsigned long long>(frame.seq), kRemoteControlTimeoutMs);
}
MGLOG_E("MG_Remote client: no SurfaceReply for seq %llu within %u ms (rc=%d)",
static_cast<unsigned long long>(frame.seq), kRemoteControlTimeoutMs,
static_cast<int>(received));
@@ -1418,6 +1478,15 @@ namespace MobileGL::MG_Remote::Client {
const Transport::SessionWait wait =
WaitForAppliedBudget(m_producer, m_events, seq, waitBudgetMs);
if (wait == Transport::SessionWait::ShutDown) {
// `dl`: SHUTDOWN IS TWO DIFFERENT EVENTS AND ONLY ONE IS A DEVICE LOSS. Stop() kills
// this same bell to wake a parked applier, so an orderly exit arrives here too - and
// latching on Dead() alone would arm the device-lost flag on every clean teardown.
// PeerHungUp() is true only when a descriptor whose far end ONLY THE SERVER held
// reported hangup, which nothing this side can cause.
if (const Transport::Doorbell* bell = m_producer.SelfDoorbell();
bell != nullptr && bell->PeerHungUp()) {
LatchDeviceLost("the barrier woke on a peer that had hung up");
}
// The doorbell died: the server went away. The only thing that returns from a
// kWaitForever park, and therefore the only way a client blocked in the barrier
// survives a server that is gone. It is teardown, not a server fault - so the answer
@@ -1564,6 +1633,12 @@ namespace MobileGL::MG_Remote::Client {
const Transport::SessionWait wait =
WaitForAppliedBudget(m_producer, m_events, target, kBarrierTimeoutMs);
if (wait == Transport::SessionWait::ShutDown) {
// `dl`: the same two-events distinction the verb barrier makes. Only a peer that
// HUNG UP is a device loss; Stop() reaches this path too.
if (const Transport::Doorbell* bell = m_producer.SelfDoorbell();
bell != nullptr && bell->PeerHungUp()) {
LatchDeviceLost("a forced run-ahead wait woke on a peer that had hung up");
}
// The doorbell died: teardown, not a fault - the same answer EmitAndWaitTails
// gives, and for the same reason.
MGLOG_E("MG_Remote client: the forced wait for %s woke on a dead doorbell; the "