[Feat] (MG_Remote, P11 B1): the apply thread's auto affinity and spin chosen per session from the peer's dial mode and data plane - a dialled-in shared-segment client leaves the apply thread only the lowest prime core, stream, forked and in-process peers keep every big core

MobileGL/MG_Remote/Server/ApplyThreadPolicy.h

@@ -0,0 +1,141 @@
+// MobileGL - MobileGL/MG_Remote/Server/ApplyThreadPolicy.h
+// Copyright (c) 2025-2026 MobileGL-Dev
+// Licensed under the GNU Lesser General Public License v3.0:
+//   https://www.gnu.org/licenses/gpl-3.0.txt
+//   https://www.gnu.org/licenses/lgpl-3.0.txt
+// SPDX-License-Identifier: LGPL-3.0-only
+// End of Source File Header
+
+// P11 B1 (CONTRACT-P11 B1): WHERE THE APPLY THREAD RUNS AND HOW LONG IT SPINS, CHOSEN PER SESSION
+// FROM WHO THE CLIENT IS AND WHICH DATA PLANE IT USES.
+//
+// `auto` used to mean one thing for every session: the apply thread pinned to every big core
+// (DetectBigCoreMask, 0xc0 on the workload device) and spinning MOBILEGL_IPC_SPIN_US before each
+// park. That is right for inproc and the same-app fork spawn (A4's measured shapes), and for a client
+// on the stream plane, whose records an io thread feeds to the apply thread. It is wrong for a
+// client that DIALLED IN from another process over SHARED SEGMENTS - an app_process-launched
+// program (the fd: endpoint), a Termux one, a root or same-app one on `unix:`: the apply thread
+// spins on the ring that client writes directly, the client's GL thread lands on the same two prime
+// cores, and on the Redmi rd12 ran 23-26 fps over shared memory against 45-54 over tcp, 118 when the
+// two were placed apart by hand (B0-CROSS-APP.md, HSPIKE.md). So `auto` now reads the peer's Hello:
+//
+//   peer, plane                     `auto` resolves to
+//   in-process (No)                 every big core, the configured spin             - unchanged
+//   forked (Fork)                   every big core, the configured spin             - unchanged
+//   dialled in (Connect), stream    every big core, the configured spin             - unchanged
+//   dialled in, shared segments     ONE prime core - the LOWEST-numbered big core, and only when the
+//                                   big set is a strict subset of at least two - and
+//                                   kConnectedPeerSpinUs unless MOBILEGL_IPC_SPIN_US was set
+//
+// The lowest, not the highest, by measurement (CONTRACT-P11 B1): the scheduler puts a foreground
+// app's hot GL thread on the HIGHEST prime core (cpu7 in 43 of 44 samples); pinning the apply
+// thread there shared that core with it (rd12 6.5 fps), pinning it to the lowest gave 120. A client
+// launched through ExternalClientHelper is also kept off the reserved core (the broker hands the
+// helper this same core as its avoid mask), so a client the scheduler would otherwise place on it -
+// an adb-shell one - is not starved either.
+//
+// An explicit mask or `off` wins for every peer, exactly as before: the operator's word is the
+// policy. A pure function, so the unit suite can hold every row without a thread or a cpufreq tree.
+
+#pragma once
+
+#include <Includes.h>
+
+#include <cstdlib>
+#include <cstring>
+
+namespace MobileGL::MG_Remote::Server {
+
+    // Who the session's client is, from its Hello (protocol.fbs DialMode).
+    enum class ApplyPeer : Uint8 {
+        InProcess = 0, // DialMode::No     - inproc: the client is another thread of this process
+        Forked = 1,    // DialMode::Fork   - the client launched this server (same app, same build)
+        Connected = 2, // DialMode::Connect - the client dialled a standing server: another process,
+                       //                     possibly another app (tcp://, unix:, fd:)
+    };
+
+    // The apply thread's spin before it parks, for a dialled-in shared-segment peer when
+    // MOBILEGL_IPC_SPIN_US is not set. 50 us - the global default - by measurement: parking at once
+    // (0) lost in every cell measured (rd12 Espryt 120 -> 74 fps on the helper route, 27 -> 17 from the
+    // shell), so the short spin stays (CONTRACT-P11 B1).
+    inline constexpr Uint32 kConnectedPeerSpinUs = 50;
+
+    struct ApplyThreadPolicy {
+        Uint64 requestedMask = 0; // 0 = make no affinity call
+        Uint32 spinUs = 0;
+        Bool recognised = true;   // false: the affinity text was not `auto`, `off` or a number
+        const char* rule = "";    // what the log line names
+    };
+
+    inline const char* ApplyPeerName(ApplyPeer peer) {
+        switch (peer) {
+        case ApplyPeer::InProcess: return "in-process";
+        case ApplyPeer::Forked: return "forked";
+        case ApplyPeer::Connected: return "connected";
+        }
+        return "unknown";
+    }
+
+    inline Uint32 PopCount64(Uint64 mask) {
+        Uint32 count = 0;
+        for (; mask != 0; mask &= mask - 1) ++count;
+        return count;
+    }
+
+    // The core `auto` reserves for a dialled-in shared-segment peer's apply thread: the lowest big
+    // core when the big set is a strict subset of the online cpus with at least two members, else 0
+    // (no prime pair to split). ExternalClientBroker.java mirrors this rule for the helper's avoid mask.
+    inline Uint64 ReservedApplyCore(Uint64 bigCoreMask, Uint64 onlineMask) {
+        const Bool asymmetric = bigCoreMask != 0 && (bigCoreMask & onlineMask) != onlineMask;
+        if (!asymmetric || PopCount64(bigCoreMask) < 2) return 0;
+        return bigCoreMask & (~bigCoreMask + 1); // the lowest set bit
+    }
+
+    // `affinity` is MOBILEGL_IPC_SERVER_AFFINITY's raw text (null or empty = `auto`); `spinExplicit`
+    // says MOBILEGL_IPC_SPIN_US was set in this process's environment and `configuredSpinUs` is its
+    // value (ConfigLoader's, 50 when unset); `sharedSegments` is the session's data plane (false =
+    // stream); `bigCoreMask` is DetectBigCoreMask()'s answer (0 when the topology is unreadable) and
+    // `onlineMask` one bit per cpu the process could see.
+    inline ApplyThreadPolicy SelectApplyThreadPolicy(const char* affinity, Bool spinExplicit, Uint32 configuredSpinUs,
+                                                     ApplyPeer peer, Bool sharedSegments, Uint64 bigCoreMask,
+                                                     Uint64 onlineMask) {
+        ApplyThreadPolicy policy;
+        policy.spinUs = configuredSpinUs;
+        const Bool automatic = affinity == nullptr || affinity[0] == '\0' || std::strcmp(affinity, "auto") == 0;
+        if (!automatic) {
+            if (std::strcmp(affinity, "off") == 0) {
+                policy.rule = "off";
+                return policy;
+            }
+            char* end = nullptr;
+            const unsigned long long parsed = std::strtoull(affinity, &end, 0);
+            if (end == affinity || (end != nullptr && *end != '\0')) {
+                policy.recognised = false;
+                policy.rule = "unrecognised";
+                return policy;
+            }
+            policy.requestedMask = static_cast<Uint64>(parsed);
+            policy.rule = "explicit mask";
+            return policy;
+        }
+        if (peer != ApplyPeer::Connected) {
+            policy.requestedMask = bigCoreMask;
+            policy.rule = "auto: every big core (in-process / forked peer)";
+            return policy;
+        }
+        if (!sharedSegments) {
+            // The stream plane: an io thread feeds the apply thread, which does not spin on the
+            // client's writes - B0 measured 7.9 ms/frame of apply CPU there against 26.8 on shm.
+            policy.requestedMask = bigCoreMask;
+            policy.rule = "auto: every big core (dialled-in peer on the stream plane)";
+            return policy;
+        }
+        policy.spinUs = spinExplicit ? configuredSpinUs : kConnectedPeerSpinUs;
+        policy.requestedMask = ReservedApplyCore(bigCoreMask, onlineMask);
+        policy.rule = policy.requestedMask != 0
+                          ? "auto: the lowest prime core, the rest left to the dialled-in shared-segment client"
+                          : "auto: no affinity for a dialled-in shared-segment client (no prime pair to split)";
+        return policy;
+    }
+
+} // namespace MobileGL::MG_Remote::Server

MobileGL/MG_Remote/Server/ServerLoop.cpp

@@ -9,6 +9,7 @@
 // P5 package v1: the apply thread, its affinity, its parking, and the EGL ownership move.
 
 #include "ServerLoop.h"
+#include "ApplyThreadPolicy.h"
 #include <MG_Remote/FatalFunnel.h>
 #include <MG_Remote/Transport/LinkMetrics.h>
 #include <MG_Backend/MGPipe/PipeInputs.h>
@@ -23,6 +24,7 @@
 #include <chrono>
 #include <cstdlib>
 #include <cstdio>
+#include <cstring>
 #include <thread>
 
 #if defined(__linux__) || defined(__ANDROID__)
@@ -82,22 +84,19 @@ namespace MobileGL::MG_Remote::Server {
             return mask;
         }
 
-        // MOBILEGL_IPC_SERVER_AFFINITY = `auto` | `off` | an explicit mask (0x... or decimal).
-        // The RAW STRING is what Config keeps, because the resolved mask is what gets logged -
-        // "an affinity that silently did nothing looks exactly like one that worked"
-        // (CONTRACT-P5 5).
-        Uint64 RequestedAffinityMask(const char* raw, Bool* outRecognised) {
-            *outRecognised = true;
-            if (raw == nullptr || raw[0] == '\0') return DetectBigCoreMask();
-            if (std::strcmp(raw, "auto") == 0) return DetectBigCoreMask();
-            if (std::strcmp(raw, "off") == 0) return 0;
-            char* end = nullptr;
-            const unsigned long long parsed = std::strtoull(raw, &end, 0);
-            if (end == raw || (end != nullptr && *end != '\0')) {
-                *outRecognised = false;
-                return 0;
-            }
-            return static_cast<Uint64>(parsed);
+        // One bit per cpu the process can count, the same count DetectBigCoreMask walks - what
+        // lets ApplyThreadPolicy tell a real prime set from a symmetric machine's "every cpu".
+        Uint64 OnlineCpuMask() {
+            const Uint cpuCount = std::min(64u, std::max(1u, std::thread::hardware_concurrency()));
+            return cpuCount >= 64 ? ~0ull : ((1ull << cpuCount) - 1);
+        }
+
+        // Whether MOBILEGL_IPC_SPIN_US was set at all: ConfigLoader answers 50 for "unset" and
+        // for "50", and the dialled-in peer's default spin (ApplyThreadPolicy.h) applies only to
+        // the first.
+        Bool SpinSetExplicitly() {
+            const char* text = std::getenv("MOBILEGL_IPC_SPIN_US");
+            return text != nullptr && text[0] != '\0';
         }
 
         void NameThisThread(const char* name) {
@@ -406,9 +405,20 @@ namespace MobileGL::MG_Remote::Server {
         // forwarder (the in-process display server's) sends to the client. Inert otherwise.
         MG_Util::Debug::SetThreadForwardsLogToPeer(true);
 
-        Bool recognised = true;
+        // MOBILEGL_IPC_SERVER_AFFINITY = `auto` | `off` | an explicit mask (0x... or decimal).
+        // The RAW STRING is what Config keeps, because the resolved mask is what gets logged -
+        // "an affinity that silently did nothing looks exactly like one that worked"
+        // (CONTRACT-P5 5). P11 B1: `auto` now depends on who the client is (ApplyThreadPolicy.h),
+        // and so does the spin before each park.
+        ServerSession& session = *m_session;
         const char* raw = AffinityStringFromConfig();
-        const Uint64 requested = RequestedAffinityMask(raw, &recognised);
+        const Bool automatic = raw == nullptr || raw[0] == '\0' || std::strcmp(raw, "auto") == 0;
+        const ApplyThreadPolicy policy =
+            SelectApplyThreadPolicy(raw, SpinSetExplicitly(), SpinUsFromConfig(), session.Peer(),
+                                    session.PeerSharedSegments(), automatic ? DetectBigCoreMask() : 0,
+                                    OnlineCpuMask());
+        const Bool recognised = policy.recognised;
+        const Uint64 requested = policy.requestedMask;
         m_affinityMask = ApplyAffinity(requested);
         if (!recognised) {
             MGLOG_E("MG_Remote server: MOBILEGL_IPC_SERVER_AFFINITY='%s' is not `auto`, `off` or "
@@ -421,11 +431,11 @@ namespace MobileGL::MG_Remote::Server {
         // an operator typed and the mask is what the kernel took.
         MGLOG_I("MG_Remote server: mgl-srv-apply started. MOBILEGL_IPC_SERVER_AFFINITY='%s' "
                 "requested mask 0x%llx, RESOLVED mask 0x%llx (0 = no affinity applied), spin "
-                "%u us",
+                "%u us; peer %s on %s, policy %s",
                 raw == nullptr ? "" : raw, static_cast<unsigned long long>(requested),
-                static_cast<unsigned long long>(m_affinityMask), SpinUsFromConfig());
+                static_cast<unsigned long long>(m_affinityMask), policy.spinUs, ApplyPeerName(session.Peer()),
+                session.PeerSharedSegments() ? "shared segments" : "stream", policy.rule);
 
-        ServerSession& session = *m_session;
         // The decoder is built HERE, on this thread, because PipeWireDecoder is "not thread
         // safe: one decoder on the apply thread, by construction" and its constructor installs
         // the process-wide apply hook.
@@ -434,7 +444,7 @@ namespace MobileGL::MG_Remote::Server {
         const auto signals = session.DataLink()->Signals();
         Transport::Doorbell& bell = session.DataLink()->ConsumerBell();
         Transport::RingConsumer& ring = session.CommandRing();
-        const Uint32 spinUs = SpinUsFromConfig();
+        const Uint32 spinUs = policy.spinUs;
 
         // THE PARK CONDITION IS THREE THINGS, NOT ONE, AND THAT IS THE WHOLE REASON THIS LOOP
         // DOES NOT CALL SessionConsumer::WaitForWork.

MobileGL/MG_Remote/Server/ServerSession.cpp

@@ -654,6 +654,10 @@ namespace MobileGL::MG_Remote::Server {
             hello->dialMode() != ::MobileGL::Wire::DialMode::Connect)
             return RefuseHandshake(transport, ::MobileGL::Wire::RefuseCode::MalformedHello,
                                    "unknown dial mode");
+        // P11 B1: remembered for the apply thread's `auto` policy (ApplyThreadPolicy.h).
+        m_peer = hello->dialMode() == ::MobileGL::Wire::DialMode::Connect ? ApplyPeer::Connected
+                 : hello->dialMode() == ::MobileGL::Wire::DialMode::Fork  ? ApplyPeer::Forked
+                                                                          : ApplyPeer::InProcess;
         const MobileGLResult compatible = ValidatePeerHandshake(transport, hello->abiMajor(),
             hello->abiMinor(), hello->wireFingerprint(), theirStamp, hello->dialMode());
         if (compatible != MOBILEGL_OK) return compatible;
@@ -665,6 +669,7 @@ namespace MobileGL::MG_Remote::Server {
             return RefuseHandshake(transport, ::MobileGL::Wire::RefuseCode::LinkTerms,
                                    "unsupported or missing link terms");
         const bool stream = terms->dataPlane() == ::MobileGL::Wire::DataPlane::Stream;
+        m_peerSharedSegments = !stream; // P11 B1: the apply thread's `auto` policy reads the plane too
         if (stream && transport.Role() == Transport::TransportRole::InProcess)
             return RefuseHandshake(transport, ::MobileGL::Wire::RefuseCode::LinkTerms,
                                    "stream requires a data connection");

MobileGL/MG_Remote/Server/ServerSession.h

@@ -57,6 +57,7 @@
 #include "../Transport/RoleMemory.h"
 #include "../Transport/SessionRings.h"
 #include "../Wire/PipeWireCodec.h"
+#include "ApplyThreadPolicy.h"
 #include "PipeApplier.h"
 
 namespace MobileGL::MG_Remote::Server {
@@ -181,6 +182,11 @@ namespace MobileGL::MG_Remote::Server {
         Uint64 CallMask() const;
 
         Bool Accepted() const;
+        // P11 B1: who the client is - its Hello's dial mode - and whether it uses shared segments
+        // (false = the stream plane), both set by Accept before Welcome. ServerLoop picks the apply
+        // thread's `auto` affinity and spin from them (ApplyThreadPolicy.h).
+        ApplyPeer Peer() const { return m_peer; }
+        Bool PeerSharedSegments() const { return m_peerSharedSegments; }
         // Teardown: after the apply thread has been joined, never before - a record still in
         // flight can still resolve a segment offset (table 3's fourth column).
         void Close();
@@ -309,6 +315,8 @@ namespace MobileGL::MG_Remote::Server {
         Bool m_consumedSet = false;
         Bool m_sizesSet = false;
         Bool m_accepted = false;
+        ApplyPeer m_peer = ApplyPeer::InProcess;
+        Bool m_peerSharedSegments = true;
         // PH-6. Written by the apply thread (the only SEG_EVENT producer), reset by Accept.
         std::atomic<Bool> m_reverseChannelForfeit{false};
         std::atomic<Uint64> m_forfeitDrops{0};