[Feat] (Build, TraceApp): ship and exec a second native binary on android

- PLAN-B.md §11 P0 lists spike A (the Android delivery chain) as a P0 deliverable,
  inherited verbatim from PLAN.md §15 P0; §8.1 inherits PLAN.md §11.1-§11.6, whose
  Android path needs a second process. Android gives an application no writable
  exec-able directory, so the only supported route is to name the binary lib*.so, let
  the packager put it in lib/<abi>/, and exec it out of
  getApplicationInfo().nativeLibraryDir. This builds that route end to end so the
  spike can be answered with evidence instead of folklore.
- New root option MOBILEGL_BUILD_SERVER_SPIKE (OFF, ANDROID-only) adds the
  MobileGLServer target from tools/spikes/server_stub/main.cpp with PREFIX "lib" /
  SUFFIX ".so" and -fPIE/-pie: an .so name does not exempt the file from Android's
  PIE requirement. Its RUNTIME_OUTPUT_DIRECTORY is pointed at
  CMAKE_LIBRARY_OUTPUT_DIRECTORY, because AGP packages what lands in the per-ABI
  library output directory and CMake would otherwise put an executable elsewhere.
- The option is opt-in on both sides. The plugin flavour cannot turn it on at all,
  and the trace flavour builds it only when asked, with
  `-Pmobilegl.buildServerSpike=ON` or MOBILEGL_BUILD_SERVER_SPIKE=ON in the
  environment; a flavour that silently carries an executable nothing loads is the
  kind of thing nobody notices until it ships. Verified both ways:
  assembleTraceDebug -Pmobilegl.buildServerSpike=ON packages
  lib/arm64-v8a/libMobileGLServer.so and `file` reports "ELF 64-bit LSB pie
  executable, ARM aarch64 ... interpreter /system/bin/linker64, for Android 26";
  the same task with no property packages only libMobileGL.so and
  libtrace_replay_runner.so.
- The stub prints one line to stdout and writes the same line to the file named by
  argv[1], then exits 0. The line carries pid/ppid/uid/gid and, decisively, the
  child's own /proc/self/attr/current: only `u:r:untrusted_app:...` proves an
  ordinary app process did the exec. An `adb run-as` shell runs in a different
  SELinux domain, so a success there would prove nothing.
- RunSpawnSpike() starts the stub with argv [serverPath, markerPath], redirects the
  child's stdout/stderr into a captured file (an app process has stdout on
  /dev/null, so a printed line would otherwise vanish), waits for it, and reports
  exit status, signal, the exec errno, the parent's own SELinux context, the marker
  content and the captured stdout - to logcat, to the returned string, and to a
  <marker>.report file, because the Activity finishes immediately afterwards.
- The child reports the errno of a REFUSED execve through a close-on-exec pipe.
  Without it the one datum the spike exists to produce is lost: the parent only ever
  sees a wait status, in which every reason has already been flattened into one exit
  code, and EACCES (SELinux, or a noexec mount) versus ENOEXEC (a packager that
  mangled the file) are opposite verdicts for the design. A successful exec closes
  the write end for free, so the parent reads EOF and reports execErrno=0.
- fork/execve only. The earlier draft also carried a posix_spawn arm behind
  `__ANDROID_API__ >= 28`, which was dead code in every configuration this repo can
  build - bionic declares posix_spawn from API 28 and the root CMakeLists.txt pins
  MOBILEGL_ANDROID_API_LEVEL to 26 and refuses to configure lower - and would have
  silently become the production path, untested, on a minSdk bump. Keeping the arm
  that actually ships means the spike measures the code the server would really use.
  Nothing happens between fork and execve except open/dup2/execve/write/_exit, all
  async-signal-safe, because the parent is a multi-threaded JVM process.
- The spike lives in its own TU, spawn_spike.cpp/.hpp, listed only by the trace
  APK's CMakeLists. Its sibling trace_replay_core.cpp is compiled verbatim by the
  DESKTOP mobilegl_trace_replay runner (tools/trace_replay/CMakeLists.txt names the
  same file), where <android/log.h> does not exist, so nothing Android-only may live
  there; spawn_spike.cpp carries an #error for anyone who adds it to that list.
- The Activity runs the spike, and nothing else, when launched with the
  `mobilegl_spike_spawn` intent extra; that mode needs no trace, no golden and no
  render surface. It is a separate JNI entry point rather than another parameter on
  the 30-argument replay call, which it shares nothing with.
- Not yet run on a device: both device locks are held by another campaign. The
  on-device verdict is the coordinator's step.
This commit is contained in:
2026-09-05 20:49:57 -04:00
parent 87ee17c68c
commit 8a239177ac
8 changed files with 513 additions and 0 deletions
@@ -234,6 +234,10 @@ target_link_libraries(glretrace_common PUBLIC retrace_common glhelpers glproc)
add_library(trace_replay_runner SHARED
trace_replay_core.cpp
trace_replay_jni.cpp
# P0 spike A. Android-only, and deliberately its own TU: trace_replay_core.cpp is
# shared verbatim with the desktop mobilegl_trace_replay runner
# (tools/trace_replay/CMakeLists.txt), which cannot see <android/log.h>.
spawn_spike.cpp
"${CMAKE_CURRENT_LIST_DIR}/../../../../../tools/trace_replay/apitrace_fbo_dump.cpp")
target_compile_features(trace_replay_runner PRIVATE cxx_std_17)
@@ -249,4 +253,5 @@ target_link_libraries(trace_replay_runner
retrace_common
image
android
log
dl)
@@ -0,0 +1,227 @@
// P0 spike A - the Android half of the delivery chain (PLAN-B.md §8.1, inheriting
// PLAN.md §11.1-§11.6). See spawn_spike.hpp for what the spike is asking.
//
// Android-only on purpose: this TU is listed only by
// android-plugin/app/src/trace/cpp/CMakeLists.txt. Its sibling trace_replay_core.cpp is
// shared with the DESKTOP mobilegl_trace_replay runner, which has no <android/log.h>,
// so nothing Android-specific may live there.
#include "spawn_spike.hpp"
#if !defined(__ANDROID__)
#error "spawn_spike.cpp is Android-only; do not add it to the desktop trace replay build"
#endif
#include <android/log.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
#include <cerrno>
#include <cstring>
#include <fstream>
#include <sstream>
#include <string>
// execve needs the environment the parent already has: a server process started from
// the app must inherit it, and handing it an empty one would change what is being tested.
extern "C" char** environ;
namespace mobilegl_trace {
namespace {
constexpr const char* kSpikeLogTag = "MobileGLTraceRunner";
std::string ReadWholeFile(const std::string& path) {
std::ifstream input(path, std::ios::binary);
if (!input) {
return {};
}
std::ostringstream contents;
contents << input.rdbuf();
std::string text = contents.str();
while (!text.empty() && (text.back() == '\n' || text.back() == '\r' || text.back() == '\0')) {
text.pop_back();
}
return text;
}
// The domain this process is in. `u:r:untrusted_app:s0:...` is the whole point of the
// spike: an exec that works from an `adb run-as` shell says nothing about whether the
// app itself is allowed to do it, because that shell is a different SELinux domain.
std::string ReadSelfSelinuxContext() {
const std::string context = ReadWholeFile("/proc/self/attr/current");
return context.empty() ? "<unreadable>" : context;
}
// Starts the child with its stdout and stderr redirected into `outputPath`, reports the
// child pid through `childPid` and, when the exec itself was refused, the child's errno
// through `execErrno`. Returns 0, or the errno of a failure that happened before the
// child existed at all.
//
// fork/execve, not posix_spawn: bionic only declares posix_spawn from API 28 while
// MobileGL ships at minSdk 26 (the root CMakeLists.txt pins MOBILEGL_ANDROID_API_LEVEL
// to 26 and refuses to configure lower), so posix_spawn is not available to the shipping
// build and this is the shape the production spawn path has to take. Nothing happens
// between fork and execve except open/dup2/execve/write/_exit, all async-signal-safe,
// because the parent is a multi-threaded JVM process.
int SpawnSpikeChild(const std::string& serverPath,
const std::string& markerPath,
const std::string& outputPath,
pid_t* childPid,
int* execErrno) {
*execErrno = 0;
char* argv[] = {const_cast<char*>(serverPath.c_str()),
const_cast<char*>(markerPath.c_str()), nullptr};
// The errno of a refused exec is the answer this spike is here to bring back, and it
// is raised in a process that cannot return anything: by the time the parent sees a
// wait status the reason has been flattened into an exit code. So the child writes
// the raw errno into a close-on-exec pipe. A successful exec closes the write end for
// free and the parent reads EOF; a refused one leaves the four bytes behind. EACCES
// (SELinux, or a noexec mount) and ENOEXEC (a mangled or non-PIE file) are entirely
// different verdicts for the design and this is the only thing that separates them.
int report[2] = {-1, -1};
if (pipe2(report, O_CLOEXEC) != 0) {
return errno;
}
const pid_t forked = fork();
if (forked < 0) {
const int forkErrno = errno;
close(report[0]);
close(report[1]);
return forkErrno;
}
if (forked == 0) {
close(report[0]);
// Without this the child's output is unobservable: an Android app process has
// stdout on /dev/null, so a printed line would vanish and the spike could not
// tell "ran and printed" apart from "never ran".
const int outputFd = open(outputPath.c_str(), O_CREAT | O_WRONLY | O_TRUNC, 0664);
if (outputFd >= 0) {
dup2(outputFd, STDOUT_FILENO);
dup2(outputFd, STDERR_FILENO);
if (outputFd != STDOUT_FILENO && outputFd != STDERR_FILENO) {
close(outputFd);
}
}
execve(serverPath.c_str(), argv, environ);
const int failure = errno;
// Only reached when the exec was refused - the one outcome this spike is about.
const ssize_t written = write(report[1], &failure, sizeof(failure));
static_cast<void>(written);
// 127 is the shell's convention for "could not exec" and is distinguishable from
// every status the stub itself can return.
_exit(127);
}
close(report[1]);
int failure = 0;
ssize_t got = 0;
// Blocks until the child either execs (the write end closes, read returns 0) or
// reports why it could not.
while ((got = read(report[0], &failure, sizeof(failure))) < 0 && errno == EINTR) {
}
close(report[0]);
if (got == static_cast<ssize_t>(sizeof(failure))) {
*execErrno = failure;
}
*childPid = forked;
return 0;
}
} // namespace
SpawnSpikeResult RunSpawnSpike(const SpawnSpikeRequest& request) {
SpawnSpikeResult result;
result.parentSelinuxContext = ReadSelfSelinuxContext();
if (request.serverPath.empty() || request.markerPath.empty()) {
result.message = "spike-spawn: serverPath and markerPath are both required";
return result;
}
// A stale marker from a previous run would otherwise be read back as this run's
// proof. Remove it first, so "the marker exists" can only mean the child wrote it.
unlink(request.markerPath.c_str());
const std::string childOutputPath = request.markerPath + ".stdout";
unlink(childOutputPath.c_str());
struct stat serverStat {};
if (stat(request.serverPath.c_str(), &serverStat) != 0) {
result.spawnErrno = errno;
result.message = "spike-spawn: " + request.serverPath + " does not exist: " +
std::strerror(errno);
__android_log_print(ANDROID_LOG_ERROR, kSpikeLogTag, "%s", result.message.c_str());
return result;
}
pid_t childPid = -1;
int execErrno = 0;
const int spawnStatus = SpawnSpikeChild(request.serverPath, request.markerPath,
childOutputPath, &childPid, &execErrno);
if (spawnStatus != 0) {
result.spawnErrno = spawnStatus;
result.message = "spike-spawn: could not start " + request.serverPath +
": spawnErrno=" + std::to_string(spawnStatus) + " (" +
std::strerror(spawnStatus) + ")";
__android_log_print(ANDROID_LOG_ERROR, kSpikeLogTag, "%s (parentSelinux=%s)",
result.message.c_str(), result.parentSelinuxContext.c_str());
return result;
}
result.spawned = true;
result.childPid = static_cast<int>(childPid);
result.execErrno = execErrno;
int waitStatus = 0;
while (waitpid(childPid, &waitStatus, 0) < 0) {
if (errno != EINTR) {
result.message = "spike-spawn: waitpid failed: " + std::string(std::strerror(errno));
__android_log_print(ANDROID_LOG_ERROR, kSpikeLogTag, "%s", result.message.c_str());
return result;
}
}
result.waitStatus = waitStatus;
if (WIFEXITED(waitStatus)) {
result.exitCode = WEXITSTATUS(waitStatus);
}
if (WIFSIGNALED(waitStatus)) {
result.termSignal = WTERMSIG(waitStatus);
}
result.markerContent = ReadWholeFile(request.markerPath);
result.childOutput = ReadWholeFile(childOutputPath);
result.succeeded =
result.execErrno == 0 && result.exitCode == 0 && !result.markerContent.empty();
std::ostringstream message;
message << "spike-spawn: " << (result.succeeded ? "OK" : "FAILED")
<< " server=" << request.serverPath
<< " pid=" << result.childPid
<< " exit=" << result.exitCode
<< " signal=" << result.termSignal
// Always printed, including on the success path, so a reader never has to
// guess whether the field was collected or merely absent.
<< " execErrno=" << result.execErrno
<< " (" << (result.execErrno == 0 ? "exec succeeded"
: std::strerror(result.execErrno)) << ")"
<< " parentSelinux=" << result.parentSelinuxContext
<< " marker=[" << result.markerContent << "]"
<< " childStdout=[" << result.childOutput << "]";
result.message = message.str();
__android_log_print(result.succeeded ? ANDROID_LOG_INFO : ANDROID_LOG_ERROR, kSpikeLogTag,
"%s", result.message.c_str());
// The Activity is normally gone as soon as the run finishes, so the verdict also goes
// to a file next to the marker; that is what a device lane copies out.
std::ofstream report(request.markerPath + ".report", std::ios::trunc);
if (report) {
report << result.message << "\n";
}
return result;
}
} // namespace mobilegl_trace
@@ -0,0 +1,60 @@
#pragma once
// ---------------------------------------------------------------------------
// P0 spike A: exec a second packaged native executable from this process.
//
// Answers one question and nothing else: can an ordinary Android application
// process (untrusted_app, NOT an `adb run-as` shell, which runs in a different
// SELinux domain and would prove nothing) exec a binary that was shipped inside
// its own APK as lib/<abi>/lib*.so? The disaggregated design needs a server
// process on Android and this is its only supported delivery route (PLAN-B.md
// §8.1, inheriting PLAN.md §11.1-§11.6).
//
// This lives beside trace_replay_core.hpp rather than inside it because
// trace_replay_core.cpp is ALSO compiled by the desktop mobilegl_trace_replay
// runner (tools/trace_replay/CMakeLists.txt names it directly), where <android/log.h>
// does not exist. The spike is Android-only, so it gets an Android-only TU;
// spawn_spike.cpp is listed only by the trace APK's CMakeLists.
//
// Nothing in the replay path calls this; it runs only when the trace Activity is
// launched with the `mobilegl_spike_spawn` intent extra.
// ---------------------------------------------------------------------------
#include <string>
namespace mobilegl_trace {
struct SpawnSpikeRequest {
// Absolute path of the executable, normally
// getApplicationInfo().nativeLibraryDir + "/libMobileGLServer.so".
std::string serverPath;
// Marker file the child is asked to write, passed to it as argv[1]. The child's
// stdout and stderr are captured next to it, with ".stdout" appended.
std::string markerPath;
};
struct SpawnSpikeResult {
bool spawned = false;
// Exec'd, waited for, exited 0, and the marker file came back non-empty.
bool succeeded = false;
// errno of the pre-fork or fork failure - the parent could not even try.
int spawnErrno = 0;
// errno of a REFUSED execve, carried out of the child over a close-on-exec pipe.
// This is the one datum the spike exists to produce: EACCES (SELinux or the mount's
// noexec) and ENOEXEC (the packager mangled the file) are different verdicts, and
// the exit status alone cannot tell them apart.
int execErrno = 0;
int childPid = -1;
int waitStatus = -1;
int exitCode = -1;
int termSignal = -1;
// /proc/self/attr/current of THIS process - the domain the exec was attempted from.
std::string parentSelinuxContext;
std::string markerContent;
std::string childOutput;
std::string message;
};
SpawnSpikeResult RunSpawnSpike(const SpawnSpikeRequest& request);
} // namespace mobilegl_trace
@@ -1,5 +1,7 @@
#include "trace_replay_core.hpp"
#include "spawn_spike.hpp"
#include <android/native_window.h>
#include <android/native_window_jni.h>
#include <jni.h>
@@ -196,3 +198,20 @@ Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunTraceReplay(JNIEnv*
}
return MakeResult(env, result);
}
// P0 spike A: exec the packaged MobileGLServer stub from this app process and report what
// happened. Deliberately a separate entry point rather than another parameter on the
// replay call - it shares nothing with a replay, and the trace lane must be able to run
// it without a trace, a golden or a surface.
extern "C" JNIEXPORT jstring JNICALL
Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunSpawnSpike(JNIEnv* env,
jclass,
jstring serverPath,
jstring markerPath) {
mobilegl_trace::SpawnSpikeRequest request;
request.serverPath = ToString(env, serverPath);
request.markerPath = ToString(env, markerPath);
const mobilegl_trace::SpawnSpikeResult result = mobilegl_trace::RunSpawnSpike(request);
return env->NewStringUTF(result.message.c_str());
}
@@ -54,6 +54,14 @@ public final class TraceReplayActivity extends Activity {
android.view.ViewGroup.LayoutParams.WRAP_CONTENT
));
// P0 spike A: when asked, exec the packaged server stub out of nativeLibraryDir
// instead of replaying anything. This mode needs no trace and no render surface.
String spikeLibrary = spawnSpikeLibrary(intent);
if (spikeLibrary != null) {
runSpawnSpike(spikeLibrary);
return;
}
SurfaceHolder holder = surfaceView.getHolder();
if (request.width > 0 && request.height > 0) {
holder.setFixedSize(request.width, request.height);
@@ -166,6 +174,53 @@ public final class TraceReplayActivity extends Activity {
String benchmarkResultPath
);
// ---------------------------------------------------------------------------
// P0 spike A: prove an APK can ship a second native executable and exec it.
//
// The exec has to happen here, in the application's own process: an `adb shell
// run-as` invocation runs in a different SELinux domain, so it can succeed while
// the real app is denied. The child reports the domain it ended up in, and the
// parent reports the domain it spawned from, so the log line stands on its own.
// ---------------------------------------------------------------------------
private static final String EXTRA_SPAWN_SPIKE = "mobilegl_spike_spawn";
private static final String DEFAULT_SPAWN_SPIKE_LIBRARY = "libMobileGLServer.so";
private static String spawnSpikeLibrary(Intent intent) {
if (!intent.hasExtra(EXTRA_SPAWN_SPIKE)) {
return null;
}
// Accepts --ez (boolean, arrives as a null string) and --es with either a truthy
// marker or the library file name to exec.
String value = intent.getStringExtra(EXTRA_SPAWN_SPIKE);
if (value == null || value.isEmpty() || "1".equals(value) || "true".equals(value)) {
return DEFAULT_SPAWN_SPIKE_LIBRARY;
}
return value;
}
private void runSpawnSpike(String libraryName) {
// The surface callbacks fire regardless; this keeps them from starting a replay
// underneath the spike.
started = true;
File outputDir = new File(request.outputDir);
String serverPath = new File(getApplicationInfo().nativeLibraryDir, libraryName)
.getAbsolutePath();
String markerPath = new File(outputDir, "spike-spawn.txt").getAbsolutePath();
statusView.setText("Running spawn spike\n" + serverPath);
new Thread(() -> {
outputDir.mkdirs();
String message = nativeRunSpawnSpike(serverPath, markerPath);
Log.i(TAG, message);
runOnUiThread(() -> {
statusView.setText(message);
finish();
});
}, "MobileGLSpawnSpike").start();
}
private static native String nativeRunSpawnSpike(String serverPath, String markerPath);
private static final class TraceReplayRequest {
final String tracePath;
final String goldenPath;