From 7ef7c7e54381af79f8d9e6635d883b9a03616ae6 Mon Sep 17 00:00:00 2001 From: Swung0x48 Date: Sat, 5 Sep 2026 20:45:21 -0400 Subject: [PATCH] [Fix] (Spikes): answer the tier question for DirectGLES too, make an OK mean bytes round-tripped through a real GPU access, and exercise T3 in the direction that makes it a tier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - plan-B §8.3 asks which tier `AcquirePersistentMap` lands in, but the probe only asked Vulkan. DirectGLES ("Espryt") reaches a persistent map through `glBufferStorageEXT` + `glMapBufferRange(PERSISTENT|COHERENT)`, not a `VkDeviceMemory` map, so a Vulkan-only answer decides nothing for that backend. Add a GLES leg to T1: the exported fd imported with `glCreateMemoryObjectsEXT` + `glImportMemoryFdEXT` + `glBufferStorageMemEXT`, then mapped PERSISTENT|COHERENT -- in-process first (isolates "GL can import this fd" from "the fd survives a process boundary"), then cross-process (new `t1gl` child). Drivers disagree about how the import must be phrased, so each attempt walks a ladder over {dedicated flag} x {import size = memory requirement or the fd's own size} x {buffer size} and reports the rung the driver accepted plus every rejected rung with its GL error -- a driver *preference* must never be reported as a missing capability. A driver that backs the storage but refuses PERSISTENT|COHERENT is reported separately from one that refuses the storage: that distinction is exactly T1 vs T2 for DirectGLES. The T0 GLES leg (`EGL_ANDROID_get_native_client_buffer` + `glBufferStorageExternalEXT` + persistent map, verified by `AHardwareBuffer_lock` on the client side) now reports every step's GL enum and requires the persistent flags for OK. - the verdict was unfalsifiable: T1 reported PARTIAL when neither leg had moved a byte. Replace it with an explicit decisive-leg model -- OK only when every decisive leg round-tripped in both directions, PARTIAL when at least one did, FAIL otherwise with the failing step and its driver error named in `why:`. Every row now opens with a per-leg trace (`vkimport[D]=rt gpu[D]=rt`). The raw `mmap` leg is informational for opaque-fd (Vulkan forbids interpreting that payload outside the driver, so a refusal is conformant) and decisive for dma-buf, where a CPU mapping is the point of the handle type. - T3 never ran the direction that would make it a tier: both ends were the importing process. Add `T3-client-memfd-server-import` (new `t3c` child) -- the client creates and writes the memfd, the server mmaps the received fd, imports the client's host pointer into a `VkDeviceMemory`, reads what the client wrote, writes back, and takes a GPU access on the client's memory, which the client then verifies through its own mapping. - no route touched the GPU, so an OK proved only that a map call returned a pointer. Every tier row now takes a real GPU access before it can be OK: `vkCmdCopyBuffer` out of the shared allocation into private staging (mismatch = the GPU could not read what the peer wrote) plus `vkCmdFillBuffer` into it, queue-idle and an explicit host-read barrier, with the peer checking the filled region through its own mapping. VkCtx grows a queue and command pool for it. - the device run executes in the `shell` SELinux domain, not the `untrusted_app` domain MobileGL runs in, and the two do not share dmabuf/gralloc rules. Print uid/pid/`/proc/self/attr/current` in a run-context header, repeat the caveat in the summary, and document in README.md how to answer it for the real domain later (exec the same binary from the trace app's spike hook, spike-A package) without implementing that here. - `vkStr()` returned a pointer into one static buffer while several results routinely appear in one format call, so all of them showed the last one; it returns std::string now, `memFlagStr` likewise, and `fmt`/`pr` carry `format(printf)` so a missed `.c_str()` is a compile error rather than UB. - `advertisedExportable` decided the status at the allocate site but not at the `vkGetMemoryFdKHR` site. One rule at every export failure now (`exportFailStatus`): advertised EXPORTABLE and then declining is FAIL, never advertised is UNSUPPORTED. Export + map + fd is factored into `exportHostVisible`. - `T0-ahb-blob-transfer` was recorded OK on the socket handoff alone. The handoff keeps its own informational row; the tier row is now composed at the end from the full import+map+compare+writeback chain over the Vulkan, GL and GPU legs. - `mmapErrno` kept the first attempt's errno after the second-chance mmap succeeded, so a working mapping carried a failure code; it is cleared on success and the first errno moves into the note. - a failed `glImportMemoryFdEXT` no longer closes the fd: EXT_memory_object_fd does not say whether ownership still transfers on failure and Mesa closes it either way, so closing risks a double close landing on the socket. Leaking a handful of dups in a short-lived probe is the safe side of that trade. - validated end to end on the host harness (lavapipe + llvmpipe, `VK_DRIVER_FILES=lvp_icd.json EGL_PLATFORM=surfaceless`): T1-opaque-fd OK, T3-external-memory-host OK, T3-memfd-cross-process OK, T3-client-memfd-server-import OK, T1-dma-buf UNSUPPORTED (not advertised exportable). The two T1-gles rows FAIL there with GL_OUT_OF_MEMORY on every ladder rung although GL_DEVICE_UUID_EXT matches the Vulkan deviceUUID -- llvmpipe's GL does not implement importing a lavapipe opaque-fd allocation, a Mesa interop gap recorded in README.md so a device FAIL stays attributable. Rebuilt for arm64-v8a with NDK r27d (PIE, android-30); the device run is pending, both device locks are held by another campaign. --- tools/spikes/extmem_probe/README.md | 162 +- tools/spikes/extmem_probe/extmem_probe.cpp | 2120 ++++++++++++++++---- 2 files changed, 1882 insertions(+), 400 deletions(-) diff --git a/tools/spikes/extmem_probe/README.md b/tools/spikes/extmem_probe/README.md index a9a6b74f..5017ba98 100644 --- a/tools/spikes/extmem_probe/README.md +++ b/tools/spikes/extmem_probe/README.md @@ -2,28 +2,51 @@ A standalone Android command-line probe that answers one question per device: -> Can a server-allocated `HOST_VISIBLE|HOST_COHERENT` `VkDeviceMemory` be shared -> with another process and mapped there, and by which route? +> Can the memory behind `AcquirePersistentMap` be shared with another process +> and mapped there — for **both** backends — and by which route? This is the P0 spike that decides the `AcquirePersistentMap` tier in plan B §8.3 (T0 = server imports a client allocation, T1 = server exports its own, T2 = give up and return `nullptr`). It links nothing from MobileGL and is not part of the project's CMake build graph. +Both backends are asked, because they reach a persistent map by different APIs: +DirectVulkan ("Magma") maps a `VkDeviceMemory`, while DirectGLES ("Espryt") +calls `glBufferStorageEXT` + `glMapBufferRange(PERSISTENT|COHERENT)`. A Vulkan +answer alone does not decide the tier for DirectGLES, so every tier has a GLES +leg. + ## What it does -* **phase A — enumeration.** Vulkan device identity + memory types, and per - handle type (`OPAQUE_FD`, `DMA_BUF`, `HOST_ALLOCATION`, `AHARDWAREBUFFER`) the +* **phase A — enumeration.** Run context (uid, pid, SELinux domain — see the + caveat below), Vulkan device identity + memory types, and per handle type + (`OPAQUE_FD`, `DMA_BUF`, `HOST_ALLOCATION`, `AHARDWAREBUFFER`) the `vkGetPhysicalDeviceExternalBufferProperties` verdict for the buffer usage MobileGL actually needs. Then a headless EGL pbuffer context reports `GL_EXT_memory_object{,_fd}`, `GL_EXT_external_buffer`, `GL_EXT_buffer_storage`, - `GL_OES_EGL_image_external{,_essl3}` and `EGL_ANDROID_get_native_client_buffer`. -* **T1 — server exports.** Allocates a `HOST_VISIBLE|HOST_COHERENT` buffer memory - with `VkExportMemoryAllocateInfo`, maps it, writes a pattern, exports an fd with - `vkGetMemoryFdKHR` (opaque-fd, then dma-buf), hands the fd to a second process - over `SCM_RIGHTS`, and has that process (a) `mmap()` the fd and (b) import it - into its own `VkDeviceMemory` and `vkMapMemory` it. Both sides write and both - sides compare, so a one-directional or copy-on-import mapping is caught. + `GL_OES_EGL_image_external{,_essl3}`, `EGL_ANDROID_get_native_client_buffer`, + and `GL_DEVICE_UUID_EXT` against the Vulkan `deviceUUID` (they must match for + an fd import to be legal, so a mismatch explains a later decline). +* **T1 — server exports (Vulkan).** Allocates a `HOST_VISIBLE|HOST_COHERENT` + buffer memory with `VkExportMemoryAllocateInfo`, maps it, writes a pattern, + takes a **GPU access** on it (below), exports an fd with `vkGetMemoryFdKHR` + (opaque-fd, then dma-buf), hands the fd to a second process over `SCM_RIGHTS`, + and has that process (a) `mmap()` the fd and (b) import it into its own + `VkDeviceMemory` and `vkMapMemory` it. Both sides write and both sides + compare, so a one-directional or copy-on-import mapping is caught. +* **T1-gles — server exports (GLES).** The same exported fd, imported as GL + buffer storage: `glCreateMemoryObjectsEXT` + `glImportMemoryFdEXT` + + `glBufferStorageMemEXT`, then `glMapBufferRange(PERSISTENT|COHERENT)` — first + **in-process** (isolates "GL can import this fd at all" from "the fd survives + a process boundary"), then **cross-process**. Because drivers disagree about + how the import must be phrased, each attempt walks a ladder over + {dedicated flag} × {import size = `VkMemoryRequirements::size` or the fd's own + size} × {buffer size}, and the report names the rung the driver accepted + (`accepted=…`) plus every rung it rejected with its GL error (`ladder: …`), so + a driver *preference* is never reported as a missing capability. A driver that + backs the storage but refuses `PERSISTENT|COHERENT` is reported separately + from one that refuses the storage — that distinction is exactly T1 vs T2 for + DirectGLES. * **T0 — server imports.** The second process allocates an `AHardwareBuffer` BLOB (`CPU_READ_OFTEN|CPU_WRITE_OFTEN|GPU_DATA_BUFFER`), writes a pattern under `AHardwareBuffer_lock`, and sends it with @@ -31,24 +54,99 @@ project's CMake build graph. ways — CPU lock, `VkDeviceMemory` imported through `VK_ANDROID_external_memory_android_hardware_buffer`, and a GL buffer created with `eglGetNativeClientBufferANDROID` + `glBufferStorageExternalEXT` mapped - persistent/coherent — writes through each, and the allocating process verifies - every write. + persistent/coherent (the DirectGLES form of T0) — takes a GPU access, writes + through each, and the allocating process verifies every write with + `AHardwareBuffer_lock`. * **T3 — host pointer import.** If `VK_EXT_external_memory_host` is advertised, - imports a memfd-backed, alignment-corrected `mmap` region as a `VkDeviceMemory` - and maps it; also passes the memfd to the second process for a cross-process - round trip. + both directions are exercised: the importing process allocates the memfd + (`T3-external-memory-host`, plus a plain cross-process memfd round trip), and — + the direction that actually makes T3 a tier — the **client** allocates the + memfd, writes to it, and the **server** mmaps the received fd, imports the + client's host pointer into a `VkDeviceMemory`, reads what the client wrote, + writes back, and takes a GPU access on the client's memory + (`T3-client-memfd-server-import`). + +**Every tier row takes a real GPU access** before it can be `OK`: +`vkCmdCopyBuffer` out of the shared allocation into a private staging buffer +(mismatch ⇒ the GPU could not read what the peer wrote) plus `vkCmdFillBuffer` +into it, `vkQueueWaitIdle`, and an explicit host-read barrier; the peer then +checks the filled region through *its* mapping. Without it an `OK` would only +mean that a map call returned a pointer, not that the tier survives GPU use. Process topology mirrors the target design (the client spawns the server): the probe re-execs `/proc/self/exe --child=` and hands the child one end of a `socketpair` on fd 3. A bare `fork()` is not usable — neither side's Vulkan driver survives it, and both sides need live Vulkan. +## Reading the verdict + +`status` is one of `OK`, `PARTIAL`, `UNSUPPORTED`, `FAIL`, `SKIP`, and the rule +is deliberately strict: + +* **`OK`** — every *decisive* leg round-tripped bytes **in both directions** + (the allocating side's payload was visible to the other side, and the other + side's write came back). A successful map call with no byte ever compared is + never `OK`. +* **`PARTIAL`** — at least one decisive leg round-tripped, but not all. +* **`FAIL`** — no decisive leg round-tripped. A `FAIL` always names the failing + step and its driver error code in `why: …`. +* **`UNSUPPORTED`** — the route's extension is absent, or the driver never + advertised the handle type as `EXPORTABLE` and then declined it. The same rule + is applied at *every* export failure site: a decline on a handle type the + driver advertised as `EXPORTABLE` is a driver bug and reports `FAIL`; the same + decline on one it never advertised reports `UNSUPPORTED`. + +Each row starts with a per-leg trace, e.g. +`rawmmap[i]=no vkimport[D]=rt gpu[D]=rt` — `[D]` decisive, `[i]` informational, +`rt` = round-tripped, `read-only`/`write-only`/`no`/`notrun` otherwise. Driver +error codes are printed verbatim (`VkResult` names, `errno`, GL enums) — that is +the payload of the spike, so do not summarise them away. + +Two details worth knowing when reading T1 output: + +* the child reads through *both* the plain `mmap` and the imported + `VkDeviceMemory` before it writes through either, because a driver whose + exported fd maps at an offset would otherwise have its payload overwritten by + the probe's own first write, and the second read would report a false failure; +* the raw `mmap` leg is **informational for opaque-fd** and decisive only for + dma-buf. Vulkan forbids interpreting an opaque-fd payload outside the driver, + so a driver that refuses it is conformant and MobileGL would never take that + route; dma-buf is the opposite — a CPU mapping is the point of the handle type. + When the direct compare fails the child scans the mapping for the exporter's + payload and reports `payloadAt=`; `payloadAt=4096` with a clean Vulkan + import (lavapipe's answer) means the fd is shareable but its offset-0 is not + the allocation's base. + +## SELinux domain caveat (important) + +Run as `adb shell /data/local/tmp/extmem_probe`, this executes in the **`shell`** +SELinux domain (`u:r:shell:s0`), **not** the `untrusted_app` domain MobileGL +actually runs in. `shell` and `untrusted_app` do not share the same rules for +dmabuf/ashmem allocators, gralloc, and device nodes, so a route that works here +can still be denied in the app — and, less often, the reverse. The probe prints +the domain it actually got in the run-context header and repeats the caveat in +the summary; record it with the results. + +To answer the question for the real domain, the same binary has to be executed +from an app process. That is **not implemented here**: the intended vehicle is +the trace app's spike hook from the spike-A package — ship `extmem_probe` as a +`jniLib`/asset, exec it from the app's own uid with its stdout redirected to +`/sdcard/MG/extmem-probe.log`, and compare the summary table with the `adb +shell` one. Any row that differs between the two is an SELinux/domain finding, +not a driver finding. + ## Build and run ```sh ANDROID_NDK=$HOME/android-sdk/ndk/27.3.13750724 ./build_android.sh /tmp/extmem-build -adb -s push /tmp/extmem-build/extmem_probe /data/local/tmp/p0-extmem/ -adb -s shell /data/local/tmp/p0-extmem/extmem_probe +``` + +One line to push, run and collect on a device: + +```sh +S=; adb -s $S push /tmp/extmem-build/extmem_probe /data/local/tmp/extmem_probe \ + && adb -s $S shell "chmod 755 /data/local/tmp/extmem_probe && /data/local/tmp/extmem_probe; echo EXIT=\$?" \ + | tee out-$S.txt ``` There is also a host build (`cmake -S . -B ` with no toolchain file). It @@ -59,22 +157,14 @@ EGL_PLATFORM=surfaceless`) proves the harness reports a working route as working, which is what makes a device-side `FAIL` attributable to the device driver rather than to this program. It is not a substitute for a device run. +**Known host-build limitation.** On lavapipe + llvmpipe the two `T1-gles` rows +report `FAIL` with `glBufferStorageMemEXT -> GL_OUT_OF_MEMORY` on every rung of +the ladder, even though `GL_DEVICE_UUID_EXT` matches the Vulkan `deviceUUID`: +llvmpipe's GL does not implement importing a lavapipe opaque-fd allocation. +That is a Mesa interop gap, not a harness defect — the T1/T3 rows are the ones +the host run validates, and they must all read `OK`. The GLES legs are validated +only on the devices. + Options: `--size=BYTES` (default 65536; the payload is split into 4 KiB regions, -one per writer), `--only-t0` / `--only-t1` / `--only-t3`. - -Output is a per-route `RESULT ` line stream plus a -summary table; `status` is one of `OK`, `PARTIAL`, `UNSUPPORTED`, `FAIL`, `SKIP`. -Driver error codes are printed verbatim (`VkResult` names, `errno`, GL enums) — -that is the payload of the spike, so do not summarise them away. - -Two details worth knowing when reading T1 output: - -* the child reads through *both* the plain `mmap` and the imported - `VkDeviceMemory` before it writes through either, because a driver whose - exported fd maps at an offset would otherwise have its payload overwritten by - the probe's own first write, and the second read would report a false failure; -* when the direct compare fails, the child scans the mapping for the exporter's - payload and reports `payloadAt=`. `payloadAt=4096` with a clean Vulkan - import (lavapipe's answer) means the fd is shareable but its offset-0 is not - the allocation's base — a route that only works if that offset is - discoverable, which opaque-fd does not promise. +one per writer — A payload, B/C/D importer writes, E GPU fill, F in-process GL +write), `--only-t0` / `--only-t1` / `--only-t3` / `--only-gles`, `--no-gles`. diff --git a/tools/spikes/extmem_probe/extmem_probe.cpp b/tools/spikes/extmem_probe/extmem_probe.cpp index 2e28f0e9..62fa063c 100644 --- a/tools/spikes/extmem_probe/extmem_probe.cpp +++ b/tools/spikes/extmem_probe/extmem_probe.cpp @@ -1,18 +1,40 @@ // extmem_probe -- MobileGL disaggregation P0 spike B (plan-B §8.3, §11 P0). // // Question this program answers, per device: -// Can a server-allocated HOST_VISIBLE|HOST_COHERENT VkDeviceMemory be shared -// with another process and mapped there, and by which route? +// Can the memory behind AcquirePersistentMap be shared with another process +// and mapped there -- for BOTH backends -- and by which route? // // T1 server exports its own allocation (VkExportMemoryAllocateInfo + // vkGetMemoryFdKHR, opaque-fd and dma-buf, handed over SCM_RIGHTS; the // importer tries plain mmap() *and* a Vulkan import + vkMapMemory) +// T1-gles the same exported fd imported into GLES (GL_EXT_memory_object +// + GL_EXT_memory_object_fd: glCreateMemoryObjectsEXT + glImportMemoryFdEXT +// + glBufferStorageMemEXT + glMapBufferRange(PERSISTENT|COHERENT)), +// first in-process, then cross-process. DirectGLES ("Espryt") reaches +// AcquirePersistentMap through a GL mapping, not a VkDeviceMemory, so the +// Vulkan-only T1 answer does not decide the tier for it. // T0 server imports a client allocation (AHardwareBuffer BLOB sent over a // unix socket, imported into VkDeviceMemory via // VK_ANDROID_external_memory_android_hardware_buffer and into a GL buffer // via EGL_ANDROID_get_native_client_buffer + glBufferStorageExternalEXT) -// T3 server imports a client host mapping (VK_EXT_external_memory_host over -// a memfd-backed mmap region) +// T3 server imports a client host mapping (VK_EXT_external_memory_host); +// both directions: the process that imports allocates the memfd, and -- +// the direction that actually makes T3 a tier -- the *client* allocates +// the memfd and the *server* imports the client's host pointer. +// +// Every route that can reach OK also takes a real GPU access (vkCmdCopyBuffer +// out of the shared allocation + vkCmdFillBuffer into it, queue-idle, host-read +// barrier), so an OK verdict means the tier survives GPU use and not merely a +// successful map call. +// +// Verdict rule (deliberately strict): a route is OK only when every decisive +// leg round-tripped bytes in both directions, PARTIAL when at least one decisive +// leg did, FAIL otherwise -- and a FAIL always names the failing step and its +// driver error code. +// +// SELINUX CAVEAT: run from `adb shell`, this executes in the `shell` domain, not +// the `untrusted_app` domain MobileGL actually runs in. See README.md; the +// summary repeats it. // // Standalone: depends on nothing from MobileGL. Build with the NDK toolchain // (see CMakeLists.txt / build_android.sh), push to /data/local/tmp and run. @@ -27,10 +49,11 @@ # define VK_USE_PLATFORM_ANDROID_KHR 1 # define PROBE_HAVE_AHB 1 #else -// The probe is an Android deliverable; the host build exists only so the T1/T3 -// harness itself can be validated against a driver that is known to implement -// those routes (lavapipe), which is what makes a device-side FAIL attributable -// to the driver rather than to this program. T0 is Android-only by nature. +// The probe is an Android deliverable; the host build exists only so the +// T1/T1-gles/T3 harness itself can be validated against a driver that is known +// to implement those routes (lavapipe/llvmpipe), which is what makes a +// device-side FAIL attributable to the driver rather than to this program. +// T0 is Android-only by nature. # define PROBE_HAVE_AHB 0 #endif @@ -85,6 +108,7 @@ static void getProp(const char* name, char* out, size_t n) { #endif } +static void pr(const char* fmt, ...) __attribute__((format(printf, 1, 2))); static void pr(const char* fmt, ...) { char buf[4096]; va_list ap; @@ -107,6 +131,7 @@ static void record(const char* route, const char* status, const std::string& det pr("RESULT %-34s %-12s %s", route, status, detail.c_str()); } +static std::string fmt(const char* f, ...) __attribute__((format(printf, 1, 2))); static std::string fmt(const char* f, ...) { char buf[1024]; va_list ap; @@ -116,7 +141,10 @@ static std::string fmt(const char* f, ...) { return std::string(buf); } -static const char* vkStr(VkResult r) { +// Returns a fresh std::string per call: several vkStr() results routinely appear +// in one format call, and a shared static buffer would make all of them show the +// last one. +static std::string vkStr(VkResult r) { switch (r) { case VK_SUCCESS: return "VK_SUCCESS"; case VK_NOT_READY: return "VK_NOT_READY"; @@ -139,27 +167,58 @@ static const char* vkStr(VkResult r) { case VK_ERROR_INVALID_EXTERNAL_HANDLE: return "VK_ERROR_INVALID_EXTERNAL_HANDLE"; case VK_ERROR_FRAGMENTATION: return "VK_ERROR_FRAGMENTATION"; case VK_ERROR_INVALID_OPAQUE_CAPTURE_ADDRESS: return "VK_ERROR_INVALID_OPAQUE_CAPTURE_ADDRESS"; - default: { - static char tmp[32]; - snprintf(tmp, sizeof(tmp), "VkResult(%d)", (int)r); - return tmp; - } + default: return fmt("VkResult(%d)", (int)r); } } +static std::string glErrStr(GLenum e) { + switch (e) { + case GL_NO_ERROR: return "GL_NO_ERROR"; + case GL_INVALID_ENUM: return "GL_INVALID_ENUM"; + case GL_INVALID_VALUE: return "GL_INVALID_VALUE"; + case GL_INVALID_OPERATION: return "GL_INVALID_OPERATION"; + case GL_OUT_OF_MEMORY: return "GL_OUT_OF_MEMORY"; + case GL_INVALID_FRAMEBUFFER_OPERATION: return "GL_INVALID_FRAMEBUFFER_OPERATION"; + default: return fmt("GL(0x%04x)", (unsigned)e); + } +} + +// drains and returns the last error, so one failing call cannot be blamed on the +// previous one +static GLenum glDrain() { + GLenum last = GL_NO_ERROR, e; + while ((e = glGetError()) != GL_NO_ERROR) last = e; + return last; +} + +static std::string readSmallFile(const char* path) { + int fd = open(path, O_RDONLY); + if (fd < 0) return fmt("<%s: errno=%d>", path, errno); + char buf[256]; + ssize_t n = read(fd, buf, sizeof(buf) - 1); + close(fd); + if (n <= 0) return ""; + buf[n] = 0; + while (n > 0 && (buf[n - 1] == '\n' || buf[n - 1] == 0)) buf[--n] = 0; + return std::string(buf); +} + // --------------------------------------------------------------------------- // payload patterns // --------------------------------------------------------------------------- static const uint64_t kRegion = 4096; // bytes per verification region +static const uint64_t kRegionCount = 8; // A..F plus slack static const uint64_t kDefaultSize = 65536; // region indices inside the shared allocation enum { - REG_A = 0, // first writer's payload - REG_B = 1, // importer write through the plain host mapping (mmap / AHB lock) + REG_A = 0, // first writer's payload (CPU, allocating side) + REG_B = 1, // importer write through the plain host mapping (mmap / AHB lock / vk map) REG_C = 2, // importer write through the imported Vulkan mapping - REG_D = 3, // importer write through the imported GL mapping + REG_D = 3, // importer write through the imported GL mapping (cross-process) + REG_E = 4, // GPU write (vkCmdFillBuffer) + REG_F = 5, // in-process GL-import write }; static void fillPattern(void* p, uint64_t bytes, uint32_t seed) { @@ -186,6 +245,63 @@ static int64_t checkRegion(const void* base, int region, uint32_t seed) { return checkPattern((const uint8_t*)base + region * kRegion, kRegion, seed); } +// vkCmdFillBuffer writes a repeating 32-bit word; -1 on match, else the first +// mismatching word index * 4 +static int64_t checkFillWord(const void* base, int region, uint32_t word) { + const uint32_t* w = (const uint32_t*)((const uint8_t*)base + region * kRegion); + for (uint64_t i = 0; i < kRegion / 4; ++i) + if (w[i] != word) return (int64_t)(i * 4); + return -1; +} + +// --------------------------------------------------------------------------- +// verdict: decisive legs must round-trip bytes, in both directions +// --------------------------------------------------------------------------- + +struct Leg { + std::string name; + bool decisive = false; // counted by the verdict; informational legs are not + bool attempted = false; + bool readOk = false; // the allocating side's bytes were visible to the other side + bool writeOk = false; // the other side's bytes came back + std::string fail; // failing step + driver error code +}; + +static const char* legVerdict(const std::vector& legs, std::string* why) { + int decisive = 0, round = 0; + std::string bad; + for (const Leg& l : legs) { + if (!l.decisive) continue; + ++decisive; + if (l.attempted && l.readOk && l.writeOk) { + ++round; + } else { + if (!bad.empty()) bad += "; "; + bad += l.name + "=" + (l.fail.empty() ? std::string("no round trip") : l.fail); + } + } + if (why) *why = bad; + if (decisive == 0) return "SKIP"; + if (round == decisive) return "OK"; + if (round > 0) return "PARTIAL"; + return "FAIL"; +} + +// compact per-leg trace that stays in the summary line +static std::string legTrace(const std::vector& legs) { + std::string s; + for (const Leg& l : legs) { + if (!s.empty()) s += " "; + const char* v = !l.attempted ? "notrun" + : (l.readOk && l.writeOk) ? "rt" + : l.readOk ? "read-only" + : l.writeOk ? "write-only" + : "no"; + s += l.name + "[" + (l.decisive ? "D" : "i") + "]=" + v; + } + return s; +} + // --------------------------------------------------------------------------- // socket message plumbing // --------------------------------------------------------------------------- @@ -199,6 +315,12 @@ enum MsgTag : uint32_t { MSG_T0_RESULT = 6, MSG_T3_OFFER = 7, MSG_T3_RESULT = 8, + MSG_T1GL_OFFER = 9, + MSG_T1GL_RESULT = 10, + MSG_T3C_REQUEST = 11, + MSG_T3C_READY = 12, + MSG_T3C_VERIFY = 13, + MSG_T3C_RESULT = 14, MSG_BYE = 99, }; @@ -360,6 +482,8 @@ struct VkCtx { VkPhysicalDevice phys = VK_NULL_HANDLE; VkDevice device = VK_NULL_HANDLE; uint32_t queueFamily = 0; + VkQueue queue = VK_NULL_HANDLE; + VkCommandPool cmdPool = VK_NULL_HANDLE; VkPhysicalDeviceMemoryProperties memProps{}; VkPhysicalDeviceProperties props{}; uint8_t deviceUUID[VK_UUID_SIZE]{}; @@ -417,7 +541,7 @@ static bool vkCtxInit(VkCtx& c, bool verbose) { VkResult r = vkCreateInstance(&ici, nullptr, &c.instance); if (r != VK_SUCCESS) { - pr("vkCreateInstance failed: %s", vkStr(r)); + pr("vkCreateInstance failed: %s", vkStr(r).c_str()); return false; } @@ -506,10 +630,21 @@ static bool vkCtxInit(VkCtx& c, bool verbose) { r = vkCreateDevice(c.phys, &dci, nullptr, &c.device); if (r != VK_SUCCESS) { - pr("vkCreateDevice failed: %s", vkStr(r)); + pr("vkCreateDevice failed: %s", vkStr(r).c_str()); return false; } + vkGetDeviceQueue(c.device, c.queueFamily, 0, &c.queue); + VkCommandPoolCreateInfo cpi{}; + cpi.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO; + cpi.flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT; + cpi.queueFamilyIndex = c.queueFamily; + VkResult pr_ = vkCreateCommandPool(c.device, &cpi, nullptr, &c.cmdPool); + if (pr_ != VK_SUCCESS) { + c.cmdPool = VK_NULL_HANDLE; + pr("vkCreateCommandPool failed: %s (GPU touch will be skipped)", vkStr(pr_).c_str()); + } + c.pGetMemoryFdKHR = (PFN_vkGetMemoryFdKHR)vkGetDeviceProcAddr(c.device, "vkGetMemoryFdKHR"); c.pGetMemoryFdPropertiesKHR = (PFN_vkGetMemoryFdPropertiesKHR)vkGetDeviceProcAddr(c.device, "vkGetMemoryFdPropertiesKHR"); @@ -526,15 +661,17 @@ static bool vkCtxInit(VkCtx& c, bool verbose) { VK_VERSION_PATCH(c.props.apiVersion), c.props.driverVersion, c.props.vendorID); char uuid[64] = {0}; for (uint32_t i = 0; i < VK_UUID_SIZE; ++i) snprintf(uuid + i * 2, 3, "%02x", c.deviceUUID[i]); - pr("deviceUUID=%s minImportedHostPointerAlignment=%llu", uuid, - (unsigned long long)c.minImportedHostPointerAlignment); + pr("deviceUUID=%s minImportedHostPointerAlignment=%llu queueFamily=%u", uuid, + (unsigned long long)c.minImportedHostPointerAlignment, c.queueFamily); } return true; } static void vkCtxDestroy(VkCtx& c) { + if (c.cmdPool) vkDestroyCommandPool(c.device, c.cmdPool, nullptr); if (c.device) vkDestroyDevice(c.device, nullptr); if (c.instance) vkDestroyInstance(c.instance, nullptr); + c.cmdPool = VK_NULL_HANDLE; c.device = VK_NULL_HANDLE; c.instance = VK_NULL_HANDLE; } @@ -553,6 +690,275 @@ static const VkBufferUsageFlags kProbeBufferUsage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_INDEX_BUFFER_BIT | VK_BUFFER_USAGE_VERTEX_BUFFER_BIT; +// --------------------------------------------------------------------------- +// GPU touch: prove the shared allocation survives a real GPU access +// +// The GPU copies `readRegion` into a private staging buffer (so a mismatch means +// the GPU could not read what the host/peer wrote) and fills `fillRegion` with a +// known word (so the caller can check, through whichever mapping it is testing, +// that a GPU write lands in the shared pages). Without this an OK verdict would +// only prove that a map call returned a pointer. +// --------------------------------------------------------------------------- + +struct GpuTouch { + bool ran = false; + VkResult submitResult = VK_NOT_READY; + int64_t readMismatch = -3; // -1 match, -3 never ran + std::string fail; +}; + +static GpuTouch gpuTouch(VkCtx& c, VkBuffer buf, int readRegion, uint32_t readSeed, int fillRegion, + uint32_t fillWord) { + GpuTouch g; + if (buf == VK_NULL_HANDLE || c.cmdPool == VK_NULL_HANDLE || c.queue == VK_NULL_HANDLE) { + g.fail = "no buffer/queue/command pool for the GPU touch"; + return g; + } + + // private host-visible staging target for the read-back + VkBufferCreateInfo sbi{}; + sbi.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; + sbi.size = kRegion; + sbi.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT; + VkBuffer staging = VK_NULL_HANDLE; + VkResult r = vkCreateBuffer(c.device, &sbi, nullptr, &staging); + if (r != VK_SUCCESS) { + g.fail = "staging vkCreateBuffer=" + vkStr(r); + return g; + } + VkMemoryRequirements sreq{}; + vkGetBufferMemoryRequirements(c.device, staging, &sreq); + int sType = pickMemType(c.memProps, sreq.memoryTypeBits, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); + if (sType < 0) { + vkDestroyBuffer(c.device, staging, nullptr); + g.fail = fmt("no HOST_VISIBLE|HOST_COHERENT staging type in bits=0x%x", sreq.memoryTypeBits); + return g; + } + VkMemoryAllocateInfo smai{}; + smai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; + smai.allocationSize = sreq.size; + smai.memoryTypeIndex = (uint32_t)sType; + VkDeviceMemory smem = VK_NULL_HANDLE; + r = vkAllocateMemory(c.device, &smai, nullptr, &smem); + if (r != VK_SUCCESS) { + vkDestroyBuffer(c.device, staging, nullptr); + g.fail = "staging vkAllocateMemory=" + vkStr(r); + return g; + } + vkBindBufferMemory(c.device, staging, smem, 0); + + VkCommandBufferAllocateInfo cai{}; + cai.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; + cai.commandPool = c.cmdPool; + cai.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; + cai.commandBufferCount = 1; + VkCommandBuffer cmd = VK_NULL_HANDLE; + r = vkAllocateCommandBuffers(c.device, &cai, &cmd); + if (r != VK_SUCCESS) { + vkFreeMemory(c.device, smem, nullptr); + vkDestroyBuffer(c.device, staging, nullptr); + g.fail = "vkAllocateCommandBuffers=" + vkStr(r); + return g; + } + + VkCommandBufferBeginInfo bi{}; + bi.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; + bi.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; + vkBeginCommandBuffer(cmd, &bi); + + // host writes are made visible to the device by the queue submit itself for + // HOST_COHERENT memory, but the shared allocation may be imported and + // non-coherent, so ask for it explicitly + VkMemoryBarrier pre{}; + pre.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; + pre.srcAccessMask = VK_ACCESS_HOST_WRITE_BIT; + pre.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT; + vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_HOST_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 1, &pre, 0, nullptr, 0, + nullptr); + + VkBufferCopy copy{}; + copy.srcOffset = (VkDeviceSize)(readRegion * kRegion); + copy.dstOffset = 0; + copy.size = kRegion; + vkCmdCopyBuffer(cmd, buf, staging, 1, ©); + vkCmdFillBuffer(cmd, buf, (VkDeviceSize)(fillRegion * kRegion), (VkDeviceSize)kRegion, fillWord); + + // device writes must be made visible to the host explicitly + VkMemoryBarrier post{}; + post.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; + post.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + post.dstAccessMask = VK_ACCESS_HOST_READ_BIT; + vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &post, 0, nullptr, 0, + nullptr); + vkEndCommandBuffer(cmd); + + VkSubmitInfo si{}; + si.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; + si.commandBufferCount = 1; + si.pCommandBuffers = &cmd; + g.submitResult = vkQueueSubmit(c.queue, 1, &si, VK_NULL_HANDLE); + if (g.submitResult == VK_SUCCESS) { + VkResult wr = vkQueueWaitIdle(c.queue); + if (wr != VK_SUCCESS) { + g.fail = "vkQueueWaitIdle=" + vkStr(wr); + } else { + void* sp = nullptr; + VkResult mr = vkMapMemory(c.device, smem, 0, VK_WHOLE_SIZE, 0, &sp); + if (mr == VK_SUCCESS && sp) { + g.readMismatch = checkPattern(sp, kRegion, readSeed); + vkUnmapMemory(c.device, smem); + g.ran = true; + if (g.readMismatch != -1) + g.fail = fmt("GPU copy out of the shared allocation mismatched at byte %lld", + (long long)g.readMismatch); + } else { + g.fail = "staging vkMapMemory=" + vkStr(mr); + } + } + } else { + g.fail = "vkQueueSubmit=" + vkStr(g.submitResult); + } + + vkFreeCommandBuffers(c.device, c.cmdPool, 1, &cmd); + vkFreeMemory(c.device, smem, nullptr); + vkDestroyBuffer(c.device, staging, nullptr); + return g; +} + +// --------------------------------------------------------------------------- +// exportable HOST_VISIBLE|HOST_COHERENT allocation + fd +// --------------------------------------------------------------------------- + +struct ExportAlloc { + VkBuffer buf = VK_NULL_HANDLE; + VkDeviceMemory mem = VK_NULL_HANDLE; + void* host = nullptr; + uint64_t allocationSize = 0; + uint64_t bufferSize = 0; + uint32_t memoryTypeIndex = 0; + uint32_t memoryTypeBits = 0; + bool dedicated = false; + int fd = -1; + bool advertisedExportable = false; + bool advertisedImportable = false; + VkResult bindResult = VK_SUCCESS; + std::string fail; // empty on success +}; + +// A failure on a handle type the driver advertised as EXPORTABLE is a driver +// bug (FAIL); the same failure on one it never advertised is simply the route +// not being there (UNSUPPORTED). One rule, used at every export failure site. +static const char* exportFailStatus(const ExportAlloc& a) { + return a.advertisedExportable ? "FAIL" : "UNSUPPORTED"; +} + +static bool exportHostVisible(VkCtx& c, VkExternalMemoryHandleTypeFlagBits ht, uint64_t size, const char* tag, + ExportAlloc& a) { + a.bufferSize = size; + + VkPhysicalDeviceExternalBufferInfo ebi{}; + ebi.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_BUFFER_INFO; + ebi.usage = kProbeBufferUsage; + ebi.handleType = ht; + VkExternalBufferProperties ebp{}; + ebp.sType = VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES; + vkGetPhysicalDeviceExternalBufferProperties(c.phys, &ebi, &ebp); + a.advertisedExportable = + (ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) != 0; + a.advertisedImportable = + (ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT) != 0; + a.dedicated = + (ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT) != 0; + pr("%s advertisedExportable=%d importable=%d dedicatedOnly=%d", tag, (int)a.advertisedExportable, + (int)a.advertisedImportable, (int)a.dedicated); + + VkExternalMemoryBufferCreateInfo ext{}; + ext.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO; + ext.handleTypes = ht; + VkBufferCreateInfo bci{}; + bci.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; + bci.pNext = &ext; + bci.size = size; + bci.usage = kProbeBufferUsage; + bci.sharingMode = VK_SHARING_MODE_EXCLUSIVE; + + VkResult r = vkCreateBuffer(c.device, &bci, nullptr, &a.buf); + if (r != VK_SUCCESS) { + a.buf = VK_NULL_HANDLE; + a.fail = "vkCreateBuffer(external)=" + vkStr(r); + return false; + } + VkMemoryRequirements req{}; + vkGetBufferMemoryRequirements(c.device, a.buf, &req); + a.allocationSize = req.size; + a.memoryTypeBits = req.memoryTypeBits; + int typeIdx = pickMemType(c.memProps, req.memoryTypeBits, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); + if (typeIdx < 0) { + a.fail = fmt("no HOST_VISIBLE|HOST_COHERENT memory type in bits=0x%x", req.memoryTypeBits); + return false; + } + a.memoryTypeIndex = (uint32_t)typeIdx; + pr("%s memReq size=%llu align=%llu typeBits=0x%x -> type %d", tag, (unsigned long long)req.size, + (unsigned long long)req.alignment, req.memoryTypeBits, typeIdx); + + VkExportMemoryAllocateInfo exportInfo{}; + exportInfo.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO; + exportInfo.handleTypes = ht; + VkMemoryDedicatedAllocateInfo ded{}; + ded.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; + ded.buffer = a.buf; + if (a.dedicated) exportInfo.pNext = &ded; + + VkMemoryAllocateInfo mai{}; + mai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; + mai.pNext = &exportInfo; + mai.allocationSize = req.size; + mai.memoryTypeIndex = a.memoryTypeIndex; + + r = vkAllocateMemory(c.device, &mai, nullptr, &a.mem); + if (r != VK_SUCCESS) { + a.mem = VK_NULL_HANDLE; + a.fail = fmt("vkAllocateMemory(export)=%s (advertisedExportable=%d)", vkStr(r).c_str(), + (int)a.advertisedExportable); + return false; + } + a.bindResult = vkBindBufferMemory(c.device, a.buf, a.mem, 0); + if (a.bindResult != VK_SUCCESS) pr("%s vkBindBufferMemory=%s (continuing)", tag, vkStr(a.bindResult).c_str()); + + r = vkMapMemory(c.device, a.mem, 0, VK_WHOLE_SIZE, 0, &a.host); + if (r != VK_SUCCESS || !a.host) { + a.host = nullptr; + a.fail = "exporter-side vkMapMemory=" + vkStr(r); + return false; + } + + VkMemoryGetFdInfoKHR gfi{}; + gfi.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR; + gfi.memory = a.mem; + gfi.handleType = ht; + r = c.pGetMemoryFdKHR(c.device, &gfi, &a.fd); + if (r != VK_SUCCESS || a.fd < 0) { + a.fd = -1; + a.fail = fmt("vkGetMemoryFdKHR=%s (advertisedExportable=%d)", vkStr(r).c_str(), (int)a.advertisedExportable); + return false; + } + pr("%s exported fd=%d -> %s", tag, a.fd, describeFd(a.fd).c_str()); + return true; +} + +static void freeExportAlloc(VkCtx& c, ExportAlloc& a) { + if (a.fd >= 0) close(a.fd); + if (a.host) vkUnmapMemory(c.device, a.mem); + if (a.mem) vkFreeMemory(c.device, a.mem, nullptr); + if (a.buf) vkDestroyBuffer(c.device, a.buf, nullptr); + a.fd = -1; + a.host = nullptr; + a.mem = VK_NULL_HANDLE; + a.buf = VK_NULL_HANDLE; +} + // --------------------------------------------------------------------------- // GLES / EGL context // --------------------------------------------------------------------------- @@ -568,6 +974,15 @@ struct GlCtx { PFNEGLGETNATIVECLIENTBUFFERANDROIDPROC pGetNativeClientBuffer = nullptr; PFNGLBUFFERSTORAGEEXTERNALEXTPROC pBufferStorageExternal = nullptr; + // GL_EXT_memory_object / GL_EXT_memory_object_fd + PFNGLCREATEMEMORYOBJECTSEXTPROC pCreateMemoryObjects = nullptr; + PFNGLDELETEMEMORYOBJECTSEXTPROC pDeleteMemoryObjects = nullptr; + PFNGLMEMORYOBJECTPARAMETERIVEXTPROC pMemoryObjectParameteriv = nullptr; + PFNGLBUFFERSTORAGEMEMEXTPROC pBufferStorageMem = nullptr; + PFNGLIMPORTMEMORYFDEXTPROC pImportMemoryFd = nullptr; + PFNGLGETUNSIGNEDBYTEI_VEXTPROC pGetUnsignedBytei_v = nullptr; + void (GL_APIENTRYP pMemoryBarrier)(GLbitfield) = nullptr; + bool hasGl(const char* n) const { for (auto& s : glExts) if (s == n) return true; @@ -578,6 +993,18 @@ struct GlCtx { if (s == n) return true; return false; } + // everything the T1 GLES leg needs + bool canImportFd() const { + return hasGl("GL_EXT_memory_object") && hasGl("GL_EXT_memory_object_fd") && pCreateMemoryObjects && + pImportMemoryFd && pBufferStorageMem; + } + std::string missingForImportFd() const { + return fmt("GL_EXT_memory_object=%d GL_EXT_memory_object_fd=%d GL_EXT_buffer_storage=%d " + "glCreateMemoryObjectsEXT=%d glImportMemoryFdEXT=%d glBufferStorageMemEXT=%d", + (int)hasGl("GL_EXT_memory_object"), (int)hasGl("GL_EXT_memory_object_fd"), + (int)hasGl("GL_EXT_buffer_storage"), (int)(pCreateMemoryObjects != nullptr), + (int)(pImportMemoryFd != nullptr), (int)(pBufferStorageMem != nullptr)); + } }; static void splitExts(const char* s, std::vector& out) { @@ -661,6 +1088,15 @@ static bool glCtxInit(GlCtx& g) { (PFNEGLGETNATIVECLIENTBUFFERANDROIDPROC)eglGetProcAddress("eglGetNativeClientBufferANDROID"); g.pBufferStorageExternal = (PFNGLBUFFERSTORAGEEXTERNALEXTPROC)eglGetProcAddress("glBufferStorageExternalEXT"); + g.pCreateMemoryObjects = (PFNGLCREATEMEMORYOBJECTSEXTPROC)eglGetProcAddress("glCreateMemoryObjectsEXT"); + g.pDeleteMemoryObjects = (PFNGLDELETEMEMORYOBJECTSEXTPROC)eglGetProcAddress("glDeleteMemoryObjectsEXT"); + g.pMemoryObjectParameteriv = + (PFNGLMEMORYOBJECTPARAMETERIVEXTPROC)eglGetProcAddress("glMemoryObjectParameterivEXT"); + g.pBufferStorageMem = (PFNGLBUFFERSTORAGEMEMEXTPROC)eglGetProcAddress("glBufferStorageMemEXT"); + g.pImportMemoryFd = (PFNGLIMPORTMEMORYFDEXTPROC)eglGetProcAddress("glImportMemoryFdEXT"); + g.pGetUnsignedBytei_v = (PFNGLGETUNSIGNEDBYTEI_VEXTPROC)eglGetProcAddress("glGetUnsignedBytei_vEXT"); + // ES 3.1 core, but loaded dynamically so an ES 3.0 context still links + g.pMemoryBarrier = (void(GL_APIENTRYP)(GLbitfield))eglGetProcAddress("glMemoryBarrier"); return true; } @@ -674,6 +1110,191 @@ static void glCtxDestroy(GlCtx& g) { g.dpy = EGL_NO_DISPLAY; } +// GL_DEVICE_UUID_EXT must equal the Vulkan deviceUUID for an fd import to be +// legal; a mismatch is the usual reason glImportMemoryFdEXT declines. +static std::string glDeviceUuidReport(GlCtx& g, const uint8_t* vkUuid, bool* matched) { + if (matched) *matched = false; + if (!g.hasGl("GL_EXT_memory_object") || !g.pGetUnsignedBytei_v) return "unavailable"; + GLint n = 0; + glDrain(); + glGetIntegerv(GL_NUM_DEVICE_UUIDS_EXT, &n); + if (glDrain() != GL_NO_ERROR || n <= 0) return "GL_NUM_DEVICE_UUIDS_EXT unreadable"; + std::string out; + for (GLint i = 0; i < n; ++i) { + GLubyte uuid[GL_UUID_SIZE_EXT] = {0}; + g.pGetUnsignedBytei_v(GL_DEVICE_UUID_EXT, (GLuint)i, uuid); + char hex[2 * GL_UUID_SIZE_EXT + 1] = {0}; + for (int k = 0; k < GL_UUID_SIZE_EXT; ++k) snprintf(hex + k * 2, 3, "%02x", uuid[k]); + if (!out.empty()) out += ","; + out += hex; + if (!memcmp(uuid, vkUuid, GL_UUID_SIZE_EXT) && matched) *matched = true; + } + return out; +} + +// --------------------------------------------------------------------------- +// GL side of T1: import an exported fd as GL buffer storage and map it +// --------------------------------------------------------------------------- + +struct GlImport { + bool memObjOk = false; + bool storageOk = false; + bool mapOk = false; + bool persistentCoherent = false; // the PERSISTENT|COHERENT map is what AcquirePersistentMap needs + GLenum errImport = GL_NO_ERROR; + GLenum errStorage = GL_NO_ERROR; + GLenum errMap = GL_NO_ERROR; // error of the PERSISTENT|COHERENT attempt + GLenum errMap2 = GL_NO_ERROR; // error of the plain MAP_READ|MAP_WRITE fallback + GLuint memObj = 0; + GLuint buf = 0; + void* ptr = nullptr; + uint64_t mappedSize = 0; + std::string variant; // which phrasing the driver accepted + std::string ladder; // every phrasing tried, with its error + std::string fail; +}; + +// Borrows `fd` (dups it per attempt; the caller keeps ownership). +// +// Drivers disagree about how this call has to be phrased -- whether the memory +// object must be flagged dedicated, and whether the buffer may be smaller than +// the imported allocation -- and a probe that tried only one phrasing would +// report a driver preference as a missing capability. So walk the ladder and +// report which rung the driver accepted. +static void glImportFdBuffer(GlCtx& g, int fd, uint64_t allocationSize, uint64_t bufferSize, bool dedicatedHint, + GlImport& o) { + struct Attempt { + bool dedicated; + uint64_t importSize; + uint64_t storageSize; + }; + std::vector attempts; + // some drivers validate the imported size against the fd's own size rather + // than against the exporter's VkMemoryRequirements::size + off_t fdSize = lseek(fd, 0, SEEK_END); + if (fdSize > 0) lseek(fd, 0, SEEK_SET); + std::vector importSizes{allocationSize}; + if (fdSize > 0 && (uint64_t)fdSize != allocationSize) importSizes.push_back((uint64_t)fdSize); + for (uint64_t imp : importSizes) { + for (bool ded : {dedicatedHint, !dedicatedHint}) { + attempts.push_back(Attempt{ded, imp, bufferSize}); + if (imp != bufferSize) attempts.push_back(Attempt{ded, imp, imp}); + } + } + glGenBuffers(1, &o.buf); + + for (const Attempt& at : attempts) { + std::string tag = fmt("[ded=%d imp=%llu store=%llu]", (int)at.dedicated, (unsigned long long)at.importSize, + (unsigned long long)at.storageSize); + glDrain(); + GLuint mo = 0; + g.pCreateMemoryObjects(1, &mo); + if (at.dedicated && g.pMemoryObjectParameteriv) { + GLint yes = GL_TRUE; + g.pMemoryObjectParameteriv(mo, GL_DEDICATED_MEMORY_OBJECT_EXT, &yes); + glDrain(); + } + int dupFd = dup(fd); + g.pImportMemoryFd(mo, (GLuint64)at.importSize, GL_HANDLE_TYPE_OPAQUE_FD_EXT, (GLint)dupFd); + GLenum eImport = glDrain(); + if (eImport != GL_NO_ERROR) { + // Deliberately NOT closed: EXT_memory_object_fd transfers ownership of + // the fd to the implementation and does not say whether that still + // happens when the import fails, and Mesa closes it either way. A + // double close would land on whatever fd the allocator handed out + // next -- the socket, in this program. At most a handful of rungs + // run, so leaking the dup is the cheap, safe side of that trade. + (void)dupFd; + if (g.pDeleteMemoryObjects) g.pDeleteMemoryObjects(1, &mo); + glDrain(); + if (!o.memObjOk) o.errImport = eImport; + o.ladder += tag + "import=" + glErrStr(eImport) + " "; + continue; + } + o.memObjOk = true; + o.errImport = GL_NO_ERROR; + + glBindBuffer(GL_ARRAY_BUFFER, o.buf); + glDrain(); + g.pBufferStorageMem(GL_ARRAY_BUFFER, (GLsizeiptr)at.storageSize, mo, 0); + GLenum eStorage = glDrain(); + o.ladder += tag + "storage=" + glErrStr(eStorage) + " "; + if (eStorage != GL_NO_ERROR) { + o.errStorage = eStorage; + if (g.pDeleteMemoryObjects) g.pDeleteMemoryObjects(1, &mo); + glDrain(); + // storage is immutable once it takes, so a failed attempt needs a + // fresh buffer name before the next rung + glBindBuffer(GL_ARRAY_BUFFER, 0); + glDeleteBuffers(1, &o.buf); + glGenBuffers(1, &o.buf); + continue; + } + o.memObj = mo; + o.errStorage = GL_NO_ERROR; + o.storageOk = true; + o.mappedSize = at.storageSize; + o.variant = tag; + break; + } + + if (!o.memObjOk) { + o.fail = "glImportMemoryFdEXT -> " + glErrStr(o.errImport) + " (ladder: " + o.ladder + ")"; + return; + } + if (!o.storageOk) { + o.fail = "glBufferStorageMemEXT -> " + glErrStr(o.errStorage) + " (ladder: " + o.ladder + ")"; + return; + } + bufferSize = o.mappedSize; + + o.ptr = glMapBufferRange(GL_ARRAY_BUFFER, 0, (GLsizeiptr)bufferSize, + GL_MAP_READ_BIT | GL_MAP_WRITE_BIT | GL_MAP_PERSISTENT_BIT_EXT | + GL_MAP_COHERENT_BIT_EXT); + o.errMap = glDrain(); + if (o.ptr) { + o.mapOk = true; + o.persistentCoherent = true; + return; + } + // A driver may back the storage but refuse the persistent/coherent flags -- + // that is exactly the T1/T2 distinction for DirectGLES, so it is reported + // separately rather than folded into one failure. + o.ptr = glMapBufferRange(GL_ARRAY_BUFFER, 0, (GLsizeiptr)bufferSize, GL_MAP_READ_BIT | GL_MAP_WRITE_BIT); + o.errMap2 = glDrain(); + if (o.ptr) { + o.mapOk = true; + o.fail = "PERSISTENT|COHERENT map refused (" + glErrStr(o.errMap) + "), only a scoped map works"; + } else { + o.fail = "glMapBufferRange persistent -> " + glErrStr(o.errMap) + ", plain -> " + glErrStr(o.errMap2); + } +} + +static void glImportPublish(GlCtx& g, GlImport& o) { + if (!o.mapOk) return; + if (o.persistentCoherent) { + if (g.pMemoryBarrier) g.pMemoryBarrier(GL_ALL_BARRIER_BITS); + } else { + glUnmapBuffer(GL_ARRAY_BUFFER); + o.ptr = nullptr; + } + glFinish(); +} + +static void glImportRelease(GlCtx& g, GlImport& o) { + if (o.buf) { + glBindBuffer(GL_ARRAY_BUFFER, o.buf); + if (o.ptr) glUnmapBuffer(GL_ARRAY_BUFFER); + glBindBuffer(GL_ARRAY_BUFFER, 0); + glDeleteBuffers(1, &o.buf); + } + if (o.memObj && g.pDeleteMemoryObjects) g.pDeleteMemoryObjects(1, &o.memObj); + glDrain(); + o.buf = 0; + o.memObj = 0; + o.ptr = nullptr; +} + // --------------------------------------------------------------------------- // child spawn // --------------------------------------------------------------------------- @@ -739,7 +1360,7 @@ static std::string reapChild(pid_t pid) { } // --------------------------------------------------------------------------- -// T1 payloads +// wire payloads // --------------------------------------------------------------------------- struct T1Offer { @@ -751,6 +1372,8 @@ struct T1Offer { uint32_t seedC; // pattern the child must write in REG_C (via imported vkMapMemory) uint32_t memoryTypeIndex; uint32_t memoryTypeBits; + uint32_t gpuWord; // word the GPU filled REG_E with + uint32_t gpuRan; // 0 -> REG_E carries nothing, do not check it }; struct T1Result { @@ -759,6 +1382,7 @@ struct T1Result { int32_t mmapErrno; int64_t mmapMismatch; // -1 == data matched int64_t mmapPatternOffset; // where the exporter's payload really starts in the mapping, -1 = not found + int64_t mmapGpuMismatch; // REG_E through the plain mapping (-2 = not checked) int32_t vkInitOk; int32_t fdPropsResult; // VkResult of vkGetMemoryFdPropertiesKHR uint32_t fdMemoryTypeBits; @@ -766,11 +1390,36 @@ struct T1Result { int32_t bindResult; int32_t mapResult; int64_t vkMismatch; // -1 == data matched + int64_t vkGpuMismatch; // REG_E through the imported mapping (-2 = not checked) int32_t wroteB; int32_t wroteC; char note[384]; }; +struct T1GlOffer { + uint64_t allocationSize; + uint64_t bufferSize; + uint32_t seedA; + uint32_t seedD; // the child writes REG_D through the imported GL mapping + uint32_t gpuWord; + uint32_t gpuRan; + uint32_t dedicated; +}; + +struct T1GlResult { + int32_t glInitOk; + int32_t haveExts; + int32_t memObjOk; + int32_t storageOk; + int32_t mapOk; + int32_t persistentCoherent; + uint32_t errImport, errStorage, errMap, errMap2; + int64_t mismatchA; + int64_t mismatchGpu; + int32_t wroteD; + char note[640]; +}; + struct T0Request { uint64_t size; uint32_t seedA; // pattern the child writes through AHardwareBuffer_lock @@ -785,10 +1434,11 @@ struct T0Alloc { }; struct T0Verify { - uint32_t seedB; // parent wrote REG_B through the imported vkMapMemory - uint32_t seedC; // parent wrote REG_C through the imported GL mapping - uint32_t seedD; // parent wrote REG_D through AHardwareBuffer_lock - uint32_t writtenMask; // bit0=B bit1=C bit2=D + uint32_t seedB; // parent wrote REG_B through the imported vkMapMemory + uint32_t seedC; // parent wrote REG_C through the imported GL mapping + uint32_t seedD; // parent wrote REG_D through AHardwareBuffer_lock + uint32_t gpuWord; // the GPU filled REG_E with this + uint32_t writtenMask; // bit0=B bit1=C bit2=D bit3=E(gpu) }; struct T0Result { @@ -797,6 +1447,7 @@ struct T0Result { int64_t mismatchB; int64_t mismatchC; int64_t mismatchD; + int64_t mismatchE; char note[192]; }; @@ -804,29 +1455,75 @@ struct T3Offer { uint64_t size; uint32_t seedA; uint32_t seedB; + uint32_t gpuWord; + uint32_t gpuRan; }; struct T3Result { int32_t mmapOk; int32_t mmapErrno; int64_t mismatch; + int64_t gpuMismatch; char note[192]; }; +// T3 in the direction that makes it a tier: the CLIENT allocates, the SERVER +// imports the client's host pointer. +struct T3cRequest { + uint64_t size; // must be a multiple of minImportedHostPointerAlignment + uint32_t seedA; // the child writes REG_A +}; + +struct T3cReady { + int32_t ok; + int32_t err; + uint64_t size; + char note[160]; +}; + +struct T3cVerify { + uint32_t seedB; // the parent wrote REG_B through the imported VkDeviceMemory + uint32_t gpuWord; // the parent's GPU filled REG_E + uint32_t mask; // bit0=B bit1=E +}; + +struct T3cResult { + int64_t mismatchB; + int64_t mismatchE; + char note[160]; +}; + // --------------------------------------------------------------------------- // Phase A: enumeration // --------------------------------------------------------------------------- -static const char* memFlagStr(VkMemoryPropertyFlags f) { - static char b[128]; - b[0] = 0; - if (f & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) strcat(b, "DEVICE_LOCAL "); - if (f & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) strcat(b, "HOST_VISIBLE "); - if (f & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) strcat(b, "HOST_COHERENT "); - if (f & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) strcat(b, "HOST_CACHED "); - if (f & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) strcat(b, "LAZY "); - if (f & VK_MEMORY_PROPERTY_PROTECTED_BIT) strcat(b, "PROTECTED "); - return b; +static std::string memFlagStr(VkMemoryPropertyFlags f) { + std::string s; + if (f & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) s += "DEVICE_LOCAL "; + if (f & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) s += "HOST_VISIBLE "; + if (f & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT) s += "HOST_COHERENT "; + if (f & VK_MEMORY_PROPERTY_HOST_CACHED_BIT) s += "HOST_CACHED "; + if (f & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT) s += "LAZY "; + if (f & VK_MEMORY_PROPERTY_PROTECTED_BIT) s += "PROTECTED "; + return s; +} + +// The whole point of this probe is what the driver does in the process MobileGL +// runs in. `adb shell` is not that process: it is the `shell` SELinux domain, +// which has access to device nodes and ashmem/dmabuf rules that `untrusted_app` +// does not necessarily share. Print the domain we actually got so a later app +// run can be compared against it. +static const char* kDomainCaveat = + "run context is `adb shell` (SELinux domain u:r:shell:s0), NOT the untrusted_app " + "domain MobileGL runs in; per-domain SELinux rules can reject a route that works here"; + +static void printRunContext() { + std::string sec = readSmallFile("/proc/self/attr/current"); + pr("=== run context ==="); + pr("uid=%d gid=%d pid=%d selinux=%s", (int)getuid(), (int)getgid(), (int)getpid(), sec.c_str()); + pr("CAVEAT: %s", kDomainCaveat); + pr(" to answer the question for the real domain, run this binary from the app " + "process (spike A's trace-app hook) rather than from adb shell -- see README.md"); } static void reportExternalBufferCaps(VkCtx& c, VkExternalMemoryHandleTypeFlagBits ht, const char* name) { @@ -838,13 +1535,12 @@ static void reportExternalBufferCaps(VkCtx& c, VkExternalMemoryHandleTypeFlagBit out.sType = VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES; vkGetPhysicalDeviceExternalBufferProperties(c.phys, &info, &out); const VkExternalMemoryProperties& p = out.externalMemoryProperties; - char feat[96]; - feat[0] = 0; - if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT) strcat(feat, "DEDICATED_ONLY "); - if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) strcat(feat, "EXPORTABLE "); - if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT) strcat(feat, "IMPORTABLE "); - if (!feat[0]) strcat(feat, ""); - pr(" externalBuffer[%s]: features=%s exportFrom=0x%x compatible=0x%x", name, feat, + std::string feat; + if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT) feat += "DEDICATED_ONLY "; + if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) feat += "EXPORTABLE "; + if (p.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT) feat += "IMPORTABLE "; + if (feat.empty()) feat = ""; + pr(" externalBuffer[%s]: features=%s exportFrom=0x%x compatible=0x%x", name, feat.c_str(), p.exportFromImportedHandleTypes, p.compatibleHandleTypes); } @@ -879,7 +1575,8 @@ static void phaseEnumerate(VkCtx& c, GlCtx& g, bool glOk) { for (uint32_t i = 0; i < c.memProps.memoryTypeCount; ++i) { const VkMemoryType& mt = c.memProps.memoryTypes[i]; pr(" [%u] heap=%u size=%lluMiB flags=%s", i, mt.heapIndex, - (unsigned long long)(c.memProps.memoryHeaps[mt.heapIndex].size >> 20), memFlagStr(mt.propertyFlags)); + (unsigned long long)(c.memProps.memoryHeaps[mt.heapIndex].size >> 20), + memFlagStr(mt.propertyFlags).c_str()); } // Only query handle types the driver actually claims: a handle type whose @@ -894,7 +1591,7 @@ static void phaseEnumerate(VkCtx& c, GlCtx& g, bool glOk) { if (!glOk) { pr("gles: context unavailable, GL extension probe skipped"); - record("A-gles-context", "FAIL", "no headless EGL context"); + record("A-gles-context", "FAIL", "no headless EGL context; every GLES leg is unanswered"); return; } pr("gles: vendor=%s renderer=%s version=%s", g.vendor.c_str(), g.renderer.c_str(), g.version.c_str()); @@ -909,12 +1606,21 @@ static void phaseEnumerate(VkCtx& c, GlCtx& g, bool glOk) { "EGL_EXT_image_dma_buf_import", "EGL_KHR_gl_texture_2D_image", }; for (const char* n : eglWanted) pr(" EGL ext %-40s %s", n, g.hasEgl(n) ? "YES" : "no"); - pr(" eglGetNativeClientBufferANDROID=%p glBufferStorageExternalEXT=%p", - (void*)g.pGetNativeClientBuffer, (void*)g.pBufferStorageExternal); + pr(" eglGetNativeClientBufferANDROID=%p glBufferStorageExternalEXT=%p", (void*)g.pGetNativeClientBuffer, + (void*)g.pBufferStorageExternal); + pr(" glCreateMemoryObjectsEXT=%p glImportMemoryFdEXT=%p glBufferStorageMemEXT=%p glMemoryObjectParameterivEXT=%p", + (void*)g.pCreateMemoryObjects, (void*)g.pImportMemoryFd, (void*)g.pBufferStorageMem, + (void*)g.pMemoryObjectParameteriv); + + bool uuidMatch = false; + std::string glUuid = glDeviceUuidReport(g, c.deviceUUID, &uuidMatch); + char vkUuid[2 * VK_UUID_SIZE + 1] = {0}; + for (uint32_t i = 0; i < VK_UUID_SIZE; ++i) snprintf(vkUuid + i * 2, 3, "%02x", c.deviceUUID[i]); + pr(" GL_DEVICE_UUID_EXT=%s vkDeviceUUID=%s match=%d", glUuid.c_str(), vkUuid, (int)uuidMatch); } // --------------------------------------------------------------------------- -// T1 parent +// T1 parent: server exports its own allocation // --------------------------------------------------------------------------- static void runT1Parent(VkCtx& c, VkExternalMemoryHandleTypeFlagBits handleType, const char* routeName, @@ -928,177 +1634,160 @@ static void runT1Parent(VkCtx& c, VkExternalMemoryHandleTypeFlagBits handleType, return; } - // exportability report first -- a driver that says "not exportable" here and - // still returns an fd is a driver bug we want on the record. - VkPhysicalDeviceExternalBufferInfo ebi{}; - ebi.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_BUFFER_INFO; - ebi.usage = kProbeBufferUsage; - ebi.handleType = handleType; - VkExternalBufferProperties ebp{}; - ebp.sType = VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES; - vkGetPhysicalDeviceExternalBufferProperties(c.phys, &ebi, &ebp); - bool advertisedExportable = - (ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) != 0; - pr("T1[%s] advertisedExportable=%d importable=%d", routeName, (int)advertisedExportable, - (int)((ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT) != 0)); - - VkExternalMemoryBufferCreateInfo ext{}; - ext.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO; - ext.handleTypes = handleType; - VkBufferCreateInfo bci{}; - bci.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; - bci.pNext = &ext; - bci.size = size; - bci.usage = kProbeBufferUsage; - bci.sharingMode = VK_SHARING_MODE_EXCLUSIVE; - - VkBuffer buf = VK_NULL_HANDLE; - VkResult r = vkCreateBuffer(c.device, &bci, nullptr, &buf); - if (r != VK_SUCCESS) { - record(routeName, "FAIL", fmt("vkCreateBuffer(external)=%s", vkStr(r))); + ExportAlloc a; + std::string tag = fmt("T1[%s]", routeName); + if (!exportHostVisible(c, handleType, size, tag.c_str(), a)) { + // one rule for every export-path failure, advertised or not + record(routeName, exportFailStatus(a), a.fail); + freeExportAlloc(c, a); return; } - VkMemoryRequirements req{}; - vkGetBufferMemoryRequirements(c.device, buf, &req); - int typeIdx = pickMemType(c.memProps, req.memoryTypeBits, - VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); - if (typeIdx < 0) { - vkDestroyBuffer(c.device, buf, nullptr); - record(routeName, "FAIL", fmt("no HOST_VISIBLE|HOST_COHERENT type in bits=0x%x", req.memoryTypeBits)); - return; - } - pr("T1[%s] memReq size=%llu align=%llu typeBits=0x%x -> type %d", routeName, - (unsigned long long)req.size, (unsigned long long)req.alignment, req.memoryTypeBits, typeIdx); - VkExportMemoryAllocateInfo exportInfo{}; - exportInfo.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO; - exportInfo.handleTypes = handleType; - VkMemoryDedicatedAllocateInfo dedicated{}; - dedicated.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; - dedicated.buffer = buf; - bool needDedicated = - (ebp.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT) != 0; - if (needDedicated) exportInfo.pNext = &dedicated; - - VkMemoryAllocateInfo mai{}; - mai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; - mai.pNext = &exportInfo; - mai.allocationSize = req.size; - mai.memoryTypeIndex = (uint32_t)typeIdx; - - VkDeviceMemory mem = VK_NULL_HANDLE; - r = vkAllocateMemory(c.device, &mai, nullptr, &mem); - if (r != VK_SUCCESS) { - vkDestroyBuffer(c.device, buf, nullptr); - record(routeName, advertisedExportable ? "FAIL" : "UNSUPPORTED", - fmt("vkAllocateMemory(export)=%s (advertisedExportable=%d)", vkStr(r), (int)advertisedExportable)); - return; - } - r = vkBindBufferMemory(c.device, buf, mem, 0); - if (r != VK_SUCCESS) pr("T1[%s] vkBindBufferMemory=%s (continuing)", routeName, vkStr(r)); - - void* host = nullptr; - r = vkMapMemory(c.device, mem, 0, VK_WHOLE_SIZE, 0, &host); - if (r != VK_SUCCESS) { - vkFreeMemory(c.device, mem, nullptr); - vkDestroyBuffer(c.device, buf, nullptr); - record(routeName, "FAIL", fmt("server-side vkMapMemory=%s", vkStr(r))); - return; - } const uint32_t seedA = 0xA5A50001u, seedB = 0xB0B00002u, seedC = 0xC0C00003u; - memset(host, 0, (size_t)size); - writeRegion(host, REG_A, seedA); + const uint32_t gpuWord = 0x5EED1234u; + memset(a.host, 0, (size_t)size); + writeRegion(a.host, REG_A, seedA); - VkMemoryGetFdInfoKHR gfi{}; - gfi.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR; - gfi.memory = mem; - gfi.handleType = handleType; - int fd = -1; - r = c.pGetMemoryFdKHR(c.device, &gfi, &fd); - if (r != VK_SUCCESS || fd < 0) { - vkUnmapMemory(c.device, mem); - vkFreeMemory(c.device, mem, nullptr); - vkDestroyBuffer(c.device, buf, nullptr); - record(routeName, "UNSUPPORTED", fmt("vkGetMemoryFdKHR=%s fd=%d (advertisedExportable=%d)", vkStr(r), fd, - (int)advertisedExportable)); - return; - } - pr("T1[%s] exported fd=%d -> %s", routeName, fd, describeFd(fd).c_str()); + // real GPU access on the shared allocation, before the handover: the GPU + // reads REG_A (host-written) and writes REG_E, which the importer then has + // to see through its own mapping. + GpuTouch gt = gpuTouch(c, a.buf, REG_A, seedA, REG_E, gpuWord); + int64_t gpuFillSeenHere = gt.ran ? checkFillWord(a.host, REG_E, gpuWord) : -3; + pr("%s gpuTouch ran=%d submit=%s readMismatch=%lld fillSeenByExporter=%lld %s", tag.c_str(), (int)gt.ran, + vkStr(gt.submitResult).c_str(), (long long)gt.readMismatch, (long long)gpuFillSeenHere, gt.fail.c_str()); int sock = -1; pid_t pid = spawnChild("t1", &sock); if (pid < 0) { - close(fd); - vkUnmapMemory(c.device, mem); - vkFreeMemory(c.device, mem, nullptr); - vkDestroyBuffer(c.device, buf, nullptr); record(routeName, "FAIL", "spawnChild failed"); + freeExportAlloc(c, a); return; } T1Offer offer{}; - offer.allocationSize = req.size; + offer.allocationSize = a.allocationSize; offer.bufferSize = size; offer.handleType = (uint32_t)handleType; offer.seedA = seedA; offer.seedB = seedB; offer.seedC = seedC; - offer.memoryTypeIndex = (uint32_t)typeIdx; - offer.memoryTypeBits = req.memoryTypeBits; + offer.memoryTypeIndex = a.memoryTypeIndex; + offer.memoryTypeBits = a.memoryTypeBits; + offer.gpuWord = gpuWord; + offer.gpuRan = (gt.ran && gpuFillSeenHere == -1) ? 1u : 0u; + + // For OPAQUE_FD the raw mmap leg is informational only: the Vulkan spec + // explicitly forbids interpreting an opaque fd payload outside the driver, + // so a driver that refuses it is conformant and MobileGL would never take + // that route. For DMA_BUF a CPU mapping is the point of the handle type, + // so there it is decisive. + const bool mmapDecisive = (handleType == VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT); std::string detail; const char* status = "FAIL"; - if (!sendMsg(sock, MSG_T1_OFFER, &offer, sizeof(offer), fd)) { + if (!sendMsg(sock, MSG_T1_OFFER, &offer, sizeof(offer), a.fd)) { detail = fmt("sendMsg(offer) errno=%d", errno); - } else { - close(fd); - fd = -1; - uint32_t tag = 0; - T1Result res{}; - size_t got = 0; - if (!recvMsg(sock, &tag, &res, sizeof(res), &got, nullptr) || tag != MSG_T1_RESULT || - got != sizeof(res)) { - detail = fmt("no T1 result from child (errno=%d, %s)", errno, reapChild(pid).c_str()); - pid = -1; - } else { - // The child wrote REG_B (mmap) and REG_C (imported vkMapMemory); check - // that the writes are visible through the *server's* own mapping. - int64_t backB = res.wroteB ? checkRegion(host, REG_B, seedB) : -2; - int64_t backC = res.wroteC ? checkRegion(host, REG_C, seedC) : -2; - - detail = fmt( - "mmap=%s(errno=%d,cmp=%lld,payloadAt=%lld,back=%lld) vkimport=%s(fdProps=%s bits=0x%x bind=%s " - "map=%s cmp=%lld back=%lld) %s", - res.mmapOk ? "ok" : "fail", res.mmapErrno, (long long)res.mmapMismatch, - (long long)res.mmapPatternOffset, (long long)backB, - res.importResult == VK_SUCCESS ? "ok" : vkStr((VkResult)res.importResult), - vkStr((VkResult)res.fdPropsResult), res.fdMemoryTypeBits, vkStr((VkResult)res.bindResult), - vkStr((VkResult)res.mapResult), (long long)res.vkMismatch, (long long)backC, res.note); - - bool mmapPath = res.mmapOk && res.mmapMismatch == -1 && backB == -1; - bool vkPath = res.importResult == VK_SUCCESS && res.mapResult == VK_SUCCESS && res.vkMismatch == -1 && - backC == -1; - if (mmapPath && vkPath) { - status = "OK"; - } else if (mmapPath || vkPath) { - status = "PARTIAL"; - } else if (!res.mmapOk && res.importResult != VK_SUCCESS) { - status = "FAIL"; - } else { - status = "PARTIAL"; - } - } - } - if (pid > 0) { + record(routeName, "FAIL", detail); sendMsg(sock, MSG_BYE, nullptr, 0, -1); - detail += " "; - detail += reapChild(pid); + reapChild(pid); + close(sock); + freeExportAlloc(c, a); + return; } + close(a.fd); + a.fd = -1; + + uint32_t tag2 = 0; + T1Result res{}; + size_t got = 0; + if (!recvMsg(sock, &tag2, &res, sizeof(res), &got, nullptr) || tag2 != MSG_T1_RESULT || got != sizeof(res)) { + detail = fmt("no T1 result from child (errno=%d, %s)", errno, reapChild(pid).c_str()); + record(routeName, "FAIL", detail); + close(sock); + freeExportAlloc(c, a); + return; + } + + // The child wrote REG_B (mmap) and REG_C (imported vkMapMemory); check that + // the writes are visible through the *server's* own mapping. + int64_t backB = res.wroteB ? checkRegion(a.host, REG_B, seedB) : -2; + int64_t backC = res.wroteC ? checkRegion(a.host, REG_C, seedC) : -2; + + std::vector legs; + { + Leg l; + l.name = "rawmmap"; + l.decisive = mmapDecisive; + l.attempted = res.mmapOk != 0; + l.readOk = res.mmapOk && res.mmapMismatch == -1 && (!offer.gpuRan || res.mmapGpuMismatch == -1); + l.writeOk = res.wroteB && backB == -1; + if (!l.attempted) + l.fail = fmt("mmap failed errno=%d(%s)", res.mmapErrno, strerror(res.mmapErrno)); + else if (!l.readOk) + l.fail = fmt("exporter payload not at offset 0 (cmp=%lld payloadAt=%lld gpuCmp=%lld)", + (long long)res.mmapMismatch, (long long)res.mmapPatternOffset, + (long long)res.mmapGpuMismatch); + else if (!l.writeOk) + l.fail = fmt("importer write not visible to exporter (back=%lld)", (long long)backB); + legs.push_back(l); + } + { + Leg l; + l.name = "vkimport"; + l.decisive = true; + l.attempted = res.importResult == VK_SUCCESS; + l.readOk = res.importResult == VK_SUCCESS && res.mapResult == VK_SUCCESS && res.vkMismatch == -1 && + (!offer.gpuRan || res.vkGpuMismatch == -1); + l.writeOk = res.wroteC && backC == -1; + if (!res.vkInitOk) + l.fail = "child Vulkan init failed"; + else if (res.importResult != VK_SUCCESS) + l.fail = "vkAllocateMemory(import)=" + vkStr((VkResult)res.importResult); + else if (res.mapResult != VK_SUCCESS) + l.fail = "importer vkMapMemory=" + vkStr((VkResult)res.mapResult); + else if (!l.readOk) + l.fail = fmt("payload mismatch cmp=%lld gpuCmp=%lld", (long long)res.vkMismatch, + (long long)res.vkGpuMismatch); + else if (!l.writeOk) + l.fail = fmt("importer write not visible to exporter (back=%lld)", (long long)backC); + legs.push_back(l); + } + { + Leg l; + l.name = "gpu"; + l.decisive = true; + l.attempted = gt.ran; + l.readOk = gt.readMismatch == -1; + l.writeOk = gpuFillSeenHere == -1; + if (!gt.ran) + l.fail = "GPU touch did not run: " + gt.fail; + else if (!l.readOk) + l.fail = fmt("GPU read of the shared allocation mismatched at %lld", (long long)gt.readMismatch); + else if (!l.writeOk) + l.fail = fmt("GPU write not visible through the exporter's map (at %lld)", (long long)gpuFillSeenHere); + legs.push_back(l); + } + + std::string why; + status = legVerdict(legs, &why); + detail = fmt( + "%s | mmap=%s(errno=%d,cmp=%lld,payloadAt=%lld,gpu=%lld,back=%lld) vkimport=%s(fdProps=%s bits=0x%x " + "bind=%s map=%s cmp=%lld gpu=%lld back=%lld) gpuTouch(submit=%s read=%lld fill=%lld) %s%s%s", + legTrace(legs).c_str(), res.mmapOk ? (res.mmapOk == 2 ? "ok-buffersize" : "ok") : "fail", res.mmapErrno, + (long long)res.mmapMismatch, (long long)res.mmapPatternOffset, (long long)res.mmapGpuMismatch, + (long long)backB, res.importResult == VK_SUCCESS ? "ok" : vkStr((VkResult)res.importResult).c_str(), + vkStr((VkResult)res.fdPropsResult).c_str(), res.fdMemoryTypeBits, vkStr((VkResult)res.bindResult).c_str(), + vkStr((VkResult)res.mapResult).c_str(), (long long)res.vkMismatch, (long long)res.vkGpuMismatch, + (long long)backC, vkStr(gt.submitResult).c_str(), (long long)gt.readMismatch, (long long)gpuFillSeenHere, + res.note, why.empty() ? "" : " | why: ", why.c_str()); + if (!mmapDecisive) + detail += " | rawmmap informational: an opaque fd is not required to be mmap-able"; + + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + detail += " "; + detail += reapChild(pid); close(sock); - if (fd >= 0) close(fd); - vkUnmapMemory(c.device, mem); - vkFreeMemory(c.device, mem, nullptr); - vkDestroyBuffer(c.device, buf, nullptr); + freeExportAlloc(c, a); record(routeName, status, detail); } @@ -1119,6 +1808,8 @@ static int childT1(int sock) { T1Result res{}; res.mmapMismatch = -3; res.vkMismatch = -3; + res.mmapGpuMismatch = -2; + res.vkGpuMismatch = -2; res.gotFd = fd; if (fd < 0) { snprintf(res.note, sizeof(res.note), "no fd received over SCM_RIGHTS"); @@ -1130,17 +1821,22 @@ static int childT1(int sock) { // (1) plain mmap of the exported fd size_t mappedLen = (size_t)offer.allocationSize; + int firstErrno = 0; void* p = mmap(nullptr, mappedLen, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); if (p == MAP_FAILED) { + firstErrno = errno; res.mmapOk = 0; - res.mmapErrno = errno; - pr("child: mmap(MAP_SHARED) failed errno=%d (%s)", errno, strerror(errno)); + res.mmapErrno = firstErrno; + pr("child: mmap(MAP_SHARED, allocationSize) failed errno=%d (%s)", firstErrno, strerror(firstErrno)); // second chance: some allocators only allow the buffer size, not the padded size mappedLen = (size_t)offer.bufferSize; p = mmap(nullptr, mappedLen, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); if (p != MAP_FAILED) { res.mmapOk = 2; - note += " [mmap needed bufferSize not allocationSize]"; + res.mmapErrno = 0; // the mapping succeeded; the first errno is history, not the verdict + note += fmt(" [mmap needed bufferSize not allocationSize; allocationSize errno=%d]", firstErrno); + } else { + res.mmapErrno = errno; } } else { res.mmapOk = 1; @@ -1152,6 +1848,7 @@ static int childT1(int sock) { res.mmapPatternOffset = -1; if (p != MAP_FAILED) { res.mmapMismatch = checkRegion(p, REG_A, offer.seedA); + if (offer.gpuRan) res.mmapGpuMismatch = checkFillWord(p, REG_E, offer.gpuWord); if (res.mmapMismatch != -1) { // Locate the exporter's payload: an fd that maps at a fixed offset from // the driver's base is still usable, but only if that offset is @@ -1168,14 +1865,18 @@ static int childT1(int sock) { } } + // writes through the plain mapping, deferred until every read is done + auto writeThroughMmap = [&]() { + if (p == MAP_FAILED) return; + writeRegion(p, REG_B, offer.seedB); + res.wroteB = 1; + msync(p, mappedLen, MS_SYNC); + }; + // (2) import the same fd into a child-side VkDeviceMemory and map it VkCtx c; if (!vkCtxInit(c, false)) { - if (p != MAP_FAILED) { - writeRegion(p, REG_B, offer.seedB); - res.wroteB = 1; - msync(p, (size_t)mappedLen, MS_SYNC); - } + writeThroughMmap(); snprintf(res.note, sizeof(res.note), "%s | child vulkan init failed", note.c_str()); sendMsg(sock, MSG_T1_RESULT, &res, sizeof(res), -1); return 4; @@ -1209,12 +1910,8 @@ static int childT1(int sock) { VkResult r = vkCreateBuffer(c.device, &bci, nullptr, &buf); if (r != VK_SUCCESS) { res.importResult = (int32_t)r; - if (p != MAP_FAILED) { - writeRegion(p, REG_B, offer.seedB); - res.wroteB = 1; - msync(p, (size_t)mappedLen, MS_SYNC); - } - snprintf(res.note, sizeof(res.note), "%s | child vkCreateBuffer=%s", note.c_str(), vkStr(r)); + writeThroughMmap(); + snprintf(res.note, sizeof(res.note), "%s | child vkCreateBuffer=%s", note.c_str(), vkStr(r).c_str()); sendMsg(sock, MSG_T1_RESULT, &res, sizeof(res), -1); vkCtxDestroy(c); return 5; @@ -1266,12 +1963,8 @@ static int childT1(int sock) { res.importResult = (int32_t)r; } else { close(importFd); - if (p != MAP_FAILED) { - writeRegion(p, REG_B, offer.seedB); - res.wroteB = 1; - msync(p, (size_t)mappedLen, MS_SYNC); - } - snprintf(res.note, sizeof(res.note), "%s | import=%s type=%d bits=0x%x", note.c_str(), vkStr(r), + writeThroughMmap(); + snprintf(res.note, sizeof(res.note), "%s | import=%s type=%d bits=0x%x", note.c_str(), vkStr(r).c_str(), typeIdx, bits); vkDestroyBuffer(c.device, buf, nullptr); sendMsg(sock, MSG_T1_RESULT, &res, sizeof(res), -1); @@ -1285,16 +1978,13 @@ static int childT1(int sock) { res.mapResult = (int32_t)r; if (r == VK_SUCCESS && host) { res.vkMismatch = checkRegion(host, REG_A, offer.seedA); + if (offer.gpuRan) res.vkGpuMismatch = checkFillWord(host, REG_E, offer.gpuWord); writeRegion(host, REG_C, offer.seedC); res.wroteC = 1; vkUnmapMemory(c.device, mem); } // now that both mappings have been read, write through the plain one too - if (p != MAP_FAILED) { - writeRegion(p, REG_B, offer.seedB); - res.wroteB = 1; - msync(p, (size_t)mappedLen, MS_SYNC); - } + writeThroughMmap(); snprintf(res.note, sizeof(res.note), "%s | childType=%d bits=0x%x", note.c_str(), typeIdx, bits); vkFreeMemory(c.device, mem, nullptr); vkDestroyBuffer(c.device, buf, nullptr); @@ -1305,6 +1995,274 @@ static int childT1(int sock) { return 0; } +// --------------------------------------------------------------------------- +// T1 for DirectGLES ("Espryt"): the exported fd imported as GL buffer storage +// +// AcquirePersistentMap on the GLES backend is glBufferStorageEXT + +// glMapBufferRange(PERSISTENT|COHERENT), not a VkDeviceMemory map, so the +// Vulkan T1 answer above does not decide the tier for that backend. The GL +// route to the same question is GL_EXT_memory_object{,_fd}: import the fd as a +// memory object, back a buffer with it, and map that buffer persistently. +// Tried in-process first (isolates "GL can import this fd at all" from +// "the fd survives a process boundary"), then cross-process. +// --------------------------------------------------------------------------- + +static const char* kT1GlSameProc = "T1-gles-memobj-fd-same-proc"; +static const char* kT1GlCrossProc = "T1-gles-memobj-fd-cross-proc"; + +static int childT1Gl(int sock) { + setRecvTimeout(sock, 30); + T1GlOffer offer{}; + uint32_t tag = 0; + size_t got = 0; + int fd = -1; + if (!recvMsg(sock, &tag, &offer, sizeof(offer), &got, &fd) || tag != MSG_T1GL_OFFER) { + pr("child: bad T1GL offer errno=%d", errno); + return 2; + } + T1GlResult res{}; + res.mismatchA = -3; + res.mismatchGpu = -2; + if (fd < 0) { + snprintf(res.note, sizeof(res.note), "no fd over SCM_RIGHTS"); + sendMsg(sock, MSG_T1GL_RESULT, &res, sizeof(res), -1); + return 3; + } + std::string note = describeFd(fd); + + GlCtx g; + if (!glCtxInit(g)) { + snprintf(res.note, sizeof(res.note), "%s | child EGL/GLES init failed", note.c_str()); + sendMsg(sock, MSG_T1GL_RESULT, &res, sizeof(res), -1); + close(fd); + return 4; + } + res.glInitOk = 1; + if (!g.canImportFd()) { + snprintf(res.note, sizeof(res.note), "%s | %s", note.c_str(), g.missingForImportFd().c_str()); + sendMsg(sock, MSG_T1GL_RESULT, &res, sizeof(res), -1); + glCtxDestroy(g); + close(fd); + return 0; + } + res.haveExts = 1; + + GlImport imp; + glImportFdBuffer(g, fd, offer.allocationSize, offer.bufferSize, offer.dedicated != 0, imp); + res.memObjOk = imp.memObjOk; + res.storageOk = imp.storageOk; + res.mapOk = imp.mapOk; + res.persistentCoherent = imp.persistentCoherent; + res.errImport = imp.errImport; + res.errStorage = imp.errStorage; + res.errMap = imp.errMap; + res.errMap2 = imp.errMap2; + if (imp.mapOk && imp.ptr) { + res.mismatchA = checkRegion(imp.ptr, REG_A, offer.seedA); + if (offer.gpuRan) res.mismatchGpu = checkFillWord(imp.ptr, REG_E, offer.gpuWord); + writeRegion(imp.ptr, REG_D, offer.seedD); + res.wroteD = 1; + glImportPublish(g, imp); + } + snprintf(res.note, sizeof(res.note), "%s | accepted=%s | ladder: %s| %s", note.c_str(), + imp.variant.empty() ? "none" : imp.variant.c_str(), imp.ladder.c_str(), imp.fail.c_str()); + glImportRelease(g, imp); + sendMsg(sock, MSG_T1GL_RESULT, &res, sizeof(res), -1); + glCtxDestroy(g); + close(fd); + return 0; +} + +static void runT1GlesParent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { + if (!glOk) { + record(kT1GlSameProc, "SKIP", "no headless GLES context"); + record(kT1GlCrossProc, "SKIP", "no headless GLES context"); + return; + } + bool uuidMatch = false; + std::string glUuid = glDeviceUuidReport(g, c.deviceUUID, &uuidMatch); + std::string uuidNote = fmt("glDeviceUUID=%s vkMatch=%d", glUuid.c_str(), (int)uuidMatch); + + if (!g.canImportFd()) { + std::string d = g.missingForImportFd() + " | " + uuidNote; + record(kT1GlSameProc, "UNSUPPORTED", d); + record(kT1GlCrossProc, "UNSUPPORTED", d); + return; + } + if (!c.hasExtMemFd || !c.pGetMemoryFdKHR) { + record(kT1GlSameProc, "UNSUPPORTED", "VK_KHR_external_memory_fd absent, nothing to import"); + record(kT1GlCrossProc, "UNSUPPORTED", "VK_KHR_external_memory_fd absent, nothing to import"); + return; + } + + ExportAlloc a; + if (!exportHostVisible(c, VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT, size, "T1-gles", a)) { + std::string d = a.fail + " | " + uuidNote; + record(kT1GlSameProc, exportFailStatus(a), d); + record(kT1GlCrossProc, exportFailStatus(a), d); + freeExportAlloc(c, a); + return; + } + + const uint32_t seedA = 0x61510001u, seedD = 0x61510004u, seedF = 0x61510006u; + const uint32_t gpuWord = 0x6C651234u; + memset(a.host, 0, (size_t)size); + writeRegion(a.host, REG_A, seedA); + GpuTouch gt = gpuTouch(c, a.buf, REG_A, seedA, REG_E, gpuWord); + int64_t gpuFillSeenHere = gt.ran ? checkFillWord(a.host, REG_E, gpuWord) : -3; + const bool gpuUsable = gt.ran && gt.readMismatch == -1 && gpuFillSeenHere == -1; + pr("T1-gles gpuTouch ran=%d submit=%s read=%lld fill=%lld %s", (int)gt.ran, vkStr(gt.submitResult).c_str(), + (long long)gt.readMismatch, (long long)gpuFillSeenHere, gt.fail.c_str()); + + // ---- (a) same process ---------------------------------------------------- + { + GlImport imp; + glImportFdBuffer(g, a.fd, a.allocationSize, size, a.dedicated, imp); + int64_t cmpA = -3, cmpGpu = -2, backF = -2; + if (imp.mapOk && imp.ptr) { + cmpA = checkRegion(imp.ptr, REG_A, seedA); + if (gpuUsable) cmpGpu = checkFillWord(imp.ptr, REG_E, gpuWord); + writeRegion(imp.ptr, REG_F, seedF); + glImportPublish(g, imp); + backF = checkRegion(a.host, REG_F, seedF); + } + + std::vector legs; + Leg l; + l.name = "gl-import"; + l.decisive = true; + l.attempted = imp.storageOk; + l.readOk = imp.mapOk && cmpA == -1 && (!gpuUsable || cmpGpu == -1); + l.writeOk = imp.mapOk && backF == -1; + if (!imp.memObjOk) + l.fail = "glImportMemoryFdEXT -> " + glErrStr(imp.errImport); + else if (!imp.storageOk) + l.fail = "glBufferStorageMemEXT -> " + glErrStr(imp.errStorage); + else if (!imp.mapOk) + l.fail = "glMapBufferRange persistent -> " + glErrStr(imp.errMap) + ", plain -> " + glErrStr(imp.errMap2); + else if (!l.readOk) + l.fail = fmt("Vulkan-written payload not visible through the GL map (cmp=%lld gpuCmp=%lld)", + (long long)cmpA, (long long)cmpGpu); + else if (!l.writeOk) + l.fail = fmt("GL-map write not visible through the Vulkan map (back=%lld)", (long long)backF); + legs.push_back(l); + // The tier needs a *persistent coherent* mapping, not a scoped one: a + // driver that only grants the scoped map cannot host AcquirePersistentMap. + Leg pc; + pc.name = "persistent-coherent"; + pc.decisive = true; + pc.attempted = imp.mapOk; + pc.readOk = imp.persistentCoherent; + pc.writeOk = imp.persistentCoherent; + if (!imp.mapOk) + pc.fail = "no mapping at all"; + else if (!imp.persistentCoherent) + pc.fail = "PERSISTENT|COHERENT refused (" + glErrStr(imp.errMap) + "), only a scoped map works"; + legs.push_back(pc); + + std::string why; + const char* status = legVerdict(legs, &why); + record(kT1GlSameProc, status, + fmt("%s | memObj=%d(%s) storage=%d(%s) accepted=%s map=%d persistentCoherent=%d(%s/%s) cmpA=%lld " + "cmpGpu=%lld backF=%lld | ladder: %s| %s | %s", + legTrace(legs).c_str(), (int)imp.memObjOk, glErrStr(imp.errImport).c_str(), (int)imp.storageOk, + glErrStr(imp.errStorage).c_str(), imp.variant.empty() ? "none" : imp.variant.c_str(), + (int)imp.mapOk, (int)imp.persistentCoherent, glErrStr(imp.errMap).c_str(), + glErrStr(imp.errMap2).c_str(), (long long)cmpA, (long long)cmpGpu, (long long)backF, + imp.ladder.c_str(), uuidNote.c_str(), why.c_str())); + glImportRelease(g, imp); + } + + // ---- (b) cross process --------------------------------------------------- + { + int sock = -1; + pid_t pid = spawnChild("t1gl", &sock); + if (pid < 0) { + record(kT1GlCrossProc, "FAIL", "spawnChild failed"); + freeExportAlloc(c, a); + return; + } + T1GlOffer offer{}; + offer.allocationSize = a.allocationSize; + offer.bufferSize = size; + offer.seedA = seedA; + offer.seedD = seedD; + offer.gpuWord = gpuWord; + offer.gpuRan = gpuUsable ? 1u : 0u; + offer.dedicated = a.dedicated ? 1u : 0u; + + if (!sendMsg(sock, MSG_T1GL_OFFER, &offer, sizeof(offer), a.fd)) { + record(kT1GlCrossProc, "FAIL", fmt("sendMsg(offer) errno=%d", errno)); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + reapChild(pid); + close(sock); + freeExportAlloc(c, a); + return; + } + T1GlResult res{}; + uint32_t tag = 0; + size_t got = 0; + if (!recvMsg(sock, &tag, &res, sizeof(res), &got, nullptr) || tag != MSG_T1GL_RESULT || got != sizeof(res)) { + record(kT1GlCrossProc, "FAIL", fmt("no reply errno=%d %s", errno, reapChild(pid).c_str())); + close(sock); + freeExportAlloc(c, a); + return; + } + int64_t backD = res.wroteD ? checkRegion(a.host, REG_D, seedD) : -2; + + std::vector legs; + Leg l; + l.name = "gl-import"; + l.decisive = true; + l.attempted = res.storageOk != 0; + l.readOk = res.mapOk && res.mismatchA == -1 && (!offer.gpuRan || res.mismatchGpu == -1); + l.writeOk = res.wroteD && backD == -1; + if (!res.glInitOk) + l.fail = "child EGL/GLES init failed"; + else if (!res.haveExts) + l.fail = "child lacks GL_EXT_memory_object{,_fd}"; + else if (!res.memObjOk) + l.fail = "glImportMemoryFdEXT -> " + glErrStr(res.errImport); + else if (!res.storageOk) + l.fail = "glBufferStorageMemEXT -> " + glErrStr(res.errStorage); + else if (!res.mapOk) + l.fail = "glMapBufferRange persistent -> " + glErrStr(res.errMap) + ", plain -> " + glErrStr(res.errMap2); + else if (!l.readOk) + l.fail = fmt("exporter payload not visible through the child's GL map (cmp=%lld gpuCmp=%lld)", + (long long)res.mismatchA, (long long)res.mismatchGpu); + else if (!l.writeOk) + l.fail = fmt("child GL-map write not visible to the exporter (back=%lld)", (long long)backD); + legs.push_back(l); + Leg pc; + pc.name = "persistent-coherent"; + pc.decisive = true; + pc.attempted = res.mapOk != 0; + pc.readOk = res.persistentCoherent != 0; + pc.writeOk = res.persistentCoherent != 0; + if (!res.mapOk) + pc.fail = "no mapping at all"; + else if (!res.persistentCoherent) + pc.fail = "PERSISTENT|COHERENT refused (" + glErrStr(res.errMap) + "), only a scoped map works"; + legs.push_back(pc); + + std::string why; + const char* status = legVerdict(legs, &why); + std::string detail = + fmt("%s | glInit=%d exts=%d memObj=%d(%s) storage=%d(%s) map=%d persistentCoherent=%d(%s/%s) " + "cmpA=%lld cmpGpu=%lld backD=%lld | %s | %s [%s] ", + legTrace(legs).c_str(), res.glInitOk, res.haveExts, res.memObjOk, glErrStr(res.errImport).c_str(), + res.storageOk, glErrStr(res.errStorage).c_str(), res.mapOk, res.persistentCoherent, + glErrStr(res.errMap).c_str(), glErrStr(res.errMap2).c_str(), (long long)res.mismatchA, + (long long)res.mismatchGpu, (long long)backD, uuidNote.c_str(), why.c_str(), res.note); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + detail += reapChild(pid); + close(sock); + record(kT1GlCrossProc, status, detail); + } + + freeExportAlloc(c, a); +} + // --------------------------------------------------------------------------- // T0: child allocates an AHardwareBuffer BLOB, parent imports it // --------------------------------------------------------------------------- @@ -1368,7 +2326,7 @@ static int childT0(int sock) { return 7; } T0Result res{}; - res.mismatchB = res.mismatchC = res.mismatchD = -2; + res.mismatchB = res.mismatchC = res.mismatchD = res.mismatchE = -2; void* q = nullptr; rc = AHardwareBuffer_lock(ahb, AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN, -1, nullptr, &q); res.lockOk = (rc == 0 && q) ? 1 : 0; @@ -1377,6 +2335,7 @@ static int childT0(int sock) { if (ver.writtenMask & 1) res.mismatchB = checkRegion(q, REG_B, ver.seedB); if (ver.writtenMask & 2) res.mismatchC = checkRegion(q, REG_C, ver.seedC); if (ver.writtenMask & 4) res.mismatchD = checkRegion(q, REG_D, ver.seedD); + if (ver.writtenMask & 8) res.mismatchE = checkFillWord(q, REG_E, ver.gpuWord); AHardwareBuffer_unlock(ahb, nullptr); } snprintf(res.note, sizeof(res.note), "mask=0x%x", ver.writtenMask); @@ -1387,18 +2346,27 @@ static int childT0(int sock) { static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { const uint32_t seedA = 0x0A0A0011u, seedB = 0x0B0B0022u, seedC = 0x0C0C0033u, seedD = 0x0D0D0044u; + const uint32_t gpuWord = 0x70701234u; + + // The composite row is recorded at the very end from the full + // import + map + compare + write-back chain; the handoff alone is only a + // diagnostic and gets its own informational row. + auto failAll = [&](const std::string& why) { + record("T0-ahb-handoff", "FAIL", why); + record("T0-ahb-blob-transfer", "FAIL", "handoff failed, nothing to import: " + why); + }; int sock = -1; pid_t pid = spawnChild("t0", &sock); if (pid < 0) { - record("T0-ahb-blob-transfer", "FAIL", "spawnChild failed"); + failAll("spawnChild failed"); return; } T0Request rq{}; rq.size = size; rq.seedA = seedA; if (!sendMsg(sock, MSG_T0_REQUEST, &rq, sizeof(rq), -1)) { - record("T0-ahb-blob-transfer", "FAIL", fmt("sendMsg errno=%d", errno)); + failAll(fmt("sendMsg errno=%d", errno)); close(sock); reapChild(pid); return; @@ -1407,12 +2375,12 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { uint32_t tag = 0; size_t got = 0; if (!recvMsg(sock, &tag, &alloc, sizeof(alloc), &got, nullptr) || tag != MSG_T0_ALLOC) { - record("T0-ahb-blob-transfer", "FAIL", fmt("no alloc reply errno=%d %s", errno, reapChild(pid).c_str())); + failAll(fmt("no alloc reply errno=%d %s", errno, reapChild(pid).c_str())); close(sock); return; } if (alloc.allocOk != 1) { - record("T0-ahb-blob-transfer", "FAIL", fmt("child alloc failed rc=%d %s", alloc.allocErr, alloc.note)); + failAll(fmt("child alloc failed rc=%d %s", alloc.allocErr, alloc.note)); close(sock); reapChild(pid); return; @@ -1422,7 +2390,7 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { AHardwareBuffer* ahb = nullptr; int rc = AHardwareBuffer_recvHandleFromUnixSocket(sock, &ahb); if (rc != 0 || !ahb) { - record("T0-ahb-blob-transfer", "FAIL", fmt("recvHandleFromUnixSocket rc=%d errno=%d", rc, errno)); + failAll(fmt("recvHandleFromUnixSocket rc=%d errno=%d", rc, errno)); close(sock); reapChild(pid); return; @@ -1431,37 +2399,46 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { AHardwareBuffer_describe(ahb, &desc); pr("T0 parent received AHB: w=%u h=%u fmt=0x%x usage=0x%llx stride=%u", desc.width, desc.height, desc.format, (unsigned long long)desc.usage, desc.stride); - record("T0-ahb-blob-transfer", "OK", fmt("socket handoff of a %llu-byte BLOB works (%s)", - (unsigned long long)size, alloc.note)); + record("T0-ahb-handoff", "OK", + fmt("socket handoff of a %llu-byte BLOB works (%s) -- handoff only, see T0-ahb-blob-transfer for the tier", + (unsigned long long)size, alloc.note)); + + uint32_t writtenMask = 0; + int64_t cpuCmp = -3, vkCmp = -3, glCmp = -3, vkGpuCmp = -2; + bool vkMapped = false, glMapped = false, glPersistent = false; + GpuTouch gt; + std::string vkFail, glFail, cpuFail, gpuFail; // (a) CPU path: AHardwareBuffer_lock on the receiving side - uint32_t writtenMask = 0; { void* p = nullptr; rc = AHardwareBuffer_lock(ahb, AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN | AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN, -1, nullptr, &p); if (rc != 0 || !p) { - record("T0-ahb-cpu-lock", "FAIL", fmt("AHardwareBuffer_lock rc=%d errno=%d", rc, errno)); + cpuFail = fmt("AHardwareBuffer_lock rc=%d errno=%d", rc, errno); + record("T0-ahb-cpu-lock", "FAIL", cpuFail); } else { - int64_t cmp = checkRegion(p, REG_A, seedA); + cpuCmp = checkRegion(p, REG_A, seedA); writeRegion(p, REG_D, seedD); writtenMask |= 4; AHardwareBuffer_unlock(ahb, nullptr); - record("T0-ahb-cpu-lock", cmp == -1 ? "OK" : "FAIL", - fmt("cross-process CPU read of the child's payload, mismatch=%lld", (long long)cmp)); + if (cpuCmp != -1) cpuFail = fmt("payload mismatch at %lld", (long long)cpuCmp); + record("T0-ahb-cpu-lock", cpuCmp == -1 ? "OK" : "FAIL", + fmt("cross-process CPU read of the client's payload, mismatch=%lld", (long long)cpuCmp)); } } - // (b) Vulkan import + // (b) Vulkan import (+ a real GPU access on the imported memory) if (!c.hasAhb || !c.pGetAhbProps) { - record("T0-ahb-vulkan-import", "UNSUPPORTED", - "VK_ANDROID_external_memory_android_hardware_buffer absent"); + vkFail = "VK_ANDROID_external_memory_android_hardware_buffer absent"; + record("T0-ahb-vulkan-import", "UNSUPPORTED", vkFail); } else { VkAndroidHardwareBufferPropertiesANDROID props{}; props.sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID; VkResult r = c.pGetAhbProps(c.device, ahb, &props); if (r != VK_SUCCESS) { - record("T0-ahb-vulkan-import", "FAIL", fmt("vkGetAndroidHardwareBufferPropertiesANDROID=%s", vkStr(r))); + vkFail = "vkGetAndroidHardwareBufferPropertiesANDROID=" + vkStr(r); + record("T0-ahb-vulkan-import", "FAIL", vkFail); } else { pr("T0 AHB props: allocationSize=%llu memoryTypeBits=0x%x", (unsigned long long)props.allocationSize, props.memoryTypeBits); @@ -1476,7 +2453,8 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { VkBuffer buf = VK_NULL_HANDLE; r = vkCreateBuffer(c.device, &bci, nullptr, &buf); if (r != VK_SUCCESS) { - record("T0-ahb-vulkan-import", "FAIL", fmt("vkCreateBuffer(AHB external)=%s", vkStr(r))); + vkFail = "vkCreateBuffer(AHB external)=" + vkStr(r); + record("T0-ahb-vulkan-import", "FAIL", vkFail); } else { int typeIdx = pickMemType(c.memProps, props.memoryTypeBits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); @@ -1497,25 +2475,37 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { VkDeviceMemory mem = VK_NULL_HANDLE; r = vkAllocateMemory(c.device, &mai, nullptr, &mem); if (r != VK_SUCCESS) { - record("T0-ahb-vulkan-import", "FAIL", - fmt("vkAllocateMemory(import AHB)=%s typeIdx=%d bits=0x%x", vkStr(r), typeIdx, - props.memoryTypeBits)); + vkFail = fmt("vkAllocateMemory(import AHB)=%s typeIdx=%d bits=0x%x", vkStr(r).c_str(), typeIdx, + props.memoryTypeBits); + record("T0-ahb-vulkan-import", "FAIL", vkFail); } else { VkResult br = vkBindBufferMemory(c.device, buf, mem, 0); + // GPU access on the client's allocation -- the part that makes + // T0 a tier rather than a successful mmap + if (br == VK_SUCCESS) { + gt = gpuTouch(c, buf, REG_A, seedA, REG_E, gpuWord); + if (gt.ran) writtenMask |= 8; + gpuFail = gt.fail; + } else { + gpuFail = "vkBindBufferMemory=" + vkStr(br); + } void* host = nullptr; VkResult mr = vkMapMemory(c.device, mem, 0, VK_WHOLE_SIZE, 0, &host); if (mr == VK_SUCCESS && host) { - int64_t cmp = checkRegion(host, REG_A, seedA); + vkMapped = true; + vkCmp = checkRegion(host, REG_A, seedA); + if (gt.ran) vkGpuCmp = checkFillWord(host, REG_E, gpuWord); writeRegion(host, REG_B, seedB); writtenMask |= 1; vkUnmapMemory(c.device, mem); - record("T0-ahb-vulkan-import", cmp == -1 ? "OK" : "PARTIAL", - fmt("imported+mapped (hostVisibleType=%d bind=%s) payload mismatch=%lld", - (int)hostVisible, vkStr(br), (long long)cmp)); + if (vkCmp != -1) vkFail = fmt("payload mismatch at %lld", (long long)vkCmp); + record("T0-ahb-vulkan-import", vkCmp == -1 ? "OK" : "PARTIAL", + fmt("imported+mapped (hostVisibleType=%d bind=%s) payload mismatch=%lld gpuFill=%lld", + (int)hostVisible, vkStr(br).c_str(), (long long)vkCmp, (long long)vkGpuCmp)); } else { - record("T0-ahb-vulkan-import", "PARTIAL", - fmt("import ok, vkMapMemory=%s (bind=%s hostVisibleType=%d bits=0x%x)", vkStr(mr), - vkStr(br), (int)hostVisible, props.memoryTypeBits)); + vkFail = fmt("vkMapMemory=%s (bind=%s hostVisibleType=%d bits=0x%x)", vkStr(mr).c_str(), + vkStr(br).c_str(), (int)hostVisible, props.memoryTypeBits); + record("T0-ahb-vulkan-import", "PARTIAL", "import ok, " + vkFail); } vkFreeMemory(c.device, mem, nullptr); } @@ -1523,48 +2513,69 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { } } } + record("T0-ahb-gpu-access", gt.ran && gt.readMismatch == -1 ? "OK" : (gt.ran ? "FAIL" : "SKIP"), + fmt("vkCmdCopyBuffer out of the client AHB + vkCmdFillBuffer into it: ran=%d submit=%s read=%lld " + "fillSeenByServerMap=%lld %s", + (int)gt.ran, vkStr(gt.submitResult).c_str(), (long long)gt.readMismatch, (long long)vkGpuCmp, + gpuFail.c_str())); // (c) GL import through EGL_ANDROID_get_native_client_buffer + EXT_external_buffer + // -- this is the DirectGLES ("Espryt") form of T0: the server backs a GL + // buffer with the client's allocation and maps it persistent/coherent. if (!glOk) { - record("T0-ahb-gl-import", "SKIP", "no GL context"); + glFail = "no GL context"; + record("T0-ahb-gl-import", "SKIP", glFail); } else if (!g.hasGl("GL_EXT_external_buffer") || !g.pBufferStorageExternal || !g.pGetNativeClientBuffer) { - record("T0-ahb-gl-import", "UNSUPPORTED", - fmt("GL_EXT_external_buffer=%d GL_EXT_buffer_storage=%d eglGetNativeClientBufferANDROID=%d " - "glBufferStorageExternalEXT=%d", - (int)g.hasGl("GL_EXT_external_buffer"), (int)g.hasGl("GL_EXT_buffer_storage"), - (int)(g.pGetNativeClientBuffer != nullptr), (int)(g.pBufferStorageExternal != nullptr))); + glFail = fmt("GL_EXT_external_buffer=%d GL_EXT_buffer_storage=%d eglGetNativeClientBufferANDROID=%d " + "glBufferStorageExternalEXT=%d", + (int)g.hasGl("GL_EXT_external_buffer"), (int)g.hasGl("GL_EXT_buffer_storage"), + (int)(g.pGetNativeClientBuffer != nullptr), (int)(g.pBufferStorageExternal != nullptr)); + record("T0-ahb-gl-import", "UNSUPPORTED", glFail); } else { EGLClientBuffer cb = g.pGetNativeClientBuffer(ahb); if (!cb) { - record("T0-ahb-gl-import", "FAIL", fmt("eglGetNativeClientBufferANDROID=NULL egl=0x%04x", eglGetError())); + glFail = fmt("eglGetNativeClientBufferANDROID=NULL egl=0x%04x", eglGetError()); + record("T0-ahb-gl-import", "FAIL", glFail); } else { GLuint b = 0; glGenBuffers(1, &b); glBindBuffer(GL_ARRAY_BUFFER, b); - while (glGetError() != GL_NO_ERROR) {} + glDrain(); g.pBufferStorageExternal(GL_ARRAY_BUFFER, 0, (GLsizeiptr)size, cb, GL_MAP_READ_BIT | GL_MAP_WRITE_BIT | GL_MAP_PERSISTENT_BIT_EXT | GL_MAP_COHERENT_BIT_EXT | GL_DYNAMIC_STORAGE_BIT_EXT); - GLenum err = glGetError(); - if (err != GL_NO_ERROR) { - record("T0-ahb-gl-import", "FAIL", fmt("glBufferStorageExternalEXT -> GL error 0x%04x", err)); + GLenum errStorage = glDrain(); + if (errStorage != GL_NO_ERROR) { + glFail = "glBufferStorageExternalEXT -> " + glErrStr(errStorage); + record("T0-ahb-gl-import", "FAIL", glFail); } else { void* m = glMapBufferRange(GL_ARRAY_BUFFER, 0, (GLsizeiptr)size, GL_MAP_READ_BIT | GL_MAP_WRITE_BIT | GL_MAP_PERSISTENT_BIT_EXT | GL_MAP_COHERENT_BIT_EXT); - GLenum merr = glGetError(); + GLenum errMap = glDrain(); + GLenum errMap2 = GL_NO_ERROR; + glPersistent = m != nullptr; if (!m) { - record("T0-ahb-gl-import", "PARTIAL", - fmt("storage ok, glMapBufferRange returned NULL (GL error 0x%04x)", merr)); + m = glMapBufferRange(GL_ARRAY_BUFFER, 0, (GLsizeiptr)size, GL_MAP_READ_BIT | GL_MAP_WRITE_BIT); + errMap2 = glDrain(); + } + if (!m) { + glFail = "glMapBufferRange persistent -> " + glErrStr(errMap) + ", plain -> " + glErrStr(errMap2); + record("T0-ahb-gl-import", "FAIL", "storage ok, " + glFail); } else { - int64_t cmp = checkRegion(m, REG_A, seedA); + glMapped = true; + glCmp = checkRegion(m, REG_A, seedA); writeRegion(m, REG_C, seedC); writtenMask |= 2; glUnmapBuffer(GL_ARRAY_BUFFER); glFinish(); - record("T0-ahb-gl-import", cmp == -1 ? "OK" : "PARTIAL", - fmt("persistent-coherent GL map of the client AHB, payload mismatch=%lld", - (long long)cmp)); + if (glCmp != -1) glFail = fmt("payload mismatch at %lld", (long long)glCmp); + if (!glPersistent) glFail += " [PERSISTENT|COHERENT refused: " + glErrStr(errMap) + "]"; + record("T0-ahb-gl-import", (glCmp == -1 && glPersistent) ? "OK" : "PARTIAL", + fmt("GL map of the client AHB: persistentCoherent=%d (persistent err=%s, plain err=%s) " + "payload mismatch=%lld", + (int)glPersistent, glErrStr(errMap).c_str(), glErrStr(errMap2).c_str(), + (long long)glCmp)); } } glBindBuffer(GL_ARRAY_BUFFER, 0); @@ -1572,38 +2583,106 @@ static void runT0Parent(VkCtx& c, GlCtx& g, bool glOk, uint64_t size) { } } - // (d) ask the child to verify everything the parent wrote + // (d) ask the client to verify everything the server wrote T0Verify ver{}; ver.seedB = seedB; ver.seedC = seedC; ver.seedD = seedD; + ver.gpuWord = gpuWord; ver.writtenMask = writtenMask; + T0Result res{}; + res.mismatchB = res.mismatchC = res.mismatchD = res.mismatchE = -3; + bool gotVerify = false; std::string wbDetail; const char* wbStatus = "FAIL"; if (!sendMsg(sock, MSG_T0_VERIFY, &ver, sizeof(ver), -1)) { wbDetail = fmt("sendMsg(verify) errno=%d", errno); + } else if (!recvMsg(sock, &tag, &res, sizeof(res), &got, nullptr) || tag != MSG_T0_RESULT) { + wbDetail = fmt("no verify reply errno=%d", errno); } else { - T0Result res{}; - if (!recvMsg(sock, &tag, &res, sizeof(res), &got, nullptr) || tag != MSG_T0_RESULT) { - wbDetail = fmt("no verify reply errno=%d", errno); - } else { - bool anyChecked = false, allOk = true; - auto acc = [&](int64_t v) { - if (v == -2) return; - anyChecked = true; - if (v != -1) allOk = false; - }; - acc(res.mismatchB); - acc(res.mismatchC); - acc(res.mismatchD); - wbStatus = !anyChecked ? "SKIP" : (allOk ? "OK" : "FAIL"); - wbDetail = fmt("mask=0x%x vkWrite=%lld glWrite=%lld cpuWrite=%lld (lock=%d)", writtenMask, - (long long)res.mismatchB, (long long)res.mismatchC, (long long)res.mismatchD, - res.lockOk); - } + gotVerify = true; + bool anyChecked = false, allOk = true; + auto acc = [&](int64_t v) { + if (v == -2 || v == -3) return; + anyChecked = true; + if (v != -1) allOk = false; + }; + acc(res.mismatchB); + acc(res.mismatchC); + acc(res.mismatchD); + acc(res.mismatchE); + wbStatus = !anyChecked ? "SKIP" : (allOk ? "OK" : "FAIL"); + wbDetail = fmt("mask=0x%x vkWrite=%lld glWrite=%lld cpuWrite=%lld gpuFill=%lld (clientLock=%d rc=%d)", + writtenMask, (long long)res.mismatchB, (long long)res.mismatchC, (long long)res.mismatchD, + (long long)res.mismatchE, res.lockOk, res.lockErr); } record("T0-ahb-writeback-to-client", wbStatus, wbDetail); + // composite tier verdict: handoff alone is not the tier + std::vector legs; + { + Leg l; + l.name = "vk-import"; + l.decisive = true; + l.attempted = vkMapped; + l.readOk = vkMapped && vkCmp == -1; + l.writeOk = gotVerify && (writtenMask & 1) && res.mismatchB == -1; + if (!l.attempted) + l.fail = vkFail.empty() ? "not attempted" : vkFail; + else if (!l.readOk) + l.fail = "server could not read the client payload: " + vkFail; + else if (!l.writeOk) + l.fail = fmt("server write not visible to the client (back=%lld)", (long long)res.mismatchB); + legs.push_back(l); + } + { + Leg l; + l.name = "gl-import"; + l.decisive = true; + l.attempted = glMapped; + l.readOk = glMapped && glCmp == -1 && glPersistent; + l.writeOk = gotVerify && (writtenMask & 2) && res.mismatchC == -1; + if (!l.attempted) + l.fail = glFail.empty() ? "not attempted" : glFail; + else if (!l.readOk) + l.fail = "GL side: " + glFail; + else if (!l.writeOk) + l.fail = fmt("server GL write not visible to the client (back=%lld)", (long long)res.mismatchC); + legs.push_back(l); + } + { + Leg l; + l.name = "gpu"; + l.decisive = true; + l.attempted = gt.ran; + l.readOk = gt.readMismatch == -1; + l.writeOk = gotVerify && (writtenMask & 8) && res.mismatchE == -1; + if (!gt.ran) + l.fail = "GPU touch did not run: " + gpuFail; + else if (!l.readOk) + l.fail = fmt("GPU read of the client allocation mismatched at %lld", (long long)gt.readMismatch); + else if (!l.writeOk) + l.fail = fmt("GPU write not visible to the client (back=%lld)", (long long)res.mismatchE); + legs.push_back(l); + } + { + Leg l; + l.name = "cpu-lock"; + l.decisive = false; // informational: proves the handle, not the tier + l.attempted = cpuCmp != -3; + l.readOk = cpuCmp == -1; + l.writeOk = gotVerify && (writtenMask & 4) && res.mismatchD == -1; + l.fail = cpuFail; + legs.push_back(l); + } + std::string why; + const char* status = legVerdict(legs, &why); + record("T0-ahb-blob-transfer", status, + fmt("%s | full chain handoff+import+map+compare+writeback | cpuCmp=%lld vkCmp=%lld glCmp=%lld " + "glPersistentCoherent=%d gpuRan=%d | %s", + legTrace(legs).c_str(), (long long)cpuCmp, (long long)vkCmp, (long long)glCmp, (int)glPersistent, + (int)gt.ran, why.c_str())); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); std::string reap = reapChild(pid); pr("T0 %s", reap.c_str()); @@ -1627,6 +2706,110 @@ static void runT0Parent(VkCtx&, GlCtx&, bool, uint64_t) { // T3: VK_EXT_external_memory_host over a memfd-backed mapping // --------------------------------------------------------------------------- +// Reserves an alignment-corrected window and places `fd` inside it. Returns the +// aligned pointer, or nullptr; `*reserveOut` must be munmap'ed with +// `mapSize + align` bytes. +static void* mapAlignedFd(int fd, uint64_t mapSize, uint64_t align, void** reserveOut, std::string* fail) { + *reserveOut = mmap(nullptr, (size_t)(mapSize + align), PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (*reserveOut == MAP_FAILED) { + *reserveOut = nullptr; + *fail = fmt("reserve mmap errno=%d(%s)", errno, strerror(errno)); + return nullptr; + } + uintptr_t base = ((uintptr_t)*reserveOut + align - 1) & ~(uintptr_t)(align - 1); + void* host = mmap((void*)base, (size_t)mapSize, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, fd, 0); + if (host == MAP_FAILED) { + *fail = fmt("mmap(fd, MAP_FIXED) errno=%d(%s)", errno, strerror(errno)); + munmap(*reserveOut, (size_t)(mapSize + align)); + *reserveOut = nullptr; + return nullptr; + } + return host; +} + +// Imports `host` as VkDeviceMemory and binds a buffer to it. +struct HostImport { + VkBuffer buf = VK_NULL_HANDLE; + VkDeviceMemory mem = VK_NULL_HANDLE; + void* mapped = nullptr; + int typeIdx = -1; + VkResult hostPtrProps = VK_NOT_READY; + VkResult createResult = VK_NOT_READY; + VkResult allocResult = VK_NOT_READY; + VkResult bindResult = VK_NOT_READY; + VkResult mapResult = VK_NOT_READY; + uint32_t bits = 0; + std::string fail; +}; + +static bool importHostPointer(VkCtx& c, void* host, uint64_t mapSize, HostImport& o) { + VkMemoryHostPointerPropertiesEXT hp{}; + hp.sType = VK_STRUCTURE_TYPE_MEMORY_HOST_POINTER_PROPERTIES_EXT; + o.hostPtrProps = c.pGetHostPtrProps(c.device, VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT, host, &hp); + if (o.hostPtrProps != VK_SUCCESS) { + o.fail = "vkGetMemoryHostPointerPropertiesEXT=" + vkStr(o.hostPtrProps); + return false; + } + VkExternalMemoryBufferCreateInfo ext{}; + ext.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO; + ext.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT; + VkBufferCreateInfo bci{}; + bci.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; + bci.pNext = &ext; + bci.size = mapSize; + bci.usage = kProbeBufferUsage; + o.createResult = vkCreateBuffer(c.device, &bci, nullptr, &o.buf); + VkMemoryRequirements req{}; + if (o.createResult == VK_SUCCESS) { + vkGetBufferMemoryRequirements(c.device, o.buf, &req); + } else { + o.buf = VK_NULL_HANDLE; + req.memoryTypeBits = 0xFFFFFFFFu; + } + o.bits = hp.memoryTypeBits & req.memoryTypeBits; + o.typeIdx = pickMemType(c.memProps, o.bits, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); + if (o.typeIdx < 0) o.typeIdx = pickMemType(c.memProps, o.bits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT); + if (o.typeIdx < 0) { + o.fail = fmt("no host-visible memory type in hostPtrBits=0x%x & reqBits=0x%x", hp.memoryTypeBits, + req.memoryTypeBits); + return false; + } + VkImportMemoryHostPointerInfoEXT imp{}; + imp.sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_HOST_POINTER_INFO_EXT; + imp.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT; + imp.pHostPointer = host; + VkMemoryAllocateInfo mai{}; + mai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; + mai.pNext = &imp; + mai.allocationSize = mapSize; + mai.memoryTypeIndex = (uint32_t)o.typeIdx; + o.allocResult = vkAllocateMemory(c.device, &mai, nullptr, &o.mem); + if (o.allocResult != VK_SUCCESS) { + o.mem = VK_NULL_HANDLE; + o.fail = fmt("vkAllocateMemory(import host ptr)=%s type=%d bits=0x%x", vkStr(o.allocResult).c_str(), o.typeIdx, + o.bits); + return false; + } + o.bindResult = (o.buf != VK_NULL_HANDLE) ? vkBindBufferMemory(c.device, o.buf, o.mem, 0) : VK_SUCCESS; + o.mapResult = vkMapMemory(c.device, o.mem, 0, VK_WHOLE_SIZE, 0, &o.mapped); + if (o.mapResult != VK_SUCCESS) { + o.mapped = nullptr; + o.fail = "vkMapMemory=" + vkStr(o.mapResult); + return false; + } + return true; +} + +static void releaseHostImport(VkCtx& c, HostImport& o) { + if (o.mapped) vkUnmapMemory(c.device, o.mem); + if (o.mem) vkFreeMemory(c.device, o.mem, nullptr); + if (o.buf) vkDestroyBuffer(c.device, o.buf, nullptr); + o.mapped = nullptr; + o.mem = VK_NULL_HANDLE; + o.buf = VK_NULL_HANDLE; +} + static int childT3(int sock) { setRecvTimeout(sock, 30); T3Offer offer{}; @@ -1636,6 +2819,7 @@ static int childT3(int sock) { if (!recvMsg(sock, &tag, &offer, sizeof(offer), &got, &fd) || tag != MSG_T3_OFFER) return 2; T3Result res{}; res.mismatch = -3; + res.gpuMismatch = -2; if (fd < 0) { snprintf(res.note, sizeof(res.note), "no fd"); sendMsg(sock, MSG_T3_RESULT, &res, sizeof(res), -1); @@ -1649,7 +2833,9 @@ static int childT3(int sock) { } else { res.mmapOk = 1; res.mismatch = checkRegion(p, REG_A, offer.seedA); + if (offer.gpuRan) res.gpuMismatch = checkFillWord(p, REG_E, offer.gpuWord); writeRegion(p, REG_B, offer.seedB); + msync(p, (size_t)offer.size, MS_SYNC); munmap(p, (size_t)offer.size); } sendMsg(sock, MSG_T3_RESULT, &res, sizeof(res), -1); @@ -1657,9 +2843,203 @@ static int childT3(int sock) { return 0; } +// The direction that makes T3 a tier: the CLIENT allocates the memory and the +// SERVER imports the client's host pointer. The child is the client here. +static int childT3Client(int sock) { + setRecvTimeout(sock, 30); + T3cRequest rq{}; + uint32_t tag = 0; + size_t got = 0; + if (!recvMsg(sock, &tag, &rq, sizeof(rq), &got, nullptr) || tag != MSG_T3C_REQUEST) return 2; + + T3cReady ready{}; + ready.size = rq.size; + int memfd = memfd_create("extmem_probe_client", 0); + if (memfd < 0) { + ready.err = errno; + snprintf(ready.note, sizeof(ready.note), "memfd_create errno=%d(%s)", errno, strerror(errno)); + sendMsg(sock, MSG_T3C_READY, &ready, sizeof(ready), -1); + return 3; + } + if (ftruncate(memfd, (off_t)rq.size) != 0) { + ready.err = errno; + snprintf(ready.note, sizeof(ready.note), "ftruncate errno=%d(%s)", errno, strerror(errno)); + sendMsg(sock, MSG_T3C_READY, &ready, sizeof(ready), -1); + close(memfd); + return 4; + } + void* p = mmap(nullptr, (size_t)rq.size, PROT_READ | PROT_WRITE, MAP_SHARED, memfd, 0); + if (p == MAP_FAILED) { + ready.err = errno; + snprintf(ready.note, sizeof(ready.note), "mmap errno=%d(%s)", errno, strerror(errno)); + sendMsg(sock, MSG_T3C_READY, &ready, sizeof(ready), -1); + close(memfd); + return 5; + } + memset(p, 0, (size_t)rq.size); + writeRegion(p, REG_A, rq.seedA); + ready.ok = 1; + snprintf(ready.note, sizeof(ready.note), "client memfd %s", describeFd(memfd).c_str()); + if (!sendMsg(sock, MSG_T3C_READY, &ready, sizeof(ready), memfd)) { + munmap(p, (size_t)rq.size); + close(memfd); + return 6; + } + + T3cVerify ver{}; + T3cResult res{}; + res.mismatchB = res.mismatchE = -2; + if (!recvMsg(sock, &tag, &ver, sizeof(ver), &got, nullptr) || tag != MSG_T3C_VERIFY) { + munmap(p, (size_t)rq.size); + close(memfd); + return 7; + } + if (ver.mask & 1) res.mismatchB = checkRegion(p, REG_B, ver.seedB); + if (ver.mask & 2) res.mismatchE = checkFillWord(p, REG_E, ver.gpuWord); + snprintf(res.note, sizeof(res.note), "mask=0x%x", ver.mask); + sendMsg(sock, MSG_T3C_RESULT, &res, sizeof(res), -1); + munmap(p, (size_t)rq.size); + close(memfd); + return 0; +} + +static void runT3ClientAllocParent(VkCtx& c, uint64_t size) { + const char* route = "T3-client-memfd-server-import"; + if (!c.hasExtMemHost || !c.pGetHostPtrProps) { + record(route, "UNSUPPORTED", "VK_EXT_external_memory_host absent"); + return; + } + uint64_t align = c.minImportedHostPointerAlignment ? c.minImportedHostPointerAlignment : 4096; + uint64_t mapSize = (size + align - 1) & ~(align - 1); + const uint32_t seedA = 0x3C3C0001u, seedB = 0x3C3C0002u, gpuWord = 0x3C3C1234u; + + int sock = -1; + pid_t pid = spawnChild("t3c", &sock); + if (pid < 0) { + record(route, "FAIL", "spawnChild failed"); + return; + } + T3cRequest rq{}; + rq.size = mapSize; + rq.seedA = seedA; + if (!sendMsg(sock, MSG_T3C_REQUEST, &rq, sizeof(rq), -1)) { + record(route, "FAIL", fmt("sendMsg(request) errno=%d", errno)); + close(sock); + reapChild(pid); + return; + } + T3cReady ready{}; + uint32_t tag = 0; + size_t got = 0; + int fd = -1; + if (!recvMsg(sock, &tag, &ready, sizeof(ready), &got, &fd) || tag != MSG_T3C_READY) { + record(route, "FAIL", fmt("no ready reply errno=%d %s", errno, reapChild(pid).c_str())); + close(sock); + return; + } + if (!ready.ok || fd < 0) { + record(route, "FAIL", fmt("client could not allocate: %s (fd=%d)", ready.note, fd)); + if (fd >= 0) close(fd); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + reapChild(pid); + close(sock); + return; + } + pr("T3c server received the client's memfd: %s", describeFd(fd).c_str()); + + void* reserve = nullptr; + std::string mapFail; + void* host = mapAlignedFd(fd, mapSize, align, &reserve, &mapFail); + if (!host) { + record(route, "FAIL", "server could not map the client's memfd: " + mapFail); + close(fd); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + reapChild(pid); + close(sock); + return; + } + + HostImport hi; + bool imported = importHostPointer(c, host, mapSize, hi); + int64_t cmpA = -3, gpuFillSeen = -3; + GpuTouch gt; + if (imported) { + cmpA = checkRegion(hi.mapped, REG_A, seedA); // server reads what the client wrote + writeRegion(hi.mapped, REG_B, seedB); // server writes back + gt = gpuTouch(c, hi.buf, REG_A, seedA, REG_E, gpuWord); + gpuFillSeen = gt.ran ? checkFillWord(hi.mapped, REG_E, gpuWord) : -3; + } + + T3cVerify ver{}; + ver.seedB = seedB; + ver.gpuWord = gpuWord; + ver.mask = (imported ? 1u : 0u) | ((gt.ran && gpuFillSeen == -1) ? 2u : 0u); + T3cResult res{}; + res.mismatchB = res.mismatchE = -3; + bool gotVerify = false; + if (sendMsg(sock, MSG_T3C_VERIFY, &ver, sizeof(ver), -1) && + recvMsg(sock, &tag, &res, sizeof(res), &got, nullptr) && tag == MSG_T3C_RESULT) { + gotVerify = true; + } + + std::vector legs; + { + Leg l; + l.name = "server-import"; + l.decisive = true; + l.attempted = imported; + l.readOk = imported && cmpA == -1; + l.writeOk = gotVerify && (ver.mask & 1) && res.mismatchB == -1; + if (!imported) + l.fail = hi.fail; + else if (!l.readOk) + l.fail = fmt("server could not read the client's payload (cmp=%lld)", (long long)cmpA); + else if (!l.writeOk) + l.fail = fmt("server write not visible to the client (back=%lld)", (long long)res.mismatchB); + legs.push_back(l); + } + { + Leg l; + l.name = "gpu"; + l.decisive = true; + l.attempted = gt.ran; + l.readOk = gt.readMismatch == -1; + l.writeOk = gotVerify && (ver.mask & 2) && res.mismatchE == -1; + if (!gt.ran) + l.fail = "GPU touch did not run: " + gt.fail; + else if (!l.readOk) + l.fail = fmt("GPU read of the client's memory mismatched at %lld", (long long)gt.readMismatch); + else if (!l.writeOk) + l.fail = fmt("GPU write not visible to the client (serverMap=%lld clientMap=%lld)", + (long long)gpuFillSeen, (long long)res.mismatchE); + legs.push_back(l); + } + std::string why; + const char* status = legVerdict(legs, &why); + std::string detail = + fmt("%s | client allocates, server imports: align=%llu size=%llu hostPtrProps=%s create=%s alloc=%s bind=%s " + "map=%s type=%d bits=0x%x | serverReadOfClient=%lld clientReadOfServer=%lld gpuRead=%lld " + "gpuFill(server=%lld,client=%lld) | %s [%s] ", + legTrace(legs).c_str(), (unsigned long long)align, (unsigned long long)mapSize, + vkStr(hi.hostPtrProps).c_str(), vkStr(hi.createResult).c_str(), vkStr(hi.allocResult).c_str(), + vkStr(hi.bindResult).c_str(), vkStr(hi.mapResult).c_str(), hi.typeIdx, hi.bits, (long long)cmpA, + (long long)res.mismatchB, (long long)gt.readMismatch, (long long)gpuFillSeen, (long long)res.mismatchE, + why.c_str(), ready.note); + + releaseHostImport(c, hi); + munmap(host, (size_t)mapSize); + if (reserve) munmap(reserve, (size_t)(mapSize + align)); + close(fd); + sendMsg(sock, MSG_BYE, nullptr, 0, -1); + detail += reapChild(pid); + close(sock); + record(route, status, detail); +} + static void runT3Parent(VkCtx& c, uint64_t size) { if (!c.hasExtMemHost || !c.pGetHostPtrProps) { record("T3-external-memory-host", "UNSUPPORTED", "VK_EXT_external_memory_host absent"); + record("T3-memfd-cross-process", "UNSUPPORTED", "VK_EXT_external_memory_host absent"); return; } uint64_t align = c.minImportedHostPointerAlignment ? c.minImportedHostPointerAlignment : 4096; @@ -1675,82 +3055,60 @@ static void runT3Parent(VkCtx& c, uint64_t size) { close(memfd); return; } - // reserve an aligned window, then place the memfd inside it - void* reserve = mmap(nullptr, (size_t)(mapSize + align), PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (reserve == MAP_FAILED) { - record("T3-external-memory-host", "FAIL", fmt("reserve mmap errno=%d", errno)); + void* reserve = nullptr; + std::string mapFail; + void* host = mapAlignedFd(memfd, mapSize, align, &reserve, &mapFail); + if (!host) { + record("T3-external-memory-host", "FAIL", mapFail); close(memfd); return; } - uintptr_t base = ((uintptr_t)reserve + align - 1) & ~(uintptr_t)(align - 1); - void* host = mmap((void*)base, (size_t)mapSize, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, memfd, 0); - if (host == MAP_FAILED) { - record("T3-external-memory-host", "FAIL", fmt("mmap(memfd, MAP_FIXED) errno=%d", errno)); - munmap(reserve, (size_t)(mapSize + align)); - close(memfd); - return; - } - const uint32_t seedA = 0x33330001u, seedB = 0x33330002u; + const uint32_t seedA = 0x33330001u, seedB = 0x33330002u, gpuWord = 0x33331234u; memset(host, 0, (size_t)mapSize); writeRegion(host, REG_A, seedA); - VkMemoryHostPointerPropertiesEXT hp{}; - hp.sType = VK_STRUCTURE_TYPE_MEMORY_HOST_POINTER_PROPERTIES_EXT; - VkResult r = c.pGetHostPtrProps(c.device, VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT, host, &hp); - if (r != VK_SUCCESS) { - record("T3-external-memory-host", "FAIL", fmt("vkGetMemoryHostPointerPropertiesEXT=%s align=%llu", vkStr(r), - (unsigned long long)align)); - } else { - VkExternalMemoryBufferCreateInfo ext{}; - ext.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO; - ext.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT; - VkBufferCreateInfo bci{}; - bci.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; - bci.pNext = &ext; - bci.size = mapSize; - bci.usage = kProbeBufferUsage; - VkBuffer buf = VK_NULL_HANDLE; - VkResult cr = vkCreateBuffer(c.device, &bci, nullptr, &buf); - VkMemoryRequirements req{}; - if (cr == VK_SUCCESS) vkGetBufferMemoryRequirements(c.device, buf, &req); - uint32_t bits = hp.memoryTypeBits & (cr == VK_SUCCESS ? req.memoryTypeBits : 0xFFFFFFFFu); - int typeIdx = pickMemType(c.memProps, bits, - VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT); - if (typeIdx < 0) typeIdx = pickMemType(c.memProps, bits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT); - if (typeIdx < 0) { - record("T3-external-memory-host", "FAIL", - fmt("no host-visible type in hostPtrBits=0x%x & reqBits=0x%x", hp.memoryTypeBits, - req.memoryTypeBits)); - } else { - VkImportMemoryHostPointerInfoEXT imp{}; - imp.sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_HOST_POINTER_INFO_EXT; - imp.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT; - imp.pHostPointer = host; - VkMemoryAllocateInfo mai{}; - mai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; - mai.pNext = &imp; - mai.allocationSize = mapSize; - mai.memoryTypeIndex = (uint32_t)typeIdx; - VkDeviceMemory mem = VK_NULL_HANDLE; - VkResult ar = vkAllocateMemory(c.device, &mai, nullptr, &mem); - if (ar != VK_SUCCESS) { - record("T3-external-memory-host", "FAIL", - fmt("vkAllocateMemory(import host ptr)=%s type=%d bits=0x%x align=%llu", vkStr(ar), typeIdx, - bits, (unsigned long long)align)); - } else { - VkResult br = (cr == VK_SUCCESS) ? vkBindBufferMemory(c.device, buf, mem, 0) : VK_SUCCESS; - void* mapped = nullptr; - VkResult mr = vkMapMemory(c.device, mem, 0, VK_WHOLE_SIZE, 0, &mapped); - int64_t cmp = -3; - if (mr == VK_SUCCESS && mapped) cmp = checkRegion(mapped, REG_A, seedA); - if (mr == VK_SUCCESS) vkUnmapMemory(c.device, mem); - record("T3-external-memory-host", (mr == VK_SUCCESS && cmp == -1) ? "OK" : "PARTIAL", - fmt("import ok (align=%llu type=%d bind=%s) vkMapMemory=%s mismatch=%lld", - (unsigned long long)align, typeIdx, vkStr(br), vkStr(mr), (long long)cmp)); - vkFreeMemory(c.device, mem, nullptr); - } - } - if (cr == VK_SUCCESS) vkDestroyBuffer(c.device, buf, nullptr); + HostImport hi; + bool imported = importHostPointer(c, host, mapSize, hi); + int64_t cmpA = -3, gpuFillSeen = -3; + GpuTouch gt; + if (imported) { + cmpA = checkRegion(hi.mapped, REG_A, seedA); + gt = gpuTouch(c, hi.buf, REG_A, seedA, REG_E, gpuWord); + gpuFillSeen = gt.ran ? checkFillWord(host, REG_E, gpuWord) : -3; + } + { + std::vector legs; + Leg l; + l.name = "import-map"; + l.decisive = true; + l.attempted = imported; + // one process on both ends here, so the "write back" direction is the + // imported mapping seeing the original mmap's bytes + l.readOk = imported && cmpA == -1; + l.writeOk = imported && cmpA == -1; + l.fail = imported ? (cmpA == -1 ? "" : fmt("payload mismatch at %lld", (long long)cmpA)) : hi.fail; + legs.push_back(l); + Leg gl; + gl.name = "gpu"; + gl.decisive = true; + gl.attempted = gt.ran; + gl.readOk = gt.readMismatch == -1; + gl.writeOk = gpuFillSeen == -1; + if (!gt.ran) + gl.fail = "GPU touch did not run: " + gt.fail; + else if (!gl.readOk) + gl.fail = fmt("GPU read mismatched at %lld", (long long)gt.readMismatch); + else if (!gl.writeOk) + gl.fail = fmt("GPU write not visible through the host mapping (at %lld)", (long long)gpuFillSeen); + legs.push_back(gl); + std::string why; + record("T3-external-memory-host", legVerdict(legs, &why), + fmt("%s | align=%llu type=%d bits=0x%x hostPtrProps=%s alloc=%s bind=%s map=%s mismatch=%lld " + "gpuRead=%lld gpuFill=%lld %s", + legTrace(legs).c_str(), (unsigned long long)align, hi.typeIdx, hi.bits, + vkStr(hi.hostPtrProps).c_str(), vkStr(hi.allocResult).c_str(), vkStr(hi.bindResult).c_str(), + vkStr(hi.mapResult).c_str(), (long long)cmpA, (long long)gt.readMismatch, + (long long)gpuFillSeen, why.c_str())); } // the same memfd handed to another process @@ -1763,6 +3121,8 @@ static void runT3Parent(VkCtx& c, uint64_t size) { off.size = mapSize; off.seedA = seedA; off.seedB = seedB; + off.gpuWord = gpuWord; + off.gpuRan = (gt.ran && gpuFillSeen == -1) ? 1u : 0u; if (!sendMsg(sock, MSG_T3_OFFER, &off, sizeof(off), memfd)) { record("T3-memfd-cross-process", "FAIL", fmt("sendMsg errno=%d", errno)); } else { @@ -1773,9 +3133,26 @@ static void runT3Parent(VkCtx& c, uint64_t size) { record("T3-memfd-cross-process", "FAIL", fmt("no reply errno=%d", errno)); } else { int64_t back = res.mmapOk ? checkRegion(host, REG_B, seedB) : -3; - record("T3-memfd-cross-process", (res.mmapOk && res.mismatch == -1 && back == -1) ? "OK" : "FAIL", - fmt("child mmap=%d errno=%d cmp=%lld writeback=%lld [%s]", res.mmapOk, res.mmapErrno, - (long long)res.mismatch, (long long)back, res.note)); + std::vector legs; + Leg l; + l.name = "peer-mmap"; + l.decisive = true; + l.attempted = res.mmapOk != 0; + l.readOk = res.mmapOk && res.mismatch == -1 && (!off.gpuRan || res.gpuMismatch == -1); + l.writeOk = res.mmapOk && back == -1; + if (!l.attempted) + l.fail = fmt("peer mmap failed errno=%d(%s)", res.mmapErrno, strerror(res.mmapErrno)); + else if (!l.readOk) + l.fail = fmt("peer could not read (cmp=%lld gpuCmp=%lld)", (long long)res.mismatch, + (long long)res.gpuMismatch); + else if (!l.writeOk) + l.fail = fmt("peer write not visible here (back=%lld)", (long long)back); + legs.push_back(l); + std::string why; + record("T3-memfd-cross-process", legVerdict(legs, &why), + fmt("%s | child mmap=%d errno=%d cmp=%lld gpuCmp=%lld writeback=%lld %s [%s]", + legTrace(legs).c_str(), res.mmapOk, res.mmapErrno, (long long)res.mismatch, + (long long)res.gpuMismatch, (long long)back, why.c_str(), res.note)); } } sendMsg(sock, MSG_BYE, nullptr, 0, -1); @@ -1783,8 +3160,9 @@ static void runT3Parent(VkCtx& c, uint64_t size) { close(sock); } + releaseHostImport(c, hi); munmap(host, (size_t)mapSize); - munmap(reserve, (size_t)(mapSize + align)); + if (reserve) munmap(reserve, (size_t)(mapSize + align)); close(memfd); } @@ -1795,7 +3173,9 @@ static void runT3Parent(VkCtx& c, uint64_t size) { static void printSummary() { char model[PROP_VALUE_MAX] = {0}; getProp("ro.product.model", model, sizeof(model)); - printf("\n=== extmem_probe summary (model=%s) ===\n", model); + std::string sec = readSmallFile("/proc/self/attr/current"); + printf("\n=== extmem_probe summary (model=%s selinux=%s) ===\n", model, sec.c_str()); + printf("NOTE: %s\n", kDomainCaveat); printf("%-34s %-12s %s\n", "ROUTE", "STATUS", "DETAIL"); for (const RouteResult& r : gResults) printf("%-34s %-12s %s\n", r.route.c_str(), r.status.c_str(), r.detail.c_str()); @@ -1806,7 +3186,7 @@ static void printSummary() { int main(int argc, char** argv) { uint64_t size = kDefaultSize; const char* childRoute = nullptr; - bool doT1 = true, doT0 = true, doT3 = true; + bool doT1 = true, doT0 = true, doT3 = true, doGles = true; for (int i = 1; i < argc; ++i) { if (!strncmp(argv[i], "--child=", 8)) { childRoute = argv[i] + 8; @@ -1815,15 +3195,19 @@ int main(int argc, char** argv) { } else if (!strcmp(argv[i], "--only-t1")) { doT0 = doT3 = false; } else if (!strcmp(argv[i], "--only-t0")) { - doT1 = doT3 = false; + doT1 = doT3 = doGles = false; } else if (!strcmp(argv[i], "--only-t3")) { - doT1 = doT0 = false; + doT1 = doT0 = doGles = false; + } else if (!strcmp(argv[i], "--only-gles")) { + doT1 = doT0 = doT3 = false; + } else if (!strcmp(argv[i], "--no-gles")) { + doGles = false; } else if (!strcmp(argv[i], "--help")) { - printf("usage: extmem_probe [--size=BYTES] [--only-t0|--only-t1|--only-t3]\n"); + printf("usage: extmem_probe [--size=BYTES] [--only-t0|--only-t1|--only-t3|--only-gles] [--no-gles]\n"); return 0; } } - if (size < 4 * kRegion) size = 4 * kRegion; + if (size < kRegionCount * kRegion) size = kRegionCount * kRegion; // A peer that has already exited must not take this process down with it. signal(SIGPIPE, SIG_IGN); @@ -1834,13 +3218,16 @@ int main(int argc, char** argv) { gRole = roleBuf; int sock = 3; if (!strcmp(childRoute, "t1")) return childT1(sock); + if (!strcmp(childRoute, "t1gl")) return childT1Gl(sock); if (!strcmp(childRoute, "t0")) return childT0(sock); if (!strcmp(childRoute, "t3")) return childT3(sock); + if (!strcmp(childRoute, "t3c")) return childT3Client(sock); pr("unknown child route %s", childRoute); return 1; } pr("extmem_probe: MobileGL disaggregation spike B, size=%llu bytes", (unsigned long long)size); + printRunContext(); VkCtx c; bool vkOk = vkCtxInit(c, true); @@ -1859,10 +3246,15 @@ int main(int argc, char** argv) { runT1Parent(c, VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT, "T1-opaque-fd", size); runT1Parent(c, VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT, "T1-dma-buf", size); } + pr("=== phase T1-gles: the same export imported as GL buffer storage ==="); + if (doGles) runT1GlesParent(c, g, glOk, size); pr("=== phase T0: client-allocated AHardwareBuffer BLOB ==="); if (doT0) runT0Parent(c, g, glOk, size); pr("=== phase T3: VK_EXT_external_memory_host ==="); - if (doT3) runT3Parent(c, size); + if (doT3) { + runT3Parent(c, size); + runT3ClientAllocParent(c, size); + } glCtxDestroy(g); vkCtxDestroy(c);