From 18a5dba0e4633f5e832f331f4c7fd312ad699a9c Mon Sep 17 00:00:00 2001 From: Swung0x48 Date: Sat, 3 Oct 2026 22:45:46 -0400 Subject: [PATCH] [Docs] (Anland): Chrome decodes video in hardware through the YUV shared images, with probes to check it --- .../skills/anland-mobilegl-plasma/SKILL.md | 4 +- .../scripts/probes/cdpmedia.py | 94 ++++ .../scripts/probes/yuvprobe.c | 410 ++++++++++++++++++ README.md | 3 +- .../notes/anland/runbook-plasma.md | 2 + 5 files changed, 510 insertions(+), 3 deletions(-) create mode 100644 .claude/skills/anland-mobilegl-plasma/scripts/probes/cdpmedia.py create mode 100644 .claude/skills/anland-mobilegl-plasma/scripts/probes/yuvprobe.c diff --git a/.claude/skills/anland-mobilegl-plasma/SKILL.md b/.claude/skills/anland-mobilegl-plasma/SKILL.md index 1cd1e442..60f7f482 100644 --- a/.claude/skills/anland-mobilegl-plasma/SKILL.md +++ b/.claude/skills/anland-mobilegl-plasma/SKILL.md @@ -126,9 +126,9 @@ timeout 25 mgrun offscreen glmark2-es2-wayland -s 1280x720 --run-forever > /tmp/ sleep 8; grep -a "presented through" /tmp/mgl-run.client.log ``` Chrome: `bash scripts/chrome-launch.sh [flags]` (log `/tmp/mgl-chrome-.client.log`, screenshot `ch-.png`; `CHROME_GPU=in-process` puts the GPU back in the browser, which cannot recover from a device loss). -Video: the container has a VA-API driver on the render node the GPU process gets (`/usr/lib/dri/msm_drm_drv_video.so`, V4L2 decode into its own NV12 dma-bufs); MobileGL cannot import those (only its own RGBA shared images; no dma-buf import in the phone's Vulkan, no `GL_OES_EGL_image_external`), and Chrome's GPU process aborts (SIGTRAP, "GPU process exited unexpectedly: exit_code=5") on the first decoded frame - window flicker + stalled video in a loop. `mobilegl-startup.sh chrome` therefore passes `--disable-accelerated-video-decode`. Diagnose a video with a CDP `Media.enable` dump (`kVideoDecoderName`); remote debugging needs a non-default `--user-data-dir`, and a fresh profile also needs `--password-store=basic` (else page loads wait forever on KWallet). +Video: Chrome decodes in hardware. The GPU process's render node has a VA-API driver in the container (`/usr/lib/dri/msm_drm_drv_video.so`, DroidSpaces' V4L2 driver): Chrome allocates its frames with `vaCreateSurfaces` and exports them (`vaExportSurfaceHandle`, one fd, R8 + GR88 layers) - the driver's own buffers, never GBM ones - and imports each as an NV12 (P010 for 10-bit) EGLImage bound to `GL_TEXTURE_EXTERNAL_OES`. MobileGL takes such a FOREIGN YUV dma-buf through its labelled CPU-copy fallback: the server maps it and copies it into a YUV AHardwareBuffer of its own (`AHardwareBuffer_lockPlanes`) at the reader's first use in each frame (any present, publishing flush or native fence), then converts that into the texture's RGBA level on the GPU (Espryt: `GL_EXT_YUV_target` with the import's colour hints, `MOBILEGL_YUV_DRIVER_CONVERSION=1` for the driver's external sampler; Magma: `VkSamplerYcbcrConversion` on the AHB's external format). A server-allocated NV12/P010 buffer (GBM `gbm_bo_create(NV12)`) needs no copy. `MOBILEGL_CHROME_VIDEO_DECODE=software` in `mobilegl-startup.sh chrome` brings back `--disable-accelerated-video-decode`. Check a video with `scripts/probes/cdpmedia.py ` in the container (CDP `Media.enable`: `kVideoDecoderName = VaapiVideoDecoder`, `kIsPlatformVideoDecoder = true`; `CDP_NAVIGATE=` / `CDP_EXPR=`), which needs Chrome started with `--remote-debugging-port` and a non-default `--user-data-dir` (a fresh profile also needs `--password-store=basic`, else page loads wait forever on KWallet), and grep Chrome's log for `exit_code` (a GPU-process restart). `scripts/probes/yuvprobe.c` pixel-checks foreign NV12/P010 imports, their per-frame refresh and a GBM NV12 buffer (`mgrun offscreen mgl-yuvprobe`, exit 0 = pass). KWin's own NV12 path (it splits a YUV buffer into R8 + GR88 imports) is not served: MobileGL imports no single-channel dma-bufs, so KWin advertises no YUV format over linux-dmabuf. Device loss: `setprop debug.mobilegl.inject_device_lost_pid ` (as root) loses the server session of the client whose pid - as it sees itself in the container - is ``, once per value, on any backend and whether or not it presents; `debug.mobilegl.inject_device_lost_at N` instead loses whichever session makes the N-th device check (Espryt presents and read-backs only). `bash scripts/chrome-recover.sh [rounds]` checks that Chrome repaints after such a loss without a restart (battery-guarded, stops below 40%). `bash scripts/kwin-recover.sh [rounds]` loses KWin's own session (default twice) and checks that KWin restarts compositing on a fresh session - same KWin, plasmashell and client processes, the screen still animating (glmark2 on Wayland, glxgears on X11, `CHROME=1` adds Chrome) - which needs the anland backend + kwin.patch reset handling deployed (`kwin-stage.sh build`). -Probes: `scripts/probes/texfmt.c` (texture formats keep alpha?), `dmahash.c` (hash a process's dma-bufs); build in the container, e.g. `gcc -O1 /root/texfmt.c -o /usr/local/bin/mgl-texfmt -lEGL`, run with `mgrun offscreen`. +Probes: `scripts/probes/texfmt.c` (texture formats keep alpha?), `dmahash.c` (hash a process's dma-bufs), `yuvprobe.c` (YUV dma-buf sampling, `gcc -O1 /root/yuvprobe.c -o /usr/local/bin/mgl-yuvprobe -lEGL -lGLESv2 -lgbm -lm`), `cdpmedia.py` (which decoder a Chrome page uses); build in the container, e.g. `gcc -O1 /root/texfmt.c -o /usr/local/bin/mgl-texfmt -lEGL`, run with `mgrun offscreen`. GPU hangs: a shader that never ends stops the WHOLE GPU (every process, MobileGL or not) until the kernel resets it - ~2 s, or ~10 s ("Preemption Fault" in dmesg) when a higher-priority context waits; nothing in user space shortens one stall. What the server does is end the guilty session so it cannot hang the GPU again (Magma: event-bracketed submissions + `mgl-gpu-watch`, budget `MOBILEGL_GPU_HANG_BUDGET_MS` / `debug.mobilegl.gpu_hang_budget_ms`, default 2000; Espryt: the driver's GUILTY reset status, which it reports after the ~2 s reset only). Repro: `scripts/probes/hangclient.c` (Wayland + GL 3.3 core; `-H 60 -N 1 -E 60` hangs every 60 frames, `-H -1` is a frame-gap observer; build with wayland-scanner's xdg-shell files, `-lwayland-client -lwayland-egl -lEGL -lOpenGL -lm`), and `gpuprobe.c` (NDK, plain adb shell; `gpuprobe 30 high` is a high-priority GLES context outside MobileGL - it both measures the device-wide stall and forces the 10 s preemption path). A result counts only if ALL hold: diff --git a/.claude/skills/anland-mobilegl-plasma/scripts/probes/cdpmedia.py b/.claude/skills/anland-mobilegl-plasma/scripts/probes/cdpmedia.py new file mode 100644 index 00000000..7c1bfb77 --- /dev/null +++ b/.claude/skills/anland-mobilegl-plasma/scripts/probes/cdpmedia.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +# cdpmedia.py [port] [seconds] - ask a Chrome started with --remote-debugging-port= (and a +# non-default --user-data-dir) which video decoder its first page's players use, and evaluate +# `window.__mglStats` / the element `#t` text of the page. Standard library only (no websocket +# module in the container): a minimal RFC 6455 client. +# python3 cdpmedia.py 9333 6 +# Prints kVideoDecoderName / kIsPlatformVideoDecoder / kVideoTracks properties and media errors as +# they arrive, then the page's stats text. +import base64, json, os, socket, struct, sys, time, urllib.request + +port = int(sys.argv[1]) if len(sys.argv) > 1 else 9333 +seconds = float(sys.argv[2]) if len(sys.argv) > 2 else 6.0 +targets = json.load(urllib.request.urlopen(f"http://127.0.0.1:{port}/json")) +page = next(t for t in targets if t.get("type") == "page") +url = page["webSocketDebuggerUrl"] +path = url.split(f"127.0.0.1:{port}", 1)[1] +sock = socket.create_connection(("127.0.0.1", port)) +key = base64.b64encode(os.urandom(16)).decode() +sock.sendall((f"GET {path} HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n" + f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n").encode()) +response = b"" +while b"\r\n\r\n" not in response: + response += sock.recv(4096) +buffer = response.split(b"\r\n\r\n", 1)[1] + + +def send(message): + data = json.dumps(message).encode() + header = bytes([0x81]) + mask = os.urandom(4) + if len(data) < 126: + header += bytes([0x80 | len(data)]) + elif len(data) < 65536: + header += bytes([0x80 | 126]) + struct.pack(">H", len(data)) + else: + header += bytes([0x80 | 127]) + struct.pack(">Q", len(data)) + sock.sendall(header + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(data))) + + +def recv_exact(n): + global buffer + while len(buffer) < n: + chunk = sock.recv(65536) + if not chunk: + raise EOFError + buffer += chunk + out, buffer = buffer[:n], buffer[n:] + return out + + +def recv(): + first, second = recv_exact(2) + length = second & 0x7F + if length == 126: + length = struct.unpack(">H", recv_exact(2))[0] + elif length == 127: + length = struct.unpack(">Q", recv_exact(8))[0] + payload = recv_exact(length) + return json.loads(payload) if first & 0x0F == 1 else None + + +send({"id": 1, "method": "Media.enable"}) +# Players created before the domain was enabled report nothing: reload so the page makes new ones. +if os.environ.get("CDP_NAVIGATE"): + send({"id": 3, "method": "Page.navigate", "params": {"url": os.environ["CDP_NAVIGATE"]}}) +elif os.environ.get("CDP_RELOAD", "1") == "1": + send({"id": 3, "method": "Page.reload"}) +deadline = time.time() + seconds +sock.settimeout(0.5) +seen = set() +while time.time() < deadline: + try: + message = recv() + except socket.timeout: + continue + if not message or "method" not in message: + continue + if message["method"] == "Media.playerPropertiesChanged": + for prop in message["params"]["properties"]: + if prop["name"] in ("kVideoDecoderName", "kIsPlatformVideoDecoder", "kVideoTracks", "kResolution"): + line = f"{prop['name']} = {prop['value']}" + if line not in seen: + seen.add(line) + print(line) + elif message["method"] in ("Media.playerErrorsRaised",): + print("media error:", json.dumps(message["params"])[:300]) +send({"id": 2, "method": "Runtime.evaluate", + "params": {"expression": os.environ.get("CDP_EXPR", "(document.getElementById('t')||{}).textContent"), "returnByValue": True}}) +sock.settimeout(3) +while True: + message = recv() + if message and message.get("id") == 2: + print("page:", message.get("result", {}).get("result", {}).get("value")) + break diff --git a/.claude/skills/anland-mobilegl-plasma/scripts/probes/yuvprobe.c b/.claude/skills/anland-mobilegl-plasma/scripts/probes/yuvprobe.c new file mode 100644 index 00000000..d85629a0 --- /dev/null +++ b/.claude/skills/anland-mobilegl-plasma/scripts/probes/yuvprobe.c @@ -0,0 +1,410 @@ +// yuvprobe.c - pixel check of YUV dma-buf sampling through MobileGL (GL_OES_EGL_image_external). +// +// gcc -O1 yuvprobe.c -o /usr/local/bin/mgl-yuvprobe -lEGL -lGLESv2 -lgbm +// mgrun offscreen mgl-yuvprobe # exit 0 = every check passed +// +// What it does, in an ES 3.0 context on a pbuffer: +// 1. FOREIGN NV12 (and P010): a dma-buf from /dev/dma_heap/system - a buffer the MobileGL server +// did not allocate, as a video decoder's would be - with padded pitches and a gap before the +// chroma plane, filled with four vertical bars (red, green, blue, white) in BT.709 narrow +// range. Imported with eglCreateImage(EGL_LINUX_DMA_BUF_EXT) and its colour hints, bound to +// GL_TEXTURE_EXTERNAL_OES, drawn through a samplerExternalOES shader into an RGBA8 FBO and read +// back: each bar must come out its colour. The buffer is then rewritten with the bars in +// reverse order and, after a native fence (what ends a frame for an offscreen producer), drawn +// again: the new order must show (the server refreshes a foreign buffer at each frame). +// 2. GBM NV12: gbm_bo_create(NV12) through the mobilegl GBM backend (a server-allocated YUV +// buffer, zero-copy), its planes printed, imported and sampled once (content undefined; the +// check is that every call succeeds). +// Also prints the dma-buf formats/modifiers EGL lists for NV12/P010. +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifndef DRM_FORMAT_NV12 +#define DRM_FORMAT_NV12 0x3231564E +#endif +#ifndef DRM_FORMAT_P010 +#define DRM_FORMAT_P010 0x30313050 +#endif + +static int g_failures = 0; +#define CHECK(cond, ...) \ + do { \ + if (!(cond)) { \ + printf("FAIL: " __VA_ARGS__); \ + printf("\n"); \ + ++g_failures; \ + } \ + } while (0) + +static PFNEGLCREATEIMAGEKHRPROC pCreateImage; +static PFNEGLDESTROYIMAGEKHRPROC pDestroyImage; +static PFNGLEGLIMAGETARGETTEXTURE2DOESPROC pTargetTexture; +static PFNEGLQUERYDMABUFFORMATSEXTPROC pQueryFormats; +static PFNEGLQUERYDMABUFMODIFIERSEXTPROC pQueryModifiers; +static PFNEGLCREATESYNCKHRPROC pCreateSync; +static PFNEGLDESTROYSYNCKHRPROC pDestroySync; +static EGLDisplay g_display; + +enum { W = 256, H = 128, BARS = 4 }; +static const unsigned char kBars[BARS][3] = {{255, 0, 0}, {0, 255, 0}, {0, 0, 255}, {255, 255, 255}}; + +// RGB -> BT.709 narrow-range codes at `bits` (8 or 10). +static void Encode709(const unsigned char rgb[3], int bits, int out[3]) { + const double kr = 0.2126, kb = 0.0722, kg = 1.0 - kr - kb; + const double r = rgb[0] / 255.0, g = rgb[1] / 255.0, b = rgb[2] / 255.0; + const double y = kr * r + kg * g + kb * b; + const double cb = (b - y) / (2.0 * (1.0 - kb)), cr = (r - y) / (2.0 * (1.0 - kr)); + const double scale = (double)(1 << (bits - 8)); + out[0] = (int)lround((16.0 + 219.0 * y) * scale); + out[1] = (int)lround((128.0 + 224.0 * cb) * scale); + out[2] = (int)lround((128.0 + 224.0 * cr) * scale); +} + +// Writes the bars (`reverse`d or not) into a semi-planar buffer: 1 byte samples, or P010's 16-bit +// samples with the 10 significant bits high. +static void FillBars(uint8_t* base, uint32_t pitchY, uint32_t offsetC, uint32_t pitchC, int bytes, int reverse) { + const int bits = bytes == 1 ? 8 : 10; + for (int y = 0; y < H; ++y) { + for (int x = 0; x < W; ++x) { + int bar = x * BARS / W; + if (reverse) bar = BARS - 1 - bar; + int code[3]; + Encode709(kBars[bar], bits, code); + uint8_t* py = base + (size_t)y * pitchY + (size_t)x * bytes; + if (bytes == 1) py[0] = (uint8_t)code[0]; + else *(uint16_t*)py = (uint16_t)(code[0] << 6); + if ((x & 1) == 0 && (y & 1) == 0) { + uint8_t* pc = base + offsetC + (size_t)(y / 2) * pitchC + (size_t)(x / 2) * 2 * bytes; + if (bytes == 1) { + pc[0] = (uint8_t)code[1]; + pc[1] = (uint8_t)code[2]; + } else { + ((uint16_t*)pc)[0] = (uint16_t)(code[1] << 6); + ((uint16_t*)pc)[1] = (uint16_t)(code[2] << 6); + } + } + } + } +} + +static GLuint Compile(GLenum type, const char* source) { + GLuint shader = glCreateShader(type); + glShaderSource(shader, 1, &source, NULL); + glCompileShader(shader); + GLint ok = 0; + glGetShaderiv(shader, GL_COMPILE_STATUS, &ok); + if (!ok) { + char log[2048]; + glGetShaderInfoLog(shader, sizeof(log), NULL, log); + printf("shader: %s\n", log); + } + return shader; +} + +static GLuint g_program, g_fbo, g_rgba; + +static void SetUpDraw(void) { + const char* vs = "#version 300 es\n" + "out vec2 uv;\n" + "void main() {\n" + " vec2 p = vec2(float((gl_VertexID << 1) & 2), float(gl_VertexID & 2));\n" + " gl_Position = vec4(p * 2.0 - 1.0, 0.0, 1.0);\n" + " uv = p;\n" + "}\n"; + const char* fs = "#version 300 es\n" + "#extension GL_OES_EGL_image_external_essl3 : require\n" + "precision mediump float;\n" + "uniform samplerExternalOES tex;\n" + "in vec2 uv;\n" + "out vec4 color;\n" + "void main() { color = texture(tex, uv); }\n"; + g_program = glCreateProgram(); + glAttachShader(g_program, Compile(GL_VERTEX_SHADER, vs)); + glAttachShader(g_program, Compile(GL_FRAGMENT_SHADER, fs)); + glLinkProgram(g_program); + GLint linked = 0; + glGetProgramiv(g_program, GL_LINK_STATUS, &linked); + CHECK(linked, "the samplerExternalOES program did not link"); + GLint type = 0, size = 0; + char name[64]; + glGetActiveUniform(g_program, 0, sizeof(name), NULL, &size, (GLenum*)&type, name); + printf("uniform 0: %s type 0x%04x\n", name, type); + CHECK(type == GL_SAMPLER_EXTERNAL_OES, "the sampler reports type 0x%04x, not GL_SAMPLER_EXTERNAL_OES", type); + glGenTextures(1, &g_rgba); + glBindTexture(GL_TEXTURE_2D, g_rgba); + glTexStorage2D(GL_TEXTURE_2D, 1, GL_RGBA8, W, H); + glGenFramebuffers(1, &g_fbo); + glBindFramebuffer(GL_FRAMEBUFFER, g_fbo); + glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, g_rgba, 0); + CHECK(glCheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE, "the readback FBO is incomplete"); + GLuint vao; + glGenVertexArrays(1, &vao); + glBindVertexArray(vao); +} + +// Draws the external texture `texture` and checks each bar's center (row `y` of the image). +static void DrawAndCheck(GLuint texture, const char* what, int reverse, int tolerance) { + glBindFramebuffer(GL_FRAMEBUFFER, g_fbo); + glViewport(0, 0, W, H); + glUseProgram(g_program); + glActiveTexture(GL_TEXTURE0); + glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture); + glUniform1i(glGetUniformLocation(g_program, "tex"), 0); + glDrawArrays(GL_TRIANGLES, 0, 3); + static unsigned char pixels[W * H * 4]; + glReadPixels(0, 0, W, H, GL_RGBA, GL_UNSIGNED_BYTE, pixels); + GLenum error = glGetError(); + CHECK(error == GL_NO_ERROR, "%s: GL error 0x%04x", what, error); + if (tolerance < 0) { + printf("%s: center pixel %d %d %d %d (content undefined)\n", what, pixels[(H / 2 * W + W / 2) * 4], + pixels[(H / 2 * W + W / 2) * 4 + 1], pixels[(H / 2 * W + W / 2) * 4 + 2], + pixels[(H / 2 * W + W / 2) * 4 + 3]); + return; + } + // Rows: GL row 0 is the image's first row (the uv mapping above), and the bars are vertical, + // so any row does; take two to catch a broken chroma row pitch. + const int rows[2] = {H / 4, (3 * H) / 4}; + for (int r = 0; r < 2; ++r) { + for (int bar = 0; bar < BARS; ++bar) { + const int x = bar * (W / BARS) + W / (2 * BARS); + const unsigned char* p = &pixels[(rows[r] * W + x) * 4]; + const unsigned char* want = kBars[reverse ? BARS - 1 - bar : bar]; + int worst = 0; + for (int c = 0; c < 3; ++c) { + int d = abs((int)p[c] - (int)want[c]); + if (d > worst) worst = d; + } + printf("%s: row %d bar %d -> %3d %3d %3d %3d (want %3d %3d %3d)%s\n", what, rows[r], bar, p[0], p[1], p[2], + p[3], want[0], want[1], want[2], worst > tolerance ? " <-- off" : ""); + CHECK(worst <= tolerance, "%s: bar %d is off by %d", what, bar, worst); + } + } +} + +static int AllocateDmaHeap(size_t size) { + int heap = open("/dev/dma_heap/system", O_RDWR | O_CLOEXEC); + if (heap < 0) { + perror("open /dev/dma_heap/system"); + return -1; + } + struct dma_heap_allocation_data data = {.len = size, .fd_flags = O_RDWR | O_CLOEXEC}; + if (ioctl(heap, DMA_HEAP_IOCTL_ALLOC, &data) != 0) { + perror("DMA_HEAP_IOCTL_ALLOC"); + close(heap); + return -1; + } + close(heap); + return (int)data.fd; +} + +static void Sync(int fd, int start) { + struct dma_buf_sync sync = {.flags = DMA_BUF_SYNC_WRITE | (start ? DMA_BUF_SYNC_START : DMA_BUF_SYNC_END)}; + ioctl(fd, DMA_BUF_IOCTL_SYNC, &sync); +} + +static EGLImageKHR Import(int fd, uint32_t fourcc, uint32_t pitchY, uint32_t offsetC, uint32_t pitchC, int modifier) { + EGLint attribs[64]; + int n = 0; +#define A(k, v) (attribs[n++] = (k), attribs[n++] = (EGLint)(v)) + A(EGL_WIDTH, W); + A(EGL_HEIGHT, H); + A(EGL_LINUX_DRM_FOURCC_EXT, fourcc); + A(EGL_DMA_BUF_PLANE0_FD_EXT, fd); + A(EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0); + A(EGL_DMA_BUF_PLANE0_PITCH_EXT, pitchY); + A(EGL_DMA_BUF_PLANE1_FD_EXT, fd); + A(EGL_DMA_BUF_PLANE1_OFFSET_EXT, offsetC); + A(EGL_DMA_BUF_PLANE1_PITCH_EXT, pitchC); + if (modifier) { + A(EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT, 0); + A(EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT, 0); + A(EGL_DMA_BUF_PLANE1_MODIFIER_LO_EXT, 0); + A(EGL_DMA_BUF_PLANE1_MODIFIER_HI_EXT, 0); + } + A(EGL_YUV_COLOR_SPACE_HINT_EXT, EGL_ITU_REC709_EXT); + A(EGL_SAMPLE_RANGE_HINT_EXT, EGL_YUV_NARROW_RANGE_EXT); + attribs[n] = EGL_NONE; +#undef A + EGLImageKHR image = pCreateImage(g_display, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, attribs); + if (image == EGL_NO_IMAGE_KHR) printf("eglCreateImage failed: 0x%04x\n", eglGetError()); + return image; +} + +static GLuint ExternalTexture(EGLImageKHR image) { + GLuint texture; + glGenTextures(1, &texture); + glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture); + pTargetTexture(GL_TEXTURE_EXTERNAL_OES, image); + GLenum error = glGetError(); + CHECK(error == GL_NO_ERROR, "glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES) -> 0x%04x", error); + GLint bound = 0; + glGetIntegerv(GL_TEXTURE_BINDING_EXTERNAL_OES, &bound); + CHECK((GLuint)bound == texture, "GL_TEXTURE_BINDING_EXTERNAL_OES reads %d", bound); + return texture; +} + +// A frame boundary as an offscreen producer makes one: a native fence. +static void EndFrame(void) { + EGLint attribs[] = {EGL_NONE}; + EGLSyncKHR sync = pCreateSync(g_display, EGL_SYNC_NATIVE_FENCE_ANDROID, attribs); + CHECK(sync != EGL_NO_SYNC_KHR, "eglCreateSync(native fence) failed"); + if (sync != EGL_NO_SYNC_KHR) pDestroySync(g_display, sync); +} + +static void ForeignTest(uint32_t fourcc, int bytes, int tolerance) { + const char* what = fourcc == DRM_FORMAT_NV12 ? "foreign NV12" : "foreign P010"; + // Padded pitches and a gap before the chroma plane, as decoders lay buffers out. + const uint32_t pitchY = W * bytes + 64, pitchC = W * bytes + 128; + const uint32_t offsetC = pitchY * (H + 16); + const size_t size = offsetC + (size_t)pitchC * (H / 2) + 4096; + int fd = AllocateDmaHeap(size); + if (fd < 0) { + CHECK(0, "%s: no dma-buf to test with", what); + return; + } + uint8_t* map = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + CHECK(map != MAP_FAILED, "%s: mmap failed", what); + if (map == MAP_FAILED) return; + Sync(fd, 1); + FillBars(map, pitchY, offsetC, pitchC, bytes, 0); + Sync(fd, 0); + EGLImageKHR image = Import(fd, fourcc, pitchY, offsetC, pitchC, fourcc == DRM_FORMAT_NV12); + CHECK(image != EGL_NO_IMAGE_KHR, "%s: import refused", what); + if (image == EGL_NO_IMAGE_KHR) return; + GLuint texture = ExternalTexture(image); + DrawAndCheck(texture, what, 0, tolerance); + EndFrame(); + // New content, same buffer: shows after the frame boundary. + Sync(fd, 1); + FillBars(map, pitchY, offsetC, pitchC, bytes, 1); + Sync(fd, 0); + EndFrame(); + DrawAndCheck(texture, what, 1, tolerance); + glDeleteTextures(1, &texture); + pDestroyImage(g_display, image); + munmap(map, size); + close(fd); +} + +static void GbmTest(void) { + int node = open("/dev/dri/renderD128", O_RDWR | O_CLOEXEC); + if (node < 0) node = open("/dev/null", O_RDWR | O_CLOEXEC); + struct gbm_device* gbm = gbm_create_device(node); + if (gbm == NULL) { + CHECK(0, "gbm_create_device failed"); + return; + } + printf("gbm backend: %s\n", gbm_device_get_backend_name(gbm)); + const int supported = gbm_device_is_format_supported(gbm, DRM_FORMAT_NV12, 0); + printf("gbm NV12 supported: %d, P010: %d\n", supported, + gbm_device_is_format_supported(gbm, DRM_FORMAT_P010, 0)); + CHECK(supported == 1, "gbm does not support NV12"); + struct gbm_bo* bo = gbm_bo_create(gbm, W, H, DRM_FORMAT_NV12, 0); + CHECK(bo != NULL, "gbm_bo_create(NV12) failed"); + if (bo != NULL) { + const int planes = gbm_bo_get_plane_count(bo); + printf("gbm NV12 bo: planes %d, stride %u/%u, offset %u/%u, modifier 0x%llx\n", planes, + gbm_bo_get_stride_for_plane(bo, 0), gbm_bo_get_stride_for_plane(bo, 1), gbm_bo_get_offset(bo, 0), + gbm_bo_get_offset(bo, 1), (unsigned long long)gbm_bo_get_modifier(bo)); + CHECK(planes == 2, "the NV12 bo has %d planes", planes); + int fd = gbm_bo_get_fd(bo); + EGLImageKHR image = + Import(fd, DRM_FORMAT_NV12, gbm_bo_get_stride_for_plane(bo, 0), gbm_bo_get_offset(bo, 1), + gbm_bo_get_stride_for_plane(bo, 1), 0); + CHECK(image != EGL_NO_IMAGE_KHR, "the server's own NV12 buffer was not imported"); + if (image != EGL_NO_IMAGE_KHR) { + GLuint texture = ExternalTexture(image); + DrawAndCheck(texture, "gbm NV12", 0, -1); + glDeleteTextures(1, &texture); + pDestroyImage(g_display, image); + } + close(fd); + gbm_bo_destroy(bo); + } + gbm_device_destroy(gbm); + close(node); +} + +int main(void) { + setvbuf(stdout, NULL, _IONBF, 0); + g_display = eglGetDisplay(EGL_DEFAULT_DISPLAY); + EGLint major, minor; + if (!eglInitialize(g_display, &major, &minor)) { + printf("FAIL: eglInitialize\n"); + return 2; + } + const char* eglExtensions = eglQueryString(g_display, EGL_EXTENSIONS); + printf("EGL %d.%d dma_buf_import: %s\n", major, minor, + strstr(eglExtensions, "EGL_EXT_image_dma_buf_import") ? "yes" : "no"); + eglBindAPI(EGL_OPENGL_ES_API); + const EGLint configAttribs[] = {EGL_SURFACE_TYPE, EGL_PBUFFER_BIT, EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT, + EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_NONE}; + EGLConfig config; + EGLint count = 0; + eglChooseConfig(g_display, configAttribs, &config, 1, &count); + if (count < 1) { + printf("FAIL: no config\n"); + return 2; + } + const EGLint pbufferAttribs[] = {EGL_WIDTH, 16, EGL_HEIGHT, 16, EGL_NONE}; + EGLSurface surface = eglCreatePbufferSurface(g_display, config, pbufferAttribs); + const EGLint contextAttribs[] = {EGL_CONTEXT_MAJOR_VERSION, 3, EGL_NONE}; + EGLContext context = eglCreateContext(g_display, config, EGL_NO_CONTEXT, contextAttribs); + if (context == EGL_NO_CONTEXT || !eglMakeCurrent(g_display, surface, surface, context)) { + printf("FAIL: no ES 3 context\n"); + return 2; + } + pCreateImage = (PFNEGLCREATEIMAGEKHRPROC)eglGetProcAddress("eglCreateImageKHR"); + pDestroyImage = (PFNEGLDESTROYIMAGEKHRPROC)eglGetProcAddress("eglDestroyImageKHR"); + pTargetTexture = (PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)eglGetProcAddress("glEGLImageTargetTexture2DOES"); + pQueryFormats = (PFNEGLQUERYDMABUFFORMATSEXTPROC)eglGetProcAddress("eglQueryDmaBufFormatsEXT"); + pQueryModifiers = (PFNEGLQUERYDMABUFMODIFIERSEXTPROC)eglGetProcAddress("eglQueryDmaBufModifiersEXT"); + pCreateSync = (PFNEGLCREATESYNCKHRPROC)eglGetProcAddress("eglCreateSyncKHR"); + pDestroySync = (PFNEGLDESTROYSYNCKHRPROC)eglGetProcAddress("eglDestroySyncKHR"); + printf("GL_RENDERER: %s\n", glGetString(GL_RENDERER)); + const char* glExtensions = (const char*)glGetString(GL_EXTENSIONS); + const int external = strstr(glExtensions, "GL_OES_EGL_image_external_essl3") != NULL; + printf("GL_OES_EGL_image_external_essl3: %s\n", external ? "yes" : "no"); + CHECK(external, "GL_OES_EGL_image_external_essl3 is not advertised"); + + EGLint formats[32], formatCount = 0; + pQueryFormats(g_display, 32, formats, &formatCount); + int nv12 = 0; + for (int i = 0; i < formatCount; ++i) { + printf("dma-buf format %.4s", (const char*)&formats[i]); + EGLuint64KHR modifiers[4]; + EGLBoolean externalOnly[4]; + EGLint modifierCount = 0; + pQueryModifiers(g_display, formats[i], 4, modifiers, externalOnly, &modifierCount); + for (int m = 0; m < modifierCount; ++m) + printf(" [modifier 0x%llx%s]", (unsigned long long)modifiers[m], externalOnly[m] ? " external-only" : ""); + printf("\n"); + nv12 |= formats[i] == (EGLint)DRM_FORMAT_NV12; + } + CHECK(nv12, "NV12 is not a listed dma-buf format"); + + SetUpDraw(); + ForeignTest(DRM_FORMAT_NV12, 1, 12); + ForeignTest(DRM_FORMAT_P010, 2, 16); + GbmTest(); + + printf("%s (%d failure(s))\n", g_failures ? "FAILED" : "PASSED", g_failures); + eglMakeCurrent(g_display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); + eglTerminate(g_display); + return g_failures ? 1 : 0; +} diff --git a/README.md b/README.md index 4da0ac05..2ac34875 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ The repository is organized into following top-level modules: 2. **MG_Impl** — front-end implementations of Graphics APIs that interact with `MG_State` and `MG_Backend`: OpenGL, plus the window-system APIs EGL, GLX, WGL, CGL and NSOpenGL. 3. **MG_Backend** — per-backend translation layer that maps front-end Graphics APIs' semantics and state into concrete backend API calls (e.g. OpenGL ES, Vulkan). 4. **MG_Pipe** and **MG_Remote** — the command stream between front-end and backend, and the client/server transport that lets the backend run in a separate process (see `docs/Disaggregated`). -5. **MG_Gbm** — a GBM backend whose buffers are images shared with the MobileGL server, so Wayland clients and compositors can exchange them as dma-bufs. +5. **MG_Gbm** — a GBM backend whose buffers are images shared with the MobileGL server, so Wayland clients and compositors can exchange them as dma-bufs (RGBA, and NV12/P010 YUV where the server's platform allocates them). YUV dma-bufs - the server's own or a video decoder's - import through `EGL_EXT_image_dma_buf_import` and are sampled through `GL_OES_EGL_image_external(_essl3)`: the backend converts the image into the texture's RGBA level at its first use in each frame (zero-copy for a server image; a foreign buffer, which the platform cannot import, is first copied into one by the CPU). 6. **MG_Util** and other utility modules. ## Third-party components @@ -173,6 +173,7 @@ MobileGL supports runtime configuration via environment variables. | `MOBILEGL_ESPRYT_AVOID_SAMPLER_MIPMAP_MIN_FILTER` | Avoid sampler mipmap minification filters. | `0`, `1` | `0` | | `MOBILEGL_COHERENT_AS_FLUSH` | Treat persistent `GL_MAP_FLUSH_EXPLICIT_BIT` maps as coherent (app-compat for engines like Flywheel that never flush them). | `0`, `1` | `0` | | `MOBILEGL_ESPRYT_FORCE_DS_READBACK_EMULATION` | Always emulate depth/stencil `glReadPixels`/`glGetTexImage` by shader sampling on Espryt, instead of using the driver's own depth/stencil readback where it has one. | `0`, `1` | `0` | +| `MOBILEGL_YUV_DRIVER_CONVERSION` | Espryt: convert YUV images with the driver's external-sampler conversion (its own colour space) instead of `GL_EXT_YUV_target` with the import's `EGL_YUV_COLOR_SPACE_HINT_EXT` / `EGL_SAMPLE_RANGE_HINT_EXT`. | `0`, `1` | `0` | | `MOBILEGL_SERVER_FRAME_STATS` | Log one `P65ServerFrame` timing line per presented frame on the server (apply vs. wall time, socket read rate). On Android also `debug.mobilegl.server_frame_stats`. | `0`, `1` | `0` | | `VK_ICD_FILENAMES` | Select the Vulkan ICD used by the Vulkan loader. | Path to an ICD JSON file | Loader default | diff --git a/docs/Disaggregated/notes/anland/runbook-plasma.md b/docs/Disaggregated/notes/anland/runbook-plasma.md index 16fbe1eb..b990b7cf 100644 --- a/docs/Disaggregated/notes/anland/runbook-plasma.md +++ b/docs/Disaggregated/notes/anland/runbook-plasma.md @@ -124,6 +124,8 @@ sleep 8; grep -a "presented through" /tmp/mgl-run.client.log Chrome：`bash scripts/chrome-launch.sh [参数]`（日志 `/tmp/mgl-chrome-.client.log`，截图 `ch-.png`）。探针：`scripts/probes/texfmt.c`（各纹理格式是否保住 alpha）、`dmahash.c`（对某进程的 dma-buf 取样哈希）；放进容器 `/root` 编，例如 `gcc -O1 /root/texfmt.c -o /usr/local/bin/mgl-texfmt -lEGL`，用 `mgrun offscreen` 跑（之前的确切编译参数未验证）。 +视频：Chrome 走硬解（VaapiVideoDecoder）。容器的 VA-API 驱动（`msm_drm_drv_video.so`，V4L2 解码）自己分配 surface 并导出 NV12 dma-buf（单 fd，R8 + GR88 两层），Chrome 以 NV12/P010 EGLImage 绑到 `GL_TEXTURE_EXTERNAL_OES`。这种**外来** YUV dma-buf 平台导入不了，MobileGL 走明确标注的 CPU 拷贝兜底：server 映射它、每帧首次使用时拷进自己的 YUV AHB（`AHardwareBuffer_lockPlanes`），再在 GPU 上转换进纹理的 RGBA 层（Espryt：`GL_EXT_YUV_target` + 导入时的色彩 hint，`MOBILEGL_YUV_DRIVER_CONVERSION=1` 改用驱动的 external sampler；Magma：AHB external format 上的 `VkSamplerYcbcrConversion`）。server 自己分配的 NV12/P010（GBM）零拷贝。`MOBILEGL_CHROME_VIDEO_DECODE=software` 恢复 `--disable-accelerated-video-decode`。检查：容器里 `python3 scripts/probes/cdpmedia.py <秒>`（CDP `Media.enable`，`kVideoDecoderName`），`scripts/probes/yuvprobe.c` 像素校验外来 NV12/P010 导入、逐帧刷新和 GBM NV12。KWin 自己的 NV12 路径（拆成 R8 + GR88 两个导入）不支持，KWin 不通告 YUV 格式。 + 结论成立要同时满足： - client 日志是 `presented through linux-dmabuf shared images`，不是 `wl_shm`；出现 `could not be presented through a linux-dmabuf shared image` 说明退回了；