Compare commits

...
Author SHA1 Message Date
swung0x48 c5569e71b3 [Feat] (tools/cts): automate Windows WGL conformance runs 2026-07-30 23:00:13 -04:00
swung0x48 7e8c32a063 [Feat] (MG_Backend, MG_Impl): expose experimental GL 4.6 CTS limits 2026-07-30 23:00:13 -04:00
swung0x48 77bd03d962 [Chore]: remove unnecessary doc 2026-07-30 21:56:17 -04:00
swung0x48 37111ae992 [Perf] (DirectVulkan): snapshot-gated consecutive-draw fast path skips SetupDraw re-resolution 2026-07-30 09:40:54 -04:00
swung0x48 6b0c2a15ab [Perf] (DirectVulkan): reuse unchanged global-UBO slices and skip identical descriptor binds 2026-07-30 08:18:21 -04:00
swung0x48 e9ffd99313 [Perf] (DirectVulkan): bake the attribute location mask and memoize the explicit-LOD eligibility probe 2026-07-30 08:18:20 -04:00
swung0x48 7c01ddea0c [Perf] (DirectVulkan): drop per-draw weak-ptr locks, re-resolves and rebuilt masks from the sampled-texture and vertex paths 2026-07-30 08:01:07 -04:00
swung0x48 2d4d6e9cfb [Perf] (DirectVulkan): skip pending-clear probes through a lock-free empty check 2026-07-30 07:02:03 -04:00
swung0x48 76b8957b99 [Perf] (DirectVulkan): memoize sampled-texture resources across draws 2026-07-30 07:02:02 -04:00
swung0x48 ec685b9fa7 [Perf] (DirectVulkan): memoize resolved vertex-input state on the VAO and dedupe vertex/index binds 2026-07-30 07:02:01 -04:00
swung0x48 a12068df52 [Perf] (DirectVulkan): reuse pipelines across per-chunk buffers and skip redundant pipeline binds 2026-07-30 05:01:46 -04:00
swung0x48 0b344792cc [Fix] (DirectVulkan): stop fence-waiting out-of-band texture uploads; reclaim transients asynchronously 2026-07-30 05:01:46 -04:00
swung0x48 9fa32bdad0 [Fix] (DirectVulkan): declare only the used colour attachment span per subpass
- every render pass declared colorAttachmentCount=8 (the full GL draw-buffer
  slot span) with trailing VK_ATTACHMENT_UNUSED references, and Adreno
  configures its per-pixel render-backend/export path from the DECLARED
  count - so every fragment of every pass paid an 8-render-target export
  cost; this was the bulk of the 1.5x per-pixel gap against
  MobileGlues+ANGLE on the same Qualcomm driver (their subpasses declare
  exactly the used span)
- measured on Adreno 650 / MC 26.2 / 1440x3044: total GPU frame time
  11.9 -> 7.5 ms (-37%, now below ANGLE's 7.87 ms), the single-quad
  swapchain blit pass alone 1.26 -> 0.40 ms, steady in-world FPS 82.8 -> 123
  under the standard cooled-start protocol, matching the
  MobileGlues+ANGLE+system-Vulkan benchmark of 123.8
- trailing UNUSED references are popped before the subpass is built (the
  entry's colorAttachmentCount and every pipeline's colour-blend span follow
  it); interior GL_NONE holes keep their slots so fragment-output locations
  still line up
- the pipeline-side fragmentOutputMask check downgrades from assert to a
  debug log: an output at a location past the trimmed span is discarded,
  which is GL's defined behaviour for a draw buffer set to GL_NONE
2026-07-30 02:45:39 -04:00
swung0x48 a4980f2b56 [Fix] (DirectVulkan): skip redundant per-draw dynamic-state commands
- viewport, scissor, blend constants, depth bias, line width and the six
  stencil parameters were re-emitted unconditionally for EVERY draw (~1500
  vkCmdSet* per frame in MC 26.2, where ANGLE emits a handful), costing CPU
  record time and GPU command-processor work for values that almost never
  change between draws
- a recording-scoped shadow now drops any vkCmdSet* whose values match what
  the command buffer already holds; valid because every PipelineFactory
  pipeline declares the same eight dynamic states, so set values persist
  across those binds
- the shadow resets at every command-buffer (re)begin (dynamic state does
  not survive the boundary) and after binding the blit or depth-mipmap
  pipelines, whose narrower dynamic sets make the untouched states undefined
  and whose raw viewport/scissor writes bypass the shadow
2026-07-30 02:45:07 -04:00
swung0x48 8ca20e28ca [Fix] (DirectVulkan): size texture backings by their defined mip level count
- every non-MSAA texture was allocated with a full mip chain regardless of
  how many levels the GL texture actually defines, so MC's 3044x1440 main
  colour and depth render targets each carried 12 levels where ANGLE
  allocates one; a level-0-only texture now gets a single-level backing and
  upgrades to the full chain exactly once when a second level is first
  defined, through the existing preserve-copy recreation path
- saves a third of the memory of every mip-less texture and keeps
  single-level render targets off the multi-mip image layout entirely, which
  also removes the surface the Adreno 650 implicit-LOD overread workaround
  (ForceExplicitLod0SamplePass) exists to defend
- measured perf-neutral on Adreno 650 / MC 26.2 (the driver keeps full UBWC
  on multi-mip render targets), so this is a memory/robustness fix, not a
  speed one
2026-07-30 02:43:32 -04:00
swung0x48 c353a2055f [Feat] (DirectVulkan): pre-pass command stream for reorderable out-of-pass work
- a draw whose sampled texture needs out-of-pass work (deferred clear
  materialization or a sampled-layout transition) used to end the active
  render pass - a full-target store+reload on a tiler - even when the only
  ordering the work needs is 'before this draw'; MC 26.2 clears an overlay
  texture every frame and samples it mid-pass, splitting the main scene pass
  once per frame for nothing
- every frame slot now carries a second primary command buffer, submitted
  strictly AHEAD of the frame command buffer in the same vkQueueSubmit; when
  the open recording has not referenced the image yet (tracked via a
  recording-generation stamp on the texture resource, advanced on every
  frame-command-buffer begin and stamped at every recorded reference:
  attachments at BeginRenderPass/attachment-write, sampled reads per draw,
  layout transitions), the clear/transition is recorded there and the active
  pass stays open - ANGLE's outside-render-pass command stream, restricted
  to the provably reorderable case
- mid-frame flushes and readback submits close and carry the pre stream with
  the frame buffer (it must never be submitted later than the recording it
  was paired with), retiring both under the same submit index; dropped
  recordings (present suspension, swapchain recreation) abandon it
- MaterializePendingClearForTexture's no-active-render-pass assert now
  applies only to the frame command buffer, since the pre stream records
  while a pass is open on the frame buffer by design
2026-07-30 02:43:13 -04:00
swung0x48 421c20984e [Fix] (DirectVulkan): stop loading and carrying dead default-framebuffer content
- EGL swap semantics make the presented colour buffer's content undefined at
  its next acquire (EGL_BUFFER_DESTROYED, the implementation default) and
  every ancillary depth/stencil buffer's content undefined after ANY swap,
  yet the default-FBO render pass reloaded both with LOAD_OP_LOAD every
  frame; SwapchainObject now tracks per-image content validity (defined when
  a pass stores into the attachment, invalidated at present) and the
  render-pass manager turns an undefined attachment's tile load into
  LOAD_OP_DONT_CARE with initialLayout=UNDEFINED, keyed into both hashes so
  the cached LOAD variants cannot be hit by mistake
- the default framebuffer's depth attachment is now attached ON DEMAND: a
  draw with depth test and stencil test both disabled (GL: a disabled test
  neither reads nor writes its buffer), and no pending depth/stencil clear,
  resolves to a depth-less pass flavour, dropping the D24S8 tile load AND
  store outright - MC 26.2 renders its GUI into its own FBO and only ever
  blits colour to the default framebuffer, so its swapchain pass carried a
  full-screen depth round-trip for nothing
- the flavour only escalates: an active depth-full pass absorbs depth-less
  draws unchanged, while a depth-using draw against a depth-less pass
  resolves to an incompatible entry and splits, its depth loading DONT_CARE
  (the content was undefined all along); the depth-less flavour is folded
  into ComputeHash and the per-draw fast-path memo so the two flavours can
  never alias
2026-07-30 02:40:47 -04:00
41 changed files with 6007 additions and 259 deletions
+2
View File
@@ -25,3 +25,5 @@ MobileGL/MG*/cmake-build*
/android-plugin/app/src/trace/jniLibs
/android-plugin/local.properties
tools/trace_replay/work/
__pycache__/
*.py[cod]
+7
View File
@@ -300,6 +300,13 @@ namespace MobileGL {
Float ViewportBoundsRangeMin = 0.0f;
Float ViewportBoundsRangeMax = 0.0f;
Int ViewportSubpixelBits = 0;
// GL 4.x fragment-interpolation offset limits. These defaults are the
// core minimums and are replaced by live GLES/Vulkan device limits.
Float MinFragmentInterpolationOffset = -0.5f;
// For four fractional bits the greatest required legal offset is
// 0.5 - 2^-4 = 0.4375 (GL 4.6 table 23.70).
Float MaxFragmentInterpolationOffset = 0.4375f;
Int FragmentInterpolationOffsetBits = 4;
Bool SupportsWideLines = false;
SizeT MaxShaderStorageBlockSize = 128 * 1024 * 1024;
Uint32 SubgroupSize = 0;
@@ -20,6 +20,7 @@
#include <MG_Util/Texture/TextureFormatProcessor.h>
#include <Config.h>
#include <algorithm>
#include <cmath>
#include <format>
namespace MobileGL::MG_Backend::DirectGLES {
@@ -603,7 +604,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
.ExtraVendor = Nullopt, // Extra vendor
.RendererGLInfo =
{
.TargetGLVersion = {3, 3, 0}, // Target OpenGL Version
.TargetGLVersion = {4, 6, 0}, // Experimental GL CTS target version
.TargetGLSLVersion = {4, 6, 0}, // Target Shading Language Version
// Baseline advertisement (no timer queries / anisotropy yet); reconciled
// once the ES capabilities exist, see UpdateAdvertisedCapabilityExtensions.
@@ -1034,6 +1035,24 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_dynamicParameters.ViewportBoundsRangeMin = m_GLESCapabilities.ViewportBoundsRangeMin;
m_dynamicParameters.ViewportBoundsRangeMax = m_GLESCapabilities.ViewportBoundsRangeMax;
m_dynamicParameters.ViewportSubpixelBits = m_GLESCapabilities.ViewportSubpixelBits;
m_dynamicParameters.MinFragmentInterpolationOffset =
std::isfinite(m_GLESCapabilities.MinFragmentInterpolationOffset) &&
m_GLESCapabilities.MinFragmentInterpolationOffset <= -0.5f
? m_GLESCapabilities.MinFragmentInterpolationOffset
: -0.5f;
m_dynamicParameters.MaxFragmentInterpolationOffset = 0.4375f;
m_dynamicParameters.FragmentInterpolationOffsetBits = 4;
if (m_GLESCapabilities.FragmentInterpolationOffsetBits >= 4 &&
std::isfinite(m_GLESCapabilities.MaxFragmentInterpolationOffset)) {
const Float requiredMaxOffset =
0.5f - std::ldexp(1.0f, -m_GLESCapabilities.FragmentInterpolationOffsetBits);
if (m_GLESCapabilities.MaxFragmentInterpolationOffset >= requiredMaxOffset) {
m_dynamicParameters.MaxFragmentInterpolationOffset =
m_GLESCapabilities.MaxFragmentInterpolationOffset;
m_dynamicParameters.FragmentInterpolationOffsetBits =
m_GLESCapabilities.FragmentInterpolationOffsetBits;
}
}
m_dynamicParameters.SupportsWideLines =
m_GLESCapabilities.AliasedLineWidthRangeMax > 1.0f || m_GLESCapabilities.SmoothLineWidthRangeMax > 1.0f;
@@ -18,6 +18,7 @@
#include "MG_Util/Texture/TextureFormatProcessor.h"
#include <Config.h>
#include <cmath>
#include <cstdlib>
#include <cstring>
@@ -505,7 +506,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.ExtraVendor = Nullopt,
.RendererGLInfo =
{
.TargetGLVersion = {3, 3, 0},
.TargetGLVersion = {4, 6, 0}, // Experimental GL CTS target version
.TargetGLSLVersion = {4, 6, 0},
// Baseline advertisement (no shader subgroup, no timer queries); a
// live backend reconciles its copy in UpdateAdvertisedExtensions.
@@ -796,6 +797,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_dynamicParameters.ViewportBoundsRangeMin = m_vulkanCaps.ViewportBoundsRangeMin;
m_dynamicParameters.ViewportBoundsRangeMax = m_vulkanCaps.ViewportBoundsRangeMax;
m_dynamicParameters.ViewportSubpixelBits = m_vulkanCaps.ViewportSubpixelBits;
m_dynamicParameters.MinFragmentInterpolationOffset =
std::isfinite(m_vulkanCaps.MinFragmentInterpolationOffset) &&
m_vulkanCaps.MinFragmentInterpolationOffset <= -0.5f
? m_vulkanCaps.MinFragmentInterpolationOffset
: -0.5f;
m_dynamicParameters.MaxFragmentInterpolationOffset = 0.4375f;
m_dynamicParameters.FragmentInterpolationOffsetBits = 4;
if (m_vulkanCaps.FragmentInterpolationOffsetBits >= 4 &&
std::isfinite(m_vulkanCaps.MaxFragmentInterpolationOffset)) {
const Float requiredMaxOffset =
0.5f - std::ldexp(1.0f, -m_vulkanCaps.FragmentInterpolationOffsetBits);
if (m_vulkanCaps.MaxFragmentInterpolationOffset >= requiredMaxOffset) {
m_dynamicParameters.MaxFragmentInterpolationOffset = m_vulkanCaps.MaxFragmentInterpolationOffset;
m_dynamicParameters.FragmentInterpolationOffsetBits =
m_vulkanCaps.FragmentInterpolationOffsetBits;
}
}
m_dynamicParameters.SupportsWideLines = m_vulkanCaps.SupportsWideLines;
m_dynamicParameters.MaxShaderStorageBlockSize =
std::min(m_vulkanCaps.MaxShaderStorageBlockSize, kMaxAdvertisedShaderStorageBlockSize);
@@ -16,18 +16,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_device = device;
m_commandPool = commandPool;
Vector<VkCommandBuffer> commandBuffers(frameCount, VK_NULL_HANDLE);
Vector<VkCommandBuffer> commandBuffers(frameCount * 2, VK_NULL_HANDLE);
VkCommandBufferAllocateInfo allocInfo{};
allocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
allocInfo.commandPool = commandPool;
allocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
allocInfo.commandBufferCount = frameCount;
allocInfo.commandBufferCount = frameCount * 2;
VkResult result = vkAllocateCommandBuffers(device, &allocInfo, commandBuffers.data());
if (result != VK_SUCCESS) {
return result;
}
for (Uint32 i = 0; i < frameCount; ++i) {
m_frames[i].commandBuffer = commandBuffers[i];
m_frames[i].preCommandBuffer = commandBuffers[frameCount + i];
}
VkSemaphoreCreateInfo semaphoreInfo{VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO};
@@ -47,9 +48,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void FrameContext::Destroy(VkDevice device, VkCommandPool commandPool) {
const Uint32 frameCount = static_cast<Uint32>(m_frames.size());
Vector<VkCommandBuffer> commandBuffers(frameCount, VK_NULL_HANDLE);
Vector<VkCommandBuffer> commandBuffers(frameCount * 2, VK_NULL_HANDLE);
for (Uint32 i = 0; i < frameCount; ++i) {
commandBuffers[i] = m_frames[i].commandBuffer;
commandBuffers[frameCount + i] = m_frames[i].preCommandBuffer;
}
for (Uint32 i = 0; i < frameCount; ++i) {
@@ -60,7 +62,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (auto& frame : m_frames) {
FreeRetiredCommandBuffers(frame);
}
vkFreeCommandBuffers(device, commandPool, frameCount, commandBuffers.data());
vkFreeCommandBuffers(device, commandPool, frameCount * 2, commandBuffers.data());
}
m_frames.clear();
currentFrameIndex = 0;
@@ -87,6 +89,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
currentFrameIndex = (currentFrameIndex + 1) % static_cast<Uint32>(m_frames.size());
GetCurrent().isCommandRecording = false;
GetCurrent().hasCommandBufferRecorded = false;
GetCurrent().isPreCommandRecording = false;
GetCurrent().hasPreCommandBufferRecorded = false;
}
VkCommandBuffer& FrameContext::BeginCommandRecording(VkCommandBufferUsageFlags flags,
@@ -118,6 +122,41 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frame.hasCommandBufferRecorded = true;
}
VkCommandBuffer FrameContext::BeginPreCommandRecording() {
auto& frame = GetCurrent();
if (frame.isPreCommandRecording) {
return frame.preCommandBuffer;
}
MOBILEGL_ASSERT(!frame.hasPreCommandBufferRecorded,
"BeginPreCommandRecording: a recorded pre stream is still awaiting submission");
VK_VERIFY(vkResetCommandBuffer(frame.preCommandBuffer, 0), "BeginPreCommandRecording, vkResetCommandBuffer");
VkCommandBufferBeginInfo beginInfo{};
beginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
VK_VERIFY(vkBeginCommandBuffer(frame.preCommandBuffer, &beginInfo),
"BeginPreCommandRecording, vkBeginCommandBuffer");
frame.isPreCommandRecording = true;
return frame.preCommandBuffer;
}
void FrameContext::EndPreCommandRecordingIfOpen() {
auto& frame = GetCurrent();
if (!frame.isPreCommandRecording) {
return;
}
VK_VERIFY(vkEndCommandBuffer(frame.preCommandBuffer), "EndPreCommandRecordingIfOpen, vkEndCommandBuffer");
frame.isPreCommandRecording = false;
frame.hasPreCommandBufferRecorded = true;
}
void FrameContext::AbandonPreCommandRecording() {
auto& frame = GetCurrent();
if (frame.isPreCommandRecording) {
VK_VERIFY(vkEndCommandBuffer(frame.preCommandBuffer), "AbandonPreCommandRecording, vkEndCommandBuffer");
}
frame.isPreCommandRecording = false;
frame.hasPreCommandBufferRecorded = false;
}
VkResult FrameContext::InitializeSwapchainSemaphores(VkDevice device, Uint32 swapchainImageCount) {
DestroySwapchainSemaphores(device);
if (swapchainImageCount == 0) {
@@ -202,17 +241,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 swapchainImageIndex) const {
const auto& frame = GetCurrent();
MOBILEGL_ASSERT(!frame.isCommandRecording, "GetSubmitInfo called while command buffer recording is still active");
MOBILEGL_ASSERT(!frame.isPreCommandRecording,
"GetSubmitInfo called while the pre-pass stream is still recording");
AssertValidSwapchainImageIndex(swapchainImageIndex);
SubmitInfoPacket packet{};
packet.waitSemaphore = frame.imageAvailableSemaphore;
packet.signalSemaphore = m_swapchainImageRenderFinishedSemaphores[swapchainImageIndex];
packet.commandBuffer = frame.commandBuffer;
Uint32 commandBufferCount = 0;
// The pre-pass stream executes strictly before the frame's commands.
if (frame.hasPreCommandBufferRecorded) {
packet.commandBuffers[commandBufferCount++] = frame.preCommandBuffer;
}
if (shouldSubmitCommandBuffer) {
packet.commandBuffers[commandBufferCount++] = frame.commandBuffer;
}
packet.submitInfo.waitSemaphoreCount = frame.imageAvailableSemaphoreConsumed ? 0U : 1U;
packet.submitInfo.pWaitSemaphores = frame.imageAvailableSemaphoreConsumed ? nullptr : &packet.waitSemaphore;
packet.submitInfo.pWaitDstStageMask = frame.imageAvailableSemaphoreConsumed ? nullptr : &packet.waitDstStageMask;
packet.submitInfo.commandBufferCount = shouldSubmitCommandBuffer ? 1U : 0U;
packet.submitInfo.pCommandBuffers = shouldSubmitCommandBuffer ? &packet.commandBuffer : nullptr;
packet.submitInfo.commandBufferCount = commandBufferCount;
packet.submitInfo.pCommandBuffers = commandBufferCount > 0 ? packet.commandBuffers : nullptr;
packet.submitInfo.signalSemaphoreCount = 1;
packet.submitInfo.pSignalSemaphores = &packet.signalSemaphore;
return packet;
@@ -276,12 +325,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_recordingObserver = observer;
}
VkResult FrameContext::RetireCurrentCommandBuffer() {
VkResult FrameContext::RetireCurrentCommandBuffer(Bool retirePreCommandBuffer) {
MOBILEGL_ASSERT(m_device != VK_NULL_HANDLE && m_commandPool != VK_NULL_HANDLE,
"RetireCurrentCommandBuffer requires an initialized FrameContext");
auto& frame = GetCurrent();
MOBILEGL_ASSERT(!frame.isCommandRecording,
"RetireCurrentCommandBuffer called while the command buffer is still recording");
MOBILEGL_ASSERT(!frame.isPreCommandRecording,
"RetireCurrentCommandBuffer called while the pre-pass stream is still recording");
VkCommandBufferAllocateInfo allocInfo{};
allocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
@@ -289,10 +340,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
allocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
allocInfo.commandBufferCount = 1;
VkCommandBuffer replacement = VK_NULL_HANDLE;
const VkResult result = vkAllocateCommandBuffers(m_device, &allocInfo, &replacement);
VkResult result = vkAllocateCommandBuffers(m_device, &allocInfo, &replacement);
if (result != VK_SUCCESS) {
return result;
}
if (retirePreCommandBuffer) {
VkCommandBuffer preReplacement = VK_NULL_HANDLE;
result = vkAllocateCommandBuffers(m_device, &allocInfo, &preReplacement);
if (result != VK_SUCCESS) {
vkFreeCommandBuffers(m_device, m_commandPool, 1, &replacement);
return result;
}
frame.retiredCommandBuffers.push_back({frame.preCommandBuffer, frame.lastSubmitIndex});
frame.preCommandBuffer = preReplacement;
}
// lastSubmitIndex was just written by the renderer for the submission
// that carried this command buffer.
frame.retiredCommandBuffers.push_back({frame.commandBuffer, frame.lastSubmitIndex});
@@ -29,7 +29,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkPipelineStageFlags waitDstStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
VkSemaphore waitSemaphore = VK_NULL_HANDLE;
VkSemaphore signalSemaphore = VK_NULL_HANDLE;
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
// [0] = pre-pass command buffer (when recorded), then the frame
// command buffer; submitInfo.pCommandBuffers points here.
VkCommandBuffer commandBuffers[2] = {VK_NULL_HANDLE, VK_NULL_HANDLE};
VkSubmitInfo submitInfo{VK_STRUCTURE_TYPE_SUBMIT_INFO};
};
@@ -52,10 +54,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct FrameData {
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
// Pre-pass work stream: out-of-pass commands (deferred clear
// materialization, sampled-layout transitions) for resources the
// frame's recording has not touched yet. Submitted immediately
// BEFORE commandBuffer in the same vkQueueSubmit, so recording
// into it never has to split the frame's active render pass.
VkCommandBuffer preCommandBuffer = VK_NULL_HANDLE;
VkSemaphore imageAvailableSemaphore = VK_NULL_HANDLE;
VkFence imageInFlightFence = VK_NULL_HANDLE;
Bool isCommandRecording = false;
Bool hasCommandBufferRecorded = false;
Bool isPreCommandRecording = false;
Bool hasPreCommandBufferRecorded = false;
Bool imageAvailableSemaphoreConsumed = false;
// Command buffers submitted mid-frame (FlushPendingCommands),
// appended in submit order; freed once their submission is known
@@ -77,6 +87,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer& BeginCommandRecording(VkCommandBufferUsageFlags flags = 0,
const VkCommandBufferInheritanceInfo* pInheritanceInfo = nullptr);
void EndCommandRecording();
// Lazily opens the pre-pass work stream (see FrameData::preCommandBuffer).
VkCommandBuffer BeginPreCommandRecording();
// Closes the pre stream if open, marking it for submission ahead of the
// frame command buffer. Safe to call when it never opened.
void EndPreCommandRecordingIfOpen();
// Drops an in-progress or recorded-but-unsubmitted pre stream (dropped
// frame recordings, swapchain recreation).
void AbandonPreCommandRecording();
VkResult InitializeSwapchainSemaphores(VkDevice device, Uint32 swapchainImageCount);
void DestroySwapchainSemaphores(VkDevice device);
Bool TransitionToPresent(VkImage image, VkImageLayout oldLayout,
@@ -91,7 +109,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// can restart while the submitted buffer is still executing. Retired
// buffers are freed after the slot's fence is next waited, or as soon
// as their submission is observed complete.
VkResult RetireCurrentCommandBuffer();
VkResult RetireCurrentCommandBuffer(Bool retirePreCommandBuffer = false);
// Frees every retired command buffer whose tagged submission index is
// known complete. Driven by the renderer's submit tracker on completion
@@ -262,6 +262,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_images.resize(imageCount, VK_NULL_HANDLE);
VK_VERIFY(vkGetSwapchainImagesKHR(device, m_swapchain, &imageCount, m_images.data()));
m_imageLayouts.assign(imageCount, VK_IMAGE_LAYOUT_UNDEFINED);
// Fresh swapchain images hold garbage until a render pass stores into them.
m_imageContentDefined.assign(imageCount, false);
m_depthStencilContentDefined.assign(imageCount, false);
CreateImageViews(device);
CreateDepthStencilResources(device, physicalDevice);
@@ -433,9 +436,39 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_images.clear();
m_imageLayouts.clear();
m_imageContentDefined.clear();
m_depthStencilContentDefined.clear();
m_preTransform = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR;
}
Bool SwapchainObject::IsImageContentDefined(Uint32 index) const {
MOBILEGL_ASSERT(index < m_imageContentDefined.size(), "Swapchain image content index out of range");
return m_imageContentDefined[index];
}
void SwapchainObject::SetImageContentDefined(Uint32 index, Bool defined) {
MOBILEGL_ASSERT(index < m_imageContentDefined.size(), "Swapchain image content index out of range");
m_imageContentDefined[index] = defined;
}
Bool SwapchainObject::IsDepthStencilContentDefined(Uint32 index) const {
MOBILEGL_ASSERT(index < m_depthStencilContentDefined.size(),
"Swapchain depth/stencil content index out of range");
return m_depthStencilContentDefined[index];
}
void SwapchainObject::SetDepthStencilContentDefined(Uint32 index, Bool defined) {
MOBILEGL_ASSERT(index < m_depthStencilContentDefined.size(),
"Swapchain depth/stencil content index out of range");
m_depthStencilContentDefined[index] = defined;
}
void SwapchainObject::SetAllDepthStencilContentUndefined() {
for (SizeT i = 0; i < m_depthStencilContentDefined.size(); ++i) {
m_depthStencilContentDefined[i] = false;
}
}
VkImage SwapchainObject::GetImage(Uint32 index) const {
MOBILEGL_ASSERT(index < m_images.size(), "Swapchain image index out of range");
return m_images[index];
@@ -52,6 +52,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void SetImageLayout(Uint32 index, VkImageLayout layout);
SizeT GetImageCount() const { return m_images.size(); }
// EGL content-validity tracking for the default framebuffer. A color
// buffer's content is undefined once its image has been presented
// (EGL_BUFFER_DESTROYED swap behaviour, the implementation default),
// and every ancillary (depth/stencil) buffer's content is undefined
// after ANY swap regardless of swap behaviour (EGL 1.5 §3.10.1). The
// render-pass manager turns an undefined attachment's tile load into
// LOAD_OP_DONT_CARE. Flags start false (a fresh swapchain image holds
// garbage) and a render pass storing into an attachment sets it back
// to defined.
Bool IsImageContentDefined(Uint32 index) const;
void SetImageContentDefined(Uint32 index, Bool defined);
Bool IsDepthStencilContentDefined(Uint32 index) const;
void SetDepthStencilContentDefined(Uint32 index, Bool defined);
void SetAllDepthStencilContentUndefined();
private:
void CreateImageViews(VkDevice device);
void CreateDepthStencilResources(VkDevice device, VkPhysicalDevice physicalDevice);
@@ -77,5 +92,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<VkDeviceMemory> m_depthStencilImageMemories;
Vector<VkImageView> m_depthStencilImageViews;
Vector<VkImageLayout> m_depthStencilImageLayouts;
Vector<Bool> m_imageContentDefined;
Vector<Bool> m_depthStencilContentDefined;
};
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -18,6 +18,7 @@
#include "MG_Util/Converters/MGToVk/TextureEnumConverter.h"
#include "MG_Util/Metrics/TextureMetrics.h"
#include <Config.h>
#include <algorithm>
#include <cstdio>
#include <cstdlib>
#include <cstring>
@@ -204,6 +205,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// The frame's descriptor sets are recycled above, so last frame's reuse target
// is gone: start the per-draw descriptor-reuse cache fresh this frame.
m_hasLastDescriptor = false;
m_lastBindValid = false;
// Re-fingerprint the bound sampler set fresh this frame so any GL object address
// reuse cannot outlive a single frame (see SamplerResolveMemo).
for (auto& memo : m_samplerResolveMemo) {
@@ -1189,6 +1191,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
bufferInfo.range = ubo.range;
dynOffset = static_cast<Uint32>(ubo.dynamicOffset);
} else {
// Global-UBO slice reuse (see GlobalUboSliceMemo): unchanged
// uniform bytes re-use the slice already uploaded this frame.
const Bool isGlobalUbo =
programObj.globalUboBinding == static_cast<Int>(binding) && element == 0;
const Uint64 uboFrameSerial = m_bufferManager->GetFrameSerial();
const Uint64 uboProgramLifetimeId = program.GetLifetimeId();
const Uint32 uboContentVersion = program.GetUBOContentVersion();
Bool reusedSlice = false;
if (isGlobalUbo) {
for (const auto& memo : m_globalUboMemo) {
if (memo.buffer != VK_NULL_HANDLE &&
memo.programLifetimeId == uboProgramLifetimeId &&
memo.frameSerial == uboFrameSerial &&
memo.uboContentVersion == uboContentVersion &&
memo.range == static_cast<VkDeviceSize>(ubo.payloadSize)) {
bufferInfo.buffer = memo.buffer;
bufferInfo.range = memo.range;
dynOffset = static_cast<Uint32>(memo.offset);
reusedSlice = true;
break;
}
}
}
if (!reusedSlice) {
BufferSlice slice{};
if (!m_bufferManager->UploadTransient(BufferKind::Uniform, frameIndex, ubo.payload,
ubo.payloadSize, m_minDynamicOffsetAlignment, slice)) {
@@ -1199,6 +1225,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
bufferInfo.buffer = slice.buffer;
bufferInfo.range = ubo.payloadSize;
dynOffset = static_cast<Uint32>(slice.offset);
if (isGlobalUbo) {
m_globalUboMemo[m_globalUboMemoNext] = GlobalUboSliceMemo{
uboProgramLifetimeId, uboFrameSerial, uboContentVersion,
slice.buffer, slice.offset, static_cast<VkDeviceSize>(ubo.payloadSize)};
m_globalUboMemoNext = (m_globalUboMemoNext + 1) % kGlobalUboMemoSize;
}
}
}
bufferInfos.push_back(bufferInfo);
// Dynamic offsets are consumed in binding order, then array element order,
@@ -1337,8 +1370,34 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_hasLastDescriptor = cacheable;
}
// Skip the driver call when this exact binding is already live on the
// command buffer (see the bind-dedup shadow in the header).
const Uint32 offsetCount = static_cast<Uint32>(dynamicOffsets.size());
Bool identicalBind = m_lastBindValid && m_lastBindSet == descriptorSet &&
m_lastBindLayout == programObj.pipelineLayout && m_lastBindPoint == bindPoint &&
m_lastBindOffsetCount == offsetCount && offsetCount <= kMaxShadowedDynamicOffsets;
if (identicalBind) {
for (Uint32 i = 0; i < offsetCount; ++i) {
if (m_lastBindOffsets[i] != dynamicOffsets[i]) {
identicalBind = false;
break;
}
}
}
if (!identicalBind) {
vkCmdBindDescriptorSets(commandBuffer, bindPoint, programObj.pipelineLayout, 0, 1,
&descriptorSet, static_cast<Uint32>(dynamicOffsets.size()), dynamicOffsets.data());
&descriptorSet, offsetCount, dynamicOffsets.data());
if (offsetCount <= kMaxShadowedDynamicOffsets) {
m_lastBindValid = true;
m_lastBindSet = descriptorSet;
m_lastBindLayout = programObj.pipelineLayout;
m_lastBindPoint = bindPoint;
m_lastBindOffsetCount = offsetCount;
std::copy_n(dynamicOffsets.data(), offsetCount, m_lastBindOffsets);
} else {
m_lastBindValid = false;
}
}
return true;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -39,6 +39,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void Shutdown();
void BeginFrame(Uint32 frameIndex);
// A command buffer (re)began recording: descriptor bindings recorded into
// the previous buffer do not carry over, so drop the bind-dedup shadow.
void OnCommandBufferBoundary() { m_lastBindValid = false; }
// A ProgramFactory eviction just destroyed this layout: purge every frame
// slot's cached descriptor sets for it, so a recycled handle value can never
// stale-hit sets written for the dead layout's bindings. The sets are
@@ -185,6 +188,35 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 m_lastDescriptorSignature = 0;
Bool m_hasLastDescriptor = false;
// vkCmdBindDescriptorSets dedup: consecutive draws with a static uniform
// block resolve to the same set AND the same dynamic offsets, so the
// driver call can be skipped outright. Command-buffer-scope state; reset
// via OnCommandBufferBoundary whenever a recording (re)begins. Keyed on
// layout+bind point, so a pipeline-layout switch always rebinds.
static constexpr Uint32 kMaxShadowedDynamicOffsets = 8;
Bool m_lastBindValid = false;
VkDescriptorSet m_lastBindSet = VK_NULL_HANDLE;
VkPipelineLayout m_lastBindLayout = VK_NULL_HANDLE;
VkPipelineBindPoint m_lastBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
Uint32 m_lastBindOffsetCount = 0;
Uint32 m_lastBindOffsets[kMaxShadowedDynamicOffsets] = {};
// Global-UBO transient-slice reuse: MC leaves the default uniform block
// untouched across long GUI/terrain runs, so the per-draw re-upload of
// the same bytes can reuse the slice uploaded earlier THIS frame (frame
// serial guards arena recycling; the content version guards writes).
struct GlobalUboSliceMemo {
Uint64 programLifetimeId = 0;
Uint64 frameSerial = 0;
Uint32 uboContentVersion = 0;
VkBuffer buffer = VK_NULL_HANDLE;
VkDeviceSize offset = 0;
VkDeviceSize range = 0;
};
static constexpr Uint32 kGlobalUboMemoSize = 4;
GlobalUboSliceMemo m_globalUboMemo[kGlobalUboMemoSize];
Uint32 m_globalUboMemoNext = 0;
// Per-binding fast path over VkSamplerManager's content-hashed sampler cache, which
// stays the source of truth: its key hashes all sampler+texture state, so two distinct
// sampler objects with identical state still resolve to one VkSampler. This memo only
@@ -58,15 +58,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VertexInputStateFactory::BackendVertexInputState& VertexInputStateFactory::GetOrCreateVertexInputState(
const MG_State::GLState::VertexArrayObject& vao) {
return GetOrCreateVertexInputState(vao, GetOrComputeHash(vao));
// Per-draw fast path: the VAO carries a pointer to its resolved entry,
// valid while its config version and the cache's eviction epoch both
// match - no re-hash, no map lookup.
const void* memoState = nullptr;
Uint64 memoEpoch = 0;
if (vao.GetBackendStateMemo(memoState, memoEpoch) && memoEpoch == m_evictionEpoch) {
const auto* entry = static_cast<const BackendVertexInputState*>(memoState);
entry->lastUsedFrameBoundary = m_frameBoundaryCounter;
return *entry;
}
const BackendVertexInputState& entry = GetOrCreateVertexInputState(vao, GetOrComputeHash(vao));
vao.SetBackendStateMemo(&entry, m_evictionEpoch);
return entry;
}
const VertexInputStateFactory::BackendVertexInputState& VertexInputStateFactory::GetOrCreateVertexInputState(
const MG_State::GLState::VertexArrayObject& vao, HashType hash) {
auto it = m_cache.find(hash);
if (it != m_cache.end()) {
it->second.lastUsedFrameBoundary = m_frameBoundaryCounter;
return it->second;
it->second->lastUsedFrameBoundary = m_frameBoundaryCounter;
return *it->second;
}
VertexInputStateBuilder builder;
@@ -172,11 +184,37 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto& state = builder.Build();
auto& entry = m_cache[hash];
auto& slot = m_cache[hash];
if (!slot) {
slot = MakeUnique<BackendVertexInputState>();
}
BackendVertexInputState& entry = *slot;
entry.hash = hash;
entry.lastUsedFrameBoundary = m_frameBoundaryCounter;
entry.bindings = builder.GetBindings();
entry.attributes = builder.GetAttributes();
// See the layoutHash declaration: hash only the resolved layout, never
// buffer identities, so identical layouts across VAOs/buffers agree.
XXHASH_VERIFY(XXH64_reset(m_hashState, 0));
for (const auto& binding : entry.bindings) {
XXHASH_VERIFY(XXH64_update(m_hashState, &binding.binding, sizeof(binding.binding)));
XXHASH_VERIFY(XXH64_update(m_hashState, &binding.stride, sizeof(binding.stride)));
XXHASH_VERIFY(XXH64_update(m_hashState, &binding.inputRate, sizeof(binding.inputRate)));
}
for (const auto& attribute : entry.attributes) {
XXHASH_VERIFY(XXH64_update(m_hashState, &attribute.location, sizeof(attribute.location)));
XXHASH_VERIFY(XXH64_update(m_hashState, &attribute.binding, sizeof(attribute.binding)));
XXHASH_VERIFY(XXH64_update(m_hashState, &attribute.format, sizeof(attribute.format)));
XXHASH_VERIFY(XXH64_update(m_hashState, &attribute.offset, sizeof(attribute.offset)));
}
XXHASH_VERIFY(XXH64_update(m_hashState, &unsupportedAttribMask, sizeof(unsupportedAttribMask)));
entry.layoutHash = XXH64_digest(m_hashState);
entry.attributeLocationMask = 0;
for (const auto& attribute : entry.attributes) {
if (attribute.location < 32u) {
entry.attributeLocationMask |= (1u << attribute.location);
}
}
entry.bindingBufferKeys = std::move(bindingBufferKeys);
entry.bindingBaseOffsets = std::move(bindingBaseOffsets);
entry.bindingAttributeLocations = std::move(bindingAttributeLocations);
@@ -205,8 +243,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (m_frameBoundaryCounter - it->second.lastUsedFrameBoundary > kRetireAgeBoundaries) {
if (m_frameBoundaryCounter - it->second->lastUsedFrameBoundary > kRetireAgeBoundaries) {
it = m_cache.erase(it);
// Invalidate every VAO's state-pointer memo: the erased node's
// address may be reused by a future insert.
++m_evictionEpoch;
} else {
++it;
}
@@ -27,9 +27,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct BackendVertexInputState {
HashType hash = 0;
// Hash of the resolved Vulkan vertex layout only (bindings, attributes,
// unsupported mask) - NO buffer identities. `hash` mixes buffer heap
// addresses so per-chunk VBOs mint a fresh identity per buffer; keying
// pipelines on that minted one VkPipeline per chunk section for an
// identical layout, defeating pipeline reuse and the per-draw memo.
// Pipelines depend only on the layout, so they key on this instead.
HashType layoutHash = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age are evicted (CPU heap only).
Uint64 lastUsedFrameBoundary = 0;
// Mutable: the VAO's state-pointer memo fast path stamps it through
// a const entry reference.
mutable Uint64 lastUsedFrameBoundary = 0;
Vector<VkVertexInputBindingDescription> bindings;
Vector<VkVertexInputAttributeDescription> attributes;
Vector<SizeT> bindingBufferKeys;
@@ -41,6 +50,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// absent from `attributes`, so without this mask the draw path cannot tell them apart from
// a genuinely disabled array and would silently feed the shader the current attribute value.
Uint32 unsupportedAttribMask = 0;
// Bitmask of `attributes[i].location` - the draw path needs it up to
// three times per draw, so it is baked once at build time.
Uint32 attributeLocationMask = 0;
VkPipelineVertexInputStateCreateInfo state{
VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO
};
@@ -80,9 +92,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VulkanRendererConfig& m_config;
VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE;
UnorderedMap<HashType, BackendVertexInputState> m_cache;
// Values are heap-allocated: FastSTL::unordered_map is open-addressing,
// so INSERT invalidates references to stored values. The draw path (and
// the VAOs' state-pointer memos) hold entry pointers across inserts;
// only the unique_ptr cell moves, never the pointee.
UnorderedMap<HashType, UniquePtr<BackendVertexInputState>> m_cache;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameBoundaryCounter = 0;
// Bumped whenever any cache entry is erased. VAOs memo a raw pointer to
// their heap-allocated entry (stable across map insert/rehash by
// construction); a memo is honored only while its recorded epoch
// matches, so an evicted entry can never be dereferenced through a
// stale memo.
Uint64 m_evictionEpoch = 1;
static inline XXH64_state_t* m_hashState = XXH64_createState();
};
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -93,6 +93,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const std::lock_guard<std::mutex> lock(m_mutex);
m_pendingClears.clear();
m_aliveObjects.clear();
m_pendingCount.store(static_cast<Uint32>(m_pendingClears.size()), std::memory_order_relaxed);
}
TextureIdentity VkClearManager::MakeTextureIdentity(MG_State::GLState::ITextureObject* texture) {
@@ -127,6 +128,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_pendingClears.erase(key);
}
m_aliveObjects.erase(identity);
m_pendingCount.store(static_cast<Uint32>(m_pendingClears.size()), std::memory_order_relaxed);
}
Bool VkClearManager::LockTextureIdentityLocked(const TextureIdentity& identity,
@@ -221,6 +223,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_aliveObjects[MakeTextureIdentity(texture.get())] = texture;
auto& pending = m_pendingClears[key];
MergeClearPayload(pending, clearPayload);
m_pendingCount.store(static_cast<Uint32>(m_pendingClears.size()), std::memory_order_relaxed);
}
void VkClearManager::QueueClear(const ClearAttachmentPayload& clearPayload,
@@ -238,6 +241,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_aliveObjects[MakeTextureIdentity(texture.get())] = texture;
auto& pending = m_pendingClears[key];
MergeClearPayload(pending, clearPayload);
m_pendingCount.store(static_cast<Uint32>(m_pendingClears.size()), std::memory_order_relaxed);
}
Bool VkClearManager::HasPendingClear(MG_State::GLState::ITextureObject* texture) {
@@ -245,6 +249,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return false;
}
if (m_pendingCount.load(std::memory_order_relaxed) == 0) {
return false; // per-draw hot path: nothing pending anywhere
}
const Uint64 lifetimeId = texture->GetLifetimeId();
const std::lock_guard<std::mutex> lock(m_mutex);
for (auto it = m_pendingClears.begin(); it != m_pendingClears.end(); ++it) {
@@ -260,6 +268,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (key.texture == nullptr) {
return false;
}
if (m_pendingCount.load(std::memory_order_relaxed) == 0) {
return false; // per-draw hot path: nothing pending anywhere
}
const std::lock_guard<std::mutex> lock(m_mutex);
if (m_pendingClears.find(key) == m_pendingClears.end()) {
@@ -287,6 +298,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (key.texture == nullptr) {
return false;
}
if (m_pendingCount.load(std::memory_order_relaxed) == 0) {
return false; // per-draw hot path: nothing pending anywhere
}
const std::lock_guard<std::mutex> lock(m_mutex);
if (!LockTextureLocked(key, outTexture)) {
@@ -325,6 +339,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (texture == nullptr) {
return false;
}
if (m_pendingCount.load(std::memory_order_relaxed) == 0) {
return false; // per-draw hot path: nothing pending anywhere
}
const Uint64 lifetimeId = texture->GetLifetimeId();
const std::lock_guard<std::mutex> lock(m_mutex);
@@ -345,6 +362,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return;
}
if (m_pendingCount.load(std::memory_order_relaxed) == 0) {
return; // per-draw hot path: nothing pending anywhere
}
const TextureIdentity identity = MakeTextureIdentity(texture);
MGLOG_D("%s: Pop all pending clears for texture %d", __func__, texture->GetExternalIndex());
const std::lock_guard<std::mutex> lock(m_mutex);
@@ -361,6 +381,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto it = m_pendingClears.find(key);
if (it != m_pendingClears.end()) {
m_pendingClears.erase(it);
m_pendingCount.store(static_cast<Uint32>(m_pendingClears.size()), std::memory_order_relaxed);
}
}
@@ -14,6 +14,7 @@
#include "MG_Util/Math/VectorTypes.h"
#include <Includes.h>
#include <atomic>
#include <unordered_map>
namespace MobileGL::MG_Backend::DirectVulkan {
@@ -120,7 +121,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
SharedPtr<MG_State::GLState::ITextureObject>& outTexture);
Uint8 m_gcCounter = 0;
public:
// Lock-free probe for the consecutive-draw fast path: any pending clear
// forces the full SetupDraw path (which materializes/consumes it).
Bool HasAnyPendingClears() const { return m_pendingCount.load(std::memory_order_relaxed) != 0; }
private:
mutable std::mutex m_mutex;
// Lock-free mirror of m_pendingClears.size(), maintained under m_mutex
// by every mutation. The per-draw probes (HasPendingClear/GetPending*)
// read it before taking the lock: during draw batches the pending set
// is almost always empty, so this turns several locked map probes per
// draw into one relaxed load.
std::atomic<Uint32> m_pendingCount{0};
std::unordered_map<PendingClearKey, ClearAttachmentPayload, PendingClearKeyHash> m_pendingClears;
std::unordered_map<TextureIdentity, WeakPtr<MG_State::GLState::ITextureObject>, TextureIdentityHash> m_aliveObjects;
};
@@ -481,7 +481,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
VkRenderPassManager::HashType VkRenderPassManager::ComputeHash(
const MG_State::GLState::FramebufferObject& fbo, Uint32 swapchainImageIndex, Bool includePendingClear) {
const MG_State::GLState::FramebufferObject& fbo, Uint32 swapchainImageIndex, Bool includePendingClear,
Bool includeDefaultFboDepthStencil) {
XXHASH_VERIFY(XXH64_reset(m_hashState, m_config.CacheVersion));
const Bool isDefaultFbo = fbo.IsDefaultFramebuffer();
if (isDefaultFbo) {
@@ -560,9 +561,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
attachment <= FramebufferAttachmentType::BackRight);
if (isDefaultColorAttachment) {
currentLayout = m_swapchainObject.GetImageLayout(swapchainImageIndex);
// Content validity feeds the attachment's loadOp (see the
// creation path), so it must key the cache as well.
if (!m_swapchainObject.IsImageContentDefined(swapchainImageIndex)) {
currentLayout = VK_IMAGE_LAYOUT_UNDEFINED;
}
} else if (attachment == FramebufferAttachmentType::Depth ||
attachment == FramebufferAttachmentType::Stencil) {
currentLayout = m_swapchainObject.GetDepthStencilImageLayout(swapchainImageIndex);
if (!m_swapchainObject.IsDepthStencilContentDefined(swapchainImageIndex)) {
currentLayout = VK_IMAGE_LAYOUT_UNDEFINED;
}
}
} else {
auto* textureResource = m_textureManager.SyncTextureAndGetDescriptor(*texture);
@@ -617,14 +626,49 @@ namespace MobileGL::MG_Backend::DirectVulkan {
combineFramebufferAttachmentObjHash(drawbuf);
}
// The depth-less default-FBO flavor omits the depth/stencil attachment
// entirely, so it must hash differently from the depth-full flavor.
const Bool depthStencilIncluded = !isDefaultFbo || includeDefaultFboDepthStencil;
XXHASH_VERIFY(XXH64_update(m_hashState, &depthStencilIncluded, sizeof(depthStencilIncluded)));
if (depthStencilIncluded) {
combineFramebufferAttachmentObjHash(FramebufferAttachmentType::Depth);
combineFramebufferAttachmentObjHash(FramebufferAttachmentType::Stencil);
}
return XXH64_digest(m_hashState);
}
RenderPassEntry& VkRenderPassManager::GetOrCreateRenderPass(const MG_State::GLState::FramebufferObject& fbo,
Uint32 swapchainImageIndex) {
Uint32 swapchainImageIndex,
Bool drawUsesDepthStencil) {
// Resolve the default-FBO depth flavor (see the header comment): keep the
// depth attachment when the caller needs it, when a depth/stencil clear is
// pending, or when the active pass already carries it (escalate-only, so
// alternating depth-less draws never split an established depth pass).
Bool includeDefaultFboDepthStencil = true;
if (fbo.IsDefaultFramebuffer()) {
Bool activeDefaultHasDepthStencil = false;
if (const auto* active = GetActiveRenderPass()) {
Bool activeIsSwapchainPass = false;
Bool activeHasSwapchainDepthStencil = false;
for (const auto& tracked : active->trackedAttachmentLayouts) {
activeIsSwapchainPass |= tracked.target == TrackedAttachmentTarget::SwapchainColor;
activeHasSwapchainDepthStencil |=
tracked.target == TrackedAttachmentTarget::SwapchainDepthStencil;
}
activeDefaultHasDepthStencil = activeIsSwapchainPass && activeHasSwapchainDepthStencil;
}
const auto& defaultDepthAtt = fbo.GetAttachment(FramebufferAttachmentType::Depth);
const auto& defaultStencilAtt = fbo.GetAttachment(FramebufferAttachmentType::Stencil);
const Bool pendingDepthStencilClear =
(defaultDepthAtt.IsTexture() && m_clearManager.HasPendingClear(defaultDepthAtt)) ||
HasPendingRenderbufferClear(defaultDepthAtt) ||
(defaultStencilAtt.IsTexture() && m_clearManager.HasPendingClear(defaultStencilAtt)) ||
HasPendingRenderbufferClear(defaultStencilAtt);
includeDefaultFboDepthStencil =
drawUsesDepthStencil || activeDefaultHasDepthStencil || pendingDepthStencilClear;
}
auto hasPendingClearOnFramebuffer = [&]() -> Bool {
const auto& drawBuffers = fbo.GetDrawBuffers();
for (auto attachment : drawBuffers) {
@@ -674,6 +718,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_rpFastFboVersion == fbo.GetObjectVersion() && m_rpFastSwapchainIndex == swapchainImageIndex &&
m_rpFastTexEpoch == m_textureManager.GetTextureImageEpoch() &&
m_rpFastRbEpoch == m_renderbufferImageEpoch &&
(!fbo.IsDefaultFramebuffer() || m_rpFastHadDepthStencil == includeDefaultFboDepthStencil) &&
m_rpFastRenderPassHash == activeRenderPass->hash && !hasPendingClearOnFramebuffer()) {
auto activeIt = m_renderPasses.find(activeRenderPass->hash);
if (activeIt != m_renderPasses.end()) {
@@ -682,7 +727,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
auto compatibilityHash = ComputeHash(fbo, swapchainImageIndex, false);
auto compatibilityHash = ComputeHash(fbo, swapchainImageIndex, false, includeDefaultFboDepthStencil);
if (activeRenderPass != nullptr &&
activeRenderPass->CompatibleWith(compatibilityHash) &&
!hasPendingClearOnFramebuffer()) {
@@ -699,10 +744,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_rpFastTexEpoch = m_textureManager.GetTextureImageEpoch();
m_rpFastRbEpoch = m_renderbufferImageEpoch;
m_rpFastRenderPassHash = activeRenderPass->hash;
m_rpFastHadDepthStencil = activeIt->second.hasDepthStencilAttachment;
activeIt->second.lastUsedFrame = m_frameCounter;
return activeIt->second;
}
auto hash = ComputeHash(fbo, swapchainImageIndex, true);
auto hash = ComputeHash(fbo, swapchainImageIndex, true, includeDefaultFboDepthStencil);
auto it = m_renderPasses.find(hash);
if (it != m_renderPasses.end()) {
it->second.lastUsedFrame = m_frameCounter;
@@ -894,6 +940,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(swapchainImageIndex < swapchainViews.size(),
"GetOrCreateRenderPass: swapchain image index out of range");
trackedColorLayout = m_swapchainObject.GetImageLayout(swapchainImageIndex);
// EGL: a presented color buffer's content is undefined when its
// image comes back around (EGL_BUFFER_DESTROYED, the default
// swap behaviour) - skip the tile load instead of reloading
// stale pixels nobody may rely on.
if (!hasClear && !m_swapchainObject.IsImageContentDefined(swapchainImageIndex)) {
trackedColorLayout = VK_IMAGE_LAYOUT_UNDEFINED;
}
trackedAttachmentLayouts.emplace_back(TrackedAttachmentLayoutInfo {
.target = TrackedAttachmentTarget::SwapchainColor,
.swapchainImageIndex = swapchainImageIndex,
@@ -913,6 +966,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
trackedAttachmentLayouts.emplace_back(TrackedAttachmentLayoutInfo {
.target = TrackedAttachmentTarget::Texture,
.texture = att.GetTexture(),
.textureRaw = att.GetTexture().get(),
.textureMipLevel = attachmentMipLevel,
.finalLayout = desc.finalLayout,
});
@@ -976,6 +1030,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
};
const auto* selectedDepthStencilAttachment = isUsableDepthStencilAttachment(depthAtt) ? &depthAtt :
(isUsableDepthStencilAttachment(stencilAtt) ? &stencilAtt : nullptr);
// Depth-less default-FBO flavor: nothing in this pass touches depth/stencil
// and their content is undefined anyway (EGL swap), so drop the attachment
// and its whole tile load + store.
if (isDefaultFbo && !includeDefaultFboDepthStencil) {
selectedDepthStencilAttachment = nullptr;
}
const Bool hasDistinctDepthAndStencilAttachments =
isUsableDepthStencilAttachment(depthAtt) && isUsableDepthStencilAttachment(stencilAtt) &&
!sameDepthStencilAttachmentObject(depthAtt, stencilAtt);
@@ -994,6 +1054,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkImageLayout trackedDepthLayout = isDefaultFbo ?
m_swapchainObject.GetDepthStencilImageLayout(swapchainImageIndex) :
VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL;
// EGL 1.5 §3.10.1: every ancillary (depth/stencil) buffer's content is
// undefined after a swap, so the first default-FBO pass of a frame can
// skip the depth/stencil tile load outright.
if (isDefaultFbo && !m_swapchainObject.IsDepthStencilContentDefined(swapchainImageIndex)) {
trackedDepthLayout = VK_IMAGE_LAYOUT_UNDEFINED;
}
depthAttachmentDescription.flags = 0;
VkSampleCountFlagBits depthAttachmentSampleCount = VK_SAMPLE_COUNT_1_BIT;
Int depthAttachmentId = 0;
@@ -1077,6 +1143,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
trackedAttachmentLayouts.emplace_back(TrackedAttachmentLayoutInfo {
.target = TrackedAttachmentTarget::Texture,
.texture = selectedDepthStencilAttachment->GetTexture(),
.textureRaw = selectedDepthStencilAttachment->GetTexture().get(),
.textureMipLevel = attachmentMipLevel,
.finalLayout = depthAttachmentDescription.finalLayout,
});
@@ -1122,6 +1189,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const Bool hasDepthStencilAttachment = depthAttachmentRef.attachment != VK_ATTACHMENT_UNUSED;
// Declare only the used colour-reference span. The GL draw-buffer array
// always spans 8 slots, so passes used to declare colorAttachmentCount=8
// with trailing VK_ATTACHMENT_UNUSED holes - and Adreno configures its
// per-pixel render-backend/export path from the DECLARED count, so every
// fragment of every pass paid the 8-target export cost (measured on
// Adreno 650 / MC 26.2: 11.9 -> 7.5 ms of GPU time per frame, with the
// single-quad swapchain blit pass alone dropping 1.26 -> 0.40 ms).
// Interior GL_NONE holes keep their slots so fragment-output locations
// still line up; a fragment output at a location past the trimmed count
// is discarded, which is exactly GL's semantic for writing to a draw
// buffer set to GL_NONE.
while (!colorAttachmentRefs.empty() &&
colorAttachmentRefs.back().attachment == VK_ATTACHMENT_UNUSED) {
colorAttachmentRefs.pop_back();
}
// Subpass
VkSubpassDescription subpassDesc;
subpassDesc.flags = 0;
@@ -1330,6 +1413,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
renderPassBeginInfo.pClearValues = clearValues.data();
vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);
// Pre-pass stream bookkeeping: this pass's attachment images are now
// referenced by the open frame recording.
if (s_textureManager != nullptr) {
for (const auto& tracked : renderPassEntry.trackedAttachmentLayouts) {
if (tracked.target == TrackedAttachmentTarget::Texture) {
if (const auto texture = tracked.texture.lock()) {
s_textureManager->StampTextureRecordingUse(texture.get());
}
}
}
}
for (const auto& pending: renderPassEntry.pendingClearAttachments) {
if (pending.hasInlinePayload) {
if (s_renderPassManager != nullptr) {
@@ -1382,11 +1476,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
case TrackedAttachmentTarget::SwapchainColor:
MOBILEGL_ASSERT(s_swapchainObject != nullptr, "EndRenderPass: swapchain object is null");
s_swapchainObject->SetImageLayout(trackedAttachment.swapchainImageIndex, trackedAttachment.finalLayout);
// The pass stored into the attachment: its content is defined
// until the image is next presented.
s_swapchainObject->SetImageContentDefined(trackedAttachment.swapchainImageIndex, true);
break;
case TrackedAttachmentTarget::SwapchainDepthStencil:
MOBILEGL_ASSERT(s_swapchainObject != nullptr, "EndRenderPass: swapchain object is null");
s_swapchainObject->SetDepthStencilImageLayout(trackedAttachment.swapchainImageIndex,
trackedAttachment.finalLayout);
s_swapchainObject->SetDepthStencilContentDefined(trackedAttachment.swapchainImageIndex, true);
break;
default:
MOBILEGL_ASSERT(false, "EndRenderPass: unsupported tracked attachment target=%d",
@@ -42,6 +42,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct TrackedAttachmentLayoutInfo {
TrackedAttachmentTarget target = TrackedAttachmentTarget::Texture;
WeakPtr<MG_State::GLState::ITextureObject> texture;
// Identity-compare shortcut for the per-draw "does the active pass use
// this sampled texture" probe: comparing this against a LIVE texture's
// address needs no weak_ptr::lock (two refcount atomics per probe).
// May dangle once the texture dies - compare only, never dereference.
MG_State::GLState::ITextureObject* textureRaw = nullptr;
WeakPtr<MG_State::GLState::RenderbufferObject> renderbuffer;
Uint32 textureMipLevel = 0;
Uint32 swapchainImageIndex = 0;
@@ -188,8 +193,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
HashType ComputeHash(
const MG_State::GLState::FramebufferObject& fbo,
Uint32 swapchainImageIndex,
Bool includePendingClear = true);
RenderPassEntry& GetOrCreateRenderPass(const MG_State::GLState::FramebufferObject& fbo, Uint32 swapchainImageIndex);
Bool includePendingClear = true,
Bool includeDefaultFboDepthStencil = true);
// drawUsesDepthStencil: whether the operation about to run inside the pass
// reads or writes the depth/stencil buffer (depth test or stencil test
// enabled, or a depth/stencil clear). Only consulted for the DEFAULT
// framebuffer: EGL undefines its ancillary buffers at every swap, so a
// default-FBO pass whose draws provably never touch depth/stencil is
// created WITHOUT the depth attachment - on a tiler that skips the whole
// depth tile load AND store. The flavor only escalates: once a pass with
// depth is active, later depth-less draws keep using it, and a depth-using
// draw against a depth-less active pass resolves to a new (incompatible)
// entry, which the caller's compatibility check turns into a pass split;
// the new pass's depth loads DONT_CARE (content was undefined all along).
RenderPassEntry& GetOrCreateRenderPass(const MG_State::GLState::FramebufferObject& fbo,
Uint32 swapchainImageIndex,
Bool drawUsesDepthStencil = true);
void QueueRenderbufferClear(GLbitfield mask, const ClearFramebufferPayload& clearPayload,
const MG_State::GLState::FramebufferObject& drawFbo);
void QueueRenderbufferClear(const ClearAttachmentPayload& clearPayload,
@@ -219,6 +238,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// image recreation.
Uint64 m_renderbufferImageEpoch = 1;
public:
// Bumped whenever a renderbuffer backing is (re)created; consecutive-draw
// snapshots include it so an attachment respecify forces a re-resolve.
Uint64 GetRenderbufferImageEpoch() const { return m_renderbufferImageEpoch; }
private:
// Per-draw fast-path memo for GetOrCreateRenderPass (dirty-flag state tracking): when the
// framebuffer state is provably unchanged since the last resolution, the active render pass
// is reused WITHOUT recomputing the expensive per-draw hash. Invalidated by FBO switch /
@@ -231,6 +257,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 m_rpFastTexEpoch = 0;
Uint64 m_rpFastRbEpoch = 0;
Uint64 m_rpFastRenderPassHash = 0;
// Whether the memoized entry carries a depth/stencil attachment; a
// default-FBO resolution whose effective depth request differs must
// miss the memo (the depth-less/depth-full flavors hash differently).
Bool m_rpFastHadDepthStencil = false;
public:
struct RenderbufferResource {
@@ -607,7 +607,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
void VkTextureManager::Shutdown() {
if (m_device != VK_NULL_HANDLE) {
ReclaimCompletedUploads(/*waitAll=*/true);
}
DestroyDeferredReleases();
++m_resourceEraseEpoch; // every memoized resource pointer dies with the map
m_textureResources.clear();
m_aliveObjects.clear();
m_storageImageTextures.clear();
@@ -629,6 +633,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frameIndex, m_deferredViewReleases.size());
m_currentFrameIndex = frameIndex;
CollectDeferredReleases(frameIndex);
ReclaimCompletedUploads();
// Frame-boundary GC: every 64 frame boundaries (~1 s at 60 fps) bounds the reclaim
// latency for dead textures regardless of draw traffic — workloads that churn
@@ -659,6 +664,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
m_aliveObjects.erase(identity);
m_storageImageTextures.erase(identity);
// Invalidate every cross-draw sampled-texture memo: the erased
// resource's address may be reused by a future emplace.
++m_resourceEraseEpoch;
}
void VkTextureManager::PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture) {
@@ -714,6 +722,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
// Cross-draw memo probe (see SyncedTextureMemoEntry): skips both map
// lookups and the (re)registration path for repeat-bound textures.
TextureResource* resourcePtr = nullptr;
for (Uint32 i = 0; i < kSyncedTextureMemoSize; ++i) {
const SyncedTextureMemoEntry& memo = m_syncedTextureMemo[i];
if (memo.texture == &texture && memo.lifetimeId == identity.lifetimeId &&
memo.eraseEpoch == m_resourceEraseEpoch) {
resourcePtr = memo.resource;
break;
}
}
if (resourcePtr == nullptr) {
auto aliveIt = m_aliveObjects.find(identity);
if (aliveIt != m_aliveObjects.end() && aliveIt->second.expired()) {
EraseTrackedTexture(aliveIt->first);
@@ -751,8 +772,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto [insertIt, _] = m_textureResources.emplace(identity, Move(initial));
it = insertIt;
}
resourcePtr = &(it->second);
m_syncedTextureMemo[m_syncedTextureMemoNext] =
SyncedTextureMemoEntry{&texture, identity.lifetimeId, m_resourceEraseEpoch, resourcePtr};
m_syncedTextureMemoNext = (m_syncedTextureMemoNext + 1) % kSyncedTextureMemoSize;
}
if (!SyncTexture(texture, it->second)) {
if (!SyncTexture(texture, *resourcePtr)) {
MGLOG_D("%s: Syncing texture %d failed", __func__, texture.GetExternalIndex());
return nullptr;
}
@@ -766,11 +792,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
if (!recorded) {
m_drawSyncedThisDraw.push_back({identity, &(it->second)});
m_drawSyncedThisDraw.push_back({identity, resourcePtr});
}
}
return &(it->second);
return resourcePtr;
}
VkImageView VkTextureManager::GetOrCreateViewAtMipLevel(MG_State::GLState::ITextureObject& texture, Uint32 mipLevel) {
@@ -1049,6 +1075,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return view;
}
void VkTextureManager::StampTextureRecordingUse(MG_State::GLState::ITextureObject* texture) {
if (texture == nullptr) {
return;
}
auto it = m_textureResources.find(MakeTextureIdentity(texture));
if (it != m_textureResources.end()) {
it->second.lastRecordingGeneration = m_recordingGeneration;
}
}
void VkTextureManager::UpdateTrackedImageLayout(MG_State::GLState::ITextureObject* texture, VkImageLayout newLayout) {
MOBILEGL_ASSERT(texture != nullptr, "UpdateTrackedImageLayout: texture is null");
auto it = m_textureResources.find(MakeTextureIdentity(texture));
@@ -1076,6 +1112,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(writtenMipLevel < resource.mipLevels,
"UpdateTrackedImageLayoutAfterAttachmentWrite: textureId=%d mipLevel=%u out of range %u",
texture->GetExternalIndex(), writtenMipLevel, resource.mipLevels);
// Pre-pass stream bookkeeping: the render pass that just ended wrote this image.
StampResourceRecordingUse(resource);
if (resource.layout != newLayout && resource.mipLevels > 1) {
VkPipelineStageFlags srcStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
@@ -1160,6 +1198,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VK_ACCESS_SHADER_READ_BIT, resource->aspect, 0, resource->mipLevels,
resource->arrayLayers);
MOBILEGL_ASSERT(ok, "TransitionTextureForSampling: transition failed for textureId=%d", texture.GetExternalIndex());
// Pre-pass stream bookkeeping: a command referencing the image was recorded.
StampResourceRecordingUse(*resource);
return ok;
}
@@ -1189,6 +1229,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource->aspect, 0, resource->mipLevels, resource->arrayLayers);
MOBILEGL_ASSERT(ok, "TransitionTextureForStorageImage: transition failed for textureId=%d",
texture.GetExternalIndex());
// Pre-pass stream bookkeeping: a command referencing the image was recorded.
StampResourceRecordingUse(*resource);
return ok;
}
@@ -1420,8 +1462,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return false;
}
const Bool isMultisampleTexture = IsMultisampleTextureUploadTarget(uploadTarget);
// A texture that has only ever defined level 0 gets a single-level backing
// (ANGLE's model). Preallocating the full chain put every render target
// onto Adreno's multi-mip image layout and grew each texture by a third
// for levels most textures never define. Once a second level is defined
// the backing is recreated ONE time with the full chain (the
// preserve-copy path below carries the pixels over), so sequentially-
// defined atlas mips do not recreate per level, and glGenerateMipmap -
// which defines every level before syncing - works unchanged.
const Uint32 backingMipLevels =
isMultisampleTexture ? 1u : std::max(mipLevels, ComputeFullMipLevelCount(texelSize));
isMultisampleTexture ? 1u
: (mipLevels > 1 ? std::max(mipLevels, ComputeFullMipLevelCount(texelSize)) : 1u);
TextureShapeInfo shapeInfo{};
const Bool supportedShape = TryResolveTextureShapeInfo(texture, uploadTarget, texelSize, shapeInfo);
MOBILEGL_ASSERT(supportedShape,
@@ -1683,6 +1734,28 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_deferredViewReleases[frameIndex].clear();
}
void VkTextureManager::ReclaimCompletedUploads(Bool waitAll) {
if (m_pendingUploadReclaims.empty()) {
return;
}
SizeT completed = 0;
for (; completed < m_pendingUploadReclaims.size(); ++completed) {
PendingUploadReclaim& entry = m_pendingUploadReclaims[completed];
if (waitAll) {
VK_VERIFY(vkWaitForFences(m_device, 1, &entry.fence, VK_TRUE, UINT64_MAX),
"vkWaitForFences(texture upload reclaim)");
} else if (vkGetFenceStatus(m_device, entry.fence) != VK_SUCCESS) {
break;
}
vkDestroyFence(m_device, entry.fence, nullptr);
vkFreeCommandBuffers(m_device, m_commandPool, 1, &entry.commandBuffer);
vmaDestroyBuffer(m_allocator, entry.stagingBuffer, entry.stagingAllocation);
}
m_pendingUploadReclaims.erase(m_pendingUploadReclaims.begin(),
m_pendingUploadReclaims.begin() + static_cast<std::ptrdiff_t>(completed));
}
void VkTextureManager::DestroyDeferredReleases() {
for (auto& deferredReleases : m_deferredReleases) {
deferredReleases.clear();
@@ -1989,11 +2062,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VK_VERIFY(vkCreateFence(m_device, &fenceInfo, nullptr, &uploadFence), "vkCreateFence(texture upload)");
VK_VERIFY(vkQueueSubmit(m_graphicsQueue, 1, &submitInfo, uploadFence), "vkQueueSubmit(texture)");
VK_VERIFY(vkWaitForFences(m_device, 1, &uploadFence, VK_TRUE, UINT64_MAX), "vkWaitForFences(texture upload)");
vkDestroyFence(m_device, uploadFence, nullptr);
vkFreeCommandBuffers(m_device, m_commandPool, 1, &commandBuffer);
vmaDestroyBuffer(m_allocator, stagingBuffer, stagingAllocation);
// Do NOT wait the fence here: this submit sits behind the previous
// frame's rendering on the queue, so a synchronous wait stalls the CPU
// until the GPU drains - a per-frame vkQueueWaitIdle for any workload
// with animated textures. Ordering against the current frame's draws is
// already guaranteed (its command buffer is submitted later, at
// present), so only the transient objects need to survive execution;
// park them until the fence signals.
m_pendingUploadReclaims.push_back({uploadFence, commandBuffer, stagingBuffer, stagingAllocation});
ReclaimCompletedUploads();
// Backstop for pathological upload storms: bound in-flight staging
// memory by blocking on the oldest upload only once the list is deep.
constexpr SizeT kMaxPendingTextureUploads = 16;
if (m_pendingUploadReclaims.size() > kMaxPendingTextureUploads) {
VK_VERIFY(vkWaitForFences(m_device, 1, &m_pendingUploadReclaims.front().fence, VK_TRUE, UINT64_MAX),
"vkWaitForFences(texture upload backstop)");
ReclaimCompletedUploads();
}
if (!ok) {
MGLOG_D("%s: texture upload cmd failed", __func__);
@@ -28,6 +28,9 @@ public:
// manager keys its per-draw fast path on this so an attachment's image recreation
// invalidates the cached render pass (dirty-flag tracking; portable to Vulkan 1.1).
Uint64 GetTextureImageEpoch() const { return m_textureImageEpoch; }
// Bumped whenever any tracked texture resource is erased; cached
// TextureResource pointers are valid only while this is unchanged.
Uint64 GetResourceEraseEpoch() const { return m_resourceEraseEpoch; }
struct TextureIdentity {
MG_State::GLState::ITextureObject* texture = nullptr;
@@ -172,6 +175,13 @@ public:
// NeedsStorageImagePreparation cannot ask for a recreate that will never happen.
Bool storageUsageResolved = false;
Uint16 syncedTextureParamsVersion = 0;
// Recording generation (VkTextureManager::GetRecordingGeneration) of the last
// command referencing this image that was recorded into the CURRENT frame
// command buffer. An image untouched by the open recording may have its
// out-of-pass work (deferred clears, sampled-layout transitions) recorded
// into the frame's PRE command buffer - which executes strictly before the
// frame's commands - instead of splitting the active render pass.
Uint64 lastRecordingGeneration = 0;
// Snapshot of ITextureObject::GetContentVersion() at the last successful sync;
// lets SyncTexture skip the whole re-check/re-upload when content is unchanged.
Uint64 syncedContentVersion = 0;
@@ -207,6 +217,7 @@ public:
std::swap(this->usageFlags, that.usageFlags);
std::swap(this->storageUsageResolved, that.storageUsageResolved);
std::swap(this->syncedTextureParamsVersion, that.syncedTextureParamsVersion);
std::swap(this->lastRecordingGeneration, that.lastRecordingGeneration);
std::swap(this->syncedContentVersion, that.syncedContentVersion);
std::swap(this->syncedMipLevelCount, that.syncedMipLevelCount);
}
@@ -307,6 +318,21 @@ public:
VkImageLayout newLayout);
Bool TransitionTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
Bool TransitionTextureForStorageImage(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
// Recording-generation bookkeeping for the pre-pass command stream. The
// generation advances every time the frame command buffer (re)begins
// recording; a resource whose stamp does not match was not referenced by
// any command in the open recording, so its out-of-pass work may safely
// execute ahead of the whole recording (in the pre command buffer).
void AdvanceRecordingGeneration() { ++m_recordingGeneration; }
void StampResourceRecordingUse(TextureResource& resource) const {
resource.lastRecordingGeneration = m_recordingGeneration;
}
// Map-lookup variant for callers that only hold the GL texture object.
void StampTextureRecordingUse(MG_State::GLState::ITextureObject* texture);
Bool WasTouchedThisRecording(const TextureResource& resource) const {
return resource.lastRecordingGeneration == m_recordingGeneration;
}
// Records that this texture is bound to a GL image unit, so its image must carry
// VK_IMAGE_USAGE_STORAGE_BIT. Must be called before NeedsStorageImagePreparation, and
// therefore before the render pass is committed: an image that has to be upgraded is
@@ -364,6 +390,9 @@ public:
private:
// Bumped in SyncTextureResource right after vmaCreateImage(texture). See GetTextureImageEpoch().
Uint64 m_textureImageEpoch = 1;
// See AdvanceRecordingGeneration. Starts above every resource's default
// stamp of 0 so a fresh resource counts as untouched.
Uint64 m_recordingGeneration = 1;
Bool SyncTexture(MG_State::GLState::ITextureObject &texture,
TextureResource &outResource);
@@ -394,6 +423,11 @@ private:
void DeferViewRelease(VkImageView view);
void CollectDeferredReleases(Uint32 frameIndex);
void DestroyDeferredReleases();
// Frees the fence/command buffer/staging buffer of every in-flight texture
// upload whose fence has signaled (submission order = completion order on
// the single queue, so the scan stops at the first still-pending entry).
// waitAll blocks on every entry - Shutdown's drain.
void ReclaimCompletedUploads(Bool waitAll = false);
static TextureIdentity MakeTextureIdentity(MG_State::GLState::ITextureObject* texture);
void EraseTrackedTexture(const TextureIdentity& identity);
void PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture);
@@ -423,6 +457,23 @@ private:
TextureResource* resource = nullptr;
};
Vector<DrawSyncedTexture> m_drawSyncedThisDraw;
// Cross-draw sampled-texture memo: the same few textures (atlas, lightmap)
// are resolved on every draw, so cache their resource pointers and skip the
// alive/resource map lookups. Node-based std::unordered_map keeps the
// pointees stable across inserts; erases bump m_resourceEraseEpoch, which
// every memo entry must match. SyncTexture still runs on memo hits, so
// content/param freshness is unaffected. A dead-then-reused texture address
// cannot false-hit: the new object carries a new lifetime id.
struct SyncedTextureMemoEntry {
const MG_State::GLState::ITextureObject* texture = nullptr;
Uint64 lifetimeId = 0;
Uint64 eraseEpoch = 0;
TextureResource* resource = nullptr;
};
static constexpr Uint32 kSyncedTextureMemoSize = 8;
SyncedTextureMemoEntry m_syncedTextureMemo[kSyncedTextureMemoSize];
Uint32 m_syncedTextureMemoNext = 0;
Uint64 m_resourceEraseEpoch = 1;
// Formats whose mutable-image probe failed on this device; their images are created
// without MUTABLE_FORMAT_BIT so repeat syncs neither re-probe nor flag-mismatch.
std::unordered_set<VkFormat> m_mutableFormatUnsupported;
@@ -432,5 +483,16 @@ private:
std::unordered_set<TextureIdentity, TextureIdentityHash> m_storageImageTextures;
Vector<Vector<TextureResource>> m_deferredReleases;
Vector<Vector<VkImageView>> m_deferredViewReleases;
// Texture uploads are submitted out-of-band but NOT waited on (waiting
// behind the queue serialized the CPU against the previous frame's GPU
// work every time an animated atlas re-uploaded). Their transient objects
// are parked here and reclaimed once the upload fence signals.
struct PendingUploadReclaim {
VkFence fence = VK_NULL_HANDLE;
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
VkBuffer stagingBuffer = VK_NULL_HANDLE;
VmaAllocation stagingAllocation = nullptr;
};
Vector<PendingUploadReclaim> m_pendingUploadReclaims;
};
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -217,6 +217,73 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return static_cast<Int>((static_cast<Int64>(value) * toExtent + fromExtent / 2) / fromExtent);
}
// Redundant dynamic-state elimination for the per-draw hot path: within one
// command-buffer recording, a vkCmdSet* whose values already match what the
// command buffer holds is skipped. Valid because every PipelineFactory
// pipeline declares the same eight dynamic states, so the values persist
// across those pipeline binds; the shadow resets whenever a recording
// (re)begins, and whenever an auxiliary pipeline with a narrower dynamic
// set (blit, depth-mipmap) binds - their static state makes the
// corresponding dynamic values undefined per the spec.
struct DynamicStateShadow {
// Last graphics pipeline bound on the frame command buffer. Pipeline
// binds are command-buffer state (they survive render-pass boundaries),
// so the same reset points that invalidate dynamic state - recording
// (re)begin and the aux blit pipelines' raw binds - are exactly the
// points where this becomes unknown.
Bool graphicsPipelineValid = false;
VkPipeline graphicsPipeline = VK_NULL_HANDLE;
// Index/vertex buffer binds are command-buffer state too. Terrain
// sections and GUI quads share one sequential index buffer, and GUI
// batches often reuse a vertex arena buffer, so skipping identical
// rebinds removes a large share of per-draw driver calls.
Bool indexBindValid = false;
VkBuffer indexBuffer = VK_NULL_HANDLE;
VkDeviceSize indexOffset = 0;
VkIndexType indexType = VK_INDEX_TYPE_MAX_ENUM;
static constexpr Uint32 kMaxShadowedVertexBindings = 8;
Bool vertexBindValid = false;
Uint32 vertexBindingCount = 0;
VkBuffer vertexBuffers[kMaxShadowedVertexBindings] = {};
VkDeviceSize vertexOffsets[kMaxShadowedVertexBindings] = {};
Bool viewportValid = false;
VkViewport viewport{};
Bool scissorValid = false;
VkRect2D scissor{};
Bool blendConstantsValid = false;
Float blendConstants[4] = {0.0f, 0.0f, 0.0f, 0.0f};
Bool depthBiasValid = false;
Float depthBiasConstantFactor = 0.0f;
Float depthBiasSlopeFactor = 0.0f;
Bool lineWidthValid = false;
Float lineWidth = 0.0f;
Bool stencilValid = false;
Uint32 stencilFrontCompareMask = 0;
Uint32 stencilBackCompareMask = 0;
Uint32 stencilFrontWriteMask = 0;
Uint32 stencilBackWriteMask = 0;
Uint32 stencilFrontReference = 0;
Uint32 stencilBackReference = 0;
};
static DynamicStateShadow g_dynamicStateShadow;
static void ResetDynamicStateShadow() {
g_dynamicStateShadow = {};
}
static void ShadowedSetScissor(VkCommandBuffer commandBuffer, const VkRect2D& scissor) {
auto& shadow = g_dynamicStateShadow;
if (shadow.scissorValid && shadow.scissor.offset.x == scissor.offset.x &&
shadow.scissor.offset.y == scissor.offset.y &&
shadow.scissor.extent.width == scissor.extent.width &&
shadow.scissor.extent.height == scissor.extent.height) {
return;
}
shadow.scissorValid = true;
shadow.scissor = scissor;
vkCmdSetScissor(commandBuffer, 0, 1, &scissor);
}
static void ApplyGLViewportState(VkCommandBuffer commandBuffer,
const IntVec2& framebufferExtent,
VkSurfaceTransformFlagBitsKHR preTransform,
@@ -246,6 +313,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
viewport.height = static_cast<float>(viewportHeight);
viewport.minDepth = depthRange.x();
viewport.maxDepth = depthRange.y();
auto& shadow = g_dynamicStateShadow;
if (shadow.viewportValid && shadow.viewport.x == viewport.x && shadow.viewport.y == viewport.y &&
shadow.viewport.width == viewport.width && shadow.viewport.height == viewport.height &&
shadow.viewport.minDepth == viewport.minDepth && shadow.viewport.maxDepth == viewport.maxDepth) {
return;
}
shadow.viewportValid = true;
shadow.viewport = viewport;
vkCmdSetViewport(commandBuffer, 0, 1, &viewport);
}
@@ -257,6 +332,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
blendColor.z(),
blendColor.w(),
};
auto& shadow = g_dynamicStateShadow;
if (shadow.blendConstantsValid && shadow.blendConstants[0] == blendConstants[0] &&
shadow.blendConstants[1] == blendConstants[1] && shadow.blendConstants[2] == blendConstants[2] &&
shadow.blendConstants[3] == blendConstants[3]) {
return;
}
shadow.blendConstantsValid = true;
shadow.blendConstants[0] = blendConstants[0];
shadow.blendConstants[1] = blendConstants[1];
shadow.blendConstants[2] = blendConstants[2];
shadow.blendConstants[3] = blendConstants[3];
vkCmdSetBlendConstants(commandBuffer, blendConstants);
}
@@ -272,8 +358,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
static void ApplyPolygonOffsetState(VkCommandBuffer commandBuffer) {
vkCmdSetDepthBias(commandBuffer, MG_State::pGLContext->GetPolygonOffsetUnits(), 0.0f,
MG_State::pGLContext->GetPolygonOffsetFactor());
const Float constantFactor = MG_State::pGLContext->GetPolygonOffsetUnits();
const Float slopeFactor = MG_State::pGLContext->GetPolygonOffsetFactor();
auto& shadow = g_dynamicStateShadow;
if (shadow.depthBiasValid && shadow.depthBiasConstantFactor == constantFactor &&
shadow.depthBiasSlopeFactor == slopeFactor) {
return;
}
shadow.depthBiasValid = true;
shadow.depthBiasConstantFactor = constantFactor;
shadow.depthBiasSlopeFactor = slopeFactor;
vkCmdSetDepthBias(commandBuffer, constantFactor, 0.0f, slopeFactor);
}
static void ApplyLineWidthState(VkCommandBuffer commandBuffer) {
@@ -288,6 +383,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
lineWidth = maxLineWidth;
}
}
auto& shadow = g_dynamicStateShadow;
if (shadow.lineWidthValid && shadow.lineWidth == lineWidth) {
return;
}
shadow.lineWidthValid = true;
shadow.lineWidth = lineWidth;
vkCmdSetLineWidth(commandBuffer, lineWidth);
}
@@ -336,15 +437,31 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static void ApplyStencilState(VkCommandBuffer commandBuffer) {
const StencilFaceState& frontStencil = MG_State::pGLContext->GetStencilState(StencilFace::Front);
const StencilFaceState& backStencil = MG_State::pGLContext->GetStencilState(StencilFace::Back);
const Uint32 frontReference = static_cast<Uint32>(std::max(frontStencil.Ref, 0));
const Uint32 backReference = static_cast<Uint32>(std::max(backStencil.Ref, 0));
auto& shadow = g_dynamicStateShadow;
if (shadow.stencilValid && shadow.stencilFrontCompareMask == frontStencil.ValueMask &&
shadow.stencilBackCompareMask == backStencil.ValueMask &&
shadow.stencilFrontWriteMask == frontStencil.WriteMask &&
shadow.stencilBackWriteMask == backStencil.WriteMask &&
shadow.stencilFrontReference == frontReference && shadow.stencilBackReference == backReference) {
return;
}
shadow.stencilValid = true;
shadow.stencilFrontCompareMask = frontStencil.ValueMask;
shadow.stencilBackCompareMask = backStencil.ValueMask;
shadow.stencilFrontWriteMask = frontStencil.WriteMask;
shadow.stencilBackWriteMask = backStencil.WriteMask;
shadow.stencilFrontReference = frontReference;
shadow.stencilBackReference = backReference;
vkCmdSetStencilCompareMask(commandBuffer, VK_STENCIL_FACE_FRONT_BIT, frontStencil.ValueMask);
vkCmdSetStencilCompareMask(commandBuffer, VK_STENCIL_FACE_BACK_BIT, backStencil.ValueMask);
vkCmdSetStencilWriteMask(commandBuffer, VK_STENCIL_FACE_FRONT_BIT, frontStencil.WriteMask);
vkCmdSetStencilWriteMask(commandBuffer, VK_STENCIL_FACE_BACK_BIT, backStencil.WriteMask);
vkCmdSetStencilReference(commandBuffer, VK_STENCIL_FACE_FRONT_BIT,
static_cast<Uint32>(std::max(frontStencil.Ref, 0)));
vkCmdSetStencilReference(commandBuffer, VK_STENCIL_FACE_BACK_BIT,
static_cast<Uint32>(std::max(backStencil.Ref, 0)));
vkCmdSetStencilReference(commandBuffer, VK_STENCIL_FACE_FRONT_BIT, frontReference);
vkCmdSetStencilReference(commandBuffer, VK_STENCIL_FACE_BACK_BIT, backReference);
}
enum class NumericDomain {
@@ -718,16 +835,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static_assert(kMaxVertexAttribs <= ProgramFactory::VkProgramObject::kMaxVertexInputLocations,
"vertexInputTypes is indexed by vertex attribute location");
static Uint32 BuildVertexInputAttributeMask(const Vector<VkVertexInputAttributeDescription>& attributes) {
Uint32 attributeMask = 0;
for (const auto& attribute : attributes) {
if (attribute.location < kMaxVertexAttribs) {
attributeMask |= (1u << attribute.location);
}
}
return attributeMask;
}
static Bool TryGetCurrentVertexAttributeFormat(GLenum glType, VkFormat& outFormat) {
switch (glType) {
case GL_FLOAT:
@@ -878,8 +985,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (trackedAttachment.target != TrackedAttachmentTarget::Texture) {
continue;
}
const auto trackedTexture = trackedAttachment.texture.lock();
if (trackedTexture && trackedTexture.get() == &texture) {
// Raw identity compare (see textureRaw): the caller's texture is
// live, so a dangling tracked pointer can never equal its address
// unless the allocator reused it - and that false positive merely
// ends the render pass early, never misses a genuine use.
if (trackedAttachment.textureRaw == &texture) {
return true;
}
}
@@ -2814,7 +2924,7 @@ void main() {
// the GetCurrentProgram + GetOrCreateProgram hash lookup every draw.
auto& vertexInputState = m_vertexInputStateFactory->GetOrCreateVertexInputState(vao);
const Uint32 activeAttribMask = programObj.activeVertexInputLocationMask;
const Uint32 vertexInputAttribMask = BuildVertexInputAttributeMask(vertexInputState.attributes);
const Uint32 vertexInputAttribMask = vertexInputState.attributeLocationMask;
const Uint32 missingAttribMask = activeAttribMask & ~vertexInputAttribMask;
const auto bindingCount = vertexInputState.bindings.size() + static_cast<SizeT>(std::popcount(missingAttribMask));
@@ -3079,8 +3189,29 @@ void main() {
}
if (bindingCount > 0) {
vkCmdBindVertexBuffers(commandBuffer, 0, static_cast<Uint32>(bindingCount), vkBuffers.data(),
vkOffsets.data());
auto& shadow = g_dynamicStateShadow;
const Uint32 count = static_cast<Uint32>(bindingCount);
Bool identical = shadow.vertexBindValid && shadow.vertexBindingCount == count &&
count <= DynamicStateShadow::kMaxShadowedVertexBindings;
if (identical) {
for (Uint32 i = 0; i < count; ++i) {
if (shadow.vertexBuffers[i] != vkBuffers[i] || shadow.vertexOffsets[i] != vkOffsets[i]) {
identical = false;
break;
}
}
}
if (!identical) {
vkCmdBindVertexBuffers(commandBuffer, 0, count, vkBuffers.data(), vkOffsets.data());
if (count <= DynamicStateShadow::kMaxShadowedVertexBindings) {
shadow.vertexBindValid = true;
shadow.vertexBindingCount = count;
std::copy_n(vkBuffers.data(), count, shadow.vertexBuffers);
std::copy_n(vkOffsets.data(), count, shadow.vertexOffsets);
} else {
shadow.vertexBindValid = false;
}
}
}
return true;
}
@@ -3150,8 +3281,17 @@ void main() {
MGLOG_E("DrawElements skipped: failed to sync resident index buffer");
return false;
}
vkCmdBindIndexBuffer(frame.commandBuffer, slice.buffer,
slice.offset + static_cast<VkDeviceSize>(pIndexBufferView->indexByteOffset), vkIndexType);
const VkDeviceSize indexBindOffset =
slice.offset + static_cast<VkDeviceSize>(pIndexBufferView->indexByteOffset);
auto& shadow = g_dynamicStateShadow;
if (!shadow.indexBindValid || shadow.indexBuffer != slice.buffer ||
shadow.indexOffset != indexBindOffset || shadow.indexType != vkIndexType) {
vkCmdBindIndexBuffer(frame.commandBuffer, slice.buffer, indexBindOffset, vkIndexType);
shadow.indexBindValid = true;
shadow.indexBuffer = slice.buffer;
shadow.indexOffset = indexBindOffset;
shadow.indexType = vkIndexType;
}
return true;
}
@@ -3653,6 +3793,10 @@ void main() {
vkCmdSetScissor(frame.commandBuffer, 0, 1, &scissor);
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
// The depth-mipmap pipeline's narrower dynamic set (viewport/scissor
// only) leaves the other dynamic states undefined; its raw scissor
// and viewport writes also bypass the shadow.
ResetDynamicStateShadow();
std::fill(depthProgramData,
depthProgramData + m_depthMipmapResources.program->GetUBOSize(),
@@ -3714,16 +3858,24 @@ void main() {
// content hash (folds program identity + link version + transform flags + shader stages),
// vertex-input hash (VAO layout), render-pass hash (render targets + the draw-buffer/format
// driven blend & write-mask gating), and the render-state version (all fixed-function state).
// Reset per-frame and on pipeline destruction so m_lastPipelineResult can never dangle.
const Uint64 vertexInputHash = m_vertexInputStateFactory->GetOrComputeHash(vao);
// Reset per-frame and on pipeline destruction so a memoized handle can never dangle.
// The identity hash mixes buffer heap addresses (per-chunk VBOs mint a new
// one per buffer); the memo and the pipeline payload key on the resolved
// LAYOUT hash instead, so draws over identical layouts share one pipeline.
// The one-arg fetch rides the VAO's state-pointer memo (no hash, no map).
auto& vis = m_vertexInputStateFactory->GetOrCreateVertexInputState(vao);
const Uint64 vertexLayoutHash = vis.layoutHash;
const Uint64 renderPassHash = renderPassEntry.hash;
const Uint renderStateVersion = MG_State::pGLContext->GetRenderStateParametersVersion();
if (m_lastPipelineValid && m_lastPipelineResult != VK_NULL_HANDLE && m_lastPipelineMode == mode &&
m_lastPipelineProgramHash == programObj.hash && m_lastPipelineVertexInputHash == vertexInputHash &&
m_lastPipelineRenderPassHash == renderPassHash &&
m_lastPipelineRenderStateVersion == renderStateVersion &&
m_lastPipelineTransformFlags == transformFlags) {
return m_lastPipelineResult;
for (Uint32 i = 0; i < m_pipelineMemoCount; ++i) {
const PipelineMemoEntry& entry = m_pipelineMemo[i];
if (entry.pipeline != VK_NULL_HANDLE && entry.mode == mode &&
entry.programHash == programObj.hash && entry.vertexInputHash == vertexLayoutHash &&
entry.renderPassHash == renderPassHash &&
entry.renderStateVersion == renderStateVersion &&
entry.transformFlags == transformFlags) {
return entry.pipeline;
}
}
#if MOBILEGL_LOG_ACTIVE_LEVEL <= MOBILEGL_LOG_LEVEL_DEBUG
@@ -3764,9 +3916,7 @@ void main() {
}
#endif
// vertexInputHash was computed above for the fast-path key; reuse it here.
auto& vis = m_vertexInputStateFactory->GetOrCreateVertexInputState(vao, vertexInputHash);
const Uint32 vertexInputAttribMask = BuildVertexInputAttributeMask(vis.attributes);
const Uint32 vertexInputAttribMask = vis.attributeLocationMask;
const Uint32 activeAttribMask = programObj.activeVertexInputLocationMask;
const Uint32 missingAttribMask = activeAttribMask & ~vertexInputAttribMask;
auto& patchedAttributes = m_patchedAttributesScratch;
@@ -3876,7 +4026,7 @@ void main() {
PipelineFactory::PipelineCreatePayload payload {
.programHash = programObj.hash,
.vertexInputHash = vertexInputHash,
.vertexInputHash = vertexLayoutHash,
.pipelineLayout = programObj.pipelineLayout,
.renderPass = renderPassEntry.renderPass,
.colorAttachmentCount = renderPassEntry.colorAttachmentCount,
@@ -3940,12 +4090,15 @@ void main() {
payload.backStencilCompareOp = VK_COMPARE_OP_ALWAYS;
}
const Uint32 fragmentOutputMask = programObj.activeFragmentOutputLocationMask;
MOBILEGL_ASSERT(
(fragmentOutputMask >> payload.colorAttachmentCount) == 0,
"GetOrCreatePipeline: fragmentOutputMask=0x%x exceeds colorAttachmentCount=%u for program=%u",
fragmentOutputMask,
payload.colorAttachmentCount,
program.GetExternalIndex());
// Outputs at locations past the render pass's trimmed colour span are
// simply discarded - GL's semantic for a fragment output whose draw
// buffer is GL_NONE (the trailing UNUSED slots no longer occupy
// references, see GetOrCreateRenderPass).
if ((fragmentOutputMask >> payload.colorAttachmentCount) != 0) {
MGLOG_D("GetOrCreatePipeline: fragmentOutputMask=0x%x exceeds colorAttachmentCount=%u for program=%u; "
"outputs past the span are discarded",
fragmentOutputMask, payload.colorAttachmentCount, program.GetExternalIndex());
}
MOBILEGL_ASSERT(payload.colorAttachmentCount <= PipelineFactory::PipelineCreatePayload::kMaxColorAttachments,
"GetOrCreatePipeline: colorAttachmentCount=%u exceeds payload capacity",
payload.colorAttachmentCount);
@@ -4180,14 +4333,16 @@ void main() {
}
VkPipeline pipeline = m_pipelineFactory->GetOrCreatePipeline(payload);
if (pipeline != VK_NULL_HANDLE) {
m_lastPipelineValid = true;
m_lastPipelineMode = mode;
m_lastPipelineProgramHash = programObj.hash;
m_lastPipelineVertexInputHash = vertexInputHash;
m_lastPipelineRenderPassHash = renderPassHash;
m_lastPipelineRenderStateVersion = renderStateVersion;
m_lastPipelineTransformFlags = transformFlags;
m_lastPipelineResult = pipeline;
PipelineMemoEntry& entry = m_pipelineMemo[m_pipelineMemoNext];
entry.mode = mode;
entry.programHash = programObj.hash;
entry.vertexInputHash = vertexLayoutHash;
entry.renderPassHash = renderPassHash;
entry.renderStateVersion = renderStateVersion;
entry.transformFlags = transformFlags;
entry.pipeline = pipeline;
m_pipelineMemoNext = (m_pipelineMemoNext + 1) % kPipelineMemoSize;
m_pipelineMemoCount = std::min(m_pipelineMemoCount + 1, kPipelineMemoSize);
}
return pipeline;
}
@@ -4289,6 +4444,129 @@ void main() {
return true;
}
Bool VulkanRenderer::TrySetupDrawFastPath(FrameContext::FrameData& frame, GLenum mode,
Flags<DrawSetupAspect> aspects, const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView) {
const SetupDrawSnapshot& snap = m_setupDrawSnapshot;
if (!snap.valid || !frame.isCommandRecording) {
return false;
}
if (snap.aspects != aspects.GetRaw() || snap.mode != mode) {
return false;
}
if (m_clearManager->HasAnyPendingClears()) {
return false;
}
const auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass();
if (activeRenderPass == nullptr || activeRenderPass->hash != snap.renderPassHash ||
snap.imageIndex != m_imageIndexAcquired) {
return false;
}
const auto& program = *MG_State::pGLContext->GetCurrentProgram();
if (program.GetLifetimeId() != snap.programLifetimeId ||
program.GetBackendStateVersion() != snap.programVersion) {
return false;
}
const auto& vao = *MG_State::pGLContext->GetBoundVertexArray();
if (static_cast<const void*>(&vao) != snap.vao || vao.GetConfigVersion() != snap.vaoConfigVersion) {
return false;
}
const auto& drawFbo =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Draw).GetBoundObject();
if (static_cast<const void*>(drawFbo.get()) != snap.drawFbo ||
drawFbo->GetObjectVersion() != snap.fboVersion) {
return false;
}
if (MG_State::pGLContext->GetRenderStateParametersVersion() != snap.renderStateVersion ||
MG_State::pGLContext->GetTextureBindGeneration() != snap.bindGeneration) {
return false;
}
if (GetShaderTransformFlags(m_swapchainObject.GetPreTransform()).GetRaw() != snap.baseTransformFlags) {
return false;
}
if (m_textureManager->GetResourceEraseEpoch() != snap.textureEraseEpoch ||
m_textureManager->GetTextureImageEpoch() != snap.textureImageEpoch ||
m_renderPassManager->GetRenderbufferImageEpoch() != snap.renderbufferImageEpoch) {
return false;
}
// Same sampled set as the snapshotting draw (program/bind keys above);
// verify content and params are untouched and every layout is still
// sampleable, then stamp recording use exactly as the full path would.
// A feedback case (sampled texture written by the active pass) fails the
// layout check and falls back to the full path's end-pass handling.
const auto& sampledTextures = m_sampledTexturesScratch;
const auto& sampledResources = m_sampledResourcesScratch;
if (sampledResources.size() != sampledTextures.size()) {
return false;
}
Uint64 contentSum = 0;
Uint64 paramsSum = 0;
for (SizeT i = 0; i < sampledTextures.size(); ++i) {
const auto* sampledTexture = sampledTextures[i];
if (sampledTexture == nullptr) {
continue;
}
const auto* resource = sampledResources[i];
if (resource == nullptr || !IsValidSampledImageLayout(resource->layout)) {
return false;
}
contentSum += sampledTexture->GetContentVersion();
paramsSum += sampledTexture->GetTextureParamsVersion();
}
if (contentSum != snap.sampledContentSum || paramsSum != snap.sampledParamsSum) {
return false;
}
for (SizeT i = 0; i < sampledTextures.size(); ++i) {
if (sampledTextures[i] != nullptr && sampledResources[i] != nullptr) {
m_textureManager->StampResourceRecordingUse(*sampledResources[i]);
}
}
// Everything the full path would re-resolve is provably unchanged; run
// only the per-draw tail.
if (!g_dynamicStateShadow.graphicsPipelineValid ||
g_dynamicStateShadow.graphicsPipeline != snap.pipeline) {
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, snap.pipeline);
g_dynamicStateShadow.graphicsPipelineValid = true;
g_dynamicStateShadow.graphicsPipeline = snap.pipeline;
}
const auto& programObj = m_programFactory->GetOrCreateProgram(
program, ProgramFactory::CompileOptionFlags(snap.resolvedTransformFlags));
if (!m_uniformManager->BindProgramUniformBuffers(frame.commandBuffer, program, programObj,
m_frameContext.GetCurrentFrameIndex())) {
return false;
}
if (!UploadAndBindVertexBuffers(frame.commandBuffer, vao, programObj, drawParams, pIndexBufferView)) {
return false;
}
if (aspects & DrawSetupAspect::IndexBuffer) {
const Bool idxUploadOk = UploadAndBindIndexBuffer(frame, vao, pIndexBufferView);
MOBILEGL_ASSERT(idxUploadOk, "SetupDraw fast path: failed to upload index buffer");
}
ApplyGLViewportState(frame.commandBuffer, snap.renderPassExtent, m_swapchainObject.GetPreTransform(),
snap.drawFboIsDefault);
ApplyBlendConstants(frame.commandBuffer);
ApplyPolygonOffsetState(frame.commandBuffer);
ApplyLineWidthState(frame.commandBuffer);
ApplyStencilState(frame.commandBuffer);
const Bool scissorEnabled = MG_State::pGLContext->IsCapabilityEnabled(CapabilityInput::ScissorTest);
VkRect2D scissor{};
if (scissorEnabled) {
const auto& scissorBox = MG_State::pGLContext->GetScissorBox();
scissor = snap.drawFboIsDefault
? MakeDefaultFramebufferScissorRect(scissorBox, snap.renderPassExtent,
m_swapchainObject.GetPreTransform())
: MakeClampedScissorRect(scissorBox, snap.renderPassExtent);
} else {
scissor.offset = {0, 0};
scissor.extent = { (Uint)snap.renderPassExtent.x(), (Uint)snap.renderPassExtent.y() };
}
ShadowedSetScissor(frame.commandBuffer, scissor);
return true;
}
Bool VulkanRenderer::SetupDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView) {
@@ -4297,6 +4575,12 @@ void main() {
// otherwise each re-run the full SyncTexture path on the same textures.
VkTextureManager::DrawSyncScope drawSyncScope(*m_textureManager);
m_textureManager->CollectGarbage();
if (TrySetupDrawFastPath(frame, mode, aspects, drawParams, pIndexBufferView)) {
return true;
}
// The fast path declined: whatever it saw may be stale. The full path
// below re-resolves everything and refreshes the snapshot on success.
m_setupDrawSnapshot.valid = false;
const auto& drawFbo =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Draw).GetBoundObject();
if (drawFbo != nullptr && IsUnsupportedFramebufferForDirectVulkan(*drawFbo)) {
@@ -4306,16 +4590,52 @@ void main() {
const auto& vao = *MG_State::pGLContext->GetBoundVertexArray();
const auto& program = *MG_State::pGLContext->GetCurrentProgram();
ProgramFactory::CompileOptionFlags transformFlags = GetShaderTransformFlags(m_swapchainObject.GetPreTransform());
const auto* programObjPtr = &m_programFactory->GetOrCreateProgram(program, transformFlags);
// Sampling a colour render target through the driver's implicit-LOD path faults the GPU on
// Adreno 650 (see ForceExplicitLod0SamplePass); ask for the explicit-LOD variant when doing
// so cannot change a texel, i.e. when every sampler this program reads is pinned to a
// single mip level.
if (UniformManager::ProgramSamplesOnlySingleLevelTextures(program, *programObjPtr)) {
transformFlags |= ProgramFactory::CompileOptionBit::ExplicitLod0Sampling;
programObjPtr = &m_programFactory->GetOrCreateProgram(program, transformFlags);
// single mip level. The probe walks every sampler binding, so its verdict is memoized
// under the sampled-set memo's key plus the sampled textures' params-version sum (level
// range and filter changes live there); the previous draw's texture list is valid for the
// sum exactly when that key matches (same program, same binds).
{
const Uint64 lodProgramLifetimeId = program.GetLifetimeId();
const Uint32 lodProgramVersion = program.GetBackendStateVersion();
const Uint64 lodBindGeneration = MG_State::pGLContext->GetTextureBindGeneration();
Bool lodMemoHit = false;
if (m_lastLodDecisionValid && m_lastSampledSetValid &&
m_lastLodProgramLifetimeId == lodProgramLifetimeId &&
m_lastLodProgramVersion == lodProgramVersion &&
m_lastLodBindGeneration == lodBindGeneration && m_lastLodBaseFlags == transformFlags &&
m_lastSampledSetProgramLifetimeId == lodProgramLifetimeId &&
m_lastSampledSetProgramVersion == lodProgramVersion &&
m_lastSampledSetBindGeneration == lodBindGeneration) {
Uint64 paramsSum = 0;
for (const auto* sampledTexture : m_sampledTexturesScratch) {
if (sampledTexture != nullptr) {
paramsSum += sampledTexture->GetTextureParamsVersion();
}
const auto& programObj = *programObjPtr;
}
if (paramsSum == m_lastLodParamsSum) {
transformFlags = m_lastLodResultFlags;
lodMemoHit = true;
}
}
if (!lodMemoHit) {
const ProgramFactory::CompileOptionFlags baseFlags = transformFlags;
const auto& baseProgramObj = m_programFactory->GetOrCreateProgram(program, transformFlags);
if (UniformManager::ProgramSamplesOnlySingleLevelTextures(program, baseProgramObj)) {
transformFlags |= ProgramFactory::CompileOptionBit::ExplicitLod0Sampling;
}
m_lastLodDecisionValid = true;
m_lastLodProgramLifetimeId = lodProgramLifetimeId;
m_lastLodProgramVersion = lodProgramVersion;
m_lastLodBindGeneration = lodBindGeneration;
m_lastLodBaseFlags = baseFlags;
m_lastLodResultFlags = transformFlags;
m_lastLodParamsSum = 0; // filled below once the sampled set is known
}
}
const auto& programObj = m_programFactory->GetOrCreateProgram(program, transformFlags);
// Begin command recording if not yet
if (!frame.isCommandRecording) {
@@ -4360,6 +4680,18 @@ void main() {
m_lastSampledSetTransformFlags = transformFlags;
m_lastSampledSetBindGeneration = bindGeneration;
}
// Complete a freshly-made LOD decision (see above): its params sum
// can only be taken once the sampled set is known. A genuine
// all-zero sum merely re-probes next draw.
if (m_lastLodDecisionValid && m_lastLodParamsSum == 0) {
Uint64 paramsSum = 0;
for (const auto* sampledTexture : sampledTextures) {
if (sampledTexture != nullptr) {
paramsSum += sampledTexture->GetTextureParamsVersion();
}
}
m_lastLodParamsSum = paramsSum;
}
}
MGLOG_D("SetupDraw: program=%u drawFbo=%u sampledTextureCount=%zu activeRenderPass=%s",
program.GetExternalIndex(), drawFbo ? drawFbo->GetExternalIndex() : 0u, sampledTextures.size(),
@@ -4383,7 +4715,10 @@ void main() {
activeRenderPass = nullptr;
}
Bool needSampledTextureTransitions = false;
for (auto* sampledTexture : sampledTextures) {
auto& sampledResources = m_sampledResourcesScratch;
sampledResources.assign(sampledTextures.size(), nullptr);
for (SizeT sampledIndex = 0; sampledIndex < sampledTextures.size(); ++sampledIndex) {
auto* sampledTexture = sampledTextures[sampledIndex];
if (!sampledTexture) {
continue;
}
@@ -4392,13 +4727,36 @@ void main() {
MOBILEGL_ASSERT(textureResource != nullptr,
"%s: SyncTextureAndGetDescriptor failed for textureId=%d",
__func__, sampledTexture->GetExternalIndex());
sampledResources[sampledIndex] = textureResource;
MGLOG_D("SetupDraw: sampled textureId=%d layout(before)=%s(%d)",
sampledTexture->GetExternalIndex(), VkImageLayoutToString(textureResource->layout),
static_cast<Int>(textureResource->layout));
if (m_clearManager->HasPendingClear(sampledTexture) ||
!IsValidSampledImageLayout(textureResource->layout)) {
// Out-of-pass work is needed (deferred clear materialization or
// a sampled-layout transition). When the open frame recording
// has not referenced this image yet, that work can execute
// ahead of the WHOLE recording - record it into the pre-pass
// stream instead of splitting the active render pass (ANGLE's
// outside-render-pass command stream, restricted to the
// provably reorderable case).
if (activeRenderPass != nullptr &&
!m_frameContext.GetCurrent().hasPreCommandBufferRecorded &&
!m_textureManager->WasTouchedThisRecording(*textureResource)) {
VkCommandBuffer preCommandBuffer = m_frameContext.BeginPreCommandRecording();
const Bool preClearReady =
MaterializePendingClearForTexture(preCommandBuffer, *sampledTexture);
MOBILEGL_ASSERT(preClearReady,
"%s: pre-pass MaterializePendingClearForTexture failed for textureId=%d",
__func__, sampledTexture->GetExternalIndex());
const Bool preTransitionReady =
m_textureManager->TransitionTextureForSampling(preCommandBuffer, *sampledTexture);
MOBILEGL_ASSERT(preTransitionReady,
"%s: pre-pass TransitionTextureForSampling failed for textureId=%d",
__func__, sampledTexture->GetExternalIndex());
continue;
}
needSampledTextureTransitions = true;
break;
}
}
@@ -4408,10 +4766,23 @@ void main() {
activeRenderPass = nullptr;
}
for (auto* sampledTexture : sampledTextures) {
for (SizeT sampledIndex = 0; sampledIndex < sampledTextures.size(); ++sampledIndex) {
auto* sampledTexture = sampledTextures[sampledIndex];
if (!sampledTexture) {
continue;
}
// Fast path: the first loop already resolved this texture, nothing
// is pending against it, and its layout is still sampleable (the
// layout re-check covers an EndRenderPass between the loops having
// rewritten an attachment's layout). Skipping the materialize +
// transition + re-resolve chain here is the difference between one
// pointer read and three calls per sampled texture per draw.
if (auto* fastResource = sampledResources[sampledIndex];
fastResource != nullptr && !m_clearManager->HasPendingClear(sampledTexture) &&
IsValidSampledImageLayout(fastResource->layout)) {
m_textureManager->StampResourceRecordingUse(*fastResource);
continue;
}
const Bool clearReady = MaterializePendingClearForTexture(frame.commandBuffer, *sampledTexture);
MOBILEGL_ASSERT(clearReady, "%s: MaterializePendingClearForTexture failed for textureId=%d",
__func__, sampledTexture->GetExternalIndex());
@@ -4422,16 +4793,28 @@ void main() {
MOBILEGL_ASSERT(transitionedResource != nullptr,
"%s: post-transition SyncTextureAndGetDescriptor failed for textureId=%d",
__func__, sampledTexture->GetExternalIndex());
// Pre-pass stream bookkeeping: the draw about to be recorded reads
// this image, so later out-of-pass work on it can no longer jump
// ahead of the recording.
m_textureManager->StampResourceRecordingUse(*transitionedResource);
MGLOG_D("SetupDraw: sampled textureId=%d layout(after)=%s(%d)",
sampledTexture->GetExternalIndex(), VkImageLayoutToString(transitionedResource->layout),
static_cast<Int>(transitionedResource->layout));
}
auto* renderPassEntry = &m_renderPassManager->GetOrCreateRenderPass(*drawFbo, m_imageIndexAcquired);
// Depth/stencil participation of THIS draw, for the default-FBO depth-less
// pass flavor (GL: a disabled depth/stencil test neither reads nor writes
// its buffer).
const Bool drawUsesDepthStencil =
MG_State::pGLContext->IsCapabilityEnabled(CapabilityInput::DepthTest) ||
MG_State::pGLContext->IsCapabilityEnabled(CapabilityInput::StencilTest);
auto* renderPassEntry =
&m_renderPassManager->GetOrCreateRenderPass(*drawFbo, m_imageIndexAcquired, drawUsesDepthStencil);
if (activeRenderPass && !activeRenderPass->CompatibleWith(*renderPassEntry)) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
activeRenderPass = nullptr;
renderPassEntry = &m_renderPassManager->GetOrCreateRenderPass(*drawFbo, m_imageIndexAcquired);
renderPassEntry =
&m_renderPassManager->GetOrCreateRenderPass(*drawFbo, m_imageIndexAcquired, drawUsesDepthStencil);
}
if (renderPassEntry->attachmentCount == 0 || renderPassEntry->extent.x() <= 0 || renderPassEntry->extent.y() <= 0) {
MGLOG_D("SetupDraw skipped: drawFbo=%u resolved to an empty render pass (attachmentCount=%u extent=%dx%d)",
@@ -4463,7 +4846,7 @@ void main() {
// Every genuinely disabled attribute the shader reads must have a current-value type we can
// synthesize a binding for; otherwise the upload below would push a null payload.
const Uint32 missingAttribMask =
activeAttribMask & ~BuildVertexInputAttributeMask(vertexInputState.attributes);
activeAttribMask & ~vertexInputState.attributeLocationMask;
for (Uint32 location = 0; location < kMaxVertexAttribs; ++location) {
if ((missingAttribMask & (1u << location)) == 0) continue;
@@ -4491,7 +4874,11 @@ void main() {
MOBILEGL_ASSERT(ok, "%s: BeginRenderPass failed", __func__);
}
if (!g_dynamicStateShadow.graphicsPipelineValid || g_dynamicStateShadow.graphicsPipeline != pipeline) {
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
g_dynamicStateShadow.graphicsPipelineValid = true;
g_dynamicStateShadow.graphicsPipeline = pipeline;
}
const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers(
frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex());
@@ -4531,7 +4918,49 @@ void main() {
scissor.offset = {0, 0};
scissor.extent = { (Uint)renderPassEntry->extent.x(), (Uint)renderPassEntry->extent.y() };
}
vkCmdSetScissor(frame.commandBuffer, 0, 1, &scissor);
ShadowedSetScissor(frame.commandBuffer, scissor);
// Snapshot the fully resolved configuration for the consecutive-draw
// fast path (see TrySetupDrawFastPath).
{
auto& snap = m_setupDrawSnapshot;
const auto* nowActiveRenderPass = VkRenderPassManager::GetActiveRenderPass();
if (nowActiveRenderPass != nullptr && !programObj.hasStorageImages) {
snap.valid = true;
snap.aspects = aspects.GetRaw();
snap.mode = mode;
snap.programLifetimeId = program.GetLifetimeId();
snap.programVersion = program.GetBackendStateVersion();
snap.vao = &vao;
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.drawFbo = drawFbo.get();
snap.fboVersion = drawFbo->GetObjectVersion();
snap.drawFboIsDefault = drawFbo->IsDefaultFramebuffer();
snap.renderStateVersion = MG_State::pGLContext->GetRenderStateParametersVersion();
snap.bindGeneration = MG_State::pGLContext->GetTextureBindGeneration();
snap.baseTransformFlags = GetShaderTransformFlags(m_swapchainObject.GetPreTransform()).GetRaw();
snap.resolvedTransformFlags = transformFlags.GetRaw();
snap.renderPassHash = nowActiveRenderPass->hash;
snap.imageIndex = m_imageIndexAcquired;
snap.textureEraseEpoch = m_textureManager->GetResourceEraseEpoch();
snap.textureImageEpoch = m_textureManager->GetTextureImageEpoch();
snap.renderbufferImageEpoch = m_renderPassManager->GetRenderbufferImageEpoch();
snap.renderPassExtent = renderPassEntry->extent;
snap.pipeline = pipeline;
Uint64 snapContentSum = 0;
Uint64 snapParamsSum = 0;
for (const auto* sampledTexture : sampledTextures) {
if (sampledTexture != nullptr) {
snapContentSum += sampledTexture->GetContentVersion();
snapParamsSum += sampledTexture->GetTextureParamsVersion();
}
}
snap.sampledContentSum = snapContentSum;
snap.sampledParamsSum = snapParamsSum;
} else {
snap.valid = false;
}
}
return true;
}
@@ -5222,8 +5651,12 @@ void main() {
if (!m_clearManager->GetPendingClears(&texture, pendingClears)) {
return true;
}
MOBILEGL_ASSERT(VkRenderPassManager::GetActiveRenderPass() == nullptr,
"MaterializePendingClearForTexture requires no active render pass");
// A pass may stay open on the FRAME command buffer while this clear is
// recorded into the pre-pass stream (a different command buffer that
// executes strictly before the frame's commands).
MOBILEGL_ASSERT(VkRenderPassManager::GetActiveRenderPass() == nullptr ||
commandBuffer != m_frameContext.GetCurrent().commandBuffer,
"MaterializePendingClearForTexture requires no active render pass on the target buffer");
auto* resource = m_textureManager->SyncTextureAndGetDescriptor(texture);
MOBILEGL_ASSERT(resource != nullptr,
@@ -5453,7 +5886,10 @@ void main() {
"TryBlitToDefaultFramebufferWithShader: failed to create sampled view for textureId=%d mip=%u",
sourceTexture->GetExternalIndex(), srcBinding.mipLevel);
auto& renderPassEntry = m_renderPassManager->GetOrCreateRenderPass(drawFbo, m_imageIndexAcquired);
// A color-only blit never touches depth/stencil: let the default-FBO pass
// it opens skip the depth attachment (depth-less flavor).
auto& renderPassEntry =
m_renderPassManager->GetOrCreateRenderPass(drawFbo, m_imageIndexAcquired, /*drawUsesDepthStencil=*/false);
const Bool ok = VkRenderPassManager::BeginRenderPass(frame.commandBuffer, renderPassEntry);
MOBILEGL_ASSERT(ok, "%s: BeginRenderPass failed", __func__);
@@ -5469,6 +5905,10 @@ void main() {
const VkPipeline pipeline = GetOrCreateBlitPipeline(renderPassEntry);
MOBILEGL_ASSERT(pipeline != VK_NULL_HANDLE, "TryBlitToDefaultFramebufferWithShader: blit pipeline is null");
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
// The blit pipeline's narrower dynamic set (viewport/scissor only)
// leaves the other dynamic states undefined; its raw viewport/scissor
// writes also bypass the shadow.
ResetDynamicStateShadow();
auto* blitProgramData = static_cast<Uint8*>(m_blitResources.program->MapUBO());
MOBILEGL_ASSERT(blitProgramData != nullptr, "TryBlitToDefaultFramebufferWithShader: blit UBO is null");
@@ -6263,9 +6703,13 @@ void main() {
if (frame.isCommandRecording) {
m_frameContext.EndCommandRecording();
frame.hasCommandBufferRecorded = true;
m_lastPipelineValid = false; // command-buffer boundary: drop the pipeline memo
InvalidatePipelineMemo(); // command-buffer boundary: drop the pipeline memo
}
if (!frame.hasCommandBufferRecorded) {
// The pre-pass stream must never be submitted later than the recording
// it was paired with (frame commands recorded after a pre-pass move
// rely on the moved work having executed first).
m_frameContext.EndPreCommandRecordingIfOpen();
if (!frame.hasCommandBufferRecorded && !frame.hasPreCommandBufferRecorded) {
return true;
}
@@ -7346,8 +7790,7 @@ void main() {
m_programFactory->OnFrameBoundary();
}
if (m_pipelineFactory && m_pipelineFactory->OnFrameBoundary() > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
InvalidatePipelineMemo();
}
if (m_vertexInputStateFactory) {
m_vertexInputStateFactory->OnFrameBoundary();
@@ -7400,8 +7843,18 @@ void main() {
submitInfo.pWaitSemaphores = &waitSemaphore;
submitInfo.pWaitDstStageMask = &waitDstStageMask;
}
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &frame.commandBuffer;
// The pre-pass stream, when recorded, executes strictly before the
// frame's commands within the same submission.
VkCommandBuffer commandBuffers[2] = {VK_NULL_HANDLE, VK_NULL_HANDLE};
Uint32 commandBufferCount = 0;
if (frame.hasPreCommandBufferRecorded) {
commandBuffers[commandBufferCount++] = frame.preCommandBuffer;
}
if (frame.hasCommandBufferRecorded) {
commandBuffers[commandBufferCount++] = frame.commandBuffer;
}
submitInfo.commandBufferCount = commandBufferCount;
submitInfo.pCommandBuffers = commandBuffers;
const VkResult result = vkQueueSubmit(m_graphicsQueue, 1, &submitInfo, fence);
if (result != VK_SUCCESS) {
MGLOG_E("SubmitPendingCommandBuffer: vkQueueSubmit returned %d", result);
@@ -7409,6 +7862,7 @@ void main() {
}
frame.imageAvailableSemaphoreConsumed = true;
frame.hasCommandBufferRecorded = false;
frame.hasPreCommandBufferRecorded = false;
RegisterSubmit(fence, pooledFence);
frame.lastSubmitIndex = m_submitCounter;
return true;
@@ -7439,6 +7893,8 @@ void main() {
}
m_frameContext.EndCommandRecording();
}
m_frameContext.EndPreCommandRecordingIfOpen();
const Bool submittingPreCommandBuffer = frame.hasPreCommandBufferRecorded;
if (!SubmitPendingCommandBuffer(frame, fence, /*pooledFence=*/true)) {
// Submit failure (device loss regime): the ended command buffer
// stays marked recorded so Present can still try to submit it.
@@ -7451,13 +7907,12 @@ void main() {
// cache and the aging sweep could destroy it while the flushed submission
// still references it. Mirrors the drops at the readback and Present
// boundaries; costs one full pipeline lookup on the next draw.
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
InvalidatePipelineMemo();
// The submitted command buffer may still be executing; recording must
// restart on a fresh one. If none can be allocated, fall back to
// draining this submission so reusing the buffer stays legal.
const VkResult retireResult = m_frameContext.RetireCurrentCommandBuffer();
const VkResult retireResult = m_frameContext.RetireCurrentCommandBuffer(submittingPreCommandBuffer);
if (retireResult != VK_SUCCESS) {
MGLOG_E("FlushPendingCommands: RetireCurrentCommandBuffer returned %d; draining submission", retireResult);
if (vkWaitForFences(m_device, 1, &fence, VK_TRUE, UINT64_MAX) == VK_SUCCESS) {
@@ -7523,6 +7978,17 @@ void main() {
}
void VulkanRenderer::OnFrameCommandRecordingBegan(VkCommandBuffer commandBuffer) {
// Dynamic state does not survive a command-buffer boundary.
ResetDynamicStateShadow();
m_setupDrawSnapshot.valid = false;
if (m_uniformManager) {
m_uniformManager->OnCommandBufferBoundary();
}
// Pre-pass stream bookkeeping: a fresh frame recording references no
// textures yet.
if (m_textureManager) {
m_textureManager->AdvanceRecordingGeneration();
}
if (m_timerQueryManager) {
m_timerQueryManager->OnFrameCommandRecordingBegan(commandBuffer, m_frameContext.GetCurrentFrameIndex(),
m_bufferManager.GetFrameSerial());
@@ -7595,9 +8061,10 @@ void main() {
if (suspendedFrame.isCommandRecording) {
m_frameContext.EndCommandRecording();
}
m_frameContext.AbandonPreCommandRecording();
suspendedFrame.isCommandRecording = false;
suspendedFrame.hasCommandBufferRecorded = false;
m_lastPipelineValid = false;
InvalidatePipelineMemo();
// The dropped recording is never submitted, so once the fence
// poll shows the pre-suspension submissions complete the frame
// transients (descriptor sets, transient arenas, deferred
@@ -7633,8 +8100,11 @@ void main() {
// performs a real, stamping lookup) and can never age out.
m_programFactory->OnFrameBoundary();
if (m_pipelineFactory->OnFrameBoundary() > 0) {
m_lastPipelineValid = false; // an aged-out pipeline may still be memoized
m_lastPipelineResult = VK_NULL_HANDLE;
InvalidatePipelineMemo(); // an aged-out pipeline may still be memoized
// A recreated pipeline could reuse a freed handle value and alias
// the bind-dedup shadow; force the next draw to re-bind.
g_dynamicStateShadow.graphicsPipelineValid = false;
m_setupDrawSnapshot.valid = false;
}
m_vertexInputStateFactory->OnFrameBoundary();
m_samplerManager->OnFrameBoundary();
@@ -7657,18 +8127,21 @@ void main() {
if (frame.isCommandRecording) {
m_frameContext.EndCommandRecording();
frame.hasCommandBufferRecorded = true;
m_lastPipelineValid = false; // command-buffer boundary: drop the pipeline memo
InvalidatePipelineMemo(); // command-buffer boundary: drop the pipeline memo
}
m_frameContext.EndPreCommandRecordingIfOpen();
const Bool shouldSubmitCommandBuffer = frame.hasCommandBufferRecorded;
// 1) Submit current frame work.
// 1) Submit current frame work (the pre-pass stream, when recorded,
// rides the same submission strictly ahead of the frame commands).
auto submitPacket = m_frameContext.GetSubmitInfo(shouldSubmitCommandBuffer, m_imageIndexAcquired);
VK_VERIFY(vkQueueSubmit(m_graphicsQueue, 1, &submitPacket.submitInfo, frame.imageInFlightFence));
RegisterSubmit(frame.imageInFlightFence, /*pooledFence=*/false);
frame.lastSubmitIndex = m_submitCounter;
frame.isCommandRecording = false;
frame.hasCommandBufferRecorded = false;
frame.hasPreCommandBufferRecorded = false;
m_swapchainObject.SetImageLayout(m_imageIndexAcquired, VK_IMAGE_LAYOUT_PRESENT_SRC_KHR);
// 2) Present current frame.
@@ -7696,6 +8169,13 @@ void main() {
result = VK_SUCCESS;
}
VK_VERIFY(result, "Present, vkQueuePresentKHR");
// EGL swap semantics: the presented color buffer's content is undefined the
// next time this image is acquired (EGL_BUFFER_DESTROYED, the default swap
// behaviour), and EVERY ancillary depth/stencil buffer's content is
// undefined after any swap. The render-pass manager turns the undefined
// attachments' next tile loads into LOAD_OP_DONT_CARE.
m_swapchainObject.SetImageContentDefined(m_imageIndexAcquired, false);
m_swapchainObject.SetAllDepthStencilContentUndefined();
// The authoritative check, done here - after the frame is presented, before the next
// acquire. This is what makes a launcher-side resolution change take effect: shrinking
// the window's buffer (SurfaceHolder.setFixedSize) moves currentExtent, the swapchain
@@ -8650,11 +9130,17 @@ void main() {
if (m_pipelineFactory) {
m_pipelineFactory->DestroyAll();
}
m_lastPipelineValid = false; // pipelines freed -> the memoized handle would dangle
InvalidatePipelineMemo(); // pipelines freed -> the memoized handle would dangle
g_dynamicStateShadow.graphicsPipelineValid = false;
m_setupDrawSnapshot.valid = false;
DestroyComputePipelines();
if (m_frameContext.GetFrameCount() > 0) {
m_frameContext.GetCurrent().isCommandRecording = false;
m_frameContext.GetCurrent().hasCommandBufferRecorded = false;
// The pre-pass stream paired with the abandoned recording is
// dropped with it (its next Begin resets the buffer).
m_frameContext.GetCurrent().isPreCommandRecording = false;
m_frameContext.GetCurrent().hasPreCommandBufferRecorded = false;
}
const Bool okArena = m_bufferManager.RecreateTransientArenas(m_frameContext.GetFrameCount());
MOBILEGL_ASSERT(okArena, "RecreateSwapchain: buffer manager transient arena initialization failed");
@@ -8809,8 +9295,7 @@ void main() {
// destroys them immediately. The memo must drop as well: it can hand out a
// cached handle without touching the factory.
if (m_pipelineFactory->EvictByRenderPasses(renderPasses) > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
InvalidatePipelineMemo();
}
}
@@ -8829,8 +9314,7 @@ void main() {
m_computePipelines.erase(computeIt);
}
if (m_pipelineFactory != nullptr && m_pipelineFactory->EvictByProgramHash(programHash) > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
InvalidatePipelineMemo();
}
if (m_uniformManager != nullptr) {
m_uniformManager->OnDescriptorSetLayoutDestroyed(descriptorSetLayout);
@@ -151,6 +151,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool SetupDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView = nullptr);
// ANGLE-style consecutive-draw fast path: SetupDraw snapshots the fully
// resolved draw configuration; the next draw whose cheap version/identity
// checks all match skips the resolution half (LOD probe, sampled-set
// walk, render-pass and pipeline resolution) and jumps straight to the
// per-draw tail. Returns false (leaving no side effects that the full
// path cannot redo idempotently) whenever anything might have changed.
Bool TrySetupDrawFastPath(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams, const IndexBufferView* pIndexBufferView);
void ClearAttachmentsOnActiveRenderPass(VkCommandBuffer commandBuffer,
const RenderPassEntry& compatibleRenderPassEntry);
@@ -455,14 +463,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// gather + synthetic vertex-input rebuild + payload hash + lookup) when the full pipeline
// state is unchanged from the previous draw. The key provably covers every pipeline field.
// Reset per-frame and on pipeline destruction so the cached handle can never dangle.
Bool m_lastPipelineValid = false;
GLenum m_lastPipelineMode = 0;
Uint64 m_lastPipelineProgramHash = 0;
Uint64 m_lastPipelineVertexInputHash = 0;
Uint64 m_lastPipelineRenderPassHash = 0;
Uint m_lastPipelineRenderStateVersion = 0;
ProgramFactory::CompileOptionFlags m_lastPipelineTransformFlags = {};
VkPipeline m_lastPipelineResult = VK_NULL_HANDLE;
// Small N-way pipeline-resolution memo (round-robin replacement). A
// single-entry memo thrashed on draw sequences that alternate a few
// pipelines (GUI text/quad program ping-pong), paying the full
// payload-hash lookup per draw; eight entries cover such working sets
// while keeping the hit path a trivial linear scan.
struct PipelineMemoEntry {
GLenum mode = 0;
Uint64 programHash = 0;
Uint64 vertexInputHash = 0;
Uint64 renderPassHash = 0;
Uint renderStateVersion = 0;
ProgramFactory::CompileOptionFlags transformFlags = {};
VkPipeline pipeline = VK_NULL_HANDLE;
};
static constexpr Uint32 kPipelineMemoSize = 8;
PipelineMemoEntry m_pipelineMemo[kPipelineMemoSize];
Uint32 m_pipelineMemoCount = 0;
Uint32 m_pipelineMemoNext = 0;
// Drops every memoized pipeline handle. Required at command-buffer
// boundaries and whenever any pipeline may have been destroyed.
void InvalidatePipelineMemo() {
m_pipelineMemoCount = 0;
m_pipelineMemoNext = 0;
}
UnorderedMap<ProgramFactory::HashType, VkPipeline> m_computePipelines;
UniquePtr<ProgramFactory> m_programFactory;
UniquePtr<UniformManager> m_uniformManager;
@@ -491,9 +515,61 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ProgramFactory::CompileOptionFlags m_lastSampledSetTransformFlags = {};
Uint64 m_lastSampledSetBindGeneration = 0;
// Memo for the per-draw explicit-LOD-0 eligibility probe
// (ProgramSamplesOnlySingleLevelTextures): same key family as the
// sampled-set memo, plus the sampled textures' params-version sum so a
// level-range or filter change re-probes. On a hit the resolved
// transform flags are reused, which also collapses the two
// GetOrCreateProgram lookups into one.
Bool m_lastLodDecisionValid = false;
Uint64 m_lastLodProgramLifetimeId = 0;
Uint32 m_lastLodProgramVersion = 0;
Uint64 m_lastLodBindGeneration = 0;
Uint64 m_lastLodParamsSum = 0;
ProgramFactory::CompileOptionFlags m_lastLodBaseFlags = {};
ProgramFactory::CompileOptionFlags m_lastLodResultFlags = {};
// Snapshot behind TrySetupDrawFastPath. Values only: the program and
// render-pass caches are open-addressing maps whose entries move on
// insert, so no pointers into them are cached; the pipeline handle is
// protected by the command-buffer-boundary reset plus the mid-frame
// pipeline-destruction resets, and monotonic epochs guard everything
// that can be destroyed or recreated between draws.
struct SetupDrawSnapshot {
Bool valid = false;
Uint8 aspects = 0;
GLenum mode = 0;
Uint64 programLifetimeId = 0;
Uint32 programVersion = 0;
const void* vao = nullptr;
Uint32 vaoConfigVersion = 0;
const void* drawFbo = nullptr;
Uint16 fboVersion = 0;
Bool drawFboIsDefault = false;
Uint renderStateVersion = 0;
Uint64 bindGeneration = 0;
Uint32 baseTransformFlags = 0;
Uint32 resolvedTransformFlags = 0;
Uint64 renderPassHash = 0;
Uint32 imageIndex = 0;
Uint64 textureEraseEpoch = 0;
Uint64 textureImageEpoch = 0;
Uint64 renderbufferImageEpoch = 0;
Uint64 sampledContentSum = 0;
Uint64 sampledParamsSum = 0;
IntVec2 renderPassExtent = {0, 0};
VkPipeline pipeline = VK_NULL_HANDLE;
};
SetupDrawSnapshot m_setupDrawSnapshot;
// Per-draw scratch buffers (clear keeps capacity) — these paths run for every
// draw call and must not allocate.
Vector<MG_State::GLState::ITextureObject*> m_sampledTexturesScratch;
// Parallel to m_sampledTexturesScratch, refilled by every SetupDraw's
// first sampled-texture loop: the resolved backend resources, so the
// post-transition loop can skip re-resolving textures whose layout is
// already sampleable.
Vector<VkTextureManager::TextureResource*> m_sampledResourcesScratch;
Vector<MG_State::GLState::ITextureObject*> m_storageImageTexturesScratch;
Vector<VkBuffer> m_vertexBuffersScratch;
Vector<VkDeviceSize> m_vertexOffsetsScratch;
@@ -465,6 +465,14 @@ namespace MobileGL::MG_Impl::GLImpl {
case GL_STENCIL_TEST:
*params = MG_State::pGLContext->IsCapabilityEnabled(CapabilityInput::StencilTest) ? GL_TRUE : GL_FALSE;
return;
case GL_MIN_FRAGMENT_INTERPOLATION_OFFSET:
case GL_MAX_FRAGMENT_INTERPOLATION_OFFSET:
case GL_FRAGMENT_INTERPOLATION_OFFSET_BITS: {
GLfloat value = 0.0f;
GetFloatv(pname, &value);
*params = value != 0.0f ? GL_TRUE : GL_FALSE;
return;
}
default:
break;
}
@@ -520,6 +528,19 @@ namespace MobileGL::MG_Impl::GLImpl {
params[1] = dynamicParameters.ViewportBoundsRangeMax;
return;
}
case GL_MIN_FRAGMENT_INTERPOLATION_OFFSET:
case GL_MAX_FRAGMENT_INTERPOLATION_OFFSET:
case GL_FRAGMENT_INTERPOLATION_OFFSET_BITS: {
const auto& dynamicParameters = MG_Backend::pActiveBackendObject->GetDynamicParameters();
if (pname == GL_MIN_FRAGMENT_INTERPOLATION_OFFSET) {
params[0] = dynamicParameters.MinFragmentInterpolationOffset;
} else if (pname == GL_MAX_FRAGMENT_INTERPOLATION_OFFSET) {
params[0] = dynamicParameters.MaxFragmentInterpolationOffset;
} else {
params[0] = static_cast<GLfloat>(dynamicParameters.FragmentInterpolationOffsetBits);
}
return;
}
case GL_DEPTH_CLEAR_VALUE:
params[0] = MG_State::pGLContext->GetClearDepth();
return;
@@ -1957,6 +1978,15 @@ namespace MobileGL::MG_Impl::GLImpl {
case GL_SUBPIXEL_BITS:
*params = std::max(dynamicParameters.ViewportSubpixelBits, kFrontendSubpixelBits);
break;
case GL_MIN_FRAGMENT_INTERPOLATION_OFFSET:
*params = static_cast<GLint>(std::lround(dynamicParameters.MinFragmentInterpolationOffset));
break;
case GL_MAX_FRAGMENT_INTERPOLATION_OFFSET:
*params = static_cast<GLint>(std::lround(dynamicParameters.MaxFragmentInterpolationOffset));
break;
case GL_FRAGMENT_INTERPOLATION_OFFSET_BITS:
*params = dynamicParameters.FragmentInterpolationOffsetBits;
break;
case GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT:
*params = static_cast<Int>(dynamicParameters.UniformBufferOffsetAlignment);
break;
@@ -104,6 +104,23 @@ namespace MobileGL {
m_backendHashMemoVersion = m_configVersion;
}
// Backend-owned resolved-state memo: an opaque pointer into the
// backend's vertex-input-state cache plus the cache's eviction
// epoch, valid while the config version matches. Lets the
// per-draw path skip the content hash AND the cache lookup; the
// epoch guards against the cache evicting the pointee.
Bool GetBackendStateMemo(const void*& outState, Uint64& outEpoch) const {
if (m_backendStateMemoVersion != m_configVersion) return false;
outState = m_backendStateMemo;
outEpoch = m_backendStateMemoEpoch;
return true;
}
void SetBackendStateMemo(const void* state, Uint64 epoch) const {
m_backendStateMemo = state;
m_backendStateMemoEpoch = epoch;
m_backendStateMemoVersion = m_configVersion;
}
private:
void BumpAttributeFormatVersion(Uint index);
void BumpAttributeBufferVersion(Uint index);
@@ -137,6 +154,9 @@ namespace MobileGL {
Uint32 m_configVersion = 0;
mutable Uint64 m_backendHashMemo = 0;
mutable Uint32 m_backendHashMemoVersion = ~0u;
mutable const void* m_backendStateMemo = nullptr;
mutable Uint64 m_backendStateMemoEpoch = 0;
mutable Uint32 m_backendStateMemoVersion = ~0u;
};
} // namespace GLState
} // namespace MG_State
@@ -34,6 +34,11 @@ namespace {
GLint maxFragmentImageUniforms = 4;
GLint maxComputeImageUniforms = 5;
bool maxGeometryImageUniformsQueried = false;
GLfloat minFragmentInterpolationOffset = -0.75f;
GLfloat maxFragmentInterpolationOffset = 0.625f;
GLint fragmentInterpolationOffsetBits = 6;
bool fragmentInterpolationLimitsQueried = false;
bool fragmentInterpolationQueryRaisesError = false;
// Emulates ANGLE-on-Vulkan: the draw reads the indirect command's
// baseInstance word and exposes it through gl_InstanceID.
bool drawLeaksBaseInstanceWord = false;
@@ -108,6 +113,14 @@ namespace {
case GL_MAX_COMPUTE_IMAGE_UNIFORMS:
*data = g_fake.maxComputeImageUniforms;
break;
case GL_FRAGMENT_INTERPOLATION_OFFSET_BITS:
g_fake.fragmentInterpolationLimitsQueried = true;
if (g_fake.fragmentInterpolationQueryRaisesError) {
g_fake.pendingError = GL_INVALID_ENUM;
} else {
*data = g_fake.fragmentInterpolationOffsetBits;
}
break;
// FillInGLESCapabilities reads the context version before running the
// baseInstance probe, which requires ES >= 3.1.
case GL_MAJOR_VERSION:
@@ -155,6 +168,22 @@ namespace {
g_fake.maxTextureMaxAnisotropyQueried = true;
data[0] = g_fake.maxTextureMaxAnisotropy;
break;
case GL_MIN_FRAGMENT_INTERPOLATION_OFFSET:
g_fake.fragmentInterpolationLimitsQueried = true;
if (g_fake.fragmentInterpolationQueryRaisesError) {
g_fake.pendingError = GL_INVALID_ENUM;
} else {
data[0] = g_fake.minFragmentInterpolationOffset;
}
break;
case GL_MAX_FRAGMENT_INTERPOLATION_OFFSET:
g_fake.fragmentInterpolationLimitsQueried = true;
if (g_fake.fragmentInterpolationQueryRaisesError) {
g_fake.pendingError = GL_INVALID_ENUM;
} else {
data[0] = g_fake.maxFragmentInterpolationOffset;
}
break;
// Two-component range queries.
case GL_ALIASED_LINE_WIDTH_RANGE:
case GL_SMOOTH_LINE_WIDTH_RANGE:
@@ -462,6 +491,52 @@ TEST(ImageUniformCapabilities, QueriesRealPerStageLimitsAndConservativelyGatesGe
EXPECT_TRUE(g_fake.maxGeometryImageUniformsQueried);
}
TEST(FragmentInterpolationCapabilities, QueriesOnlyWhenSupportedAndPreservesDriverLimits) {
const auto funcs = MakeFakeGLESFunctions();
ResetFakeDriver();
g_fake.maxVertexSsboBlocks = 0;
MobileGL::MG_External::GLESCapabilities unsupportedCaps;
ASSERT_TRUE(MobileGL::MG_Util::BackendLoader::FillInGLESCapabilities(unsupportedCaps, funcs));
EXPECT_FALSE(unsupportedCaps.SupportsShaderMultisampleInterpolation);
EXPECT_FALSE(g_fake.fragmentInterpolationLimitsQueried);
EXPECT_FLOAT_EQ(unsupportedCaps.MinFragmentInterpolationOffset, -0.5f);
EXPECT_FLOAT_EQ(unsupportedCaps.MaxFragmentInterpolationOffset, 0.4375f);
EXPECT_EQ(unsupportedCaps.FragmentInterpolationOffsetBits, 4);
ResetFakeDriver();
g_fake.maxVertexSsboBlocks = 0;
g_fake.extensions.emplace_back("GL_OES_shader_multisample_interpolation");
// A stale error from an earlier capability probe must not make the optional
// interpolation query look like it failed.
g_fake.pendingError = GL_INVALID_OPERATION;
MobileGL::MG_External::GLESCapabilities supportedCaps;
ASSERT_TRUE(MobileGL::MG_Util::BackendLoader::FillInGLESCapabilities(supportedCaps, funcs));
EXPECT_TRUE(supportedCaps.SupportsShaderMultisampleInterpolation);
EXPECT_TRUE(g_fake.fragmentInterpolationLimitsQueried);
EXPECT_FLOAT_EQ(supportedCaps.MinFragmentInterpolationOffset, g_fake.minFragmentInterpolationOffset);
EXPECT_FLOAT_EQ(supportedCaps.MaxFragmentInterpolationOffset, g_fake.maxFragmentInterpolationOffset);
EXPECT_EQ(supportedCaps.FragmentInterpolationOffsetBits, g_fake.fragmentInterpolationOffsetBits);
EXPECT_EQ(funcs.glGetError(), GL_NO_ERROR);
}
TEST(FragmentInterpolationCapabilities, QueryErrorIsDrainedAndFallsBackToCoreMinimums) {
ResetFakeDriver();
g_fake.maxVertexSsboBlocks = 0;
g_fake.extensions.emplace_back("GL_OES_shader_multisample_interpolation");
g_fake.fragmentInterpolationQueryRaisesError = true;
const auto funcs = MakeFakeGLESFunctions();
MobileGL::MG_External::GLESCapabilities caps;
ASSERT_TRUE(MobileGL::MG_Util::BackendLoader::FillInGLESCapabilities(caps, funcs));
EXPECT_TRUE(g_fake.fragmentInterpolationLimitsQueried);
EXPECT_FLOAT_EQ(caps.MinFragmentInterpolationOffset, -0.5f);
EXPECT_FLOAT_EQ(caps.MaxFragmentInterpolationOffset, 0.4375f);
EXPECT_EQ(caps.FragmentInterpolationOffsetBits, 4);
EXPECT_EQ(funcs.glGetError(), GL_NO_ERROR);
}
// The extension string is what apps gate on (LWJGL builds GLCapabilities from it), so advertising
// it on a driver that cannot filter anisotropically would leave them silently on trilinear.
TEST(TextureAnisotropyCapabilities, ExtensionIsAdvertisedOnlyWhenTheHostDriverSupportsIt) {
+92 -6
View File
@@ -197,13 +197,13 @@ TEST(DirectGLESSanity, AdvertisesDepthTextureForGlmarkShadowScenes) {
EXPECT_NE(std::find(extensions.begin(), extensions.end(), MobileGL::E_GL_ARB_depth_texture), extensions.end());
}
TEST(DirectGLESSanity, AdvertisesVoxyRequiredRenderingExtensionsWithoutRaisingGLVersion) {
TEST(DirectGLESSanity, AdvertisesVoxyRequiredRenderingExtensionsAtExperimentalCTSVersion) {
MobileGL::MG_Backend::DirectGLES::BackendObject_DirectGLES backend;
const auto& rendererInfo = backend.GetRendererInfo().RendererGLInfo;
const auto& extensions = rendererInfo.Extensions;
EXPECT_EQ(rendererInfo.TargetGLVersion.Major, 3);
EXPECT_EQ(rendererInfo.TargetGLVersion.Minor, 3);
EXPECT_EQ(rendererInfo.TargetGLVersion.Major, 4);
EXPECT_EQ(rendererInfo.TargetGLVersion.Minor, 6);
EXPECT_EQ(rendererInfo.TargetGLVersion.Patch, 0);
EXPECT_NE(std::find(extensions.begin(), extensions.end(), MobileGL::E_GL_ARB_compute_shader),
@@ -397,13 +397,13 @@ TEST(DirectVulkanSanity, RenderPassExtentUsesSwapchainSizeOnlyForDefaultFramebuf
MobileGL::IntVec2(512, 512));
}
TEST(DirectVulkanSanity, AdvertisesVoxyRequiredRenderingExtensionsWithoutRaisingGLVersion) {
TEST(DirectVulkanSanity, AdvertisesVoxyRequiredRenderingExtensionsAtExperimentalCTSVersion) {
MobileGL::MG_Backend::DirectVulkan::BackendObject_DirectVulkan backend;
const auto& rendererInfo = backend.GetRendererInfo().RendererGLInfo;
const auto& extensions = rendererInfo.Extensions;
EXPECT_EQ(rendererInfo.TargetGLVersion.Major, 3);
EXPECT_EQ(rendererInfo.TargetGLVersion.Minor, 3);
EXPECT_EQ(rendererInfo.TargetGLVersion.Major, 4);
EXPECT_EQ(rendererInfo.TargetGLVersion.Minor, 6);
EXPECT_EQ(rendererInfo.TargetGLVersion.Patch, 0);
EXPECT_NE(std::find(extensions.begin(), extensions.end(), MobileGL::E_GL_ARB_compute_shader),
@@ -516,6 +516,50 @@ TEST(DirectGLESSanity, PreservesHostPerStageImageUniformLimits) {
EXPECT_EQ(params.MaxComputeImageUniforms, 5);
}
TEST(FragmentInterpolationCapabilities, PlumbsGLESAndBothVulkanPropertyPaths) {
using namespace MobileGL;
MG_External::GLESCapabilities glesCaps;
glesCaps.MinFragmentInterpolationOffset = -0.75f;
glesCaps.MaxFragmentInterpolationOffset = 0.625f;
glesCaps.FragmentInterpolationOffsetBits = 6;
MG_Backend::DirectGLES::BackendObject_DirectGLES glesBackend;
glesBackend.ApplyGLESCapabilitiesForTesting(glesCaps);
EXPECT_FLOAT_EQ(glesBackend.GetDynamicParameters().MinFragmentInterpolationOffset, -0.75f);
EXPECT_FLOAT_EQ(glesBackend.GetDynamicParameters().MaxFragmentInterpolationOffset, 0.625f);
EXPECT_EQ(glesBackend.GetDynamicParameters().FragmentInterpolationOffsetBits, 6);
VkPhysicalDeviceProperties properties{};
// A common Vulkan limit pair: max is one representable 4-bit step below 0.5.
properties.limits.minInterpolationOffset = -0.5f;
properties.limits.maxInterpolationOffset = 0.4375f;
properties.limits.subPixelInterpolationOffsetBits = 4;
MG_External::VulkanCapabilities vkCaps;
MG_Util::BackendLoader::FillInVulkanCapabilities(vkCaps, properties);
EXPECT_FLOAT_EQ(vkCaps.MinFragmentInterpolationOffset, -0.5f);
EXPECT_FLOAT_EQ(vkCaps.MaxFragmentInterpolationOffset, 0.4375f);
EXPECT_EQ(vkCaps.FragmentInterpolationOffsetBits, 4);
vkCaps.MinFragmentInterpolationOffset = -0.875f;
vkCaps.MaxFragmentInterpolationOffset = 0.75f;
vkCaps.FragmentInterpolationOffsetBits = 7;
MG_Backend::DirectVulkan::BackendObject_DirectVulkan vkBackend;
vkBackend.ApplyVulkanCapabilitiesForTesting(vkCaps);
EXPECT_FLOAT_EQ(vkBackend.GetDynamicParameters().MinFragmentInterpolationOffset, -0.875f);
EXPECT_FLOAT_EQ(vkBackend.GetDynamicParameters().MaxFragmentInterpolationOffset, 0.75f);
EXPECT_EQ(vkBackend.GetDynamicParameters().FragmentInterpolationOffsetBits, 7);
// Invalid/zero host data cannot under-advertise the OpenGL 4 minimums.
MG_External::VulkanCapabilities invalidCaps;
invalidCaps.MinFragmentInterpolationOffset = 0.0f;
invalidCaps.MaxFragmentInterpolationOffset = 0.0f;
invalidCaps.FragmentInterpolationOffsetBits = 0;
vkBackend.ApplyVulkanCapabilitiesForTesting(invalidCaps);
EXPECT_LE(vkBackend.GetDynamicParameters().MinFragmentInterpolationOffset, -0.5f);
EXPECT_FLOAT_EQ(vkBackend.GetDynamicParameters().MaxFragmentInterpolationOffset, 0.4375f);
EXPECT_EQ(vkBackend.GetDynamicParameters().FragmentInterpolationOffsetBits, 4);
}
TEST(DirectVulkanSanity, AdvertisesSubgroupOnlyWhenVulkanReportsUsableSupport) {
using namespace MobileGL;
@@ -638,6 +682,48 @@ TEST(GetterSanity, ClampsMaxVertexAttribsToCurrentValueStorageCapacity) {
MG_State::pGLContext.reset();
}
TEST(GetterSanity, ReportsFragmentInterpolationLimitsForFloatAndIntegerQueries) {
using namespace MobileGL;
auto previousContext = Move(MG_State::pGLContext);
auto previousBackend = Move(MG_Backend::pActiveBackendObject);
MG_State::pGLContext = MakeUnique<MG_State::GLState::GLContext>();
MG_Backend::DynamicBackendParameters params;
params.MinFragmentInterpolationOffset = -0.75f;
params.MaxFragmentInterpolationOffset = 0.4375f;
params.FragmentInterpolationOffsetBits = 6;
MG_Backend::pActiveBackendObject = MakeUnique<DynamicParameterBackend>(params);
GLfloat floatValue = 0.0f;
MG_Impl::GLImpl::GetFloatv(GL_MIN_FRAGMENT_INTERPOLATION_OFFSET, &floatValue);
EXPECT_FLOAT_EQ(floatValue, -0.75f);
MG_Impl::GLImpl::GetFloatv(GL_MAX_FRAGMENT_INTERPOLATION_OFFSET, &floatValue);
EXPECT_FLOAT_EQ(floatValue, 0.4375f);
MG_Impl::GLImpl::GetFloatv(GL_FRAGMENT_INTERPOLATION_OFFSET_BITS, &floatValue);
EXPECT_FLOAT_EQ(floatValue, 6.0f);
GLint intValue = 0;
MG_Impl::GLImpl::GetIntegerv(GL_MIN_FRAGMENT_INTERPOLATION_OFFSET, &intValue);
EXPECT_EQ(intValue, -1);
MG_Impl::GLImpl::GetIntegerv(GL_MAX_FRAGMENT_INTERPOLATION_OFFSET, &intValue);
EXPECT_EQ(intValue, 0);
MG_Impl::GLImpl::GetIntegerv(GL_FRAGMENT_INTERPOLATION_OFFSET_BITS, &intValue);
EXPECT_EQ(intValue, 6);
GLboolean boolValue = GL_FALSE;
MG_Impl::GLImpl::GetBooleanv(GL_MIN_FRAGMENT_INTERPOLATION_OFFSET, &boolValue);
EXPECT_EQ(boolValue, GL_TRUE);
MG_Impl::GLImpl::GetBooleanv(GL_MAX_FRAGMENT_INTERPOLATION_OFFSET, &boolValue);
EXPECT_EQ(boolValue, GL_TRUE);
MG_Impl::GLImpl::GetBooleanv(GL_FRAGMENT_INTERPOLATION_OFFSET_BITS, &boolValue);
EXPECT_EQ(boolValue, GL_TRUE);
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
MG_Backend::pActiveBackendObject = Move(previousBackend);
MG_State::pGLContext = Move(previousContext);
}
TEST(GetterSanity, PerStageImageUniformQueriesMatchShaderCompilerLimits) {
using namespace MobileGL;
@@ -9,6 +9,7 @@
#include "Loader.h"
#include "MG_Util/Types.h"
#include <Config.h>
#include <cmath>
#if defined(_WIN32)
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN 1
@@ -838,8 +839,14 @@ namespace MobileGL::MG_Util::BackendLoader {
if (std::strcmp(extension, "GL_NV_shader_noperspective_interpolation") == 0) {
caps.SupportsNoperspectiveInterpolation = true;
}
if (std::strcmp(extension, "GL_OES_shader_multisample_interpolation") == 0) {
caps.SupportsShaderMultisampleInterpolation = true;
}
}
}
caps.SupportsShaderMultisampleInterpolation =
caps.SupportsShaderMultisampleInterpolation || caps.GLESVersion.Major > 3 ||
(caps.GLESVersion.Major == 3 && caps.GLESVersion.Minor >= 2);
// Detect optional raster/color-mask entry points by whether they loaded. glColorMaski is GLES
// 3.2 core (no extension string), so pointer presence is the reliable signal for all of these.
@@ -900,6 +907,9 @@ namespace MobileGL::MG_Util::BackendLoader {
GLint maxColorAttachments = 8;
GLint maxClipDistances = 8;
GLint maxViewports = 16;
GLfloat minFragmentInterpolationOffset = -0.5f;
GLfloat maxFragmentInterpolationOffset = 0.4375f;
GLint fragmentInterpolationOffsetBits = 4;
glesFuncs.glGetFloatv(GL_ALIASED_LINE_WIDTH_RANGE, aliasedLineWidthRange);
glesFuncs.glGetFloatv(GL_SMOOTH_LINE_WIDTH_RANGE, smoothLineWidthRange);
glesFuncs.glGetFloatv(GL_SMOOTH_LINE_WIDTH_GRANULARITY, &smoothLineWidthGranularity);
@@ -950,6 +960,29 @@ namespace MobileGL::MG_Util::BackendLoader {
glesFuncs.glGetIntegerv(GL_MAX_VIEWPORTS, &maxViewports);
glesFuncs.glGetIntegerv(GL_MAX_VIEWPORT_DIMS, maxViewportDims);
glesFuncs.glGetIntegerv(GL_VIEWPORT_SUBPIXEL_BITS, &viewportSubpixelBits);
if (caps.SupportsShaderMultisampleInterpolation && glesFuncs.glGetFloatv) {
const auto drainErrors = [&glesFuncs]() {
Bool hadError = false;
if (glesFuncs.glGetError) {
while (glesFuncs.glGetError() != GL_NO_ERROR) hadError = true;
}
return hadError;
};
// Isolate these optional queries from errors raised by preceding capability
// probes, then consume any query error so initialization never leaks it into
// the application's first glGetError call.
drainErrors();
glesFuncs.glGetFloatv(GL_MIN_FRAGMENT_INTERPOLATION_OFFSET, &minFragmentInterpolationOffset);
glesFuncs.glGetFloatv(GL_MAX_FRAGMENT_INTERPOLATION_OFFSET, &maxFragmentInterpolationOffset);
glesFuncs.glGetIntegerv(GL_FRAGMENT_INTERPOLATION_OFFSET_BITS, &fragmentInterpolationOffsetBits);
if (drainErrors()) {
MGLOG_W("Fragment interpolation limit query failed; using OpenGL minimums");
minFragmentInterpolationOffset = -0.5f;
maxFragmentInterpolationOffset = 0.4375f;
fragmentInterpolationOffsetBits = 4;
}
}
// Only legal to query once the extension has been seen in the loop above, hence not batched
// with the unconditional probes: on a driver without it this raises GL_INVALID_ENUM.
if (caps.SupportsTextureFilterAnisotropy) {
@@ -1007,6 +1040,19 @@ namespace MobileGL::MG_Util::BackendLoader {
caps.ViewportBoundsRangeMin = viewportBoundsRange[0];
caps.ViewportBoundsRangeMax = viewportBoundsRange[1];
caps.ViewportSubpixelBits = viewportSubpixelBits;
caps.MinFragmentInterpolationOffset =
std::isfinite(minFragmentInterpolationOffset) && minFragmentInterpolationOffset <= -0.5f
? minFragmentInterpolationOffset
: -0.5f;
caps.MaxFragmentInterpolationOffset = 0.4375f;
caps.FragmentInterpolationOffsetBits = 4;
if (fragmentInterpolationOffsetBits >= 4 && std::isfinite(maxFragmentInterpolationOffset)) {
const Float requiredMaxOffset = 0.5f - std::ldexp(1.0f, -fragmentInterpolationOffsetBits);
if (maxFragmentInterpolationOffset >= requiredMaxOffset) {
caps.MaxFragmentInterpolationOffset = maxFragmentInterpolationOffset;
caps.FragmentInterpolationOffsetBits = fragmentInterpolationOffsetBits;
}
}
MGLOG_I(" GL_ALIASED_LINE_WIDTH_RANGE: [%.3f, %.3f]", caps.AliasedLineWidthRangeMin,
caps.AliasedLineWidthRangeMax);
MGLOG_I(" GL_SMOOTH_LINE_WIDTH_RANGE: [%.3f, %.3f]", caps.SmoothLineWidthRangeMin,
@@ -1055,6 +1055,9 @@ namespace MobileGL {
// SPIRV-Cross's `#extension ... : require` would fail to compile and MobileGL falls back
// to stripping the NoPerspective decoration (smooth interpolation) via StripNoPerspectivePass.
Bool SupportsNoperspectiveInterpolation = false;
// GLES 3.2 core or GL_OES_shader_multisample_interpolation exposes
// interpolateAtOffset and the three fragment-offset limit queries.
Bool SupportsShaderMultisampleInterpolation = false;
// GL_RENDERER contains "ANGLE".
Bool IsAngleRenderer = false;
// GL_RENDERER contains both "ANGLE" and "llvmpipe".
@@ -1120,6 +1123,9 @@ namespace MobileGL {
Float ViewportBoundsRangeMin = 0.0f;
Float ViewportBoundsRangeMax = 0.0f;
Int ViewportSubpixelBits = 0;
Float MinFragmentInterpolationOffset = -0.5f;
Float MaxFragmentInterpolationOffset = 0.4375f;
Int FragmentInterpolationOffsetBits = 4;
};
} // namespace MG_External
@@ -9,6 +9,7 @@
#include "Loader.h"
#include <Config.h>
#include <cmath>
namespace MobileGL::MG_Util::BackendLoader {
namespace {
@@ -40,6 +41,25 @@ namespace MobileGL::MG_Util::BackendLoader {
return MaxSampleCountFromFlags(commonFlags);
}
void FillFragmentInterpolationLimits(MG_External::VulkanCapabilities& caps,
const VkPhysicalDeviceLimits& limits) {
caps.MinFragmentInterpolationOffset =
std::isfinite(limits.minInterpolationOffset) && limits.minInterpolationOffset <= -0.5f
? limits.minInterpolationOffset
: -0.5f;
caps.MaxFragmentInterpolationOffset = 0.4375f;
caps.FragmentInterpolationOffsetBits = 4;
const Int bits = static_cast<Int>(limits.subPixelInterpolationOffsetBits);
if (bits >= 4 && std::isfinite(limits.maxInterpolationOffset)) {
const Float requiredMaxOffset = 0.5f - std::ldexp(1.0f, -bits);
if (limits.maxInterpolationOffset >= requiredMaxOffset) {
caps.MaxFragmentInterpolationOffset = limits.maxInterpolationOffset;
caps.FragmentInterpolationOffsetBits = bits;
}
}
}
VulkanDynamicFunctions LoadVulkanDynamicFunctions(VkInstance instance) {
VulkanDynamicFunctions loaded{};
if (instance == VK_NULL_HANDLE) {
@@ -171,6 +191,7 @@ namespace MobileGL::MG_Util::BackendLoader {
caps.ViewportBoundsRangeMin = p.limits.viewportBoundsRange[0];
caps.ViewportBoundsRangeMax = p.limits.viewportBoundsRange[1];
caps.ViewportSubpixelBits = static_cast<Int>(p.limits.viewportSubPixelBits);
FillFragmentInterpolationLimits(caps, p.limits);
VkPhysicalDeviceFeatures supportedFeatures{};
vkGetPhysicalDeviceFeatures(physicalDevice, &supportedFeatures);
@@ -261,6 +282,7 @@ namespace MobileGL::MG_Util::BackendLoader {
caps.ViewportBoundsRangeMin = properties.limits.viewportBoundsRange[0];
caps.ViewportBoundsRangeMax = properties.limits.viewportBoundsRange[1];
caps.ViewportSubpixelBits = static_cast<Int>(properties.limits.viewportSubPixelBits);
FillFragmentInterpolationLimits(caps, properties.limits);
caps.SupportsWideLines = false;
// This helper only receives properties, not VkPhysicalDeviceFeatures. Leave optional
// stage writes disabled rather than inferring them from descriptor limits alone.
@@ -68,6 +68,9 @@ namespace MobileGL {
Float ViewportBoundsRangeMin = 0.0f;
Float ViewportBoundsRangeMax = 0.0f;
Int ViewportSubpixelBits = 0;
Float MinFragmentInterpolationOffset = -0.5f;
Float MaxFragmentInterpolationOffset = 0.4375f;
Int FragmentInterpolationOffsetBits = 4;
Bool SupportsWideLines = false;
// Storage-image descriptors are limited per stage by
// maxPerStageDescriptorStorageImages, but writes/atomics outside compute additionally
@@ -1,78 +0,0 @@
# MobileGL POST Format Capability Tables
## Goal
Expose the format-capability results used during MobileGL backend startup in the Android plugin's driver POST screen. The screen must show the exact `Full`, `Caveat`, or `None` result for every backend, target, internal format, and capability without duplicating the backend's detection rules.
## Existing Architecture
- `DriverPost.cpp` probes the device GLES and Vulkan drivers before `MobileGL::Initialize()` and returns a `BackendPostReport` for each backend.
- `DriverPostJni.cpp` serializes those reports to JSON for `PostActivity`.
- `PostActivity` uses platform Android views and already supports collapsible check details and a collapsible raw report.
- Backend startup fills a `FormatCapabilityCache` in the DirectGLES and DirectVulkan `InitCapabilities()` paths. `FullCaps` takes precedence over `CaveatCaps`; an absent bit means `None`.
- The capability matrix contains 12 targets, 75 internal formats, and 14 capability columns per backend.
## Selected Approach
Extract callable format-probe entry points from the existing DirectGLES and DirectVulkan implementations. Backend startup and the POST will call these same functions, so their results cannot drift.
The POST will run each probe while its temporary driver resources are still valid:
- DirectGLES: after the GLES function table and capabilities have been populated, while the 1x1 pbuffer context is current.
- DirectVulkan: after selecting the physical device, while the Vulkan instance and physical device handles remain valid.
The resulting optional `FormatCapabilityCache` will be stored in each `BackendPostReport`. Failure to obtain a format table will not discard the existing POST checks or change their verdict; the UI will instead report that the format table is unavailable.
## JSON Contract
The JNI report will add an optional `formatCapabilities` object to each backend. To avoid repeating tens of thousands of status strings, the object will contain:
- one ordered capability-name array;
- one entry per target;
- one compact row per internal format containing the format name, a Full bitmask, and a Caveat bitmask.
Java resolves each cell in this order:
1. Full bit present: `Full`.
2. Otherwise Caveat bit present: `Caveat`.
3. Otherwise: `None`.
This preserves the backend's current precedence and keeps the raw JSON reasonably small.
## Android UI
Each backend section keeps its existing verdict, renderer string, and check table. A new `Format capabilities` subsection follows it.
- Each of the 11 texture targets and `Renderbuffer` is a separate, initially collapsed table.
- Target headers can be expanded independently.
- Table content is created on expansion and removed when collapsed, preventing the activity from retaining roughly 27,000 status views.
- Each expanded table is placed in a horizontal scroll container.
- The first column contains internal-format names. The remaining columns use the ordered capability names from the JSON report.
- Every status cell displays its status text and uses the conventional color mapping:
- `Full`: green background with white text.
- `Caveat`: yellow background with black text.
- `None`: red background with white text.
- Header and format-name cells use neutral dark backgrounds consistent with the existing POST theme.
- The existing raw-report toggle remains available at the end of the screen.
## Performance and Lifecycle
- The existing single-flight native POST and cached JSON behavior remains unchanged.
- Format tables are lazily materialized and discarded on collapse.
- The JSON carries bitmasks rather than repeated `Full`, `Caveat`, and `None` strings.
- Existing configuration-change handling remains unchanged.
## Validation
1. Run focused source checks and `git diff --check`.
2. Build the Android plugin APK with the repository's current Gradle workflow.
3. If an Android target is connected, install the APK and open `PostActivity`.
4. Verify both backend sections, all target toggles, horizontal scrolling, visible cell text, and the green/yellow/red mapping.
5. Confirm collapsing a table removes its generated content and expanding it recreates the same values.
## Non-Goals
- Changing the meanings of `Full`, `Caveat`, or `None`.
- Changing POST verdict rules.
- Displaying sample-count vectors in this iteration.
- Replacing the existing platform-view UI with Compose, AppCompat, or WebView.
+28 -1
View File
@@ -1,4 +1,31 @@
# Running the OpenGL CTS (VK-GL-CTS / KHR-GL33) against MobileGL on Android
# Running the OpenGL CTS against MobileGL
This directory contains two supported paths:
- Android arm64 / MobileGL EGL: the KHR-GL33 workflow documented below and in
`skills/gl-cts-on-mobilegl/SKILL.md`.
- Windows x64 / MobileGL WGL: the GL30-GL46 pipeline in
`scripts/wgl_glcts_pipeline.py`, documented by
`skills/wgl-gl-cts-on-mobilegl/SKILL.md`.
Windows prerequisites are Git, Python 3.9+, CMake, Visual Studio 2022's Desktop
C++ workload, and a Vulkan SDK visible to CMake. DirectVulkan also needs a
working Vulkan loader plus a GPU-vendor ICD and driver; the SDK alone is not a
GPU driver.
For Windows, start with:
```powershell
python tools\cts\scripts\wgl_glcts_pipeline.py --help
```
The pipeline builds MobileGL as a drop-in `opengl32.dll`, builds or reuses
`glcts.exe`, checks that WGL loaded MobileGL rather than the system driver,
resumes individual suites after crashes/timeouts, and writes Markdown plus JSON
reports below the printed `runs/<first-16-of-run-fingerprint>` directory. Its
manifest records provenance and the runner settings used to validate a resume.
## Android KHR-GL33 workflow
Goal: measure how much of the OpenGL 3.3 core-profile conformance suite MobileGL
passes, separately for each backend (`DirectGLES`, `DirectVulkan`).
+545
View File
@@ -0,0 +1,545 @@
#!/usr/bin/env python
"""Build a GL 3.0--3.3 CTS conformance matrix from dEQP QPA logs.
The report deliberately scores against the unique cases in each supplied
caselist. A case that has not produced a result therefore cannot disappear
from the denominator and make a partial run look conformant.
QPA parsing and crash/hang sidecar handling follow :mod:`qpa_report`:
* a later QPA observation of a case wins;
* ``crashed.txt`` upgrades a missing/incomplete result to ``Crash``;
* ``hung.txt`` upgrades a missing/incomplete/crash result to ``DeviceHang``.
Example::
python cts_matrix_report.py \
--gl30-caselist gl30-main.txt --gl30-results runs/gl30 \
--gl31-caselist gl31-main.txt --gl31-results runs/gl31 \
--gl32-caselist gl32-main.txt --gl32-results runs/gl32 \
--gl33-caselist gl33-main.txt --gl33-results runs/gl33 \
--json runs/cts-matrix.json
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
from collections import Counter, defaultdict
from datetime import datetime, timezone
from typing import Iterable, Optional, Sequence
try: # Works both as a directly executed script and as a package import.
from . import qpa_report
except ImportError: # pragma: no cover - exercised by the command-line tests
import qpa_report
VERSIONS = ("gl30", "gl31", "gl32", "gl33")
ACCEPTED_STATUSES = (
"Pass",
"NotSupported",
"QualityWarning",
"CompatibilityWarning",
"Waiver",
)
ACCEPTED = frozenset(ACCEPTED_STATUSES)
CHUNK_QPA = re.compile(r"^chunk(\d+)\.qpa$", re.IGNORECASE)
class ReportInputError(ValueError):
"""An input path cannot be used to construct a meaningful report."""
def _read_non_comment_lines(path: str) -> list[str]:
try:
# Match run_cts_windows.py: Khronos lists are UTF-8 and may carry a BOM.
with open(path, "r", encoding="utf-8-sig", errors="strict") as fh:
return [
line.strip()
for line in fh
if line.strip() and not line.lstrip().startswith("#")
]
except (OSError, UnicodeError) as exc:
raise ReportInputError(f"cannot read {path}: {exc}") from exc
def read_caselist(path: str) -> tuple[list[str], dict[str, int]]:
"""Return unique cases in file order and repeated caselist entries.
The mustpass files consumed by glcts and ``run_cts.py`` are one case per
non-empty, non-comment line, so this intentionally uses the same syntax.
"""
entries = _read_non_comment_lines(path)
counts = Counter(entries)
unique = list(dict.fromkeys(entries))
duplicates = {case: count for case, count in counts.items() if count > 1}
return unique, duplicates
def _collect_qpa_files(paths: Sequence[str]) -> list[str]:
missing = [path for path in paths if not os.path.exists(path)]
if missing:
raise ReportInputError(
"result path(s) do not exist: " + ", ".join(sorted(missing))
)
# qpa_report.collect provides the established directory-recursion rules.
# De-duplicate aliases so specifying the same directory twice does not
# manufacture duplicate observations.
files = qpa_report.collect(paths)
by_identity: dict[str, str] = {}
for path in files:
if not os.path.isfile(path):
raise ReportInputError(f"QPA input is not a file: {path}")
absolute = os.path.abspath(path)
by_identity.setdefault(os.path.normcase(absolute), absolute)
def order_key(value: str) -> tuple[str, str, int, str]:
absolute = os.path.abspath(value)
directory = os.path.normcase(os.path.dirname(absolute))
filename = os.path.normcase(os.path.basename(absolute))
match = CHUNK_QPA.fullmatch(filename)
if match:
# run_cts_windows.py uses a minimum width of four digits, not a
# fixed width. Numeric ordering is therefore required once a run
# reaches chunk10000; lexical ordering would put it before
# chunk9999 and break the later-observation-wins rule.
return directory, "chunk", int(match.group(1)), filename
# Preserve a deterministic, name-based position for foreign/legacy
# QPA files while grouping numeric runner chunks at the lexical
# position occupied by the "chunk" basename.
return directory, filename, -1, filename
return sorted(by_identity.values(), key=order_key)
def _ratio(numerator: int, denominator: int) -> float:
return numerator / denominator if denominator else 0.0
def _sidecar(paths: Sequence[str], name: str) -> set[str]:
"""Load a run_cts.py sidecar with qpa_report-compatible lookup rules."""
return qpa_report.load_sidecar(paths, name)
def build_version_report(
version: str, caselist: str, result_paths: Sequence[str]
) -> dict:
"""Build the serialisable report for one GL mustpass version."""
expected_cases, expected_duplicates = read_caselist(caselist)
expected = set(expected_cases)
qpa_files = _collect_qpa_files(result_paths)
results: dict[str, str] = {}
observation_history: dict[str, list[dict[str, str]]] = defaultdict(list)
for qpa_file in qpa_files:
for case, status in qpa_report.parse_qpa(qpa_file):
observation_history[case].append(
{"file": qpa_file, "status": status}
)
results[case] = status
crashed = _sidecar(result_paths, "crashed.txt")
hung = _sidecar(result_paths, "hung.txt")
explicit_unrun = _sidecar(result_paths, "unrun.txt")
skipped = _sidecar(result_paths, "skipped.txt")
# Keep this order and these guards in lock-step with qpa_report.py.
for case in crashed:
if results.get(case, "Incomplete") == "Incomplete":
results[case] = "Crash"
for case in hung:
if results.get(case, "Incomplete") in ("Incomplete", "Crash"):
results[case] = "DeviceHang"
# A begin/end pair without <Result>, or a QPA truncated mid-case, is not a
# completed observation. Sidecars above may upgrade it to Crash/Hang;
# anything still Incomplete must stay in the expected denominator as unrun.
incomplete_results = {
case for case, status in results.items() if status == "Incomplete"
}
for case in incomplete_results:
del results[case]
expected_results = {
case: status for case, status in results.items() if case in expected
}
unexpected_results = {
case: status for case, status in results.items() if case not in expected
}
# Missing cases are inferred from the caselist even if unrun.txt itself is
# missing or stale. This is the invariant that prevents partial-run rate
# inflation.
unrun_cases = expected - set(expected_results)
declared_not_measured = explicit_unrun | skipped
undeclared_unrun = unrun_cases - declared_not_measured
stale_unrun = (explicit_unrun | skipped) & set(expected_results)
counts = Counter(expected_results.values())
strict_pass = counts["Pass"]
accepted = sum(counts[status] for status in ACCEPTED)
result_count = len(expected_results)
expected_count = len(expected)
crash_count = counts["Crash"]
hang_count = counts["DeviceHang"]
duplicate_cases = {
case: {
"observations": len(history),
"extra_observations": len(history) - 1,
"final_status": results.get(case, "Incomplete"),
"history": history,
}
for case, history in sorted(observation_history.items())
if len(history) > 1
}
duplicate_observations = sum(
item["extra_observations"] for item in duplicate_cases.values()
)
sidecar_unknown = {
name: sorted(cases - expected)
for name, cases in (
("crashed.txt", crashed),
("hung.txt", hung),
("unrun.txt", explicit_unrun),
("skipped.txt", skipped),
)
if cases - expected
}
errors: list[str] = []
warnings: list[str] = []
if not expected_count:
errors.append("caselist has no cases")
if expected_duplicates:
errors.append(
f"caselist has {sum(n - 1 for n in expected_duplicates.values())} "
"duplicate entry/entries"
)
if not qpa_files:
errors.append("no .qpa files found")
if unexpected_results:
errors.append(
f"{len(unexpected_results)} result case(s) are absent from the caselist"
)
if sidecar_unknown:
errors.append("one or more sidecars name cases absent from the caselist")
if undeclared_unrun:
errors.append(
f"{len(undeclared_unrun)} missing result case(s) are not declared by "
"unrun.txt/skipped.txt"
)
if stale_unrun:
warnings.append(
f"{len(stale_unrun)} case(s) declared unrun/skipped also have a result"
)
if duplicate_observations:
warnings.append(
f"{duplicate_observations} duplicate QPA observation(s); last result wins"
)
if incomplete_results:
warnings.append(
f"{len(incomplete_results)} QPA case(s) ended without a final result and were treated as unrun"
)
if errors:
state = "ERROR"
elif unrun_cases:
state = "INCOMPLETE"
else:
state = "OK"
return {
"version": version,
"inputs": {
"caselist": os.path.abspath(caselist),
"result_paths": [os.path.abspath(path) for path in result_paths],
"qpa_files": qpa_files,
},
"expected": expected_count,
"result": result_count,
"pass": strict_pass,
"accepted": accepted,
"crash": crash_count,
"hang": hang_count,
"unrun": len(unrun_cases),
"duplicate": duplicate_observations,
"counts": dict(sorted(counts.items())),
"coverage": {
"numerator": result_count,
"denominator": expected_count,
"rate": _ratio(result_count, expected_count),
},
"rates": {
# These are the report's conformance rates. Expected, not merely
# measured results, is the denominator.
"denominator": "expected",
"strict_pass_only": _ratio(strict_pass, expected_count),
"conformance_accepted": _ratio(accepted, expected_count),
# Useful for comparison with qpa_report.py, whose denominator is
# cases with a result. Never presented as the conformance rate.
"measured_only_strict_pass": _ratio(strict_pass, result_count),
"measured_only_conformance_accepted": _ratio(
accepted, result_count
),
},
"strict_pass_rate": _ratio(strict_pass, expected_count),
"conformance_accepted_rate": _ratio(accepted, expected_count),
"validation": {
"state": state,
"ok": state == "OK",
"errors": errors,
"warnings": warnings,
"invariant_expected_equals_result_plus_unrun": (
expected_count == result_count + len(unrun_cases)
),
"undeclared_unrun": sorted(undeclared_unrun),
"stale_unrun_or_skipped": sorted(stale_unrun),
"sidecar_cases_absent_from_caselist": sidecar_unknown,
},
"cases": {
"results": dict(sorted(expected_results.items())),
"unrun": sorted(unrun_cases),
"unexpected_results": dict(sorted(unexpected_results.items())),
"incomplete_results": sorted(incomplete_results),
"duplicate_results": duplicate_cases,
"duplicate_caselist_entries": dict(sorted(expected_duplicates.items())),
},
}
def build_matrix(suites: dict[str, tuple[str, Sequence[str]]]) -> dict:
"""Build all four version reports and their case-weighted aggregate."""
version_reports = {
version: build_version_report(version, *suites[version])
for version in VERSIONS
}
totals = {
key: sum(report[key] for report in version_reports.values())
for key in (
"expected",
"result",
"pass",
"accepted",
"crash",
"hang",
"unrun",
"duplicate",
)
}
status_counts: Counter[str] = Counter()
for report in version_reports.values():
status_counts.update(report["counts"])
states = {report["validation"]["state"] for report in version_reports.values()}
if "ERROR" in states:
overall_state = "ERROR"
elif "INCOMPLETE" in states:
overall_state = "INCOMPLETE"
else:
overall_state = "OK"
overall = {
**totals,
"counts": dict(sorted(status_counts.items())),
"aggregation": "weighted_by_expected_cases",
"coverage": {
"numerator": totals["result"],
"denominator": totals["expected"],
"rate": _ratio(totals["result"], totals["expected"]),
},
"rates": {
"denominator": "expected",
"strict_pass_only": _ratio(totals["pass"], totals["expected"]),
"conformance_accepted": _ratio(
totals["accepted"], totals["expected"]
),
"measured_only_strict_pass": _ratio(
totals["pass"], totals["result"]
),
"measured_only_conformance_accepted": _ratio(
totals["accepted"], totals["result"]
),
},
"strict_pass_rate": _ratio(totals["pass"], totals["expected"]),
"conformance_accepted_rate": _ratio(
totals["accepted"], totals["expected"]
),
"validation": {
"state": overall_state,
"ok": overall_state == "OK",
"invariant_expected_equals_result_plus_unrun": (
totals["expected"] == totals["result"] + totals["unrun"]
),
},
}
return {
"schema_version": 1,
"generated_at": datetime.now(timezone.utc).isoformat(),
"accepted_statuses": list(ACCEPTED_STATUSES),
"rate_policy": {
"denominator": "unique expected cases from each caselist",
"unrun_cases": "included in the denominator and never accepted",
"duplicate_results": "last QPA result wins, matching qpa_report.py",
},
"versions": version_reports,
"overall": overall,
}
def _percent(numerator: int, denominator: int) -> str:
if not denominator:
return "n/a"
return f"{100.0 * numerator / denominator:.2f}% ({numerator}/{denominator})"
def render_markdown(report: dict) -> str:
"""Render the compact terminal-facing conformance table."""
header = (
"| Suite | Expected | Result | Pass | Accepted | Crash | Hang | Unrun | "
"Duplicate | Coverage | Strict Pass-only | Conformance-accepted | Validation |"
)
separator = (
"|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|:---:|"
)
rows = [header, separator]
for version in VERSIONS:
item = report["versions"][version]
rows.append(
"| {version} | {expected} | {result} | {pass_count} | {accepted} | "
"{crash} | {hang} | {unrun} | {duplicate} | {coverage} | {strict} | "
"{accepted_rate} | {state} |".format(
version=version.upper(),
expected=item["expected"],
result=item["result"],
pass_count=item["pass"],
accepted=item["accepted"],
crash=item["crash"],
hang=item["hang"],
unrun=item["unrun"],
duplicate=item["duplicate"],
coverage=_percent(item["result"], item["expected"]),
strict=_percent(item["pass"], item["expected"]),
accepted_rate=_percent(item["accepted"], item["expected"]),
state=item["validation"]["state"],
)
)
overall = report["overall"]
rows.append(
"| **Overall (weighted)** | **{expected}** | **{result}** | **{pass_count}** | "
"**{accepted}** | **{crash}** | **{hang}** | **{unrun}** | **{duplicate}** | "
"**{coverage}** | **{strict}** | **{accepted_rate}** | **{state}** |".format(
expected=overall["expected"],
result=overall["result"],
pass_count=overall["pass"],
accepted=overall["accepted"],
crash=overall["crash"],
hang=overall["hang"],
unrun=overall["unrun"],
duplicate=overall["duplicate"],
coverage=_percent(overall["result"], overall["expected"]),
strict=_percent(overall["pass"], overall["expected"]),
accepted_rate=_percent(overall["accepted"], overall["expected"]),
state=overall["validation"]["state"],
)
)
rows.extend(
(
"",
"Rates use unique **Expected** caselist cases as the denominator; unrun cases "
"remain in that denominator and are not accepted.",
"Accepted statuses: " + ", ".join(f"`{s}`" for s in ACCEPTED_STATUSES) + ".",
"Duplicate is the number of extra QPA observations; the last observation wins.",
)
)
details: list[str] = []
for version in VERSIONS:
validation = report["versions"][version]["validation"]
messages = validation["errors"] + validation["warnings"]
if messages:
details.append(
f"- **{version.upper()} {validation['state']}**: " + "; ".join(messages)
)
if details:
rows.extend(("", "Validation details:", "", *details))
return "\n".join(rows)
def _write_json(path: str, report: dict) -> None:
parent = os.path.dirname(os.path.abspath(path))
os.makedirs(parent, exist_ok=True)
with open(path, "w", encoding="utf-8", newline="\n") as fh:
json.dump(report, fh, indent=2, sort_keys=True)
fh.write("\n")
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
for version in VERSIONS:
parser.add_argument(
f"--{version}-caselist",
f"--{version}-case-list",
required=True,
help=f"{version.upper()} mustpass caselist",
)
parser.add_argument(
f"--{version}-results",
f"--{version}-result-dir",
f"--{version}-results-dir",
action="append",
required=True,
help=f"{version.upper()} result directory or QPA file (repeatable)",
)
parser.add_argument(
"--json",
dest="json_out",
default="cts_matrix_report.json",
help="JSON output path (default: ./cts_matrix_report.json)",
)
parser.add_argument(
"--allow-incomplete",
action="store_true",
help="return success even when validation is ERROR/INCOMPLETE",
)
return parser
def main(argv: Optional[Iterable[str]] = None) -> int:
args = _parser().parse_args(argv)
suites = {
version: (
getattr(args, f"{version}_caselist"),
getattr(args, f"{version}_results"),
)
for version in VERSIONS
}
try:
report = build_matrix(suites)
_write_json(args.json_out, report)
except (OSError, ReportInputError) as exc:
print(f"cts_matrix_report: {exc}", file=sys.stderr)
return 2
print(render_markdown(report))
print(f"\nJSON: {os.path.abspath(args.json_out)}")
if not args.allow_incomplete and not report["overall"]["validation"]["ok"]:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+540
View File
@@ -0,0 +1,540 @@
#!/usr/bin/env python
"""Summarise any number of GL CTS suites and MobileGL backends.
Each repeatable suite specification consists of four values: backend, label,
caselist, and result directory. For example::
python cts_multi_report.py \
--suite DirectGLES gl30 gl30-main.txt runs/gles/gl30 \
--suite DirectVulkan gl30 gl30-main.txt runs/vulkan/gl30 \
--markdown cts-summary.md --json cts-summary.json
A compact comma form is accepted as well::
--suite=DirectGLES,gl31,gl31-main.txt,runs/gles/gl31
Per-suite parsing and validation deliberately delegate to
``cts_matrix_report`` so QPA ordering, sidecar upgrades, accepted statuses,
unrun handling, and duplicate-result semantics cannot drift between reports.
All conformance rates use unique expected caselist cases as their denominator.
Backend subtotals and the overall total are therefore case-weighted, not an
unweighted average of suite percentages.
"""
from __future__ import annotations
import argparse
from collections import Counter
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
import sys
from typing import Iterable, Optional, Sequence
try: # Direct script execution and package imports are both supported.
from . import cts_matrix_report
except ImportError: # pragma: no cover - covered through CLI-style tests
import cts_matrix_report
SUPPORTED_BACKENDS = ("DirectGLES", "DirectVulkan")
SUM_FIELDS = (
"expected",
"result",
"pass",
"accepted",
"crash",
"hang",
"unrun",
"duplicate",
)
class MultiReportInputError(ValueError):
"""Suite specifications cannot produce an unambiguous report."""
@dataclass(frozen=True)
class SuiteSpec:
backend: str
label: str
caselist: str
result_dir: str
@property
def suite_id(self) -> str:
return f"{self.backend}/{self.label}"
def _ratio(numerator: int, denominator: int) -> float:
return numerator / denominator if denominator else 0.0
def _validation_state(items: Sequence[dict]) -> str:
states = {item["validation"]["state"] for item in items}
if "ERROR" in states:
return "ERROR"
if "INCOMPLETE" in states:
return "INCOMPLETE"
return "OK"
def aggregate_reports(items: Sequence[dict], suite_state_keys: Sequence[str]) -> dict:
"""Return an expected-case-weighted aggregate for suite reports."""
if len(items) != len(suite_state_keys):
raise MultiReportInputError("internal suite/state key count mismatch")
totals = {
field: sum(int(item[field]) for item in items)
for field in SUM_FIELDS
}
status_counts: Counter[str] = Counter()
for item in items:
status_counts.update(item["counts"])
state = _validation_state(items)
expected = totals["expected"]
result = totals["result"]
suite_states = {
key: item["validation"]["state"]
for key, item in zip(suite_state_keys, items)
}
return {
**totals,
"suite_count": len(items),
"counts": dict(sorted(status_counts.items())),
"aggregation": "weighted_by_expected_cases",
"coverage": {
"numerator": result,
"denominator": expected,
"rate": _ratio(result, expected),
},
"rates": {
"denominator": "expected",
"strict_pass_only": _ratio(totals["pass"], expected),
"conformance_accepted": _ratio(totals["accepted"], expected),
"measured_only_strict_pass": _ratio(totals["pass"], result),
"measured_only_conformance_accepted": _ratio(
totals["accepted"], result
),
},
# Keep the convenient aliases used by cts_matrix_report consumers.
"strict_pass_rate": _ratio(totals["pass"], expected),
"conformance_accepted_rate": _ratio(totals["accepted"], expected),
"validation": {
"state": state,
"ok": state == "OK",
"suite_states": suite_states,
"invariant_expected_equals_result_plus_unrun": (
expected == result + totals["unrun"]
),
},
}
def _caselist_fingerprint(path: str) -> tuple[str, int]:
cases, _duplicates = cts_matrix_report.read_caselist(path)
payload = "\n".join(cases).encode("utf-8") + b"\n"
return hashlib.sha256(payload).hexdigest(), len(cases)
def _read_provenance(
spec: SuiteSpec,
require_run_state: bool,
expected_run_identity: Optional[str],
) -> dict:
path = os.path.join(spec.result_dir, "run_state.json")
if not os.path.isfile(path):
if require_run_state or expected_run_identity is not None:
raise MultiReportInputError(
"suite result directory has no run_state.json; invocation provenance "
f"cannot be verified: {spec.result_dir}"
)
return {"state": "UNVERIFIED", "run_state": None}
try:
with open(path, "r", encoding="utf-8") as handle:
state = json.load(handle)
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise MultiReportInputError(f"cannot read suite run identity {path}: {exc}") from exc
if not isinstance(state, dict):
raise MultiReportInputError(f"suite run identity must be a JSON object: {path}")
if state.get("backend") != spec.backend:
raise MultiReportInputError(
f"suite {spec.suite_id} is labelled {spec.backend}, but run_state.json "
f"records {state.get('backend')!r}"
)
fingerprint, case_count = _caselist_fingerprint(spec.caselist)
if state.get("caselist_sha256") != fingerprint or state.get("case_count") != case_count:
raise MultiReportInputError(
f"suite {spec.suite_id} run_state.json belongs to a different caselist"
)
invocation_identity = state.get("invocation_identity")
if (
expected_run_identity is not None
and invocation_identity != expected_run_identity
):
raise MultiReportInputError(
f"suite {spec.suite_id} run_state.json belongs to a different CTS invocation"
)
return {
"state": "VERIFIED",
"run_state": os.path.abspath(path),
"invocation_identity": invocation_identity,
}
def _validate_specs(
specs: Sequence[SuiteSpec],
require_run_state: bool,
expected_run_identity: Optional[str],
) -> dict[str, dict]:
if not specs:
raise MultiReportInputError("at least one --suite specification is required")
seen: set[tuple[str, str]] = set()
seen_result_dirs: list[tuple[str, str]] = []
provenance: dict[str, dict] = {}
for spec in specs:
if spec.backend not in SUPPORTED_BACKENDS:
raise MultiReportInputError(
f"unsupported backend {spec.backend!r}; expected one of "
+ ", ".join(SUPPORTED_BACKENDS)
)
if not spec.label.strip():
raise MultiReportInputError("suite label cannot be empty")
identity = (spec.backend, spec.label)
if identity in seen:
raise MultiReportInputError(
f"duplicate suite specification for {spec.backend}/{spec.label}"
)
seen.add(identity)
if not os.path.isdir(spec.result_dir):
raise MultiReportInputError(
f"suite result directory does not exist: {spec.result_dir}"
)
physical_result_dir = os.path.normcase(
os.path.realpath(os.path.abspath(spec.result_dir))
)
for previous_dir, previous_suite in seen_result_dirs:
try:
common_dir = os.path.commonpath(
[previous_dir, physical_result_dir]
)
except ValueError:
continue
if common_dir in (previous_dir, physical_result_dir):
raise MultiReportInputError(
f"suite {spec.suite_id} uses a result directory which overlaps "
f"{previous_suite}: {spec.result_dir}"
)
seen_result_dirs.append((physical_result_dir, spec.suite_id))
provenance[spec.suite_id] = _read_provenance(
spec, require_run_state, expected_run_identity
)
return provenance
def build_report(
specs: Sequence[SuiteSpec],
require_run_state: bool = True,
expected_run_identity: Optional[str] = None,
) -> dict:
"""Build suite, per-backend, and overall serialisable reports."""
provenance = _validate_specs(
specs, require_run_state, expected_run_identity
)
suite_reports: list[dict] = []
backend_order: list[str] = []
for spec in specs:
if spec.backend not in backend_order:
backend_order.append(spec.backend)
item = cts_matrix_report.build_version_report(
spec.label, spec.caselist, [spec.result_dir]
)
# ``version`` is the generic label argument in build_version_report;
# expose explicit multi-report terminology while retaining all of its
# validation and case-level evidence.
item.pop("version", None)
item["backend"] = spec.backend
item["label"] = spec.label
item["suite_id"] = spec.suite_id
item["provenance"] = provenance[spec.suite_id]
if item["provenance"]["state"] == "UNVERIFIED":
item["validation"]["warnings"].append(
"result directory has no run_state.json; backend provenance is unverified"
)
suite_reports.append(item)
backends: dict[str, dict] = {}
for backend in backend_order:
backend_items = [
item for item in suite_reports if item["backend"] == backend
]
labels = [item["label"] for item in backend_items]
aggregate = aggregate_reports(backend_items, labels)
aggregate["backend"] = backend
aggregate["suite_labels"] = labels
backends[backend] = aggregate
overall = aggregate_reports(
suite_reports, [item["suite_id"] for item in suite_reports]
)
overall["backend_count"] = len(backends)
overall["backends"] = backend_order
return {
"schema_version": 1,
"generated_at": datetime.now(timezone.utc).isoformat(),
"accepted_statuses": list(cts_matrix_report.ACCEPTED_STATUSES),
"rate_policy": {
"denominator": "unique expected cases from each suite caselist",
"unrun_cases": "included in the denominator and never accepted",
"backend_aggregation": "weighted by expected cases",
"overall_aggregation": "weighted by expected cases across backend-suite pairs",
"duplicate_results": "last QPA result wins, matching qpa_report.py",
},
"suites": suite_reports,
"backends": backends,
"overall": overall,
}
def _percent(numerator: int, denominator: int) -> str:
if not denominator:
return "n/a"
return f"{100.0 * numerator / denominator:.2f}% ({numerator}/{denominator})"
def _markdown_cell(value: object) -> str:
return str(value).replace("|", r"\|").replace("\r", " ").replace("\n", " ")
def _table_row(backend: str, label: str, item: dict, bold: bool = False) -> str:
values = [
backend,
label,
str(item["expected"]),
str(item["result"]),
str(item["pass"]),
str(item["accepted"]),
str(item["crash"]),
str(item["hang"]),
str(item["unrun"]),
str(item["duplicate"]),
_percent(item["result"], item["expected"]),
_percent(item["pass"], item["expected"]),
_percent(item["accepted"], item["expected"]),
item["validation"]["state"],
]
values = [_markdown_cell(value) for value in values]
if bold:
values = [f"**{value}**" for value in values]
return "| " + " | ".join(values) + " |"
def render_markdown(report: dict) -> str:
lines = [
"# GL CTS multi-suite conformance report",
"",
(
"| Backend | Suite | Expected | Result | Pass | Accepted | Crash | Hang | "
"Unrun | Duplicate | Coverage | Strict Pass-only | Conformance-accepted | Validation |"
),
"|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|:---:|",
]
for backend in report["backends"]:
for item in report["suites"]:
if item["backend"] == backend:
lines.append(_table_row(backend, item["label"], item))
subtotal = report["backends"][backend]
lines.append(
_table_row(backend, f"{backend} weighted subtotal", subtotal, bold=True)
)
lines.append(
_table_row(
"All backends",
"Overall weighted",
report["overall"],
bold=True,
)
)
lines.extend(
[
"",
(
"Rates use unique **Expected** caselist cases as the denominator. "
"Unrun cases remain in the denominator and are not accepted."
),
"Accepted statuses: "
+ ", ".join(
f"`{status}`" for status in report["accepted_statuses"]
)
+ ".",
(
"Duplicate is the number of extra QPA observations; the final "
"observation wins."
),
]
)
details: list[str] = []
for item in report["suites"]:
validation = item["validation"]
messages = validation["errors"] + validation["warnings"]
if messages:
details.append(
f"- **{_markdown_cell(item['suite_id'])} {validation['state']}**: "
+ "; ".join(_markdown_cell(message) for message in messages)
)
if details:
lines.extend(["", "## Validation details", "", *details])
return "\n".join(lines) + "\n"
def _write_text(path: str, contents: str) -> None:
absolute = os.path.abspath(path)
os.makedirs(os.path.dirname(absolute), exist_ok=True)
with open(absolute, "w", encoding="utf-8", newline="\n") as handle:
handle.write(contents)
def _write_json(path: str, report: dict) -> None:
_write_text(path, json.dumps(report, indent=2, sort_keys=True) + "\n")
def _normalise_compact_suite_args(argv: Sequence[str]) -> list[str]:
"""Expand ``--suite=b,l,c,r`` into the four-value argparse form."""
result: list[str] = []
index = 0
while index < len(argv):
token = argv[index]
if token.startswith("--suite="):
compact = token.split("=", 1)[1]
parts = compact.split(",", 3)
if len(parts) != 4:
raise MultiReportInputError(
"compact --suite expects backend,label,caselist,result-dir"
)
result.extend(["--suite", *parts])
index += 1
continue
if token == "--suite" and index + 1 < len(argv) and argv[index + 1].count(",") >= 3:
parts = argv[index + 1].split(",", 3)
result.extend(["--suite", *parts])
index += 2
continue
result.append(token)
index += 1
return result
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--suite",
action="append",
nargs=4,
required=True,
metavar=("BACKEND", "LABEL", "CASELIST", "RESULT_DIR"),
help=(
"suite specification; repeat for every backend/suite pair "
f"(backends: {', '.join(SUPPORTED_BACKENDS)})"
),
)
parser.add_argument(
"--markdown",
default="cts_multi_report.md",
help="Markdown output path (default: ./cts_multi_report.md)",
)
parser.add_argument(
"--json",
dest="json_out",
default="cts_multi_report.json",
help="JSON output path (default: ./cts_multi_report.json)",
)
parser.add_argument(
"--allow-incomplete",
action="store_true",
help="return success even when one or more suites are ERROR/INCOMPLETE",
)
parser.add_argument(
"--adopt-legacy",
dest="allow_unverified_provenance",
action="store_true",
help="accept legacy result directories without run_state.json (provenance remains unverified)",
)
parser.add_argument(
"--allow-unverified-provenance",
dest="allow_unverified_provenance",
action="store_true",
help=argparse.SUPPRESS,
)
parser.add_argument(
"--expected-run-identity",
help="require every suite run_state.json to contain this controller fingerprint",
)
return parser
def _specs_from_args(values: Sequence[Sequence[str]]) -> list[SuiteSpec]:
return [
SuiteSpec(
backend=backend.strip(),
label=label.strip(),
caselist=caselist,
result_dir=result_dir,
)
for backend, label, caselist, result_dir in values
]
def main(argv: Optional[Iterable[str]] = None) -> int:
raw_argv = list(argv) if argv is not None else sys.argv[1:]
try:
normalised = _normalise_compact_suite_args(raw_argv)
except MultiReportInputError as exc:
print(f"cts_multi_report: {exc}", file=sys.stderr)
return 2
args = _parser().parse_args(normalised)
if os.path.normcase(os.path.abspath(args.markdown)) == os.path.normcase(
os.path.abspath(args.json_out)
):
print("cts_multi_report: Markdown and JSON paths must differ", file=sys.stderr)
return 2
try:
report = build_report(
_specs_from_args(args.suite),
require_run_state=not args.allow_unverified_provenance,
expected_run_identity=args.expected_run_identity,
)
markdown = render_markdown(report)
_write_text(args.markdown, markdown)
_write_json(args.json_out, report)
except (
OSError,
MultiReportInputError,
cts_matrix_report.ReportInputError,
) as exc:
print(f"cts_multi_report: {exc}", file=sys.stderr)
return 2
print(markdown, end="")
print(f"\nMarkdown: {os.path.abspath(args.markdown)}")
print(f"JSON: {os.path.abspath(args.json_out)}")
if not args.allow_incomplete and not report["overall"]["validation"]["ok"]:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+878
View File
@@ -0,0 +1,878 @@
#!/usr/bin/env python
"""Run a local Windows glcts executable and resume across process failures.
The desktop CTS normally runs a complete caselist in one process. That is a
poor fit for testing a developing OpenGL implementation: one access violation
or GPU hang prevents every later case from running. This driver gives each
invocation the cases which have not produced a result yet, preserves one QPA
and stdout/stderr pair per invocation, and starts another process after a
crash.
Existing ``chunkNNNN.qpa`` files and ``crashed.txt``/``hung.txt`` sidecars are
read on startup, so invoking the same command and output directory resumes an
interrupted run. A timeout is based on *idle QPA time*, not total process wall
time: a healthy invocation may legitimately run thousands of cases for hours.
Example (values beginning with ``--`` use argparse's ``=`` spelling)::
py run_cts_windows.py \
--exe D:\\glcts\\glcts.exe --workdir D:\\glcts \
--caselist D:\\glcts\\mustpass\\gl30.txt --outdir D:\\results\\gl30 \
--backend DirectVulkan \
--deqp-arg=--deqp-surface-type=window
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import signal
import subprocess
import sys
import tempfile
import time
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Iterable, Optional, Sequence
CASE_START = re.compile(r"^#beginTestCaseResult\s+(\S+)")
CASE_END = re.compile(r"^#endTestCaseResult(?:\s|$)")
CASE_TERM = re.compile(r"^#terminateTestCaseResult(?:\s|$)")
CASE_RESULT = re.compile(r'<Result\s+StatusCode="[^"]+"')
CHUNK_ARTIFACT = re.compile(r"^chunk(\d+)(?:\.|$)", re.IGNORECASE)
CHUNK_META = re.compile(r"^chunk(\d+)\.meta\.json$", re.IGNORECASE)
RECOVERY_SIDECAR_NAMES = frozenset(
{"crashed.txt", "hung.txt", "unrun.txt", "skipped.txt", "remaining.txt"}
)
CONTROLLED_DEQP_OPTIONS = {
"--deqp-caselist-file",
"--deqp-log-filename",
}
ATOMIC_REPLACE_ATTEMPTS = 8
ATOMIC_REPLACE_INITIAL_BACKOFF_SECONDS = 0.025
ATOMIC_REPLACE_MAX_BACKOFF_SECONDS = 0.2
class RunnerError(Exception):
"""A user/configuration error which should not be attributed to a case."""
@dataclass
class QpaProgress:
"""Cases recorded by a QPA and its unterminated tail, if any."""
recorded: list[str]
in_flight: Optional[str]
begin_count: int
@dataclass
class ProcessOutcome:
returncode: Optional[int]
duration_seconds: float
timed_out: bool = False
timeout_reason: Optional[str] = None
interrupted: bool = False
launch_error: Optional[str] = None
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def read_caselist(path: Path) -> list[str]:
"""Read a dEQP text caselist, preserving order and removing duplicates."""
try:
lines = path.read_text(encoding="utf-8-sig", errors="strict").splitlines()
except (OSError, UnicodeError) as exc:
raise RunnerError(f"cannot read caselist {path}: {exc}") from exc
cases: list[str] = []
seen: set[str] = set()
for raw in lines:
case = raw.strip()
if not case or case.startswith("#") or case in seen:
continue
cases.append(case)
seen.add(case)
if not cases:
raise RunnerError(f"caselist contains no test cases: {path}")
return cases
def read_name_set(path: Path) -> set[str]:
if not path.is_file():
return set()
try:
return {
line.strip()
for line in path.read_text(encoding="utf-8-sig", errors="replace").splitlines()
if line.strip() and not line.lstrip().startswith("#")
}
except OSError as exc:
raise RunnerError(f"cannot read recovery file {path}: {exc}") from exc
def _replace_with_retry(source: Path, destination: Path) -> None:
"""Replace a state file, tolerating brief Windows access-denied races.
Antivirus/indexing tools can momentarily open ``remaining.txt`` without
delete sharing. Windows then reports either ``PermissionError`` or a
generic ``OSError`` carrying ``winerror == 5``. Retry only those cases;
disk, path, and programming errors remain immediately visible.
"""
for attempt in range(ATOMIC_REPLACE_ATTEMPTS):
try:
os.replace(source, destination)
return
except OSError as exc:
retryable = isinstance(exc, PermissionError) or getattr(exc, "winerror", None) == 5
if not retryable or attempt + 1 >= ATOMIC_REPLACE_ATTEMPTS:
raise
delay = min(
ATOMIC_REPLACE_INITIAL_BACKOFF_SECONDS * (2**attempt),
ATOMIC_REPLACE_MAX_BACKOFF_SECONDS,
)
time.sleep(delay)
def atomic_write_text(path: Path, text: str) -> None:
"""Replace a small state file without exposing a partially-written copy."""
path.parent.mkdir(parents=True, exist_ok=True)
fd, temporary = tempfile.mkstemp(prefix=f".{path.name}.", suffix=".tmp", dir=str(path.parent))
temporary_path = Path(temporary)
try:
with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as handle:
handle.write(text)
handle.flush()
os.fsync(handle.fileno())
_replace_with_retry(temporary_path, path)
finally:
try:
temporary_path.unlink()
except FileNotFoundError:
pass
def atomic_write_json(path: Path, value: object) -> None:
atomic_write_text(path, json.dumps(value, indent=2, sort_keys=True) + "\n")
def write_case_file(path: Path, cases: Iterable[str]) -> None:
values = list(cases)
atomic_write_text(path, "\n".join(values) + ("\n" if values else ""))
def scan_qpa(path: Path) -> QpaProgress:
"""Return cases with a final result and the unfinished tail, if any.
``#terminateTestCaseResult`` is a completed result (usually Crash or
Timeout). ``#endTestCaseResult`` only completes a case when its XML carried
a ``<Result StatusCode=...>``. A truncated case that already wrote Result is
also recoverable; a case with no Result remains eligible for a retry.
"""
if not path.is_file():
return QpaProgress([], None, 0)
recorded: list[str] = []
current: Optional[str] = None
has_result = False
begin_count = 0
try:
with path.open("r", encoding="utf-8", errors="replace") as handle:
for raw_line in handle:
line = raw_line.lstrip("\ufeff")
match = CASE_START.match(line)
if match:
if current is not None and has_result:
recorded.append(current)
current = match.group(1)
has_result = False
begin_count += 1
continue
if current is not None and CASE_RESULT.search(line):
has_result = True
continue
if current is not None and CASE_TERM.match(line):
recorded.append(current)
current = None
has_result = False
continue
if current is not None and CASE_END.match(line):
if has_result:
recorded.append(current)
current = None
has_result = False
except OSError as exc:
raise RunnerError(f"cannot read QPA {path}: {exc}") from exc
if current is not None and has_result:
recorded.append(current)
current = None
return QpaProgress(recorded, current, begin_count)
def numbered_files(outdir: Path, pattern: re.Pattern[str]) -> list[tuple[int, Path]]:
found: list[tuple[int, Path]] = []
try:
children = list(outdir.iterdir())
except OSError as exc:
raise RunnerError(f"cannot list output directory {outdir}: {exc}") from exc
for path in children:
match = pattern.match(path.name)
if match:
found.append((int(match.group(1)), path))
found.sort(key=lambda item: item[0])
return found
def next_chunk_number(outdir: Path) -> int:
numbers = [number for number, _path in numbered_files(outdir, CHUNK_ARTIFACT)]
return max(numbers, default=-1) + 1
def load_meta_classifications(outdir: Path, expected: set[str]) -> tuple[set[str], set[str]]:
"""Recover an atomic classification written just before sidecar updates."""
crashed: set[str] = set()
hung: set[str] = set()
for _number, path in numbered_files(outdir, CHUNK_META):
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError):
# A damaged metadata file is diagnostic only. QPA and sidecars are
# authoritative and must still allow recovery.
continue
if not isinstance(value, dict):
continue
case = value.get("classified_case")
classification = value.get("classification")
if not isinstance(case, str) or case not in expected:
continue
if classification == "DeviceHang":
hung.add(case)
elif classification == "Crash":
crashed.add(case)
crashed.difference_update(hung)
return crashed, hung
def result_qpa_files(outdir: Path) -> list[Path]:
"""Return every QPA a directory-based report would consume."""
found: list[Path] = []
try:
for root, directories, names in os.walk(outdir):
directories.sort(key=str.casefold)
for name in sorted(names, key=str.casefold):
if name.casefold().endswith(".qpa"):
found.append(Path(root) / name)
except OSError as exc:
raise RunnerError(f"cannot scan output directory {outdir}: {exc}") from exc
return found
def recover_results(outdir: Path, expected: set[str]) -> tuple[set[str], set[str], set[str]]:
recorded: set[str] = set()
for path in result_qpa_files(outdir):
progress = scan_qpa(path)
recorded.update(case for case in progress.recorded if case in expected)
crashed = read_name_set(outdir / "crashed.txt") & expected
hung = read_name_set(outdir / "hung.txt") & expected
meta_crashed, meta_hung = load_meta_classifications(outdir, expected)
crashed.update(meta_crashed)
hung.update(meta_hung)
crashed.difference_update(hung)
return recorded, crashed, hung
def caselist_fingerprint(cases: Sequence[str]) -> str:
payload = "\n".join(cases).encode("utf-8") + b"\n"
return hashlib.sha256(payload).hexdigest()
def recovery_artifacts(outdir: Path) -> list[Path]:
"""Return prior-run evidence which must not be adopted implicitly."""
found = set(result_qpa_files(outdir))
try:
children = list(outdir.iterdir())
except OSError as exc:
raise RunnerError(f"cannot list output directory {outdir}: {exc}") from exc
found.update(
path
for path in children
if CHUNK_ARTIFACT.match(path.name)
or path.name.casefold() in RECOVERY_SIDECAR_NAMES
)
return sorted(
found,
key=lambda path: str(path.relative_to(outdir)).casefold(),
)
def check_run_identity(
outdir: Path,
backend: str,
cases: Sequence[str],
invocation_identity: Optional[str] = None,
adopt_legacy: bool = False,
) -> None:
"""Refuse to silently mix different suites/backends in one result dir."""
path = outdir / "run_state.json"
fingerprint = caselist_fingerprint(cases)
if path.is_file():
try:
state = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise RunnerError(f"cannot read run identity {path}: {exc}") from exc
if not isinstance(state, dict):
raise RunnerError(f"run identity must be a JSON object: {path}")
if state.get("backend") != backend:
raise RunnerError(
f"output directory belongs to backend {state.get('backend')!r}, not {backend!r}: {outdir}"
)
if state.get("caselist_sha256") != fingerprint:
raise RunnerError(f"output directory belongs to a different caselist: {outdir}")
stored_invocation_identity = state.get("invocation_identity")
if (
stored_invocation_identity is not None
or invocation_identity is not None
) and stored_invocation_identity != invocation_identity:
raise RunnerError(f"output directory belongs to a different CTS invocation: {outdir}")
return
legacy_artifacts = recovery_artifacts(outdir)
if legacy_artifacts and not adopt_legacy:
examples = ", ".join(path.name for path in legacy_artifacts[:3])
raise RunnerError(
"output directory contains CTS recovery artifacts but no run_state.json; "
f"refusing to adopt unverified legacy results ({examples}). Re-run with "
"--adopt-legacy only after verifying the backend, caselist, and invocation."
)
atomic_write_json(
path,
{
"version": 1,
"backend": backend,
"case_count": len(cases),
"caselist_sha256": fingerprint,
"invocation_identity": invocation_identity,
"adopted_legacy": bool(legacy_artifacts),
"created_utc": utc_now(),
},
)
def persist_sidecars(
outdir: Path,
ordered_cases: Sequence[str],
crashed: set[str],
hung: set[str],
remaining: Sequence[str],
) -> None:
write_case_file(outdir / "crashed.txt", (case for case in ordered_cases if case in crashed))
write_case_file(outdir / "hung.txt", (case for case in ordered_cases if case in hung))
write_case_file(outdir / "unrun.txt", remaining)
write_case_file(outdir / "remaining.txt", remaining)
def parse_environment(values: Sequence[str]) -> dict[str, str]:
result: dict[str, str] = {}
for value in values:
if "=" not in value:
raise RunnerError(f"--env expects NAME=VALUE, got {value!r}")
name, contents = value.split("=", 1)
if not name or "\x00" in name or "=" in name:
raise RunnerError(f"invalid environment variable name in {value!r}")
result[name] = contents
return result
def validate_deqp_args(values: Sequence[str]) -> None:
for value in values:
option = value.split("=", 1)[0].lower()
if option in CONTROLLED_DEQP_OPTIONS:
raise RunnerError(f"{option} is controlled by this runner and cannot be supplied via --deqp-arg")
def resolve_paths(
exe_value: str,
workdir_value: Optional[str],
caselist_value: str,
outdir_value: str,
) -> tuple[Path, Path, Path, Path]:
launch_dir = Path.cwd()
requested_exe = Path(exe_value).expanduser()
if workdir_value:
workdir = Path(workdir_value).expanduser().resolve()
elif requested_exe.is_absolute():
workdir = requested_exe.resolve().parent
else:
workdir = launch_dir
if requested_exe.is_absolute():
exe = requested_exe.resolve()
else:
in_workdir = (workdir / requested_exe).resolve()
in_launch_dir = (launch_dir / requested_exe).resolve()
exe = in_workdir if in_workdir.is_file() else in_launch_dir
caselist = Path(caselist_value).expanduser().resolve()
outdir = Path(outdir_value).expanduser().resolve()
if not exe.is_file():
raise RunnerError(f"glcts executable does not exist: {exe}")
if not workdir.is_dir():
raise RunnerError(f"working directory does not exist: {workdir}")
if not caselist.is_file():
raise RunnerError(f"caselist does not exist: {caselist}")
return exe, workdir, caselist, outdir
def qpa_signature(path: Path) -> Optional[tuple[int, int]]:
try:
stat = path.stat()
except FileNotFoundError:
return None
except OSError:
# A transient sharing violation must not kill a healthy process. The
# next poll will retry and the idle clock retains its previous value.
return None
return stat.st_size, stat.st_mtime_ns
def kill_process_tree(process: subprocess.Popen[bytes]) -> None:
"""Force-stop the process and descendants, with a parent-only fallback."""
if process.poll() is not None:
return
if os.name == "nt":
# /T is essential: CTS/platform helpers can outlive the top-level
# process, retain the QPA/DLL, and poison the next continuation round.
taskkill = Path(os.environ.get("SystemRoot", r"C:\Windows")) / "System32" / "taskkill.exe"
command = [str(taskkill), "/PID", str(process.pid), "/T", "/F"]
try:
subprocess.run(
command,
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=20,
check=False,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
)
except (OSError, subprocess.TimeoutExpired):
pass
else:
try:
os.killpg(process.pid, signal.SIGKILL)
except (ProcessLookupError, PermissionError, OSError):
pass
try:
process.wait(timeout=10)
return
except subprocess.TimeoutExpired:
pass
try:
process.kill()
except OSError:
pass
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
pass
def run_process(
command: Sequence[str],
workdir: Path,
environment: dict[str, str],
qpa_path: Path,
stdout_path: Path,
stderr_path: Path,
idle_timeout: float,
max_round_seconds: float,
poll_seconds: float,
) -> ProcessOutcome:
"""Run one CTS chunk, killing its tree only after QPA progress stalls."""
started = time.monotonic()
with stdout_path.open("wb") as stdout_handle, stderr_path.open("wb") as stderr_handle:
popen_options: dict[str, object] = {
"cwd": str(workdir),
"env": environment,
"stdin": subprocess.DEVNULL,
"stdout": stdout_handle,
"stderr": stderr_handle,
}
if os.name == "nt":
popen_options["creationflags"] = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
else:
popen_options["start_new_session"] = True
try:
process = subprocess.Popen(list(command), **popen_options) # type: ignore[arg-type]
except OSError as exc:
message = f"failed to launch {command[0]}: {exc}\n"
stderr_handle.write(message.encode("utf-8", errors="replace"))
stderr_handle.flush()
return ProcessOutcome(None, time.monotonic() - started, launch_error=str(exc))
last_signature = qpa_signature(qpa_path)
last_progress = time.monotonic()
timed_out = False
timeout_reason: Optional[str] = None
interrupted = False
try:
while True:
try:
returncode = process.wait(timeout=poll_seconds)
break
except subprocess.TimeoutExpired:
pass
now = time.monotonic()
signature = qpa_signature(qpa_path)
if signature is not None and signature != last_signature:
last_signature = signature
last_progress = now
if idle_timeout > 0 and now - last_progress >= idle_timeout:
timed_out = True
timeout_reason = "qpa-idle"
kill_process_tree(process)
returncode = process.poll()
break
if max_round_seconds > 0 and now - started >= max_round_seconds:
timed_out = True
timeout_reason = "max-round"
kill_process_tree(process)
returncode = process.poll()
break
except KeyboardInterrupt:
interrupted = True
kill_process_tree(process)
returncode = process.poll()
return ProcessOutcome(
returncode=returncode,
duration_seconds=time.monotonic() - started,
timed_out=timed_out,
timeout_reason=timeout_reason,
interrupted=interrupted,
)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Run Windows glcts against MobileGL, resuming across crashes and GPU hangs."
)
parser.add_argument("--exe", required=True, help="path to glcts.exe")
parser.add_argument(
"--workdir",
help="glcts working directory (default: executable directory, or current directory for a relative exe)",
)
parser.add_argument("--caselist", required=True, help="mustpass/caselist text file")
parser.add_argument("--outdir", required=True, help="persistent result directory")
parser.add_argument("--backend", required=True, choices=("DirectGLES", "DirectVulkan"))
parser.add_argument(
"--run-identity",
help="controller fingerprint for executable, data, arguments, and environment",
)
parser.add_argument(
"--adopt-legacy",
action="store_true",
help=(
"adopt existing chunk/sidecar results which predate run_state.json; "
"disabled by default because their provenance cannot be verified"
),
)
parser.add_argument(
"--env",
action="append",
default=[],
metavar="NAME=VALUE",
help="extra child environment variable (repeatable)",
)
parser.add_argument(
"--deqp-arg",
action="append",
default=[],
metavar="ARG",
help="extra glcts argument; repeat and use --deqp-arg=--option=value for leading dashes",
)
parser.add_argument(
"--idle-timeout",
type=float,
default=300.0,
metavar="SECONDS",
help="kill a chunk after this many seconds with no QPA size/mtime change (0 disables; default: 300)",
)
parser.add_argument(
"--max-round-seconds",
type=float,
default=0.0,
metavar="SECONDS",
help="optional total wall limit for one invocation (0 disables; default: 0)",
)
parser.add_argument("--poll-seconds", type=float, default=1.0, help=argparse.SUPPRESS)
parser.add_argument(
"--max-rounds",
type=int,
default=10000,
help="maximum glcts invocations in this runner process (default: 10000)",
)
parser.add_argument(
"--max-empty-streak",
type=int,
default=3,
help="abort after this many invocations record no case at all; no case is blamed (default: 3)",
)
return parser
def execute(args: argparse.Namespace) -> int:
if args.idle_timeout < 0 or args.max_round_seconds < 0:
raise RunnerError("timeout values must be non-negative")
if args.poll_seconds <= 0:
raise RunnerError("--poll-seconds must be greater than zero")
if args.max_rounds <= 0 or args.max_empty_streak <= 0:
raise RunnerError("--max-rounds and --max-empty-streak must be greater than zero")
validate_deqp_args(args.deqp_arg)
extra_environment = parse_environment(args.env)
exe, workdir, caselist_path, outdir = resolve_paths(
args.exe, args.workdir, args.caselist, args.outdir
)
outdir.mkdir(parents=True, exist_ok=True)
cases = read_caselist(caselist_path)
expected = set(cases)
check_run_identity(
outdir,
args.backend,
cases,
args.run_identity,
adopt_legacy=args.adopt_legacy,
)
recorded, crashed, hung = recover_results(outdir, expected)
accounted = recorded | crashed | hung
remaining = [case for case in cases if case not in accounted]
persist_sidecars(outdir, cases, crashed, hung, remaining)
existing_qpas = len(result_qpa_files(outdir))
print(
f"[run_cts_windows] {args.backend}: expected {len(cases)}, recovered {len(accounted)} "
f"({existing_qpas} QPA chunk(s), {len(crashed)} crash, {len(hung)} hang)"
)
if not remaining:
print(f"[run_cts_windows] complete: all {len(cases)} expected cases are accounted")
return 0
environment = os.environ.copy()
environment.update(extra_environment)
# --backend is authoritative even if the inherited or extra environment
# already contains a different value.
environment["MOBILEGL_BACKEND_TYPE"] = args.backend
common_arguments = [
"--deqp-terminate-on-device-lost=disable",
"--deqp-log-images=disable",
"--deqp-log-shader-sources=disable",
]
chunk_number = next_chunk_number(outdir)
rounds = 0
empty_streak = 0
interrupted = False
fatal_launch_error = False
started_all = time.monotonic()
while remaining and rounds < args.max_rounds:
prefix = f"chunk{chunk_number:04d}"
remaining_path = outdir / "remaining.txt"
qpa_path = outdir / f"{prefix}.qpa"
stdout_path = outdir / f"{prefix}.stdout.log"
stderr_path = outdir / f"{prefix}.stderr.log"
meta_path = outdir / f"{prefix}.meta.json"
# The number allocator considers every chunk artifact, so these should
# be new. Refuse to truncate evidence if a foreign file races us.
for artifact in (qpa_path, stdout_path, stderr_path, meta_path):
if artifact.exists():
raise RunnerError(f"refusing to overwrite existing chunk artifact: {artifact}")
write_case_file(remaining_path, remaining)
command = [
str(exe),
f"--deqp-caselist-file={remaining_path}",
f"--deqp-log-filename={qpa_path}",
*common_arguments,
*args.deqp_arg,
]
print(
f"[run_cts_windows] {prefix}: launching {len(remaining)} remaining case(s); "
f"idle timeout {args.idle_timeout:g}s"
)
chunk_started_utc = utc_now()
outcome = run_process(
command,
workdir,
environment,
qpa_path,
stdout_path,
stderr_path,
args.idle_timeout,
args.max_round_seconds,
args.poll_seconds,
)
progress = scan_qpa(qpa_path)
before = set(accounted)
for case in progress.recorded:
if case in expected:
recorded.add(case)
accounted.add(case)
classification: Optional[str] = None
classified_case: Optional[str] = None
in_flight = progress.in_flight if progress.in_flight in expected else None
if not outcome.interrupted and in_flight is not None and in_flight not in accounted:
classified_case = in_flight
if outcome.timed_out:
classification = "DeviceHang"
hung.add(in_flight)
crashed.discard(in_flight)
else:
classification = "Crash"
crashed.add(in_flight)
accounted.add(in_flight)
new_accounted = len(accounted - before)
if new_accounted:
empty_streak = 0
elif progress.begin_count == 0:
# No #begin marker means there is no evidence that the first
# remaining case was reached. Retry the identical caselist, then
# abort rather than manufacturing a string of false Crash results.
empty_streak += 1
else:
# A log containing only already-accounted cases is also no forward
# progress, but it is a different failure mode. Bound it with the
# same guard while retaining the QPA evidence.
empty_streak += 1
remaining = [case for case in cases if case not in accounted]
metadata = {
"version": 1,
"chunk": chunk_number,
"started_utc": chunk_started_utc,
"finished_utc": utc_now(),
"duration_seconds": round(outcome.duration_seconds, 3),
"returncode": outcome.returncode,
"timed_out": outcome.timed_out,
"timeout_reason": outcome.timeout_reason,
"interrupted": outcome.interrupted,
"launch_error": outcome.launch_error,
"qpa_begin_count": progress.begin_count,
"qpa_recorded_count": len(progress.recorded),
"in_flight": progress.in_flight,
"classification": classification,
"classified_case": classified_case,
"new_accounted": new_accounted,
"remaining": len(remaining),
}
# Metadata is committed first. If the runner itself dies between this
# write and the sidecars, recovery can reconstruct the classification.
atomic_write_json(meta_path, metadata)
persist_sidecars(outdir, cases, crashed, hung, remaining)
rounds += 1
elapsed_minutes = (time.monotonic() - started_all) / 60.0
detail = ""
if classification:
detail = f", {classification}={classified_case}"
if outcome.timed_out:
detail += f", timeout={outcome.timeout_reason}"
print(
f"[run_cts_windows] {prefix}: +{new_accounted}, accounted "
f"{len(accounted)}/{len(cases)}, remaining {len(remaining)}{detail} "
f"({elapsed_minutes:.1f} min)"
)
chunk_number += 1
if outcome.interrupted:
interrupted = True
print("[run_cts_windows] interrupted; process tree stopped and state preserved", file=sys.stderr)
break
if outcome.launch_error:
fatal_launch_error = True
print(
f"[run_cts_windows] launch failed; see {stderr_path.name}: {outcome.launch_error}",
file=sys.stderr,
)
break
if empty_streak >= args.max_empty_streak:
print(
f"[run_cts_windows] aborting after {empty_streak} consecutive chunks made no "
"case progress; no unobserved case was labelled Crash/Hang",
file=sys.stderr,
)
break
# Recompute from the persisted evidence so the final completeness claim is
# subject to the exact same recovery path as a later invocation.
final_recorded, final_crashed, final_hung = recover_results(outdir, expected)
final_accounted = final_recorded | final_crashed | final_hung
final_remaining = [case for case in cases if case not in final_accounted]
persist_sidecars(outdir, cases, final_crashed, final_hung, final_remaining)
if not final_remaining and final_accounted == expected:
print(
f"[run_cts_windows] complete: all {len(cases)} expected cases are accounted "
f"({len(final_crashed)} crash, {len(final_hung)} hang, {rounds} new invocation(s))"
)
return 0
print(
f"[run_cts_windows] INCOMPLETE: {len(final_accounted)}/{len(cases)} accounted; "
f"{len(final_remaining)} listed in {outdir / 'unrun.txt'}",
file=sys.stderr,
)
if interrupted:
return 130
if fatal_launch_error:
return 3
return 4
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
try:
return execute(args)
except RunnerError as exc:
print(f"[run_cts_windows] ERROR: {exc}", file=sys.stderr)
return 2
except OSError as exc:
print(f"[run_cts_windows] ERROR: filesystem/process operation failed: {exc}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())
+251
View File
@@ -0,0 +1,251 @@
import contextlib
import io
import json
import tempfile
import unittest
from pathlib import Path
try:
from . import cts_matrix_report as report
except ImportError: # Allows `python test_cts_matrix_report.py`.
import cts_matrix_report as report
def qpa_case(case, status):
return (
f"#beginTestCaseResult {case}\n"
f'<Result StatusCode="{status}"/>\n'
"#endTestCaseResult\n"
)
class MatrixReportTests(unittest.TestCase):
def test_utf8_bom_caselist_matches_runner_semantics(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_bytes(b"\xef\xbb\xbfcase.a\n")
results = root / "results"
results.mkdir()
(results / "run.qpa").write_text(
qpa_case("case.a", "Pass"), encoding="utf-8"
)
item = report.build_version_report("gl30", str(caselist), [str(results)])
self.assertEqual(1, item["expected"])
self.assertEqual({"case.a": "Pass"}, item["cases"]["results"])
self.assertEqual("OK", item["validation"]["state"])
def test_incomplete_qpa_is_unrun_not_a_completed_result(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_text("case.a\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "run.qpa").write_text(
"#beginTestCaseResult case.a\n#endTestCaseResult\n",
encoding="utf-8",
)
(results / "unrun.txt").write_text("case.a\n", encoding="utf-8")
item = report.build_version_report("gl46", str(caselist), [str(results)])
self.assertEqual(0, item["result"])
self.assertEqual(1, item["unrun"])
self.assertEqual(["case.a"], item["cases"]["incomplete_results"])
self.assertEqual("INCOMPLETE", item["validation"]["state"])
def test_incomplete_qpa_is_upgraded_by_crash_sidecar(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_text("case.a\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "run.qpa").write_text(
"#beginTestCaseResult case.a\n", encoding="utf-8"
)
(results / "crashed.txt").write_text("case.a\n", encoding="utf-8")
item = report.build_version_report("gl46", str(caselist), [str(results)])
self.assertEqual("Crash", item["cases"]["results"]["case.a"])
self.assertEqual([], item["cases"]["incomplete_results"])
self.assertEqual("OK", item["validation"]["state"])
def test_chunk_numbers_above_four_digits_use_numeric_order(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_text("case.a\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "alpha.qpa").write_text("# no results\n", encoding="utf-8")
(results / "chunk9999.qpa").write_text(
qpa_case("case.a", "Fail"), encoding="utf-8"
)
(results / "chunk10000.qpa").write_text(
qpa_case("case.a", "Pass"), encoding="utf-8"
)
(results / "zeta.qpa").write_text("# no results\n", encoding="utf-8")
item = report.build_version_report(
"gl46", str(caselist), [str(results)]
)
self.assertEqual(
["alpha.qpa", "chunk9999.qpa", "chunk10000.qpa", "zeta.qpa"],
[Path(path).name for path in item["inputs"]["qpa_files"]],
)
self.assertEqual("Pass", item["cases"]["results"]["case.a"])
self.assertEqual(1, item["duplicate"])
def test_qpa_sidecars_duplicates_and_expected_denominator(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "gl30.txt"
caselist.write_text("\n".join("abcdefg") + "\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "chunk0000.qpa").write_text(
qpa_case("a", "Fail") + "#beginTestCaseResult e\n",
encoding="utf-8",
)
(results / "chunk0001.qpa").write_text(
qpa_case("a", "Pass")
+ qpa_case("b", "NotSupported")
+ qpa_case("c", "QualityWarning")
+ qpa_case("d", "Fail"),
encoding="utf-8",
)
(results / "crashed.txt").write_text("e\n", encoding="utf-8")
(results / "hung.txt").write_text("f\n", encoding="utf-8")
(results / "unrun.txt").write_text("g\n", encoding="utf-8")
item = report.build_version_report(
"gl30", str(caselist), [str(results)]
)
self.assertEqual(item["expected"], 7)
self.assertEqual(item["result"], 6)
self.assertEqual(item["pass"], 1)
self.assertEqual(item["accepted"], 3)
self.assertEqual(item["crash"], 1)
self.assertEqual(item["hang"], 1)
self.assertEqual(item["unrun"], 1)
self.assertEqual(item["duplicate"], 1)
self.assertEqual(item["cases"]["results"]["a"], "Pass")
self.assertEqual(item["cases"]["results"]["e"], "Crash")
self.assertEqual(item["cases"]["results"]["f"], "DeviceHang")
self.assertAlmostEqual(item["strict_pass_rate"], 1 / 7)
self.assertAlmostEqual(item["conformance_accepted_rate"], 3 / 7)
self.assertAlmostEqual(
item["rates"]["measured_only_conformance_accepted"], 3 / 6
)
self.assertEqual(item["validation"]["state"], "INCOMPLETE")
self.assertEqual(item["validation"]["errors"], [])
self.assertTrue(
item["validation"]["invariant_expected_equals_result_plus_unrun"]
)
def test_missing_result_is_inferred_and_rejected_when_not_declared(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_text("a\nb\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "run.qpa").write_text(qpa_case("a", "Pass"), encoding="utf-8")
item = report.build_version_report(
"gl31", str(caselist), [str(results)]
)
self.assertEqual(item["unrun"], 1)
self.assertEqual(item["cases"]["unrun"], ["b"])
self.assertEqual(item["validation"]["state"], "ERROR")
self.assertEqual(item["validation"]["undeclared_unrun"], ["b"])
self.assertIn("not declared", item["validation"]["errors"][0])
self.assertEqual(item["strict_pass_rate"], 0.5)
def test_cli_emits_markdown_json_and_weighted_overall(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
statuses = {
"gl30": "Pass",
"gl31": "Fail",
"gl32": "NotSupported",
"gl33": None,
}
argv = []
for version, status in statuses.items():
caselist = root / f"{version}.txt"
caselist.write_text(f"{version}.case\n", encoding="utf-8")
result_dir = root / f"{version}-results"
result_dir.mkdir()
qpa = result_dir / "run.qpa"
qpa.write_text(
qpa_case(f"{version}.case", status) if status else "# empty run\n",
encoding="utf-8",
)
if status is None:
(result_dir / "unrun.txt").write_text(
f"{version}.case\n", encoding="utf-8"
)
argv.extend(
[
f"--{version}-caselist",
str(caselist),
f"--{version}-results",
str(result_dir),
]
)
json_path = root / "matrix.json"
argv.extend(["--json", str(json_path)])
stdout = io.StringIO()
with contextlib.redirect_stdout(stdout):
rc = report.main(argv)
self.assertEqual(rc, 1) # GL33 is explicitly incomplete.
markdown = stdout.getvalue()
self.assertIn("| Suite | Expected | Result", markdown)
self.assertIn("| **Overall (weighted)**", markdown)
payload = json.loads(json_path.read_text(encoding="utf-8"))
overall = payload["overall"]
self.assertEqual(overall["expected"], 4)
self.assertEqual(overall["result"], 3)
self.assertEqual(overall["pass"], 1)
self.assertEqual(overall["accepted"], 2)
self.assertEqual(overall["unrun"], 1)
self.assertEqual(overall["strict_pass_rate"], 0.25)
self.assertEqual(overall["conformance_accepted_rate"], 0.5)
self.assertEqual(overall["aggregation"], "weighted_by_expected_cases")
self.assertEqual(overall["validation"]["state"], "INCOMPLETE")
def test_duplicate_caselist_and_unexpected_result_are_validation_errors(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
caselist = root / "cases.txt"
caselist.write_text("a\na\n", encoding="utf-8")
results = root / "results"
results.mkdir()
(results / "run.qpa").write_text(
qpa_case("a", "Pass") + qpa_case("outside", "Pass"),
encoding="utf-8",
)
item = report.build_version_report(
"gl32", str(caselist), [str(results)]
)
self.assertEqual(item["cases"]["duplicate_caselist_entries"], {"a": 2})
self.assertEqual(item["cases"]["unexpected_results"], {"outside": "Pass"})
self.assertEqual(item["validation"]["state"], "ERROR")
self.assertEqual(len(item["validation"]["errors"]), 2)
if __name__ == "__main__":
unittest.main()
+342
View File
@@ -0,0 +1,342 @@
import contextlib
import io
import json
import tempfile
import unittest
from pathlib import Path
try:
from . import cts_multi_report as report
except ImportError: # Allows `python test_cts_multi_report.py`.
import cts_multi_report as report
def qpa_case(case: str, status: str) -> str:
return (
f"#beginTestCaseResult {case}\n"
f'<Result StatusCode="{status}"/>\n'
"#endTestCaseResult\n"
)
def write_run_state(
caselist: Path,
result_dir: Path,
backend: str,
invocation_identity=None,
) -> None:
fingerprint, case_count = report._caselist_fingerprint(str(caselist))
(result_dir / "run_state.json").write_text(
json.dumps(
{
"version": 1,
"backend": backend,
"case_count": case_count,
"caselist_sha256": fingerprint,
"invocation_identity": invocation_identity,
}
),
encoding="utf-8",
)
def make_inputs(
root: Path,
name: str,
cases: list[str],
qpa: str,
backend: str = "DirectGLES",
):
caselist = root / f"{name}.txt"
caselist.write_text("\n".join(cases) + "\n", encoding="utf-8")
result_dir = root / f"{name}-results"
result_dir.mkdir()
(result_dir / "chunk0000.qpa").write_text(qpa, encoding="utf-8")
write_run_state(caselist, result_dir, backend)
return caselist, result_dir
class MultiReportTests(unittest.TestCase):
def test_backend_aggregate_is_weighted_by_expected_cases(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
small_cases, small_results = make_inputs(
root, "small", ["small.pass"], qpa_case("small.pass", "Pass")
)
large_cases, large_results = make_inputs(
root,
"large",
["large.pass", "large.crash", "large.hang"],
qpa_case("large.pass", "Pass"),
)
(large_results / "crashed.txt").write_text(
"large.crash\n", encoding="utf-8"
)
(large_results / "hung.txt").write_text(
"large.hang\n", encoding="utf-8"
)
payload = report.build_report(
[
report.SuiteSpec(
"DirectGLES", "small", str(small_cases), str(small_results)
),
report.SuiteSpec(
"DirectGLES", "large", str(large_cases), str(large_results)
),
]
)
aggregate = payload["backends"]["DirectGLES"]
self.assertEqual(4, aggregate["expected"])
self.assertEqual(4, aggregate["result"])
self.assertEqual(2, aggregate["pass"])
self.assertEqual(2, aggregate["accepted"])
self.assertEqual(1, aggregate["crash"])
self.assertEqual(1, aggregate["hang"])
self.assertEqual(0, aggregate["unrun"])
# (1 accepted + 1 accepted) / (1 expected + 3 expected), not
# the unweighted mean of 100% and 33.3%.
self.assertEqual(0.5, aggregate["conformance_accepted_rate"])
self.assertEqual("weighted_by_expected_cases", aggregate["aggregation"])
self.assertEqual("OK", aggregate["validation"]["state"])
def test_declared_unrun_is_incomplete_and_cli_returns_nonzero(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root, "missing", ["case.a", "case.b"], qpa_case("case.a", "Pass")
)
(result_dir / "unrun.txt").write_text("case.b\n", encoding="utf-8")
markdown_path = root / "report.md"
json_path = root / "report.json"
argv = [
"--suite",
"DirectGLES",
"gl30",
str(caselist),
str(result_dir),
"--markdown",
str(markdown_path),
"--json",
str(json_path),
]
with contextlib.redirect_stdout(io.StringIO()):
returncode = report.main(argv)
self.assertEqual(1, returncode)
self.assertTrue(markdown_path.is_file())
payload = json.loads(json_path.read_text(encoding="utf-8"))
suite = payload["suites"][0]
self.assertEqual(2, suite["expected"])
self.assertEqual(1, suite["result"])
self.assertEqual(1, suite["unrun"])
self.assertEqual("INCOMPLETE", suite["validation"]["state"])
self.assertEqual("INCOMPLETE", payload["overall"]["validation"]["state"])
self.assertEqual(0.5, payload["overall"]["conformance_accepted_rate"])
def test_dual_backend_cli_outputs_markdown_and_json(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist = root / "gl30.txt"
caselist.write_text("gl30.case\n", encoding="utf-8")
gles = root / "gles"
vulkan = root / "vulkan"
gles.mkdir()
vulkan.mkdir()
(gles / "run.qpa").write_text(
qpa_case("gl30.case", "Pass"), encoding="utf-8"
)
(vulkan / "run.qpa").write_text(
qpa_case("gl30.case", "Fail"), encoding="utf-8"
)
write_run_state(caselist, gles, "DirectGLES")
write_run_state(caselist, vulkan, "DirectVulkan")
markdown_path = root / "dual.md"
json_path = root / "dual.json"
argv = [
f"--suite=DirectGLES,gl30,{caselist},{gles}",
"--suite",
"DirectVulkan",
"gl30",
str(caselist),
str(vulkan),
"--markdown",
str(markdown_path),
"--json",
str(json_path),
]
stdout = io.StringIO()
with contextlib.redirect_stdout(stdout):
returncode = report.main(argv)
self.assertEqual(0, returncode)
payload = json.loads(json_path.read_text(encoding="utf-8"))
self.assertEqual({"DirectGLES", "DirectVulkan"}, set(payload["backends"]))
self.assertEqual(1, payload["backends"]["DirectGLES"]["accepted"])
self.assertEqual(0, payload["backends"]["DirectVulkan"]["accepted"])
self.assertEqual(2, payload["overall"]["expected"])
self.assertEqual(1, payload["overall"]["accepted"])
self.assertEqual(0.5, payload["overall"]["conformance_accepted_rate"])
markdown = markdown_path.read_text(encoding="utf-8")
self.assertIn("DirectGLES weighted subtotal", markdown)
self.assertIn("DirectVulkan weighted subtotal", markdown)
self.assertIn("Overall weighted", markdown)
self.assertIn("Markdown:", stdout.getvalue())
def test_duplicate_qpa_result_uses_last_observation(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root,
"duplicate",
["case.a"],
qpa_case("case.a", "Fail"),
backend="DirectVulkan",
)
(result_dir / "chunk0001.qpa").write_text(
qpa_case("case.a", "Pass"), encoding="utf-8"
)
payload = report.build_report(
[
report.SuiteSpec(
"DirectVulkan", "gl33", str(caselist), str(result_dir)
)
]
)
suite = payload["suites"][0]
self.assertEqual("Pass", suite["cases"]["results"]["case.a"])
self.assertEqual(1, suite["duplicate"])
self.assertEqual(1, payload["overall"]["duplicate"])
self.assertEqual(1.0, payload["overall"]["strict_pass_rate"])
self.assertEqual("OK", suite["validation"]["state"])
self.assertIn("last result wins", suite["validation"]["warnings"][0])
def test_backend_provenance_mismatch_is_rejected(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root,
"provenance",
["case.a"],
qpa_case("case.a", "Pass"),
backend="DirectVulkan",
)
with self.assertRaises(report.MultiReportInputError):
report.build_report(
[report.SuiteSpec("DirectGLES", "gl30", str(caselist), str(result_dir))]
)
def test_missing_provenance_requires_explicit_legacy_opt_in(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root, "legacy", ["case.a"], qpa_case("case.a", "Pass")
)
(result_dir / "run_state.json").unlink()
spec = report.SuiteSpec("DirectGLES", "gl30", str(caselist), str(result_dir))
with self.assertRaises(report.MultiReportInputError):
report.build_report([spec])
payload = report.build_report([spec], require_run_state=False)
self.assertEqual("UNVERIFIED", payload["suites"][0]["provenance"]["state"])
def test_expected_run_identity_accepts_match_and_rejects_mismatch(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root, "identity", ["case.a"], qpa_case("case.a", "Pass")
)
write_run_state(
caselist, result_dir, "DirectGLES", invocation_identity="identity-a"
)
spec = report.SuiteSpec(
"DirectGLES", "gl30", str(caselist), str(result_dir)
)
payload = report.build_report(
[spec], expected_run_identity="identity-a"
)
self.assertEqual(
"identity-a",
payload["suites"][0]["provenance"]["invocation_identity"],
)
with self.assertRaises(report.MultiReportInputError):
report.build_report([spec], expected_run_identity="identity-b")
def test_expected_identity_rejects_legacy_state_and_missing_state(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root, "legacy-identity", ["case.a"], qpa_case("case.a", "Pass")
)
spec = report.SuiteSpec(
"DirectGLES", "gl30", str(caselist), str(result_dir)
)
with self.assertRaises(report.MultiReportInputError):
report.build_report([spec], expected_run_identity="identity-a")
(result_dir / "run_state.json").unlink()
with self.assertRaises(report.MultiReportInputError):
report.build_report(
[spec],
require_run_state=False,
expected_run_identity="identity-a",
)
def test_duplicate_physical_result_directory_is_rejected(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist, result_dir = make_inputs(
root, "duplicate-dir", ["case.a"], qpa_case("case.a", "Pass")
)
with self.assertRaises(report.MultiReportInputError):
report.build_report(
[
report.SuiteSpec(
"DirectGLES", "gl30", str(caselist), str(result_dir)
),
report.SuiteSpec(
"DirectGLES",
"gl31",
str(caselist),
str(result_dir / "."),
),
]
)
def test_ancestor_and_descendant_result_directories_are_rejected(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist = root / "cases.txt"
caselist.write_text("case.a\n", encoding="utf-8")
parent = root / "results"
child = parent / "nested"
child.mkdir(parents=True)
(parent / "chunk0000.qpa").write_text(
qpa_case("case.a", "Pass"), encoding="utf-8"
)
(child / "chunk0000.qpa").write_text(
qpa_case("case.a", "Fail"), encoding="utf-8"
)
write_run_state(caselist, parent, "DirectGLES")
write_run_state(caselist, child, "DirectVulkan")
with self.assertRaises(report.MultiReportInputError):
report.build_report(
[
report.SuiteSpec(
"DirectGLES", "gl30", str(caselist), str(parent)
),
report.SuiteSpec(
"DirectVulkan", "gl30", str(caselist), str(child)
),
]
)
if __name__ == "__main__":
unittest.main()
+401
View File
@@ -0,0 +1,401 @@
import sys
import tempfile
import time
import unittest
from pathlib import Path
from unittest import mock
import run_cts_windows as runner
def qpa_closed(case: str, status: str = "Pass") -> str:
return (
f"#beginTestCaseResult {case}\n"
f'<Result StatusCode="{status}">ok</Result>\n'
"#endTestCaseResult\n"
)
def command_path(command, option):
prefix = option + "="
return Path(next(value[len(prefix) :] for value in command if value.startswith(prefix)))
class AtomicWriteTests(unittest.TestCase):
def test_access_denied_retries_then_replace_succeeds(self):
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / "remaining.txt"
real_replace = runner.os.replace
attempts = 0
def flaky_replace(source, destination):
nonlocal attempts
attempts += 1
if attempts == 1:
raise PermissionError(13, "temporarily denied", str(destination))
if attempts == 2:
error = OSError("temporary WinError 5")
error.winerror = 5
raise error
real_replace(source, destination)
with mock.patch.object(runner.os, "replace", side_effect=flaky_replace), mock.patch.object(
runner.time, "sleep"
) as sleep:
runner.atomic_write_text(target, "case.a\n")
self.assertEqual(3, attempts)
self.assertEqual("case.a\n", target.read_text(encoding="utf-8"))
self.assertEqual(2, sleep.call_count)
self.assertEqual(
[
mock.call(runner.ATOMIC_REPLACE_INITIAL_BACKOFF_SECONDS),
mock.call(runner.ATOMIC_REPLACE_INITIAL_BACKOFF_SECONDS * 2),
],
sleep.call_args_list,
)
self.assertEqual([], list(target.parent.glob(".remaining.txt.*.tmp")))
def test_permanent_access_denied_stops_after_bounded_attempts(self):
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / "remaining.txt"
def always_denied(_source, _destination):
error = OSError("persistent WinError 5")
error.winerror = 5
raise error
with mock.patch.object(
runner.os, "replace", side_effect=always_denied
) as replace, mock.patch.object(runner.time, "sleep") as sleep:
with self.assertRaises(OSError) as raised:
runner.atomic_write_text(target, "case.a\n")
self.assertEqual(5, raised.exception.winerror)
self.assertEqual(runner.ATOMIC_REPLACE_ATTEMPTS, replace.call_count)
self.assertEqual(runner.ATOMIC_REPLACE_ATTEMPTS - 1, sleep.call_count)
self.assertFalse(target.exists())
self.assertEqual([], list(target.parent.glob(".remaining.txt.*.tmp")))
def test_non_access_error_is_not_retried(self):
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / "remaining.txt"
error = OSError(28, "disk full")
with mock.patch.object(
runner.os, "replace", side_effect=error
) as replace, mock.patch.object(runner.time, "sleep") as sleep:
with self.assertRaises(OSError):
runner.atomic_write_text(target, "case.a\n")
self.assertEqual(1, replace.call_count)
sleep.assert_not_called()
class QpaParsingTests(unittest.TestCase):
def test_terminate_is_a_completed_result(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "chunk0000.qpa"
path.write_text(
"#beginTestCaseResult KHR-GL30.a\n"
"#terminateTestCaseResult Crash\n"
"#beginTestCaseResult KHR-GL30.b\n",
encoding="utf-8",
)
progress = runner.scan_qpa(path)
self.assertEqual(["KHR-GL30.a"], progress.recorded)
self.assertEqual("KHR-GL30.b", progress.in_flight)
self.assertEqual(2, progress.begin_count)
def test_end_without_result_is_not_accounted(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "chunk0000.qpa"
path.write_text(
"#beginTestCaseResult KHR-GL46.incomplete\n"
"#endTestCaseResult\n"
+ qpa_closed("KHR-GL46.complete"),
encoding="utf-8",
)
progress = runner.scan_qpa(path)
self.assertEqual(["KHR-GL46.complete"], progress.recorded)
self.assertIsNone(progress.in_flight)
def test_result_written_before_truncated_eof_is_recovered(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "chunk0000.qpa"
path.write_text(
"#beginTestCaseResult KHR-GL46.complete\n"
'<Result StatusCode="Pass">ok</Result>\n',
encoding="utf-8",
)
progress = runner.scan_qpa(path)
self.assertEqual(["KHR-GL46.complete"], progress.recorded)
self.assertIsNone(progress.in_flight)
class RunIdentityTests(unittest.TestCase):
def test_non_object_run_state_is_a_controlled_error(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
(outdir / "run_state.json").write_text("null\n", encoding="utf-8")
with self.assertRaises(runner.RunnerError):
runner.check_run_identity(outdir, "DirectVulkan", ["case.a"])
def test_controller_identity_prevents_mixed_invocations(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a"], invocation_identity="identity-a"
)
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a"], invocation_identity="identity-a"
)
with self.assertRaises(runner.RunnerError):
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a"], invocation_identity="identity-b"
)
def test_controller_identity_cannot_be_downgraded_by_omission(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a"], invocation_identity="identity-a"
)
with self.assertRaises(runner.RunnerError):
runner.check_run_identity(outdir, "DirectVulkan", ["case.a"])
def test_legacy_artifacts_require_explicit_adoption(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
(outdir / "chunk0000.qpa").write_text(
qpa_closed("case.a"), encoding="utf-8"
)
with self.assertRaises(runner.RunnerError):
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a"], invocation_identity="identity-a"
)
self.assertFalse((outdir / "run_state.json").exists())
runner.check_run_identity(
outdir,
"DirectVulkan",
["case.a"],
invocation_identity="identity-a",
adopt_legacy=True,
)
state = runner.json.loads(
(outdir / "run_state.json").read_text(encoding="utf-8")
)
self.assertTrue(state["adopted_legacy"])
def test_foreign_nested_qpa_and_skipped_sidecar_are_legacy_evidence(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
nested = outdir / "old"
nested.mkdir()
qpa = nested / "legacy.qpa"
qpa.write_text(qpa_closed("case.a"), encoding="utf-8")
skipped = outdir / "skipped.txt"
skipped.write_text("case.b\n", encoding="utf-8")
self.assertEqual(
{qpa, skipped}, set(runner.recovery_artifacts(outdir))
)
with self.assertRaises(runner.RunnerError):
runner.check_run_identity(
outdir, "DirectVulkan", ["case.a", "case.b"]
)
recorded, crashed, hung = runner.recover_results(
outdir, {"case.a", "case.b"}
)
self.assertEqual({"case.a"}, recorded)
self.assertEqual(set(), crashed)
self.assertEqual(set(), hung)
def test_non_object_or_non_string_meta_classification_is_ignored(self):
with tempfile.TemporaryDirectory() as temporary:
outdir = Path(temporary)
(outdir / "chunk0000.meta.json").write_text("null\n", encoding="utf-8")
(outdir / "chunk0001.meta.json").write_text("[]\n", encoding="utf-8")
(outdir / "chunk0002.meta.json").write_text(
'{"classified_case": [], "classification": "Crash"}\n', encoding="utf-8"
)
self.assertEqual(
(set(), set()), runner.load_meta_classifications(outdir, {"case.a"})
)
class RunnerRecoveryTests(unittest.TestCase):
def run_args(self, root: Path, caselist: Path, outdir: Path, *extra: str):
return [
"--exe",
sys.executable,
"--workdir",
str(root),
"--caselist",
str(caselist),
"--outdir",
str(outdir),
"--backend",
"DirectVulkan",
*extra,
]
def test_crash_tail_is_quarantined_and_next_chunk_resumes(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist = root / "cases.txt"
outdir = root / "results"
caselist.write_text("KHR-GL30.a\nKHR-GL30.b\nKHR-GL30.c\n", encoding="utf-8")
seen_remaining = []
def fake_run(command, workdir, environment, qpa_path, stdout_path, stderr_path, *timeouts):
del workdir, timeouts
seen_remaining.append(
command_path(command, "--deqp-caselist-file")
.read_text(encoding="utf-8")
.splitlines()
)
stdout_path.write_text("fake stdout\n", encoding="utf-8")
stderr_path.write_text("fake stderr\n", encoding="utf-8")
self.assertEqual("DirectVulkan", environment["MOBILEGL_BACKEND_TYPE"])
if len(seen_remaining) == 1:
qpa_path.write_text(
qpa_closed("KHR-GL30.a") + "#beginTestCaseResult KHR-GL30.b\n",
encoding="utf-8",
)
return runner.ProcessOutcome(0xC0000005, 0.1)
qpa_path.write_text(qpa_closed("KHR-GL30.c"), encoding="utf-8")
return runner.ProcessOutcome(0, 0.1)
with mock.patch.object(runner, "run_process", side_effect=fake_run):
result = runner.main(self.run_args(root, caselist, outdir))
self.assertEqual(0, result)
self.assertEqual(
[
["KHR-GL30.a", "KHR-GL30.b", "KHR-GL30.c"],
["KHR-GL30.c"],
],
seen_remaining,
)
self.assertEqual("KHR-GL30.b\n", (outdir / "crashed.txt").read_text(encoding="utf-8"))
self.assertEqual("", (outdir / "hung.txt").read_text(encoding="utf-8"))
self.assertEqual("", (outdir / "unrun.txt").read_text(encoding="utf-8"))
def test_existing_qpa_and_sidecar_are_recovered(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist = root / "cases.txt"
outdir = root / "results"
outdir.mkdir()
caselist.write_text("KHR-GL31.a\nKHR-GL31.b\nKHR-GL31.c\n", encoding="utf-8")
(outdir / "chunk0000.qpa").write_text(qpa_closed("KHR-GL31.a"), encoding="utf-8")
(outdir / "crashed.txt").write_text("KHR-GL31.b\n", encoding="utf-8")
seen_remaining = []
def fake_run(command, workdir, environment, qpa_path, stdout_path, stderr_path, *timeouts):
del workdir, environment, stdout_path, stderr_path, timeouts
seen_remaining.extend(
command_path(command, "--deqp-caselist-file")
.read_text(encoding="utf-8")
.splitlines()
)
qpa_path.write_text(qpa_closed("KHR-GL31.c"), encoding="utf-8")
return runner.ProcessOutcome(0, 0.1)
with mock.patch.object(runner, "run_process", side_effect=fake_run):
result = runner.main(
self.run_args(root, caselist, outdir, "--adopt-legacy")
)
self.assertEqual(0, result)
self.assertEqual(["KHR-GL31.c"], seen_remaining)
self.assertTrue((outdir / "chunk0001.qpa").is_file())
def test_repeated_no_output_aborts_without_false_case_blame(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
caselist = root / "cases.txt"
outdir = root / "results"
caselist.write_text("KHR-GL32.a\nKHR-GL32.b\n", encoding="utf-8")
seen_remaining = []
def fake_run(command, workdir, environment, qpa_path, stdout_path, stderr_path, *timeouts):
del workdir, environment, stdout_path, stderr_path, timeouts
seen_remaining.append(
command_path(command, "--deqp-caselist-file")
.read_text(encoding="utf-8")
.splitlines()
)
qpa_path.write_text("#sessionInfo releaseName fake\n", encoding="utf-8")
return runner.ProcessOutcome(
1, 0.1, timed_out=True, timeout_reason="qpa-idle"
)
with mock.patch.object(runner, "run_process", side_effect=fake_run):
result = runner.main(
self.run_args(root, caselist, outdir, "--max-empty-streak", "2")
)
self.assertEqual(4, result)
self.assertEqual(
[["KHR-GL32.a", "KHR-GL32.b"], ["KHR-GL32.a", "KHR-GL32.b"]],
seen_remaining,
)
self.assertEqual("", (outdir / "crashed.txt").read_text(encoding="utf-8"))
self.assertEqual("", (outdir / "hung.txt").read_text(encoding="utf-8"))
self.assertEqual(
"KHR-GL32.a\nKHR-GL32.b\n",
(outdir / "unrun.txt").read_text(encoding="utf-8"),
)
class ProcessTimeoutTests(unittest.TestCase):
def test_qpa_activity_prevents_idle_timeout(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
qpa = root / "active.qpa"
helper = (
"import pathlib,sys,time\n"
"path=pathlib.Path(sys.argv[1])\n"
"for size in range(1, 9):\n"
" path.write_text('x' * size, encoding='utf-8')\n"
" time.sleep(0.08)\n"
)
outcome = runner.run_process(
[sys.executable, "-c", helper, str(qpa)],
root,
dict(runner.os.environ),
qpa,
root / "stdout.log",
root / "stderr.log",
idle_timeout=0.2,
max_round_seconds=0,
poll_seconds=0.03,
)
self.assertFalse(outcome.timed_out)
self.assertEqual(0, outcome.returncode)
def test_idle_timeout_really_stops_process(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
started = time.monotonic()
outcome = runner.run_process(
[sys.executable, "-c", "import time; time.sleep(30)"],
root,
dict(runner.os.environ),
root / "never-created.qpa",
root / "stdout.log",
root / "stderr.log",
idle_timeout=0.2,
max_round_seconds=0,
poll_seconds=0.05,
)
elapsed = time.monotonic() - started
self.assertTrue(outcome.timed_out)
self.assertEqual("qpa-idle", outcome.timeout_reason)
self.assertLess(elapsed, 10)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,294 @@
import argparse
import json
from pathlib import Path
import struct
import tempfile
import unittest
import wgl_glcts_pipeline as pipeline
def write_fake_pe(path: Path, machine: int = pipeline.PE_MACHINE_AMD64, payload: bytes = b"") -> None:
data = bytearray(0x88)
data[0:2] = b"MZ"
struct.pack_into("<I", data, 0x3C, 0x80)
data[0x80:0x84] = b"PE\0\0"
struct.pack_into("<H", data, 0x84, machine)
path.write_bytes(bytes(data) + payload)
class ArgumentTests(unittest.TestCase):
def test_versions_accept_gl_and_dotted_spellings(self):
self.assertEqual("30", pipeline.normalize_version("GL30"))
self.assertEqual("46", pipeline.normalize_version("4.6"))
with self.assertRaises(argparse.ArgumentTypeError):
pipeline.normalize_version("4.7")
def test_environment_assignment_validation(self):
self.assertEqual(("MOBILEGL_TEST", "a=b"), pipeline.parse_assignment("MOBILEGL_TEST=a=b"))
with self.assertRaises(argparse.ArgumentTypeError):
pipeline.parse_assignment("9BAD=value")
def test_windows_environment_keys_are_canonical_and_last_wins(self):
self.assertEqual(
{"FOO": "x", "PATH": "second"},
pipeline.canonicalize_windows_environment(
[("Path", "first"), ("FOO", "x"), ("pAtH", "second")]
),
)
class CommandTests(unittest.TestCase):
def test_visual_studio_configure_commands_are_x64_and_wgl_default(self):
mobilegl = pipeline.mobilegl_configure_command(
Path("C:/src/MobileGL"), Path("D:/work/mg"), "Visual Studio 17 2022", "x64", []
)
self.assertIn("-A", mobilegl)
self.assertIn("x64", mobilegl)
self.assertIn("-DMOBILEGL_BUILD_TEST=OFF", mobilegl)
cts = pipeline.cts_configure_command(
Path("D:/src/VK-GL-CTS"), Path("D:/work/cts"), "Visual Studio 17 2022", "x64", []
)
self.assertIn("-DDEQP_TARGET=default", cts)
self.assertNotIn("-DDEQP_TARGET=mobilegl", cts)
def test_runner_command_contains_identity_preserving_wgl_flags(self):
command = pipeline.runner_command(
Path("runner.py"),
Path("runtime/glcts.exe"),
Path("cts/modules"),
Path("gl46-main.txt"),
Path("results/gl46"),
"DirectVulkan",
300,
0,
10000,
{"MOBILEGL_LOG_FILE_PATH": "result/mobilegl.log"},
pipeline.DEFAULT_DEQP_ARGS,
"run-fingerprint",
)
self.assertIn("--backend", command)
self.assertIn("DirectVulkan", command)
self.assertIn("--deqp-arg=--deqp-gl-context-type=wgl", command)
self.assertIn("--deqp-arg=--deqp-surface-type=fbo", command)
self.assertIn("--env", command)
self.assertIn("MOBILEGL_LOG_FILE_PATH=result/mobilegl.log", command)
self.assertIn("--run-identity", command)
self.assertIn("run-fingerprint", command)
def test_report_command_requires_the_pipeline_run_identity(self):
command = pipeline.report_command(
Path("report.py"),
[("DirectVulkan", "gl46", Path("gl46.txt"), Path("results/gl46"))],
Path("summary.md"),
Path("summary.json"),
False,
"run-fingerprint",
)
self.assertIn("--expected-run-identity", command)
self.assertIn("run-fingerprint", command)
class RuntimeTests(unittest.TestCase):
def test_pe_machine_rejects_non_x64(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "x86.dll"
write_fake_pe(path, machine=0x14C)
self.assertEqual(0x14C, pipeline.pe_machine(path))
with self.assertRaises(pipeline.PipelineError):
pipeline.require_x64_pe(path, "test DLL")
def test_runtime_is_hash_keyed_and_copies_only_declared_files(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
glcts = root / "source-glcts.exe"
mobilegl = root / "source-MobileGL.dll"
write_fake_pe(glcts, payload=b"glcts")
write_fake_pe(mobilegl, payload=b"mobilegl")
sources = {"glcts.exe": glcts, "opengl32.dll": mobilegl}
fingerprint, hashes = pipeline.runtime_fingerprint(sources)
runtime = pipeline.assemble_runtime(root / "work", sources, fingerprint, hashes)
self.assertEqual(fingerprint[:16], runtime.name)
self.assertEqual(hashes["glcts.exe"], pipeline.sha256_file(runtime / "glcts.exe"))
self.assertEqual(hashes["opengl32.dll"], pipeline.sha256_file(runtime / "opengl32.dll"))
manifest = json.loads((runtime / "manifest.json").read_text(encoding="utf-8"))
self.assertEqual(fingerprint, manifest["fingerprint"])
self.assertFalse((runtime / "libEGL.dll").exists())
def test_run_fingerprint_changes_with_execution_semantics(self):
base = pipeline.run_fingerprint(
"runtime", "data", {"runner": "tool"}, {"30": "caselist"}, ["--deqp-surface-type=fbo"], {"FLAG": "1"}
)
self.assertEqual(
base,
pipeline.run_fingerprint(
"runtime", "data", {"runner": "tool"}, {"30": "caselist"}, ["--deqp-surface-type=fbo"], {"FLAG": "1"}
),
)
self.assertNotEqual(
base,
pipeline.run_fingerprint(
"runtime", "data", {"runner": "tool"}, {"30": "caselist"}, ["--deqp-surface-type=window"], {"FLAG": "1"}
),
)
self.assertNotEqual(
base,
pipeline.run_fingerprint(
"runtime", "data", {"runner": "tool"}, {"30": "different"}, ["--deqp-surface-type=fbo"], {"FLAG": "1"}
),
)
self.assertNotEqual(
base,
pipeline.run_fingerprint(
"runtime", "different-data", {"runner": "tool"}, {"30": "caselist"}, ["--deqp-surface-type=fbo"], {"FLAG": "1"}
),
)
self.assertNotEqual(
base,
pipeline.run_fingerprint(
"runtime", "data", {"runner": "different-tool"}, {"30": "caselist"}, ["--deqp-surface-type=fbo"], {"FLAG": "1"}
),
)
timeout_baseline = pipeline.run_fingerprint(
"runtime",
"data",
{"runner": "tool"},
{"30": "caselist"},
["--deqp-surface-type=fbo"],
{"FLAG": "1"},
{"idle_timeout_seconds": 300.0, "max_round_seconds": 0.0},
)
idle_changed = pipeline.run_fingerprint(
"runtime",
"data",
{"runner": "tool"},
{"30": "caselist"},
["--deqp-surface-type=fbo"],
{"FLAG": "1"},
{"idle_timeout_seconds": 1.0, "max_round_seconds": 0.0},
)
max_round_changed = pipeline.run_fingerprint(
"runtime",
"data",
{"runner": "tool"},
{"30": "caselist"},
["--deqp-surface-type=fbo"],
{"FLAG": "1"},
{"idle_timeout_seconds": 300.0, "max_round_seconds": 60.0},
)
self.assertNotEqual(timeout_baseline, idle_changed)
self.assertNotEqual(timeout_baseline, max_round_changed)
def test_tracked_environment_is_case_insensitive_and_narrow(self):
ambient, effective = pipeline.tracked_run_environment(
{
"Path": "ambient-path",
"mobilegl_debug": "0",
"LibGL_Driver": "ambient-libgl",
"vK_iCd_fIlEnAmEs": "ambient-icd",
"ANGLE_DEFAULT_PLATFORM": "vulkan",
"Egl_Test": "1",
"D3D_Feature": "1",
"DxVk_Config": "ambient-dxvk",
"HOME": "ignored",
"PATH_EXTRA": "ignored",
"MOBILEGL": "ignored",
},
{"pAtH": "explicit-path", "vk_icd_filenames": "explicit-icd", "CUSTOM": "kept"},
)
self.assertEqual("ambient-path", ambient["PATH"])
self.assertNotIn("HOME", ambient)
self.assertNotIn("PATH_EXTRA", ambient)
self.assertNotIn("MOBILEGL", ambient)
self.assertEqual("explicit-path", effective["PATH"])
self.assertEqual("explicit-icd", effective["VK_ICD_FILENAMES"])
self.assertEqual("kept", effective["CUSTOM"])
def test_reserved_environment_names_cannot_hide_behind_case(self):
overrides = pipeline.canonicalize_windows_environment(
[("mobilegl_backend_type", "DirectGLES")]
)
self.assertEqual(
{"MOBILEGL_BACKEND_TYPE"},
pipeline.CONTROLLED_ENVIRONMENT_NAMES & set(overrides),
)
def test_directgles_requires_complete_angle_runtime(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
glcts = root / "glcts.exe"
mobilegl = root / "MobileGL.dll"
write_fake_pe(glcts)
write_fake_pe(mobilegl)
with self.assertRaises(pipeline.PipelineError):
pipeline.runtime_source_files(glcts, mobilegl, ["DirectGLES"], root / "angle")
angle = root / "angle"
angle.mkdir()
for name in pipeline.ANGLE_REQUIRED_DLLS:
write_fake_pe(angle / name, payload=name.encode("ascii"))
files = pipeline.runtime_source_files(glcts, mobilegl, ["DirectGLES"], angle)
self.assertEqual(
{"glcts.exe", "opengl32.dll", *pipeline.ANGLE_REQUIRED_DLLS}, set(files)
)
class CaselistTests(unittest.TestCase):
def test_preflight_prefers_small_buffer_case(self):
with tempfile.TemporaryDirectory() as temporary:
caselist = Path(temporary) / "gl30-main.txt"
caselist.write_text(
"KHR-GL30.api.coverage\nKHR-GL30.buffer_objects.gen_buffers\n",
encoding="utf-8",
)
self.assertEqual("KHR-GL30.buffer_objects.gen_buffers", pipeline.choose_preflight_case(caselist))
class PreflightTests(unittest.TestCase):
def write_identity(self, root: Path, renderer: str, version: str = "4.6"):
qpa = root / "chunk0000.qpa"
qpa.write_text(
'#sessionInfo vendor "MobileGL-Dev"\n'
f'#sessionInfo renderer "{renderer}"\n'
'#sessionInfo commandLineParameters "--deqp-gl-context-type=wgl --deqp-surface-type=fbo"\n',
encoding="utf-8",
)
log = root / "mobilegl.log"
log.write_text(f"Target OpenGL Version: {version}\n", encoding="utf-8")
return qpa, log
def test_identity_accepts_both_mobilegl_renderers(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
qpa, log = self.write_identity(root, "Magma (MobileGL Core)")
identity = pipeline.parse_preflight_identity([qpa], log, "DirectVulkan", (4, 6))
self.assertEqual("4.6", identity["target_gl_version"])
qpa, log = self.write_identity(root, "Espryt (MobileGL Core)")
identity = pipeline.parse_preflight_identity([qpa], log, "DirectGLES", (3, 3))
self.assertIn("Espryt", identity["renderer"])
def test_identity_rejects_system_driver_or_low_version(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
qpa, log = self.write_identity(root, "NVIDIA GeForce RTX", version="4.6")
qpa.write_text(
'#sessionInfo vendor "NVIDIA Corporation"\n'
'#sessionInfo renderer "NVIDIA GeForce RTX"\n'
'#sessionInfo commandLineParameters "--deqp-gl-context-type=wgl"\n',
encoding="utf-8",
)
with self.assertRaises(pipeline.PipelineError):
pipeline.parse_preflight_identity([qpa], log, "DirectVulkan", (4, 6))
qpa, log = self.write_identity(root, "Magma (MobileGL Core)", version="4.5")
with self.assertRaises(pipeline.PipelineError):
pipeline.parse_preflight_identity([qpa], log, "DirectVulkan", (4, 6))
if __name__ == "__main__":
unittest.main()
+914
View File
@@ -0,0 +1,914 @@
#!/usr/bin/env python
"""Build MobileGL's Windows WGL shim and run Khronos OpenGL CTS suites.
The pipeline intentionally keeps the build, runtime, results, and reports in an
explicit work root. Each run is keyed by the hashes of glcts.exe, opengl32.dll,
and (for DirectGLES) the ANGLE runtime, so resuming can never silently combine
results from different binaries.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import struct
import subprocess
import sys
from datetime import datetime, timezone
from typing import Iterable, Mapping, Optional, Sequence
SUPPORTED_VERSIONS = ("30", "31", "32", "33", "40", "41", "42", "43", "44", "45", "46")
SUPPORTED_BACKENDS = ("DirectGLES", "DirectVulkan")
ANGLE_REQUIRED_DLLS = ("libEGL.dll", "libGLESv2.dll", "d3dcompiler_47.dll")
ANGLE_OPTIONAL_DLLS = ("dxcompiler.dll", "dxil.dll")
PE_MACHINE_AMD64 = 0x8664
TRACKED_ENVIRONMENT_NAMES = frozenset({"PATH"})
TRACKED_ENVIRONMENT_PREFIXES = (
"MOBILEGL_",
"LIBGL_",
"VK_",
"ANGLE_",
"EGL_",
"D3D_",
"DXVK_",
)
CONTROLLED_ENVIRONMENT_NAMES = frozenset(
{"MOBILEGL_BACKEND_TYPE", "MOBILEGL_LOG_FILE_PATH"}
)
DEFAULT_DEQP_ARGS = (
"--deqp-gl-context-type=wgl",
"--deqp-surface-type=fbo",
"--deqp-gl-config-name=rgba8888d24s8",
"--deqp-surface-width=64",
"--deqp-surface-height=-1",
"--deqp-base-seed=3",
"--deqp-visibility=hidden",
"--deqp-watchdog=enable",
"--deqp-crashhandler=enable",
)
SESSION_VENDOR = re.compile(r'^#sessionInfo vendor "([^"]*)"', re.MULTILINE)
SESSION_RENDERER = re.compile(r'^#sessionInfo renderer "([^"]*)"', re.MULTILINE)
SESSION_COMMAND_LINE = re.compile(r'^#sessionInfo commandLineParameters "([^"]*)"', re.MULTILINE)
TARGET_GL_VERSION = re.compile(r"Target OpenGL Version:\s*(\d+)\.(\d+)")
class PipelineError(RuntimeError):
"""A configuration, build, or identity error."""
def repository_root() -> Path:
return Path(__file__).resolve().parents[3]
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def normalize_version(value: str) -> str:
normalized = value.strip().lower().removeprefix("gl").replace(".", "")
if normalized not in SUPPORTED_VERSIONS:
supported = ", ".join(f"gl{version}" for version in SUPPORTED_VERSIONS)
raise argparse.ArgumentTypeError(f"unsupported GL suite {value!r}; choose one of: {supported}")
return normalized
def gl_version_tuple(version: str) -> tuple[int, int]:
return int(version[0]), int(version[1])
def parse_assignment(value: str) -> tuple[str, str]:
name, separator, setting = value.partition("=")
if not separator or not name or "\x00" in value:
raise argparse.ArgumentTypeError(f"expected NAME=VALUE, got {value!r}")
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", name):
raise argparse.ArgumentTypeError(f"invalid environment variable name {name!r}")
return name, setting
def canonicalize_windows_environment(
items: Iterable[tuple[str, str]],
) -> dict[str, str]:
"""Canonicalize environment keys using Windows' case-insensitive rules."""
result: dict[str, str] = {}
for name, value in items:
result[name.upper()] = value
return result
def tracked_run_environment(
inherited: Mapping[str, str], overrides: Mapping[str, str]
) -> tuple[dict[str, str], dict[str, str]]:
"""Return tracked ambient values and the effective values used for identity."""
canonical_inherited = canonicalize_windows_environment(inherited.items())
ambient = {
name: value
for name, value in canonical_inherited.items()
if name in TRACKED_ENVIRONMENT_NAMES
or name.startswith(TRACKED_ENVIRONMENT_PREFIXES)
}
effective = dict(ambient)
effective.update(canonicalize_windows_environment(overrides.items()))
return dict(sorted(ambient.items())), dict(sorted(effective.items()))
def command_text(command: Sequence[object]) -> str:
return subprocess.list2cmdline([str(part) for part in command])
def run_command(
command: Sequence[object],
*,
cwd: Optional[Path] = None,
env: Optional[Mapping[str, str]] = None,
check: bool = True,
capture: bool = False,
) -> subprocess.CompletedProcess[str]:
rendered = command_text(command)
location = f" (cwd={cwd})" if cwd else ""
print(f"[wgl_glcts_pipeline] $ {rendered}{location}", flush=True)
completed = subprocess.run(
[str(part) for part in command],
cwd=str(cwd) if cwd else None,
env=dict(env) if env else None,
text=True,
capture_output=capture,
check=False,
)
if check and completed.returncode != 0:
detail = ""
if capture:
detail = f"\nstdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
raise PipelineError(f"command failed with exit code {completed.returncode}: {rendered}{detail}")
return completed
def require_file(path: Path, label: str) -> Path:
resolved = path.expanduser().resolve()
if not resolved.is_file():
raise PipelineError(f"{label} does not exist or is not a file: {resolved}")
return resolved
def require_directory(path: Path, label: str) -> Path:
resolved = path.expanduser().resolve()
if not resolved.is_dir():
raise PipelineError(f"{label} does not exist or is not a directory: {resolved}")
return resolved
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def sha256_directory(path: Path) -> str:
digest = hashlib.sha256()
files = sorted(
(candidate for candidate in path.rglob("*") if candidate.is_file()),
key=lambda candidate: candidate.relative_to(path).as_posix(),
)
if not files:
raise PipelineError(f"directory contains no files to fingerprint: {path}")
for candidate in files:
relative = candidate.relative_to(path).as_posix()
digest.update(relative.encode("utf-8"))
digest.update(b"\0")
digest.update(sha256_file(candidate).encode("ascii"))
digest.update(b"\n")
return digest.hexdigest()
def pe_machine(path: Path) -> int:
with path.open("rb") as stream:
if stream.read(2) != b"MZ":
raise PipelineError(f"not a PE executable: {path}")
stream.seek(0x3C)
offset_bytes = stream.read(4)
if len(offset_bytes) != 4:
raise PipelineError(f"truncated PE header: {path}")
pe_offset = struct.unpack("<I", offset_bytes)[0]
stream.seek(pe_offset)
if stream.read(4) != b"PE\0\0":
raise PipelineError(f"invalid PE signature: {path}")
machine_bytes = stream.read(2)
if len(machine_bytes) != 2:
raise PipelineError(f"truncated PE COFF header: {path}")
return struct.unpack("<H", machine_bytes)[0]
def require_x64_pe(path: Path, label: str) -> None:
machine = pe_machine(path)
if machine != PE_MACHINE_AMD64:
raise PipelineError(f"{label} must be an x64 PE (machine 0x8664), got 0x{machine:04x}: {path}")
def git_snapshot(path: Path) -> dict[str, object]:
snapshot: dict[str, object] = {"path": str(path)}
try:
head = run_command(
["git", "-C", path, "rev-parse", "HEAD"], check=True, capture=True
).stdout.strip()
status = run_command(
["git", "-C", path, "status", "--porcelain"], check=True, capture=True
).stdout
snapshot.update({"head": head, "dirty": bool(status.strip())})
except (OSError, PipelineError):
snapshot.update({"head": None, "dirty": None})
return snapshot
def generator_arguments(generator: str, architecture: str) -> list[str]:
arguments = ["-G", generator]
if generator.lower().startswith("visual studio"):
arguments.extend(["-A", architecture])
return arguments
def mobilegl_configure_command(
repo_root: Path,
build_dir: Path,
generator: str,
architecture: str,
extra: Iterable[str],
) -> list[str]:
return [
"cmake",
"-S",
str(repo_root),
"-B",
str(build_dir),
*generator_arguments(generator, architecture),
"-DMOBILEGL_BUILD_TEST=OFF",
"-DMOBILEGL_BUILD_BENCHMARK=OFF",
"-DMOBILEGL_BUILD_TRACE_REPLAY=OFF",
"-DMOBILEGL_ENABLE_TRACY=OFF",
"-DMOBILEGL_FORCE_RELEASE_OPT=ON",
*extra,
]
def cts_configure_command(
cts_source: Path,
build_dir: Path,
generator: str,
architecture: str,
extra: Iterable[str],
) -> list[str]:
return [
"cmake",
"-S",
str(cts_source),
"-B",
str(build_dir),
*generator_arguments(generator, architecture),
"-DDEQP_TARGET=default",
"-DDEQP_SUPPORT_DRM=OFF",
*extra,
]
def build_command(build_dir: Path, configuration: str, target: str, jobs: int) -> list[str]:
command = ["cmake", "--build", str(build_dir), "--config", configuration, "--target", target]
if jobs > 0:
command.extend(["--parallel", str(jobs)])
return command
def verify_mobilegl_sources(repo_root: Path) -> None:
required = (
repo_root / "CMakeLists.txt",
repo_root / "MobileGL" / "MG_Impl" / "WGLImpl" / "WGLImpl.cpp",
repo_root / "3rdparty" / "glslang" / "CMakeLists.txt",
repo_root / "3rdparty" / "SPIRV-Cross" / "CMakeLists.txt",
repo_root / "3rdparty" / "Vulkan-Headers" / "CMakeLists.txt",
)
missing = [str(path) for path in required if not path.is_file()]
if missing:
raise PipelineError(
"MobileGL source/submodules are incomplete:\n "
+ "\n ".join(missing)
+ f"\nRun: git -C {repo_root} submodule update --init --recursive"
)
def verify_cts_sources(cts_source: Path) -> None:
required = (
cts_source / "CMakeLists.txt",
cts_source / "external" / "openglcts" / "CMakeLists.txt",
)
missing = [str(path) for path in required if not path.is_file()]
if missing:
raise PipelineError(
"VK-GL-CTS source/external packages are incomplete:\n "
+ "\n ".join(missing)
+ f"\nRun: {sys.executable} {cts_source / 'external' / 'fetch_sources.py'}"
)
def discover_mobilegl_dll(build_dir: Path, configuration: str) -> Path:
preferred = (
build_dir / configuration / "opengl32.dll",
build_dir / "MobileGL" / configuration / "opengl32.dll",
build_dir / "opengl32.dll",
)
for candidate in preferred:
if candidate.is_file():
return candidate.resolve()
candidates = sorted({path.resolve() for path in build_dir.rglob("opengl32.dll") if path.is_file()})
if len(candidates) == 1:
return candidates[0]
if not candidates:
raise PipelineError(f"MobileGL build produced no opengl32.dll under {build_dir}")
raise PipelineError("multiple opengl32.dll candidates; pass --mobilegl-dll explicitly:\n " + "\n ".join(map(str, candidates)))
def discover_glcts_exe(build_dir: Path, configuration: str) -> Path:
preferred = (
build_dir / "external" / "openglcts" / "modules" / configuration / "glcts.exe",
build_dir / "external" / "openglcts" / "modules" / "glcts.exe",
)
for candidate in preferred:
if candidate.is_file():
return candidate.resolve()
candidates = sorted({path.resolve() for path in build_dir.rglob("glcts.exe") if path.is_file()})
if len(candidates) == 1:
return candidates[0]
if not candidates:
raise PipelineError(f"CTS build produced no glcts.exe under {build_dir}")
raise PipelineError("multiple glcts.exe candidates; pass --glcts-exe explicitly:\n " + "\n ".join(map(str, candidates)))
def default_cts_modules_dir(cts_build_dir: Path) -> Path:
return cts_build_dir / "external" / "openglcts" / "modules"
def find_caselist_root(cts_modules_dir: Path, cts_source: Path) -> Path:
relative = Path("gl_cts/data/mustpass/gl/khronos_mustpass/main")
candidates = (cts_modules_dir / relative, cts_source / "external" / "openglcts" / "modules" / relative)
for candidate in candidates:
if candidate.is_dir():
return candidate.resolve()
raise PipelineError("Khronos GL mustpass directory was not found; checked:\n " + "\n ".join(map(str, candidates)))
def caselist_for(caselist_root: Path, version: str) -> Path:
return require_file(caselist_root / f"gl{version}-main.txt", f"GL{version} mustpass caselist")
def runtime_source_files(
glcts_exe: Path,
mobilegl_dll: Path,
backends: Sequence[str],
angle_dir: Optional[Path],
) -> dict[str, Path]:
files = {"glcts.exe": glcts_exe, "opengl32.dll": mobilegl_dll}
if "DirectGLES" in backends:
if angle_dir is None:
raise PipelineError("--angle-dir is required when DirectGLES is selected")
angle_dir = require_directory(angle_dir, "ANGLE runtime directory")
for name in ANGLE_REQUIRED_DLLS:
files[name] = require_file(angle_dir / name, f"ANGLE {name}")
for name in ANGLE_OPTIONAL_DLLS:
candidate = angle_dir / name
if candidate.is_file():
files[name] = candidate.resolve()
return files
def runtime_fingerprint(files: Mapping[str, Path]) -> tuple[str, dict[str, str]]:
hashes = {name: sha256_file(path) for name, path in sorted(files.items())}
digest = hashlib.sha256()
for name, file_hash in hashes.items():
digest.update(f"{name}\0{file_hash}\n".encode("utf-8"))
return digest.hexdigest(), hashes
def run_fingerprint(
runtime_hash: str,
cts_data_hash: str,
tool_hashes: Mapping[str, str],
caselist_hashes: Mapping[str, str],
deqp_args: Sequence[str],
environment: Mapping[str, str],
result_semantics: Optional[Mapping[str, object]] = None,
) -> str:
identity = {
"version": 2,
"runtime_fingerprint": runtime_hash,
"cts_data_sha256": cts_data_hash,
"tool_hashes": dict(sorted(tool_hashes.items())),
"caselist_hashes": dict(sorted(caselist_hashes.items())),
"deqp_args": list(deqp_args),
"environment": dict(sorted(environment.items())),
"result_semantics": dict(sorted((result_semantics or {}).items())),
}
return hashlib.sha256(
json.dumps(identity, sort_keys=True, separators=(",", ":")).encode("utf-8")
).hexdigest()
def assemble_runtime(
work_root: Path,
sources: Mapping[str, Path],
fingerprint: str,
hashes: Mapping[str, str],
) -> Path:
runtime_dir = work_root / "runtime" / fingerprint[:16]
runtime_dir.mkdir(parents=True, exist_ok=True)
for name, source in sources.items():
require_x64_pe(source, name)
destination = runtime_dir / name
if source.resolve() != destination.resolve():
shutil.copy2(source, destination)
if sha256_file(destination) != hashes[name]:
raise PipelineError(f"runtime copy hash mismatch: {destination}")
manifest = {
"version": 1,
"created_utc": utc_now(),
"fingerprint": fingerprint,
"files": {
name: {"source": str(source), "sha256": hashes[name]}
for name, source in sorted(sources.items())
},
}
write_json(runtime_dir / "manifest.json", manifest)
return runtime_dir
def write_json(path: Path, value: object) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + ".tmp")
temporary.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8")
os.replace(temporary, path)
def read_cases(caselist: Path) -> list[str]:
cases: list[str] = []
for raw_line in caselist.read_text(encoding="utf-8-sig").splitlines():
line = raw_line.strip()
if line and not line.startswith("#"):
cases.append(line)
if not cases:
raise PipelineError(f"caselist contains no cases: {caselist}")
return cases
def choose_preflight_case(caselist: Path) -> str:
cases = read_cases(caselist)
preferred_suffixes = (
".buffer_objects.gen_buffers",
".CommonBugs.CommonBug_GetProgramivActiveUniformBlockMaxNameLength",
)
for suffix in preferred_suffixes:
for case in cases:
if case.endswith(suffix):
return case
return cases[0]
def runner_command(
runner: Path,
runtime_exe: Path,
workdir: Path,
caselist: Path,
outdir: Path,
backend: str,
idle_timeout: float,
max_round_seconds: float,
max_rounds: int,
environment: Mapping[str, str],
deqp_args: Sequence[str],
run_identity: Optional[str] = None,
) -> list[str]:
command = [
sys.executable,
str(runner),
"--exe",
str(runtime_exe),
"--workdir",
str(workdir),
"--caselist",
str(caselist),
"--outdir",
str(outdir),
"--backend",
backend,
"--idle-timeout",
str(idle_timeout),
"--max-round-seconds",
str(max_round_seconds),
"--max-rounds",
str(max_rounds),
]
if run_identity is not None:
command.extend(["--run-identity", run_identity])
for name, value in sorted(environment.items()):
command.extend(["--env", f"{name}={value}"])
command.extend(f"--deqp-arg={argument}" for argument in deqp_args)
return command
def parse_preflight_identity(
qpa_files: Sequence[Path],
mobilegl_log: Path,
backend: str,
minimum_version: tuple[int, int],
) -> dict[str, object]:
if not qpa_files:
raise PipelineError(f"{backend} preflight produced no QPA file")
qpa_text = "\n".join(path.read_text(encoding="utf-8", errors="replace") for path in qpa_files)
vendors = SESSION_VENDOR.findall(qpa_text)
renderers = SESSION_RENDERER.findall(qpa_text)
command_lines = SESSION_COMMAND_LINE.findall(qpa_text)
if not vendors or "MobileGL" not in vendors[-1]:
raise PipelineError(f"{backend} preflight did not load MobileGL (vendor={vendors[-1] if vendors else None!r})")
expected_renderer = "Espryt" if backend == "DirectGLES" else "Magma"
if not renderers or expected_renderer not in renderers[-1]:
raise PipelineError(
f"{backend} preflight renderer mismatch: expected {expected_renderer!r}, "
f"got {renderers[-1] if renderers else None!r}"
)
if not command_lines or "--deqp-gl-context-type=wgl" not in command_lines[-1]:
raise PipelineError(f"{backend} preflight did not record a WGL context")
if not mobilegl_log.is_file():
raise PipelineError(f"{backend} preflight did not create MobileGL log: {mobilegl_log}")
log_text = mobilegl_log.read_text(encoding="utf-8", errors="replace")
versions = [(int(major), int(minor)) for major, minor in TARGET_GL_VERSION.findall(log_text)]
if not versions:
raise PipelineError(f"{backend} preflight log contains no target OpenGL version")
actual_version = versions[-1]
if actual_version < minimum_version:
raise PipelineError(
f"{backend} reports GL {actual_version[0]}.{actual_version[1]}, "
f"but selected suites require at least {minimum_version[0]}.{minimum_version[1]}"
)
return {
"backend": backend,
"vendor": vendors[-1],
"renderer": renderers[-1],
"target_gl_version": f"{actual_version[0]}.{actual_version[1]}",
"qpa_files": [str(path) for path in qpa_files],
"mobilegl_log": str(mobilegl_log),
}
def run_preflight(
*,
runner: Path,
runtime_exe: Path,
cts_modules_dir: Path,
caselist: Path,
preflight_root: Path,
backend: str,
idle_timeout: float,
environment: Mapping[str, str],
deqp_args: Sequence[str],
minimum_version: tuple[int, int],
run_identity: str,
) -> dict[str, object]:
outdir = preflight_root / backend.lower()
outdir.mkdir(parents=True, exist_ok=True)
case_file = outdir / "case.txt"
case_file.write_text(choose_preflight_case(caselist) + "\n", encoding="utf-8")
log_path = outdir / "mobilegl.log"
child_environment = dict(environment)
child_environment["MOBILEGL_LOG_FILE_PATH"] = str(log_path)
command = runner_command(
runner,
runtime_exe,
cts_modules_dir,
case_file,
outdir,
backend,
min(idle_timeout, 120.0) if idle_timeout > 0 else 120.0,
180.0,
1,
child_environment,
deqp_args,
run_identity,
)
# A developing driver may fail the chosen case or terminate during deinit.
# Identity is the gate: the QPA and MobileGL log must prove which WGL driver ran.
run_command(command, cwd=repository_root(), check=False)
identity = parse_preflight_identity(sorted(outdir.glob("chunk*.qpa")), log_path, backend, minimum_version)
print(
f"[wgl_glcts_pipeline] preflight {backend}: {identity['renderer']} | "
f"GL {identity['target_gl_version']}"
)
return identity
def report_command(
reporter: Path,
suites: Sequence[tuple[str, str, Path, Path]],
markdown: Path,
json_out: Path,
allow_incomplete: bool,
expected_run_identity: Optional[str] = None,
) -> list[str]:
command = [sys.executable, str(reporter)]
for backend, label, caselist, results in suites:
command.extend(["--suite", backend, label, str(caselist), str(results)])
command.extend(["--markdown", str(markdown), "--json", str(json_out)])
if expected_run_identity is not None:
command.extend(["--expected-run-identity", expected_run_identity])
if allow_incomplete:
command.append("--allow-incomplete")
return command
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Build MobileGL WGL, assemble a local glcts runtime, run GL30-GL46, and report results."
)
parser.add_argument("--repo-root", type=Path, default=repository_root(), help="MobileGL worktree root")
parser.add_argument("--cts-source", type=Path, required=True, help="VK-GL-CTS source checkout")
parser.add_argument("--work-root", type=Path, required=True, help="build/result root (kept outside source)")
parser.add_argument("--angle-dir", type=Path, help="x64 ANGLE directory for DirectGLES")
parser.add_argument("--backends", nargs="+", choices=SUPPORTED_BACKENDS, default=list(SUPPORTED_BACKENDS))
parser.add_argument("--versions", nargs="+", type=normalize_version, default=list(SUPPORTED_VERSIONS))
parser.add_argument("--configuration", default="Release")
parser.add_argument("--generator", default="Visual Studio 17 2022")
parser.add_argument("--architecture", default="x64")
parser.add_argument("--jobs", type=int, default=max(1, os.cpu_count() or 1))
parser.add_argument("--mobilegl-build-dir", type=Path)
parser.add_argument("--cts-build-dir", type=Path)
parser.add_argument("--mobilegl-dll", type=Path, help="reuse an existing MobileGL/opengl32 DLL")
parser.add_argument("--glcts-exe", type=Path, help="reuse an existing glcts.exe")
parser.add_argument("--cts-modules-dir", type=Path, help="glcts working directory containing gl_cts data")
parser.add_argument("--skip-mobilegl-build", action="store_true")
parser.add_argument("--skip-cts-build", action="store_true")
parser.add_argument("--skip-preflight", action="store_true")
parser.add_argument("--skip-run", action="store_true")
parser.add_argument("--skip-report", action="store_true")
parser.add_argument("--allow-incomplete-report", action="store_true")
parser.add_argument("--continue-on-suite-error", action="store_true")
parser.add_argument("--idle-timeout", type=float, default=300.0)
parser.add_argument("--max-round-seconds", type=float, default=0.0)
parser.add_argument("--max-rounds", type=int, default=10000)
parser.add_argument("--env", action="append", type=parse_assignment, default=[], metavar="NAME=VALUE")
parser.add_argument(
"--deqp-arg", action="append", default=[], metavar="ARG", help="extra glcts option; use --deqp-arg=--x=y"
)
parser.add_argument(
"--mobilegl-cmake-arg", action="append", default=[], metavar="ARG", help="extra MobileGL configure option"
)
parser.add_argument("--cts-cmake-arg", action="append", default=[], metavar="ARG", help="extra CTS configure option")
return parser
def execute(args: argparse.Namespace) -> int:
if os.name != "nt":
raise PipelineError("this pipeline builds and exercises the Windows WGL target and must run on Windows")
if shutil.which("cmake") is None and (not args.skip_mobilegl_build or not args.skip_cts_build):
raise PipelineError("cmake was not found on PATH")
if args.jobs < 0 or args.max_rounds <= 0:
raise PipelineError("--jobs must be >= 0 and --max-rounds must be > 0")
repo_root = require_directory(args.repo_root, "MobileGL worktree")
cts_source = require_directory(args.cts_source, "VK-GL-CTS checkout")
work_root = args.work_root.expanduser().resolve()
work_root.mkdir(parents=True, exist_ok=True)
versions = list(dict.fromkeys(args.versions))
backends = list(dict.fromkeys(args.backends))
minimum_version = max(gl_version_tuple(version) for version in versions)
extra_environment = canonicalize_windows_environment(args.env)
reserved_environment = CONTROLLED_ENVIRONMENT_NAMES & set(extra_environment)
if reserved_environment:
raise PipelineError("the pipeline controls these environment variables: " + ", ".join(sorted(reserved_environment)))
configuration = args.configuration
mobilegl_build_dir = (args.mobilegl_build_dir or work_root / f"mobilegl-build-{configuration.lower()}").resolve()
cts_build_dir = (args.cts_build_dir or work_root / f"cts-build-wgl-{configuration.lower()}").resolve()
if not args.skip_mobilegl_build:
verify_mobilegl_sources(repo_root)
mobilegl_build_dir.mkdir(parents=True, exist_ok=True)
run_command(
mobilegl_configure_command(
repo_root, mobilegl_build_dir, args.generator, args.architecture, args.mobilegl_cmake_arg
),
cwd=repo_root,
)
run_command(build_command(mobilegl_build_dir, configuration, "MobileGL", args.jobs), cwd=repo_root)
mobilegl_dll = (
require_file(args.mobilegl_dll, "MobileGL DLL")
if args.mobilegl_dll
else discover_mobilegl_dll(mobilegl_build_dir, configuration)
)
if not args.skip_cts_build:
verify_cts_sources(cts_source)
cts_build_dir.mkdir(parents=True, exist_ok=True)
run_command(
cts_configure_command(cts_source, cts_build_dir, args.generator, args.architecture, args.cts_cmake_arg),
cwd=cts_source,
)
run_command(build_command(cts_build_dir, configuration, "glcts", args.jobs), cwd=cts_source)
glcts_exe = (
require_file(args.glcts_exe, "glcts executable")
if args.glcts_exe
else discover_glcts_exe(cts_build_dir, configuration)
)
cts_modules_dir = require_directory(
args.cts_modules_dir or default_cts_modules_dir(cts_build_dir), "CTS modules/working directory"
)
cts_data_dir = require_directory(cts_modules_dir / "gl_cts" / "data", "CTS gl_cts data directory")
cts_data_hash = sha256_directory(cts_data_dir)
caselist_root = find_caselist_root(cts_modules_dir, cts_source)
caselists = {version: caselist_for(caselist_root, version) for version in versions}
runtime_sources = runtime_source_files(glcts_exe, mobilegl_dll, backends, args.angle_dir)
fingerprint, hashes = runtime_fingerprint(runtime_sources)
runtime_dir = assemble_runtime(work_root, runtime_sources, fingerprint, hashes)
runtime_exe = runtime_dir / "glcts.exe"
runner = require_file(repo_root / "tools" / "cts" / "scripts" / "run_cts_windows.py", "Windows CTS runner")
reporter = require_file(repo_root / "tools" / "cts" / "scripts" / "cts_multi_report.py", "CTS reporter")
matrix_reporter = require_file(
repo_root / "tools" / "cts" / "scripts" / "cts_matrix_report.py", "CTS matrix reporter"
)
qpa_reporter = require_file(repo_root / "tools" / "cts" / "scripts" / "qpa_report.py", "QPA parser")
tool_hashes = {
"wgl_glcts_pipeline.py": sha256_file(Path(__file__).resolve()),
"run_cts_windows.py": sha256_file(runner),
"cts_multi_report.py": sha256_file(reporter),
"cts_matrix_report.py": sha256_file(matrix_reporter),
"qpa_report.py": sha256_file(qpa_reporter),
}
deqp_args = [*DEFAULT_DEQP_ARGS, *args.deqp_arg]
caselist_hashes = {version: sha256_file(path) for version, path in caselists.items()}
ambient_environment, identity_environment = tracked_run_environment(
os.environ, extra_environment
)
result_semantics = {
"idle_timeout_seconds": args.idle_timeout,
"max_round_seconds": args.max_round_seconds,
}
execution_settings = {
**result_semantics,
"max_rounds": args.max_rounds,
"continue_on_suite_error": args.continue_on_suite_error,
}
execution_fingerprint = run_fingerprint(
fingerprint,
cts_data_hash,
tool_hashes,
caselist_hashes,
deqp_args,
identity_environment,
result_semantics,
)
run_root = work_root / "runs" / execution_fingerprint[:16]
report_root = run_root / "reports"
manifest = {
"version": 1,
"created_utc": utc_now(),
"run_fingerprint": execution_fingerprint,
"runtime_fingerprint": fingerprint,
"runtime_hashes": hashes,
"tool_hashes": tool_hashes,
"cts_data": {"path": str(cts_data_dir), "sha256": cts_data_hash},
"runtime_dir": str(runtime_dir),
"mobilegl": git_snapshot(repo_root),
"vk_gl_cts": git_snapshot(cts_source),
"configuration": configuration,
"generator": args.generator,
"architecture": args.architecture,
"backends": backends,
"versions": versions,
"caselists": {version: {"path": str(path), "sha256": caselist_hashes[version]} for version, path in caselists.items()},
"deqp_args": deqp_args,
"environment_overrides": extra_environment,
"ambient_environment": ambient_environment,
"identity_environment": identity_environment,
"result_semantics": result_semantics,
"execution_settings": execution_settings,
}
write_json(run_root / "manifest.json", manifest)
print(f"[wgl_glcts_pipeline] runtime fingerprint: {fingerprint}")
print(f"[wgl_glcts_pipeline] run fingerprint: {execution_fingerprint}")
print(f"[wgl_glcts_pipeline] run root: {run_root}")
identities: list[dict[str, object]] = []
if not args.skip_preflight:
first_caselist = caselists[versions[0]]
preflight_root = run_root / "preflight"
for backend in backends:
identities.append(
run_preflight(
runner=runner,
runtime_exe=runtime_exe,
cts_modules_dir=cts_modules_dir,
caselist=first_caselist,
preflight_root=preflight_root,
backend=backend,
idle_timeout=args.idle_timeout,
environment=extra_environment,
deqp_args=deqp_args,
minimum_version=minimum_version,
run_identity=execution_fingerprint,
)
)
manifest["preflight"] = identities
write_json(run_root / "manifest.json", manifest)
suites: list[tuple[str, str, Path, Path]] = []
suite_errors: list[dict[str, object]] = []
for backend in backends:
for version in versions:
label = f"gl{version}"
result_dir = run_root / "results" / backend.lower() / label
suites.append((backend, label, caselists[version], result_dir))
if args.skip_run:
continue
result_dir.mkdir(parents=True, exist_ok=True)
child_environment = dict(extra_environment)
child_environment["MOBILEGL_LOG_FILE_PATH"] = str(result_dir / "mobilegl.log")
command = runner_command(
runner,
runtime_exe,
cts_modules_dir,
caselists[version],
result_dir,
backend,
args.idle_timeout,
args.max_round_seconds,
args.max_rounds,
child_environment,
deqp_args,
execution_fingerprint,
)
completed = run_command(command, cwd=repo_root, check=False)
if completed.returncode != 0:
suite_errors.append({"backend": backend, "suite": label, "returncode": completed.returncode})
if not args.continue_on_suite_error:
break
if suite_errors and not args.continue_on_suite_error:
break
report_returncode: Optional[int] = None
if not args.skip_report:
report_root.mkdir(parents=True, exist_ok=True)
completed = run_command(
report_command(
reporter,
suites,
report_root / "gl-cts-summary.md",
report_root / "gl-cts-summary.json",
args.allow_incomplete_report or bool(suite_errors),
execution_fingerprint,
),
cwd=repo_root,
check=False,
)
report_returncode = completed.returncode
manifest["suite_errors"] = suite_errors
manifest["report_returncode"] = report_returncode
manifest["finished_utc"] = utc_now()
write_json(run_root / "manifest.json", manifest)
if suite_errors:
print(f"[wgl_glcts_pipeline] {len(suite_errors)} suite runner(s) incomplete; see manifest/report", file=sys.stderr)
returncodes = {int(item["returncode"]) for item in suite_errors}
if 130 in returncodes:
return 130
if 2 in returncodes:
return 2
if 3 in returncodes:
return 3
return 4
if report_returncode is not None and report_returncode != 0:
print(f"[wgl_glcts_pipeline] report validation failed with exit code {report_returncode}", file=sys.stderr)
return report_returncode
return 0
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
try:
return execute(args)
except PipelineError as exc:
print(f"[wgl_glcts_pipeline] ERROR: {exc}", file=sys.stderr)
return 2
except OSError as exc:
print(f"[wgl_glcts_pipeline] ERROR: filesystem/process operation failed: {exc}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -16,3 +16,4 @@ Each skill is a self-contained package, matching the layout used by
| Skill | What it does |
| --- | --- |
| [gl-cts-on-mobilegl](gl-cts-on-mobilegl/SKILL.md) | Build VK-GL-CTS `glcts` as a standalone Android arm64 binary against MobileGL's own EGL, run KHR-GL33, and report a per-backend OpenGL 3.3 core conformance rate. |
| [wgl-gl-cts-on-mobilegl](wgl-gl-cts-on-mobilegl/SKILL.md) | Build MobileGL's Windows x64 WGL drop-in, run GL30-GL46 core CTS against DirectGLES and DirectVulkan, resume safely, and emit validated reports. |
@@ -0,0 +1,143 @@
---
name: wgl-gl-cts-on-mobilegl
description: Build MobileGL as a Windows x64 WGL drop-in opengl32.dll, build or reuse VK-GL-CTS glcts, run Khronos GL30 through GL46 core suites against DirectGLES and DirectVulkan, resume after crashes or idle timeouts, and produce validated Markdown/JSON conformance reports. Use when Codex needs to compile MobileGL's WGL target, connect it to desktop OpenGL CTS on Windows, rerun selected GL core mustpass lists, or verify that CTS loaded MobileGL instead of the system OpenGL driver.
---
# WGL OpenGL CTS on MobileGL
Use `tools/cts/scripts/wgl_glcts_pipeline.py` as the single entry point. It
configures both Visual Studio builds, assembles a private runtime, verifies the
loaded WGL implementation, calls the crash-resuming runner, and generates the
multi-suite report.
## Prerequisites
- Run on Windows x64 with Git, Python 3.9 or newer, CMake, Visual Studio 2022's
Desktop C++ workload, and a Vulkan SDK visible to MobileGL's CMake configure.
- Use the already selected MobileGL worktree. Inspect `git status` first and do
not create another worktree unless the user explicitly asks.
- Initialize MobileGL submodules:
```powershell
git submodule update --init --recursive
```
- Prepare a VK-GL-CTS checkout at a citable release tag, then fetch its pinned
externals:
```powershell
git -C D:\VK-GL-CTS checkout opengl-cts-4.6.8.1
python D:\VK-GL-CTS\external\fetch_sources.py
```
- For DirectGLES, provide one x64 ANGLE directory containing matching
`libEGL.dll`, `libGLESv2.dll`, and `d3dcompiler_47.dll`. DirectVulkan does not
need ANGLE.
- For DirectVulkan, provide a working Vulkan loader plus a GPU-vendor ICD and
driver. The Vulkan SDK alone does not provide a usable GPU device.
## Run the full matrix
From the MobileGL worktree root:
```powershell
python tools\cts\scripts\wgl_glcts_pipeline.py `
--cts-source D:\VK-GL-CTS `
--work-root D:\MobileGL-WGL-CTS `
--angle-dir C:\path\to\angle-x64 `
--backends DirectGLES DirectVulkan `
--versions gl30 gl31 gl32 gl33 gl40 gl41 gl42 gl43 gl44 gl45 gl46
```
The defaults deliberately reproduce the proven Windows setup:
- Visual Studio 17 2022, x64, Release;
- VK-GL-CTS `DEQP_TARGET=default`, which selects desktop WGL on Windows;
- MobileGL copied beside `glcts.exe` as `opengl32.dll`;
- WGL context, FBO surface, `rgba8888d24s8`, hidden window, watchdog and crash
handler enabled;
- `--deqp-terminate-on-device-lost=disable` supplied by the underlying runner.
Do not replace the FBO surface with the default framebuffer when comparing
backends; it changes the readback path and invalidates comparison with the
established runs.
## Mandatory preflight
Leave preflight enabled for a new binary. It must prove all of the following
before the full suite starts:
- QPA vendor contains `MobileGL`;
- DirectGLES renderer contains `Espryt`, or DirectVulkan contains `Magma`;
- QPA command line records `--deqp-gl-context-type=wgl`;
- MobileGL's log reports a GL version at least as high as the highest selected
suite.
Treat any preflight failure as a hard stop. It commonly means `glcts.exe` loaded
the system `opengl32.dll`, ANGLE DLLs have the wrong architecture, or the driver
still reports too low a GL version.
## Common variants
Run DirectVulkan only, without ANGLE:
```powershell
python tools\cts\scripts\wgl_glcts_pipeline.py `
--cts-source D:\VK-GL-CTS --work-root D:\MobileGL-WGL-CTS `
--backends DirectVulkan --versions gl43 gl44 gl45 gl46
```
Build and preflight without starting a multi-hour CTS run:
```powershell
python tools\cts\scripts\wgl_glcts_pipeline.py `
--cts-source D:\VK-GL-CTS --work-root D:\MobileGL-WGL-CTS `
--angle-dir C:\path\to\angle-x64 `
--skip-run --skip-report
```
Reuse previously built binaries with `--skip-mobilegl-build`,
`--skip-cts-build`, `--mobilegl-dll`, `--glcts-exe`, and
`--cts-modules-dir`. Continue to use a modules directory containing the
`gl_cts` data tree; the executable directory alone is insufficient.
Pass extra dEQP options with the equals form so argparse does not consume the
leading dashes:
```powershell
--deqp-arg=--deqp-log-images=enable
```
## Resume and artifacts
Repeat the exact command and `--work-root` to resume. The runner recovers
completed QPA cases plus `crashed.txt` and `hung.txt`, then schedules only
unaccounted cases.
The pipeline prints full SHA-256 fingerprints and uses their first 16
hexadecimal characters as directory names: `runtime/<runtime-prefix>` and
`runs/<run-prefix>`. The run fingerprint covers the CTS data tree,
runner/report tools, caselists, dEQP arguments, explicit `--env` values, tracked
ambient GL/Vulkan environment, and the timeout settings that determine
Crash/Hang classification. The manifest records those inputs plus
`max_rounds`, suite-error continuation policy, source commits and dirty state,
preflight identity, suite errors, and report status.
The runner refuses to attach a new `run_state.json` to old QPA or sidecar files
by default. Invoke `run_cts_windows.py --adopt-legacy` directly only after
verifying that those artifacts match the backend, caselist, binaries, and dEQP
arguments. Pipeline reports require every suite state to match the current run
fingerprint.
Read the final outputs at:
```text
<work-root>/runs/<run-prefix>/reports/gl-cts-summary.md
<work-root>/runs/<run-prefix>/reports/gl-cts-summary.json
```
Use the exact run root printed by the pipeline.
Do not claim completeness when the report validation is incomplete or when the
manifest records suite errors. Keep `NotSupported` separate from hard failures
when prioritizing implementation work.
@@ -0,0 +1,4 @@
interface:
display_name: "WGL GL CTS on MobileGL"
short_description: "Build MobileGL WGL and run GL30-GL46 CTS on Windows"
default_prompt: "Use $wgl-gl-cts-on-mobilegl to build MobileGL's WGL DLL and run the selected OpenGL CTS suites on Windows."