swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-11-07 01:53:24 +09:00
Code Issues Packages Projects Releases Wiki Activity

[Perf] (P8-D, Magma): the wire arm issues glDraw*Indirect / glMultiDraw*Indirect[Count] as vkCmdDraw*Indirect[Count] from the wire store instead of expanding them on the CPU behind a GPU-idle wait (create-indirect split: 321 waits / ~8 s -> 0), with an INDIRECT_COMMAND_READ barrier once per shader write; windr[] PipeStats counters, WireIndirectDrawScenario on both backends x every arm, notes/p8/D.md

Browse Source
This commit is contained in:
swung0x48
2026-09-29 22:32:49 -04:00
parent 8d3e2317a6
commit d639e54616
15 changed files with 826 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MobileGL/MG_Backend/DirectVulkan/DirectVulkan.cpp
+24 -4
View File
@@ -370,16 +370,36 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#endif // !MOBILEGL_BUILD_DISAGGREGATED
#if MOBILEGL_BUILD_DISAGGREGATED
// The verb's handles identify server stores. Readback orders GPU-produced
// command/count bytes before CPU tier expansion; never inspect a client binding.
// The verb's handles identify server stores; never inspect a client binding.
//
// P8-D: THE GPU READS THE WORDS (VulkanRenderer::DrawWireIndirectNative, WireDraw.inc), as
// on the monolith arm. The CPU expansion below is kept for the one shape the monolith arm
// also expands - a COUNT form the device cannot issue natively (no VK_KHR_draw_indirect_count,
// or maxdrawcount > 1 without multiDrawIndirect: VulkanRenderer.cpp's
// MultiDrawElementsIndirectCount) - where ReadWireBuffer orders GPU-produced command/count
// bytes before the expansion.
void DrawWireIndirect(GLenum mode, GLenum type, const void* indirect, GLsizei drawcount,
GLsizei stride, Bool indexed, Bool counted = false, GLintptr countOffset = 0) {
if (drawcount <= 0) return;
if (drawcount <= 0 || countOffset < 0) return;
if (pVulkanRenderer->DrawWireIndirectNative(mode, type, reinterpret_cast<Uint64>(indirect), drawcount,
stride, indexed, counted, static_cast<Uint64>(countOffset)))
return;
auto& buffers = pVulkanRenderer->GetWireBufferManager();
const auto& state = MG_Pipe::MGPipeApplier();
// P8-D: this call reads its words on the CPU; the waits those reads take are its own.
struct ExpansionCount {
VkBufferManager& buffers;
const Uint64 waitsBefore = buffers.GetWireHostWaitCount();
~ExpansionCount() {
if (!MG_Util::PipeStats::Enabled()) return;
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireIndirectCpuExpansions, 1);
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireHostWaitsIndirect,
buffers.GetWireHostWaitCount() - waitsBefore);
}
} expansionCount{buffers};
if (counted) {
Uint32 count = 0;
if (countOffset < 0 || !buffers.ReadWireBuffer(state.VerbIndirectParameterBuffer,
if (!buffers.ReadWireBuffer(state.VerbIndirectParameterBuffer,
static_cast<Uint64>(countOffset), sizeof(count), &count)) return;
drawcount = static_cast<GLsizei>(std::min<Uint32>(count, static_cast<Uint32>(drawcount)));
if (!drawcount) return;
MobileGL/MG_Backend/DirectVulkan/Renderer/VkBufferManager.cpp
+23 -1
View File
@@ -22,6 +22,7 @@
#include <android/hardware_buffer.h>
#endif
#include <Config.h>
#include <chrono>
#include <cstdlib>
#endif
@@ -271,6 +272,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource->lastUseSerial = 0;
resource->lastUseSubmitIndex = 0; // M2 r2: the new store carries no submission yet (B3's stamp is per store)
resource->gpuWritesPending = false;
resource->indirectReadBarrierPending = false;
resource->stagedCoverage.clear();
++m_sliceEpochCounter;
if (resource->size == 0) return;
@@ -315,7 +317,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
barrier.dstAccessMask = VK_ACCESS_HOST_READ_BIT | VK_ACCESS_HOST_WRITE_BIT;
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_HOST_BIT,
0, 1, &barrier, 0, nullptr, 0, nullptr);
if (!pVulkanRenderer->WaitForSubmitIndex(pVulkanRenderer->GetSyncPointSubmitIndex(), UINT64_MAX, true)) {
// P8-D: counted here, past the idle early-out, so `whw` is waits TAKEN, not calls.
++m_wireHostWaits;
const Bool timed = MG_Util::PipeStats::Enabled();
const auto waitStart = timed ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{};
const Bool waited =
pVulkanRenderer->WaitForSubmitIndex(pVulkanRenderer->GetSyncPointSubmitIndex(), UINT64_MAX, true);
if (timed) {
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireHostWaits, 1);
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireHostWaitMicros,
static_cast<Uint64>(std::chrono::duration_cast<std::chrono::microseconds>(
std::chrono::steady_clock::now() - waitStart).count()));
}
if (!waited) {
return false;
}
resource.lastUseSerial = 0;
@@ -594,6 +608,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource->lastUseSerial = m_frameSerial;
resource->stagedCoverage.clear();
resource->gpuWritesPending = true;
resource->indirectReadBarrierPending = true;
++m_sliceEpochCounter;
// The existing client dirty-state protocol accepts one whole-store
// notification. The store remains on the GPU and exact later SubData
@@ -606,6 +621,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MG_Pipe::gMGPipeCallbacks.OnGpuWritten(res, 1, &range);
}
Bool VkBufferManager::TakeWireIndirectReadBarrier(MG_Pipe::MGPipeHandle res) {
auto* resource = FindWireBuffer(res);
if (!resource || !resource->indirectReadBarrierPending) return false;
resource->indirectReadBarrierPending = false;
return true;
}
// ---- P11 B2: T0 - THE CLIENT'S AHardwareBuffer AS A WIRE STORE -----------------------------
// The serial a T0 store is released under: its last use, or - when no GPU command has named it
MobileGL/MG_Backend/DirectVulkan/Renderer/VkBufferManager.h
+13
View File
@@ -125,6 +125,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool CopyWireBufferSubWordRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
Uint32 frameIndex, const BufferSlice& dst, Uint64 dstSkip);
void MarkWireBufferGpuWritten(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size);
// P8-D: whether a shader write was marked on this store since the last indirect draw
// read it natively, clearing the mark. The caller records the INDIRECT_COMMAND_READ
// barrier before the draw when this answers true. Transfer writes need no mark: their own
// after-barriers already make them visible to MEMORY_READ at ALL_COMMANDS.
Bool TakeWireIndirectReadBarrier(MG_Pipe::MGPipeHandle res);
// P8-D: every WaitForWireBufferHostAccess that actually waited, over this manager's life -
// the attribution base a caller reads before and after its own ReadWireBuffer.
Uint64 GetWireHostWaitCount() const { return m_wireHostWaits; }
// Resource-op entry points. All run on the server apply owner.
void CreateWireBuffer(MG_Pipe::MGPipeHandle res, const MG_Pipe::MGPResourceDesc& desc);
@@ -257,7 +265,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// P11 B2 (T0): `buffer` is the client's imported AHardwareBuffer. Its readback posts
// no bytes (the client reads its own pages once the wait below it is done).
Bool imported = false;
// P8-D: a shader write was marked (MarkWireBufferGpuWritten) and no indirect draw has
// recorded its INDIRECT_COMMAND_READ barrier since (TakeWireIndirectReadBarrier).
Bool indirectReadBarrierPending = false;
};
// P8-D: see GetWireHostWaitCount.
Uint64 m_wireHostWaits = 0;
static Uint64 WireBufferKey(MG_Pipe::MGPipeHandle res) {
return (static_cast<Uint64>(res.Gen) << 32) | res.Slot;
}
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.cpp
+13
View File
@@ -3582,6 +3582,9 @@ void main() {
m_device = VK_NULL_HANDLE;
}
s_vkCmdDrawIndexedIndirectCount = nullptr;
#if MOBILEGL_BUILD_DISAGGREGATED
s_vkCmdWireDrawIndirectCount = nullptr;
#endif
s_vkCmdDrawMultiEXT = nullptr;
s_vkCmdDrawMultiIndexedEXT = nullptr;
@@ -15933,6 +15936,16 @@ void main() {
MGLOG_W("VK_KHR_draw_indirect_count enabled but vkCmdDrawIndexedIndirectCount entry point is missing, will continue as if VK_KHR_draw_indirect_count is not supported!");
m_drawIndirectCountExtensionEnabled = false;
}
#if MOBILEGL_BUILD_DISAGGREGATED
// P8-D: the wire arm's glMultiDrawArraysIndirectCount. Null = that verb reads its count on
// the CPU (WireDraw.inc's DrawWireIndirectNative answers false for it).
s_vkCmdWireDrawIndirectCount = reinterpret_cast<PFNDrawIndexedIndirectCountFunc>(
vkGetDeviceProcAddr(m_device, "vkCmdDrawIndirectCountKHR"));
if (s_vkCmdWireDrawIndirectCount == nullptr) {
s_vkCmdWireDrawIndirectCount = reinterpret_cast<PFNDrawIndexedIndirectCountFunc>(
vkGetDeviceProcAddr(m_device, "vkCmdDrawIndirectCount"));
}
#endif
s_vkCmdDrawMultiEXT = nullptr;
s_vkCmdDrawMultiIndexedEXT = nullptr;
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.h
+18
View File
@@ -160,6 +160,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// GetSyncPointSubmitIndex(), which answers m_submitCounter when nothing is recorded -
// true for a fence taken at that instant, wrong for work about to be recorded.
Uint64 GetWireNextSubmitIndex() const { return m_submitCounter + 1; }
// P8-D: the wire arm's indirect draw families, issued as vkCmdDraw[Indexed]Indirect[Count]
// from the wire stores MGPipeApplier().VerbIndirectBuffer / VerbIndirectParameterBuffer
// name. `offset` / `countOffset` are byte offsets into those stores; `stride` 0 means
// tightly packed. FALSE = this device cannot issue the COUNT form natively (the monolith
// arm's own condition, see the definition in WireDraw.inc) and the caller must read the
// words on the CPU; TRUE = issued, or declined by name.
Bool DrawWireIndirectNative(GLenum mode, GLenum type, Uint64 offset, GLsizei drawcount, GLsizei stride,
Bool indexed, Bool counted, Uint64 countOffset);
// P11 B2 (T0): what importing a client's AHardwareBuffer as a wire buffer store needs from
// this device. False when the device did not take VK_ANDROID_external_memory_android_
// hardware_buffer at creation (not Android, not advertised, or a monolith device).
@@ -680,6 +688,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void CollectWireObjects(Uint64 completedSubmit, Bool all = false);
Vector<WireRetiredObjects> m_wireRetiredObjects;
Uint32 m_wirePreparationDepth = 0;
// P8-D: set only while DrawWireIndirectNative is inside SetupDraw, so SetupWireDraw can
// tell the wire arm's own indirect draw from a monolith indirect entry point reached under
// a transport (still the `buffer-legacy-arm` role violation).
Bool m_wireNativeIndirectDraw = false;
void DispatchWireCompute(GLuint x, GLuint y, GLuint z);
void RewindWireDescriptorSetsIfDue();
UniquePtr<RenderPassEntry> m_wireDrawPass;
@@ -921,6 +933,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkDeviceSize countBufferOffset, Uint32 maxDrawCount,
Uint32 stride);
static inline PFNDrawIndexedIndirectCountFunc s_vkCmdDrawIndexedIndirectCount = nullptr;
#if MOBILEGL_BUILD_DISAGGREGATED
// P8-D: vkCmdDrawIndirectCount (same signature, same extension), for the wire arm's
// glMultiDrawArraysIndirectCount. The monolith arm never loads it: it reads that count on
// the CPU (DirectVulkan.cpp's MultiDrawArraysIndirectCount).
static inline PFNDrawIndexedIndirectCountFunc s_vkCmdWireDrawIndirectCount = nullptr;
#endif
// VK_EXT_multi_draw entry points, loaded at device creation when the extension
// (and its multiDraw feature) is enabled; null otherwise.
static inline PFN_vkCmdDrawMultiEXT s_vkCmdDrawMultiEXT = nullptr;
MobileGL/MG_Backend/DirectVulkan/Renderer/WireDeclines.def
+6
View File
@@ -104,3 +104,9 @@ MGL_WIRE_DECLINE(ShapeCreateImageFailed)
// Appended by the B3 review round: the preserve-copy arm of a reallocation was the one exit in
// SyncWireTextureShape still leaving no trace at all.
MGL_WIRE_DECLINE(ShapePreserveCopyFailed)
// ---- VulkanRenderer::DrawWireIndirectNative (P8-D), in statement order -----------------------
MGL_WIRE_DECLINE(IndirectBufferUnbound)
MGL_WIRE_DECLINE(IndirectCommandRange)
MGL_WIRE_DECLINE(IndirectCountRange)
MGL_WIRE_DECLINE(IndirectIndexStore)
MobileGL/MG_Backend/DirectVulkan/Renderer/WireDraw.inc
+138 -3
View File
@@ -207,7 +207,10 @@
MGL_WIRE_DECLINE_AT(ProgramLinkOrSpirv, "program is not linked or carries no SPIR-V");
return false;
}
if (aspects & DrawSetupAspect::IndirectDrawBuffer)
// P8-D: the wire arm's own indirect draw (DrawWireIndirectNative) is the one caller that
// may pass this aspect; a monolith indirect entry point reached here is still the role
// violation.
if ((aspects & DrawSetupAspect::IndirectDrawBuffer) && !m_wireNativeIndirectDraw)
WireBufferLegacyFatal();
MakeXfbWritesVisible();
const auto* fbo = state.DrawFramebuffer();
@@ -310,8 +313,10 @@
const SizeT stride = static_cast<SizeT>(attr.Stride);
SizeT count = stride ? 1 + (static_cast<SizeT>(slice.size - offset) - elementSize) / stride : 1;
// Direct non-indexed draws bound the conversion; indexed/multi-draw may
// name arbitrary vertices, so preserve the entire valid source stream.
if (stride && !(aspects & DrawSetupAspect::IndexBuffer) && !resource.Divisor)
// name arbitrary vertices, so preserve the entire valid source stream. So may an
// indirect draw (P8-D): its vertex range is in GPU words nobody read.
if (stride && !(aspects & DrawSetupAspect::IndexBuffer) &&
!(aspects & DrawSetupAspect::IndirectDrawBuffer) && !resource.Divisor)
count = std::min<SizeT>(count, static_cast<SizeT>(drawParams.firstVertex) + drawParams.vertexCount);
if (!count) {
MGL_WIRE_DECLINE_AT(VertexStreamCountZero, "the converted vertex stream would hold no elements");
@@ -601,6 +606,136 @@
return true;
}
// ---- P8-D: THE WIRE ARM'S INDIRECT DRAWS READ THEIR WORDS ON THE GPU ----------------------
//
// The shape the monolith arm records (VulkanRenderer.cpp's MultiDraw{Elements,Arrays}Indirect
// and MultiDrawElementsIndirectCount), with the wire store's own VkBuffer in place of the
// frontend buffer's resident slice. Before this, every wire indirect draw read its commands
// through ReadWireBuffer and expanded them on the CPU, and a store a shader had written made
// that read a whole-GPU wait (WaitForWireBufferHostAccess) per draw.
//
// THE ONE FALSE: a COUNT form this device cannot issue - the monolith arm's own condition for
// vkCmdDrawIndexedIndirectCount (MultiDrawElementsIndirectCount), extended to the non-indexed
// twin the monolith never loads. Everything else is issued here or declined by name.
Bool VulkanRenderer::DrawWireIndirectNative(GLenum mode, GLenum type, Uint64 offset, GLsizei drawcount,
GLsizei stride, Bool indexed, Bool counted, Uint64 countOffset) {
if (drawcount <= 0) return true;
if (counted) {
const auto countEntry = indexed ? s_vkCmdDrawIndexedIndirectCount : s_vkCmdWireDrawIndirectCount;
if (!m_drawIndirectCountExtensionEnabled || countEntry == nullptr ||
(!m_multiDrawIndirectFeatureEnabled && drawcount != 1))
return false;
}
const auto& state = MG_Pipe::MGPipeApplier();
// GL's command sizes (GL 4.6 10.3.10), which are also Vulkan's.
const Uint64 commandSize = indexed ? 5 * sizeof(Uint32) : 4 * sizeof(Uint32);
if (stride == 0) stride = static_cast<GLsizei>(commandSize);
BufferSlice commands{};
if (MG_Pipe::MGPipeHandleIsNull(state.VerbIndirectBuffer) ||
!m_bufferManager.AcquireWireSlice(BufferKind::Indirect, state.VerbIndirectBuffer, commands)) {
MGL_WIRE_DECLINE_AT(IndirectBufferUnbound, "an indirect draw names no live indirect buffer store");
return true;
}
const Uint64 span = static_cast<Uint64>(stride) * static_cast<Uint64>(drawcount - 1) + commandSize;
// GL refuses a stride that is not a multiple of four (INVALID_VALUE), and Vulkan requires
// it for drawCount > 1; a peer's record that breaks either is declined, not issued.
if (stride < static_cast<GLsizei>(commandSize) || (stride & 3) != 0 || (offset & 3u) != 0 ||
offset > commands.size || span > commands.size - offset) {
MGL_WIRE_DECLINE_AT(IndirectCommandRange, "the indirect commands lie outside their store or are misaligned");
return true;
}
BufferSlice parameters{};
if (counted) {
if (MG_Pipe::MGPipeHandleIsNull(state.VerbIndirectParameterBuffer) ||
!m_bufferManager.AcquireWireSlice(BufferKind::Indirect, state.VerbIndirectParameterBuffer, parameters) ||
(countOffset & 3u) != 0 || countOffset > parameters.size || parameters.size - countOffset < sizeof(Uint32)) {
MGL_WIRE_DECLINE_AT(IndirectCountRange, "the indirect draw count lies outside its parameter store");
return true;
}
}
// The index view is the WHOLE bound element store past its binding offset, as the monolith
// arm binds it: the commands name their own firstIndex, which only the GPU reads.
IndexBufferView indices{};
DrawCmdParam vertexRange{};
vertexRange.instanceCount = 1;
Flags<DrawSetupAspect> aspects = DrawSetupAspect::IndirectDrawBuffer;
if (indexed) {
const SizeT width = MG_Util::GetGLTypeSize(type);
BufferSlice elements{};
if (width == 0 || MG_Pipe::MGPipeHandleIsNull(state.IndexBuffer.Res) ||
!m_bufferManager.AcquireWireSlice(BufferKind::Index, state.IndexBuffer.Res, elements) ||
state.IndexBuffer.Offset > elements.size) {
MGL_WIRE_DECLINE_AT(IndirectIndexStore, "an indexed indirect draw names no usable element store");
return true;
}
indices.indexType = type;
indices.indexByteOffset = 0;
indices.indexByteSize = static_cast<SizeT>(elements.size - state.IndexBuffer.Offset);
vertexRange.vertexCount = static_cast<Uint32>(indices.indexByteSize / width);
aspects |= DrawSetupAspect::IndexBuffer;
}
// A shader write the store was marked with (MarkWireBufferGpuWritten) is made visible to
// the indirect read here, once per write, so a program that omits GL_COMMAND_BARRIER_BIT
// is not left racing a read the old CPU path used to take after a whole-GPU wait. The
// application's own glMemoryBarrier is BuildMemoryBarrierForGlBarriers, as on the monolith
// arm; transfer writes carry their own MEMORY_READ after-barrier (VkBufferManager.cpp).
const Bool commandsWritten = m_bufferManager.TakeWireIndirectReadBarrier(state.VerbIndirectBuffer);
const Bool countWritten = counted && m_bufferManager.TakeWireIndirectReadBarrier(state.VerbIndirectParameterBuffer);
if (commandsWritten || countWritten) {
RetireWireDrawPass();
auto& barrierFrame = m_frameContext.GetCurrent();
if (!barrierFrame.isCommandRecording) m_frameContext.BeginCommandRecording();
if (VkRenderPassManager::GetActiveRenderPass() != nullptr)
VkRenderPassManager::EndRenderPass(barrierFrame.commandBuffer);
VkMemoryBarrier barrier{VK_STRUCTURE_TYPE_MEMORY_BARRIER};
barrier.srcAccessMask = VK_ACCESS_MEMORY_WRITE_BIT;
barrier.dstAccessMask = VK_ACCESS_INDIRECT_COMMAND_READ_BIT;
vkCmdPipelineBarrier(barrierFrame.commandBuffer, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
VK_PIPELINE_STAGE_DRAW_INDIRECT_BIT, 0, 1, &barrier, 0, nullptr, 0, nullptr);
if (MG_Util::PipeStats::Enabled())
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireIndirectBarriers, 1);
}
auto& frame = m_frameContext.GetCurrent();
m_wireNativeIndirectDraw = true;
const Bool ready = SetupDraw(frame, mode, aspects, vertexRange, indexed ? &indices : nullptr);
m_wireNativeIndirectDraw = false;
if (!ready) return true;
MOBILEGL_ASSERT(frame.isCommandRecording, "%s: frame recording was not started", __func__);
const VkCommandBuffer commandBuffer = frame.commandBuffer;
const VkDeviceSize first = commands.offset + offset;
const Uint32 count = static_cast<Uint32>(drawcount);
const Uint32 byteStride = static_cast<Uint32>(stride);
if (counted) {
const auto countEntry = indexed ? s_vkCmdDrawIndexedIndirectCount : s_vkCmdWireDrawIndirectCount;
countEntry(commandBuffer, commands.buffer, first, parameters.buffer, parameters.offset + countOffset,
count, byteStride);
} else if (count == 1 || (!m_multiDrawForceUnrollIndirect && m_multiDrawIndirectFeatureEnabled)) {
// maxDrawIndirectCount bounds one call; GL's drawcount does not.
const Uint32 limit = std::max<Uint32>(1u, m_physicalDevice.properties.limits.maxDrawIndirectCount);
for (Uint32 base = 0; base < count; base += limit) {
const Uint32 chunk = std::min(count - base, limit);
const VkDeviceSize at = first + static_cast<VkDeviceSize>(base) * byteStride;
if (indexed) vkCmdDrawIndexedIndirect(commandBuffer, commands.buffer, at, chunk, byteStride);
else vkCmdDrawIndirect(commandBuffer, commands.buffer, at, chunk, byteStride);
}
} else {
// No multiDrawIndirect: one indirect draw per command, still read by the GPU
// (the monolith arm's own fallback).
for (Uint32 i = 0; i < count; ++i) {
const VkDeviceSize at = first + static_cast<VkDeviceSize>(i) * byteStride;
if (indexed) vkCmdDrawIndexedIndirect(commandBuffer, commands.buffer, at, 1, 0);
else vkCmdDrawIndirect(commandBuffer, commands.buffer, at, 1, 0);
}
}
if (MG_Util::PipeStats::Enabled())
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireIndirectNativeDraws, 1);
return true;
}
void VulkanRenderer::DispatchWireCompute(GLuint x, GLuint y, GLuint z) {
RetireWireDrawPass();
auto& frame = m_frameContext.GetCurrent();
MobileGL/MG_IntegrationTest/CMakeLists.txt
+35
View File
@@ -4648,6 +4648,41 @@ if (MOBILEGL_BUILD_DISAGGREGATED)
endforeach()
endif()
# P8-D: the Magma wire arm issues indirect draws natively (docs/Disaggregated/notes/p8/D.md).
# Public GL, so the source joins the binary unguarded and the ambient `DirectGLES.` /
# `DirectVulkan.` monolith discovery runs it too. Split arms: DirectGLES through the registration
# macro (which also replays it on Magma's informational tier), Magma's gated tier by hand, all
# three transports - the counter halves read only on the inproc arm (WireIndirectPeek.h) and the
# pixel halves everywhere, so there is no arm-only property to declare.
target_sources(MobileGLIntegrationTest PRIVATE Harness/WireIndirectPeek.cpp Scenarios/WireIndirectDrawScenario.cpp)
if (MOBILEGL_BUILD_DISAGGREGATED)
mgl_itest_register_split_arms("WireIndirectDrawScenario.*" "")
set(MGL_ITEST_MAGMA_INDIRECT_FLAVOUR_ENV
"MOBILEGL_BACKEND_TYPE=DirectVulkan"
"MOBILEGL_IPC_ROLE_SPLIT_STATE=1" "MOBILEGL_IPC_STRICT_ERRORS=1"
"MOBILEGL_ITEST_REQUIRE_GPU=1" "MGITEST_SPLIT_LANE=1"
${MGL_ITEST_CAPABILITY_ENV} ${MGL_ITEST_VULKAN_ENV})
foreach(mglItestArm IN LISTS MGL_ITEST_MAGMA_ARMS)
mgl_itest_magma_arm_shape(${mglItestArm} indirect)
mgl_itest_join_environment(MGL_ITEST_MAGMA_INDIRECT_ENVIRONMENT
${MGL_ITEST_MAGMA_INDIRECT_FLAVOUR_ENV} ${mglItestArmTransportEnv})
foreach(indirectCase
EveryIndirectFormIsIssuedNativelyFromItsStore
ComputeWrittenCommandsDrawWithoutAHostWait
ComputeWrittenCountWordIsTheOneTheGpuReads
ShaderWrittenCommandsWithoutAnApplicationBarrierAreOrdered)
gtest_discover_tests(MobileGLIntegrationTest
TEST_PREFIX "DirectVulkan.${mglItestArm}.Indirect."
TEST_FILTER "WireIndirectDrawScenario.${indirectCase}"
DISCOVERY_TIMEOUT 30
PROPERTIES LABELS "${mglItestMagmaLabels}"
TIMEOUT ${MGL_ITEST_TIMEOUT}
${mglItestArmProperties}
ENVIRONMENT "${MGL_ITEST_MAGMA_INDIRECT_ENVIRONMENT}\;MOBILEGL_LOG_FILE_PATH=${CMAKE_CURRENT_BINARY_DIR}/magma-indirect${mglItestArmLogTag}-${indirectCase}.log")
endforeach()
endforeach()
endif()
# =====================================================================================
# P7 PACKAGE L: THE INFORMATIONAL MAGMA CENSUS - TIER 2 OF TWO
# =====================================================================================
MobileGL/MG_IntegrationTest/Harness/WireIndirectPeek.cpp
+37
View File
@@ -0,0 +1,37 @@
// MobileGL - MobileGL/MG_IntegrationTest/Harness/WireIndirectPeek.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "WireIndirectPeek.h"
#if !defined(__ANDROID__)
#include <MG_Pipe/MGPipe.h>
#if MOBILEGL_PIPE_PUSH
#include <MG_Util/Metrics/PipeStats.h>
#define MGITEST_WIRE_INDIRECT_PEEK_LIVE 1
#endif
#endif
namespace MGITest {
#if defined(MGITEST_WIRE_INDIRECT_PEEK_LIVE)
bool PeekWireIndirectCounters(WireIndirectCounters* out) {
if (out == nullptr) return false;
namespace Stats = MobileGL::MG_Util::PipeStats;
if (!Stats::Enabled()) Stats::SetEnabledForTesting(true);
out->nativeDraws = Stats::TotalCalls(Stats::CallClass::WireIndirectNativeDraws);
out->cpuExpansions = Stats::TotalCalls(Stats::CallClass::WireIndirectCpuExpansions);
out->barriers = Stats::TotalCalls(Stats::CallClass::WireIndirectBarriers);
out->hostWaits = Stats::TotalCalls(Stats::CallClass::WireHostWaits);
out->indirectWaits = Stats::TotalCalls(Stats::CallClass::WireHostWaitsIndirect);
return true;
}
#else
bool PeekWireIndirectCounters(WireIndirectCounters*) { return false; }
#endif
} // namespace MGITest
MobileGL/MG_IntegrationTest/Harness/WireIndirectPeek.h
+28
View File
@@ -0,0 +1,28 @@
// MobileGL - MobileGL/MG_IntegrationTest/Harness/WireIndirectPeek.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// P8-D: the Magma wire arm's indirect-draw counters (PipeStats `windr[...]`), read as run totals
// out of THIS process - P4aFinalFixPeek.cpp's shape. Process-wide means the SERVER's counts only
// where the server is this process: monolith and inproc, never spawn or tcp. The first call arms
// the counters (a case reads before and after its workload, so it must peek before it draws).
// False where the reading cannot be taken: a pull build, Android.
#pragma once
namespace MGITest {
struct WireIndirectCounters {
unsigned long long nativeDraws = 0; // wire-indirect-native-draws (`wind`)
unsigned long long cpuExpansions = 0; // wire-indirect-cpu-expansions (`wixp`)
unsigned long long barriers = 0; // wire-indirect-barriers (`wibar`)
unsigned long long hostWaits = 0; // wire-host-waits (`whw`)
unsigned long long indirectWaits = 0; // wire-host-waits-indirect (`whwi`)
};
bool PeekWireIndirectCounters(WireIndirectCounters* out);
} // namespace MGITest
MobileGL/MG_IntegrationTest/Scenarios/WireIndirectDrawScenario.cpp
+365
View File
@@ -0,0 +1,365 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/WireIndirectDrawScenario.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// P8-D: THE MAGMA WIRE ARM ISSUES INDIRECT DRAWS FROM THE WIRE STORE, AS THE MONOLITH ARM DOES.
//
// Before P8-D every wire indirect draw read its commands (and its count word) through
// ReadWireBuffer and expanded them on the CPU, and a store a shader had written turned that read
// into a whole-GPU wait per draw (create-indirect: 321 expansions, 327 waits, ~4-8 s of waiting
// per replay on lavapipe). The cases pin both halves of the change:
// * PIXELS, on every arm and both backends where the backend's own semantics allow it - each
// indirect form draws exactly the strips its commands name, and GPU-written commands are the
// ones the draw sees (the poison the buffers start with names the OTHER strips);
// * THE SERVER'S OWN COUNTERS (Harness/WireIndirectPeek.h), where they can be read - Magma on
// the inproc arm, whose server is this process: every call issued natively, no CPU
// expansion, no host wait, and the INDIRECT_COMMAND_READ barrier exactly once per write.
//
// Four strips across the viewport, one per quarter; a command draws one strip. Green = drawn,
// red = the clear. The VBO opens with three padding vertices so a command that loses its `first`
// or `firstIndex` draws a degenerate triangle rather than the right strip.
#include <array>
#include <cstdint>
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#include "../Harness/SplitRuntimePeek.h"
#include "../Harness/WireIndirectPeek.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glext.h>
namespace MGITest {
namespace {
constexpr int kStrips = 4;
constexpr std::uint32_t kPad = 3;
constexpr std::uint32_t kStripVertices = 6;
constexpr const char* kVertexSource = R"(#version 430 core
layout(location = 0) in vec2 aPos;
void main() { gl_Position = vec4(aPos, 0.0, 1.0); }
)";
constexpr const char* kFragmentSource = R"(#version 430 core
out vec4 oColor;
void main() { oColor = vec4(0.0, 1.0, 0.0, 1.0); }
)";
// Copies `source` into `words`, one word per invocation: the compute write under test.
constexpr const char* kCopySource = R"(#version 430 core
layout(local_size_x = 1) in;
layout(std430, binding = 0) buffer Words { uint words[]; };
layout(std430, binding = 1) readonly buffer Source { uint source[]; };
void main() { words[gl_GlobalInvocationID.x] = source[gl_GlobalInvocationID.x]; }
)";
struct ArraysCommand {
std::uint32_t count, instanceCount, first, baseInstance;
};
struct ElementsCommand {
std::uint32_t count, instanceCount, firstIndex;
std::int32_t baseVertex;
std::uint32_t baseInstance;
};
// A padded elements command, for the stride case: 32 bytes, the tail unread.
struct PaddedElementsCommand {
ElementsCommand command;
std::uint32_t padding[3];
};
ArraysCommand Arrays(int strip) { return {kStripVertices, 1, kPad + kStripVertices * strip, 0}; }
ElementsCommand Elements(int strip) { return {kStripVertices, 1, kStripVertices * strip, 0, 0}; }
using Strips = std::array<bool, kStrips>;
class WireIndirectDrawScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
m_magma = Gl().BackendName() == "DirectVulkan";
m_transport = PeekSplitRuntime().transportName;
std::string error;
m_program = CompileProgram(kVertexSource, kFragmentSource, &error);
ASSERT_NE(m_program, 0u) << error;
m_copy = CompileCompute(kCopySource);
ASSERT_NE(m_copy, 0u);
std::vector<float> vertices(2 * kPad, 0.0f);
for (int strip = 0; strip < kStrips; ++strip) {
const float x0 = -1.0f + 0.5f * static_cast<float>(strip);
const float x1 = x0 + 0.5f;
const float quad[12] = {x0, -1.0f, x1, -1.0f, x1, 1.0f, x0, -1.0f, x1, 1.0f, x0, 1.0f};
vertices.insert(vertices.end(), quad, quad + 12);
}
std::vector<std::uint32_t> indices;
for (std::uint32_t i = 0; i < kStripVertices * kStrips; ++i) indices.push_back(kPad + i);
glGenVertexArrays(1, &m_vao);
glBindVertexArray(m_vao);
glGenBuffers(1, &m_vbo);
glBindBuffer(GL_ARRAY_BUFFER, m_vbo);
glBufferData(GL_ARRAY_BUFFER, static_cast<GLsizeiptr>(vertices.size() * sizeof(float)),
vertices.data(), GL_STATIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 2 * sizeof(float), nullptr);
glGenBuffers(1, &m_ebo);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, m_ebo);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, static_cast<GLsizeiptr>(indices.size() * sizeof(std::uint32_t)),
indices.data(), GL_STATIC_DRAW);
ASSERT_EQ(FirstGLError(), GLenum(GL_NO_ERROR)) << "scene setup left a GL error behind";
// The counters are armed by the first peek, so it has to come before any draw.
if (CountersReadable()) {
ASSERT_TRUE(PeekWireIndirectCounters(&m_before))
<< "Magma on the inproc arm must be able to read the wire arm's own counters";
}
}
void TearDown() override {
if (!Ready()) return;
glBindVertexArray(0);
for (GLuint* buffer : {&m_vbo, &m_ebo, &m_indirect, &m_parameter, &m_source}) {
if (*buffer != 0) glDeleteBuffers(1, buffer);
*buffer = 0;
}
if (m_vao != 0) glDeleteVertexArrays(1, &m_vao);
if (m_program != 0) glDeleteProgram(m_program);
if (m_copy != 0) glDeleteProgram(m_copy);
}
static GLuint CompileCompute(const char* source) {
const GLuint shader = glCreateShader(GL_COMPUTE_SHADER);
glShaderSource(shader, 1, &source, nullptr);
glCompileShader(shader);
GLint ok = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &ok);
if (ok != GL_TRUE) {
glDeleteShader(shader);
return 0;
}
const GLuint program = glCreateProgram();
glAttachShader(program, shader);
glLinkProgram(program);
glDeleteShader(shader);
glGetProgramiv(program, GL_LINK_STATUS, &ok);
if (ok != GL_TRUE) {
glDeleteProgram(program);
return 0;
}
return program;
}
template <typename T>
static void Fill(GLuint& name, GLenum target, const std::vector<T>& data) {
if (name == 0) glGenBuffers(1, &name);
glBindBuffer(target, name);
glBufferData(target, static_cast<GLsizeiptr>(data.size() * sizeof(T)), data.data(), GL_DYNAMIC_COPY);
}
// The compute write: `words` (already holding its poison) receives `data` from the GPU.
template <typename T>
void ComputeWrite(GLuint words, const std::vector<T>& data) {
Fill(m_source, GL_SHADER_STORAGE_BUFFER, data);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, words);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, m_source);
glUseProgram(m_copy);
glDispatchCompute(static_cast<GLuint>(data.size() * sizeof(T) / sizeof(std::uint32_t)), 1, 1);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, 0);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, 0);
}
template <typename DrawFn>
Image Render(DrawFn&& draw) {
BindDefaultFramebuffer();
glViewport(0, 0, Gl().Width(), Gl().Height());
glDisable(GL_DEPTH_TEST);
ClearTo(1.0f, 0.0f, 0.0f, 1.0f);
glUseProgram(m_program);
glBindVertexArray(m_vao);
glBindBuffer(GL_DRAW_INDIRECT_BUFFER, m_indirect);
draw();
return ReadPixels(Gl().Width(), Gl().Height());
}
static void ExpectStrips(const Image& image, const Strips& drawn, const std::string& what) {
for (int strip = 0; strip < kStrips; ++strip) {
const int x = image.Width() * (2 * strip + 1) / (2 * kStrips);
EXPECT_STREQ(image.ColorName(x, image.Height() / 2), drawn[strip] ? "green" : "red")
<< what << ": strip " << strip;
}
}
// Magma's server in THIS process: the inproc arm. Spawn and tcp servers are other
// processes, and the monolith arm never takes the wire path at all.
bool CountersReadable() const { return m_magma && m_transport == "inproc"; }
WireIndirectCounters Delta() const {
WireIndirectCounters now{};
EXPECT_TRUE(PeekWireIndirectCounters(&now));
return {now.nativeDraws - m_before.nativeDraws, now.cpuExpansions - m_before.cpuExpansions,
now.barriers - m_before.barriers, now.hostWaits - m_before.hostWaits,
now.indirectWaits - m_before.indirectWaits};
}
bool m_magma = false;
std::string m_transport;
WireIndirectCounters m_before{};
GLuint m_program = 0, m_copy = 0, m_vao = 0, m_vbo = 0, m_ebo = 0;
GLuint m_indirect = 0, m_parameter = 0, m_source = 0;
};
// Every indirect form from CPU-written commands. On Magma/inproc the server must have
// issued all six natively, expanded none, waited for nothing and recorded no barrier (no
// shader ever wrote these stores).
TEST_F(WireIndirectDrawScenario, EveryIndirectFormIsIssuedNativelyFromItsStore) {
if (!Ready() || IsSkipped()) return;
// Arrays commands at 0.., elements commands after them: the offsets are byte offsets
// into one store, as an application packs them.
const std::vector<ArraysCommand> arrays = {Arrays(0), Arrays(1), Arrays(3), Arrays(2)};
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, arrays);
Image image = Render([&] { glDrawArraysIndirect(GL_TRIANGLES, reinterpret_cast<const void*>(16)); });
ExpectStrips(image, {false, true, false, false}, "glDrawArraysIndirect at byte 16");
image = Render([&] { glMultiDrawArraysIndirect(GL_TRIANGLES, nullptr, 3, 0); });
ExpectStrips(image, {true, true, false, true}, "glMultiDrawArraysIndirect, 3 commands");
const std::vector<PaddedElementsCommand> padded = {{Elements(3), {}}, {Elements(0), {}}};
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, padded);
image = Render([&] {
glMultiDrawElementsIndirect(GL_TRIANGLES, GL_UNSIGNED_INT, nullptr, 2, sizeof(PaddedElementsCommand));
});
ExpectStrips(image, {true, false, false, true}, "glMultiDrawElementsIndirect, stride 32");
image = Render([&] {
glDrawElementsIndirect(GL_TRIANGLES, GL_UNSIGNED_INT,
reinterpret_cast<const void*>(sizeof(PaddedElementsCommand)));
});
ExpectStrips(image, {true, false, false, false}, "glDrawElementsIndirect at byte 32");
// The count forms: three commands each, a count word that allows fewer.
Fill(m_parameter, GL_PARAMETER_BUFFER, std::vector<std::uint32_t>{7, 2, 1});
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ArraysCommand>{Arrays(0), Arrays(1), Arrays(2)});
image = Render([&] {
glMultiDrawArraysIndirectCount(GL_TRIANGLES, nullptr, 4, 3, 0);
});
ExpectStrips(image, {true, true, false, false}, "glMultiDrawArraysIndirectCount, count word 2 of 3");
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ElementsCommand>{Elements(3), Elements(2), Elements(1)});
image = Render([&] {
glMultiDrawElementsIndirectCount(GL_TRIANGLES, GL_UNSIGNED_INT, nullptr, 8, 3, 0);
});
ExpectStrips(image, {false, false, false, true}, "glMultiDrawElementsIndirectCount, count word 1 of 3");
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
if (!CountersReadable()) return;
const WireIndirectCounters delta = Delta();
EXPECT_EQ(delta.nativeDraws, 6u) << "every indirect call must be issued as vkCmdDraw*Indirect[Count]";
EXPECT_EQ(delta.cpuExpansions, 0u) << "no indirect call may read its words on the CPU";
EXPECT_EQ(delta.indirectWaits, 0u) << "an indirect call took a host wait";
EXPECT_EQ(delta.hostWaits, 0u) << "a host wait was taken with no GPU write anywhere";
EXPECT_EQ(delta.barriers, 0u) << "no shader wrote these stores, so no barrier is due";
}
// Commands a compute shader wrote, behind the application's own GL_COMMAND_BARRIER_BIT.
// The buffers start with commands for the OTHER strips, so a read of stale words shows.
// On Magma/inproc: issued natively, and - the point of the package - NO host wait.
TEST_F(WireIndirectDrawScenario, ComputeWrittenCommandsDrawWithoutAHostWait) {
if (!Ready() || IsSkipped()) return;
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ArraysCommand>{Arrays(2), Arrays(3)});
ComputeWrite(m_indirect, std::vector<ArraysCommand>{Arrays(0), Arrays(1)});
glMemoryBarrier(GL_COMMAND_BARRIER_BIT);
Image image = Render([&] { glMultiDrawArraysIndirect(GL_TRIANGLES, nullptr, 2, 0); });
ExpectStrips(image, {true, true, false, false}, "glMultiDrawArraysIndirect of compute-written commands");
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ElementsCommand>{Elements(0), Elements(1)});
ComputeWrite(m_indirect, std::vector<ElementsCommand>{Elements(3), Elements(2)});
glMemoryBarrier(GL_COMMAND_BARRIER_BIT);
image = Render([&] { glMultiDrawElementsIndirect(GL_TRIANGLES, GL_UNSIGNED_INT, nullptr, 2, 0); });
ExpectStrips(image, {false, false, true, true}, "glMultiDrawElementsIndirect of compute-written commands");
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
if (!CountersReadable()) return;
const WireIndirectCounters delta = Delta();
EXPECT_EQ(delta.nativeDraws, 2u) << "both draws must be issued natively";
EXPECT_EQ(delta.cpuExpansions, 0u) << "no indirect call may read its words on the CPU";
EXPECT_EQ(delta.indirectWaits, 0u)
<< "an indirect draw of shader-written commands waited for the GPU on the host";
EXPECT_EQ(delta.hostWaits, 0u) << "a host wait was taken";
EXPECT_EQ(delta.barriers, 2u) << "one INDIRECT_COMMAND_READ barrier per freshly written store";
}
// The COUNT word a compute shader wrote. Magma only: Espryt reads that word from a
// server shadow GPU writes do not update, on both arms (OPEN-QUESTIONS 15, P8-C's
// package), so its answer here is a known divergence rather than this package's claim.
TEST_F(WireIndirectDrawScenario, ComputeWrittenCountWordIsTheOneTheGpuReads) {
if (!Ready() || IsSkipped()) return;
if (!m_magma) {
GTEST_SKIP() << "Espryt reads the count word from its shadow on both arms (OQ15, P8-C)";
}
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ElementsCommand>{Elements(0), Elements(3), Elements(2)});
Fill(m_parameter, GL_PARAMETER_BUFFER, std::vector<std::uint32_t>{3});
ComputeWrite(m_parameter, std::vector<std::uint32_t>{1});
glMemoryBarrier(GL_COMMAND_BARRIER_BIT);
Image image = Render([&] {
glMultiDrawElementsIndirectCount(GL_TRIANGLES, GL_UNSIGNED_INT, nullptr, 0, 3, 0);
});
ExpectStrips(image, {true, false, false, false}, "glMultiDrawElementsIndirectCount, shader-written count 1");
if (m_transport == "monolith") {
// DirectVulkan.cpp's monolith MultiDrawArraysIndirectCount reads this word from the
// frontend shadow without SyncGpuWrites: recorded for dev (ID-P8-3), not this lane's.
return;
}
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ArraysCommand>{Arrays(1), Arrays(2), Arrays(3)});
Fill(m_parameter, GL_PARAMETER_BUFFER, std::vector<std::uint32_t>{0});
ComputeWrite(m_parameter, std::vector<std::uint32_t>{2});
glMemoryBarrier(GL_COMMAND_BARRIER_BIT);
image = Render([&] { glMultiDrawArraysIndirectCount(GL_TRIANGLES, nullptr, 0, 3, 0); });
ExpectStrips(image, {false, true, true, false}, "glMultiDrawArraysIndirectCount, shader-written count 2");
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
if (!CountersReadable()) return;
const WireIndirectCounters delta = Delta();
EXPECT_EQ(delta.nativeDraws, 2u);
EXPECT_EQ(delta.cpuExpansions, 0u) << "a count word was read on the CPU";
EXPECT_EQ(delta.hostWaits, 0u) << "a host wait was taken";
EXPECT_EQ(delta.barriers, 2u) << "one barrier per draw whose count store a shader wrote";
}
// Flywheel's shape (create-indirect's trace: 322 cull-then-draw rounds, no glMemoryBarrier
// between them). GL leaves it undefined; the wire arm orders it anyway, once per write,
// because the CPU path it replaced did (by waiting). Magma's split arms only: the monolith
// arm records no dependency here (recorded for dev), and Espryt's is its driver's.
TEST_F(WireIndirectDrawScenario, ShaderWrittenCommandsWithoutAnApplicationBarrierAreOrdered) {
if (!Ready() || IsSkipped()) return;
if (!m_magma || m_transport == "monolith") {
GTEST_SKIP() << "the implicit INDIRECT_COMMAND_READ barrier is the Magma wire arm's (P8-D)";
}
Fill(m_indirect, GL_DRAW_INDIRECT_BUFFER, std::vector<ArraysCommand>{Arrays(3), Arrays(2)});
ComputeWrite(m_indirect, std::vector<ArraysCommand>{Arrays(0), Arrays(1)});
Image image = Render([&] { glMultiDrawArraysIndirect(GL_TRIANGLES, nullptr, 2, 0); });
ExpectStrips(image, {true, true, false, false}, "the first draw after the write");
image = Render([&] { glDrawArraysIndirect(GL_TRIANGLES, reinterpret_cast<const void*>(16)); });
ExpectStrips(image, {false, true, false, false}, "a second draw, no write between");
ComputeWrite(m_indirect, std::vector<ArraysCommand>{Arrays(2), Arrays(3)});
image = Render([&] { glMultiDrawArraysIndirect(GL_TRIANGLES, nullptr, 2, 0); });
ExpectStrips(image, {false, false, true, true}, "the draw after a second write");
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
if (!CountersReadable()) return;
const WireIndirectCounters delta = Delta();
EXPECT_EQ(delta.nativeDraws, 3u);
EXPECT_EQ(delta.cpuExpansions, 0u);
EXPECT_EQ(delta.hostWaits, 0u) << "a host wait was taken";
EXPECT_EQ(delta.barriers, 2u) << "exactly one barrier per write: the second draw reads no new write";
}
} // namespace
} // namespace MGITest
MobileGL/MG_Test/Util/PipeStatsTest.cpp
+19
View File
@@ -574,4 +574,23 @@ namespace {
}
EXPECT_EQ(PS::FrameCount(), PS::SummaryFramePeriod());
}
#if MOBILEGL_PIPE_PUSH
// P8-D: the Magma wire arm's indirect-draw bracket. Its short names are what the create
// fixtures' measurement (notes/p8/D.md) reads off the server's line, and its long names are
// what Harness/WireIndirectPeek.cpp's readers expect in the JSON dump.
TEST_F(PipeStatsTest, WireIndirectCountersPrintInTheirOwnBracket) {
PS::AddCalls(PS::CallClass::WireIndirectNativeDraws, 3);
PS::AddCalls(PS::CallClass::WireIndirectCpuExpansions, 2);
PS::AddCalls(PS::CallClass::WireHostWaitMicros, 1500);
const String line = PS::FormatWindowLine();
EXPECT_NE(line.find("windr[wind=3 wixp=2 wibar=0 whw=0 whwi=0 whwus=1500]"), String::npos) << line;
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireIndirectNativeDraws), "wire-indirect-native-draws");
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireIndirectCpuExpansions), "wire-indirect-cpu-expansions");
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireIndirectBarriers), "wire-indirect-barriers");
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireHostWaits), "wire-host-waits");
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireHostWaitsIndirect), "wire-host-waits-indirect");
EXPECT_STREQ(PS::NameOf(PS::CallClass::WireHostWaitMicros), "wire-host-wait-us");
}
#endif
} // namespace
MobileGL/MG_Util/Metrics/PipeStats.cpp
+11
View File
@@ -238,6 +238,8 @@ namespace MobileGL::MG_Util::PipeStats {
"shader-image-emissions", "client-tex-upload-emissions", "tex-remint-pulls",
"resident-subdata-emissions",
"residual-pulls", "server-verb-boundaries", "wire-records",
"wire-indirect-native-draws", "wire-indirect-cpu-expansions", "wire-indirect-barriers",
"wire-host-waits", "wire-host-waits-indirect", "wire-host-wait-us",
#endif
};
const char* const kGateNames[kGateCount] = {
@@ -690,6 +692,15 @@ namespace MobileGL::MG_Util::PipeStats {
line += " wlivepk=" + std::to_string(Read(g_gauges[static_cast<Uint32>(Gauge::WireStoresPeak)]));
line += " wdefpk=" + std::to_string(Read(g_gauges[static_cast<Uint32>(Gauge::WireDeferredBytesPeak)]));
line += " wdefsync=" + std::to_string(Read(g_gauges[static_cast<Uint32>(Gauge::WireDeferredSyncs)]));
// P8-D: the Magma wire arm's indirect draws and its host-access waits, windowed like the
// other CallClass fields (per window, not per frame: the create fixtures hold one frame).
line += "] windr[wind=" +
std::to_string(calls[static_cast<Uint32>(CallClass::WireIndirectNativeDraws)]);
line += " wixp=" + std::to_string(calls[static_cast<Uint32>(CallClass::WireIndirectCpuExpansions)]);
line += " wibar=" + std::to_string(calls[static_cast<Uint32>(CallClass::WireIndirectBarriers)]);
line += " whw=" + std::to_string(calls[static_cast<Uint32>(CallClass::WireHostWaits)]);
line += " whwi=" + std::to_string(calls[static_cast<Uint32>(CallClass::WireHostWaitsIndirect)]);
line += " whwus=" + std::to_string(calls[static_cast<Uint32>(CallClass::WireHostWaitMicros)]);
#endif
line += "] gates[";
for (Uint32 i = 0; i < kGateCount; ++i) {
MobileGL/MG_Util/Metrics/PipeStats.h
+13
View File
@@ -270,6 +270,19 @@ namespace MobileGL::MG_Util::PipeStats {
// that goes with it (FormatWindowLine prints the run total instead when the window holds
// no Present).
WireRecords,
// P8-D: the Magma wire arm's indirect draws, on the server (`windr[...]` on the line).
// `wind` - indirect draw calls issued as vkCmdDraw*Indirect[Count] from the wire store.
// `wixp` - indirect draw calls whose words were read on the CPU instead (the count word,
// or every command, through ReadWireBuffer).
// `wibar` - the INDIRECT_COMMAND_READ barriers recorded for a store marked GPU-written.
// `whw` / `whwus` - WaitForWireBufferHostAccess calls that actually waited, and their
// microseconds, from any caller; `whwi` - the subset an indirect read caused.
WireIndirectNativeDraws,
WireIndirectCpuExpansions,
WireIndirectBarriers,
WireHostWaits,
WireHostWaitsIndirect,
WireHostWaitMicros,
#endif
Count
};
docs/Disaggregated/notes/p8/D.md
+83
View File
@@ -0,0 +1,83 @@
# P8-D：Magma wire 臂原生 indirect draw
> 2026-09-29，分支 `p8/d`（基线 `8d3e2317`）。计划：[`PLAN-P8.md`](PLAN-P8.md) 包 D；审计 §4、§6 风险 3。主机 lavapipe，只动 Magma（Espryt 作对照）。
## 1. 结论
- wire 臂的 `glDraw*Indirect` / `glMultiDraw*Indirect[Count]` 改为从 wire store 自己的 `VkBuffer` 发 `vkCmdDraw[Indexed]Indirect[Count]`：`VulkanRenderer::DrawWireIndirectNative`（`MobileGL/MG_Backend/DirectVulkan/Renderer/WireDraw.inc:620`）。
- `DrawWireIndirect`（`MobileGL/MG_Backend/DirectVulkan/DirectVulkan.cpp:381`）先走原生；CPU 展开只留给设备发不出的 COUNT 形态（与 monolith 同条件，§3）。
- create-indirect × Magma × split：indirect 引起的 host 等待 321 → 0，等待时间 8.07 s → 0.013 s（剩下 6 次是应用回读，§2）；SSIM 各臂不变。
- 新增 barrier：着色器写过的 store 在下一次 indirect 读前补一个 `INDIRECT_COMMAND_READ` barrier，每次写一次（`WireDraw.inc:686-698`）。create-indirect 的 322 次 cull→draw 里 320 次没有 `glMemoryBarrier`，旧 CPU 路径靠整 GPU 等待碰巧有序，原生路径靠这个 barrier。
## 2. 测量（主机，`MOBILEGL_PIPE_STATS=1 MOBILEGL_PIPE_STATS_PERIOD=1`，server 日志 `windr[...]` 求和）
两个 fixture 都只有 1 帧（frames=1），所以"每帧"= 每次重放。主机上同时有 7 个包在跑，毫秒只看量级。
| fixture | 臂 | draws | wind | wixp | wibar | whw | whwi | whwus | SSIM |
|---|---|---:|---:|---:|---:|---:|---:|---:|---|
| create-indirect | monolith（前后相同） | 13172 | 0 | 0 | 0 | 0 | 0 | 0 | 0.999956667 |
| create-indirect | inproc，改前 | 13172 | 0 | 321 | 0 | 327 | 321 | 8065617 | 0.999956667 |
| create-indirect | inproc，改后 | 13172 | 321 | 0 | 321 | 6 | 0 | 12540 | 0.999956667 |
| create-indirect | spawn，改前 | 13202 | 0 | 322 | 0 | 328 | 322 | 8360015 | 0.999956667 |
| create-indirect | spawn，改后 | 13202 | 322 | 0 | 322 | 6 | 0 | 15769 | 0.999956667 |
| create-instancing | 三臂，前后相同 | 14158 / 14194 | 0 | 0 | 0 | 0 | 0 | 0 | 0.999952470 |
- 字段：`wind` 原生 indirect 调用、`wixp` CPU 展开、`wibar` 补的 barrier、`whw` / `whwus` 真正等了的 `WaitForWireBufferHostAccess` 次数 / 微秒（`MobileGL/MG_Backend/DirectVulkan/Renderer/VkBufferManager.cpp:321`）、`whwi` 其中由 indirect 读引起的。定义在 `MobileGL/MG_Util/Metrics/PipeStats.h:280-285`，打印在 `MobileGL/MG_Util/Metrics/PipeStats.cpp:697`。
- 改后剩下的 6 次等待全部来自 `ReadbackWireBuffer`（`VkBufferManager.cpp:574`，临时探针取调用方地址 + `addr2line`，未提交）：应用对 6 个 store 的回读，改前也是这 6 次（327 − 321）。
- 同一 trace（apitrace dump 到 target call）：322 次 `glMultiDrawElementsIndirect`、2901 次 `glDispatchCompute`、9 次 `glMemoryBarrier`；前 320 次 draw 与其 cull dispatch 之间没有应用 barrier，只有最后 2 次前有 `GL_COMMAND_BARRIER_BIT`。每次 draw 前都有 compute 写它的命令 store，所以 `wibar` = draw 数。
- create-instancing 没有 indirect draw，也没有等待：它的 p99 差异不归 D。
## 3. monolith 何时在 CPU 上读 indirect 字（D 只在同样的场合保留 CPU 路径）
| monolith 场合 | 位置 | wire 臂（本包后） |
|---|---|---|
| 没绑 `GL_DRAW_INDIRECT_BUFFER`：命令在 client 内存 | `DirectVulkan.cpp:684`、`:825`、`:884` | client 在兼容档拒绝（`MobileGL/MG_Remote/Client/EmitTables.cpp:730` `CLIENT_COMMANDS`）；store 为空则具名拒绝 `IndirectBufferUnbound` |
| `glMultiDrawArraysIndirectCount`：总在 CPU 读 count 字 | `DirectVulkan.cpp:720-757` | 原生 `vkCmdDrawIndirectCount`（`MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.cpp:15942` 加载，只在 disaggregated 构建） |
| `glMultiDrawElementsIndirectCount`：无 `VK_KHR_draw_indirect_count`，或 maxdrawcount > 1 且无 `multiDrawIndirect` → CPU 循环 | `VulkanRenderer.cpp:13421-13444` | 同条件（`WireDraw.inc:623-628`）返回 false，走原 CPU 展开（`DirectVulkan.cpp:390-413`，计入 `wixp`） |
| 无 `multiDrawIndirect` 或 stride 不是 4 的倍数：逐条原生 indirect，仍由 GPU 读 | `VulkanRenderer.cpp:13510`、`:13615` | 同（`WireDraw.inc:725-733`）；stride 非 4 倍数在 GL 是 INVALID_VALUE，wire 上具名拒绝 `IndirectCommandRange` |
| client 顶点数组 + arrays indirect：CPU 读命令定上传范围，draw 仍原生 | `VulkanRenderer.cpp:13576` | 不可达：client 把 client 数组拷成自有 buffer |
| `GL_LINE_LOOP`：indirect 不改写，退化为 LINE_STRIP | `MobileGL/MG_Util/Converters/MGToVk/RenderStateEnumConverter.cpp:27` | 同 |
| uint8 索引无扩展 / 非固定 restart 索引 / 非对齐偏移：整段索引在 CPU 改写，命令仍 GPU 读 | 两臂都在各自的索引上传里 | `WireDraw.inc:391`，索引视图是整个 EBO（`WireDraw.inc:674`，同 monolith） |
| baseInstance / gl_DrawID | 原生（monolith 不查 `drawIndirectFirstInstance`） | 原生；旧 CPU 展开的第 3 档把 gl_DrawID 全给 0（`VulkanRenderer.cpp:13154`），原生反而更对 |
## 4. 设计
| 项 | 内容 |
|---|---|
| 入口 | `DrawWireIndirect`（`DirectVulkan.cpp:381`）→ `DrawWireIndirectNative`；false 才走 CPU 展开 |
| store | `AcquireWireSlice` 取 indirect / count / 元素 store（打 `lastUseSerial`，之后的 `glBufferSubData` 会排在这次读之后）；范围、4 字节对齐、count 字越界都是具名拒绝（`MobileGL/MG_Backend/DirectVulkan/Renderer/WireDeclines.def:109-112`） |
| draw 准备 | 复用 `SetupDraw` → `SetupWireDraw`，带 `IndirectDrawBuffer`；该 aspect 原来是 `buffer-legacy-arm` Fatal，现只对 monolith 入口仍 Fatal（`WireDraw.inc:213`，`m_wireNativeIndirectDraw`）；顶点流转换不再按 first+count 截断（`WireDraw.inc:319`） |
| 发射 | 同 monolith 的三档：COUNT 原生、`multiDrawIndirect` 一次发、否则逐条；另按 `maxDrawIndirectCount` 分块（`WireDraw.inc:712-733`） |
| barrier | `MarkWireBufferGpuWritten` 置 `indirectReadBarrierPending`（`VkBufferManager.cpp:611`），`TakeWireIndirectReadBarrier`（`VkBufferManager.cpp:624`）取走；有则结束 render pass 并记 `ALL_COMMANDS/MEMORY_WRITE → DRAW_INDIRECT/INDIRECT_COMMAND_READ`。transfer 写已有 `MEMORY_READ` 后置 barrier（`VkBufferManager.cpp:514`、`:1437`），不置位。应用自己的 `glMemoryBarrier(GL_COMMAND_BARRIER_BIT)` 仍走 `BuildMemoryBarrierForGlBarriers`（`VulkanRenderer.cpp:7873`），与 monolith 一样 |
| G1 | 全部在 `MOBILEGL_BUILD_DISAGGREGATED` / `MOBILEGL_PIPE_PUSH` 下；pull 构建 `added=0 removed=0` |
## 5. 门与 red-once
用例 `WireIndirectDrawScenario`（`MobileGL/MG_IntegrationTest/Scenarios/WireIndirectDrawScenario.cpp:225`、`:274`、`:302`、`:340`），读数经 `MobileGL/MG_IntegrationTest/Harness/WireIndirectPeek.cpp:22`（进程内 PipeStats，只在 inproc 臂看得到 server）。登记：`MobileGL/MG_IntegrationTest/CMakeLists.txt:4651-4684`——Magma 门控三臂 `DirectVulkan.{Split,Spawn,Tcp}.Indirect.`、DirectGLES 三臂（经宏，含 Magma 信息层）、两后端 monolith（环境发现）。
| 用例 | 断言 | 两后端 / 各臂 |
|---|---|---|
| `EveryIndirectFormIsIssuedNativelyFromItsStore` | 6 种 indirect 形态像素；inproc Magma：wind=6、wixp=0、whw=0、wibar=0 | 两后端全臂 |
| `ComputeWrittenCommandsDrawWithoutAHostWait` | compute 写命令（初值毒化为别的条带）+ 应用 barrier，像素；inproc Magma：wind=2、wixp=0、whw=0、wibar=2 | 两后端全臂 |
| `ComputeWrittenCountWordIsTheOneTheGpuReads` | compute 写 count 字，像素；inproc Magma 计数 | Magma；Espryt 跳过（count 字读 server 影子，OQ15 / P8-C）；Magma monolith 跳过 arrays-count 半（§6） |
| `ShaderWrittenCommandsWithoutAnApplicationBarrierAreOrdered` | Flywheel 形状（无应用 barrier）；inproc Magma：每次写恰好 1 个 barrier（wibar=2 / 3 次 draw） | Magma split 三臂；monolith 与 Espryt 跳过 |
| red-once | 变异 | 载体 | 观察 |
|---|---|---|---|
| 1 | `DrawWireIndirectNative` 开头 `return false`（强制 CPU 路径） | 4 用例 × 3 个 inproc 登记（`.Split.Indirect.` 门控、`.Split.` 信息层、`.Split.Full.`） | 12 红；`ComputeWrittenCommandsDrawWithoutAHostWait`：nativeDraws 0、cpuExpansions 2、indirectWaits 2、hostWaits 2；spawn/tcp 只断言像素，保持绿（CPU 路径像素是对的） |
| 2 | 去掉 barrier（`if (false && ...)`） | 同上 | 3 个 compute 写用例 × 3 个 inproc 登记 = 9 红，全部是 `delta.barriers` 0 ≠ 2；**像素在 llvmpipe 上不红**（lavapipe 按序执行），spawn/tcp 绿 |
| 3 | 去掉 Magma monolith 的 arrays-count 跳过 | `DirectVulkan.WireIndirectDrawScenario.ComputeWrittenCountWordIsTheOneTheGpuReads` | 红：compute 写的 count=2，条带 1、2 未画（monolith 读到陈旧影子 count 0） |
| fixture 门 | 改前 = 旧 CPU 路径 | create-indirect × Magma × {inproc, spawn} | whwi 321 / 322 > 0（§2 改前行）；改后 0 |
## 6. monolith 缺陷（记给 dev，ID-P8-3，本包不修）
| 缺陷 | 位置 | 证据 |
|---|---|---|
| Magma monolith `glMultiDrawArraysIndirectCount` 从前端影子读 count 字，不 `SyncGpuWrites`：着色器写的 count 读成旧值 | `DirectVulkan.cpp:749-757` | red-once 3（主机 lavapipe 红） |
| Magma monolith 对"compute 写 → indirect 读"没有任何依赖：不补 barrier，render pass 的外部依赖也不含 `COMPUTE_SHADER` / `DRAW_INDIRECT`（`MobileGL/MG_Backend/DirectVulkan/Renderer/VkRenderPassManager.cpp:1413-1419`）。create-indirect 的 320 次 cull→draw 不发 `glMemoryBarrier`（GL 未定义），真 GPU 上 cull 结果与 indirect 读竞争 | `VulkanRenderer.cpp:7727-7787`（dispatch 后无 barrier） | 推断：主机 lavapipe 按序执行，red 不出；可能与 ID-P7-4（红米上 create-indirect monolith 坏）有关，需设备验证 |
## 7. 找到但未做
- `DispatchComputeIndirect` 的 wire 臂同样经 `ReadWireBuffer` 在 CPU 读 group 数（`DirectVulkan.cpp:938-947`），GPU 写过则整 GPU 等待；create fixtures 里没有 `glDispatchComputeIndirect`，本包未改（monolith 是原生 `vkCmdDispatchIndirect`）。
- 补的 barrier 只覆盖 indirect 读；同一批 cull 写的实例 SSBO 被顶点着色器读时，两臂都没有 compute→vertex 的依赖（应用没发 barrier 时），不在 D 范围。
- 同一 store 既作 indirect 又作本 draw 的 SSBO 时，SetupWireDraw 解析描述符会再次置位，下一次 indirect draw 会多一个 barrier（保守，不影响正确性）。