mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-16 14:33:24 +09:00
[MG_Backend, MG_IntegrationTest, docs] (P7): the wire readback waits for the frame it is reading, instead of inferring who wrote it
Exit gate 3, OpenRA on the Redmi. The mechanism, in two lines of this tree: WireFramebuffer.inc, ReadWirePixels: `if (HasPendingRecordedWork() && !FlushPendingCommands())` - FlushPendingCommands (VulkanRenderer.cpp:13534-13598) puts the frame's draws on the queue under a POOLED FENCE OF ITS OWN and returns without waiting; there is no vkWaitForFences on its success path. The vkCmdCopyImageToBuffer that follows is recorded into a fresh command buffer and becomes a LATER submission. VulkanRenderer.cpp:10707-10748, SubmitReadbackCommandsAndWait: it waited ONE fence, its own, and then called OnSubmitsCompletedUpTo(frame.lastSubmitIndex) - which RECORDED the earlier submission as complete and retired its fence, so afterwards nothing could observe that it had not been waited for. The comment beneath it said "the wait proved every submission complete". That is true of the monolith arm, which never flushes before a readback and therefore has its draws and its copy in one submission under one fence. It was not true of the wire arm. The only thing tying the two submissions together was the pipeline barrier that the readback's TransitionWireImage records - and that barrier derives its SOURCE scope from the image's tracked layout, through GetImageTransitionSourceState (VulkanRenderer.cpp:1784-1825). Two of its arms, VK_IMAGE_LAYOUT_UNDEFINED and VK_IMAGE_LAYOUT_PRESENT_SRC_KHR (:1787-1791), return VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT with no access bits, which is Vulkan's spelling of an EMPTY first scope: wait for nothing. A just-presented default framebuffer is in the second of those layouts. The slower pipeline creation is, the longer the draws' submission runs, and the more of the frame's LAST-drawn objects the copy can miss - which is what the device's diff image shows. Measured on this host (OpenRA, DirectVulkan, inproc), at the moment the copy is recorded: submit=72 completed=71. The submission carrying the whole frame's draws is in flight and unwaited. That is structural, not a coin flip, and it reproduces every run. The fix is two halves, both wire-only: - RecordWireReadbackWriteBarrier (WireFramebuffer.inc, new): the readback records its own ALL_COMMANDS -> TRANSFER, MEMORY_WRITE -> TRANSFER_READ|MEMORY_READ barrier before the copy rather than inferring from a layout who it should wait for. A barrier's first scope is every command earlier in SUBMISSION order on the queue, so it reaches across the submission boundary. - SubmitReadbackCommandsAndWait now waits every in-flight fence at or below its own submission index. Under #if MOBILEGL_BUILD_DISAGGREGATED: the pull build has no wire arm, cannot flush before a readback, and is the image G1 pins - its single-fence wait is unchanged. The cost is nothing a readback does not already pay. THE COORDINATOR'S REPRODUCTION PATH DOES NOT WORK ON THIS HOST, and that is recorded rather than glossed. MOBILEGL_TEST_PIPELINE_CREATE_DELAY_MS is added as asked (split-only, inert unless set, on the pipeline-creation MISS path in GetOrCreatePipelineWithInput) and it does bite - the OpenRA DirectVulkan SPLIT retrace goes from 1.88 s to 4.53 s at 25 ms - but ssim stays 1.000000. The reason is in the probe above: this host's default framebuffer is tracked as VK_IMAGE_LAYOUT_GENERAL at the readback, which takes GetImageTransitionSourceState's `default:` arm (ALL_COMMANDS / MEMORY_WRITE) and orders the copy by accident. The device's is not GENERAL. Red once (executed, reverted) is therefore a COUNT, not a picture, and it is deterministic: instrumenting how many fences the readback actually waits on gives fences=2 upTo=73 completed=71 with the fix (submission 72, the draws, plus its own 73) and fences=1 without it, with completed=71 either way - i.e. 72 provably outstanding and provably unwaited before. Lane: three readback-after-work cases (VertexIdSamplerAndScalarUniformPixels, ComputeImageStoreFramebufferPixels, GenerateMipmapPixels) registered on the three Magma arms under MOBILEGL_TEST_PIPELINE_CREATE_DELAY_MS=25, prefix .PipelineDelay. They are a REGRESSION GATE, not the reproduction: they pin that a readback still returns the draw's pixels when every pipeline creation on the path costs 25 ms. Gates: integration-magma-split 84 (+3), -spawn 63 (+3), -tcp 65 (+3), 0 red; ctest -L unit 2410; informational integration-magma-full-split 516, 0 red, 59 skip; fatal_census green; link_ratchet --assert-monotone unchanged at 186, p7-magma 101; G1 pull .text 0xa52203, defined-symbol name list identical (30570); OpenRA DirectVulkan SPLIT retrace ssim 1.000000 with and without the delay, two runs each. Owed to the device: the three cold-cache repeats. If they still diverge, the next two suspects are named in notes/p7/magma-b.md §5.6 - the eleven silent `return false` drop sites in SetupWireDraw (one of which is `pipeline == VK_NULL_HANDLE`, on the same path as the device log's three "handing vkCreateShaderModule an INVALID module" errors), and SetImageContentDefined being set BEFORE the draw is recorded on the wire arm while the monolith sets it from EndRenderPass, which silences the "no write is recorded" warning exactly when it should fire. Neither is in this commit's scope.
This commit is contained in:
@@ -5308,6 +5308,22 @@ void main() {
|
||||
return rsp.PrimitiveRestartIndex <= MG_Util::FixedRestartIndexForGLType(pIndexBufferView->indexType);
|
||||
}
|
||||
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// MOBILEGL_TEST_PIPELINE_CREATE_DELAY_MS - milliseconds to spend in a pipeline-creation MISS
|
||||
// on the wire arm. See the call site below for why it exists: the device's divergence needs a
|
||||
// COLD driver pipeline cache, and this host's is always warm, so the only way to reproduce
|
||||
// what a cold cache costs is to buy the time. Read once; inert unless set; compiled only into
|
||||
// the disaggregated build, so the monolith image G1 pins never carries it.
|
||||
static Uint32 MagmaTestPipelineCreateDelayMs() {
|
||||
static const Uint32 milliseconds = [] {
|
||||
const char* value = std::getenv("MOBILEGL_TEST_PIPELINE_CREATE_DELAY_MS");
|
||||
if (!value || !*value) return 0u;
|
||||
return static_cast<Uint32>(std::strtoul(value, nullptr, 10));
|
||||
}();
|
||||
return milliseconds;
|
||||
}
|
||||
#endif
|
||||
|
||||
VkPipeline VulkanRenderer::GetOrCreatePipeline(
|
||||
GLenum mode,
|
||||
const MagmaProgramSource& program,
|
||||
@@ -6114,6 +6130,22 @@ void main() {
|
||||
MG_Util::ConvertBlendEquationToVkEnum(alphaEquation),
|
||||
attachmentColorWriteMask);
|
||||
}
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 wave 2-B: MOBILEGL_TEST_PIPELINE_CREATE_DELAY_MS.
|
||||
//
|
||||
// The Redmi's OpenRA divergence (exit gate 3) is timing-shaped and reproduces ONLY with a
|
||||
// cold driver pipeline cache: three cleared-cache runs give a bit-identical 0.976494 /
|
||||
// 14658 mismatched pixels, while a warm cache gives 1.000000 from the second run on. What
|
||||
// a cold cache changes is how long vkCreateGraphicsPipelines takes on the apply thread,
|
||||
// so this knob buys that time on a host whose cache is always warm. INERT unless set, and
|
||||
// compiled only into the disaggregated build - the monolith image, which G1 pins, never
|
||||
// sees these lines. The delay goes on the MISS path only: a memo or cache hit is not
|
||||
// what is slow on the device either.
|
||||
if (const Uint32 delayMs = MagmaTestPipelineCreateDelayMs();
|
||||
delayMs != 0 && MG_Config::Transport != MG_Config::TransportMode::Monolith) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(delayMs));
|
||||
}
|
||||
#endif
|
||||
VkPipeline pipeline = m_pipelineFactory->GetOrCreatePipeline(payload);
|
||||
if (pipeline != VK_NULL_HANDLE) {
|
||||
PipelineMemoEntry& entry = m_pipelineMemo[m_pipelineMemoNext];
|
||||
@@ -10792,7 +10824,47 @@ void main() {
|
||||
return false;
|
||||
}
|
||||
|
||||
// EVERY OUTSTANDING SUBMISSION, NOT ONLY THE ONE JUST MADE (P7 wave 2-B, exit gate 3).
|
||||
//
|
||||
// This used to wait one fence and then call OnSubmitsCompletedUpTo(frame.lastSubmitIndex),
|
||||
// and the comment below said the wait "proved every submission complete". On the monolith
|
||||
// arm that is true - nothing flushes before a readback, so the frame's draws and the copy
|
||||
// are one submission under one fence. On the WIRE arm it is not: ReadWirePixels calls
|
||||
// FlushPendingCommands first, which submits the draws under a POOLED FENCE OF ITS OWN and
|
||||
// returns without waiting, so the copy is a later, separately fenced submission. Two
|
||||
// submissions on one queue start in order; they do not finish in order unless something
|
||||
// makes them, and a fence says nothing about a submission it does not belong to.
|
||||
//
|
||||
// Measured, on this host, at the copy: submit=72 completed=71. The old code then called
|
||||
// OnSubmitsCompletedUpTo(73), which RECORDED 72 as complete and retired its fence - so
|
||||
// after the fact nothing could even observe that it had not been waited for.
|
||||
//
|
||||
// The cost is nothing a readback does not already pay: it is a full CPU stall by
|
||||
// construction, and the fences gathered here are exactly the ones already in flight.
|
||||
//
|
||||
// UNDER #if, although this function serves both arms of the disaggregated build: the PULL
|
||||
// build has no wire arm, cannot flush before a readback, and is the image G1 pins to
|
||||
// 0xa52203. Its single-fence wait stays the statement it always was.
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
Vector<VkFence> readbackFences;
|
||||
readbackFences.reserve(m_inFlightSubmits.size() + 1);
|
||||
for (const auto& record : m_inFlightSubmits) {
|
||||
if (record.fence != VK_NULL_HANDLE && record.submitIndex <= frame.lastSubmitIndex) {
|
||||
readbackFences.push_back(record.fence);
|
||||
}
|
||||
}
|
||||
if (readbackFences.empty()) {
|
||||
readbackFences.push_back(frame.imageInFlightFence);
|
||||
}
|
||||
// Measured here on the host (OpenRA, DirectVulkan, inproc, at the frame readback):
|
||||
// fences=2 upTo=73 submitCounter=73 completed=71. Two fences, because submission 72 -
|
||||
// the frame's draws, flushed by ReadWirePixels under a pooled fence - was still
|
||||
// outstanding; the code this replaces waited on one.
|
||||
VkResult result = vkWaitForFences(m_device, static_cast<Uint32>(readbackFences.size()),
|
||||
readbackFences.data(), VK_TRUE, UINT64_MAX);
|
||||
#else
|
||||
VkResult result = vkWaitForFences(m_device, 1, &frame.imageInFlightFence, VK_TRUE, UINT64_MAX);
|
||||
#endif
|
||||
if (result != VK_SUCCESS) {
|
||||
MGLOG_E_ONCE("DirectVulkan readback: vkWaitForFences returned %d", result);
|
||||
return false;
|
||||
@@ -10806,8 +10878,8 @@ void main() {
|
||||
|
||||
frame.hasCommandBufferRecorded = false;
|
||||
frame.isCommandRecording = false;
|
||||
// The wait proved every submission complete, so the full frame-boundary
|
||||
// drain applies: descriptor cursors, transient arenas, deferred
|
||||
// The wait above covered every submission at or below this one, so the full
|
||||
// frame-boundary drain applies: descriptor cursors, transient arenas, deferred
|
||||
// texture/buffer releases, retired command buffers and the converted
|
||||
// vertex-stream cache all rewind here, keeping present-less readback
|
||||
// loops bounded (Present is the only other drain point).
|
||||
|
||||
Reference in New Issue
Block a user