swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-09-12 14:18:31 +09:00
Code Issues Packages Projects Releases Wiki Activity

[Refactor] (MG_State, MG_Backend): PipeResource storage layer + zero-copy coherent persistent maps

Browse Source 
Introduce a Mesa pipe_resource-style PipeResource that owns a GL buffer's bytes
and its backend GPU resource, abstracting WHERE the authoritative bytes live:
 - Shadow mode (non-persistent buffers): a CPU Vector; the backend keeps its own
   GPU copy in sync via BufferBackendOps, exactly as before.
 - Persistent mode (coherent GL_MAP_PERSISTENT maps): the backend's host-visible,
   COHERENT, persistently-mapped GPU memory is the single source of truth. The app
   writes into it directly, every reader resolves against it, and NO per-write
   backend transfer happens. The CPU shadow is released.

BufferObject no longer owns a raw shadow Vector; it holds a PipeResource and
exposes one accessor, MappedData(), that all readers go through. Every buffer-data
consumer (UBO payload, PBO texture upload, indirect draws, resident/streamed
uploads, both backends) was migrated from GetDataReadOnly()->data() to
MappedData(), so a persistent buffer's readers see GPU memory - not a stale
shadow. That stale-shadow inconsistency is what corrupted rendering (wrong UBOs ->
misplaced/"lost" vertices) in the first zero-copy attempt (625c8a6, reverted in
896cafc); routing every consumer through one accessor makes it structurally
impossible.

Backends provide the map via BufferBackendOps::AcquirePersistentMap:
 - DirectVulkan: a HOST_VISIBLE|HOST_COHERENT (required, not just requested),
   persistently mapped resident VkBuffer carrying every usage, seeded from the
   shadow, never recreated; AcquireResidentSlice binds it directly.
 - DirectGLES: EXT_buffer_storage immutable persistent+coherent glMapBufferRange,
   falling back to the shadow when the extension is absent.

Fixes the ~7GB GpuMemory OOM + 100%-CPU/ANR running modern Blaze3D Minecraft on
both Magma and Espryt (per-draw whole-buffer re-upload of the coherent persistent
ring buffer), without the coherency/stale-read hazards of the reverted attempt.

BufferTest: zero-copy stress guard (15,360 draws -> 0 per-draw transfers, and every
reader resolves to GPU memory) + a shadow-fallback test. Host suite: 203/203 pass.
Device verification pending.
This commit is contained in:
swung0x48
2026-07-11 20:18:38 -04:00
parent 0f99d93300
commit c1743aa42d
15 changed files with 577 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MobileGL/MG_State/GLState/BufferState/BufferObject.cpp
+71 -35
View File
@@ -24,12 +24,11 @@ namespace MobileGL::MG_State::GLState {
BufferObject::BufferObject(Uint externalIndex)
: m_externalIndex(externalIndex), m_size(0), m_usage(BufferUsage::StaticDraw), m_isMapped(false),
m_mappingAccess(BufferMappingAccessBit::Null), m_mappedRange({0, 0}), m_dataPtr(MakeShared<Data>()),
m_ownsStagingData{} {}
m_mappingAccess(BufferMappingAccessBit::Null), m_mappedRange({0, 0}), m_ownsStagingData{} {}
BufferObject::~BufferObject() {
if (m_backendResource && g_bufferBackendOps && g_bufferBackendOps->OnDestroy) {
g_bufferBackendOps->OnDestroy(std::move(m_backendResource));
if (m_resource.Backend() && g_bufferBackendOps && g_bufferBackendOps->OnDestroy) {
g_bufferBackendOps->OnDestroy(m_resource.ReleaseBackend());
}
}
@@ -56,13 +55,22 @@ namespace MobileGL::MG_State::GLState {
}
}
void BufferObject::NotifyContentWrite(SizeT offset, SizeT size) {
if (m_resource.IsGpuResident()) {
// The write already landed in coherent GPU memory; the backend has no separate
// copy to sync. Only bump the serial so cached transient slices invalidate.
++m_changeSerial;
return;
}
NotifySubData(offset, size);
}
void BufferObject::Respecify(SizeT size, const void* data) {
ReleaseMemory();
m_size = size;
m_dataPtr->reserve(std::bit_ceil(size)); // power-of-2 reserve
m_dataPtr->resize(size);
m_resource.ResizeShadow(size);
if (data && size > 0) {
Memcpy(m_dataPtr->data(), data, size);
Memcpy(m_resource.Bytes(), data, size);
}
m_isImmutableStorage = false;
m_storageFlags = 0;
@@ -76,12 +84,11 @@ namespace MobileGL::MG_State::GLState {
void BufferObject::AllocateImmutableStorage(SizeT size, const void* data, GLbitfield storageFlags) {
ReleaseMemory();
m_size = size;
m_dataPtr->reserve(std::bit_ceil(size));
m_dataPtr->resize(size);
m_resource.ResizeShadow(size);
if (data) {
Memcpy(m_dataPtr->data(), data, size);
Memcpy(m_resource.Bytes(), data, size);
} else if (size > 0) {
Memset(m_dataPtr->data(), 0, size);
Memset(m_resource.Bytes(), 0, size);
}
m_isImmutableStorage = true;
m_storageFlags = storageFlags;
@@ -94,8 +101,8 @@ namespace MobileGL::MG_State::GLState {
data.size, m_size);
MOBILEGL_ASSERT(!m_isMapped || (m_mappingAccess & BufferMappingAccessBit::Persistent),
"Cannot upload data while buffer is non-persistently mapped.");
Memcpy(m_dataPtr->data() + atOffset, data.data, data.size);
NotifySubData(atOffset, data.size);
Memcpy(m_resource.Bytes() + atOffset, data.data, data.size);
NotifyContentWrite(atOffset, data.size);
}
void BufferObject::SetUsage(BufferUsage usage) {
@@ -105,10 +112,13 @@ namespace MobileGL::MG_State::GLState {
void BufferObject::ReleaseMemory() {
if (!m_isMapped) return;
if (m_mappingAccess & BufferMappingAccessBit::Write) { // if we wrote to the buffer
if (!(m_mappingAccess & BufferMappingAccessBit::FlushExplicit)) { // if we didn't flush explicitly
if (m_mappingAccess & BufferMappingAccessBit::Write) { // if we wrote to the buffer
// A persistent GPU-resident map wrote straight into coherent GPU memory, so
// there is nothing to copy back and no range to push down on unmap.
if (!m_resource.IsGpuResident() &&
!(m_mappingAccess & BufferMappingAccessBit::FlushExplicit)) { // if we didn't flush explicitly
if (!(m_mappingAccess & BufferMappingAccessBit::Persistent)) {
Memcpy(m_dataPtr->data() + m_mappedRange.start, m_stagingData.data(),
Memcpy(m_resource.Bytes() + m_mappedRange.start, m_stagingData.data(),
m_mappedRange.end - m_mappedRange.start);
}
NotifyFlushMappedRange(m_mappedRange, m_mappingAccess);
@@ -135,14 +145,20 @@ namespace MobileGL::MG_State::GLState {
MOBILEGL_ASSERT(end <= m_mappedRange.end, "Flush range out of bounds: mappedRange.end (%zu) < end (%zu)",
m_mappedRange.end, end);
// FLUSH_EXPLICIT maps are never GPU-resident (only coherent maps are adopted), so
// the staged bytes must be copied into the shadow before the backend reads them.
if (!(m_mappingAccess & BufferMappingAccessBit::Persistent)) {
Memcpy(m_dataPtr->data() + start, m_stagingData.data() + offset, length);
Memcpy(m_resource.Bytes() + start, m_stagingData.data() + offset, length);
}
NotifyFlushMappedRange({start, end}, m_mappingAccess);
}
void BufferObject::SyncPersistentMappedRange() {
if (!m_isMapped) return;
// GPU-resident: the app already wrote directly into coherent GPU memory. This is
// the whole point of the persistent-map path - the per-draw whole-buffer re-upload
// that used to run here is gone.
if (m_resource.IsGpuResident()) return;
if (!(m_mappingAccess & BufferMappingAccessBit::Persistent)) return;
if (!(m_mappingAccess & BufferMappingAccessBit::Write)) return;
if (m_mappingAccess & BufferMappingAccessBit::FlushExplicit) return;
@@ -153,6 +169,8 @@ namespace MobileGL::MG_State::GLState {
void BufferObject::SyncMappedRangeForGpuRead(Range1D range) {
if (!m_isMapped) return;
// GPU-resident maps already alias GPU-visible memory; nothing to push.
if (m_resource.IsGpuResident()) return;
if (!(m_mappingAccess & BufferMappingAccessBit::Persistent)) return;
if (!(m_mappingAccess & BufferMappingAccessBit::Write)) return;
// Non-FLUSH_EXPLICIT persistent maps are already covered wholesale by
@@ -170,7 +188,7 @@ namespace MobileGL::MG_State::GLState {
MOBILEGL_ASSERT(atOffset + data.size <= m_size,
"WritebackFromBackend out of bounds: atOffset (%zu) + data.size (%zu) > m_size (%zu)", atOffset,
data.size, m_size);
Memcpy(m_dataPtr->data() + atOffset, data.data, data.size);
Memcpy(m_resource.Bytes() + atOffset, data.data, data.size);
++m_changeSerial;
}
@@ -181,15 +199,15 @@ namespace MobileGL::MG_State::GLState {
"UploadSubData out of bounds: atOffset (%zu) + data.size (%zu) > m_size (%zu)", atOffset,
data.size, m_size);
Memcpy(m_dataPtr->data() + atOffset, data.data, data.size);
NotifySubData(atOffset, data.size);
Memcpy(m_resource.Bytes() + atOffset, data.data, data.size);
NotifyContentWrite(atOffset, data.size);
}
void BufferObject::DownloadSubData(void* dst, SizeT atOffset, SizeT size) const {
MOBILEGL_ASSERT(atOffset + size <= m_size,
"DownloadSubData out of bounds: atOffset (%zu) + size (%zu) > m_size (%zu)", atOffset, size,
m_size);
Memcpy(dst, m_dataPtr->data() + atOffset, size);
Memcpy(dst, m_resource.Bytes() + atOffset, size);
}
void BufferObject::CopyDataFrom(const SharedPtr<BufferObject>& src, SizeT srcOffset, SizeT dstOffset, SizeT size) {
@@ -204,9 +222,8 @@ namespace MobileGL::MG_State::GLState {
"Destination buffer copy out of bounds: dstOffset (%zu) + size (%zu) > m_size (%zu)", dstOffset,
size, m_size);
const Uint8* srcData = src->m_dataPtr->data() + srcOffset;
Memcpy(m_dataPtr->data() + dstOffset, srcData, size);
NotifySubData(dstOffset, size);
Memcpy(m_resource.Bytes() + dstOffset, src->m_resource.Bytes() + srcOffset, size);
NotifyContentWrite(dstOffset, size);
}
void* BufferObject::AcquireMemory(Bool markMapped, Bool read, Bool write) {
@@ -222,14 +239,14 @@ namespace MobileGL::MG_State::GLState {
if (!(m_mappingAccess &
(BufferMappingAccessBit::InvalidateRange | BufferMappingAccessBit::InvalidateBuffer))) {
Memcpy(m_stagingData.data(), m_dataPtr->data(), m_size);
Memcpy(m_stagingData.data(), m_resource.Bytes(), m_size);
}
return m_stagingData.data();
}
}
return m_dataPtr->data();
return m_resource.Bytes();
}
void* BufferObject::AcquireMemoryRange(Range1D range, Flags<BufferMappingAccessBit> access) {
@@ -242,7 +259,20 @@ namespace MobileGL::MG_State::GLState {
if (access & BufferMappingAccessBit::Persistent) {
m_ownsStagingData = false;
return m_dataPtr->data() + range.start;
// Zero-copy: for a coherent (non-FLUSH_EXPLICIT) persistent write map, ask the
// active backend for host-visible, coherent GPU storage and adopt it as the
// single source of truth. The backend seeds it from the current shadow before
// returning; AdoptPersistentMap then releases the shadow. Falls back to the
// shadow when the backend declines (returns null). Only attempted once - the
// storage is immutable and outlives unmap/remap.
if (!m_resource.IsGpuResident() && (access & BufferMappingAccessBit::Write) &&
!(access & BufferMappingAccessBit::FlushExplicit) && g_bufferBackendOps &&
g_bufferBackendOps->AcquirePersistentMap) {
if (void* base = g_bufferBackendOps->AcquirePersistentMap(*this)) {
m_resource.AdoptPersistentMap(base);
}
}
return m_resource.Bytes() + range.start;
}
if (access & BufferMappingAccessBit::Write) {
@@ -250,18 +280,22 @@ namespace MobileGL::MG_State::GLState {
m_ownsStagingData = true;
if (!(access & (BufferMappingAccessBit::InvalidateRange | BufferMappingAccessBit::InvalidateBuffer))) {
Memcpy(m_stagingData.data(), m_dataPtr->data() + range.start, m_stagingData.size());
Memcpy(m_stagingData.data(), m_resource.Bytes() + range.start, m_stagingData.size());
}
return m_stagingData.data();
} else {
m_ownsStagingData = false;
return m_dataPtr->data() + range.start;
return m_resource.Bytes() + range.start;
}
}
const SharedPtr<Data>& BufferObject::GetDataReadOnly() const {
return m_dataPtr;
const Uint8* BufferObject::MappedData() const {
return m_resource.Bytes();
}
Bool BufferObject::IsBackendPersistentMapped() const {
return m_resource.IsGpuResident();
}
SizeT BufferObject::GetSize() const {
@@ -281,11 +315,11 @@ namespace MobileGL::MG_State::GLState {
}
const SharedPtr<BackendBufferResource>& BufferObject::GetBackendResource() const {
return m_backendResource;
return m_resource.Backend();
}
void BufferObject::SetBackendResource(SharedPtr<BackendBufferResource> resource) {
m_backendResource = std::move(resource);
m_resource.SetBackend(std::move(resource));
}
Bool BufferObject::IsMapped() const {
@@ -299,12 +333,14 @@ namespace MobileGL::MG_State::GLState {
void* BufferObject::GetMappedPointer() const {
if (!m_isMapped) return nullptr;
if (m_mappingAccess & BufferMappingAccessBit::Persistent) {
return const_cast<Uint8*>(m_dataPtr->data()) + m_mappedRange.start;
// GPU-resident maps return the coherent GPU pointer; shadow-backed persistent
// maps return the shadow. m_resource.Bytes() resolves both.
return const_cast<Uint8*>(m_resource.Bytes()) + m_mappedRange.start;
}
if (m_ownsStagingData) {
return const_cast<Uint8*>(m_stagingData.data());
}
return const_cast<Uint8*>(m_dataPtr->data()) + m_mappedRange.start;
return const_cast<Uint8*>(m_resource.Bytes()) + m_mappedRange.start;
}
Flags<BufferMappingAccessBit> BufferObject::GetMappingAccess() const {
MobileGL/MG_State/GLState/BufferState/BufferObject.h
+33 -10
View File
@@ -9,6 +9,7 @@
#pragma once
#include <Includes.h>
#include <MG_Util/Math/VectorTypes.h>
#include "PipeResource.h"
namespace MobileGL {
enum class BufferTarget {
@@ -59,13 +60,9 @@ namespace MobileGL {
namespace MG_State::GLState {
class BufferObject;
// Opaque, refcounted handle to the backend's storage for one buffer
// (the pipe_resource analogue). The frontend owns the reference; the
// active backend derives from it and attaches its own payload.
class BackendBufferResource {
public:
virtual ~BackendBufferResource() = default;
};
// BackendBufferResource and PipeResource (the storage abstraction that holds
// either the CPU shadow or the backend's persistently-mapped GPU memory) live
// in PipeResource.h.
// Immediate buffer transfer interface implemented by the active backend
// (the pipe_context buffer-op analogue). Ops are invoked at GL call time,
@@ -91,6 +88,17 @@ namespace MobileGL {
// Final release of the backend resource (called from ~BufferObject).
// The backend defers actual destruction until the GPU is done with it.
void (*OnDestroy)(SharedPtr<BackendBufferResource>&& resource) = nullptr;
// Zero-copy persistent mapping. For a coherent (non-FLUSH_EXPLICIT) persistent
// write map, the backend may hand back a host-visible, COHERENT, persistently
// mapped pointer into its own GPU storage for the whole buffer [0, size),
// created with every buffer usage and seeded from the shadow. From that point
// the GPU buffer is the single source of truth: the app writes into it
// directly, all reads/writes resolve against it (HostData()), and NO further
// backend transfer ops are dispatched for this buffer. Returns nullptr when the
// backend cannot back the map; the frontend then keeps the CPU-shadow model.
// Must be idempotent: a second call for an already-backed buffer returns the
// same base pointer.
void* (*AcquirePersistentMap)(BufferObject& bufferObject) = nullptr;
};
// Registered by the active backend at init, cleared at shutdown.
@@ -148,7 +156,16 @@ namespace MobileGL {
BufferUsage GetUsage() const;
Range1D GetMappedRange() const;
void* GetMappedPointer() const;
const SharedPtr<Data>& GetDataReadOnly() const;
// Host-visible base pointer to the buffer's authoritative bytes for
// [0, GetSize()): the coherent persistent GPU map when the buffer is
// persistent-resident, otherwise the CPU shadow. Every reader goes through
// this so no consumer branches on where the bytes live (the class of bug
// that a partial persistent-map redirect would reintroduce).
const Uint8* MappedData() const;
// True once the buffer's bytes were adopted into backend GPU memory (a
// coherent persistent map): reads/writes hit GPU memory and no per-write
// backend transfer op is dispatched.
Bool IsBackendPersistentMapped() const;
Flags<BufferMappingAccessBit> GetMappingAccess() const;
GLbitfield GetStorageFlags() const;
Uint GetExternalIndex() const;
@@ -163,11 +180,18 @@ namespace MobileGL {
void NotifyRespecify();
void NotifySubData(SizeT offset, SizeT size);
void NotifyFlushMappedRange(Range1D range, Flags<BufferMappingAccessBit> appAccess);
// A content write of [offset, offset+size) just landed in m_resource. For a
// persistent GPU-resident buffer the bytes are already in coherent GPU memory,
// so this only bumps the change serial; otherwise it dispatches a backend
// SubData transfer to sync the backend's separate GPU copy.
void NotifyContentWrite(SizeT offset, SizeT size);
const Uint m_externalIndex = 0;
SizeT m_size = 0;
BufferUsage m_usage = BufferUsage::StaticDraw;
SharedPtr<Data> m_dataPtr;
// Owns the buffer's bytes (CPU shadow or backend persistent GPU map) and
// the backend GPU resource. All data access goes through it.
PipeResource m_resource;
Bool m_isMapped;
Flags<BufferMappingAccessBit> m_mappingAccess;
Bool m_isImmutableStorage = false;
@@ -176,7 +200,6 @@ namespace MobileGL {
Range1D m_mappedRange;
Vector<Uint8> m_stagingData;
Bool m_ownsStagingData;
SharedPtr<BackendBufferResource> m_backendResource;
};
} // namespace MG_State::GLState
} // namespace MobileGL
MobileGL/MG_State/GLState/BufferState/PipeResource.h
+83
View File
@@ -0,0 +1,83 @@
// MobileGL - MobileGL/MG_State/GLState/BufferState/PipeResource.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_Util/Types.h>
#include <bit>
namespace MobileGL::MG_State::GLState {
// Opaque, refcounted handle to the backend's GPU storage for one buffer
// (the driver-side resource). The active backend derives from it and attaches
// its own payload (VkBufferResource / GLESBufferResource). Held by PipeResource.
class BackendBufferResource {
public:
virtual ~BackendBufferResource() = default;
};
// Mesa pipe_resource analogue for a GL buffer's storage. It owns the buffer's
// bytes and its backend GPU resource, and abstracts WHERE the authoritative
// bytes live so no caller has to branch on the mode:
//
// - Shadow mode (default, non-persistent buffers): the bytes live in a CPU
// Vector (the shadow). GL writes mutate the shadow; the active backend keeps
// its own GPU copy in sync via BufferBackendOps (glBufferData/SubData/...).
//
// - Persistent mode (coherent GL_MAP_PERSISTENT maps): the bytes live in the
// backend's host-visible, COHERENT, persistently-mapped GPU memory. That GPU
// buffer is the single source of truth - the app writes into it directly,
// every read/write resolves against it, and NO per-write backend transfer
// happens. The CPU shadow is released on adoption.
//
// Bytes() always returns a host-visible base pointer valid for [0, size) in both
// modes, so readers/writers just call Bytes() (the size lives on the owning
// BufferObject). (Named Bytes(), not Data(), to avoid colliding with the type
// alias Data = Vector<Uint8> used for the shadow.)
class PipeResource {
public:
Uint8* Bytes() { return m_gpuMapped != nullptr ? static_cast<Uint8*>(m_gpuMapped) : m_shadow->data(); }
const Uint8* Bytes() const {
return m_gpuMapped != nullptr ? static_cast<const Uint8*>(m_gpuMapped) : m_shadow->data();
}
// True once the buffer's bytes have been adopted into backend GPU memory.
Bool IsGpuResident() const { return m_gpuMapped != nullptr; }
// Shadow (re)allocation for non-persistent storage (glBufferData /
// glBufferStorage before any persistent map). Mirrors the previous
// power-of-two reserve + exact resize of the old m_dataPtr.
void ResizeShadow(SizeT size) {
m_shadow->reserve(std::bit_ceil(size == 0 ? SizeT{1} : size));
m_shadow->resize(size);
}
// Direct shadow access, used only by the backend's upload-from-shadow path,
// which never runs for a GPU-resident (persistent) buffer.
Data& Shadow() { return *m_shadow; }
const Data& Shadow() const { return *m_shadow; }
// Transition to persistent GPU residency: adopt the backend's coherent
// mapped base as the source of truth and drop the CPU shadow. The caller
// must have already seeded the GPU memory from the shadow (via the backend
// AcquirePersistentMap op) before calling this.
void AdoptPersistentMap(void* mappedBase) {
m_gpuMapped = mappedBase;
m_shadow->clear();
m_shadow->shrink_to_fit();
}
// Backend GPU resource, owned here in both modes.
const SharedPtr<BackendBufferResource>& Backend() const { return m_backend; }
void SetBackend(SharedPtr<BackendBufferResource> backend) { m_backend = std::move(backend); }
SharedPtr<BackendBufferResource> ReleaseBackend() { return std::move(m_backend); }
private:
SharedPtr<Data> m_shadow = MakeShared<Data>();
void* m_gpuMapped = nullptr;
SharedPtr<BackendBufferResource> m_backend;
};
} // namespace MobileGL::MG_State::GLState