Compare commits

...
21 Commits
Author SHA1 Message Date
swung0x48 7dbe0e5c78 [Docs] (.git-blame-ignore-revs, CONTRIBUTING.md): list the reformat for git blame and update the format rules 2026-10-16 06:30:59 -04:00
swung0x48 f4994ee36e [Chore] (MG_Backend, MG_Backend/DirectGLES, MG_Backend/DirectVulkan, MG_Backend/MGPipe, MG_Backend/Record, MG_Benchmark/Buffer, MG_Benchmark/Container, MG_Benchmark/Driver, MG_Benchmark/Program, MG_Benchmark/ShaderCache, MG_Benchmark/Transpile, MG_Gbm, MG_Impl, MG_Impl/CGLImpl, MG_Impl/DyldInterpose, MG_Impl/EGLImpl, MG_Impl/GLImpl, MG_Impl/GLXImpl, MG_Impl/NSOpenGLImpl, MG_Impl/Pipe, MG_Impl/WGLImpl, MG_IntegrationTest, MG_IntegrationTest/Harness, MG_IntegrationTest/Scenarios, MG_Pipe, MG_Remote, MG_Remote/Client, MG_Remote/Protocol, MG_Remote/Server, MG_Remote/Transport, MG_Remote/Wire, MG_State/EGLState, MG_State/GLState, MG_Test, MG_Test/Backend, MG_Test/BackendLoader, MG_Test/Buffer, MG_Test/EGLState, MG_Test/Framebuffer, MG_Test/Gbm, MG_Test/Pipe, MG_Test/Pipeline, MG_Test/Program, MG_Test/Query, MG_Test/SelfTest, MG_Test/ShaderTranspiler, MG_Test/State, MG_Test/Swapchain, MG_Test/Texture, MG_Test/Util, MG_Test/VertexArray, MG_Test/Wire, MG_Util/Async, MG_Util/BackendLoaders, MG_Util/Converters, MG_Util/Damage, MG_Util/Debug, MG_Util/Math, MG_Util/Metrics, MG_Util/SelfTest, MG_Util/ShaderTranspiler, MG_Util/Texture, MG_Util/X11, MobileGL, scripts/data, tools/cts, tools/device_bench, tools/spikes, tools/trace_replay, tools/wgl-smoke): clang-format every C and C++ source
- Layout only: token streams match up to split literals, continuations and using order
- The size totals fall, so the structure baseline is lowered
2026-10-16 06:30:38 -04:00
swung0x48 76b10cc35b [CI] (scripts): read the dirty-bit name table in any line layout 2026-10-16 06:26:09 -04:00
swung0x48 58c727f74e [Tools] (scripts): format exactly the files the format check covers
- Skip MG_Impl/GLImpl/Exporting/Definitions.cpp, whose table clang-format cannot wrap
2026-10-16 06:26:07 -04:00
swung0x48 351bb35a96 [CI] (scripts/data): lower the baselines after the BackendCaps.h move 2026-10-16 05:47:58 -04:00
swung0x48 f7e065aedb [Docs] (MG_Backend/Record, MG_Impl/GLImpl, MG_Impl/Pipe, MG_Remote, MG_Remote/Client): cite BackendObject.h symbols, not lines 2026-10-16 05:47:43 -04:00
swung0x48 778014c013 [Refactor] (MG_Backend, MG_Backend/DirectVulkan, MG_Pipe, MG_Remote/Client, MG_State/GLState, MG_Test/Pipe, MG_Util/SelfTest, MG_Util/ShaderTranspiler, MobileGL, scripts): move the backend capability types into BackendCaps.h
- Config.h includes only BackendCaps.h, no longer both backends; pActiveBackendObject is declared with the interface
2026-10-16 05:47:42 -04:00
swung0x48 cf1b7f6ed8 [CI] (scripts/ci, scripts/data): lower the baselines after the DirectGLES split 2026-10-16 05:17:25 -04:00
swung0x48 91294a19f8 [Docs] (MG_Backend/DirectGLES, MG_Backend/DirectVulkan, MG_Backend/Record, MG_Impl/Pipe, MG_IntegrationTest/Scenarios, MG_Pipe, MG_Remote/Client, MG_Remote/Server, MG_State/GLState, MG_Test, MG_Test/Backend, MG_Test/Buffer, MG_Test/Pipe, MG_Test/Wire, scripts, scripts/ci): cite symbols, not lines, in the split backend files 2026-10-16 05:16:51 -04:00
swung0x48 d1f220c1a2 [Refactor] (CMakeLists.txt, MG_Backend/DirectGLES): split Managers.cpp by object kind
- helpers and state several files share lose `static`; ManagersInternal.h declares them
2026-10-16 05:10:03 -04:00
swung0x48 b0f73e3795 [Refactor] (CMakeLists.txt, MG_Backend/DirectGLES): split DirectGLES.cpp by area
- helpers and state several files share lose `static`; DirectGLESInternal.h declares them
2026-10-16 05:09:31 -04:00
swung0x48 1ec54d876f [CI] (scripts/ci): read every DirectGLES source in the memo purity gate 2026-10-16 04:47:34 -04:00
swung0x48 5973bf353b [CI] (scripts/data): lower the hygiene and structure baselines after the renderer split 2026-10-16 04:30:51 -04:00
swung0x48 16994dc994 [Docs] (MG_Backend/Record, MG_IntegrationTest/Scenarios): cite symbols the split left past their files' ends 2026-10-16 04:30:26 -04:00
swung0x48 7f7c596213 [Docs] (docs/Disaggregated, docs/MobileVA, docs/Performance, MG_Impl/Pipe, MG_IntegrationTest/Scenarios, MG_State/GLState, MG_Test/Buffer, MG_Test/Wire, scripts/ci): name the renderer's new files 2026-10-16 04:28:53 -04:00
swung0x48 98c8e8833c [Refactor] (CMakeLists.txt, MG_Backend/DirectVulkan): split VulkanRenderer.cpp and its fragments by area
- code moves unchanged; helpers several files use are in VulkanRendererInternal.h
2026-10-16 04:28:47 -04:00
swung0x48 b0e229f2a3 [Refactor] (MG_Backend/DirectVulkan): make the renderer's two file-scope singletons inline 2026-10-16 04:20:26 -04:00
swung0x48 01f80b0a0a [Refactor] (CMakeLists.txt, MG_Backend, MG_Backend/DirectGLES, MG_Backend/DirectVulkan, MG_Backend/MGPipe, MG_Test/Framebuffer, MG_Test/Texture): move the client readback packing out of DirectGLES 2026-10-16 04:20:19 -04:00
swung0x48 deae0fc4c1 [Refactor] (MG_Backend/DirectGLES): drop the readback stores' pack-buffer branch, which never runs 2026-10-16 04:05:43 -04:00
swung0x48 f90537c3cb [CI] (.github/workflows, CONTRIBUTING.md, scripts, scripts/data): add the layering and size ratchets 2026-10-16 04:01:29 -04:00
swung0x48 26a30ecd02 [CI] (CONTRIBUTING.md, scripts): skip moved lines in the changed-lines format check 2026-10-16 04:01:27 -04:00
775 changed files with 72808 additions and 72111 deletions
+5
View File
@@ -0,0 +1,5 @@
# Layout-only commits for `git blame` to look past:
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# clang-format every C and C++ source
f4994ee36e3aa2d53c22370e471c976a3fae719f
+10 -4
View File
@@ -1,8 +1,9 @@
name: Text hygiene
# CONTRIBUTING.md's checks: the pushed commits' messages, the source-text rules and ratchets, and
# clang-format on each commit's changed lines. This is its own workflow without a concurrency
# group, so a later push never cancels it and every push's commits are checked.
# CONTRIBUTING.md's checks: the pushed commits' messages, the source-text rules and ratchets,
# clang-format on each commit's changed lines, and the layering and size ratchets. This is its own
# workflow without a concurrency group, so a later push never cancels it and every push's commits
# are checked.
on:
push:
branches: ['**']
@@ -29,6 +30,11 @@ jobs:
"${RUNNER_TEMP}/clang-format/bin/pip" install --quiet 'clang-format==23.1.3'
echo "CLANG_FORMAT=${RUNNER_TEMP}/clang-format/bin/clang-format" >> "${GITHUB_ENV}"
- name: Lint self-test
run: python3 scripts/check_text_hygiene.py self-test
run: |
python3 scripts/check_text_hygiene.py self-test
python3 scripts/check_structure.py self-test
- name: Check commits, source text and formatting
run: python3 scripts/check_text_hygiene.py ci
- name: Check layering and file and function size
if: ${{ !cancelled() }}
run: python3 scripts/check_structure.py ci
+32
View File
@@ -496,12 +496,31 @@ set(SOURCE_FILES
MobileGL/MG_Backend/Init.cpp
MobileGL/MG_Backend/BackendObject.cpp
MobileGL/MG_Backend/ClientReadback.cpp
MobileGL/MG_Backend/GpuHangWatch.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLES.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESBind.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESBlit.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESBuffers.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESContext.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESCopy.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESDraw.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESReadback.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESSharedImage.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESState.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESSync.cpp
MobileGL/MG_Backend/DirectGLES/DirectGLESTextures.cpp
MobileGL/MG_Backend/DirectGLES/BackendObject_DirectGLES.cpp
MobileGL/MG_Backend/DirectGLES/Utils.cpp
MobileGL/MG_Backend/DirectGLES/Managers.cpp
MobileGL/MG_Backend/DirectGLES/ManagersBuffers.cpp
MobileGL/MG_Backend/DirectGLES/ManagersFramebuffers.cpp
MobileGL/MG_Backend/DirectGLES/ManagersPrograms.cpp
MobileGL/MG_Backend/DirectGLES/ManagersSamplers.cpp
MobileGL/MG_Backend/DirectGLES/ManagersTextureSync.cpp
MobileGL/MG_Backend/DirectGLES/ManagersTextures.cpp
MobileGL/MG_Backend/DirectGLES/ManagersVertexArrays.cpp
MobileGL/MG_Backend/DirectGLES/MultiDraw.cpp
MobileGL/MG_Backend/DirectVulkan/DirectVulkan.cpp
@@ -509,6 +528,19 @@ set(SOURCE_FILES
MobileGL/MG_Backend/DirectVulkan/MagmaSession.cpp
MobileGL/MG_Backend/DirectVulkan/VmaImpl.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRenderer.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererCopy.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererDevice.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererDraw.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererPipeline.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererPresent.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererSubmit.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireColorBlit.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireDepthMipmap.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireDraw.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireFramebuffer.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireMultisampleResolve.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireSharedImage.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/WireYuvImage.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/SwapchainObject.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/FrameContext.cpp
MobileGL/MG_Backend/DirectVulkan/Renderer/PipelineFactory.cpp
+20 -4
View File
@@ -52,14 +52,22 @@ Example:
## Code
- Format the lines you change with `.clang-format`, using clang-format 23. Install it with `pip install 'clang-format==23.1.3'`; `git clang-format <base>` from the same package formats only the lines changed since `<base>`. `scripts/format_code.sh` reformats whole files, so do not use it on files you only touched.
- `.inc` files are fragments included in the middle of a `.cpp` file, inside a namespace, or are generated tables. clang-format cannot format them on their own, so the check skips them. Match the indentation of the file that includes them.
- C and C++ sources are formatted with `.clang-format`, using clang-format 23. Install it with `pip install 'clang-format==23.1.3'`. `git clang-format <base>` from the same package formats the lines changed since `<base>`, and `scripts/format_code.sh` formats every file the check covers. The tree is already formatted, so neither touches lines you did not change.
- The commit that reformatted the whole tree is listed in `.git-blame-ignore-revs`. Run `git config blame.ignoreRevsFile .git-blame-ignore-revs` once so that `git blame` looks past it.
- `.inc` files are fragments included in the middle of a `.cpp` file, inside a namespace, or are generated tables. clang-format cannot format them on their own, so the check skips them. Match the indentation of the file that includes them. The check also skips generated files and the GL export table, `MG_Impl/GLImpl/Exporting/Definitions.cpp`.
- Text files are UTF-8.
- These source files under `MobileGL/`, `scripts/` and `tools/` start with the license header:
- C, C++, `.inc`, GLSL, Java and JavaScript modules, with `//` comments;
- Python, shell, CMake modules and `CMakeLists.txt`, with `#` comments.
- Generated files and files under another license are exempt. `python3 scripts/update-source-file-header.py` adds the header.
- Keep files to about 3,000 lines and functions to about 300 lines.
- Modules are layered. A file includes files of its own layer and of the layers below it, never above:
1. the root headers (`Includes.h`, `Config.h`), `MG_Util` and `MG_State`;
2. `MG_Pipe`;
3. `MG_Backend`, `MG_Remote` and `MG_Impl`;
4. the root `.cpp` files and `MG_Gbm`;
5. `MG_Test`, `MG_IntegrationTest` and `MG_Benchmark`.
- Each backend is isolated. Files under `MG_Backend/DirectGLES/` or `MG_Backend/DirectVulkan/` are included only by the same backend, by the registry files directly in `MG_Backend/`, by the root `.cpp` files and by tests.
- Platform-specific APIs stay behind an abstraction, never raw in logic code.
- A fix that restores intended behaviour ships unconditionally. Add a switch only for a real trade-off.
- A cache keyed on object identity uses a monotonic lifetime id, never a GL name or a pointer, because both are recycled.
@@ -85,7 +93,7 @@ These fail on any violation:
- a text file that is not UTF-8;
- an external project name in a comment;
- a `file:line` citation in code that points past the end of the file;
- changed lines that clang-format would change.
- changed lines that clang-format would change (a line moved unchanged, the same commit removing its exact text elsewhere, is not a changed line).
These are ratchets: each total may only go down.
@@ -97,4 +105,12 @@ These are ratchets: each total may only go down.
`python3 scripts/check_text_hygiene.py tree --update-baseline` lowers `scripts/data/text_hygiene_baseline.json` after a cleanup. It never raises it, and CI rejects a raised baseline.
Not checked by CI: the size limits, the platform, switch and lifetime-id rules, planning ids outside comments, English outside comments, and the regeneration step for generated code.
`python3 scripts/check_structure.py ci` checks three more ratchets over the C and C++ sources under `MobileGL/`, against `scripts/data/structure_baseline.json`:
- includes that break the layering or the backend isolation above;
- lines beyond 3,000 in a file;
- lines beyond 300 in a function body.
`python3 scripts/check_structure.py report` lists what each total counts, and `tree --update-baseline` lowers the baseline the same way.
Not checked by CI: the platform, switch and lifetime-id rules, planning ids outside comments, English outside comments, and the regeneration step for generated code.
+422
View File
@@ -0,0 +1,422 @@
// MobileGL - MobileGL/BackendCaps.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_State/GLState/TextureState/TextureEnum.h>
// The vocabulary every layer uses to talk about a backend's capabilities: which backend it is,
// what each texture format supports, and the probed limits and features. It holds values only;
// the backend itself is MG_Backend/BackendObject.h.
namespace MobileGL {
enum class BackendType {
DirectGLES,
DirectVulkan,
BackendTypeCount,
Unknown = -1
};
namespace MG_Backend {
enum class FormatCapability : Uint64 {
Creatable = 1ull << 0,
Sampled = 1ull << 1,
LinearFilter = 1ull << 2,
GenerateMipmap = 1ull << 3,
TextureGather = 1ull << 4,
TextureShadow = 1ull << 5,
FramebufferRenderable = 1ull << 6,
FramebufferLayered = 1ull << 7,
MultisampleTexture = 1ull << 8,
MultisampleRenderbuffer = 1ull << 9,
ColorAttachment = 1ull << 10,
DepthAttachment = 1ull << 11,
StencilAttachment = 1ull << 12,
TextureBuffer = 1ull << 13
};
using FormatCapabilityFlags = Flags<FormatCapability>;
inline constexpr Array<FormatCapability, 14> kReportedFormatCapabilities = {
FormatCapability::Creatable,
FormatCapability::Sampled,
FormatCapability::LinearFilter,
FormatCapability::GenerateMipmap,
FormatCapability::TextureGather,
FormatCapability::TextureShadow,
FormatCapability::FramebufferRenderable,
FormatCapability::FramebufferLayered,
FormatCapability::MultisampleTexture,
FormatCapability::MultisampleRenderbuffer,
FormatCapability::ColorAttachment,
FormatCapability::DepthAttachment,
FormatCapability::StencilAttachment,
FormatCapability::TextureBuffer,
};
inline constexpr SizeT kFormatCapabilityTextureTargetCount =
static_cast<SizeT>(TextureTarget::TextureTargetCount);
inline constexpr SizeT kFormatCapabilityRenderbufferTargetIndex = kFormatCapabilityTextureTargetCount;
inline constexpr SizeT kFormatCapabilityTargetCount = kFormatCapabilityTextureTargetCount + 1;
inline constexpr SizeT kFormatCapabilityFormatCount =
static_cast<SizeT>(TextureInternalFormat::TextureInternalFormatCount);
using FormatCapabilityTable =
Array<Array<FormatCapabilityFlags, kFormatCapabilityFormatCount>, kFormatCapabilityTargetCount>;
using FormatSampleCountTable =
Array<Array<Vector<Int>, kFormatCapabilityFormatCount>, kFormatCapabilityTargetCount>;
struct FormatCapabilityCache {
FormatCapabilityTable FullCaps{};
FormatCapabilityTable CaveatCaps{};
FormatSampleCountTable SampleCounts{};
void Clear();
};
Bool HasFormatCapability(FormatCapabilityFlags caps, FormatCapability capability);
SizeT GetFormatCapabilityTargetIndex(TextureTarget target);
SizeT GetRenderbufferFormatCapabilityTargetIndex();
const char* GetFormatCapabilityName(FormatCapability capability);
String GetFormatCapabilityTargetName(SizeT targetIndex);
void PrintFormatCapabilities(const FormatCapabilityCache& cache);
// Device features that decide whether an extension string may be advertised, beyond the
// per-backend positional flags of BuildAdvertisedExtensions. Each is the PROBED driver
// capability (an ES version / extension on DirectGLES, a VkPhysicalDeviceFeatures bit on
// DirectVulkan), never a vendor guess. Defaults describe the no-device case.
struct AdvertisedFeatureSupport {
// ES 3.2 or GL_EXT_tessellation_shader; VkPhysicalDeviceFeatures::tessellationShader.
Bool TessellationShader = false;
// At least one cull distance: GL_EXT_clip_cull_distance with a non-zero
// GL_MAX_CULL_DISTANCES; VkPhysicalDeviceFeatures::shaderCullDistance with a non-zero
// maxCullDistances.
Bool CullDistance = false;
// GL_EXT_clip_control on DirectGLES. DirectVulkan applies clip control in its own
// position fixup and does not read this.
Bool ClipControl = false;
};
// Coarse GPU vendor identity for gating device-specific quirks. Detected from the
// Vulkan physical-device vendorID or the GLES GL_VENDOR/GL_RENDERER strings; stays
// Unknown when detection is inconclusive, in which case auto-gated quirks stay off.
enum class GpuVendorKind : Uint8 {
Unknown = 0,
Qualcomm,
Arm,
Nvidia,
Amd,
Intel,
ImgTec,
// Software rasterizers (llvmpipe/lavapipe, SwiftShader).
Software,
};
struct DynamicBackendParameters {
Uint64 UniformBufferOffsetAlignment = 256;
// GL_SHADER_STORAGE_BUFFER_OFFSET_ALIGNMENT, which is a SEPARATE limit from the
// uniform one and is routinely larger: Adreno 830 reports 32 for uniform buffers and
// 64 for storage buffers. Answering the storage query with the uniform value let an
// application bind a storage range at an offset the driver cannot address, which it
// accepted without error and then wrote somewhere else entirely.
Uint64 ShaderStorageBufferOffsetAlignment = 256;
// GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT. 1.0 means the backend cannot filter anisotropically,
// which is also why the extension is not advertised in that case.
Float MaxTextureMaxAnisotropy = 1.0f;
Float AliasedLineWidthRangeMin = 1.0f;
Float AliasedLineWidthRangeMax = 1.0f;
Float SmoothLineWidthRangeMin = 1.0f;
Float SmoothLineWidthRangeMax = 1.0f;
Float SmoothLineWidthGranularity = 1.0f;
Float PointSizeRangeMin = 1.0f;
Float PointSizeRangeMax = 1.0f;
Float PointSizeGranularity = 1.0f;
Int Max3DTextureSize = 16384;
Int MaxArrayTextureLayers = 2048;
Int MaxCubeMapTextureSize = 16384;
Int MaxFramebufferWidth = 16384;
Int MaxFramebufferHeight = 16384;
Int MaxFramebufferLayers = 2048;
Int MaxRenderbufferSize = 16384;
Int MaxTextureSize = 16384;
Int MaxColorTextureSamples = 1;
Int MaxDepthTextureSamples = 1;
Int MaxFramebufferSamples = 1;
Int MaxIntegerSamples = 1;
Int MaxSamples = 1;
Int MaxSampleMaskWords = 1;
// Tessellation limits; defaults are the GL 4.0 core minimums.
Int MaxPatchVertices = 32;
Int MaxTessGenLevel = 64;
// GL_MIN/MAX_PROGRAM_TEXTURE_GATHER_OFFSET. Defaults are the GL 4.0 core
// minimums, which every ES 3.1 driver also guarantees.
Int MinProgramTextureGatherOffset = -8;
Int MaxProgramTextureGatherOffset = 7;
Int MaxTextureImageUnits = 32;
Int MaxVertexTextureImageUnits = 32;
Int MaxComputeTextureImageUnits = 32;
Int MaxCombinedTextureImageUnits = 192;
Int MaxVertexAttribs = 16;
Int MaxComputeShaderStorageBlocks = 8;
Int MaxCombinedShaderStorageBlocks = 32;
// Per-stage GL_MAX_*_SHADER_STORAGE_BLOCKS. Zero is a legal answer for the four
// non-compute, non-fragment stages and these defaults are the spec minimums, not
// placeholders: GL 4.6 table 23.64 and ES 3.2 table 21.44 both set the minimum for
// vertex, tessellation control, tessellation evaluation and geometry at 0, and only
// fragment (8 in GL, 4 in ES) and compute are guaranteed to have any. Every real ARM
// GLES driver takes that allowance - a Mali-G925 reports 0 for all four - so a
// backend that cannot honour a graphics-stage storage block MUST report 0 here
// rather than a hopeful number. Advertising a non-zero count the driver will refuse
// does not make the block work; it only moves the failure from an honest
// "unsupported" at query time to a backend link error the frontend never surfaces,
// after which every draw with that program silently renders nothing.
Int MaxVertexShaderStorageBlocks = 0;
Int MaxTessControlShaderStorageBlocks = 0;
Int MaxTessEvaluationShaderStorageBlocks = 0;
Int MaxGeometryShaderStorageBlocks = 0;
Int MaxFragmentShaderStorageBlocks = 8;
Int MaxComputeUniformBlocks = 12;
Int MaxComputeWorkGroupInvocations = 128;
// GL_MAX_COMPUTE_WORK_GROUP_COUNT / GL_MAX_COMPUTE_WORK_GROUP_SIZE, one value per
// axis. These six, with the invocations limit above, are the only indexed limits a
// backend genuinely OWNS - the device answers them (glGetIntegeri_v on DirectGLES,
// VkPhysicalDeviceLimits::maxComputeWorkGroupCount/Size on DirectVulkan) - and so
// the only ones that survive the retirement of the GetIntegeri_v table entry: they
// cross the MGPipe boundary inside MGPCaps, by inclusion of this struct (plan B
// section 4.4.1). Every other indexed pname names frontend state. RAW driver
// answers, like the invocations limit: GL_Getter and the compile environment floor
// them at the shared MIN_COMPUTE_WORK_GROUP_* minimums themselves. The defaults are
// the GL 4.3 core minimums (table 23.60) and describe the no-backend case, as
// MaxClipDistances' does.
Int MaxComputeWorkGroupCount[3] = {65535, 65535, 65535};
Int MaxComputeWorkGroupSize[3] = {1024, 1024, 64};
Int MaxShaderStorageBufferBindings = 8;
Int MaxTextureBufferSize = 65536;
// GL_TEXTURE_BUFFER_OFFSET_ALIGNMENT; 1 means the offset is unconstrained.
Int TextureBufferOffsetAlignment = 1;
Int MaxUniformBufferBindings = 24;
Int MaxUniformBlockSize = 16384;
Int MaxImageUnits = 8;
Int MaxCombinedImageUniforms = 8;
Int MaxVertexImageUniforms = 0;
Int MaxTessControlImageUniforms = 0;
Int MaxTessEvaluationImageUniforms = 0;
Int MaxGeometryImageUniforms = 0;
Int MaxFragmentImageUniforms = 8;
Int MaxComputeImageUniforms = 8;
Int MaxDrawBuffers = 8;
Int MaxColorAttachments = 8;
// GL_MAX_CLIP_DISTANCES. Zero is a legal answer here, not a placeholder, and a
// backend that cannot host a clip distance MUST report it: advertising eight the
// backend will refuse does not make gl_ClipDistance work, it only moves the failure
// from an honest "unsupported" at query time to a backend shader-compile error the
// frontend never surfaces, after which every draw with that program silently renders
// nothing. DirectGLES fills it from GL_EXT_clip_cull_distance, DirectVulkan from the
// shaderClipDistance device feature. The DEFAULT stays at the GL 4.3 core minimum
// because it describes the no-backend case (standalone shader compiles, unit tests),
// where there is no device to be honest about and BuildTBuiltInResource still has to
// hand glslang a workable gl_MaxClipDistances.
Int MaxClipDistances = 8;
// GL_MAX_CULL_DISTANCES and GL_MAX_COMBINED_CLIP_AND_CULL_DISTANCES, under exactly
// the contract stated for MaxClipDistances above: ZERO IS A LEGAL ANSWER and a
// backend that cannot host a cull distance MUST report it. The failure this prevents
// is worse than the clip one, because cull distance discards the whole primitive:
// glslang bounds gl_CullDistance[i] against maxCullDistances and expands
// gl_MaxCullDistances from it, SPIRV-Cross then emits
// `#extension GL_EXT_clip_cull_distance : require` into the ESSL, and a host driver
// without that extension rejects the program in an info log nobody surfaces. These
// used to be bare 8s inside BuildTBuiltInResource with no backend consulted at all.
// The DEFAULTS are the GL 4.5 core minimums for the same reason MaxClipDistances'
// is: they describe the no-backend case (standalone compiles, unit tests).
Int MaxCullDistances = 8;
Int MaxCombinedClipAndCullDistances = 8;
Int MaxViewports = 16;
// GL_LAYER_PROVOKING_VERTEX / GL_VIEWPORT_INDEX_PROVOKING_VERTEX: which vertex of a
// primitive supplies gl_Layer and gl_ViewportIndex. GL 4.6 table 23.65 makes
// GL_UNDEFINED_VERTEX a legal answer for both, and it is the honest default - naming
// a convention is a statement about behaviour, so a backend that does not pin one
// must not claim it does. DirectGLES fills the layer one from the ES 3.2 query and
// the viewport one from GL_OES_viewport_array, and leaves UNDEFINED where the
// capability is absent: without the viewport array extension only viewport 0 is ever
// rasterized, so no convention selects anything. DirectVulkan keeps UNDEFINED for
// both - which vertex provokes is decided per pipeline by
// VulkanRenderer::SelectProvokingVertexMode out of VK_EXT_provoking_vertex,
// provokingVertexModePerPipeline and the topology, so no single convention is true
// of the backend.
Uint32 LayerProvokingVertex = GL_UNDEFINED_VERTEX;
Uint32 ViewportIndexProvokingVertex = GL_UNDEFINED_VERTEX;
Int MaxViewportWidth = 16384;
Int MaxViewportHeight = 16384;
Float ViewportBoundsRangeMin = 0.0f;
Float ViewportBoundsRangeMax = 0.0f;
Int ViewportSubpixelBits = 0;
// GL 4.x fragment-interpolation offset limits. These defaults are the
// core minimums and are replaced by live GLES/Vulkan device limits.
Float MinFragmentInterpolationOffset = -0.5f;
// For four fractional bits the greatest required legal offset is
// 0.5 - 2^-4 = 0.4375 (GL 4.6 table 23.70).
Float MaxFragmentInterpolationOffset = 0.4375f;
Int FragmentInterpolationOffsetBits = 4;
Bool SupportsWideLines = false;
// Whether a framebuffer whose depth and stencil attachments are distinct
// images can be rendered to. GL only requires support when both refer to the
// same image and lets an implementation answer GL_FRAMEBUFFER_UNSUPPORTED
// otherwise, which is what DirectVulkan (one combined attachment) and the
// real ES drivers behind DirectGLES both do. Defaults to true so a backend
// that never sets it keeps the permissive behaviour.
Bool SupportsDistinctDepthStencilAttachments = true;
// Whether attaching a single layer of a 3D or array texture to a framebuffer actually
// renders to that layer. DirectGLES hands the layer straight to
// glFramebufferTextureLayer, so it does; DirectVulkan maps a GL layer onto a Vulkan
// array layer with no notion of a 3D depth slice, so it does not yet. Defaults to false
// so a backend that never sets it gets the conservative answer.
// Which layered texture targets this backend can attach ONE layer of to a framebuffer
// and then really clear, render and read back that layer. Bit (1u << TextureTarget) is
// set for each supported target. Deliberately per target rather than one flag: the three
// ways a GL layer maps onto Vulkan are independent capabilities. A 2D or 2D multisample
// array layer IS a VkImage array layer and needs nothing extra; a 3D texture's layer is
// a z slice, which needs a 2D-array-compatible image and a per-slice clear that
// vkCmdClearColorImage cannot express; a cube map array needs an image shape and the
// imageCubeArray feature before it can be attached at any layer at all. Defaults to 0 so
// a backend that never sets it gets the conservative answer.
Uint32 PerLayerFramebufferAttachmentTargets = 0;
static constexpr Uint32 PerLayerFramebufferAttachmentBit(TextureTarget target) {
return (static_cast<Int>(target) >= 0 &&
static_cast<Int>(target) < static_cast<Int>(TextureTarget::TextureTargetCount))
? (1u << static_cast<Uint32>(target))
: 0u;
}
Bool SupportsPerLayerFramebufferAttachment(TextureTarget target) const {
const Uint32 bit = PerLayerFramebufferAttachmentBit(target);
return bit != 0 && (PerLayerFramebufferAttachmentTargets & bit) != 0;
}
// Whether this backend can CONSUME a shader module that still declares 64-bit floats,
// i.e. whether `double` survives the transpile instead of being narrowed to `float`
// (ShaderTranspiler::DemoteFloat64Pass). Detected, never assumed:
// * DirectVulkan sets it from VkPhysicalDeviceFeatures::shaderFloat64, the feature
// VUID-VkShaderModuleCreateInfo-pCode-08740 requires before a module declaring
// OpCapability Float64 may be created at all. lavapipe has it; Adreno and Mali
// both report VK_FALSE, so no real mobile device does.
// * DirectGLES can NEVER have it. GLSL ES has no 64-bit float type in any version
// or extension, so SPIRV-Cross cannot emit one ("FP64 not supported in ES
// profile") and the demotion there is mathematically mandatory, always.
// Defaults to false so a backend that never sets it - and the no-backend case, which
// is what standalone shader compiles and the unit tests run under - keeps the
// demotion, which is the behaviour that works everywhere.
Bool SupportsShaderFloat64 = false;
// Whether glVertexAttribLFormat / glVertexArrayAttribLFormat can be honoured, i.e.
// whether a 64-bit vertex attribute can actually reach a shader unconverted. Detected,
// never assumed: DirectVulkan needs VkPhysicalDeviceFeatures::shaderFloat64 (the
// attribute travels as its 32-bit word pair, so no VK_FORMAT_R64* is required, but the
// bitcast result is Float64); DirectGLES can never have it, ESSL having no fp64 type at
// all. Defaults to false so a backend that never sets it gets the conservative answer.
//
// INDEPENDENT of SupportsShaderFloat64, and it has to be: this flag decides a VkFormat
// from the VAO ATTRIBUTE alone, which does not know what type the shader declared, and
// glVertexAttribFormat(GL_DOUBLE) feeding a plain `in vec4` is both legal and common
// (KHR-GL43.vertex_attrib_binding.basic-input-case4/5, advanced-bindingUpdate). A
// backend with native fp64 that still cannot FETCH 64 bits keeps this false and relies
// on the per-MODULE rule in ShaderCompiler::SanitizeAndOptimizeBinary instead: a vertex
// module that declares a 64-bit float INPUT is demoted whole, so the two shader-side
// halves (PackDoubleVertexInputsPass and VertexInputStateFactory::ToVkVertexFormat)
// still see one consistent world.
Bool SupportsFloat64VertexAttributes = false;
// Whether a TESSELLATION stage of this backend may access gl_PointSize - i.e.
// whether a module declaring OpCapability TessellationPointSize can reach the
// driver at all. DirectVulkan sets both this and the geometry twin from the one
// shaderTessellationAndGeometryPointSize feature; DirectGLES sets them
// independently from the EXT/OES_tessellation_point_size /
// geometry_point_size extension pairs (PointSizeTier), which really do come
// separately. When absent, ProgramSpirvTask demotes the built-in to an ordinary
// varying program-wide (ShaderCompiler::
// DemoteTessellationGeometryPointSizeForProgram); MOBILEGL_POINT_SIZE_DEMOTION
// overrides the detection in either direction at backend init.
//
// Defaults TRUE, deliberately against the house "assume absent" rule: false
// ARMS a rewrite, so the conservative no-backend answer (standalone compiles,
// unit tests) is the one that leaves modules untouched. A backend that never
// sets it gets standard modules and, at worst, the old honest declines.
Bool SupportsTessellationPointSize = true;
// The geometry-stage twin (OpCapability GeometryPointSize).
Bool SupportsGeometryPointSize = true;
Uint64 MaxShaderStorageBlockSize = 128 * 1024 * 1024;
Uint32 SubgroupSize = 0;
Uint32 SubgroupSupportedStages = 0;
Uint32 SubgroupSupportedFeatures = 0;
// GL_SAMPLE_POSITION (GL 4.6 core 14.3.1.1) as the device actually places the samples
// of a 2-, 4-, 8- and 16-sample pixel: (x, y) pairs in [0, 1], the N-sample row starting
// at pair N - 2. Bit log2(N) of SamplePositionCountMask says row N was filled in; a row
// the backend could not determine stays unset rather than being guessed. Read and
// written through Get/SetSamplePositions only.
static constexpr Int kMaxSamplePositionCount = 16;
Float SamplePositions[60] = {}; // 2 * (2 + 4 + 8 + 16)
Uint32 SamplePositionCountMask = 0;
Bool SubgroupQuadOperationsInAllStages = false;
GpuVendorKind GpuVendor = GpuVendorKind::Unknown;
// GL_EXT_fragment_shading_rate and its two companions. Each is a statement about the
// device, so all of them default to "absent" and the extension strings are advertised
// only from these: DirectGLES fills them from the driver's own
// EXT_fragment_shading_rate(_primitive, _attachment), DirectVulkan from
// VK_KHR_fragment_shading_rate's pipeline / primitive / attachment features.
// SupportsFragmentShadingRate glShadingRateEXT and gl_ShadingRateEXT
// SupportsPrimitiveFragmentShadingRate gl_PrimitiveShadingRateEXT, combiner op 0
// SupportsAttachmentFragmentShadingRate the R8UI rate attachment, combiner op 1
Bool SupportsFragmentShadingRate = false;
Bool SupportsPrimitiveFragmentShadingRate = false;
Bool SupportsAttachmentFragmentShadingRate = false;
// The four GL_FRAGMENT_SHADING_RATE_*_SUPPORTED_EXT booleans, as the device answers them.
Bool FragmentShadingRateNonTrivialCombiners = false;
Bool FragmentShadingRateWithShaderDepthStencilWrites = false;
Bool FragmentShadingRateWithSampleMask = false;
Bool FragmentShadingRatePrimitiveWithMultiViewport = false;
Bool FragmentShadingRateAttachmentWithDefaultFramebuffer = false;
// What glGetFragmentShadingRatesEXT answers, one mask per sample count 1, 2, 4 .. 64
// (index = log2(samples)): bit (log2(w) << 2 | log2(h)) is set when that rate is
// honoured exactly at that sample count, i.e. gl_ShadingRateEXT reads it back
// unclamped. Zero masks mean no rate at all, which is what a sample count the device
// cannot shade coarsely answers.
Uint16 FragmentShadingRateMasks[7] = {};
// The GL_{MIN,MAX}_FRAGMENT_SHADING_RATE_ATTACHMENT_TEXEL_{WIDTH,HEIGHT}_EXT,
// _ASPECT_RATIO_EXT and _LAYERS_EXT limits. Meaningful only with the attachment.
Int MinFragmentShadingRateAttachmentTexelWidth = 0;
Int MinFragmentShadingRateAttachmentTexelHeight = 0;
Int MaxFragmentShadingRateAttachmentTexelWidth = 0;
Int MaxFragmentShadingRateAttachmentTexelHeight = 0;
Int MaxFragmentShadingRateAttachmentTexelAspectRatio = 0;
Int MaxFragmentShadingRateAttachmentLayers = 0;
static constexpr Bool IsSamplePositionCount(Int samples) {
return samples >= 2 && samples <= kMaxSamplePositionCount && (samples & (samples - 1)) == 0;
}
static constexpr Uint32 SamplePositionCountBit(Int samples) {
return 1u << static_cast<Uint32>(std::countr_zero(static_cast<Uint32>(samples)));
}
// `xy` holds `samples` (x, y) pairs.
void SetSamplePositions(Int samples, const Float* xy) {
if (!IsSamplePositionCount(samples)) return;
for (Int i = 0; i < 2 * samples; ++i)
SamplePositions[2 * (samples - 2) + i] = xy[i];
SamplePositionCountMask |= SamplePositionCountBit(samples);
}
Bool GetSamplePosition(Int samples, Int index, Float* outXY) const {
if (!IsSamplePositionCount(samples) || index < 0 || index >= samples ||
(SamplePositionCountMask & SamplePositionCountBit(samples)) == 0) {
return false;
}
outXY[0] = SamplePositions[2 * (samples - 2 + index)];
outXY[1] = SamplePositions[2 * (samples - 2 + index) + 1];
return true;
}
};
} // namespace MG_Backend
} // namespace MobileGL
+12 -8
View File
@@ -8,7 +8,7 @@
#pragma once
#include <Includes.h>
#include <MG_Backend/BackendObjects.h>
#include <BackendCaps.h>
namespace MobileGL::MG_Config {
inline const String ProjectName = "MobileGL";
@@ -520,11 +520,11 @@ namespace MobileGL::MG_Config {
// MG_Remote` must be non-empty in build-split (CONTRACT-P5.md table 3, and the CI job
// P5 adds beside build-linux-verify).
enum class TransportMode : Uint8 {
Monolith = 0, // today's in-library backend; no MG_Remote object is constructed
InProcess = 1, // P5: a real apply thread in this process, over the same G3 codec
Spawn = 2, // P6: fork/exec MobileGLServer, socketpair
UnixSocket = 3,// P6: connect to an existing AF_UNIX endpoint (Endpoint = <path>)
NamedPipe = 4, // P6: Windows named pipe (Endpoint = <name>)
Monolith = 0, // today's in-library backend; no MG_Remote object is constructed
InProcess = 1, // P5: a real apply thread in this process, over the same G3 codec
Spawn = 2, // P6: fork/exec MobileGLServer, socketpair
UnixSocket = 3, // P6: connect to an existing AF_UNIX endpoint (Endpoint = <path>)
NamedPipe = 4, // P6: Windows named pipe (Endpoint = <name>)
};
// P13 W5: THE RECORD ARM'S OWN KNOBS. They kept their MOBILEGL_IPC_* names (logs, CI lanes and
@@ -799,8 +799,12 @@ namespace MobileGL::MG_Config {
// wire and on monolith alike ("monolith is the push variant"). W4 switched monolith family by
// family behind MOBILEGL_PIPE_DATA_ARM; W5 built the record arm into every library; W6 deleted
// the frontend arm and the knob. The predicate stays as a name for the sites that read it.
inline constexpr Bool DataArmIsRecord() { return true; }
inline constexpr Bool DataArmIsRecord() {
return true;
}
// ...and the record arm runs in this process over stores that ALIAS the frontend's shadows
// (monolith) instead of copying them (every wire).
inline Bool RecordArmAliasesFrontend() { return Transport == TransportMode::Monolith; }
inline Bool RecordArmAliasesFrontend() {
return Transport == TransportMode::Monolith;
}
} // namespace MobileGL::MG_Config
+21 -25
View File
@@ -115,7 +115,8 @@ namespace MobileGL::MG_ConfigLoader {
continue;
}
String value = Trimmed(line.substr(equals + 1));
if (value.size() >= 2 && (value.front() == '"' || value.front() == '\'') && value.back() == value.front()) {
if (value.size() >= 2 && (value.front() == '"' || value.front() == '\'') &&
value.back() == value.front()) {
value = value.substr(1, value.size() - 2);
}
out.emplace(key, value);
@@ -262,8 +263,7 @@ namespace MobileGL::MG_ConfigLoader {
if (lowered.empty() || lowered == "auto") return MG_Config::QuirkOverride::Auto;
if (lowered == "forceon" || lowered == "on") return MG_Config::QuirkOverride::ForceOn;
if (lowered == "forceoff" || lowered == "off" || lowered == "no") return MG_Config::QuirkOverride::ForceOff;
return IsTruthyValue(it->second) ? MG_Config::QuirkOverride::ForceOn
: MG_Config::QuirkOverride::ForceOff;
return IsTruthyValue(it->second) ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
// Multi-draw mode is a named-value preference: unset keeps Auto (best supported tier),
@@ -394,8 +394,7 @@ namespace MobileGL::MG_ConfigLoader {
features.MagmaMaxDrawsPerCommandBuffer =
QueryEnvUint32("MOBILEGL_MAGMA_MAX_DRAWS_PER_COMMAND_BUFFER", 16384, 0, 1u << 24);
features.MagmaDescriptorTrimFrames = QueryEnvUint32("MOBILEGL_MAGMA_DESCRIPTOR_TRIM_FRAMES", 120, 0, 1u << 20);
features.EsprytAvoidSamplerMipmapMinFilter =
QueryEnvFlag("MOBILEGL_ESPRYT_AVOID_SAMPLER_MIPMAP_MIN_FILTER");
features.EsprytAvoidSamplerMipmapMinFilter = QueryEnvFlag("MOBILEGL_ESPRYT_AVOID_SAMPLER_MIPMAP_MIN_FILTER");
features.EsprytAvoidExplicitLodBias = QueryEnvFlag("MOBILEGL_ESPRYT_AVOID_EXPLICIT_LOD_BIAS");
features.EsprytUnlocatedIoBlocks = QueryEnvQuirkOverride("MOBILEGL_ESPRYT_UNLOCATED_IO_BLOCKS");
features.EsprytDeferGeometryEmits = QueryEnvQuirkOverride("MOBILEGL_ESPRYT_DEFER_GEOMETRY_EMITS");
@@ -409,8 +408,7 @@ namespace MobileGL::MG_ConfigLoader {
features.EsprytDisableUploadRing = QueryEnvFlag("MOBILEGL_ESPRYT_DISABLE_UPLOAD_RING");
features.EsprytDisableInvalidateFlush = QueryEnvFlag("MOBILEGL_ESPRYT_DISABLE_INVALIDATE_FLUSH");
features.DisableLargeBufferAdoption = QueryEnvFlag("MOBILEGL_DISABLE_LARGE_BUFFER_ADOPTION");
features.EsprytForceDepthStencilReadbackEmulation =
QueryEnvFlag("MOBILEGL_ESPRYT_FORCE_DS_READBACK_EMULATION");
features.EsprytForceDepthStencilReadbackEmulation = QueryEnvFlag("MOBILEGL_ESPRYT_FORCE_DS_READBACK_EMULATION");
features.RelaxedSemantics = QueryEnvFlag("MOBILEGL_RELAXED_SEMANTICS");
features.MagmaDisableBlendedDepthWriteQuirk =
QueryEnvQuirkOverride("MOBILEGL_MAGMA_DISABLE_BLENDED_DEPTH_WRITE");
@@ -419,13 +417,10 @@ namespace MobileGL::MG_ConfigLoader {
features.EsprytMultiDrawMode = QueryEnvGLESMultiDrawMode("MOBILEGL_ESPRYT_MULTIDRAW_MODE");
features.AsyncShaderCompile = QueryEnvQuirkOverride("MOBILEGL_ASYNC_SHADER_COMPILE");
features.AsyncShaderCompileThreads = QueryEnvUint32("MOBILEGL_ASYNC_SHADER_COMPILE_THREADS", 0, 0, 64);
features.AsyncOptimisticShaderStatus =
QueryEnvQuirkOverride("MOBILEGL_ASYNC_OPTIMISTIC_SHADER_STATUS");
features.AsyncOptimisticShaderStatus = QueryEnvQuirkOverride("MOBILEGL_ASYNC_OPTIMISTIC_SHADER_STATUS");
features.ShaderTranslationCache = QueryEnvQuirkOverride("MOBILEGL_SHADER_CACHE");
features.EsprytViewportArrayEmulation =
QueryEnvQuirkOverride("MOBILEGL_ESPRYT_FORCE_VIEWPORT_ARRAY_EMULATION");
features.EsprytWidenPacked16Storage =
QueryEnvQuirkOverride("MOBILEGL_ESPRYT_WIDEN_PACKED16_STORAGE");
features.EsprytViewportArrayEmulation = QueryEnvQuirkOverride("MOBILEGL_ESPRYT_FORCE_VIEWPORT_ARRAY_EMULATION");
features.EsprytWidenPacked16Storage = QueryEnvQuirkOverride("MOBILEGL_ESPRYT_WIDEN_PACKED16_STORAGE");
features.MagmaPrimGenQueryReroute = QueryEnvQuirkOverride("MOBILEGL_MAGMA_PRIMGEN_QUERY_REROUTE");
features.MagmaE5RawReads = QueryEnvQuirkOverride("MOBILEGL_MAGMA_E5_RAW_READS");
features.MagmaSplitSliceCopies = QueryEnvQuirkOverride("MOBILEGL_MAGMA_SPLIT_SLICE_COPIES");
@@ -444,14 +439,12 @@ namespace MobileGL::MG_ConfigLoader {
// nothing to run; it is refused here, by name, before any backend is brought up, rather
// than discovered as an armless subsystem at its first draw. Bit 63 (CSO content
// addressing off) stays a switch.
if ((features.PipePush & MG_Pipe::kMGPipeSubsystemsMigratedAtP5e) !=
MG_Pipe::kMGPipeSubsystemsMigratedAtP5e) {
if ((features.PipePush & MG_Pipe::kMGPipeSubsystemsMigratedAtP5e) != MG_Pipe::kMGPipeSubsystemsMigratedAtP5e) {
MGLOG_F("MGPipe: Fatal{PipeSubsystemsFixedOn, \"MOBILEGL_PIPE_PUSH=0x%llx clears subsystem "
"bit(s) 0x%llx; bits 0-13 are fixed on since P13 - only bit 63 may be set or "
"cleared\"}",
static_cast<unsigned long long>(features.PipePush),
static_cast<unsigned long long>(MG_Pipe::kMGPipeSubsystemsMigratedAtP5e &
~features.PipePush));
static_cast<unsigned long long>(MG_Pipe::kMGPipeSubsystemsMigratedAtP5e & ~features.PipePush));
std::abort();
}
features.PipeVerify = QueryEnvFlag("MOBILEGL_PIPE_VERIFY");
@@ -629,9 +622,9 @@ namespace MobileGL::MG_ConfigLoader {
// P15: derived from MOBILEGL_FRAMES_IN_FLIGHT (Config.h states the mapping); the explicit
// knob stays as an override for experiments.
{
const Uint32 derived = std::max<Uint32>(1u, std::min<Uint32>(8u, MG_Config::Features.FramesInFlight > 2
? MG_Config::Features.FramesInFlight - 2
: 1u));
const Uint32 derived = std::max<Uint32>(
1u, std::min<Uint32>(8u, MG_Config::Features.FramesInFlight > 2 ? MG_Config::Features.FramesInFlight - 2
: 1u));
ipc.PresentCredit = QueryEnvUint32("MOBILEGL_IPC_PRESENT_CREDIT", derived, 1, 8);
}
// CONTRACT-P6 D5b's reply bound and P7's cold-start budget (Config.h has the semantics).
@@ -650,7 +643,8 @@ namespace MobileGL::MG_ConfigLoader {
// and the dual-block census is empty (no apply reads a client-filled field), so the
// shipped arm now matches what is tested - and with two blocks the client's fill is its
// own and needs no applier quiesce. 0 still turns it off (the negative control does).
ipc.RoleSplitState = QueryEnvQuirkOverride("MOBILEGL_IPC_ROLE_SPLIT_STATE") != MG_Config::QuirkOverride::ForceOff;
ipc.RoleSplitState =
QueryEnvQuirkOverride("MOBILEGL_IPC_ROLE_SPLIT_STATE") != MG_Config::QuirkOverride::ForceOff;
if (MG_Config::Features.PipeVerify && ipc.RoleSplitState) {
MGLOG_W("Config: MOBILEGL_IPC_ROLE_SPLIT_STATE=1 is incompatible with "
"MOBILEGL_PIPE_VERIFY (the comparator owns the single fill block); "
@@ -725,10 +719,12 @@ namespace MobileGL::MG_ConfigLoader {
"cold-start=%ums strict=%d audit=%d role-split-state=%d affinity='%s' surface=%s",
ipc.RingMb, ipc.StageMb, MG_Config::RecordArm.WireDeferredMb, ipc.SpinUs, ipc.EventWaitMs,
MG_Config::RecordArm.PersistentBlockKb, ipc.CreateWindow, ipc.PboReadbackSync, ipc.PollEscalate,
ipc.AdoptTier == 0 ? "0" : ipc.AdoptTier == 1 ? "1" : ipc.AdoptTier == 2 ? "2" : "unset(T0)",
ipc.AllowAdoptT0,
ipc.VerbBarrier, ipc.RunAhead, ipc.PresentCredit, ipc.ControlTimeoutMs, ipc.ColdStartMs,
static_cast<int>(ipc.StrictErrors), static_cast<int>(ipc.Audit),
ipc.AdoptTier == 0 ? "0"
: ipc.AdoptTier == 1 ? "1"
: ipc.AdoptTier == 2 ? "2"
: "unset(T0)",
ipc.AllowAdoptT0, ipc.VerbBarrier, ipc.RunAhead, ipc.PresentCredit, ipc.ControlTimeoutMs,
ipc.ColdStartMs, static_cast<int>(ipc.StrictErrors), static_cast<int>(ipc.Audit),
static_cast<int>(ipc.RoleSplitState), ipc.ServerAffinity.c_str(),
MG_Config::ServerOwnedWindowSurfaces() ? "server" : "offscreen");
if (ipc.VerbBarrier == 0) {
+9 -9
View File
@@ -107,14 +107,14 @@
// 2026-08-13 renumbering unchanged; the contract is and stays
// "INFO builds: asserts OFF; DEBUG builds: asserts ON".
#if MOBILEGL_LOG_ACTIVE_LEVEL <= MOBILEGL_LOG_LEVEL_DEBUG
#define MOBILEGL_ASSERT(condition, ...) \
do { \
if (!(condition)) { \
MGLOG_F("Assertion failed" __VA_OPT__(": ") __VA_ARGS__); \
MGLOG_F(" at %s:%d (%s)", __FILE__, __LINE__, __func__); \
TRAP; \
} \
} while (0)
#define MOBILEGL_ASSERT(condition, ...) \
do { \
if (!(condition)) { \
MGLOG_F("Assertion failed" __VA_OPT__(": ") __VA_ARGS__); \
MGLOG_F(" at %s:%d (%s)", __FILE__, __LINE__, __func__); \
TRAP; \
} \
} while (0)
#else
#define MOBILEGL_ASSERT(condition, ...)
#define MOBILEGL_ASSERT(condition, ...)
#endif
+1
View File
@@ -7,6 +7,7 @@
// End of Source File Header
#include "Config.h"
#include <MG_Backend/BackendObject.h>
namespace MobileGL {
namespace MG_Config {
+1 -1
View File
@@ -124,7 +124,7 @@ namespace MobileGL {
// TODO: add and use Destroy functions for other subsystems
}
}
} // namespace
void Initialize() {
if (g_isInitialized) {
+10 -11
View File
@@ -24,10 +24,8 @@ namespace MobileGL::MG_Backend {
return std::this_thread::get_id();
}
const char* GetFormatCapabilitySupportString(const FormatCapabilityCache& cache,
SizeT targetIndex,
SizeT formatIndex,
FormatCapability capability) {
const char* GetFormatCapabilitySupportString(const FormatCapabilityCache& cache, SizeT targetIndex,
SizeT formatIndex, FormatCapability capability) {
if (HasFormatCapability(cache.FullCaps[targetIndex][formatIndex], capability)) return "Full";
if (HasFormatCapability(cache.CaveatCaps[targetIndex][formatIndex], capability)) return "Caveat";
return "None";
@@ -58,9 +56,7 @@ namespace MobileGL::MG_Backend {
return line.str();
}
String BuildFormatCapabilityRow(const FormatCapabilityCache& cache,
SizeT targetIndex,
SizeT formatIndex,
String BuildFormatCapabilityRow(const FormatCapabilityCache& cache, SizeT targetIndex, SizeT formatIndex,
SizeT formatNameWidth) {
const auto format = static_cast<TextureInternalFormat>(formatIndex);
std::ostringstream line;
@@ -397,8 +393,12 @@ namespace MobileGL::MG_Backend {
}
} // namespace
const MG_Util::Damage::Region& CurrentPresentDamage() { return PresentDamageSlot(); }
void SetCurrentPresentDamage(const MG_Util::Damage::Region& damage) { PresentDamageSlot() = damage; }
const MG_Util::Damage::Region& CurrentPresentDamage() {
return PresentDamageSlot();
}
void SetCurrentPresentDamage(const MG_Util::Damage::Region& damage) {
PresentDamageSlot() = damage;
}
Bool BackendObject::SwapEGLBuffers(EGLDisplay dpy, EGLSurface draw) {
const std::lock_guard<std::recursive_mutex> lock(m_eglStateMutex);
@@ -458,8 +458,7 @@ namespace MobileGL::MG_Backend {
void BackendObject::DestroyPendingEGLSurfaceIfUnused(EGLSurface surface) {
auto surfaceIt = m_eglSurfaces.find(surface);
if (surfaceIt == m_eglSurfaces.end() || !surfaceIt->second.DestroyPending ||
IsEGLSurfaceCurrent(surface)) {
if (surfaceIt == m_eglSurfaces.end() || !surfaceIt->second.DestroyPending || IsEGLSurfaceCurrent(surface)) {
return;
}
+15 -416
View File
@@ -8,6 +8,7 @@
#pragma once
#include <Includes.h>
#include <BackendCaps.h>
#include "MG_State/GLState/TextureState/TextureEnum.h"
#include <MG_Util/Damage/Damage.h>
#include <MG_Pipe/MGPipeHandles.h>
@@ -17,14 +18,7 @@ namespace MobileGL {
class FramebufferObject;
class ITextureObject;
class RenderbufferObject;
}
enum class BackendType {
DirectGLES,
DirectVulkan,
BackendTypeCount,
Unknown = -1
};
} // namespace MG_State::GLState
namespace MG_Backend {
// One endpoint of a glCopyImageSubData. GL 4.6 core 18.3.2 accepts GL_RENDERBUFFER
@@ -49,73 +43,6 @@ namespace MobileGL {
}
};
enum class FormatCapability : Uint64 {
Creatable = 1ull << 0,
Sampled = 1ull << 1,
LinearFilter = 1ull << 2,
GenerateMipmap = 1ull << 3,
TextureGather = 1ull << 4,
TextureShadow = 1ull << 5,
FramebufferRenderable = 1ull << 6,
FramebufferLayered = 1ull << 7,
MultisampleTexture = 1ull << 8,
MultisampleRenderbuffer = 1ull << 9,
ColorAttachment = 1ull << 10,
DepthAttachment = 1ull << 11,
StencilAttachment = 1ull << 12,
TextureBuffer = 1ull << 13
};
using FormatCapabilityFlags = Flags<FormatCapability>;
inline constexpr Array<FormatCapability, 14> kReportedFormatCapabilities = {
FormatCapability::Creatable,
FormatCapability::Sampled,
FormatCapability::LinearFilter,
FormatCapability::GenerateMipmap,
FormatCapability::TextureGather,
FormatCapability::TextureShadow,
FormatCapability::FramebufferRenderable,
FormatCapability::FramebufferLayered,
FormatCapability::MultisampleTexture,
FormatCapability::MultisampleRenderbuffer,
FormatCapability::ColorAttachment,
FormatCapability::DepthAttachment,
FormatCapability::StencilAttachment,
FormatCapability::TextureBuffer,
};
inline constexpr SizeT kFormatCapabilityTextureTargetCount =
static_cast<SizeT>(TextureTarget::TextureTargetCount);
inline constexpr SizeT kFormatCapabilityRenderbufferTargetIndex = kFormatCapabilityTextureTargetCount;
inline constexpr SizeT kFormatCapabilityTargetCount = kFormatCapabilityTextureTargetCount + 1;
inline constexpr SizeT kFormatCapabilityFormatCount =
static_cast<SizeT>(TextureInternalFormat::TextureInternalFormatCount);
using FormatCapabilityTable =
Array<Array<FormatCapabilityFlags, kFormatCapabilityFormatCount>, kFormatCapabilityTargetCount>;
using FormatSampleCountTable =
Array<Array<Vector<Int>, kFormatCapabilityFormatCount>, kFormatCapabilityTargetCount>;
struct FormatCapabilityCache {
FormatCapabilityTable FullCaps{};
FormatCapabilityTable CaveatCaps{};
FormatSampleCountTable SampleCounts{};
void Clear();
};
Bool HasFormatCapability(FormatCapabilityFlags caps, FormatCapability capability);
SizeT GetFormatCapabilityTargetIndex(TextureTarget target);
SizeT GetRenderbufferFormatCapabilityTargetIndex();
const char* GetFormatCapabilityName(FormatCapability capability);
String GetFormatCapabilityTargetName(SizeT targetIndex);
void PrintFormatCapabilities(const FormatCapabilityCache& cache);
// Opaque backend fence-sync handle, created by GLFunctionsTable::FenceSync
// and released by GLFunctionsTable::DeleteSync.
using BackendSyncHandle = void*;
@@ -139,8 +66,8 @@ namespace MobileGL {
void (*MultiDrawElementsIndirect)(GLenum mode, GLenum type, const void* indirect, GLsizei drawcount,
GLsizei stride);
void (*MultiDrawArraysIndirect)(GLenum mode, const void* indirect, GLsizei drawcount, GLsizei stride);
void (*MultiDrawElementsIndirectCount)(GLenum mode, GLenum type, const void* indirect,
GLintptr drawcount, GLsizei maxdrawcount, GLsizei stride);
void (*MultiDrawElementsIndirectCount)(GLenum mode, GLenum type, const void* indirect, GLintptr drawcount,
GLsizei maxdrawcount, GLsizei stride);
void (*MultiDrawArraysIndirectCount)(GLenum mode, const void* indirect, GLintptr drawcount,
GLsizei maxdrawcount, GLsizei stride);
void (*DrawRangeElementsBaseVertex)(GLenum mode, GLuint start, GLuint end, GLsizei count, GLenum type,
@@ -178,18 +105,16 @@ namespace MobileGL {
GLint dstX1, GLint dstY1, GLbitfield mask, GLenum filter);
void (*BlitNamedFramebuffer)(const SharedPtr<MG_State::GLState::FramebufferObject>& readFramebuffer,
const SharedPtr<MG_State::GLState::FramebufferObject>& drawFramebuffer,
GLint srcX0, GLint srcY0, GLint srcX1, GLint srcY1,
GLint dstX0, GLint dstY0, GLint dstX1, GLint dstY1,
GLbitfield mask, GLenum filter);
GLint srcX0, GLint srcY0, GLint srcX1, GLint srcY1, GLint dstX0, GLint dstY0,
GLint dstX1, GLint dstY1, GLbitfield mask, GLenum filter);
void (*CopyTexImage2D)(GLenum target, GLint level, GLenum internalformat, GLint x, GLint y, GLsizei width,
GLsizei height, GLint border);
void (*CopyTexSubImage2D)(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y,
GLsizei width, GLsizei height);
void (*CopyImageSubData)(const CopyImageEndpoint& src,
GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY, GLint srcZ,
const CopyImageEndpoint& dst,
GLenum dstTarget, GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ,
GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void (*CopyImageSubData)(const CopyImageEndpoint& src, GLenum srcTarget, GLint srcLevel, GLint srcX,
GLint srcY, GLint srcZ, const CopyImageEndpoint& dst, GLenum dstTarget,
GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ, GLsizei srcWidth,
GLsizei srcHeight, GLsizei srcDepth);
void (*GenerateMipmap)(GLenum target);
void (*ReadPixels)(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type,
void* pixels);
@@ -309,336 +234,6 @@ namespace MobileGL {
void (*SetSwapInterval)(Int interval);
};
// Device features that decide whether an extension string may be advertised, beyond the
// per-backend positional flags of BuildAdvertisedExtensions. Each is the PROBED driver
// capability (an ES version / extension on DirectGLES, a VkPhysicalDeviceFeatures bit on
// DirectVulkan), never a vendor guess. Defaults describe the no-device case.
struct AdvertisedFeatureSupport {
// ES 3.2 or GL_EXT_tessellation_shader; VkPhysicalDeviceFeatures::tessellationShader.
Bool TessellationShader = false;
// At least one cull distance: GL_EXT_clip_cull_distance with a non-zero
// GL_MAX_CULL_DISTANCES; VkPhysicalDeviceFeatures::shaderCullDistance with a non-zero
// maxCullDistances.
Bool CullDistance = false;
// GL_EXT_clip_control on DirectGLES. DirectVulkan applies clip control in its own
// position fixup and does not read this.
Bool ClipControl = false;
};
// Coarse GPU vendor identity for gating device-specific quirks. Detected from the
// Vulkan physical-device vendorID or the GLES GL_VENDOR/GL_RENDERER strings; stays
// Unknown when detection is inconclusive, in which case auto-gated quirks stay off.
enum class GpuVendorKind : Uint8 {
Unknown = 0,
Qualcomm,
Arm,
Nvidia,
Amd,
Intel,
ImgTec,
// Software rasterizers (llvmpipe/lavapipe, SwiftShader).
Software,
};
struct DynamicBackendParameters {
Uint64 UniformBufferOffsetAlignment = 256;
// GL_SHADER_STORAGE_BUFFER_OFFSET_ALIGNMENT, which is a SEPARATE limit from the
// uniform one and is routinely larger: Adreno 830 reports 32 for uniform buffers and
// 64 for storage buffers. Answering the storage query with the uniform value let an
// application bind a storage range at an offset the driver cannot address, which it
// accepted without error and then wrote somewhere else entirely.
Uint64 ShaderStorageBufferOffsetAlignment = 256;
// GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT. 1.0 means the backend cannot filter anisotropically,
// which is also why the extension is not advertised in that case.
Float MaxTextureMaxAnisotropy = 1.0f;
Float AliasedLineWidthRangeMin = 1.0f;
Float AliasedLineWidthRangeMax = 1.0f;
Float SmoothLineWidthRangeMin = 1.0f;
Float SmoothLineWidthRangeMax = 1.0f;
Float SmoothLineWidthGranularity = 1.0f;
Float PointSizeRangeMin = 1.0f;
Float PointSizeRangeMax = 1.0f;
Float PointSizeGranularity = 1.0f;
Int Max3DTextureSize = 16384;
Int MaxArrayTextureLayers = 2048;
Int MaxCubeMapTextureSize = 16384;
Int MaxFramebufferWidth = 16384;
Int MaxFramebufferHeight = 16384;
Int MaxFramebufferLayers = 2048;
Int MaxRenderbufferSize = 16384;
Int MaxTextureSize = 16384;
Int MaxColorTextureSamples = 1;
Int MaxDepthTextureSamples = 1;
Int MaxFramebufferSamples = 1;
Int MaxIntegerSamples = 1;
Int MaxSamples = 1;
Int MaxSampleMaskWords = 1;
// Tessellation limits; defaults are the GL 4.0 core minimums.
Int MaxPatchVertices = 32;
Int MaxTessGenLevel = 64;
// GL_MIN/MAX_PROGRAM_TEXTURE_GATHER_OFFSET. Defaults are the GL 4.0 core
// minimums, which every ES 3.1 driver also guarantees.
Int MinProgramTextureGatherOffset = -8;
Int MaxProgramTextureGatherOffset = 7;
Int MaxTextureImageUnits = 32;
Int MaxVertexTextureImageUnits = 32;
Int MaxComputeTextureImageUnits = 32;
Int MaxCombinedTextureImageUnits = 192;
Int MaxVertexAttribs = 16;
Int MaxComputeShaderStorageBlocks = 8;
Int MaxCombinedShaderStorageBlocks = 32;
// Per-stage GL_MAX_*_SHADER_STORAGE_BLOCKS. Zero is a legal answer for the four
// non-compute, non-fragment stages and these defaults are the spec minimums, not
// placeholders: GL 4.6 table 23.64 and ES 3.2 table 21.44 both set the minimum for
// vertex, tessellation control, tessellation evaluation and geometry at 0, and only
// fragment (8 in GL, 4 in ES) and compute are guaranteed to have any. Every real ARM
// GLES driver takes that allowance - a Mali-G925 reports 0 for all four - so a
// backend that cannot honour a graphics-stage storage block MUST report 0 here
// rather than a hopeful number. Advertising a non-zero count the driver will refuse
// does not make the block work; it only moves the failure from an honest
// "unsupported" at query time to a backend link error the frontend never surfaces,
// after which every draw with that program silently renders nothing.
Int MaxVertexShaderStorageBlocks = 0;
Int MaxTessControlShaderStorageBlocks = 0;
Int MaxTessEvaluationShaderStorageBlocks = 0;
Int MaxGeometryShaderStorageBlocks = 0;
Int MaxFragmentShaderStorageBlocks = 8;
Int MaxComputeUniformBlocks = 12;
Int MaxComputeWorkGroupInvocations = 128;
// GL_MAX_COMPUTE_WORK_GROUP_COUNT / GL_MAX_COMPUTE_WORK_GROUP_SIZE, one value per
// axis. These six, with the invocations limit above, are the only indexed limits a
// backend genuinely OWNS - the device answers them (glGetIntegeri_v on DirectGLES,
// VkPhysicalDeviceLimits::maxComputeWorkGroupCount/Size on DirectVulkan) - and so
// the only ones that survive the retirement of the GetIntegeri_v table entry: they
// cross the MGPipe boundary inside MGPCaps, by inclusion of this struct (plan B
// section 4.4.1). Every other indexed pname names frontend state. RAW driver
// answers, like the invocations limit: GL_Getter and the compile environment floor
// them at the shared MIN_COMPUTE_WORK_GROUP_* minimums themselves. The defaults are
// the GL 4.3 core minimums (table 23.60) and describe the no-backend case, as
// MaxClipDistances' does.
Int MaxComputeWorkGroupCount[3] = {65535, 65535, 65535};
Int MaxComputeWorkGroupSize[3] = {1024, 1024, 64};
Int MaxShaderStorageBufferBindings = 8;
Int MaxTextureBufferSize = 65536;
// GL_TEXTURE_BUFFER_OFFSET_ALIGNMENT; 1 means the offset is unconstrained.
Int TextureBufferOffsetAlignment = 1;
Int MaxUniformBufferBindings = 24;
Int MaxUniformBlockSize = 16384;
Int MaxImageUnits = 8;
Int MaxCombinedImageUniforms = 8;
Int MaxVertexImageUniforms = 0;
Int MaxTessControlImageUniforms = 0;
Int MaxTessEvaluationImageUniforms = 0;
Int MaxGeometryImageUniforms = 0;
Int MaxFragmentImageUniforms = 8;
Int MaxComputeImageUniforms = 8;
Int MaxDrawBuffers = 8;
Int MaxColorAttachments = 8;
// GL_MAX_CLIP_DISTANCES. Zero is a legal answer here, not a placeholder, and a
// backend that cannot host a clip distance MUST report it: advertising eight the
// backend will refuse does not make gl_ClipDistance work, it only moves the failure
// from an honest "unsupported" at query time to a backend shader-compile error the
// frontend never surfaces, after which every draw with that program silently renders
// nothing. DirectGLES fills it from GL_EXT_clip_cull_distance, DirectVulkan from the
// shaderClipDistance device feature. The DEFAULT stays at the GL 4.3 core minimum
// because it describes the no-backend case (standalone shader compiles, unit tests),
// where there is no device to be honest about and BuildTBuiltInResource still has to
// hand glslang a workable gl_MaxClipDistances.
Int MaxClipDistances = 8;
// GL_MAX_CULL_DISTANCES and GL_MAX_COMBINED_CLIP_AND_CULL_DISTANCES, under exactly
// the contract stated for MaxClipDistances above: ZERO IS A LEGAL ANSWER and a
// backend that cannot host a cull distance MUST report it. The failure this prevents
// is worse than the clip one, because cull distance discards the whole primitive:
// glslang bounds gl_CullDistance[i] against maxCullDistances and expands
// gl_MaxCullDistances from it, SPIRV-Cross then emits
// `#extension GL_EXT_clip_cull_distance : require` into the ESSL, and a host driver
// without that extension rejects the program in an info log nobody surfaces. These
// used to be bare 8s inside BuildTBuiltInResource with no backend consulted at all.
// The DEFAULTS are the GL 4.5 core minimums for the same reason MaxClipDistances'
// is: they describe the no-backend case (standalone compiles, unit tests).
Int MaxCullDistances = 8;
Int MaxCombinedClipAndCullDistances = 8;
Int MaxViewports = 16;
// GL_LAYER_PROVOKING_VERTEX / GL_VIEWPORT_INDEX_PROVOKING_VERTEX: which vertex of a
// primitive supplies gl_Layer and gl_ViewportIndex. GL 4.6 table 23.65 makes
// GL_UNDEFINED_VERTEX a legal answer for both, and it is the honest default - naming
// a convention is a statement about behaviour, so a backend that does not pin one
// must not claim it does. DirectGLES fills the layer one from the ES 3.2 query and
// the viewport one from GL_OES_viewport_array, and leaves UNDEFINED where the
// capability is absent: without the viewport array extension only viewport 0 is ever
// rasterized, so no convention selects anything. DirectVulkan keeps UNDEFINED for
// both - which vertex provokes is decided per pipeline by
// VulkanRenderer::SelectProvokingVertexMode out of VK_EXT_provoking_vertex,
// provokingVertexModePerPipeline and the topology, so no single convention is true
// of the backend.
Uint32 LayerProvokingVertex = GL_UNDEFINED_VERTEX;
Uint32 ViewportIndexProvokingVertex = GL_UNDEFINED_VERTEX;
Int MaxViewportWidth = 16384;
Int MaxViewportHeight = 16384;
Float ViewportBoundsRangeMin = 0.0f;
Float ViewportBoundsRangeMax = 0.0f;
Int ViewportSubpixelBits = 0;
// GL 4.x fragment-interpolation offset limits. These defaults are the
// core minimums and are replaced by live GLES/Vulkan device limits.
Float MinFragmentInterpolationOffset = -0.5f;
// For four fractional bits the greatest required legal offset is
// 0.5 - 2^-4 = 0.4375 (GL 4.6 table 23.70).
Float MaxFragmentInterpolationOffset = 0.4375f;
Int FragmentInterpolationOffsetBits = 4;
Bool SupportsWideLines = false;
// Whether a framebuffer whose depth and stencil attachments are distinct
// images can be rendered to. GL only requires support when both refer to the
// same image and lets an implementation answer GL_FRAMEBUFFER_UNSUPPORTED
// otherwise, which is what DirectVulkan (one combined attachment) and the
// real ES drivers behind DirectGLES both do. Defaults to true so a backend
// that never sets it keeps the permissive behaviour.
Bool SupportsDistinctDepthStencilAttachments = true;
// Whether attaching a single layer of a 3D or array texture to a framebuffer actually
// renders to that layer. DirectGLES hands the layer straight to
// glFramebufferTextureLayer, so it does; DirectVulkan maps a GL layer onto a Vulkan
// array layer with no notion of a 3D depth slice, so it does not yet. Defaults to false
// so a backend that never sets it gets the conservative answer.
// Which layered texture targets this backend can attach ONE layer of to a framebuffer
// and then really clear, render and read back that layer. Bit (1u << TextureTarget) is
// set for each supported target. Deliberately per target rather than one flag: the three
// ways a GL layer maps onto Vulkan are independent capabilities. A 2D or 2D multisample
// array layer IS a VkImage array layer and needs nothing extra; a 3D texture's layer is
// a z slice, which needs a 2D-array-compatible image and a per-slice clear that
// vkCmdClearColorImage cannot express; a cube map array needs an image shape and the
// imageCubeArray feature before it can be attached at any layer at all. Defaults to 0 so
// a backend that never sets it gets the conservative answer.
Uint32 PerLayerFramebufferAttachmentTargets = 0;
static constexpr Uint32 PerLayerFramebufferAttachmentBit(TextureTarget target) {
return (static_cast<Int>(target) >= 0 &&
static_cast<Int>(target) < static_cast<Int>(TextureTarget::TextureTargetCount))
? (1u << static_cast<Uint32>(target))
: 0u;
}
Bool SupportsPerLayerFramebufferAttachment(TextureTarget target) const {
const Uint32 bit = PerLayerFramebufferAttachmentBit(target);
return bit != 0 && (PerLayerFramebufferAttachmentTargets & bit) != 0;
}
// Whether this backend can CONSUME a shader module that still declares 64-bit floats,
// i.e. whether `double` survives the transpile instead of being narrowed to `float`
// (ShaderTranspiler::DemoteFloat64Pass). Detected, never assumed:
// * DirectVulkan sets it from VkPhysicalDeviceFeatures::shaderFloat64, the feature
// VUID-VkShaderModuleCreateInfo-pCode-08740 requires before a module declaring
// OpCapability Float64 may be created at all. lavapipe has it; Adreno and Mali
// both report VK_FALSE, so no real mobile device does.
// * DirectGLES can NEVER have it. GLSL ES has no 64-bit float type in any version
// or extension, so SPIRV-Cross cannot emit one ("FP64 not supported in ES
// profile") and the demotion there is mathematically mandatory, always.
// Defaults to false so a backend that never sets it - and the no-backend case, which
// is what standalone shader compiles and the unit tests run under - keeps the
// demotion, which is the behaviour that works everywhere.
Bool SupportsShaderFloat64 = false;
// Whether glVertexAttribLFormat / glVertexArrayAttribLFormat can be honoured, i.e.
// whether a 64-bit vertex attribute can actually reach a shader unconverted. Detected,
// never assumed: DirectVulkan needs VkPhysicalDeviceFeatures::shaderFloat64 (the
// attribute travels as its 32-bit word pair, so no VK_FORMAT_R64* is required, but the
// bitcast result is Float64); DirectGLES can never have it, ESSL having no fp64 type at
// all. Defaults to false so a backend that never sets it gets the conservative answer.
//
// INDEPENDENT of SupportsShaderFloat64, and it has to be: this flag decides a VkFormat
// from the VAO ATTRIBUTE alone, which does not know what type the shader declared, and
// glVertexAttribFormat(GL_DOUBLE) feeding a plain `in vec4` is both legal and common
// (KHR-GL43.vertex_attrib_binding.basic-input-case4/5, advanced-bindingUpdate). A
// backend with native fp64 that still cannot FETCH 64 bits keeps this false and relies
// on the per-MODULE rule in ShaderCompiler::SanitizeAndOptimizeBinary instead: a vertex
// module that declares a 64-bit float INPUT is demoted whole, so the two shader-side
// halves (PackDoubleVertexInputsPass and VertexInputStateFactory::ToVkVertexFormat)
// still see one consistent world.
Bool SupportsFloat64VertexAttributes = false;
// Whether a TESSELLATION stage of this backend may access gl_PointSize - i.e.
// whether a module declaring OpCapability TessellationPointSize can reach the
// driver at all. DirectVulkan sets both this and the geometry twin from the one
// shaderTessellationAndGeometryPointSize feature; DirectGLES sets them
// independently from the EXT/OES_tessellation_point_size /
// geometry_point_size extension pairs (PointSizeTier), which really do come
// separately. When absent, ProgramSpirvTask demotes the built-in to an ordinary
// varying program-wide (ShaderCompiler::
// DemoteTessellationGeometryPointSizeForProgram); MOBILEGL_POINT_SIZE_DEMOTION
// overrides the detection in either direction at backend init.
//
// Defaults TRUE, deliberately against the house "assume absent" rule: false
// ARMS a rewrite, so the conservative no-backend answer (standalone compiles,
// unit tests) is the one that leaves modules untouched. A backend that never
// sets it gets standard modules and, at worst, the old honest declines.
Bool SupportsTessellationPointSize = true;
// The geometry-stage twin (OpCapability GeometryPointSize).
Bool SupportsGeometryPointSize = true;
Uint64 MaxShaderStorageBlockSize = 128 * 1024 * 1024;
Uint32 SubgroupSize = 0;
Uint32 SubgroupSupportedStages = 0;
Uint32 SubgroupSupportedFeatures = 0;
// GL_SAMPLE_POSITION (GL 4.6 core 14.3.1.1) as the device actually places the samples
// of a 2-, 4-, 8- and 16-sample pixel: (x, y) pairs in [0, 1], the N-sample row starting
// at pair N - 2. Bit log2(N) of SamplePositionCountMask says row N was filled in; a row
// the backend could not determine stays unset rather than being guessed. Read and
// written through Get/SetSamplePositions only.
static constexpr Int kMaxSamplePositionCount = 16;
Float SamplePositions[60] = {}; // 2 * (2 + 4 + 8 + 16)
Uint32 SamplePositionCountMask = 0;
Bool SubgroupQuadOperationsInAllStages = false;
GpuVendorKind GpuVendor = GpuVendorKind::Unknown;
// GL_EXT_fragment_shading_rate and its two companions. Each is a statement about the
// device, so all of them default to "absent" and the extension strings are advertised
// only from these: DirectGLES fills them from the driver's own
// EXT_fragment_shading_rate(_primitive, _attachment), DirectVulkan from
// VK_KHR_fragment_shading_rate's pipeline / primitive / attachment features.
// SupportsFragmentShadingRate glShadingRateEXT and gl_ShadingRateEXT
// SupportsPrimitiveFragmentShadingRate gl_PrimitiveShadingRateEXT, combiner op 0
// SupportsAttachmentFragmentShadingRate the R8UI rate attachment, combiner op 1
Bool SupportsFragmentShadingRate = false;
Bool SupportsPrimitiveFragmentShadingRate = false;
Bool SupportsAttachmentFragmentShadingRate = false;
// The four GL_FRAGMENT_SHADING_RATE_*_SUPPORTED_EXT booleans, as the device answers them.
Bool FragmentShadingRateNonTrivialCombiners = false;
Bool FragmentShadingRateWithShaderDepthStencilWrites = false;
Bool FragmentShadingRateWithSampleMask = false;
Bool FragmentShadingRatePrimitiveWithMultiViewport = false;
Bool FragmentShadingRateAttachmentWithDefaultFramebuffer = false;
// What glGetFragmentShadingRatesEXT answers, one mask per sample count 1, 2, 4 .. 64
// (index = log2(samples)): bit (log2(w) << 2 | log2(h)) is set when that rate is
// honoured exactly at that sample count, i.e. gl_ShadingRateEXT reads it back
// unclamped. Zero masks mean no rate at all, which is what a sample count the device
// cannot shade coarsely answers.
Uint16 FragmentShadingRateMasks[7] = {};
// The GL_{MIN,MAX}_FRAGMENT_SHADING_RATE_ATTACHMENT_TEXEL_{WIDTH,HEIGHT}_EXT,
// _ASPECT_RATIO_EXT and _LAYERS_EXT limits. Meaningful only with the attachment.
Int MinFragmentShadingRateAttachmentTexelWidth = 0;
Int MinFragmentShadingRateAttachmentTexelHeight = 0;
Int MaxFragmentShadingRateAttachmentTexelWidth = 0;
Int MaxFragmentShadingRateAttachmentTexelHeight = 0;
Int MaxFragmentShadingRateAttachmentTexelAspectRatio = 0;
Int MaxFragmentShadingRateAttachmentLayers = 0;
static constexpr Bool IsSamplePositionCount(Int samples) {
return samples >= 2 && samples <= kMaxSamplePositionCount && (samples & (samples - 1)) == 0;
}
static constexpr Uint32 SamplePositionCountBit(Int samples) {
return 1u << static_cast<Uint32>(std::countr_zero(static_cast<Uint32>(samples)));
}
// `xy` holds `samples` (x, y) pairs.
void SetSamplePositions(Int samples, const Float* xy) {
if (!IsSamplePositionCount(samples)) return;
for (Int i = 0; i < 2 * samples; ++i) SamplePositions[2 * (samples - 2) + i] = xy[i];
SamplePositionCountMask |= SamplePositionCountBit(samples);
}
Bool GetSamplePosition(Int samples, Int index, Float* outXY) const {
if (!IsSamplePositionCount(samples) || index < 0 || index >= samples ||
(SamplePositionCountMask & SamplePositionCountBit(samples)) == 0) {
return false;
}
outXY[0] = SamplePositions[2 * (samples - 2 + index)];
outXY[1] = SamplePositions[2 * (samples - 2 + index) + 1];
return true;
}
};
enum class WindowBackend {
Android,
X11,
@@ -866,7 +461,8 @@ namespace MobileGL {
virtual Bool SwapEGLBuffers(EGLDisplay dpy, EGLSurface draw);
// eglSwapBuffersWithDamage: `damage` (GL window coordinates) is a hint for the native
// present. A backend with nothing to hand it to swaps as SwapEGLBuffers does.
virtual Bool SwapEGLBuffersWithDamage(EGLDisplay dpy, EGLSurface draw, const MG_Util::Damage::Region& damage) {
virtual Bool SwapEGLBuffersWithDamage(EGLDisplay dpy, EGLSurface draw,
const MG_Util::Damage::Region& damage) {
(void)damage;
return SwapEGLBuffers(dpy, draw);
}
@@ -977,5 +573,8 @@ namespace MobileGL {
void DestroyPendingEGLSurfaceIfUnused(EGLSurface surface);
void ReleaseEGLCurrentThread(const std::thread::id& threadKey);
};
extern UniquePtr<BackendObject>& pActiveBackendObject;
extern GlobalBackendFunctionsTable gBackendFunctionsTable;
} // namespace MG_Backend
} // namespace MobileGL
-3
View File
@@ -13,9 +13,6 @@
#include "DirectVulkan/BackendObject_DirectVulkan.h"
namespace MobileGL::MG_Backend {
extern UniquePtr<BackendObject>& pActiveBackendObject;
extern GlobalBackendFunctionsTable gBackendFunctionsTable;
#if MOBILEGL_BUILD_DISAGGREGATED
// P5 v1. The counterpart of Init()'s single split hook, and a NO-OP in every run that is
// not split. It exists ONLY in a split build: G1 admits no new symbol in the pull build,
+491
View File
@@ -0,0 +1,491 @@
// MobileGL - MobileGL/MG_Backend/ClientReadback.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ClientReadback.h"
#include <MG_Backend/MGPipe/PipeInputs.h>
#include <MG_Util/Math/HalfFloat.h>
#include <MG_Util/Math/SmallFloat.h>
#include <algorithm>
#include <cmath>
namespace MobileGL::MG_Backend {
// ---- Client-format readback conversion helpers -------------------------------------------------
// ReadPixels/GetTexImage read a guaranteed wide RGBA(_INTEGER) layout from the ES driver and repack
// it on the CPU into the client's (format, type) layout. Everything here is pure byte shuffling so
// unit tests can assert the exact packed words; field positions follow GL 3.3 table 3.6 and mirror
// the GL CTS packed_pixels oracle (glcPackedPixelsTests.cpp pack_UNSIGNED_* helpers).
namespace ClientReadback {
using MG_Util::DecodeHalfBitsToFloat;
using MG_Util::EncodeFloatToHalfBits;
Bool GetReadbackChannelMapping(GLenum format, ReadbackChannelMapping& outMapping) {
switch (format) {
case GL_RED:
outMapping = {{0, 0, 0, 0}, 1, false};
return true;
case GL_RED_INTEGER:
outMapping = {{0, 0, 0, 0}, 1, true};
return true;
// Desktop-GL single-channel client formats (GL CTS packed_pixels rgba8_format_green/blue):
// the destination holds one component sourced from the named channel of the wide RGBA read.
// GL_ALPHA is mapped here from the raw enum because the state layer folds it into Red for the
// legacy alpha-texture upload hack.
case GL_GREEN:
outMapping = {{1, 0, 0, 0}, 1, false};
return true;
case GL_GREEN_INTEGER:
outMapping = {{1, 0, 0, 0}, 1, true};
return true;
case GL_BLUE:
outMapping = {{2, 0, 0, 0}, 1, false};
return true;
case GL_BLUE_INTEGER:
outMapping = {{2, 0, 0, 0}, 1, true};
return true;
case GL_ALPHA:
outMapping = {{3, 0, 0, 0}, 1, false};
return true;
case GL_ALPHA_INTEGER:
outMapping = {{3, 0, 0, 0}, 1, true};
return true;
case GL_RG:
outMapping = {{0, 1, 0, 0}, 2, false};
return true;
case GL_RG_INTEGER:
outMapping = {{0, 1, 0, 0}, 2, true};
return true;
case GL_RGB:
outMapping = {{0, 1, 2, 0}, 3, false};
return true;
case GL_RGB_INTEGER:
outMapping = {{0, 1, 2, 0}, 3, true};
return true;
case GL_BGR:
outMapping = {{2, 1, 0, 0}, 3, false};
return true;
case GL_BGR_INTEGER:
outMapping = {{2, 1, 0, 0}, 3, true};
return true;
case GL_RGBA:
outMapping = {{0, 1, 2, 3}, 4, false};
return true;
case GL_RGBA_INTEGER:
outMapping = {{0, 1, 2, 3}, 4, true};
return true;
case GL_BGRA:
outMapping = {{2, 1, 0, 3}, 4, false};
return true;
case GL_BGRA_INTEGER:
outMapping = {{2, 1, 0, 3}, 4, true};
return true;
default:
return false;
}
}
Bool GetPackedReadbackLayout(GLenum type, PackedReadbackLayout& out) {
switch (type) {
// Non-REV types pack the first format component starting at the most significant bit,
// *_REV types starting at the least significant bit (GL CTS pack_UNSIGNED_SHORT_5_6_5:
// R bits 15-11; pack_UNSIGNED_SHORT_1_5_5_5_REV: R bits 4-0, A bit 15).
case GL_UNSIGNED_BYTE_3_3_2:
out = {3, {3, 3, 2, 0}, {5, 2, 0, 0}, 1, false};
return true;
case GL_UNSIGNED_BYTE_2_3_3_REV:
out = {3, {3, 3, 2, 0}, {0, 3, 6, 0}, 1, false};
return true;
case GL_UNSIGNED_SHORT_5_6_5:
out = {3, {5, 6, 5, 0}, {11, 5, 0, 0}, 2, false};
return true;
case GL_UNSIGNED_SHORT_5_6_5_REV:
out = {3, {5, 6, 5, 0}, {0, 5, 11, 0}, 2, false};
return true;
case GL_UNSIGNED_SHORT_4_4_4_4:
out = {4, {4, 4, 4, 4}, {12, 8, 4, 0}, 2, false};
return true;
case GL_UNSIGNED_SHORT_4_4_4_4_REV:
out = {4, {4, 4, 4, 4}, {0, 4, 8, 12}, 2, false};
return true;
case GL_UNSIGNED_SHORT_5_5_5_1:
out = {4, {5, 5, 5, 1}, {11, 6, 1, 0}, 2, false};
return true;
case GL_UNSIGNED_SHORT_1_5_5_5_REV:
out = {4, {5, 5, 5, 1}, {0, 5, 10, 15}, 2, false};
return true;
case GL_UNSIGNED_INT_8_8_8_8:
out = {4, {8, 8, 8, 8}, {24, 16, 8, 0}, 4, false};
return true;
case GL_UNSIGNED_INT_8_8_8_8_REV:
out = {4, {8, 8, 8, 8}, {0, 8, 16, 24}, 4, false};
return true;
case GL_UNSIGNED_INT_10_10_10_2:
out = {4, {10, 10, 10, 2}, {22, 12, 2, 0}, 4, false};
return true;
case GL_UNSIGNED_INT_2_10_10_10_REV:
out = {4, {10, 10, 10, 2}, {0, 10, 20, 30}, 4, false};
return true;
// Packed-float RGB types: fields hold unsigned small floats; 5_9_9_9_REV's shared 5-bit
// exponent (bits 31-27) is emitted by EncodeSharedExponentRGB9E5, not a component field.
case GL_UNSIGNED_INT_10F_11F_11F_REV:
out = {3, {11, 11, 10, 0}, {0, 11, 22, 0}, 4, true};
return true;
case GL_UNSIGNED_INT_5_9_9_9_REV:
out = {3, {9, 9, 9, 0}, {0, 9, 18, 0}, 4, true};
return true;
default:
return false;
}
}
SizeT GetReadbackComponentSize(GLenum type) {
PackedReadbackLayout packedLayout{};
if (GetPackedReadbackLayout(type, packedLayout)) {
return packedLayout.byteSize;
}
switch (type) {
case GL_UNSIGNED_BYTE:
case GL_BYTE:
return 1;
case GL_UNSIGNED_SHORT:
case GL_SHORT:
case GL_HALF_FLOAT:
return 2;
case GL_UNSIGNED_INT:
case GL_INT:
case GL_FLOAT:
return 4;
default:
return 0;
}
}
SizeT GetReadbackDstPixelSize(const ReadbackChannelMapping& mapping, GLenum type) {
PackedReadbackLayout packedLayout{};
if (GetPackedReadbackLayout(type, packedLayout)) {
if (packedLayout.fieldCount != mapping.channelCount) {
return 0; // 3-field packed types pair with 3-component formats only, 4 with 4
}
if (mapping.isInteger && packedLayout.isFloatPacked) {
return 0; // packed-float RGB types never pair with integer formats
}
return packedLayout.byteSize;
}
if (mapping.isInteger && (type == GL_FLOAT || type == GL_HALF_FLOAT)) {
return 0;
}
const SizeT componentSize = GetReadbackComponentSize(type);
return componentSize == 0 ? 0 : static_cast<SizeT>(mapping.channelCount) * componentSize;
}
namespace {
void WritePackedReadbackWord(Uint8* dst, Uint32 word, SizeT byteSize) {
switch (byteSize) {
case 1: {
const auto out = static_cast<Uint8>(word);
Memcpy(dst, &out, sizeof(out));
break;
}
case 2: {
const auto out = static_cast<Uint16>(word);
Memcpy(dst, &out, sizeof(out));
break;
}
default:
Memcpy(dst, &word, sizeof(word));
break;
}
}
} // namespace
// Shared encoders live in MG_Util/Math/SmallFloat.h so the upload conversion
// (PixelStoreProcessor) uses byte-identical packing; kept exported here for unit tests.
Uint32 EncodeFloatToUnsignedF11(Float value) {
return MG_Util::EncodeFloatToUnsignedF11(value);
}
Uint32 EncodeFloatToUnsignedF10(Float value) {
return MG_Util::EncodeFloatToUnsignedF10(value);
}
Uint32 EncodeSharedExponentRGB9E5(const Float rgb[3]) {
return MG_Util::EncodeSharedExponentRGB9E5(rgb);
}
void ConvertWideReadbackRow(const Uint8* src, Uint8* dst, SizeT width, GLenum wideType,
const ReadbackChannelMapping& mapping, GLenum type) {
PackedReadbackLayout packedLayout{};
const Bool isPacked = GetPackedReadbackLayout(type, packedLayout);
const SizeT dstComponentSize = GetReadbackComponentSize(type);
const SizeT dstPixelBytes = GetReadbackDstPixelSize(mapping, type);
const SizeT srcPixelBytes = 4 * GetReadbackComponentSize(wideType);
for (SizeT col = 0; col < width; ++col) {
const Uint8* srcPixel = src + col * srcPixelBytes;
Uint8* dstPixel = dst + col * dstPixelBytes;
if (mapping.isInteger) {
Int64 srcValues[4];
for (Int c = 0; c < 4; ++c) {
srcValues[c] = wideType == GL_INT
? static_cast<Int64>(reinterpret_cast<const Int32*>(srcPixel)[c])
: static_cast<Int64>(reinterpret_cast<const Uint32*>(srcPixel)[c]);
}
if (isPacked) {
// Integer sources clamp each component to the unsigned range of its field
// (GL 3.3 section 4.3.1 final conversion).
Uint32 word = 0;
for (Int ch = 0; ch < packedLayout.fieldCount; ++ch) {
const Int64 fieldMax = (Int64{1} << packedLayout.width[ch]) - 1;
const auto v = static_cast<Uint32>(
std::clamp<Int64>(srcValues[mapping.sourceChannel[ch]], 0, fieldMax));
word |= v << packedLayout.shift[ch];
}
WritePackedReadbackWord(dstPixel, word, packedLayout.byteSize);
} else {
for (Int ch = 0; ch < mapping.channelCount; ++ch) {
const Int64 v = srcValues[mapping.sourceChannel[ch]];
Uint8* dstComponent = dstPixel + static_cast<SizeT>(ch) * dstComponentSize;
switch (type) {
case GL_UNSIGNED_BYTE:
*dstComponent = static_cast<Uint8>(std::clamp<Int64>(v, 0, 255));
break;
case GL_BYTE: {
const auto out = static_cast<Int8>(std::clamp<Int64>(v, -128, 127));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_SHORT: {
const auto out = static_cast<Uint16>(std::clamp<Int64>(v, 0, 65535));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_SHORT: {
const auto out = static_cast<Int16>(std::clamp<Int64>(v, -32768, 32767));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_INT: {
const auto out = static_cast<Uint32>(std::clamp<Int64>(v, 0, 4294967295LL));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_INT: {
const auto out = static_cast<Int32>(std::clamp<Int64>(v, -2147483648LL, 2147483647LL));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
default:
break;
}
}
}
} else {
Float srcValues[4];
switch (wideType) {
case GL_UNSIGNED_BYTE:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = static_cast<Float>(srcPixel[c]) / 255.0f;
}
break;
case GL_BYTE:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = std::max(
static_cast<Float>(reinterpret_cast<const Int8*>(srcPixel)[c]) / 127.0f, -1.0f);
}
break;
case GL_UNSIGNED_SHORT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = static_cast<Float>(reinterpret_cast<const Uint16*>(srcPixel)[c]) / 65535.0f;
}
break;
case GL_SHORT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = std::max(
static_cast<Float>(reinterpret_cast<const Int16*>(srcPixel)[c]) / 32767.0f, -1.0f);
}
break;
case GL_HALF_FLOAT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = DecodeHalfBitsToFloat(reinterpret_cast<const Uint16*>(srcPixel)[c]);
}
break;
default: // GL_FLOAT
for (Int c = 0; c < 4; ++c) {
srcValues[c] = reinterpret_cast<const Float*>(srcPixel)[c];
}
break;
}
if (isPacked) {
Uint32 word = 0;
if (packedLayout.isFloatPacked) {
const Float fields[3] = {srcValues[mapping.sourceChannel[0]],
srcValues[mapping.sourceChannel[1]],
srcValues[mapping.sourceChannel[2]]};
word = type == GL_UNSIGNED_INT_5_9_9_9_REV
? EncodeSharedExponentRGB9E5(fields)
: (EncodeFloatToUnsignedF11(fields[0]) << packedLayout.shift[0]) |
(EncodeFloatToUnsignedF11(fields[1]) << packedLayout.shift[1]) |
(EncodeFloatToUnsignedF10(fields[2]) << packedLayout.shift[2]);
} else {
// Normalized encode: round(clamp(v, 0, 1) * (2^bits - 1)) into each field.
for (Int ch = 0; ch < packedLayout.fieldCount; ++ch) {
const auto fieldMax = static_cast<Float>((1u << packedLayout.width[ch]) - 1u);
const auto v = static_cast<Uint32>(std::llround(
std::clamp(srcValues[mapping.sourceChannel[ch]], 0.0f, 1.0f) * fieldMax));
word |= v << packedLayout.shift[ch];
}
}
WritePackedReadbackWord(dstPixel, word, packedLayout.byteSize);
} else {
for (Int ch = 0; ch < mapping.channelCount; ++ch) {
const Float v = srcValues[mapping.sourceChannel[ch]];
Uint8* dstComponent = dstPixel + static_cast<SizeT>(ch) * dstComponentSize;
switch (type) {
case GL_UNSIGNED_BYTE:
*dstComponent = static_cast<Uint8>(std::llround(std::clamp(v, 0.0f, 1.0f) * 255.0));
break;
case GL_BYTE: {
const auto out = static_cast<Int8>(std::llround(std::clamp(v, -1.0f, 1.0f) * 127.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_SHORT: {
const auto out = static_cast<Uint16>(std::llround(std::clamp(v, 0.0f, 1.0f) * 65535.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_SHORT: {
const auto out = static_cast<Int16>(std::llround(std::clamp(v, -1.0f, 1.0f) * 32767.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_INT: {
const auto out = static_cast<Uint32>(
std::llround(static_cast<Double>(std::clamp(v, 0.0f, 1.0f)) * 4294967295.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_INT: {
const auto out = static_cast<Int32>(
std::llround(static_cast<Double>(std::clamp(v, -1.0f, 1.0f)) * 2147483647.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_FLOAT:
Memcpy(dstComponent, &v, sizeof(v));
break;
case GL_HALF_FLOAT: {
const Uint16 out = EncodeFloatToHalfBits(v);
Memcpy(dstComponent, &out, sizeof(out));
break;
}
default:
break;
}
}
}
}
}
}
static SizeT AlignReadbackRow(SizeT rowBytes, Int alignment) {
const SizeT align = alignment > 0 ? static_cast<SizeT>(alignment) : 1;
return (rowBytes + align - 1) / align * align;
}
// Walks the client-side destination the PACK parameters describe and hands each row to
// `fillRow(slice, row, dstRow)`, which writes width * dstPixelBytes bytes of finished client
// texels. Shared by the converting and the raw-word stores so both address the destination
// identically. The destination is always client memory: a readback into a pack buffer takes
// the verb path and never reaches these stores.
// applyPackImageParams: GL_PACK_IMAGE_HEIGHT / GL_PACK_SKIP_IMAGES apply only to GetTexImage
// of 3D/array images; ReadPixels and 2D GetTexImage ignore them (GL 3.3 sections 4.3.1, 6.1.4).
// Per the GL addressing rules, slice k row j lands at
// SKIP_IMAGES*imageStride + SKIP_ROWS*rowStride + SKIP_PIXELS*pixelBytes
// + k*imageStride + j*rowStride, with imageStride = max(IMAGE_HEIGHT, sliceHeight)*rowStride.
template <typename FillRow>
static Bool StoreClientRows(SizeT dstPixelBytes, SizeT swapGroupSize, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, void* pixels, Bool applyPackImageParams, FillRow&& fillRow) {
// Destination layout is computed from the client-side PACK parameters; only the actual pixel
// rows are written so skip regions of the destination stay untouched.
const auto packParams = MG_Pipe::gPipeInputs->GetPixelStoreParameters(false);
const SizeT rowPixels = static_cast<SizeT>(packParams.RowLength > 0 ? packParams.RowLength : width);
const SizeT dstRowStride = AlignReadbackRow(rowPixels * dstPixelBytes, packParams.Alignment);
const SizeT imageRows = applyPackImageParams && packParams.ImageHeight > 0
? static_cast<SizeT>(packParams.ImageHeight)
: static_cast<SizeT>(sliceHeight);
const SizeT dstImageStride = imageRows * dstRowStride;
const SizeT skipImages =
applyPackImageParams ? static_cast<SizeT>(std::max(packParams.SkipImages, 0)) : SizeT{0};
const SizeT dstSkipOffset = skipImages * dstImageStride +
static_cast<SizeT>(std::max(packParams.SkipRows, 0)) * dstRowStride +
static_cast<SizeT>(std::max(packParams.SkipPixels, 0)) * dstPixelBytes;
const SizeT dstRowBytes = static_cast<SizeT>(width) * dstPixelBytes;
Vector<Uint8> convertedRow(dstRowBytes);
for (GLsizei slice = 0; slice < sliceCount; ++slice) {
for (GLsizei row = 0; row < sliceHeight; ++row) {
fillRow(slice, row, convertedRow.data());
if (packParams.SwapBytes && swapGroupSize > 1) {
for (SizeT offset = 0; offset + swapGroupSize <= dstRowBytes; offset += swapGroupSize) {
std::reverse(convertedRow.data() + offset, convertedRow.data() + offset + swapGroupSize);
}
}
const SizeT dstOffset = dstSkipOffset + static_cast<SizeT>(slice) * dstImageStride +
static_cast<SizeT>(row) * dstRowStride;
Memcpy(static_cast<Uint8*>(pixels) + dstOffset, convertedRow.data(), dstRowBytes);
}
}
return true;
}
// Repacks wide RGBA(_INTEGER) rows into the client's (format, type) layout, honoring the
// client-side PACK parameters. `wide` holds
// `sliceHeight * sliceCount` rows of `width` texels (slice-major, tightly stacked),
// 4 components x GetReadbackComponentSize(wideType) bytes each.
Bool StoreWideRowsToClient(const Uint8* wide, GLenum wideType, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, const ReadbackChannelMapping& mapping, GLenum type, void* pixels,
Bool applyPackImageParams) {
const SizeT dstPixelBytes = GetReadbackDstPixelSize(mapping, type);
if (dstPixelBytes == 0) {
return false;
}
PackedReadbackLayout packedLayout{};
const Bool isPackedType = GetPackedReadbackLayout(type, packedLayout);
const SizeT swapGroupSize = isPackedType ? packedLayout.byteSize : GetReadbackComponentSize(type);
const SizeT srcPixelBytes = 4 * GetReadbackComponentSize(wideType);
return StoreClientRows(
dstPixelBytes, swapGroupSize, width, sliceHeight, sliceCount, pixels, applyPackImageParams,
[&](GLsizei slice, GLsizei row, Uint8* dstRow) {
const SizeT flatRow =
static_cast<SizeT>(slice) * static_cast<SizeT>(sliceHeight) + static_cast<SizeT>(row);
const Uint8* srcRow = wide + flatRow * static_cast<SizeT>(width) * srcPixelBytes;
ConvertWideReadbackRow(srcRow, dstRow, static_cast<SizeT>(width), wideType, mapping, type);
});
}
Bool StorePackedWordsToClient(const Uint8* srcWords, GLsizei width, GLsizei sliceHeight, GLsizei sliceCount,
GLenum type, void* pixels, Bool applyPackImageParams) {
PackedReadbackLayout packedLayout{};
if (!GetPackedReadbackLayout(type, packedLayout) || packedLayout.byteSize != 4) {
return false;
}
const SizeT srcRowBytes = static_cast<SizeT>(width) * 4;
return StoreClientRows(4, packedLayout.byteSize, width, sliceHeight, sliceCount, pixels,
applyPackImageParams, [&](GLsizei slice, GLsizei row, Uint8* dstRow) {
const SizeT flatRow =
static_cast<SizeT>(slice) * static_cast<SizeT>(sliceHeight) +
static_cast<SizeT>(row);
Memcpy(dstRow, srcWords + flatRow * srcRowBytes, srcRowBytes);
});
}
} // namespace ClientReadback
} // namespace MobileGL::MG_Backend
+75
View File
@@ -0,0 +1,75 @@
// MobileGL - MobileGL/MG_Backend/ClientReadback.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
namespace MobileGL::MG_Backend {
// Pure CPU helpers of the client-format readback conversion (ReadPixels/GetTexImage repack a wide
// RGBA(_INTEGER) read into the caller's (format, type) layout). Kept context-free so unit tests can
// exercise the exact packing the GL CTS packed_pixels oracle compares against.
namespace ClientReadback {
struct ReadbackChannelMapping {
Int sourceChannel[4]; // RGBA source channel feeding each destination component
Int channelCount; // destination component count
Bool isInteger;
};
Bool GetReadbackChannelMapping(GLenum format, ReadbackChannelMapping& outMapping);
// Byte size of one destination component of `type`; packed types report the packed word size.
// 0 = type not supported by the conversion path.
SizeT GetReadbackComponentSize(GLenum type);
// Bit-field layout of a GL packed pixel type. width/shift are indexed in the client format's
// component order (matching ReadbackChannelMapping); shift is the LSB position of the field in
// the packed word: non-REV types pack the first component from the MSB, *_REV types from the
// LSB (GL 3.3 table 3.6; field positions mirror the GL CTS glcPackedPixelsTests pack_* oracle).
struct PackedReadbackLayout {
Int fieldCount; // format components stored in the packed word
Int width[4]; // bit width of each component's field
Int shift[4]; // LSB bit position of each component's field
SizeT byteSize; // packed word size in bytes (1, 2 or 4)
Bool isFloatPacked; // 10F_11F_11F_REV / 5_9_9_9_REV: fields hold unsigned small floats
};
Bool GetPackedReadbackLayout(GLenum type, PackedReadbackLayout& out);
// Unsigned small-float encoders (EXT_packed_float / EXT_texture_shared_exponent semantics).
Uint32 EncodeFloatToUnsignedF11(Float value);
Uint32 EncodeFloatToUnsignedF10(Float value);
Uint32 EncodeSharedExponentRGB9E5(const Float rgb[3]);
// Destination bytes per pixel for a (format mapping, type) readback pair; 0 when the pair is
// not convertible (unknown type, packed field count != format component count, floating-point
// or packed-float type with an integer format).
SizeT GetReadbackDstPixelSize(const ReadbackChannelMapping& mapping, GLenum type);
// Repacks one row of wide RGBA(_INTEGER) texels (4 components of wideType each) into the
// client's (format, type) layout. src holds width * 4 * GetReadbackComponentSize(wideType)
// bytes, dst receives width * GetReadbackDstPixelSize(mapping, type) bytes.
void ConvertWideReadbackRow(const Uint8* src, Uint8* dst, SizeT width, GLenum wideType,
const ReadbackChannelMapping& mapping, GLenum type);
// Stores wide RGBA(_INTEGER) rows into client memory, honoring the client-side PACK
// pixel-store parameters (row length, alignment, skips, swap-bytes, and - when
// applyPackImageParams - image height/skip images). Shared by the DirectGLES and DirectVulkan
// readback conversion paths.
Bool StoreWideRowsToClient(const Uint8* wide, GLenum wideType, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, const ReadbackChannelMapping& mapping, GLenum type, void* pixels,
Bool applyPackImageParams);
// Stores packed 32-bit source words verbatim, with the same destination addressing and PACK
// parameters as StoreWideRowsToClient. For the sources whose storage word already IS the
// client word (MG_Util::IsRawPackedPixelTransfer): routing those through the wide float
// intermediate re-encodes them, and the RGB9_E5 encoder canonicalizes
// the shared exponent, so glGetTexImage would answer with different bits than were stored.
// `srcWords` holds sliceHeight * sliceCount tightly stacked rows of `width` 32-bit words.
// False when `type` is not a 4-byte packed type.
Bool StorePackedWordsToClient(const Uint8* srcWords, GLsizei width, GLsizei sliceHeight, GLsizei sliceCount,
GLenum type, void* pixels, Bool applyPackImageParams);
} // namespace ClientReadback
} // namespace MobileGL::MG_Backend
@@ -50,14 +50,20 @@ namespace MobileGL::MG_Backend::DirectGLES {
// the recorded tuple; only that exact tuple may skip. Process-wide like the native state it
// mirrors; under split exactly one DirectGLES object exists (the server's), and only the
// apply thread reaches these.
enum class NativeBindState : Uint8 { NotBound, FreshFromSurfaceCreation, BoundForTuple };
enum class NativeBindState : Uint8 {
NotBound,
FreshFromSurfaceCreation,
BoundForTuple
};
NativeBindState g_nativeBindState = NativeBindState::NotBound;
EGLDisplay g_nativeBoundDpy = EGL_NO_DISPLAY;
EGLSurface g_nativeBoundDraw = EGL_NO_SURFACE;
EGLSurface g_nativeBoundRead = EGL_NO_SURFACE;
EGLContext g_nativeBoundCtx = EGL_NO_CONTEXT;
void NoteNativeContextGone() { g_nativeBindState = NativeBindState::NotBound; }
void NoteNativeContextGone() {
g_nativeBindState = NativeBindState::NotBound;
}
void NoteNativeContextFreshFromSurfaceCreation() {
g_nativeBindState = NativeBindState::FreshFromSurfaceCreation;
}
@@ -70,8 +76,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
Bool NativeBindCanBeSkippedFor(EGLDisplay dpy, EGLSurface draw, EGLSurface read, EGLContext ctx) {
switch (g_nativeBindState) {
case NativeBindState::NotBound: return false;
case NativeBindState::FreshFromSurfaceCreation: return true;
case NativeBindState::NotBound:
return false;
case NativeBindState::FreshFromSurfaceCreation:
return true;
case NativeBindState::BoundForTuple:
return g_nativeBoundDpy == dpy && g_nativeBoundDraw == draw && g_nativeBoundRead == read &&
g_nativeBoundCtx == ctx;
@@ -364,8 +372,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// for EVERY colour format while GL_MAX_SAMPLES - ES 3.0 core, so genuinely answered -
// reports 4. Once the frontend validates against this list, that would reject every
// multisample renderbuffer on such a context.
Int GetGLESRenderbufferFormatMaxSamples(const MG_External::GLESCapabilities& capabilities,
GLenum imageFormat) {
Int GetGLESRenderbufferFormatMaxSamples(const MG_External::GLESCapabilities& capabilities, GLenum imageFormat) {
const Bool isInteger = imageFormat == GL_RED_INTEGER || imageFormat == GL_RG_INTEGER ||
imageFormat == GL_RGB_INTEGER || imageFormat == GL_RGBA_INTEGER;
return isInteger ? capabilities.MaxIntegerSamples : capabilities.MaxSamples;
@@ -474,8 +481,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
GLint prevFramebuffer = 0, prevRenderbuffer = 0;
gl.glGetIntegerv(GL_FRAMEBUFFER_BINDING, &prevFramebuffer);
gl.glGetIntegerv(GL_RENDERBUFFER_BINDING, &prevRenderbuffer);
for (Int samples = 2; samples <= DynamicBackendParameters::kMaxSamplePositionCount &&
samples <= capabilities.MaxSamples;
for (Int samples = 2;
samples <= DynamicBackendParameters::kMaxSamplePositionCount && samples <= capabilities.MaxSamples;
samples *= 2) {
GLuint framebuffer = 0, renderbuffer = 0;
gl.glGenFramebuffers(1, &framebuffer);
@@ -502,8 +509,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
// A count the driver refused leaves its error behind; it must not reach the application.
if (gl.glGetError) {
for (Int drained = 0; drained < 16 && gl.glGetError() != GL_NO_ERROR; ++drained) {
}
for (Int drained = 0; drained < 16 && gl.glGetError() != GL_NO_ERROR; ++drained) {}
}
}
@@ -728,12 +734,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
const Flags<PixelFormatNormalizeOptionBit> forcedProbeOptions =
(outerHasForcedFallback || widenUnconditionally) ? forcedOptions | renderTargetOptions
: forcedOptions;
hasForcedFallback =
BuildFallbackProbeFormatInfo(requestedInternalFormat, forcedProbeOptions, true,
fallbackInfo);
hasForcedFallback = BuildFallbackProbeFormatInfo(requestedInternalFormat, forcedProbeOptions,
true, fallbackInfo);
if (!hasForcedFallback) {
BuildFallbackProbeFormatInfo(requestedInternalFormat,
driverOptions | renderTargetOptions, false, fallbackInfo);
BuildFallbackProbeFormatInfo(requestedInternalFormat, driverOptions | renderTargetOptions,
false, fallbackInfo);
}
// HONEST STATUS OF THE FORCED PATH. A forced fallback is only ever built
// for ANGLE (GetForcedPixelFormatNormalizeOptions returns nothing for any
@@ -1162,7 +1167,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
const std::lock_guard<std::recursive_mutex> lock(m_eglStateMutex);
const auto state = m_eglSurfaces.find(surface);
const auto native = m_nativeSurfaces.find(surface);
if (state == m_eglSurfaces.end() || state->second.Kind != SurfaceKind::Pbuffer || native == m_nativeSurfaces.end()) {
if (state == m_eglSurfaces.end() || state->second.Kind != SurfaceKind::Pbuffer ||
native == m_nativeSurfaces.end()) {
return BackendObject::ResizeEGLWindowSurface(surface, width, height);
}
// A pbuffer has no resize: the client surface gets a new native pbuffer at the new size,
@@ -1398,9 +1404,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// driver dequeues); only the client has to be told.
const auto native = m_nativeSurfaces.find(client);
if (native == m_nativeSurfaces.end()) continue;
published = DirectGLES::RepublishWindowSurfaceShape(native->second,
reinterpret_cast<NativeWindowType>(window)) ||
published;
published =
DirectGLES::RepublishWindowSurfaceShape(native->second, reinterpret_cast<NativeWindowType>(window)) ||
published;
}
return published;
}
@@ -1443,8 +1449,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
Vector<GLExtension> BuildAdvertisedExtensions(Bool timerQueriesSupported, Bool anisotropicFilteringSupported,
Bool drawIndirectSupported,
Bool nonZeroIndirectBaseInstanceSupported,
Bool drawIndirectSupported, Bool nonZeroIndirectBaseInstanceSupported,
Bool textureViewSupported, Bool cubeMapArraySupported,
const AdvertisedFeatureSupport& features) {
Vector<GLExtension> extensions = {
@@ -1452,23 +1457,21 @@ namespace MobileGL::MG_Backend::DirectGLES {
// TargetGLVersion is {4,6,0}, and a list that stopped at OpenGL40 told an
// application feature-detecting off these tokens the opposite of what
// GL_MAJOR_VERSION / GL_MINOR_VERSION told it.
V_OpenGL30, V_OpenGL31, V_OpenGL32, V_OpenGL33, V_OpenGL40, V_OpenGL41, V_OpenGL42, V_OpenGL43,
V_OpenGL44, V_OpenGL45, V_OpenGL46,
E_GL_ARB_draw_buffers_blend,
V_OpenGL30, V_OpenGL31, V_OpenGL32, V_OpenGL33, V_OpenGL40, V_OpenGL41, V_OpenGL42, V_OpenGL43, V_OpenGL44,
V_OpenGL45, V_OpenGL46, E_GL_ARB_draw_buffers_blend,
// The GLSL 1.00/1.10-era ARB trio plus NPOT textures: all core since GL 2.0 and
// trivially true for every context this backend claims (4.6). KWin 6's
// EglContext::checkSupported() gates desktop-GL compositing on exactly these four
// strings, so withholding them fails its renderer bring-up on an otherwise capable
// implementation.
E_GL_ARB_shader_objects, E_GL_ARB_vertex_shader, E_GL_ARB_fragment_shader,
E_GL_ARB_texture_non_power_of_two,
E_GL_ARB_compute_shader, E_GL_ARB_shader_storage_buffer_object, E_GL_ARB_shader_image_load_store,
E_GL_ARB_clear_buffer_object, E_GL_ARB_program_interface_query, E_GL_ARB_framebuffer_object, E_GL_EXT_framebuffer_object,
E_GL_ARB_depth_texture, E_GL_ARB_buffer_storage, E_GL_ARB_texture_storage,
E_GL_ARB_texture_storage_multisample, E_GL_ARB_clear_texture, E_GL_ARB_direct_state_access,
E_GL_ARB_multi_draw_indirect, E_GL_ARB_indirect_parameters, E_GL_ARB_shader_draw_parameters,
E_GL_ARB_gpu_shader5, E_GL_ARB_multi_bind, E_GL_ARB_shading_language_420pack,
E_GL_ARB_vertex_attrib_binding,
E_GL_ARB_texture_non_power_of_two, E_GL_ARB_compute_shader, E_GL_ARB_shader_storage_buffer_object,
E_GL_ARB_shader_image_load_store, E_GL_ARB_clear_buffer_object, E_GL_ARB_program_interface_query,
E_GL_ARB_framebuffer_object, E_GL_EXT_framebuffer_object, E_GL_ARB_depth_texture, E_GL_ARB_buffer_storage,
E_GL_ARB_texture_storage, E_GL_ARB_texture_storage_multisample, E_GL_ARB_clear_texture,
E_GL_ARB_direct_state_access, E_GL_ARB_multi_draw_indirect, E_GL_ARB_indirect_parameters,
E_GL_ARB_shader_draw_parameters, E_GL_ARB_gpu_shader5, E_GL_ARB_multi_bind,
E_GL_ARB_shading_language_420pack, E_GL_ARB_vertex_attrib_binding,
// Both are core from GL 3.2/3.3 on and implemented here for
// every advertised version, but an app targeting 3.0/3.1
// only reaches them through the extension string - the CTS
@@ -1808,8 +1811,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
void BackendObject_DirectGLES::UpdateDynamicBackendParameters() {
m_dynamicParameters.UniformBufferOffsetAlignment = m_GLESCapabilities.UniformBufferOffsetAlignment;
m_dynamicParameters.ShaderStorageBufferOffsetAlignment =
m_GLESCapabilities.ShaderStorageBufferOffsetAlignment;
m_dynamicParameters.ShaderStorageBufferOffsetAlignment = m_GLESCapabilities.ShaderStorageBufferOffsetAlignment;
m_dynamicParameters.MaxTextureMaxAnisotropy = m_GLESCapabilities.MaxTextureMaxAnisotropy;
m_dynamicParameters.AliasedLineWidthRangeMin = m_GLESCapabilities.AliasedLineWidthRangeMin;
m_dynamicParameters.AliasedLineWidthRangeMax = m_GLESCapabilities.AliasedLineWidthRangeMax;
@@ -1962,8 +1964,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_dynamicParameters.SubgroupSupportedFeatures = subgroups.features;
m_dynamicParameters.SubgroupQuadOperationsInAllStages = subgroups.quadInAllStages;
MGLOG_I("DirectGLES: GL_KHR_shader_subgroup %s (size %u, stages 0x%x, features 0x%x)",
SubgroupImplementationName(ResolveEsprytSubgroupImplementation(m_GLESCapabilities)),
subgroups.size, subgroups.stages, subgroups.features);
SubgroupImplementationName(ResolveEsprytSubgroupImplementation(m_GLESCapabilities)), subgroups.size,
subgroups.stages, subgroups.features);
}
// Follows the line above, and must: OpenGL ES has no double-precision vertex format and no
// fp64 type to consume one with, so a 64-bit vertex attribute has nowhere to land here.
@@ -1977,11 +1979,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// exercised on a healthy driver (the pinned integration lane); =0 restores the
// detected answer's declines.
m_dynamicParameters.SupportsTessellationPointSize =
m_GLESCapabilities.TessellationPointSizeSupport !=
MG_External::GLESCapabilities::PointSizeTier::None;
m_GLESCapabilities.TessellationPointSizeSupport != MG_External::GLESCapabilities::PointSizeTier::None;
m_dynamicParameters.SupportsGeometryPointSize =
m_GLESCapabilities.GeometryPointSizeSupport !=
MG_External::GLESCapabilities::PointSizeTier::None;
m_GLESCapabilities.GeometryPointSizeSupport != MG_External::GLESCapabilities::PointSizeTier::None;
switch (MG_Config::Features.PointSizeDemotion) {
case MG_Config::QuirkOverride::ForceOn:
MGLOG_I("DirectGLES: MOBILEGL_POINT_SIZE_DEMOTION=1 - treating tessellation/geometry "
@@ -15,8 +15,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Populates the same format-capability cache used by backend startup. The caller
// must keep the supplied GLES context current for the duration of this call.
void PopulateFormatCapabilities(const MG_External::GLESFunctionsTable& gl,
const MG_External::GLESCapabilities& capabilities,
FormatCapabilityCache& cache);
const MG_External::GLESCapabilities& capabilities, FormatCapabilityCache& cache);
// Clamps a requested sample count down to what the ES driver can really deliver for this
// format on this format-capability target: the probed per-format list when there is one, the
@@ -118,8 +117,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// plus the feature set in `features`. The MOBILEGL_DISABLE_TIMERQUERY escape hatch is applied
// inside.
Vector<GLExtension> BuildAdvertisedExtensions(Bool timerQueriesSupported, Bool anisotropicFilteringSupported,
Bool drawIndirectSupported,
Bool nonZeroIndirectBaseInstanceSupported,
Bool drawIndirectSupported, Bool nonZeroIndirectBaseInstanceSupported,
Bool textureViewSupported, Bool cubeMapArraySupported,
const AdvertisedFeatureSupport& features = {});
@@ -27,5 +27,5 @@ namespace MobileGL::MG_Backend::DirectGLES {
const Bool server = MG_Config::Transport != MG_Config::TransportMode::Monolith;
return {g_backendContextGeneration, server ? MG_Pipe::MGPipeApplierContextSerial() : 0, server};
}
}
} // namespace MobileGL::MG_Backend::DirectGLES
#endif
File diff suppressed because it is too large Load Diff
+10 -9
View File
@@ -75,11 +75,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
GLsizei height, GLint border);
void CopyTexSubImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y, GLsizei width,
GLsizei height);
void CopyImageSubData(const CopyImageEndpoint& src,
GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY, GLint srcZ,
const CopyImageEndpoint& dst,
GLenum dstTarget, GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ,
GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void CopyImageSubData(const CopyImageEndpoint& src, GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY,
GLint srcZ, const CopyImageEndpoint& dst, GLenum dstTarget, GLint dstLevel, GLint dstX,
GLint dstY, GLint dstZ, GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void GenerateMipmap(GLenum target);
const GLubyte* GetString(GLenum name);
void ReadPixels(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels);
@@ -291,8 +289,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
SizeT operator()(const TwinKey& key) const {
// The 64-bit mix SlotAllocator and the applier's registry already use for their own
// pair keys, so every keyed table in this tree hashes a session the same way.
Uint64 mixed = key.SessionKey ^ (key.ShareGroupKey + 0x9E3779B97F4A7C15ull +
(key.SessionKey << 6) + (key.SessionKey >> 2));
Uint64 mixed = key.SessionKey ^
(key.ShareGroupKey + 0x9E3779B97F4A7C15ull + (key.SessionKey << 6) + (key.SessionKey >> 2));
mixed ^= mixed >> 33;
mixed *= 0xFF51AFD7ED558CCDull;
mixed ^= mixed >> 33;
@@ -344,7 +342,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
CacheEntry Cache[kCacheEntries]{};
};
inline constinit thread_local NativeContextThreadState t_nativeContextThread;
inline Uint64 CurrentNativeContextSerial() { return t_nativeContextThread.BoundSerial; }
inline Uint64 CurrentNativeContextSerial() {
return t_nativeContextThread.BoundSerial;
}
class PerNativeContextBase {
public:
@@ -391,7 +391,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
template <typename F>
void ForEach(F&& visit) {
const std::lock_guard<std::mutex> lock(m_mutex);
for (auto& entry : m_bySerial) visit(entry.second);
for (auto& entry : m_bySerial)
visit(entry.second);
}
void Clear() {
const std::lock_guard<std::mutex> lock(m_mutex);
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,707 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/DirectGLESBuffers.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DirectGLESInternal.h"
namespace MobileGL::MG_Backend::DirectGLES {
namespace BufferImpl {
// See Managers.h for why these two are named functions over the applier state and
// nothing else.
Uint ResolveDrawVertexBuffersFromRecord(const MG_Pipe::MGPipeApplierState& st, DrawVertexBufferRequest* out,
Uint capacity) {
Uint count = 0;
const Uint32 begin = st.VertexBufferStart;
const Uint32 end = begin + st.VertexBufferCount;
for (Uint32 i = begin; i < end && i < MG_Pipe::kMGPipeMaxVertexAttribs; ++i) {
const MG_Pipe::MGPVertexBuffer& binding = st.VertexBuffers[i];
if (MG_Pipe::MGPipeHandleIsNull(binding.Res)) continue;
// A {slot, gen} compare, not a raw address compare: an address a successor
// object can reproduce is the identity hazard this phase exists to remove.
Bool alreadySeen = false;
for (Uint j = 0; j < count; ++j) {
if (out[j].Res == binding.Res) {
alreadySeen = true;
break;
}
}
if (alreadySeen) continue;
if (count >= capacity) {
MGLOG_E_ONCE("MGPipe: the vertex-buffer window named more distinct buffers than "
"a draw can hold (%u); the tail is dropped",
capacity);
break;
}
out[count].Res = binding.Res;
out[count].BindingIndex = binding.BindingIndex;
++count;
}
return count;
}
DrawIndexBufferRequest ResolveDrawIndexBufferFromRecord(const MG_Pipe::MGPipeApplierState& st) {
DrawIndexBufferRequest request;
request.Res = st.IndexBuffer.Res;
request.Serial = st.IndexBufferSerial;
return request;
}
// True from a draw or dispatch that ran with a storage buffer bound until the next
// barrier that orders storage-buffer writes. Every bound storage point counts as written,
// the client's own conservative set (see serverGpuWritten below).
//
// WHAT IT IS FOR: two dispatches with no barrier between them that write DIFFERENT
// components of one vector in the same storage buffer. GL leaves the two unordered, which
// allows them to run concurrently, but every write still lands - they are separate memory
// locations. A driver that stores a vector component as a read-modify-write of the whole
// vector loses one of them instead: KHR-GL43.shader_storage_buffer_object.
// advanced-switchBuffers-cs writes digest.x .. digest.w of one uvec4 from four
// back-to-back dispatches and read back a zero component in 8 runs of 20 on Adreno (0 of
// 20 with a storage barrier between the dispatches, 0 of 20 with uint[4]). The fix is the
// barrier the application did not owe: PrepareForCompute issues it when the previous
// shader work may have written a storage buffer and nothing ordered those writes since.
//
// Compute only. A dispatch that follows a dispatch is the shape the test has and the one
// a driver may overlap; draws keep their own ordering and are not taxed, they only SET
// the flag here - one store in the walk that already visits every bound point.
Bool g_shaderStorageWritesUnordered = false;
// The record arm of SyncBufferBindingPoints. Same shape, same order, same branches; the
// four frontend reads become the applier's window and a handle per entry.
//
// WHAT THE WINDOW IS: ShaderBufferCount[class] starting at ShaderBufferStart[class],
// which is the client's touched high-water mark clamped to the 84 the wire carries
// (ruling 10). It is NOT clamped on the client to the device's ceiling, so the ES clamp
// below stays exactly where it was - a client reading GL_MAX_UNIFORM_BUFFER_BINDINGS
// would be answering a driver question from the wrong side of the wire.
//
// WHOLE-VS-RANGE IS `Offset == 0 && Size == kMGPipeWholeBuffer` and obj->GetSize()
// disappears from this function: a base binding does not freeze an extent, the client
// deliberately does not resolve one, and the re-resolution happens HERE against the
// server's own storage - which is what makes a glBufferData issued between the emission
// and this apply bind the NEW extent, as GL does.
void SyncBufferBindingPointsByRecord(Uint32 shaderBufferClass, GLenum glTarget) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
const MG_Pipe::MGPipeApplierState& st = MG_Pipe::MGPipeApplier();
const Uint32 start = st.ShaderBufferStart[shaderBufferClass];
SizeT end = static_cast<SizeT>(start) + st.ShaderBufferCount[shaderBufferClass];
// ...and never past what the ES driver itself can hold, exactly as the frontend
// walk does and for the same reason recorded there.
if (glTarget == GL_UNIFORM_BUFFER && g_GLESCapabilities.MaxUniformBufferBindings > 0) {
end = std::min(end, static_cast<SizeT>(g_GLESCapabilities.MaxUniformBufferBindings));
}
for (SizeT i = start; i < end; ++i) {
const MG_Pipe::MGPBufferRange& entry = st.BoundShaderBuffers[shaderBufferClass][i];
if (MG_Pipe::MGPipeHandleIsNull(entry.Res)) {
BindBufferBaseCached(glTarget, static_cast<GLuint>(i), 0);
continue;
}
auto* backendResource = EnsureBufferResourceForHandle(nullptr, entry.Res);
if (!backendResource || backendResource->id == 0) {
MGLOG_E_ONCE("No backend buffer found for %s binding point %zu (handle {%u, %u}).",
MG_Util::ConvertGLEnumToString(glTarget).c_str(), i, entry.Res.Slot, entry.Res.Gen);
continue;
}
// P8-C: the SERVER's half of the mark MarkShaderStorageBuffersGpuWritten no longer
// makes under a transport - on this side's twin, for this side's CPU readers
// (SplitHostBytesForCpuRead). Every bound storage point, which is the client's
// own conservative set (ShaderBufferEmit.h marks every one writable); a uniform
// point is read-only by definition.
if (shaderBufferClass != MG_Pipe::kMGPipeShaderBufferClassUniform) {
backendResource->serverGpuWritten = true;
g_shaderStorageWritesUnordered = true;
}
const auto backendBufferId = backendResource->id;
if (entry.Offset == 0 && entry.Size == MG_Pipe::kMGPipeWholeBuffer) {
BindBufferBaseCached(glTarget, static_cast<GLuint>(i), backendBufferId);
} else {
// CLAMPED AGAINST THE SERVER's OWN STORAGE, which is what the frontend walk
// clamped against obj->GetSize() for: a range that outruns the storage is a
// GL_INVALID_VALUE the driver would raise on a bind the application already
// made legally against a buffer that has since shrunk.
const SizeT storage = backendResource->storageSize;
const SizeT rangeStart = std::min<SizeT>(static_cast<SizeT>(entry.Offset), storage);
const SizeT rangeEnd = std::min<SizeT>(static_cast<SizeT>(entry.Offset + entry.Size), storage);
BindBufferRangeCached(glTarget, static_cast<GLuint>(i), backendBufferId,
static_cast<GLintptr>(rangeStart),
static_cast<GLsizeiptr>(rangeEnd - rangeStart));
}
}
}
void SyncAtomicCounterBuffers(const Vector<Int>& glBindings, Int esslBindingTop) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
// P5e (sb, §5.6): THE RECORD ARM. `glBindings` stays the backend program's own list
// - it is server-owned since the link and names which GL counter bindings THIS
// program declares - and only the points themselves move sides. The bound is the
// applier's window instead of GetBufferBindingPointCount's sticky forward, which is
// the row this site was the Espryt reader of.
const MG_Pipe::MGPipeApplierState& st = MG_Pipe::MGPipeApplier();
const Uint32 windowStart = st.ShaderBufferStart[MG_Pipe::kMGPipeShaderBufferClassAtomicCounter];
const SizeT windowEnd =
static_cast<SizeT>(windowStart) + st.ShaderBufferCount[MG_Pipe::kMGPipeShaderBufferClassAtomicCounter];
for (const Int glBinding : glBindings) {
if (glBinding < 0) continue;
const Int esslBinding = esslBindingTop - glBinding;
// Already diagnosed once when the block was transpiled; nothing was bound to
// it there either, so there is nothing to unbind here.
if (esslBinding < 0) continue;
// OUTSIDE THE WINDOW IS "NOTHING BOUND", which is what the frontend array's
// default says too - so it takes the unbind arm rather than being skipped.
// Skipping would leave whatever the driver last had at that ESSL point,
// which is the hole the frontend walk's `continue` never had because its
// bound was the whole 84-point array.
const MG_Pipe::MGPBufferRange* entry = nullptr;
if (static_cast<SizeT>(glBinding) >= windowStart && static_cast<SizeT>(glBinding) < windowEnd) {
entry = &st.BoundShaderBuffers[MG_Pipe::kMGPipeShaderBufferClassAtomicCounter]
[static_cast<SizeT>(glBinding)];
}
if (entry == nullptr || MG_Pipe::MGPipeHandleIsNull(entry->Res)) {
BindBufferBaseCached(GL_SHADER_STORAGE_BUFFER, static_cast<Uint>(esslBinding), 0);
continue;
}
auto* backendResource = EnsureBufferResourceForHandle(nullptr, entry->Res);
if (!backendResource || backendResource->id == 0) {
MGLOG_E_ONCE("No backend buffer found for atomic counter binding point %d "
"(handle {%u, %u}).",
glBinding, entry->Res.Slot, entry->Res.Gen);
continue;
}
if (entry->Offset == 0 && entry->Size == MG_Pipe::kMGPipeWholeBuffer) {
BindBufferBaseCached(GL_SHADER_STORAGE_BUFFER, static_cast<Uint>(esslBinding), backendResource->id);
} else {
const SizeT storage = backendResource->storageSize;
const SizeT rangeStart = std::min<SizeT>(static_cast<SizeT>(entry->Offset), storage);
const SizeT rangeEnd = std::min<SizeT>(static_cast<SizeT>(entry->Offset + entry->Size), storage);
BindBufferRangeCached(GL_SHADER_STORAGE_BUFFER, static_cast<Uint>(esslBinding), backendResource->id,
static_cast<GLintptr>(rangeStart),
static_cast<GLsizeiptr>(rangeEnd - rangeStart));
}
// AND NO MarkBufferGpuWritten HERE. It was the second of the three backend
// GPU-write sites and it reached into a client object; the client's
// MarkAtomicCounterBindings (GpuWritePending.cpp) already marks the same set
// from MarkGpuWritesForDraw, WIDER on purpose - it marks every touched
// counter point rather than only the ones this program declares, which is
// the safe direction for an over-approximate set.
//
// P8-C: the server's own twin is marked instead, for the server's own CPU
// readers (SplitHostBytesForCpuRead) - a counter buffer later bound as a
// parameter or element buffer is read on this side.
backendResource->serverGpuWritten = true;
}
return;
}
} // namespace BufferImpl
namespace XfbImpl {
// One scratch ES buffer serves every scattered capture: only one span can be
// recording at a time (the driver would reject a second Begin), so its contents
// are consumed by the End that follows.
GLuint g_scatterBufferId = 0;
SizeT g_scatterBufferSize = 0;
UnorderedMap<GLuint, XfbObjectState> g_xfbObjects;
GLuint g_currentXfbName = 0;
// Cached address of g_xfbObjects[g_currentXfbName]: PrepareForDraw consults
// CurrentXfb on EVERY draw (StartPendingTransformFeedback) and the map
// lookup was pure per-draw overhead for the overwhelmingly common no-capture
// case. Open addressing keeps values in the bucket array, so ANY insert can
// rehash and move them - and erase moves them too, by shifting the rest of the
// probe cluster into the hole, which reaches entries other than the erased one.
// Every site that mutates the map or rebinds the current name resets this to
// null instead of reasoning about stability, and CurrentXfb re-resolves lazily.
XfbObjectState* g_currentXfbState = nullptr;
// P13 W6: one role - the record arm's (the frontend arm's GL-name keyed state went).
Array<RoleXfbState, 1> g_roleXfbState;
namespace {
RoleXfbState& ActiveXfbState() {
auto& state = g_roleXfbState[0];
if (state.NativeGeneration != g_backendContextGeneration) {
state = {};
state.NativeGeneration = g_backendContextGeneration;
}
return state;
}
} // namespace
XfbObjectState& CurrentXfb() {
auto& state = ActiveXfbState();
const Uint64 lifetime = MG_Pipe::MGPipeApplier().BoundStreamOutputLifetimeId;
if (state.Current != lifetime || state.Cached == nullptr || state.NeedsBind)
BindTransformFeedback(0); // identity is the applier's lifetime, not this argument
auto& g_currentXfbState = state.Cached;
auto& g_xfbObjects = state.Objects;
const auto g_currentXfbName = state.Current;
if (g_currentXfbState == nullptr) {
g_currentXfbState = &g_xfbObjects[g_currentXfbName];
}
return *g_currentXfbState;
}
namespace {
Bool AreTransformFeedbackObjectsSupported() {
return g_GLESFuncs.glGenTransformFeedbacks != nullptr &&
g_GLESFuncs.glBindTransformFeedback != nullptr &&
g_GLESFuncs.glDeleteTransformFeedbacks != nullptr &&
g_GLESFuncs.glPauseTransformFeedback != nullptr &&
g_GLESFuncs.glResumeTransformFeedback != nullptr;
}
// How wide one OnBufferWriteback may be for a capture range of `rangeBytes`, which is
// the whole range when the transport imposes no limit (monolith, where the callback is
// a direct call with no ring under it) or when the range already fits one.
//
// BOTH XFB WRITEBACK PRODUCERS ASK THE SAME FUNCTION. They post from different sources
// - one from a live glMapBufferRange mapping, one from a staged Vector<Uint8> - but
// they post the same KIND of record into the same ring, and a second spelling of the
// width is how one of them ends up being the producer that still kills the server.
SizeT XfbWritebackSliceStep(SizeT rangeBytes) {
const Uint64 slice = MG_Pipe::MGPipeBufferWritebackSliceBytes();
if (slice == 0 || slice >= static_cast<Uint64>(rangeBytes)) return rangeBytes;
return static_cast<SizeT>(slice);
}
// Mirrors one capture span's results into the frontend CPU shadows. The GPU wrote
// the capture buffers behind the frontend's back, so the shadows that back
// MapBuffer/GetBufferSubData still hold the pre-draw bytes. Buffers whose storage
// the backend already owns (coherent persistent map) need nothing: reads resolve
// against that storage directly.
void ReadbackCapturedRanges(Vector<XfbCaptureTarget>& targets) {
if (g_GLESFuncs.glMapBufferRange == nullptr || g_GLESFuncs.glUnmapBuffer == nullptr) {
MGLOG_E_ONCE("EndTransformFeedback: the ES driver exposes no glMapBufferRange/glUnmapBuffer, so "
"captured data can never reach the application's buffers");
}
if (targets.empty()) {
// The span closed with nothing to mirror back. Either the deferred Begin
// never ran (a span with no draw - legal) or it ran and found no bound
// capture buffer, which is not.
MGLOG_D("EndTransformFeedback: capture span closed with no recorded targets");
}
if (g_GLESFuncs.glMapBufferRange != nullptr && g_GLESFuncs.glUnmapBuffer != nullptr) {
for (const auto& target : targets) {
// P5c: under an active transport the frontend object is client
// memory (rule E), so the persistence question is the server
// resource's and the captured bytes go back as a writeback EVENT -
// WritebackFromBackend from the apply thread is the R1/R2 shape.
const MG_Pipe::MGPipeHandle res = target.handle;
auto* resource = BufferImpl::FindBufferResourceForHandle(res);
if (resource == nullptr) continue;
if (resource->persistentMapped) {
// P8-C: nothing to copy - the capture reaches a coherent map -
// but this side's CPU readers must finish before reading it.
resource->serverGpuWritten = true;
continue;
}
const SizeT size = target.end - target.start;
BufferImpl::BindBufferId(BufferImpl::TempBufferTarget, target.backendId);
void* mapped = g_GLESFuncs.glMapBufferRange(BufferImpl::TempBufferTarget,
static_cast<GLintptr>(target.start),
static_cast<GLsizeiptr>(size), GL_MAP_READ_BIT);
if (mapped == nullptr) {
MGLOG_E_ONCE("EndTransformFeedback: failed to map backend buffer %u [%zu, %zu) for "
"capture readback (ES error %s); the captured data will NOT be visible to "
"the application",
target.backendId, target.start, target.end,
MG_Util::ConvertGLEnumToString(TakeXfbDriverError()).c_str());
continue;
}
if (MG_Pipe::gMGPipeCallbacks.OnBufferWriteback != nullptr) {
// ONE EVENT PER SLICE, NOT ONE PER CAPTURE. The bytes travel
// INLINE in a SEG_EVENT record and the ring refuses any record
// above half its capacity outright, so a whole-range post of a
// capture wider than ~128 KiB was Fatal{EventRingOverflow} - the
// SERVER dying on a large buffer. The slicing shape is
// BufferObject.cpp's one ring over: in-order delivery makes the
// last slice's landing imply every earlier one, so the client
// needs nothing new to reassemble them.
//
// INSIDE THE MAPPING, deliberately: the map is the expensive part
// and the slices are reads of one live pointer, so this unmaps
// once below however many events it posts.
const SizeT step = XfbWritebackSliceStep(size);
for (SizeT off = 0; off < size; off += step) {
const SizeT bytes = std::min(step, size - off);
MG_Pipe::gMGPipeCallbacks.OnBufferWriteback(
res, target.start + off,
MG_Pipe::MGPBlobRef{reinterpret_cast<Uint64>(static_cast<Uint8*>(mapped) + off),
static_cast<Uint64>(bytes), MG_Pipe::kMGHostSpanSegNone, 0});
}
} else {
MGLOG_E_ONCE("EndTransformFeedback: no reverse channel is installed, so the "
"captured bytes of buffer {%u,%u} cannot reach the client shadow",
res.Slot, res.Gen);
}
// P8-C: the server's own staged copy takes the same bytes, from the
// same mapping, so this side's CPU readers see the capture too.
BufferImpl::StageServerWrittenRange(*resource, res, mapped, target.start, size);
g_GLESFuncs.glUnmapBuffer(BufferImpl::TempBufferTarget);
// Ops_H_Readback's order: writeback, unmap, THEN the serial
// stamp, so the next draw does not re-upload the server's stale
// staged bytes over the capture that just landed.
resource->syncedChangeSerial = BufferImpl::ResourceSerialForHandle(res);
BufferImpl::BumpBufferMutationEpoch();
continue;
}
}
targets.clear();
}
// Distributes the gap-free records the driver captured into the application's
// buffers at the offsets the GL layout asks for. Only the bytes a varying actually
// occupies are written, so the holes gl_SkipComponents asks for keep whatever the
// application had put there - which is the whole point of the feature.
void ScatterCapturedRecords(XfbObjectState& xfb) {
auto& g_scatterBufferId = xfb.serverScatterBuffer;
const XfbProgramSource program{xfb.scatterProgram, xfb.scatterArchive};
if (!program || xfb.targets.empty()) return;
if (g_GLESFuncs.glMapBufferRange == nullptr || g_GLESFuncs.glUnmapBuffer == nullptr) return;
const SizeT packedStride = program->GetTransformFeedbackPackedStride();
const SizeT modelledVertices =
static_cast<SizeT>(MG_Pipe::gPipeInputs->GetTransformFeedbackCapturedVertices());
const SizeT vertices = std::min<SizeT>(modelledVertices, xfb.scatterCapacityVertices);
if (packedStride == 0 || vertices == 0) {
// The scatter path redirected the DRIVER's capture into the scratch buffer,
// so bailing here leaves the application's buffers holding their pre-draw
// bytes - a total data loss, not a no-op. The vertex count is the CPU model
// (AccountTransformFeedbackPrimitives), which is 0 for any draw mode
// CountPrimitivesForDraw does not know and for the instanced/indirect entry
// points that never call it.
MGLOG_E_ONCE("EndTransformFeedback: scattered capture discarded - packedStride=%zu, "
"CPU-modelled captured vertices=%zu, scratch capacity=%zu. The capture buffers keep "
"their pre-draw contents.",
packedStride, modelledVertices, xfb.scatterCapacityVertices);
return;
}
BufferImpl::BindBufferId(BufferImpl::TempBufferTarget, g_scatterBufferId);
const void* packed = g_GLESFuncs.glMapBufferRange(
BufferImpl::TempBufferTarget, 0, static_cast<GLsizeiptr>(packedStride * vertices), GL_MAP_READ_BIT);
if (packed == nullptr) {
MGLOG_E_ONCE("EndTransformFeedback: failed to map the scatter capture buffer");
return;
}
// One staged copy per destination buffer: start from what the application had
// (the shadow is authoritative - uploads go shadow -> ES, and previous captures
// were mirrored back into it), patch the captured varyings in, then push the
// whole range down once.
for (SizeT targetIndex = 0; targetIndex < xfb.targets.size(); ++targetIndex) {
const auto& target = xfb.targets[targetIndex];
// By BUFFER index, not by position in the compacted list - see XfbCaptureTarget.
const SizeT stride = program->GetTransformFeedbackStride(static_cast<Uint32>(target.bufferIndex));
if (stride == 0) continue;
const SizeT rangeBytes = target.end - target.start;
Vector<Uint8> staged(rangeBytes);
// P5c: under an active transport BufferObject::MappedData on the apply
// thread is Fatal{RoleViolation, "buffer-legacy-arm"} (CONTRACT-P5C §3.8) -
// the pre-capture bytes are the SERVER's staged shadow, and the reconciled
// range goes back as a writeback EVENT instead of a WritebackFromBackend
// poke into client memory.
MG_Pipe::MGPipeHandle splitRes = MG_Pipe::kMGPipeNullHandle;
BufferImpl::GLESBufferResource* splitResource = nullptr;
splitRes = target.handle;
splitResource = BufferImpl::FindBufferResourceForHandle(splitRes);
if (splitResource == nullptr) {
// THE ONLY DISCARD LEFT, and it is the impossible one: this target
// was recorded by BindBufferRange/Base against a handle the applier
// had already minted a twin for, so no twin here means the slot
// table and the capture list disagree about one resource. There is
// no backendId to upload into either, so there is nothing to do but
// name it.
MGLOG_E_ONCE("EndTransformFeedback: a scattered capture target (handle {%u,%u}) has "
"no server twin at all; its capture is discarded",
splitRes.Slot, splitRes.Gen);
continue;
}
// P8-C: made current first - a store a shader wrote here is refreshed.
const Uint8* hostBytes =
BufferImpl::SplitHostBytesForCpuRead(*splitResource, splitRes, "xfb_scatter_pre_capture");
// A NULL SHADOW IS NOT A LOST CAPTURE. It is the ORPHANED store
// (glBufferData(size, NULL) with no resource_subdata since), which
// M-3 declares legal: every byte the application has not staged is
// UNDEFINED by its own declaration, and `staged` is already
// value-initialised to zero - which is byte for byte what the monolith
// arm scatters over when it reads a fresh MappedData(). Skipping the
// Memcpy is therefore the WHOLE of the difference; skipping the TARGET
// threw the varyings away and left the application's buffer holding its
// pre-draw bytes under GL_NO_ERROR, the silent-loss class this file's
// preamble exists to close. The streaming idiom that hits it - orphan,
// partial glBufferSubData, capture into the rest - is ordinary.
if (hostBytes != nullptr) {
// Managers.cpp's pool-reuse ladder guards its whole-store
// MGL_SERVER_STAGED_REQUIRE with exactly this predicate and for
// exactly this reason: for a store whose content the application
// SUPPLIED a coverage gap is a missing record and zero-filling past
// it is silent data loss (Fatal by name); for an orphaned one the
// gap is its own undefined content and the read is legal.
if (BufferImpl::ResourceContentIsDeclared(splitRes)) {
BufferImpl::RequireStagedCoverage(*splitResource, hostBytes, target.start, target.end,
"xfb_scatter_pre_capture");
}
Memcpy(staged.data(), hostBytes + target.start, rangeBytes);
}
for (const auto& varying : program->GetTransformFeedbackVaryings()) {
if (varying.bufferIndex != target.bufferIndex) continue;
for (SizeT v = 0; v < vertices; ++v) {
const SizeT dstOffset = v * stride + varying.offsetBytes;
if (dstOffset + varying.byteSize > rangeBytes) break;
Memcpy(staged.data() + dstOffset,
static_cast<const Uint8*>(packed) + v * packedStride + varying.packedOffsetBytes,
varying.byteSize);
}
}
if (MG_Pipe::gMGPipeCallbacks.OnBufferWriteback != nullptr) {
// Sliced for the reason the readback path above is sliced: the bytes
// are INLINE in a SEG_EVENT record and the ring refuses a record wider
// than half its capacity outright, so one whole-range post of a
// capture over ~128 KiB was Fatal{EventRingOverflow} and the server
// died. Only the EVENT is cut; the glBufferSubData below stays one
// call over the whole `staged` vector, because the ES store has no
// ring and cutting it would change how many backend calls one
// application call makes.
const SizeT step = XfbWritebackSliceStep(rangeBytes);
for (SizeT off = 0; off < rangeBytes; off += step) {
const SizeT bytes = std::min(step, rangeBytes - off);
MG_Pipe::gMGPipeCallbacks.OnBufferWriteback(
splitRes, target.start + off,
MG_Pipe::MGPBlobRef{reinterpret_cast<Uint64>(staged.data() + off),
static_cast<Uint64>(bytes), MG_Pipe::kMGHostSpanSegNone, 0});
}
} else {
MGLOG_E_ONCE("EndTransformFeedback: no reverse channel is installed, so the "
"scattered capture of buffer {%u,%u} cannot reach the client shadow",
splitRes.Slot, splitRes.Gen);
}
// P8-C: the staged copy takes the composed range too, so this side's CPU
// readers see the capture (before P8 it kept the pre-capture bytes).
BufferImpl::StageServerWrittenRange(*splitResource, splitRes, staged.data(), target.start,
rangeBytes);
// Ops_H_Readback's stamp, for its reason: the GL store now holds the
// capture, and the next draw must not re-upload an older range over it.
splitResource->syncedChangeSerial = BufferImpl::ResourceSerialForHandle(splitRes);
// Serial bumped with no backend op (see ReadbackCapturedRanges).
BufferImpl::BumpBufferMutationEpoch();
if (g_GLESFuncs.glBufferSubData != nullptr) {
BufferImpl::BindBufferId(BufferImpl::TempBufferTarget, target.backendId);
g_GLESFuncs.glBufferSubData(BufferImpl::TempBufferTarget, static_cast<GLintptr>(target.start),
static_cast<GLsizeiptr>(rangeBytes), staged.data());
}
BufferImpl::BindBufferId(BufferImpl::TempBufferTarget, g_scatterBufferId);
}
g_GLESFuncs.glUnmapBuffer(BufferImpl::TempBufferTarget);
xfb.targets.clear();
}
} // namespace
Bool AreTransformFeedbacksSupported() {
return g_GLESFuncs.glBeginTransformFeedback != nullptr && g_GLESFuncs.glEndTransformFeedback != nullptr &&
g_GLESFuncs.glTransformFeedbackVaryings != nullptr;
}
Bool IsCaptureSpanOpen() {
const auto& xfb = CurrentXfb();
return (xfb.pending || xfb.started) && !xfb.paused;
}
void BeginTransformFeedback(GLenum primitiveMode) {
if (!AreTransformFeedbacksSupported()) return;
auto& xfb = CurrentXfb();
xfb.primitiveMode = primitiveMode;
xfb.pending = true;
xfb.started = false;
xfb.paused = false;
xfb.targets.clear();
}
void EndTransformFeedback() {
auto& xfb = CurrentXfb();
const Bool wasPending = xfb.pending;
xfb.pending = false;
xfb.paused = false;
if (!xfb.started) {
// A span that never drew is legal and captures nothing by definition; one that
// is STILL pending here drew nothing the backend saw, which for a span the
// application expected data from is the whole bug in one line.
MGLOG_D("EndTransformFeedback: closing a span the driver never opened (pending=%d)",
wasPending ? 1 : 0);
return;
}
xfb.started = false;
g_GLESFuncs.glEndTransformFeedback();
if (const GLenum endError = TakeXfbDriverError(); endError != GL_NO_ERROR) {
MGLOG_E_ONCE("EndTransformFeedback: the ES driver rejected glEndTransformFeedback with %s - the "
"driver's capture state and MobileGL's have diverged",
MG_Util::ConvertGLEnumToString(endError).c_str());
}
if (xfb.scattered) {
ScatterCapturedRecords(xfb);
xfb.scattered = false;
xfb.scatterProgram.reset();
xfb.scatterArchive.reset();
} else {
ReadbackCapturedRanges(xfb.targets);
}
}
void PauseTransformFeedback() {
auto& xfb = CurrentXfb();
xfb.paused = true;
// A span the driver never opened (paused before the first draw) has nothing to
// pause; the flag above is what holds the deferred Begin back until the resume.
if (!xfb.started || g_GLESFuncs.glPauseTransformFeedback == nullptr) return;
g_GLESFuncs.glPauseTransformFeedback();
}
void ResumeTransformFeedback() {
auto& xfb = CurrentXfb();
xfb.paused = false;
if (!xfb.started || g_GLESFuncs.glResumeTransformFeedback == nullptr) return;
g_GLESFuncs.glResumeTransformFeedback();
}
void BindTransformFeedback(GLuint name) {
auto& roleState = ActiveXfbState();
auto& g_currentXfbState = roleState.Cached;
auto& g_xfbObjects = roleState.Objects;
auto& g_currentXfbName = roleState.Current;
const Uint64 key = MG_Pipe::MGPipeApplier().BoundStreamOutputLifetimeId;
(void)name;
roleState.NeedsBind = false;
g_currentXfbState = nullptr; // name changes; operator[] below may also rehash
// The capture buffer bindings are the OBJECT's, not the context's: the bind below
// swaps all of them for whatever the target object holds, which the redundant-bind
// shadow has never seen.
BufferImpl::InvalidateTransformFeedbackBindingShadows();
if (!AreTransformFeedbackObjectsSupported()) {
// Without driver objects there is only the default span; keep the frontend
// name so the bookkeeping below stays consistent.
g_currentXfbName = key;
return;
}
auto& xfb = g_xfbObjects[key];
// Even each virtual context's name-0 object gets its own native object.
if (xfb.esId == 0) {
g_GLESFuncs.glGenTransformFeedbacks(1, &xfb.esId);
}
g_GLESFuncs.glBindTransformFeedback(GL_TRANSFORM_FEEDBACK, xfb.esId);
g_currentXfbName = key;
}
void DeleteTransformFeedback(GLuint name) {
auto& roleState = ActiveXfbState();
auto& g_currentXfbState = roleState.Cached;
auto& g_xfbObjects = roleState.Objects;
auto& g_currentXfbName = roleState.Current;
const Uint64 key = MG_Pipe::MGPipeApplier().VerbDeleteStreamOutputLifetimeId;
(void)name;
// A generated but never-bound name has no native object or lifetime.
if (key == 0) return;
const auto it = g_xfbObjects.find(key);
if (it == g_xfbObjects.end()) return;
if (it->second.esId != 0 && g_GLESFuncs.glDeleteTransformFeedbacks != nullptr) {
g_GLESFuncs.glDeleteTransformFeedbacks(1, &it->second.esId);
}
g_currentXfbState = nullptr; // erase shifts the probe cluster, moving other entries
g_xfbObjects.erase(it);
// The frontend reverts to the default object when the bound one is deleted.
if (g_currentXfbName == key) {
// The following BindStreamOutput names this context's actual
// default lifetime. Never create a native object for key 0.
g_currentXfbName = 0;
roleState.NeedsBind = true;
BufferImpl::InvalidateTransformFeedbackBindingShadows();
return;
}
}
// The ES context went away (or is being torn down): the spans, their buffer ids, the
// driver objects and the frontend objects they pinned all belonged to it.
void OnBackendContextDestroyed() {
for (auto& state : g_roleXfbState)
state = {};
g_currentXfbState = nullptr;
g_xfbObjects.clear();
g_currentXfbName = 0;
g_scatterBufferId = 0;
g_scatterBufferSize = 0;
BufferImpl::InvalidateTransformFeedbackBindingShadows();
}
} // namespace XfbImpl
namespace VertexArrayImpl {
// THE CLIENT-MEMORY VERTEX ARRAY UPLOAD, and the one read of this family that P5e does
// NOT retire - it MOVES the decision instead, which is what ID-82 / ruling 2 settles.
//
// The bytes of a client-memory array exist only in the application's own memory: the
// emitter publishes Res == kMGPipeNullHandle for such an attribute by design
// (VertexInputEmit.h) and this upload is where the server used to dereference
// attrib.Offset as a raw client pointer (Managers.cpp's
// SyncClientSideAttributesForDrawArrays, kimi audit row 14). Under run-ahead those bytes
// are movable and the read is torn by construction, so the CLIENT refuses the draw by
// name before it is ever emitted (EmitTables.cpp, Fatal{UnmigratedVerb,
// "DrawArrays+CLIENT_ARRAYS"}); staging them as a record tail is P8's.
//
// What is left here is therefore reachable in exactly two situations, and both are safe:
// * monolith / the push-monolith build - one process, one thread, no wire;
// * a LOCKSTEP split session - the client is parked in WaitForApplied for
// this very record, so its memory is stable
// (rule F is scoped to unbarriered records,
// ruling 4). Escalation (ii) of §2.1 pins the
// same thing from the other side: a draw
// carrying kDrawClientArrays is BARRIERED.
// Making it monolith-only instead would silently drop the attribute upload of a
// lockstep split session, which is a wrong picture today - before ra has landed
// anything - so the refusal is the client's to raise, not this arm's to assume.
//
// The frontend VAO is taken ONLY when the applier's own vertex-buffer window says a
// client-sourced attribute exists, which is what retires the per-draw
// GetBoundVertexArray row of the ordinary (VBO-backed) DrawArrays - the strict lane's
// `GetBoundVertexArray@DrawArrays` marker - without touching this path's behaviour.
// `static` on purpose (G1): the pull build must not gain an exported name for a
// function that only re-homes two identical inline blocks the two DrawArrays entry
// points used to carry. The same spelling governs the three below, and for a second
// reason on top of the symbol count: a Debug build drops CXX_VISIBILITY_PRESET to
// `default` (CMakeLists.txt:707-719), so an external-linkage helper here would be a
// DYNAMIC export in exactly the configuration a symbol gate cannot see it in.
//
// ALL FLAVORS, NOT JUST THE SPLIT ARM. These three were inside
// `#if MOBILEGL_BUILD_DISAGGREGATED` for one round and that was a misreading of their
// own gate: the body's first statement returns early when `MG_Config::Transport !=
// Monolith`, i.e. they exist FOR the monolith arm - and the monolith arm is what the
// pull/verify/push builds run. Their callers sit in entry points that compile in every
// flavor (the indexed family, both indirect executors), so a guard here breaks the
// pull build outright rather than merely omitting a split-arm path. In those flavors
// the guard's own test folds to a compile-time false (Config.h:566) and the body is
// the transport-free upload it always was.
//
// ---- THE MONOLITH ARM'S CLIENT-MEMORY SNAPSHOT, PER DRAW -----------------------------
//
// A client-memory attribute is uploaded HERE because its bytes have no store on this side
// to bind. SyncClientSideVertexArraysForDrawArrays below does that for the non-indexed
// family, whose fetched elements are exactly the contiguous (first, count) run. Every
// OTHER draw shape fetches by index and by instance, so its elements have to come from
// MGPipeClientFetchPlan - the same answer the wire arm's owned snapshot is built from
// (MG_Impl/Pipe/OwnedDrawInputs.h, which is why the two arms agree by construction).
//
// WHAT WAS MISSING BEFORE THIS: an indexed draw over a client array uploaded nothing at
// all. The ES context kept whatever the last VBO-backed attribute had left for that
// index - normally the array DISABLED, i.e. every vertex reading the generic current
// value - so the draw painted nothing, with no GL error anywhere to say why.
struct ClientSnapshotSources {
const SharedPtr<MG_State::GLState::BufferObject>* ElementBuffer = nullptr;
};
} // namespace VertexArrayImpl
} // namespace MobileGL::MG_Backend::DirectGLES
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,893 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/DirectGLESSharedImage.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DirectGLESInternal.h"
namespace MobileGL::MG_Backend::DirectGLES {
#if MOBILEGL_BUILD_DISAGGREGATED
// ---- SHARED IMAGES (DirectGLES.h; docs/archive/notes/anland/plan-ahb-dmabuf.md) ------
namespace SharedImageImpl {
#if defined(__ANDROID__)
namespace {
namespace SI = MG_Remote::Server::SharedImages;
// EGL_ANDROID_image_native_buffer's target and EGL_KHR_image_base's attribute, spelled
// here so no extension header is needed.
constexpr EGLenum kEglNativeBufferAndroid = 0x3140;
constexpr EGLint kEglImagePreservedKhr = 0x30D2;
using GetNativeClientBufferFn = EGLClientBuffer (*)(const AHardwareBuffer*);
using CreateImageKhrFn = void* (*)(EGLDisplay, EGLContext, EGLenum, EGLClientBuffer, const EGLint*);
using DestroyImageKhrFn = EGLBoolean (*)(EGLDisplay, void*);
using ImageTargetFn = void (*)(GLenum, void*);
// EGL_KHR_fence_sync / EGL_KHR_wait_sync / EGL_ANDROID_native_fence_sync, likewise.
constexpr EGLenum kEglSyncNativeFenceAndroid = 0x3144;
constexpr EGLint kEglSyncNativeFenceFdAndroid = 0x3145;
constexpr EGLint kEglConditionSatisfiedKhr = 0x30F6;
using CreateSyncKhrFn = void* (*)(EGLDisplay, EGLenum, const EGLint*);
using DestroySyncKhrFn = EGLBoolean (*)(EGLDisplay, void*);
using WaitSyncKhrFn = EGLint (*)(EGLDisplay, void*, EGLint);
using ClientWaitSyncKhrFn = EGLint (*)(EGLDisplay, void*, EGLint, Uint64);
using DupNativeFenceFdFn = EGLint (*)(EGLDisplay, void*);
struct EntryPoints {
Bool Resolved = false;
Bool Usable = false;
String Why;
GetNativeClientBufferFn GetNativeClientBuffer = nullptr;
CreateImageKhrFn CreateImage = nullptr;
DestroyImageKhrFn DestroyImage = nullptr;
ImageTargetFn TargetTexture2D = nullptr;
ImageTargetFn TargetRenderbuffer = nullptr;
// Fences as sync_files: what lets one session's GPU wait for another's without a
// CPU in between. Without them every wait below is a CPU wait.
Bool FencesUsable = false;
CreateSyncKhrFn CreateSync = nullptr;
DestroySyncKhrFn DestroySync = nullptr;
WaitSyncKhrFn WaitSync = nullptr;
ClientWaitSyncKhrFn ClientWaitSync = nullptr;
DupNativeFenceFdFn DupNativeFenceFd = nullptr;
// YUV images: sampled as external textures (GL_OES_EGL_image_external_essl3), and
// through GL_EXT_YUV_target's raw Y'CbCr sampler where the driver has it, so the
// conversion is the import's own (its colour hints) rather than the driver's guess.
Bool ExternalEssl3 = false;
Bool YuvTarget = false;
};
// None of these is in the loader's tables, and eglGetProcAddress hands back live-looking
// stubs for entry points a driver lacks, so the extension strings decide first. Resolved
// once, from the first call that has a context current to read the strings from; the
// answer is never modified afterwards, so it is read without the lock.
const EntryPoints& ResolvedEntryPoints() {
static auto* mutex = new std::mutex();
static auto* entry = new EntryPoints();
const std::lock_guard<std::mutex> lock(*mutex);
if (entry->Resolved) return *entry;
if (!g_EGLFuncs.eglGetProcAddress || !g_EGLFuncs.eglQueryString || !g_GLESFuncs.glGetString) {
entry->Why = "the loader has no eglGetProcAddress/eglQueryString/glGetString";
return *entry;
}
const char* egl =
g_Display != EGL_NO_DISPLAY ? g_EGLFuncs.eglQueryString(g_Display, EGL_EXTENSIONS) : nullptr;
const char* gl = reinterpret_cast<const char*>(g_GLESFuncs.glGetString(GL_EXTENSIONS));
if (egl == nullptr || gl == nullptr) {
entry->Why = "no current display and context to read the extension strings from";
return *entry; // asked again on the next call
}
entry->Resolved = true;
if (HasExtensionWord(egl, "EGL_KHR_fence_sync") && HasExtensionWord(egl, "EGL_KHR_wait_sync") &&
HasExtensionWord(egl, "EGL_ANDROID_native_fence_sync")) {
entry->CreateSync =
reinterpret_cast<CreateSyncKhrFn>(g_EGLFuncs.eglGetProcAddress("eglCreateSyncKHR"));
entry->DestroySync =
reinterpret_cast<DestroySyncKhrFn>(g_EGLFuncs.eglGetProcAddress("eglDestroySyncKHR"));
entry->WaitSync = reinterpret_cast<WaitSyncKhrFn>(g_EGLFuncs.eglGetProcAddress("eglWaitSyncKHR"));
entry->ClientWaitSync =
reinterpret_cast<ClientWaitSyncKhrFn>(g_EGLFuncs.eglGetProcAddress("eglClientWaitSyncKHR"));
entry->DupNativeFenceFd = reinterpret_cast<DupNativeFenceFdFn>(
g_EGLFuncs.eglGetProcAddress("eglDupNativeFenceFDANDROID"));
entry->FencesUsable = entry->CreateSync && entry->DestroySync && entry->WaitSync &&
entry->ClientWaitSync && entry->DupNativeFenceFd;
}
if (!entry->FencesUsable) {
MGLOG_I("Shared images: no EGL native fence sync / wait sync; sessions order their access "
"to an image with CPU waits");
}
if (!HasExtensionWord(egl, "EGL_KHR_image_base") ||
!HasExtensionWord(egl, "EGL_ANDROID_image_native_buffer") ||
!HasExtensionWord(egl, "EGL_ANDROID_get_native_client_buffer")) {
entry->Why = "EGL_KHR_image_base / EGL_ANDROID_image_native_buffer / "
"EGL_ANDROID_get_native_client_buffer is not advertised";
return *entry;
}
if (!HasExtensionWord(gl, "GL_OES_EGL_image")) {
entry->Why = "GL_OES_EGL_image is not advertised";
return *entry;
}
entry->ExternalEssl3 = HasExtensionWord(gl, "GL_OES_EGL_image_external_essl3");
entry->YuvTarget = HasExtensionWord(gl, "GL_EXT_YUV_target");
entry->GetNativeClientBuffer = reinterpret_cast<GetNativeClientBufferFn>(
g_EGLFuncs.eglGetProcAddress("eglGetNativeClientBufferANDROID"));
entry->CreateImage =
reinterpret_cast<CreateImageKhrFn>(g_EGLFuncs.eglGetProcAddress("eglCreateImageKHR"));
entry->DestroyImage =
reinterpret_cast<DestroyImageKhrFn>(g_EGLFuncs.eglGetProcAddress("eglDestroyImageKHR"));
entry->TargetTexture2D =
reinterpret_cast<ImageTargetFn>(g_EGLFuncs.eglGetProcAddress("glEGLImageTargetTexture2DOES"));
entry->TargetRenderbuffer = reinterpret_cast<ImageTargetFn>(
g_EGLFuncs.eglGetProcAddress("glEGLImageTargetRenderbufferStorageOES"));
if (!entry->GetNativeClientBuffer || !entry->CreateImage || !entry->DestroyImage ||
!entry->TargetTexture2D || !entry->TargetRenderbuffer) {
entry->Why = "eglGetProcAddress gave no address for an advertised EGLImage entry point";
return *entry;
}
entry->Usable = true;
return *entry;
}
void DrainErrors() {
for (Uint guard = 0; guard < 16 && g_GLESFuncs.glGetError() != GL_NO_ERROR; ++guard) {}
}
} // namespace
#endif
struct EglImage {
#if defined(__ANDROID__)
SI::ImageRef Image;
void* Handle = nullptr;
EGLDisplay Display = EGL_NO_DISPLAY;
Uint64 DisplayTerminations = 0;
DestroyImageKhrFn Destroy = nullptr;
// AcquireForSampling's memory, touched only by the session whose twin holds this: the
// write generation last waited for and the native context the wait was queued in (a
// GPU wait orders that context alone), and the read frame the image was last noted in.
Uint64 WaitedGeneration = 0;
Uint64 WaitedContext = 0;
Uint64 NotedFrame = 0;
// A YUV image (ConvertYuvForSampling): an external texture over the EGLImage, the source
// of the conversion into the texture's own RGBA storage, and the reader's frame the
// conversion last ran in.
GLuint ExternalTexture = 0;
Uint64 ConvertedFrame = 0;
~EglImage() {
// eglDestroyImageKHR needs the display, not a context, so a session's twin teardown
// (no context current) still releases the image; only exit() and a terminated
// display - which destroyed the image itself - skip it.
if (Handle == nullptr || Destroy == nullptr || InProcessExit()) return;
if (g_displayTerminations.load(std::memory_order_acquire) != DisplayTerminations) return;
// Texture names are the share group's: deleting one needs any context of it, and
// without one current the name goes with the display.
if (ExternalTexture != 0 && t_native.boundContext != EGL_NO_CONTEXT)
g_GLESFuncs.glDeleteTextures(1, &ExternalTexture);
Destroy(Display, Handle);
}
#endif
};
#if defined(__ANDROID__)
EglImageRef CreateEglImage(Uint64 id, String& why) {
const EntryPoints& entry = ResolvedEntryPoints();
if (!entry.Usable) {
why = entry.Why;
return nullptr;
}
SI::ImageRef image = SI::Find(id);
if (image == nullptr) {
why = "no live shared image has this id";
return nullptr;
}
if (image->Native == nullptr) {
why = "the image has no AHardwareBuffer";
return nullptr;
}
const EGLClientBuffer buffer =
entry.GetNativeClientBuffer(static_cast<const AHardwareBuffer*>(image->Native));
if (buffer == nullptr) {
why = "eglGetNativeClientBufferANDROID returned null";
return nullptr;
}
const EGLint attribs[] = {kEglImagePreservedKhr, EGL_TRUE, EGL_NONE};
void* handle = entry.CreateImage(g_Display, EGL_NO_CONTEXT, kEglNativeBufferAndroid, buffer, attribs);
if (handle == nullptr) {
char text[96];
std::snprintf(text, sizeof(text), "eglCreateImageKHR failed (EGL error 0x%04x)",
g_EGLFuncs.eglGetError ? static_cast<unsigned>(g_EGLFuncs.eglGetError()) : 0u);
why = text;
return nullptr;
}
auto result = MakeShared<EglImage>();
result->Image = std::move(image);
result->Handle = handle;
result->Display = g_Display;
result->DisplayTerminations = g_displayTerminations.load(std::memory_order_acquire);
result->Destroy = entry.DestroyImage;
return result;
}
Bool TargetBoundTexture2D(const EglImageRef& image) {
const EntryPoints& entry = ResolvedEntryPoints();
if (!entry.Usable || image == nullptr || image->Handle == nullptr) return false;
DrainErrors();
entry.TargetTexture2D(GL_TEXTURE_2D, image->Handle);
return g_GLESFuncs.glGetError() == GL_NO_ERROR;
}
namespace {
// ONE PRESENT TARGET PER IMAGE, PROCESS-WIDE: the EGLImage and a renderbuffer whose
// storage it is. Every native context is in one share group, so the renderbuffer is
// one object for every session's context; only the framebuffer it is attached to is
// per context. Kept until no session holds the image any more (pruned on every present
// - the cache's own reference is then the only one), with a small cap so a client that
// cycles through many images cannot pile them up between presents.
struct PresentTarget {
EglImageRef Image;
GLuint Renderbuffer = 0;
Uint64 LastUse = 0;
// A present has written the image through this target (its content is a frame).
Bool Written = false;
};
struct PresentTargets {
std::mutex Mutex;
std::unordered_map<Uint64, PresentTarget> ById;
Uint64 UseClock = 0;
};
constexpr SizeT kMaxPresentTargets = 8;
PresentTargets& Targets() {
static auto* targets = new PresentTargets();
return *targets;
}
// The framebuffer a present draws through, one per native context (container objects
// are never shared). It carries an attachment only for the duration of a present, so
// no context's framebuffer keeps a dropped target's renderbuffer alive.
PerNativeContext<GLuint> g_presentFramebuffers;
Bool TargetIsCurrent(const PresentTarget& target) {
return target.Image != nullptr &&
target.Image->DisplayTerminations == g_displayTerminations.load(std::memory_order_acquire);
}
void DeleteTarget(PresentTarget& target) {
if (target.Renderbuffer != 0 && TargetIsCurrent(target)) {
g_GLESFuncs.glDeleteRenderbuffers(1, &target.Renderbuffer);
}
target.Renderbuffer = 0;
target.Image = nullptr;
}
// Caller holds the mutex and has a context of the share group current.
void PruneLocked(PresentTargets& targets) {
for (auto it = targets.ById.begin(); it != targets.ById.end();) {
if (!TargetIsCurrent(it->second) || it->second.Image->Image.use_count() <= 1) {
DeleteTarget(it->second);
it = targets.ById.erase(it);
} else {
++it;
}
}
}
GLuint MakeRenderbuffer(const EglImage& image, String& why) {
GLuint renderbuffer = 0;
g_GLESFuncs.glGenRenderbuffers(1, &renderbuffer);
if (renderbuffer == 0) {
why = "glGenRenderbuffers gave no name";
return 0;
}
// Espryt keeps no renderbuffer-binding shadow; put the driver's binding back as found.
GLint previous = 0;
g_GLESFuncs.glGetIntegerv(GL_RENDERBUFFER_BINDING, &previous);
DrainErrors();
g_GLESFuncs.glBindRenderbuffer(GL_RENDERBUFFER, renderbuffer);
ResolvedEntryPoints().TargetRenderbuffer(GL_RENDERBUFFER, image.Handle);
const GLenum error = g_GLESFuncs.glGetError();
g_GLESFuncs.glBindRenderbuffer(GL_RENDERBUFFER, static_cast<GLuint>(previous));
if (error != GL_NO_ERROR) {
char text[96];
std::snprintf(text, sizeof(text), "glEGLImageTargetRenderbufferStorageOES -> 0x%04x",
static_cast<unsigned>(error));
why = text;
g_GLESFuncs.glDeleteRenderbuffers(1, &renderbuffer);
return 0;
}
return renderbuffer;
}
// Everything the current context has submitted, waited out on the CPU: the fallback for
// a driver that cannot hand a fence to another session.
Bool WaitForCompletion() {
if (g_GLESFuncs.glFenceSync && g_GLESFuncs.glClientWaitSync && g_GLESFuncs.glDeleteSync) {
if (GLsync fence = g_GLESFuncs.glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0)) {
constexpr GLuint64 kSliceNs = 1000ull * 1000 * 1000;
GLenum status = GL_TIMEOUT_EXPIRED;
GLbitfield flags = GL_SYNC_FLUSH_COMMANDS_BIT;
for (Uint slice = 0; slice < 10 && status == GL_TIMEOUT_EXPIRED; ++slice) {
status = g_GLESFuncs.glClientWaitSync(fence, flags, kSliceNs);
flags = 0;
}
g_GLESFuncs.glDeleteSync(fence);
if (status == GL_ALREADY_SIGNALED || status == GL_CONDITION_SATISFIED) return true;
if (status == GL_TIMEOUT_EXPIRED) {
MGLOG_E("Shared images: the submitted work did not complete within 10 s");
return false;
}
// GL_WAIT_FAILED: glFinish is the same promise without the object.
}
}
g_GLESFuncs.glFinish();
return true;
}
// How long a CPU wait on another session's fence may take before the order is given up
// on (logged): a session whose GPU work never completes must not wedge its peers.
constexpr Uint32 kFenceCpuWaitMs = 2000;
void CloseFd(int fd) {
if (fd >= 0) ::close(fd);
}
// The current context's LATER commands wait for `fence` (a sync_file) on the GPU; a CPU
// wait when the driver cannot. Takes ownership of `fence`: a native fence sync owns it
// from a successful create on, every other path closes it here.
void WaitForFenceBeforeLaterCommands(int fence) {
if (fence < 0) return;
const EntryPoints& entry = ResolvedEntryPoints();
if (entry.FencesUsable) {
const EGLint attribs[] = {kEglSyncNativeFenceFdAndroid, fence, EGL_NONE};
if (void* sync = entry.CreateSync(g_Display, kEglSyncNativeFenceAndroid, attribs)) {
if (entry.WaitSync(g_Display, sync, 0) != EGL_TRUE) {
constexpr Uint64 kWaitNs = Uint64{kFenceCpuWaitMs} * 1000 * 1000;
if (entry.ClientWaitSync(g_Display, sync, 0, kWaitNs) != kEglConditionSatisfiedKhr) {
MGLOG_E_ONCE("Shared images: a peer session's fence could not be waited for");
}
}
// A pending server wait keeps the sync alive past this (EGL_KHR_wait_sync).
entry.DestroySync(g_Display, sync);
return;
}
}
if (!SI::SyncFile::Wait(fence, kFenceCpuWaitMs)) {
MGLOG_E_ONCE("Shared images: a peer session's fence did not signal within %u ms",
static_cast<unsigned>(kFenceCpuWaitMs));
}
CloseFd(fence);
}
// A sync_file for everything the current context has submitted so far (flushed, so it
// will signal), in `*fence`. False when the driver cannot export one; the caller then
// waits on the CPU and publishes -1.
Bool ExportFence(int* fence) {
*fence = -1;
const EntryPoints& entry = ResolvedEntryPoints();
if (!entry.FencesUsable) return false;
const EGLint attribs[] = {EGL_NONE};
void* sync = entry.CreateSync(g_Display, kEglSyncNativeFenceAndroid, attribs);
if (sync == nullptr) return false;
// The fence has no descriptor until its command reaches the driver's queue.
g_GLESFuncs.glFlush();
const EGLint fd = entry.DupNativeFenceFd(g_Display, sync);
entry.DestroySync(g_Display, sync);
if (fd < 0) return false;
*fence = fd;
return true;
}
// ONE READ FRAME PER SESSION: what the session has sampled since its last frame
// boundary. A session is served by exactly one apply thread, so the thread owns it -
// nothing about it is process-wide, and a session's end (its thread's) drops it. `Id`
// is drawn from one counter across all sessions and renewed at every boundary, so an
// EglImage's NotedFrame is a once-per-frame gate no other session's frame can match.
struct ReadFrame {
Uint64 Id = 0;
SI::ReadTracker Tracker;
// The session ends frames at publishing flushes (PublishPendingAccesses): it writes
// images, so its first use of one in a frame waits for that image's pending reads.
Bool ImplicitSync = false;
// When a YUV image is converted again (ConvertYuvForSampling): renewed at EVERY
// boundary - present, publishing flush, and each native fence whatever the session
// is - since an offscreen producer (a browser's GPU process) may have no other.
Uint64 YuvEpoch = 0;
};
std::atomic<Uint64> g_readFrameIds{0};
ReadFrame& CurrentReadFrame() {
thread_local ReadFrame frame;
if (frame.Id == 0) frame.Id = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
if (frame.YuvEpoch == 0) frame.YuvEpoch = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
return frame;
}
void RenewYuvEpoch(ReadFrame& frame) {
frame.YuvEpoch = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
}
} // namespace
void AcquireForSampling(const EglImageRef& image) {
if (image == nullptr || image->Image == nullptr) return;
EglImage& state = *image;
// Steady state: one lock and a compare (the write fence, once signaled, is dropped by
// the registry and nothing is duplicated any more).
Uint64 generation = 0;
int fence = SI::DupWriteFence(*state.Image, &generation);
const Uint64 context = CurrentNativeContextSerial();
if (generation != state.WaitedGeneration || context != state.WaitedContext) {
state.WaitedGeneration = generation;
state.WaitedContext = context;
WaitForFenceBeforeLaterCommands(fence);
fence = -1;
}
CloseFd(fence);
ReadFrame& frame = CurrentReadFrame();
if (state.NotedFrame != frame.Id) {
// A session that renders into images (it ends frames with a publishing flush) may be
// about to overwrite this one: what other sessions still sample goes first.
if (frame.ImplicitSync) {
for (const int read : SI::DupPendingReadFences(*state.Image))
WaitForFenceBeforeLaterCommands(read);
}
frame.Tracker.NoteRead(state.Image);
state.NotedFrame = frame.Id;
}
}
Bool IsYuv(const EglImageRef& image) {
return image != nullptr && image->Image != nullptr && SI::FourccIsYuv(image->Image->Fourcc);
}
namespace {
// THE CONVERSION PASS: one triangle over the texture, sampling the image through an
// external texture. Programs are the share group's and rebuilt with a new context
// generation; the framebuffer and vertex array are per context (never shared).
struct YuvConversion {
Uint ContextGeneration = ~0u;
GLuint ExternalProgram = 0; // the driver's conversion (samplerExternalOES)
GLuint TargetProgram = 0; // the import's own (GL_EXT_YUV_target)
GLint TargetMatrix = -1;
GLint TargetOffset = -1;
Bool Failed = false;
NativeContextName Framebuffer;
NativeContextName VertexArray;
};
YuvConversion& Conversion() {
static auto* conversion = new YuvConversion();
return *conversion;
}
constexpr const char* kYuvVertexSource =
"#version 300 es\n"
"out vec2 vUv;\n"
"void main() {\n"
" vec2 p = vec2(float((gl_VertexID << 1) & 2), float(gl_VertexID & 2));\n"
" gl_Position = vec4(p * 2.0 - 1.0, 0.0, 1.0);\n"
" vUv = p;\n"
"}\n";
constexpr const char* kYuvExternalFragmentSource =
"#version 300 es\n"
"#extension GL_OES_EGL_image_external_essl3 : require\n"
"precision highp float;\n"
"uniform samplerExternalOES uSource;\n"
"in vec2 vUv;\n"
"out vec4 oColor;\n"
"void main() { oColor = vec4(texture(uSource, vUv).rgb, 1.0); }\n";
constexpr const char* kYuvTargetFragmentSource =
"#version 300 es\n"
"#extension GL_EXT_YUV_target : require\n"
"precision highp float;\n"
"uniform __samplerExternal2DY2YEXT uSource;\n"
"uniform mat3 uMatrix;\n"
"uniform vec3 uOffset;\n"
"in vec2 vUv;\n"
"out vec4 oColor;\n"
"void main() {\n"
" vec3 ycbcr = texture(uSource, vUv).xyz;\n"
" oColor = vec4(clamp(uMatrix * (ycbcr - uOffset), 0.0, 1.0), 1.0);\n"
"}\n";
GLuint BuildYuvProgram(const char* fragmentSource) {
const GLuint vertexShader = g_GLESFuncs.glCreateShader(GL_VERTEX_SHADER);
const GLuint fragmentShader = g_GLESFuncs.glCreateShader(GL_FRAGMENT_SHADER);
GLint linked = GL_FALSE;
GLuint program = 0;
if (vertexShader != 0 && fragmentShader != 0) {
g_GLESFuncs.glShaderSource(vertexShader, 1, &kYuvVertexSource, nullptr);
g_GLESFuncs.glCompileShader(vertexShader);
g_GLESFuncs.glShaderSource(fragmentShader, 1, &fragmentSource, nullptr);
g_GLESFuncs.glCompileShader(fragmentShader);
program = g_GLESFuncs.glCreateProgram();
if (program != 0) {
g_GLESFuncs.glAttachShader(program, vertexShader);
g_GLESFuncs.glAttachShader(program, fragmentShader);
g_GLESFuncs.glLinkProgram(program);
g_GLESFuncs.glGetProgramiv(program, GL_LINK_STATUS, &linked);
}
}
if (vertexShader != 0) g_GLESFuncs.glDeleteShader(vertexShader);
if (fragmentShader != 0) g_GLESFuncs.glDeleteShader(fragmentShader);
if (linked != GL_TRUE) {
if (program != 0) g_GLESFuncs.glDeleteProgram(program);
return 0;
}
return program;
}
// MOBILEGL_YUV_DRIVER_CONVERSION=1: always the driver's conversion (samplerExternalOES),
// even where GL_EXT_YUV_target would let the import's hints decide.
Bool DriverConversionForced() {
static const Bool forced = [] {
const char* value = std::getenv("MOBILEGL_YUV_DRIVER_CONVERSION");
return value != nullptr && value[0] == '1';
}();
return forced;
}
} // namespace
void ConvertYuvForSampling(const EglImageRef& image, GLuint target, Uint32 width, Uint32 height) {
if (!IsYuv(image) || target == 0 || width == 0 || height == 0) return;
EglImage& state = *image;
ReadFrame& frame = CurrentReadFrame();
if (state.ConvertedFrame == frame.YuvEpoch) return;
// Whatever happens below, this frame does not try again.
state.ConvertedFrame = frame.YuvEpoch;
const EntryPoints& entry = ResolvedEntryPoints();
if (!entry.Usable || !entry.ExternalEssl3) {
MGLOG_E_ONCE("Shared images: a YUV image cannot be sampled here (%s)",
entry.Usable ? "no GL_OES_EGL_image_external_essl3" : entry.Why.c_str());
return;
}
// THE CPU-COPY FALLBACK, for a buffer this server did not allocate (SharedImageRegistry.h).
String why;
if (state.Image->IsForeign() && !SI::RefreshForeign(*state.Image, why)) {
MGLOG_E_ONCE("Shared images: foreign YUV image %llu could not be refreshed: %s",
static_cast<unsigned long long>(state.Image->Id), why.c_str());
}
YuvConversion& conversion = Conversion();
if (conversion.ContextGeneration != g_backendContextGeneration) {
conversion.ExternalProgram = 0;
conversion.TargetProgram = 0;
conversion.TargetMatrix = conversion.TargetOffset = -1;
conversion.Failed = false;
conversion.ContextGeneration = g_backendContextGeneration;
}
if (conversion.Failed) return;
const Bool ownConversion = entry.YuvTarget && !DriverConversionForced();
GLuint& program = ownConversion ? conversion.TargetProgram : conversion.ExternalProgram;
if (program == 0) {
program = BuildYuvProgram(ownConversion ? kYuvTargetFragmentSource : kYuvExternalFragmentSource);
if (program == 0) {
conversion.Failed = true;
MGLOG_E_ONCE("Shared images: the YUV conversion program could not be built");
return;
}
if (ownConversion) {
conversion.TargetMatrix = g_GLESFuncs.glGetUniformLocation(program, "uMatrix");
conversion.TargetOffset = g_GLESFuncs.glGetUniformLocation(program, "uOffset");
}
MGLOG_I("Shared images: YUV images convert through %s",
ownConversion ? "GL_EXT_YUV_target with the import's colour hints"
: "the driver's external-sampler conversion");
}
if (conversion.Framebuffer == 0) g_GLESFuncs.glGenFramebuffers(1, &conversion.Framebuffer);
if (conversion.VertexArray == 0) g_GLESFuncs.glGenVertexArrays(1, &conversion.VertexArray);
if (conversion.Framebuffer == 0 || conversion.VertexArray == 0) return;
// Every binding goes through Espryt's shadows or a guard that puts it back.
ScopedEmulationDrawState drawState; // also activates TextureImpl::TempTextureUnit
ScopedFramebufferBinding binding(/*saveRead=*/false, /*saveDraw=*/true);
DrainErrors();
if (state.ExternalTexture == 0) {
g_GLESFuncs.glGenTextures(1, &state.ExternalTexture);
g_GLESFuncs.glBindTexture(GL_TEXTURE_EXTERNAL_OES, state.ExternalTexture);
entry.TargetTexture2D(GL_TEXTURE_EXTERNAL_OES, state.Handle);
g_GLESFuncs.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
g_GLESFuncs.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
g_GLESFuncs.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
g_GLESFuncs.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
const GLenum error = g_GLESFuncs.glGetError();
if (error != GL_NO_ERROR) {
MGLOG_E_ONCE("Shared images: YUV image %llu cannot be bound as an external texture (0x%04x)",
static_cast<unsigned long long>(state.Image->Id), static_cast<unsigned>(error));
}
} else {
g_GLESFuncs.glBindTexture(GL_TEXTURE_EXTERNAL_OES, state.ExternalTexture);
}
FramebufferImpl::BindFramebufferId(GL_DRAW_FRAMEBUFFER, conversion.Framebuffer);
g_GLESFuncs.glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, target, 0);
const GLenum status = g_GLESFuncs.glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER);
if (status == GL_FRAMEBUFFER_COMPLETE) {
VertexArrayImpl::BindBackendVAOId(conversion.VertexArray);
g_GLESFuncs.glUseProgram(program);
g_GLESFuncs.glUniform1i(g_GLESFuncs.glGetUniformLocation(program, "uSource"),
static_cast<GLint>(TextureImpl::TempTextureUnit));
if (ownConversion) {
// A Y2Y sample is the stored code normalized as the format stores it: 8 bits, or
// P010's 10 in the high end of 16.
const double sampleMax = state.Image->Fourcc == SI::kFourccP010 ? 65535.0 / 64.0 : 255.0;
const auto c = SI::Yuv::ConversionFor(SI::GetYuvHints(*state.Image), sampleMax);
g_GLESFuncs.glUniformMatrix3fv(conversion.TargetMatrix, 1, GL_FALSE, c.Matrix);
g_GLESFuncs.glUniform3fv(conversion.TargetOffset, 1, c.Offset);
}
g_GLESFuncs.glViewport(0, 0, static_cast<GLsizei>(width), static_cast<GLsizei>(height));
g_GLESFuncs.glDrawArrays(GL_TRIANGLES, 0, 3);
const GLenum error = g_GLESFuncs.glGetError();
if (error != GL_NO_ERROR) {
MGLOG_E_ONCE("Shared images: the YUV conversion draw failed (0x%04x)",
static_cast<unsigned>(error));
}
} else {
MGLOG_E_ONCE("Shared images: a YUV conversion target is not renderable (0x%04x)",
static_cast<unsigned>(status));
}
g_GLESFuncs.glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, 0, 0);
g_GLESFuncs.glBindTexture(GL_TEXTURE_EXTERNAL_OES, 0);
}
Bool PublishPendingAccesses() {
ReadFrame& frame = CurrentReadFrame();
frame.ImplicitSync = true;
RenewYuvEpoch(frame);
// Sub-data waiting for a sync that nothing of this session would otherwise trigger.
if (t_native.boundContext != EGL_NO_CONTEXT) TextureImpl::WritePendingSharedImageUploads();
if (frame.Tracker.Pending() == 0) return true;
int fence = -1;
if (t_native.boundContext != EGL_NO_CONTEXT && !ExportFence(&fence)) {
// No fence to publish: the work is waited out here, and published as completed.
(void)WaitForCompletion();
}
frame.Tracker.PublishFrameAsWrite(fence);
frame.Id = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
return true;
}
Bool ExportNativeFence(int* fence) {
*fence = -1;
ReadFrame& frame = CurrentReadFrame();
RenewYuvEpoch(frame);
// No context current: nothing of this session is in flight to fence.
if (t_native.boundContext == EGL_NO_CONTEXT) return true;
TextureImpl::WritePendingSharedImageUploads();
int exported = -1;
if (!ExportFence(&exported)) {
// No descriptor to give: the work is waited out, so the caller may report it done.
exported = -1;
(void)WaitForCompletion();
}
// A session that writes images (one that ends frames at flushes) ends one here too; a
// compositor's reads stay for its swap, which comes after the fence's work anyway.
if (frame.ImplicitSync && frame.Tracker.Pending() != 0) {
frame.Tracker.PublishFrameAsWrite(exported >= 0 ? ::dup(exported) : -1);
frame.Id = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
}
*fence = exported;
return true;
}
void PublishPendingReads() {
ReadFrame& frame = CurrentReadFrame();
RenewYuvEpoch(frame);
if (frame.Tracker.Pending() == 0) return;
int fence = -1;
// No context current: the reads' context is gone (and its work with it), and there is
// nothing to fence or wait from - the frame is only forgotten.
if (t_native.boundContext != EGL_NO_CONTEXT && !ExportFence(&fence)) {
// FALLBACK: there is no fence to hand a writer, so the frame's reads are waited out
// here instead - once this session moves on (and, say, releases the buffer to its
// producer), nothing it submitted still samples the image.
(void)WaitForCompletion();
}
// A session that writes images publishes every boundary as a write: an image it rendered
// into must not leave the frame without its generation moving.
if (frame.ImplicitSync)
frame.Tracker.PublishFrameAsWrite(fence);
else
frame.Tracker.PublishFrame(fence);
frame.Id = g_readFrameIds.fetch_add(1, std::memory_order_relaxed) + 1;
}
Bool BlitDefaultFramebufferTo(const SharedImageView& view, const MG_Util::Damage::Region& region) {
if (view.Id == 0 || view.Width == 0 || view.Height == 0 || view.NativeBuffer == nullptr) return false;
if (!IsBackendContextCurrentOnThisThread()) {
MGLOG_E_ONCE("Shared image present: no backend context is current on the calling thread");
return false;
}
if (LatchIfDeviceLost("shared-image-present")) return false;
const EGLSurface surface = CurrentContextDrawSurface();
EGLint sourceWidth = 0;
EGLint sourceHeight = 0;
if (surface == EGL_NO_SURFACE || !g_EGLFuncs.eglQuerySurface ||
!g_EGLFuncs.eglQuerySurface(g_Display, surface, EGL_WIDTH, &sourceWidth) ||
!g_EGLFuncs.eglQuerySurface(g_Display, surface, EGL_HEIGHT, &sourceHeight) || sourceWidth <= 0 ||
sourceHeight <= 0) {
MGLOG_E_ONCE("Shared image present: the current context has no default framebuffer to copy");
return false;
}
const SI::ImageRef image = SI::Find(view.Id);
if (image == nullptr) {
MGLOG_E_ONCE("Shared image present: image %llu is no longer live",
static_cast<unsigned long long>(view.Id));
return false;
}
// WRITE AFTER READ: the frames that still sample the image finish before the copy
// overwrites it - queued on this context's GPU timeline, or waited out on the CPU when
// the driver has no fence to queue. Outside the targets' lock, which every session's
// present takes.
if (ResolvedEntryPoints().FencesUsable) {
for (const int fence : SI::DupPendingReadFences(*image))
WaitForFenceBeforeLaterCommands(fence);
} else if (!SI::WaitForReads(*image, kFenceCpuWaitMs)) {
MGLOG_E_ONCE("Shared image present: image %llu's readers did not finish within %u ms",
static_cast<unsigned long long>(view.Id), static_cast<unsigned>(kFenceCpuWaitMs));
}
Bool copied = false;
{
auto& targets = Targets();
const std::lock_guard<std::mutex> lock(targets.Mutex);
PruneLocked(targets);
auto it = targets.ById.find(view.Id);
if (it != targets.ById.end() && it->second.Image->Image->Native != view.NativeBuffer) {
DeleteTarget(it->second);
targets.ById.erase(it);
it = targets.ById.end();
}
if (it == targets.ById.end()) {
String why;
PresentTarget target;
target.Image = CreateEglImage(view.Id, why);
if (target.Image != nullptr) target.Renderbuffer = MakeRenderbuffer(*target.Image, why);
if (target.Renderbuffer == 0) {
MGLOG_E_ONCE("Shared image present: image %llu cannot be rendered to: %s",
static_cast<unsigned long long>(view.Id), why.c_str());
return false;
}
while (targets.ById.size() >= kMaxPresentTargets) {
auto oldest = targets.ById.begin();
for (auto candidate = targets.ById.begin(); candidate != targets.ById.end(); ++candidate) {
if (candidate->second.LastUse < oldest->second.LastUse) oldest = candidate;
}
DeleteTarget(oldest->second);
targets.ById.erase(oldest);
}
it = targets.ById.emplace(view.Id, std::move(target)).first;
}
it->second.LastUse = ++targets.UseClock;
// A target written before keeps what it was given; one new to this process is
// copied whole whatever the client asked (it cannot know the image's history here).
const Bool wholeTarget = !it->second.Written;
it->second.Written = true;
GLuint& framebuffer = g_presentFramebuffers.Current();
if (framebuffer == 0) g_GLESFuncs.glGenFramebuffers(1, &framebuffer);
if (framebuffer == 0) {
MGLOG_E_ONCE("Shared image present: glGenFramebuffers gave no name");
return false;
}
// Every binding goes through Espryt's shadows (BindFramebufferId, the scissor test's
// render-state shadow) and is put back by the guards, so the application's state and
// the caches that describe it are what they were. A blit is scissored like a draw.
ScopedScissorDisable scissorOff;
ScopedFramebufferBinding binding(/*saveRead=*/true, /*saveDraw=*/true);
FramebufferImpl::BindFramebufferId(GL_READ_FRAMEBUFFER, 0);
FramebufferImpl::BindFramebufferId(GL_DRAW_FRAMEBUFFER, framebuffer);
g_GLESFuncs.glFramebufferRenderbuffer(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER,
it->second.Renderbuffer);
const GLenum status = g_GLESFuncs.glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER);
if (status == GL_FRAMEBUFFER_COMPLETE) {
// The default framebuffer's read buffer is context state the application may
// have set to GL_NONE; the copy reads the back buffer either way.
GLint readBuffer = GL_BACK;
g_GLESFuncs.glGetIntegerv(GL_READ_BUFFER, &readBuffer);
if (readBuffer != GL_BACK) g_GLESFuncs.glReadBuffer(GL_BACK);
DrainErrors();
const Bool sameExtent = static_cast<Uint32>(sourceWidth) == view.Width &&
static_cast<Uint32>(sourceHeight) == view.Height;
// TOP ROW FIRST: GL's top row (y = height - 1) lands in the image's first memory
// row, which an AHardwareBuffer-backed renderbuffer addresses as y = 0.
//
// Only where the image differs from the frame: the client keeps, per image, the
// damage it missed (BufferDamageTracker), and a partial blit keeps the rest of
// the renderbuffer as GL promises (checked texel for texel against the frame on
// Adreno). What once left old texels around the rectangles was the scissor
// guard above trusting another session's shadow: the copy was clipped, and
// the missed-damage bookkeeping then never brought those texels over again.
if (sameExtent && !wholeTarget && !region.IsFull()) {
// Each rectangle flipped the same way (an empty region copies nothing: the
// image already is the frame).
for (const MG_Util::Damage::FlippedCopy& copy :
MG_Util::Damage::FlippedCopies(region, sourceWidth, sourceHeight)) {
const MG_Util::Damage::Rect& from = copy.Source;
const MG_Util::Damage::Rect& to = copy.Destination;
g_GLESFuncs.glBlitFramebuffer(from.X, from.Y, from.X + from.Width, from.Y + from.Height,
to.X, to.Y + to.Height, to.X + to.Width, to.Y,
GL_COLOR_BUFFER_BIT, GL_NEAREST);
}
} else {
g_GLESFuncs.glBlitFramebuffer(0, 0, sourceWidth, sourceHeight, 0,
static_cast<GLint>(view.Height), static_cast<GLint>(view.Width),
0, GL_COLOR_BUFFER_BIT, sameExtent ? GL_NEAREST : GL_LINEAR);
}
const GLenum error = g_GLESFuncs.glGetError();
copied = error == GL_NO_ERROR;
if (readBuffer != GL_BACK) g_GLESFuncs.glReadBuffer(static_cast<GLenum>(readBuffer));
if (!copied) {
MGLOG_E_ONCE("Shared image present: glBlitFramebuffer %dx%d -> %ux%u failed (0x%04x)",
sourceWidth, sourceHeight, view.Width, view.Height, static_cast<unsigned>(error));
}
} else {
MGLOG_E_ONCE("Shared image present: image %llu's framebuffer is incomplete (0x%04x)",
static_cast<unsigned long long>(view.Id), static_cast<unsigned>(status));
}
g_GLESFuncs.glFramebufferRenderbuffer(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, 0);
}
if (!copied) return false;
// The copy's completion goes to the readers as a fence and the present returns at once;
// a reader that binds the image waits for it on its own GPU. Without fence export the
// copy is waited out here and published as already complete.
int fence = -1;
if (!ExportFence(&fence) && !WaitForCompletion()) return false;
SI::PublishWrite(*image, fence);
// A session that samples images and presents into one (a nested compositor) closes its
// read frame here too.
PublishPendingReads();
return true;
}
void ForgetPresentTargetsAfterTerminate() {
auto& targets = Targets();
const std::lock_guard<std::mutex> lock(targets.Mutex);
// TargetIsCurrent is already false for every entry, so nothing reaches the driver.
for (auto& entry : targets.ById)
DeleteTarget(entry.second);
targets.ById.clear();
}
#else
EglImageRef CreateEglImage(Uint64 id, String& why) {
(void)id;
why = "shared images are imported through AHardwareBuffer, which only Android has";
return nullptr;
}
Bool TargetBoundTexture2D(const EglImageRef& image) {
(void)image;
return false;
}
Bool BlitDefaultFramebufferTo(const SharedImageView& view, const MG_Util::Damage::Region& region) {
(void)view, (void)region;
return false;
}
void AcquireForSampling(const EglImageRef& image) {
(void)image;
}
Bool IsYuv(const EglImageRef& image) {
(void)image;
return false;
}
void ConvertYuvForSampling(const EglImageRef& image, GLuint target, Uint32 width, Uint32 height) {
(void)image, (void)target, (void)width, (void)height;
}
void PublishPendingReads() {}
Bool PublishPendingAccesses() {
return true;
}
Bool ExportNativeFence(int* fence) {
*fence = -1;
return true;
}
void ForgetPresentTargetsAfterTerminate() {}
#endif
} // namespace SharedImageImpl
#endif
} // namespace MobileGL::MG_Backend::DirectGLES
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,682 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/DirectGLESSync.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DirectGLESInternal.h"
namespace MobileGL::MG_Backend::DirectGLES {
namespace {
// Backend fence handle: a native ES sync plus the ES context
// generation it was created under.
struct GLESSyncObject {
GLsync esSync = nullptr;
Uint contextGeneration = 0;
};
// Backend timer-query handle: a native GL query object name plus the
// ES context generation it was created under. Stale-generation
// handles read as available with a zero result, and deleting them
// only frees the wrapper (the dead ES context already reclaimed the
// query object).
struct GLESQueryObject {
GLuint queryId = 0;
Uint contextGeneration = 0;
// Non-zero for the core (non-timer) query targets - occlusion and transform
// feedback primitives - and then holds the glBeginQuery target, which glEndQuery
// needs back. Their results are counts reachable only through the core
// glGetQueryObjectuiv getter: GL_EXT_disjoint_timer_query's 64-bit
// glGetQueryObjectui64vEXT is timer-specific and may be entirely absent on
// drivers that otherwise fully support these core ES queries.
GLenum coreTarget = 0;
};
} // namespace
// EGL ground-truth verification stamp. glvnd's eglGetCurrentContext performs
// fork detection with a real getpid() syscall on every call, and this
// predicate sits 2-3 deep in every draw - measured at 16% of the render
// thread on a live workload. NOT thread_local although the stamp is
// per-owner-thread state: a shared-library thread_local costs a
// __tls_get_addr call per access, which itself showed at 1.5% of the draw
// loop. Plain globals are equivalent because only the thread that passes the
// owner-identity check below can ever stamp or trust them, only one thread
// can be the owner at a time, and the ONLY writers of the owner id -
// MakeCurrent/ReleaseCurrent - reset the stamps before publishing a new
// owner, so a stamp can never leak across an ownership change. Atomics
// (relaxed) because the resets may come from a non-owner thread; the
// owner-id release/acquire pairing orders them.
std::atomic<Uint64> g_eglVerifiedFrameSerial{~0ull};
std::atomic<Uint> g_eglVerifiedContextGeneration{0};
Bool IsBackendContextCurrentOnThisThread() {
const NativeContextTuple& tuple = ActiveNativeContext();
if (tuple.Context == EGL_NO_CONTEXT) {
return false;
}
if (g_backendContextOwnerThread.load(std::memory_order_acquire) != std::this_thread::get_id()) {
return false;
}
// Belt and braces: EGL itself is the ground truth. A migration that bypassed
// MakeCurrent()/ReleaseCurrent() must not leave a stale ownership claim
// standing, or GL calls would silently no-op while shadow bookkeeping (bind
// cache, synced serials) still advances. Re-verify once per (frame, context
// generation) rather than per call: an external migration is caught at the
// next frame boundary instead of the next call, which recovers the
// bookkeeping just the same, without paying a syscall on every draw.
const Uint64 frameSerial = g_currentFrameSerial.load(std::memory_order_relaxed);
if (g_eglVerifiedFrameSerial.load(std::memory_order_relaxed) == frameSerial &&
g_eglVerifiedContextGeneration.load(std::memory_order_relaxed) == g_syncContextGeneration) {
return true;
}
if (g_EGLFuncs.eglGetCurrentContext && g_EGLFuncs.eglGetCurrentContext() != tuple.Context) {
return false;
}
g_eglVerifiedFrameSerial.store(frameSerial, std::memory_order_relaxed);
g_eglVerifiedContextGeneration.store(g_syncContextGeneration, std::memory_order_relaxed);
return true;
}
BackendSyncHandle FenceSync() {
// ES fences can only be created on the thread that owns the ES context
// (Flywheel and friends fence on the render thread, which does).
// Returning null makes the frontend fall back to an always-signaled
// sync object.
if (!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glFenceSync) {
return nullptr;
}
GLsync esSync = g_GLESFuncs.glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
if (esSync == nullptr) {
return nullptr;
}
return new GLESSyncObject{esSync, g_syncContextGeneration};
}
GLenum ClientWaitSync(BackendSyncHandle handle, GLbitfield flags, GLuint64 timeout) {
const auto* sync = static_cast<GLESSyncObject*>(handle);
if (sync == nullptr) {
return GL_ALREADY_SIGNALED;
}
// The creating ES context is gone: its GPU work either completed or
// died with the context; waiting is meaningless either way.
if (sync->contextGeneration != g_syncContextGeneration) {
return GL_ALREADY_SIGNALED;
}
// Degraded path: a thread that does not own the ES context cannot
// issue GL calls, so report signaled instead of blocking on state we
// cannot observe. Fence waits normally arrive on the render thread,
// which owns the context.
if (!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glClientWaitSync) {
return GL_ALREADY_SIGNALED;
}
return g_GLESFuncs.glClientWaitSync(sync->esSync, flags & GL_SYNC_FLUSH_COMMANDS_BIT, timeout);
}
void WaitSync(BackendSyncHandle handle, GLbitfield flags, GLuint64 timeout) {
(void)flags;
(void)timeout;
const auto* sync = static_cast<GLESSyncObject*>(handle);
if (sync == nullptr || sync->contextGeneration != g_syncContextGeneration ||
!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glWaitSync) {
return;
}
// ES 3.0 requires flags == 0 and timeout == GL_TIMEOUT_IGNORED.
g_GLESFuncs.glWaitSync(sync->esSync, 0, GL_TIMEOUT_IGNORED);
}
void DeleteSync(BackendSyncHandle handle) {
auto* sync = static_cast<GLESSyncObject*>(handle);
if (sync == nullptr) {
return;
}
if (sync->contextGeneration == g_syncContextGeneration && IsBackendContextCurrentOnThisThread() &&
g_GLESFuncs.glDeleteSync) {
g_GLESFuncs.glDeleteSync(sync->esSync);
}
// Otherwise the ES sync is abandoned; the ES context reclaims all of
// its sync objects when it is destroyed.
delete sync;
}
Bool GetSyncStatus(BackendSyncHandle handle) {
const auto* sync = static_cast<GLESSyncObject*>(handle);
if (sync == nullptr || sync->contextGeneration != g_syncContextGeneration ||
!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glGetSynciv) {
return true;
}
GLint status = GL_SIGNALED;
GLsizei length = 0;
g_GLESFuncs.glGetSynciv(sync->esSync, GL_SYNC_STATUS, 1, &length, &status);
return status == GL_SIGNALED;
}
// GL timer queries, backed by GL_EXT_disjoint_timer_query. The desktop
// tokens from glext.h are used throughout: GL_TIME_ELAPSED (0x88BF),
// GL_TIMESTAMP (0x8E28), GL_QUERY_RESULT (0x8866) and
// GL_QUERY_RESULT_AVAILABLE (0x8867) are numerically identical to their
// _EXT counterparts.
Bool AreTimerQueriesSupported() {
return g_GLESCapabilities.SupportsDisjointTimerQuery && g_GLESFuncs.glGenQueries &&
g_GLESFuncs.glDeleteQueries && g_GLESFuncs.glBeginQuery && g_GLESFuncs.glEndQuery &&
g_GLESFuncs.glGetQueryObjectuiv && g_GLESFuncs.glQueryCounterEXT && g_GLESFuncs.glGetQueryObjectui64vEXT;
}
namespace {
// The entry point the resolved tier's support ships, or null when there is none.
MG_External::GLES::glTexBuffer_PTR ResolveTexBufferEntryPoint() {
using Tier = MG_External::GLESCapabilities::TextureBufferTier;
switch (g_GLESCapabilities.TextureBufferSupport) {
case Tier::ExtensionEXT:
return g_GLESFuncs.glTexBufferEXT ? g_GLESFuncs.glTexBufferEXT : g_GLESFuncs.glTexBuffer;
case Tier::ExtensionOES:
return g_GLESFuncs.glTexBufferOES ? g_GLESFuncs.glTexBufferOES : g_GLESFuncs.glTexBuffer;
case Tier::CoreEs32:
return g_GLESFuncs.glTexBuffer;
case Tier::None:
default:
return nullptr;
}
}
MG_External::GLES::glTexBufferRange_PTR ResolveTexBufferRangeEntryPoint() {
using Tier = MG_External::GLESCapabilities::TextureBufferTier;
switch (g_GLESCapabilities.TextureBufferSupport) {
case Tier::ExtensionEXT:
return g_GLESFuncs.glTexBufferRangeEXT ? g_GLESFuncs.glTexBufferRangeEXT : g_GLESFuncs.glTexBufferRange;
case Tier::ExtensionOES:
return g_GLESFuncs.glTexBufferRangeOES ? g_GLESFuncs.glTexBufferRangeOES : g_GLESFuncs.glTexBufferRange;
case Tier::CoreEs32:
return g_GLESFuncs.glTexBufferRange;
case Tier::None:
default:
return nullptr;
}
}
} // namespace
Bool AreBufferTexturesSupported() {
// Both halves matter. The tier is what the driver ADVERTISES, and it is only meaningful
// once the capabilities have been filled in; the resolved pointer is what MobileGL can
// actually call, through the spelling that tier's support ships. Gating on the
// unsuffixed name alone would call an entry point an EXT/OES driver never exported.
return g_GLESCapabilities.TextureBufferSupport != MG_External::GLESCapabilities::TextureBufferTier::None &&
ResolveTexBufferEntryPoint() != nullptr;
}
void CallTexBuffer(GLenum target, GLenum internalFormat, GLuint buffer) {
MG_External::GLES::glTexBuffer_PTR entryPoint = ResolveTexBufferEntryPoint();
if (entryPoint == nullptr) {
return;
}
entryPoint(target, internalFormat, buffer);
}
Bool CallTexBufferRange(GLenum target, GLenum internalFormat, GLuint buffer, GLintptr offset, GLsizeiptr size) {
MG_External::GLES::glTexBufferRange_PTR entryPoint = ResolveTexBufferRangeEntryPoint();
if (entryPoint == nullptr) {
return false;
}
entryPoint(target, internalFormat, buffer, offset, size);
return true;
}
const char* GetBufferTextureTierName() {
using Tier = MG_External::GLESCapabilities::TextureBufferTier;
switch (g_GLESCapabilities.TextureBufferSupport) {
case Tier::CoreEs32:
return "core (ES 3.2)";
case Tier::ExtensionEXT:
return "GL_EXT_texture_buffer";
case Tier::ExtensionOES:
return "GL_OES_texture_buffer";
case Tier::None:
default:
return "unsupported";
}
}
BackendQueryHandle BeginTimeElapsedQuery() {
// Query objects can only be created on the thread that owns the ES
// context (MC's F3 profiler queries on the render thread, which
// does). Returning null makes the frontend fall back to an
// immediately available zero result.
if (!IsBackendContextCurrentOnThisThread() || !AreTimerQueriesSupported()) {
return nullptr;
}
GLuint queryId = 0;
g_GLESFuncs.glGenQueries(1, &queryId);
if (queryId == 0) {
return nullptr;
}
g_GLESFuncs.glBeginQuery(GL_TIME_ELAPSED, queryId);
return new GLESQueryObject{queryId, g_syncContextGeneration};
}
void EndTimeElapsedQuery(BackendQueryHandle handle) {
const auto* query = static_cast<GLESQueryObject*>(handle);
if (query == nullptr || query->contextGeneration != g_syncContextGeneration ||
!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glEndQuery) {
return;
}
// ES tracks the active query per target, not per object, so the
// handle only guards the degraded paths above.
g_GLESFuncs.glEndQuery(GL_TIME_ELAPSED);
}
BackendQueryHandle QueryCounterTimestamp() {
if (!IsBackendContextCurrentOnThisThread() || !AreTimerQueriesSupported()) {
return nullptr;
}
GLuint queryId = 0;
g_GLESFuncs.glGenQueries(1, &queryId);
if (queryId == 0) {
return nullptr;
}
g_GLESFuncs.glQueryCounterEXT(queryId, GL_TIMESTAMP);
return new GLESQueryObject{queryId, g_syncContextGeneration};
}
// The core (non-timer) query targets - occlusion and transform feedback primitives.
// Unlike the timer queries above these are core ES (no GL_EXT_disjoint_timer_query
// needed), so they share one begin/end pair keyed on the glBeginQuery target.
static Bool AreCoreQueriesSupported() {
return g_GLESFuncs.glGenQueries && g_GLESFuncs.glDeleteQueries && g_GLESFuncs.glBeginQuery &&
g_GLESFuncs.glEndQuery && g_GLESFuncs.glGetQueryObjectuiv;
}
static BackendQueryHandle BeginCoreQuery(GLenum target) {
if (!IsBackendContextCurrentOnThisThread() || !AreCoreQueriesSupported()) {
return nullptr;
}
GLuint queryId = 0;
g_GLESFuncs.glGenQueries(1, &queryId);
if (queryId == 0) {
return nullptr;
}
g_GLESFuncs.glBeginQuery(target, queryId);
return new GLESQueryObject{queryId, g_syncContextGeneration, target};
}
static void EndCoreQuery(BackendQueryHandle handle) {
const auto* query = static_cast<GLESQueryObject*>(handle);
if (query == nullptr || query->coreTarget == 0 || query->contextGeneration != g_syncContextGeneration ||
!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glEndQuery) {
return;
}
g_GLESFuncs.glEndQuery(query->coreTarget);
}
BackendQueryHandle BeginOcclusionQuery() {
// ES only implements the boolean ANY_SAMPLES_PASSED variant, not an exact
// GL_SAMPLES_PASSED count; the frontend already coerces ANY_SAMPLES_PASSED*
// targets to boolean, and desktop GL_SAMPLES_PASSED reads a 0/1 approximation.
return BeginCoreQuery(GL_ANY_SAMPLES_PASSED);
}
void EndOcclusionQuery(BackendQueryHandle handle) {
EndCoreQuery(handle);
}
// GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN / GL_PRIMITIVES_GENERATED. The frontend
// otherwise counts primitives on the CPU from the draw calls, which cannot see a
// geometry shader's amplification; the real driver's counters are exact. Returning
// null keeps that CPU accounting as the fallback.
BackendQueryHandle BeginXfbPrimitivesQuery(Bool generated) {
// GL_PRIMITIVES_GENERATED is only a legal query target from ES 3.2 on (it comes
// with geometry shaders); issuing it earlier just leaves a stray GL_INVALID_ENUM
// that some later unrelated glGetError would report as its own failure.
if (generated && g_GLESCapabilities.GLESVersion.Major * 10 + g_GLESCapabilities.GLESVersion.Minor < 32) {
return nullptr;
}
return BeginCoreQuery(generated ? GL_PRIMITIVES_GENERATED : GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN);
}
void EndXfbPrimitivesQuery(BackendQueryHandle handle) {
EndCoreQuery(handle);
}
Bool IsQueryResultAvailable(BackendQueryHandle handle) {
const auto* query = static_cast<GLESQueryObject*>(handle);
// Null/stale handles report available so the frontend proceeds to
// GetQueryResult64, which finalizes them as zero. A thread that does
// not own the ES context also reports available: GetQueryResult64
// then returns false and the frontend keeps the handle for a later
// read from the owning thread.
if (query == nullptr || query->contextGeneration != g_syncContextGeneration ||
!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glGetQueryObjectuiv) {
return true;
}
GLuint available = GL_FALSE;
g_GLESFuncs.glGetQueryObjectuiv(query->queryId, GL_QUERY_RESULT_AVAILABLE, &available);
return available != GL_FALSE;
}
Bool GetQueryResult64(BackendQueryHandle handle, Bool wait, Uint64* outNanoseconds) {
*outNanoseconds = 0;
const auto* query = static_cast<GLESQueryObject*>(handle);
// Null handles never had a GL query object, handles from a
// since-destroyed ES context lost theirs, and missing entry points
// can never produce a reading (belt and braces: the creators already
// require them): zero is the FINAL result in all three cases, so
// report it as produced and let the frontend cache it and release
// the handle.
if (query == nullptr || query->contextGeneration != g_syncContextGeneration ||
!g_GLESFuncs.glGetQueryObjectuiv || (query->coreTarget == 0 && !g_GLESFuncs.glGetQueryObjectui64vEXT)) {
return true;
}
// A thread that does not own the ES context cannot issue GL calls,
// but the result still lands on the owning context eventually: report
// "not obtainable yet" so the frontend keeps the handle and a later
// availability poll / result read from the owning thread can still
// produce the real value.
if (!IsBackendContextCurrentOnThisThread()) {
return false;
}
if (wait) {
// Reading GL_QUERY_RESULT blocks in the driver until the result
// lands, but only after the commands were flushed; flush once,
// then poll availability for a bounded ~100ms before dropping to
// a glFinish as the last resort (ClientWaitSync has no polling
// loop to mirror - it delegates its timeout to the driver, which
// a query-object read cannot do).
if (g_GLESFuncs.glFlush) {
g_GLESFuncs.glFlush();
}
constexpr Int kMaxAvailabilityPolls = 1000; // ~100ms at 100us per poll
GLuint available = GL_FALSE;
for (Int i = 0; i < kMaxAvailabilityPolls && available == GL_FALSE; ++i) {
g_GLESFuncs.glGetQueryObjectuiv(query->queryId, GL_QUERY_RESULT_AVAILABLE, &available);
if (available == GL_FALSE) {
std::this_thread::sleep_for(std::chrono::microseconds(100));
}
}
if (available == GL_FALSE && g_GLESFuncs.glFinish) {
g_GLESFuncs.glFinish();
}
}
// GL_EXT_disjoint_timer_query's GPU_DISJOINT_EXT signal is
// deliberately ignored: after a disjoint event (power state change,
// context switch) the result may be garbage, which is tolerable for
// an F3 GPU% readout, and consuming the latched flag here could hide
// the event from another observer.
if (query->coreTarget != 0) {
GLuint result32 = 0;
g_GLESFuncs.glGetQueryObjectuiv(query->queryId, GL_QUERY_RESULT, &result32);
*outNanoseconds = static_cast<Uint64>(result32);
return true;
}
GLuint64 result = 0;
g_GLESFuncs.glGetQueryObjectui64vEXT(query->queryId, GL_QUERY_RESULT, &result);
*outNanoseconds = static_cast<Uint64>(result);
return true;
}
void DeleteBackendQuery(BackendQueryHandle handle) {
auto* query = static_cast<GLESQueryObject*>(handle);
if (query == nullptr) {
return;
}
if (query->contextGeneration == g_syncContextGeneration && IsBackendContextCurrentOnThisThread() &&
g_GLESFuncs.glDeleteQueries) {
g_GLESFuncs.glDeleteQueries(1, &query->queryId);
}
// Otherwise the GL query object is abandoned; the ES context reclaims
// all of its query objects when it is destroyed.
delete query;
}
Int64 GetGpuTimestampNs() {
// Synchronous GPU clock sample; 0 tells the frontend GL_TIMESTAMP
// getter to fall back.
if (!IsBackendContextCurrentOnThisThread() || !AreTimerQueriesSupported() || !g_GLESFuncs.glGetInteger64v) {
return 0;
}
GLint64 timestamp = 0;
g_GLESFuncs.glGetInteger64v(GL_TIMESTAMP, &timestamp);
return static_cast<Int64>(timestamp);
}
Uint64 CurrentFrameSerial() {
return g_currentFrameSerial.load(std::memory_order_relaxed);
}
Uint64 CompletedFrameSerial() {
return g_completedFrameSerial.load(std::memory_order_relaxed);
}
namespace {
// THE WATERMARK IS CONTIGUOUS. Fences signal in submission order only within ONE context,
// and the serials are handed out to every native context of every session in turn, so the
// highest signaled serial says nothing about an older fence of another context still
// pending. A serial is complete when no live fence at or below it is pending.
void PollFrameFences() {
if (!g_GLESFuncs.glGetSynciv) return;
Uint64 lowestPending = ~Uint64{0};
for (FrameFence& slot : g_frameFenceRing) {
if (!slot.sync) continue;
if (slot.contextGeneration != g_syncContextGeneration) {
slot.sync = nullptr; // its context reclaimed it, and its work is over
continue;
}
GLint status = GL_SIGNALED;
GLsizei length = 0;
g_GLESFuncs.glGetSynciv(slot.sync, GL_SYNC_STATUS, 1, &length, &status);
if (status == GL_SIGNALED) {
if (g_GLESFuncs.glDeleteSync) g_GLESFuncs.glDeleteSync(slot.sync);
slot.sync = nullptr;
} else if (slot.serial < lowestPending) {
lowestPending = slot.serial;
}
}
const Uint64 completed =
lowestPending == ~Uint64{0} ? g_currentFrameSerial.load(std::memory_order_relaxed) : lowestPending - 1;
if (completed > g_completedFrameSerial.load(std::memory_order_relaxed)) {
g_completedFrameSerial.store(completed, std::memory_order_relaxed);
}
}
FrameFence* OldestPendingFrameFence() {
FrameFence* oldest = nullptr;
for (FrameFence& slot : g_frameFenceRing) {
if (!slot.sync || slot.contextGeneration != g_syncContextGeneration) continue;
if (!oldest || slot.serial < oldest->serial) oldest = &slot;
}
return oldest;
}
} // namespace
Bool WaitForFrameSerialCompleted(Uint64 serial, Uint64 timeoutNs) {
if (CompletedFrameSerial() >= serial) return true;
if (!IsBackendContextCurrentOnThisThread() || !g_GLESFuncs.glClientWaitSync) return false;
// Oldest pending fence first: the watermark can only pass a serial once every older fence
// (of any context) has signaled. Each fence was flushed when it was made (Present's swap),
// so waiting from this context needs no flush of the other.
for (int guard = 0; guard < kFrameFenceRingDepth + 1; ++guard) {
PollFrameFences();
if (CompletedFrameSerial() >= serial) return true;
FrameFence* oldest = OldestPendingFrameFence();
if (!oldest) return false;
const GLenum status = g_GLESFuncs.glClientWaitSync(oldest->sync, GL_SYNC_FLUSH_COMMANDS_BIT, timeoutNs);
if (status != GL_ALREADY_SIGNALED && status != GL_CONDITION_SATISFIED) return false;
}
PollFrameFences();
return CompletedFrameSerial() >= serial;
}
#if MOBILEGL_BUILD_DISAGGREGATED
namespace {
// Once per thread: a latched session declines everything after this verb anyway, and the
// apply thread is the session's own.
thread_local Bool t_deviceLossLatched = false;
Bool LatchOnResetStatus(const char* site, GLenum status) {
if (status == GL_NO_ERROR) return false;
// ONLY THE GUILTY CONTEXT ENDS ITS SESSION. The contexts ask for reset notification
// (EnsureNativeContext) so the driver can name the context whose work hung or faulted the
// GPU; a reset it reports to an INNOCENT context - one whose work only waited behind the
// stall, the compositor's above all - is noted and the session goes on as it did before
// contexts were told. Adreno 750 measured: the guilty context reads GUILTY once, after
// which the driver drops everything it submits; innocent ones read NO_ERROR.
static thread_local Bool innocentNoted = false;
if (status == GL_INNOCENT_CONTEXT_RESET) {
if (!innocentNoted) {
innocentNoted = true;
MGLOG_W("Espryt:%s - the driver reports a GPU reset this session's context did not cause "
"(GL_INNOCENT_CONTEXT_RESET); the session goes on",
site != nullptr ? site : "?");
}
return false;
}
t_deviceLossLatched = true;
(void)MG_Pipe::MGPipeSessionLatch(MG_Pipe::MGPipeFatalFamily::DeviceLost,
"MGPipe: Fatal{BackendDeviceLost, \"Espryt:%s\"} - this session's EGL "
"context was reset (glGetGraphicsResetStatus 0x%x); the session ends "
"and its client reads a lost context, every other session keeps running",
site != nullptr ? site : "?", static_cast<unsigned>(status));
return true;
}
} // namespace
Bool LatchIfDeviceLost(const char* site, Bool debugKnob) {
if (MG_Config::Transport == MG_Config::TransportMode::Monolith) return false;
if (t_deviceLossLatched) return true;
GLenum status = GL_NO_ERROR;
if (g_GLESFuncs.glGetGraphicsResetStatus != nullptr && IsBackendContextCurrentOnThisThread()) {
status = g_GLESFuncs.glGetGraphicsResetStatus();
}
if (status == GL_NO_ERROR && debugKnob && MG_Pipe::MGPipeDebugDeviceLossDue())
status = GL_UNKNOWN_CONTEXT_RESET;
return LatchOnResetStatus(site, status);
}
Bool LatchIfGuiltyBeforeApply() {
if (MG_Config::Transport == MG_Config::TransportMode::Monolith) return false;
if (t_deviceLossLatched) return true;
// Asked every time the apply thread wakes, so it must leave nothing behind: not
// IsBackendContextCurrentOnThisThread, whose once-per-frame EGL verification stamp it would
// spend at the top of the loop (where the session may be between two of its contexts) - only
// whether this thread holds a native context and the backend's ownership.
if (g_GLESFuncs.glGetGraphicsResetStatus == nullptr || t_native.boundContext == EGL_NO_CONTEXT ||
g_backendContextOwnerThread.load(std::memory_order_acquire) != std::this_thread::get_id())
return false;
return LatchOnResetStatus("apply", g_GLESFuncs.glGetGraphicsResetStatus());
}
#endif
void Present() {
#if MOBILEGL_BUILD_DISAGGREGATED
if (LatchIfDeviceLost("present", /*debugKnob=*/false)) return;
#endif
// Insert one fence per frame BEFORE the swap (eglSwapBuffers' implicit flush
// makes it reachable), then non-blocking-poll prior frames' fences AFTER to
// advance the completed-frame watermark that gates buffer-pool recycling.
const Bool canFence = IsBackendContextCurrentOnThisThread() && g_GLESFuncs.glFenceSync;
Uint64 presentedSerial = 0;
if (canFence) {
const Uint64 serial = g_currentFrameSerial.fetch_add(1, std::memory_order_relaxed) + 1;
presentedSerial = serial;
FrameFence& slot = g_frameFenceRing[serial % kFrameFenceRingDepth];
if (slot.sync && slot.contextGeneration == g_syncContextGeneration) {
// The slot's fence is a ring-depth of serials old and may be another context's:
// dropping it unsignaled would let the watermark pass work nobody waited for.
if (g_GLESFuncs.glClientWaitSync) {
constexpr Uint64 kOldFenceWaitNs = 100ull * 1000 * 1000;
(void)g_GLESFuncs.glClientWaitSync(slot.sync, GL_SYNC_FLUSH_COMMANDS_BIT, kOldFenceWaitNs);
}
if (g_GLESFuncs.glDeleteSync) g_GLESFuncs.glDeleteSync(slot.sync);
}
slot = {g_GLESFuncs.glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0), g_syncContextGeneration, serial};
}
#if MOBILEGL_BUILD_DISAGGREGATED
// The session's frame boundary: the shared images its frame sampled get the frame's fence,
// so their producers' next copies wait for it.
SharedImageImpl::PublishPendingReads();
#endif
// P14 S4: the session's surface, not the process's.
//
// EXT -> KHR -> plain. The loader fills eglSwapBuffersWithDamageEXT from whichever
// spelling the driver provides (Adreno exposes only eglSwapBuffersWithDamageKHR), so
// the null check below is the whole chain: a driver with neither presents with plain
// eglSwapBuffers. Passing n_rects == 0 is not a partial present - the extension
// defines it as "the entire surface is implicitly damaged and the behaviour is
// equivalent to calling eglSwapBuffers" - and the wire protocol carries no damage
// rectangles for the server to forward, so the full surface is what is meant anyway.
//
// The damage is the client's, from the present record (CurrentPresentDamage): GL window
// coordinates, which is what the native call takes. Whole surface = no rectangles.
//
// ONLY A WINDOW SURFACE TAKES RECTANGLES: the platform's swap-with-damage hands them to the
// surface's native window, and a pbuffer has none (Android's dereferences it regardless).
NativeSessionState& session = ActiveNativeSession(); // before the lock: it takes it too
const EGLSurface drawSurface = session.Draw;
Bool windowSurface = false;
if (drawSurface != EGL_NO_SURFACE) {
const std::lock_guard<std::mutex> lock(g_nativeRegistryMutex);
if (const auto* entry = FindSessionSurfaceLocked(session, drawSurface))
windowSurface = entry->Window != static_cast<NativeWindowType>(0);
}
EGLBoolean mglSwapped = EGL_FALSE;
if (g_EGLFuncs.eglSwapBuffersWithDamageEXT != nullptr) {
const MG_Util::Damage::Region& damage = MG_Backend::CurrentPresentDamage();
EGLint rects[MG_Util::Damage::kMaxRects * 4] = {};
const EGLint count = static_cast<EGLint>(MG_Util::Damage::NativeSwapRects(windowSurface, damage, rects));
mglSwapped =
g_EGLFuncs.eglSwapBuffersWithDamageEXT(g_Display, drawSurface, count > 0 ? rects : nullptr, count);
// A DRIVER THAT NAMES THE ENTRY POINT AND THEN REFUSES THE SWAP. Some vendor EGLs
// export eglSwapBuffersWithDamageEXT and answer EGL_FALSE on a window surface without
// setting an error (measured on a 4.14 vendor kernel with an Adreno driver; the plain
// entry point presents the same surface happily). Keeping the frame on that answer
// queues nothing at all: the compositor composites into a window whose buffer queue
// never sees a buffer. Fall back to the plain call instead of dropping the frame.
if (mglSwapped == EGL_FALSE && g_EGLFuncs.eglSwapBuffers != nullptr) {
MGLOG_W_ONCE("DirectGLES: this driver's eglSwapBuffersWithDamageEXT refused a window "
"surface; presenting through eglSwapBuffers for the rest of the process");
mglSwapped = g_EGLFuncs.eglSwapBuffers(g_Display, drawSurface);
}
} else if (g_EGLFuncs.eglSwapBuffers != nullptr) {
mglSwapped = g_EGLFuncs.eglSwapBuffers(g_Display, drawSurface);
}
NoteDrawSurfaceMoved(); // a window surface takes a new size at its swap
if (drawSurface != EGL_NO_SURFACE) {
const std::lock_guard<std::mutex> lock(g_nativeRegistryMutex);
if (auto* entry = FindSessionSurfaceLocked(session, drawSurface)) ++entry->Presents;
}
// A frame boundary: the next frame's age is asked anew, and its window buffer use flushed.
t_ageAskedSurface = EGL_NO_SURFACE;
t_windowBufferFlushIssued = false;
// P15: MOBILEGL_FRAMES_IN_FLIGHT, Espryt's half. Magma bounds the frames the GPU has
// queued with its frame contexts (frame S+1 reuses the context of frame S+1-N); this is
// the same bound on the ES driver: after presenting S, wait for S+1-N to complete. The
// platform's own queueBuffer throttle already waits for S-1, so only N = 1 is stricter
// than what Android does by itself; larger N is the same contract stated, not a change.
const Uint32 framesInFlight = MG_Config::Features.FramesInFlight;
if (presentedSerial != 0 && framesInFlight != 0 && framesInFlight < static_cast<Uint32>(kFrameFenceRingDepth) &&
g_GLESFuncs.glClientWaitSync && presentedSerial + 1 > framesInFlight) {
const Uint64 target = presentedSerial + 1 - framesInFlight;
const FrameFence& oldest = g_frameFenceRing[target % kFrameFenceRingDepth];
if (oldest.sync && oldest.serial == target && oldest.contextGeneration == g_syncContextGeneration) {
constexpr Uint64 kFramesInFlightWaitNs = 1000ull * 1000 * 1000;
(void)g_GLESFuncs.glClientWaitSync(oldest.sync, GL_SYNC_FLUSH_COMMANDS_BIT, kFramesInFlightWaitNs);
}
}
if (canFence) PollFrameFences();
// After the watermark advanced: retire grown-away ring stores and record the
// frame's ring high-water marks for slot reclamation.
BufferImpl::UboRingOnPresent();
BufferImpl::UnpackRingOnPresent();
BufferImpl::UploadRingOnPresent();
BufferImpl::TrimBufferPool();
// THE frame boundary for the MGPipe counters: publish this frame's plots, fold the
// frame into the run totals and, every 120th frame, emit the summary line.
if (MG_Util::PipeStats::Enabled()) {
MG_Util::PipeStats::OnPresent();
}
}
} // namespace MobileGL::MG_Backend::DirectGLES
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+48 -65
View File
@@ -307,7 +307,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
template <typename StateObject, typename BackendObject MGB_TWIN_KIND_PARAM>
class StateBackendObjectRegistry {
public:
using BackendPtr = SharedPtr<BackendObject>;
using SlotTable = BackendSlotTable<StateObject, BackendObject, kKind>;
// P14 S6: the ONE bucket of this kind the calling thread is in. Every handle-keyed entry
@@ -435,13 +434,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (EsprytSlotTablesEnabled()) return m_slotTables.Snapshot();
return {};
}
SizeT BucketCount() const {
return EsprytSlotTablesEnabled() ? m_slotTables.BucketCount() : 0;
}
SizeT BucketCount() const { return EsprytSlotTablesEnabled() ? m_slotTables.BucketCount() : 0; }
// Drop one {session, share group}. Answers whether such a bucket existed.
Bool DropBucket(const TwinKey& key) {
return EsprytSlotTablesEnabled() && m_slotTables.Drop(key);
}
Bool DropBucket(const TwinKey& key) { return EsprytSlotTablesEnabled() && m_slotTables.Drop(key); }
// Drop every bucket of one session - the difference between "this session's twins go"
// and "the process's do" - and answer how many groups that took with it.
Uint32 DropBucketsOfSession(Uint64 sessionKey) {
@@ -634,8 +629,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// view record is live with Desc.ViewOf naming a freed slot. No draw can land in between, and
// PipeTextureRecordForHandle tests Live and Gen, so that window answers null (= not clean)
// rather than a recycled stranger's record.
const MG_Pipe::MGPipeResourceRecord* PipeTextureStorageRecordForRecord(
const MG_Pipe::MGPipeResourceRecord& record);
const MG_Pipe::MGPipeResourceRecord* PipeTextureStorageRecordForRecord(const MG_Pipe::MGPipeResourceRecord& record);
// P5e (fb): the texture's own TARGET, from its descriptor. The image-unit bind needs it
// (glBindImageTexture's layered/format rules are per target) and MGPImageView has no room
@@ -748,8 +742,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// The DISTINCT buffers the draw's enabled attributes fetch from, deduped on {slot, gen}.
// A null Res is skipped: it is a disabled slot below the window's high-water mark, or a
// client-memory array whose bytes have no store to ensure (P8 stages those).
Uint ResolveDrawVertexBuffersFromRecord(const MG_Pipe::MGPipeApplierState& st,
DrawVertexBufferRequest* out, Uint capacity);
Uint ResolveDrawVertexBuffersFromRecord(const MG_Pipe::MGPipeApplierState& st, DrawVertexBufferRequest* out,
Uint capacity);
struct DrawIndexBufferRequest {
MG_Pipe::MGPipeHandle Res = MG_Pipe::kMGPipeNullHandle;
@@ -895,8 +889,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// P5c (hd): the staged-coverage assertion (StagedShadowStore::RequireCoverage) for a
// read of the server shadow outside the upload ladders - the indirect command-byte
// resolver. A no-op for a base that is not this resource's server shadow.
void RequireStagedCoverage(GLESBufferResource& resource, const Uint8* hostBase, SizeT start,
SizeT end, const char* site);
void RequireStagedCoverage(GLESBufferResource& resource, const Uint8* hostBase, SizeT start, SizeT end,
const char* site);
// P11 B2: THE CPU-READABLE BYTES OF A SPLIT SERVER'S BUFFER TWIN, for the handle-arm
// readers that have no frontend object (indirect commands, primitive-restart
@@ -945,7 +939,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
//
// `GL_MAP_INVALIDATE_RANGE_BIT` is not a hint, it is an ASSERTION THAT THE OLD BYTES
// ARE DEAD - and it is only true of the bytes this call is about to rewrite from the
// authoritative shadow. Managers.cpp:1125-1128 records what happens when it is not:
// authoritative shadow. ManagersBuffers.cpp's FlushPendingRangesFrom records what happens when it is not:
// widening the map to page bounds "looked free and was not - the widened bytes
// clobbered GPU-written data (an SSBO counter beside the app's SubData) with the stale
// shadow". It fails SILENTLY, unlike tier 3, which only stalls.
@@ -965,9 +959,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
//
// Returns the glMapBufferRange access bits, or 0 when tier 1 must not be taken.
inline constexpr SizeT kEsprytInvalidateRangeMinBytes = 128u * 1024u;
constexpr GLbitfield InvalidateFlushAccessFor(SizeT queuedStart, SizeT queuedEnd, SizeT mapStart,
SizeT mapEnd, SizeT limit, SizeT storageSize,
Bool ringCopyInFlight) {
constexpr GLbitfield InvalidateFlushAccessFor(SizeT queuedStart, SizeT queuedEnd, SizeT mapStart, SizeT mapEnd,
SizeT limit, SizeT storageSize, Bool ringCopyInFlight) {
if (mapEnd <= mapStart) return 0u;
// THE WIDENING REFUSAL. Not >=, not "covers": exactly, in both directions. A map
// narrower than the queued range leaves bytes unwritten inside a range it has just
@@ -986,8 +979,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// instead, GPU-ordered behind the copies. The whole-buffer orphan-map is exempt: it
// rewrites every byte from the shadow the queued copies were taken from.
if (!wholeBuffer && ringCopyInFlight) return 0u;
return GL_MAP_WRITE_BIT |
(wholeBuffer ? GL_MAP_INVALIDATE_BUFFER_BIT : GL_MAP_INVALIDATE_RANGE_BIT);
return GL_MAP_WRITE_BIT | (wholeBuffer ? GL_MAP_INVALIDATE_BUFFER_BIT : GL_MAP_INVALIDATE_RANGE_BIT);
}
// P15: WRITE RENAMING FOR A SMALL STORE. The draw-time drain sends a partial range below
@@ -1142,8 +1134,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// frames, so re-referencing them would need per-bind pinning — rewriting
// GetUBOSize() bytes once per program per frame is far cheaper.
struct UboRingAllocation {
Uint32 contentVersion = ~0u; // frontend UBO content version held at `offset`
Uint32 ringGeneration = 0; // ring identity the slot lives in (0 = never valid)
Uint32 contentVersion = ~0u; // frontend UBO content version held at `offset`
Uint32 ringGeneration = 0; // ring identity the slot lives in (0 = never valid)
Uint64 frameSerial = ~Uint64{0}; // frame the slot was written in
SizeT offset = 0;
};
@@ -1161,7 +1153,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
// this frame and ring generation, Allocated when a fresh slot was carved (`outWrite`
// is where its bytes go), Failed when allocation failed. `outBufferId` names the ring
// store for Reused and Allocated.
enum class UboRingSlotState : Uint8 { Unavailable, Reused, Allocated, Failed };
enum class UboRingSlotState : Uint8 {
Unavailable,
Reused,
Allocated,
Failed
};
UboRingSlotState UboRingPrepareSlot(UboRingAllocation& slot, Uint32 contentVersion, Uint64 frameSerial,
SizeT bindSize, void*& outWrite, Uint& outBufferId);
void* UboRingMappedPtr();
@@ -1509,7 +1506,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
Uint64 m_syncedContextSerial = 0;
};
extern TwinRegistry<MG_State::GLState::VertexArrayObject, BackendVertexArrayObject, MG_Pipe::MGPipeKind::VertexElementsCso>
extern TwinRegistry<MG_State::GLState::VertexArrayObject, BackendVertexArrayObject,
MG_Pipe::MGPipeKind::VertexElementsCso>
g_backendVertexArrayObjects;
// P5e (id), CONTRACT-P5E §4.1 / §4.2: THE VAO TWIN, RESOLVED BY THE HANDLE THE RECORD
@@ -1628,8 +1626,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
inline Bool IsMultisampleTextureTarget(TextureTarget target) {
return target == TextureTarget::Texture2DMultisample ||
target == TextureTarget::Texture2DMultisampleArray;
return target == TextureTarget::Texture2DMultisample || target == TextureTarget::Texture2DMultisampleArray;
}
inline Bool SupportsWrapR(TextureTarget target) {
@@ -1833,8 +1830,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
//
// The contextId / samplingGeneration arguments are GONE: the applier's ContextSerial
// and TextureShutterSerial are what the caller's keys already carry.
Bool IsDrawSyncCleanByRecord(MG_Pipe::MGPipeHandle res,
const MG_Pipe::MGPipeResourceRecord& record) const;
Bool IsDrawSyncCleanByRecord(MG_Pipe::MGPipeHandle res, const MG_Pipe::MGPipeResourceRecord& record) const;
private:
void RecreateBackendTexture();
@@ -2069,8 +2065,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// ResolveTextureTwin is the lookup without the sync, for the callers that need the twin
// to answer a shape question (an image bind's target and storage format come off the
// twin, not off a new wire field) after tx2's sync has already run this frame.
SharedPtr<BackendTextureObject>& SyncTextureToBackendByHandle(
MG_Pipe::MGPipeHandle texture, Bool imageBindableStorageRequired = false);
SharedPtr<BackendTextureObject>& SyncTextureToBackendByHandle(MG_Pipe::MGPipeHandle texture,
Bool imageBindableStorageRequired = false);
BackendTextureObject* ResolveTextureTwin(MG_Pipe::MGPipeHandle texture);
// NoteDriverSideTextureWrite's HANDLE ARM, for the same paths when the caller holds a
// Texture handle (a framebuffer surface's Res, an image unit's Res, VerbMipRes,
@@ -2186,9 +2182,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
*/
GLenum m_backendDrawBuffers[MG_State::GLState::FramebufferObject::MAX_DRAW_BUFFERS] = {GL_NONE};
static constexpr Uint MAX_COLOR_ATTACHMENT_SLOTS =
static_cast<Uint>(FramebufferAttachmentType::Color31) -
static_cast<Uint>(FramebufferAttachmentType::Color0) + 1;
static constexpr Uint MAX_COLOR_ATTACHMENT_SLOTS = static_cast<Uint>(FramebufferAttachmentType::Color31) -
static_cast<Uint>(FramebufferAttachmentType::Color0) + 1;
/* Where each frontend GL_COLOR_ATTACHMENTn image physically lives in the backend ES
framebuffer, as a GL_COLOR_ATTACHMENTm enum. ES only accepts glDrawBuffers bufs[s] ==
GL_COLOR_ATTACHMENTs, so a GL draw-buffer slot s naming attachment a forces a's image
@@ -2272,7 +2267,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
void ApplyReadBufferFromRecord(const MG_Pipe::MGPFramebufferState& record, Uint glNameForDiag);
};
extern TwinRegistry<MG_State::GLState::FramebufferObject, BackendFramebufferObject, MG_Pipe::MGPipeKind::Framebuffer>
extern TwinRegistry<MG_State::GLState::FramebufferObject, BackendFramebufferObject,
MG_Pipe::MGPipeKind::Framebuffer>
g_backendFramebufferObjects;
// P4a (D-C2 as corrected by ID-19): the applier's record for THE FRAMEBUFFER OBJECT this
@@ -2306,8 +2302,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// The index answers about a TEXTURE SLOT and returns whole FRAMEBUFFER handles, so a
// recycled framebuffer slot never answers for its predecessor. See the definition for
// why nothing prunes a dead row.
void NoteFramebufferTextureAttachments(MG_Pipe::MGPipeHandle fbo,
const MG_Pipe::MGPFramebufferState& record);
void NoteFramebufferTextureAttachments(MG_Pipe::MGPipeHandle fbo, const MG_Pipe::MGPFramebufferState& record);
// The record's surface for an attachment POINT, or null when this record does not
// describe that point at all (Color8..Color31, and the default framebuffer's FRONT/BACK
// tokens). MGPFramebufferState carries Color[8] + Depth + Stencil, which is every point
@@ -2436,7 +2431,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// makes forgetting impossible, and that call is written below.
//
// IT IS GATED, AND HERE IS THE HANDSHAKE, because the definition is `static` in E's file
// on the tree this package was built against (esprytdraw v2, DirectGLES.cpp:2789) and an
// on the tree this package was built against (esprytdraw v2; now in DirectGLESInternal.h) and an
// internal-linkage function cannot be called from Managers.cpp. E's verification round
// drops that one keyword; D's verification round flips this constant to 1, in this file,
// one line. Neither side can do it silently: the flip has no other reader and the
@@ -2539,7 +2534,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
namespace SamplerImpl {
class BackendSamplerObject; // for PrgramImpl's sampler-pass memo rows below
}
} // namespace SamplerImpl
// Image uniforms take their unit from the layout(binding=N) qualifier baked into
// the transpiled ESSL; unlike samplers they must not (and in ES cannot) be
@@ -2686,9 +2681,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
Bool PointSizeDemoted() const { return PointSizeWasDemoted; }
Uint GetUBOSize() const { return GlobalUboSize; }
const Vector<ShaderStage>& GetLinkedShaderStages() const { return LinkedStages; }
const UnorderedMap<String, Int>& GetShaderStorageBlockBindingOverrides() const {
return StorageOverrides;
}
const UnorderedMap<String, Int>& GetShaderStorageBlockBindingOverrides() const { return StorageOverrides; }
// THE DEBUG LOG IS DELETED ON THIS ARM (CONTRACT-P5E §5.5). The snapshot is
// GL-thread-owned ShaderObject SharedPtrs - the exact shape rule C forbids an
// applier entry point to reach - and its one consumer in the build is an MGLOG_D
@@ -2734,9 +2727,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// arrays is keyed by its full "[0]"-terminated spelling - which is why the
// suffixed lookup is tried before the trailing subscript is read as an index.
Int GetUniformLocation(const String& name) const;
Int GetActiveUniformBlocksCount() const {
return static_cast<Int>(Link->glBlockIndexToTProgram.size());
}
Int GetActiveUniformBlocksCount() const { return static_cast<Int>(Link->glBlockIndexToTProgram.size()); }
const String& GetUniformBlockName(Uint index) const;
GLenum GetTransformFeedbackBufferMode() const { return Link->xfbBufferMode; }
@@ -2878,20 +2869,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
// `layout(vertices = N) out`, so a program built for one patch size is stale for
// another and the draw path has to say so. -1 compares equal to itself for every
// program that has a control stage of its own, i.e. for all but a handful.
Int GetPassthroughTessControlPatchVertices() const {
return m_passthroughTessControlPatchVertices;
}
Int GetPassthroughTessControlPatchVertices() const { return m_passthroughTessControlPatchVertices; }
// GL_PATCH_DEFAULT_{OUTER,INNER}_LEVEL the same synthesized stage was built with, for
// the same reason: ES has neither the state nor an entry point to forward it to, so
// glPatchParameterfv's values are compiled in as literals and a program built with one
// set is stale for another. Meaningless (and never read) when the patch-vertices field
// above is -1, which is the gate the draw path tests first.
const FloatVec4& GetPassthroughTessControlOuterLevel() const {
return m_passthroughTessControlOuterLevel;
}
const FloatVec2& GetPassthroughTessControlInnerLevel() const {
return m_passthroughTessControlInnerLevel;
}
const FloatVec4& GetPassthroughTessControlOuterLevel() const { return m_passthroughTessControlOuterLevel; }
const FloatVec2& GetPassthroughTessControlInnerLevel() const { return m_passthroughTessControlInnerLevel; }
Bool HasGlobalUboBlock() const { return m_globalUboBackendBlockIndex >= 0; }
const Vector<Int>& GetUniformBlockBackendIndices() const { return m_uniformBlockBackendIndices; }
@@ -2969,10 +2954,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// own - which ES 3.2 rejects outright. Called from SyncToBackend after every real
// stage has been attached and before the link; see the definition for why it cannot
// regress a program that works today.
void AttachPassthroughTessControlStage(
const ProgramBuildSource& src, Int tessEvalShaderIndex,
const Vector<Vector<unsigned int>>& shaderSpirvs, const String& vertexStageEssl,
const String& tessEvalStageEssl);
void AttachPassthroughTessControlStage(const ProgramBuildSource& src, Int tessEvalShaderIndex,
const Vector<Vector<unsigned int>>& shaderSpirvs,
const String& vertexStageEssl, const String& tessEvalStageEssl);
// One stage's SPIR-V through the DirectGLES pass chain and SPIRV-Cross, producing
// the raw emitted ESSL and the interface blocks this stage's XFB flattening
@@ -2985,10 +2969,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
const ImageFormatBakeInputs& imageFormatBake,
const UnorderedMap<String, Int>& storageBlockBindingOverrides,
const std::map<String, String>& inputBlockRenames,
const std::map<String, String>& outputBlockRenames,
Bool stripInputBlockLocations, Bool stripOutputBlockLocations,
Int atomicCounterEsslBindingTop, Bool enableSpirvValidation,
String& outSource,
const std::map<String, String>& outputBlockRenames, Bool stripInputBlockLocations,
Bool stripOutputBlockLocations, Int atomicCounterEsslBindingTop,
Bool enableSpirvValidation, String& outSource,
std::set<String>& outFlattenedXfbBlockNames,
Vector<Int>& outAtomicCounterGlBindings, String& outError) const;
@@ -3193,7 +3176,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// bind_sampler_states; the caller that knows the unit passes it here.
//
// Defaulted so a caller that has no unit - the backend's OWN raw-depth-fetch
// sampler (DirectGLES.cpp:217), a SamplerObject the client has never seen and for
// sampler (GetRawDepthFetchSampler no longer calls this), a SamplerObject the client has never seen and for
// which no record can exist - keeps working: that arm reads the object, which is
// the authority for server-owned state. An APPLICATION sampler reaching here
// without a handle is the E-side call-site gap and says so once.
@@ -3309,9 +3292,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// look at it.
// P14 S6: and a bucket registry for the same reason the buffer one above is - a sampler
// view's twin is a driver object of the texture's group.
using BackendSamplerViewTable = SlotTableRegistry<MG_State::GLState::ITextureObject,
BackendSamplerViewObject,
MG_Pipe::MGPipeKind::SamplerViewCso>;
using BackendSamplerViewTable = SlotTableRegistry<MG_State::GLState::ITextureObject, BackendSamplerViewObject,
MG_Pipe::MGPipeKind::SamplerViewCso>;
extern BackendSamplerViewTable g_backendSamplerViews;
BackendSamplerViewObject* FindSamplerViewForHandle(MG_Pipe::MGPipeHandle view);
@@ -3358,7 +3340,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
Uint64 m_syncedResourceSerial = 0;
};
extern TwinRegistry<MG_State::GLState::RenderbufferObject, BackendRenderbufferObject, MG_Pipe::MGPipeKind::Renderbuffer>
extern TwinRegistry<MG_State::GLState::RenderbufferObject, BackendRenderbufferObject,
MG_Pipe::MGPipeKind::Renderbuffer>
g_backendRenderbufferObjects;
} // namespace RenderbufferImpl
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,675 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/ManagersInternal.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "Managers.h"
#include "SubgroupPolicy.h"
#include <MG_Backend/MGPipe/PipeInputs.h>
// P3a: the handle-shaped resource op table, the applier's records and the reverse channel.
// Both headers are compiled into the library only under push, so they are included here
// under the same condition - a pull build must gain no declaration it cannot link.
#include <MG_Pipe/MGPipeCallbacks.h>
#include <MG_Pipe/MGPipeHostSpan.h>
#include <MG_Pipe/PipeApply.h>
// P5e (pg): the DEFINITION of the archive MGPipeShaderCsoRecord holds by SharedPtr. PipeApply.h
// needs only the name - the deleter is captured where the archive is constructed - but the
// program build reads through it, so the one translation unit that does needs the whole type.
#include <MG_State/GLState/ProgramState/ProgramArtifactsCodec.h>
// R-11's server-owned staging copy. Header-only and package v1's; see its own header block for
// why GLESBufferResource does not simply gain a member.
#include <MG_Backend/Record/StagedShadow.h>
#include <MG_Backend/Record/StagedTextureStore.h>
#if MOBILEGL_BUILD_DISAGGREGATED
#include <MG_Remote/Server/ServerLoop.h>
// P5c (ct): object_death's producer (CONTRACT-P5C.md §5.2) - the death notice's split arm
// emits the record through the client's emit helper instead of hopping a stack struct to the
// apply thread.
#include <MG_Remote/Client/WireTables.h>
// P11 B2 (T0): the client's AHardwareBuffer store - its reference counting and, on Android, the
// allocation the POST self-test makes.
#include <MG_Remote/Transport/AdoptT0.h>
#if defined(__ANDROID__)
#include <android/hardware_buffer.h>
#endif
#endif
#include "Utils.h"
#include "DirectGLES.h"
#include "BackendObject_DirectGLES.h"
#include <Config.h>
#include <MG_State/GLState/StateObjectDeathNotice.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/TranslationCache.h>
#include <MG_Util/BackendLoaders/OpenGL/Loader.h>
#include <MG_Util/Metrics/PipeStats.h>
#include <MG_Util/Converters/GLToStr/GLEnumConverter.h>
#include <MG_Util/Converters/MGToGL/DataTypeConverter.h>
#include <MG_Util/Converters/MGToGL/BufferEnumConverter.h>
#include <MG_Util/Converters/GLToMG/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/Converters/MGToGL/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToStr/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToStr/FramebufferEnumConverter.h>
#include <MG_State/GLState/TextureState/TextureObjectBuffer.h>
#include <MG_Util/Converters/GLToMG/FramebufferEnumConverter.h>
#include <MG_Util/Converters/MGToGL/FramebufferEnumConverter.h>
#include <MG_State/GLState/FramebufferState/FramebufferObject.h>
#include <algorithm>
#include <cctype>
#include <cmath>
#include <cstdlib>
#include <limits>
#include <map>
#include <mutex>
#include <cstring>
#include <regex>
#include "ContextEpoch.h"
namespace MobileGL::MG_Backend::DirectGLES {
constexpr const char* BASE_INSTANCE_UNIFORM_NAME = "mg_BaseInstance";
constexpr const char* DRAW_ID_UNIFORM_NAME = "mg_DrawID";
constexpr const char* BASE_VERTEX_UNIFORM_NAME = "mg_BaseVertex";
constexpr const char* BASE_INSTANCE_WORD_INDEX_UNIFORM_NAME = "mg_BaseInstanceWordIndex";
constexpr const char* INDIRECT_PARAMS_BLOCK_NAME = "mg_IndirectParams";
Bool ViewportArrayEmulationEnabled();
GLenum ResolveBackendMinFilter(const SamplerParameters& samplerParams, Bool avoidMipmapMinFilter);
String EmulateBaseInstanceInVertexShader(String source, GLenum shaderType);
String PromoteDrawParameterGlobalsToUniforms(String source, GLenum shaderType);
constexpr const char* VIEWPORT_PASS_MASK_UNIFORM_NAME = "mg_ViewportPassMask";
Bool PromoteViewportIndexGlobalToVarying(String& source);
Bool InjectViewportIndexPassGate(String& source);
namespace BufferImpl {
using MG_State::GLState::BackendBufferResource;
using MG_State::GLState::BufferObject;
extern Uint g_bufferContextGeneration;
// --- Persistent-mapped bump rings (see Managers.h) -----------------------
// Shared machinery behind BOTH the global-UBO ring and the texture
// unpack-PBO ring: one EXT_buffer_storage persistent|coherent map per ring,
// monotonic head/tail cursors, and reclamation riding the Present()
// frame-fence watermark. The two rings differ only in size cap, offset
// alignment and log label - the reclamation, context-loss and
// emergency-drain rules are the part that was hard to get right, so they
// are shared rather than copied.
constexpr SizeT kUboRingInitialBytes = 4u * 1024u * 1024u;
constexpr SizeT kUboRingMaxBytes = 64u * 1024u * 1024u;
constexpr SizeT kUnpackRingInitialBytes = 4u * 1024u * 1024u;
constexpr SizeT kUnpackRingMaxBytes = 64u * 1024u * 1024u;
// A PBO-sourced glTexSubImage constrains the offset only to the pixel
// TYPE's size (GL_INVALID_OPERATION otherwise), and no ES client type is
// wider than 4 bytes - unlike a UBO bind, which owes the driver
// GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT. 64 covers every type with room to
// spare and keeps consecutive staged blocks off each other's cache lines.
constexpr SizeT kUnpackRingAlignment = 64;
// glCopyBufferSubData carries no offset-alignment requirement at all; 64
// keeps staged blocks cache-line separated, same as the unpack ring.
constexpr SizeT kUploadRingInitialBytes = 4u * 1024u * 1024u;
constexpr SizeT kUploadRingMaxBytes = 64u * 1024u * 1024u;
constexpr SizeT kUploadRingAlignment = 64;
struct PersistentRingStore {
Uint id = 0;
Uint8* mappedPtr = nullptr;
SizeT size = 0;
// Monotonic linear cursors: `head` counts every byte ever allocated
// (incl. wrap padding); everything below `tail` is GPU-complete. Ring
// offset of a linear position is pos % size, so in-flight bytes are
// head - tail and must stay <= size.
Uint64 head = 0;
Uint64 tail = 0;
Uint32 generation = 0; // bumped on every (re)create/grow; 0 = never valid
Uint contextGeneration = 0;
SizeT alignment = 256;
// A hard storage-creation failure under this context; stop retrying
// per use (cleared when the context generation moves on).
Bool creationFailed = false;
};
// Grown-away ring stores: deletable only once the GPU finished the last
// frame that could reference them (same watermark as the buffer pool).
struct RetiredRingStore {
Uint id = 0;
Uint contextGeneration = 0;
Uint64 retireSerial = 0;
};
// Present()-time high-water marks: every byte below headAtPresent was
// written during frames <= frameSerial, so once frameSerial completes,
// tail may advance to headAtPresent. FIFO by construction.
struct RingFrameMark {
Uint64 frameSerial = 0;
Uint64 headAtPresent = 0;
};
// One ring: its live store, its two reclamation lists, and the immutable
// knobs that tell it apart from the other one.
struct PersistentRing {
PersistentRingStore store;
Vector<RetiredRingStore> retired;
Vector<RingFrameMark> frameMarks;
SizeT initialBytes = 0;
SizeT maxBytes = 0;
// 0: take the offset alignment from GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT
// at store-creation time (the UBO ring's binds require it).
SizeT fixedAlignment = 0;
const char* label = "";
};
// THE RINGS ARE PER NATIVE CONTEXT. Their slots are reclaimed by frame marks recorded at
// Present, which say "everything this context wrote before its fence N is done once N is".
// One process-wide set used to be shared by every session's contexts: a fence from one
// context retired bytes another context's not-yet-executed copies still had to read
// (a glTexSubImage from the unpack ring is a deferred GPU read), and the next session's
// staging overwrote them - its pixels landed, at the wrong offsets, in the first one's
// textures. With one set per context, a context's bytes wait for that context's own
// fence. (The buffers themselves are shared objects; only who stages into them is split.)
// Staging ring for app buffer updates whose destination store may still be
// referenced by in-flight GPU work. Mali's glBufferSubData resolves that WAR
// hazard by BLOCKING in the call (osup_sync_object_wait) until every
// referencing job retires - under Minecraft 26.3's per-frame UBO and
// chunk-mesh SubData streams that serialized whole frames (~1 fps while
// chunks stream in). Staging the bytes here and issuing a
// glCopyBufferSubData instead keeps the hazard on the GPU timeline where it
// is just job ordering, and the CPU never waits.
struct RingSet {
PersistentRing ubo{{}, {}, {}, kUboRingInitialBytes, kUboRingMaxBytes, 0, "Global-UBO ring"};
PersistentRing unpack{
{}, {}, {}, kUnpackRingInitialBytes, kUnpackRingMaxBytes, kUnpackRingAlignment, "Texture unpack ring"};
PersistentRing upload{
{}, {}, {}, kUploadRingInitialBytes, kUploadRingMaxBytes, kUploadRingAlignment, "Buffer upload ring"};
RingSet();
~RingSet();
RingSet(const RingSet&) = delete;
RingSet& operator=(const RingSet&) = delete;
};
extern DirectGLES::PerNativeContext<RingSet> g_ringSets;
namespace {
Bool CanTouchGLNow() {
return DirectGLES::IsBackendContextCurrentOnThisThread();
}
} // namespace
MG_Record::StagedShadowStore& ServerStaged();
namespace {
// THE COVERAGE RULE FOR THE THREE-TIER FLUSH DRAIN, ASSERTED BEFORE THE CALL AND NOT
// INSIDE IT. FlushPendingRangesFrom is a G5-pinned body (p3a_untouched_regions.sh's
// PINNED_FUNCTIONS, ID-41): the split arm CALLS it, it does not re-spell it, and it
// does not add a line to it either. Its tier-1 arm - the range-invalidating map -
// copies with its own Memcpy and never reaches UploadRangeFrom, so the one tier that
// DECLARES THE OLD BYTES DEAD is the one tier no later check can see; the only place
// left to say "every queued range is inside the staged coverage" is the instant before
// the drain takes the queue. The clamp is the drain's own (limit = the smaller of the
// frontend size and the backend store; bytes past either end have nowhere to land and
// are not this rule's subject), so the two agree about which bytes are meant.
//
// Fires only for a base that IS this resource's server shadow: RequireCoverage
// answers nothing for the legacy arm's MappedData(), which is valid for the whole
// store. `pendingMutex` is taken here and released before the drain takes it again.
void RequireStagedCoverageForPendingRanges(GLESBufferResource& resource, const Uint8* hostBase,
SizeT frontendSize, const char* site) {
if (hostBase == nullptr) return;
const SizeT limit = std::min(frontendSize, resource.storageSize);
const std::lock_guard<std::mutex> lock(resource.pendingMutex);
for (const auto& range : resource.pendingRanges) {
const SizeT end = std::min(range.end, limit);
const SizeT start = std::min(range.start, end);
if (start == end) continue;
ServerStaged().RequireCoverage(&resource, hostBase, start, end, site);
}
}
// The four hostBytes sites read the same in both builds. The non-split expansion is the
// ORIGINAL EXPRESSION, character for character - `raw - offset` - so a push or verify build
// compiles exactly what it compiled before R-11 and the split arm is the only new behaviour.
#define MGL_SERVER_STAGED_ADOPT(res, width, bytes, offset, size) \
ServerStaged().Adopt(&(res), (width), (bytes), (offset), (size))
#define MGL_SERVER_STAGED_REQUIRE_PENDING(res, base, frontendSize, site) \
RequireStagedCoverageForPendingRanges((res), (base), (frontendSize), (site))
} // namespace
// Ring machinery shared with the UBO/unpack rings; defined further down in
// this same unnamed namespace.
Bool RingAllocate(PersistentRing& ring, SizeT size, SizeT& outOffset);
// Push every queued range of `resource` from the shadow into the backend
// store, without ever letting a driver resolve the WAR hazard against
// in-flight frames at the WHOLE BUFFER's expense. Three tiers:
//
// 1. glMapBufferRange(WRITE | INVALIDATE_RANGE) + memcpy. The entire
// mapped range is rewritten from the authoritative shadow, so
// declaring its old bytes dead is exact - and it lets the driver
// swap fresh pages in for JUST that range. This is the only tier
// whose cost scales with the RANGE on this Mali driver: both the
// immediate glBufferSubData (pre-queueing) and a staged
// glCopyBufferSubData into a busy MUTABLE store ghost the whole
// destination with a worker-thread memcpy - Minecraft 26.3 streams
// ~1MB section meshes into 128MB arenas about nine times a frame
// during a camera pan, and 9 x 128MB of ghosting per frame is
// ~380ms, the measured 2-4 fps. (Backing the arenas with immutable
// stores also kills the ghost, but eagerly commits every arena's
// full extent - +hundreds of MB - which LMK'd the whole device.)
// 2. The staging ring + glCopyBufferSubData: the copy is ordered on
// the GPU timeline, no CPU wait (MOBILEGL_ESPRYT_DISABLE_INVALIDATE_FLUSH
// forces this tier as the map path's negative control).
// 3. Direct glBufferSubData (potentially stalling) when neither the
// map entry points nor the ring exist.
//
// The ranges are flushed AS QUEUED (VecRange1D::Add already merges
// near-adjacent ones): bytes, not flush calls, are the cost axis here,
// and collapsing a scattered flush into its union re-copied nearly whole
// chunk-mesh arenas every frame.
// The caller owns syncedChangeSerial; this only drains the queue.
//
// ONE body for both arms, and deliberately so: the three-tier decision (whole-buffer
// orphan map / >= 128 KiB range-invalidating map / staged ring copy) is what the MC
// 26.3 p99 depends on, and a second copy of it for the handle arm is exactly how a
// tier silently changes. The arms differ only in where `hostBase` and `frontendSize`
// come from.
//
// R-11'S PARAMETER, AND IT IS A PARAMETER RATHER THAN AN ASSUMPTION (P5 b1).
// `hostBase` is re-read at every use today precisely because a shadow resize or an
// adoption moves what an earlier base pointed at. Under split the server may hold
// no pointer into the client's shadow at all, so the base becomes a SNAPSHOT taken
// into SEG_STAGE at emission - and a snapshot covers a RANGE, not the store. The
// two extra arguments say which range `hostBase` is good for; the default is the
// whole store, which is exactly what a live shadow is, so every caller today is
// byte-identical. The moment w1 passes a real snapshot extent, tier 1's widening
// refusal below stops being unreachable and a too-narrow snapshot is named instead
// of silently clobbering GPU-written bytes with stale ones.
constexpr SizeT kHostBaseCoversWholeStore = ~static_cast<SizeT>(0);
void FlushPendingRangesFrom(GLESBufferResource& resource, const Uint8* hostBase, SizeT frontendSize,
SizeT hostBaseFrom = 0, SizeT hostBaseTo = kHostBaseCoversWholeStore);
void DrainResidentWritesNow(GLESBufferResource& resource);
namespace {
// The two-argument spelling the legacy arm's call sites use, kept so those sites
// are the SAME text in both builds (G1). The object was never read.
void DrainResidentWritesNow(GLESBufferResource& resource, BufferObject& bufferObject) {
(void)bufferObject;
DrainResidentWritesNow(resource);
}
// ---- P3a: the same seven ops plus the create/unmap pair, BY HANDLE ------------
//
// Every body below is its Ops_* counterpart above with exactly the substitutions
// D-A2's table names and nothing else: the frontend object's GetSize() / GetUsage()
// / HasDefinedContent() / MappedData() / GetChangeSerial() become the applier
// record's Desc.Width / Desc.Usage / Desc.HasDefinedContent, the shadow base the
// call carried, and the record's server-owned Serial. Every branch survives -
// pendingRespecify early-out, the off-thread queue, the adopted zero-copy stamp,
// the upload-ring kill switch, the Mali WAR-stall queue-only default.
//
// NOTHING here reads a frontend type. The one thing a handle op cannot do is ask
// an object for its shadow, which is why GLESBufferResource::hostBytes exists: the
// three content-carrying calls hand the base over and the later drains read it.
// The applier's record for this resource, or null when the client never created it
// (or created it into a slot that has since been recycled).
const MG_Pipe::MGPipeResourceRecord* ResourceRecordOf(MG_Pipe::MGPipeHandle res) {
if (MG_Pipe::MGPipeHandleIsNull(res)) return nullptr;
const auto& records = MG_Pipe::MGPipeApplier().Resources;
if (res.Slot >= records.size()) return nullptr;
const auto& record = records[res.Slot];
if (!record.Live || record.Gen != res.Gen) return nullptr;
return &record;
}
// The server-owned MGGen this backend mirrors in syncedChangeSerial. Zero for a
// resource with no record, which is the same "never synced" answer a frontend
// change serial of zero gave.
Uint64 ResourceSerialOf(MG_Pipe::MGPipeHandle res) {
const auto* record = ResourceRecordOf(res);
return record != nullptr ? record->Serial : 0;
}
SizeT ResourceWidthOf(MG_Pipe::MGPipeHandle res) {
const auto* record = ResourceRecordOf(res);
return record != nullptr ? static_cast<SizeT>(record->Desc.Width) : 0;
}
} // namespace
#if MOBILEGL_BUILD_DISAGGREGATED && defined(__ANDROID__)
using MG_Util::BackendLoader::HasExtensionWord;
#endif // MOBILEGL_BUILD_DISAGGREGATED && __ANDROID__
const Vector<TextureUploadTarget>& StagedUploadTargetsForPipeTarget(Uint8 pipeResourceTarget);
TextureTarget StagedTextureTargetForPipeTarget(Uint8 pipeResourceTarget);
void BumpBufferMutationEpoch();
GLESBufferResource* FindBufferResourceForHandle(MG_Pipe::MGPipeHandle res);
SizeT ResourceWidthForHandle(MG_Pipe::MGPipeHandle res);
Uint64 ResourceSerialForHandle(MG_Pipe::MGPipeHandle res);
GLESBufferResource* EnsureBufferResourceForHandle(
const SharedPtr<MG_State::GLState::BufferObject>& bufferObject, MG_Pipe::MGPipeHandle res);
void BindBufferId(GLenum target, Uint id);
void BindPixelUnpackBufferId(Uint id);
void NoteBufferIdDeleted(Uint id);
Bool CreateRingStorage(PersistentRing& ring, SizeT minBytes);
Bool RingAllocateSlow(PersistentRing& ring, SizeT size, SizeT& outOffset);
Bool RingAllocate(PersistentRing& ring, SizeT size, SizeT& outOffset);
Bool RingAllocateSlow(PersistentRing& ring, SizeT size, SizeT& outOffset);
Bool UnpackRingAvailable();
Bool UnpackRingAllocate(SizeT size, SizeT& outOffset);
void* UnpackRingMappedPtr();
Uint UnpackRingBufferId();
SizeT UnpackRingMaxBytes();
} // namespace BufferImpl
// The release-build VOICE for StateBackendObjectRegistry::GetOrCreateByHandle's two
// silent refusals, shared by the kinds P4a re-keys so the wording cannot drift between
// them. It is the shape GetOrCreateBufferResourceForHandle gives P3a's buffer family,
// lifted into one template because a copy per kind is a chance per kind to write one of
// them differently.
//
// ITS CALLERS IN THIS PACKAGE ARE THE FRAMEBUFFER ATTACHMENT WALK'S TWO
// (FramebufferImpl::SyncAttachmentSurface: the texture registry and the renderbuffer
// registry), because that is where a HANDLE arrives in a payload - MGPSurface::Res -
// and an ADOPTION is what the arm owes rather than a lookup. The other three re-keyed
// kinds resolve at sites in DirectGLES.cpp (package E's SyncCurrentFBO, the lazy
// sampler mint and SyncCurrentProgram), so their first adoption is E's to write; this
// template is what they call so all five say the same thing.
//
// WHY A VOICE AT ALL: the table asserts, and MOBILEGL_ASSERT compiles out at INFO -
// which all three gate builds and every shipped build are - so an object that silently
// stops being twinned is exactly the failure mode the refusal exists to replace.
//
// Returns null on the legacy arm too, so a caller that reaches this without checking
// its family's arm gets nothing rather than a twin the legacy arm would never see.
//
// SPLIT IN TWO (review N-3), and the split is what makes the two refusals REACHABLE.
// Both of this package's call sites stand behind a cross-check that the record's Res
// equals registry.HandleOf(frontendObject) - the LIVE handle - so by the time the
// adoption ran, the slot was in bounds and the generation was current BY CONSTRUCTION
// and neither refusal below could ever fire: the letter of M-4 was closed and its
// substance was not. The record's handle is the UNTRUSTED input and the frontend is the
// corroboration, so the validity question is asked FIRST, on its own, at both sites;
// the adoption then performs the resolve. Asking it first is also side-effect free,
// which the ordering rule N-1 states for the framebuffer refusal wants here too:
// GetOrCreateByHandle can reset an incumbent twin, and a handle this predicate rejects
// must not reach it.
template <typename Registry>
Bool PipeTwinHandleIsAdoptable(Registry& registry, MG_Pipe::MGPipeHandle handle, const char* kindName) {
if (MG_Pipe::MGPipeHandleIsNull(handle)) return false;
if (handle.Slot >= Registry::SlotTable::kMaxHandleSlot) {
#if MOBILEGL_BUILD_DISAGGREGATED
MG_Pipe::MGPipeSessionFail( // @Ph-declined (ID-P7-1): PH-2 stays Fatal, CONTRACT-P7 §12
MG_Pipe::MGPipeFatalFamily::ProtocolCorruption,
"MGPipe: Fatal{ProtocolCorruption, \"BackendSlotTable.HandleSlot\"} - "
"%s handle slot %u is past this table's %u bound",
kindName, handle.Slot, Registry::SlotTable::kMaxHandleSlot);
#else
MGLOG_E_ONCE("MGPipe: %s handle slot %u is past the backend table's %u bound - "
"refusing to twin it",
kindName, handle.Slot, Registry::SlotTable::kMaxHandleSlot);
return false;
#endif
}
const Uint32 liveGen = registry.LiveGenAt(handle.Slot);
if (liveGen != 0 && liveGen > handle.Gen) {
#if MOBILEGL_BUILD_DISAGGREGATED
MG_Pipe::MGPipeSessionFail( // @Ph-declined (ID-P7-1): PH-2 stays Fatal, CONTRACT-P7 §12
MG_Pipe::MGPipeFatalFamily::ProtocolCorruption,
"MGPipe: Fatal{ProtocolCorruption, \"BackendSlotTable.Generation\"} - "
"%s handle {%u, %u} names generation behind live generation %u",
kindName, handle.Slot, handle.Gen, liveGen);
#else
// SlotTables.h:301-321: forward is a recycle and resets the twin, BACKWARD is
// refused, because adopting it would release the incumbent LIVE twin's driver
// id and then stamp the slot back to the dead object's generation, after which
// the incumbent's own FindByHandle refuses it and it is silently handed a
// fresh, empty twin - a leak AND an object that loses its storage with no
// diagnostic. That is the shape commit d7655247 fixed on the buffer side.
MGLOG_E_ONCE("MGPipe: %s handle {%u, %u} names a generation BEHIND the live "
"twin's %u - refusing rather than dropping the incumbent's driver "
"object",
kindName, handle.Slot, handle.Gen, liveGen);
return false;
#endif
}
return true;
}
namespace {
template <typename Registry>
typename Registry::BackendPtr* AdoptTwinByHandle(Registry& registry, MG_Pipe::MGPipeHandle handle,
const char* kindName) {
if (!PipeTwinHandleIsAdoptable(registry, handle, kindName)) return nullptr;
return registry.GetOrCreateByHandle(handle);
}
} // namespace
// ---- P4a: the record readers (Managers.h documents the contract) ----
namespace {
// One shape for all five, so a kind cannot grow a different liveness rule than its
// neighbours: index the applier's own dense table, refuse a slot it has not grown to,
// refuse a record that is not Live, and refuse a generation that has moved on under
// the caller. The last of the three is the one that matters: a stale handle resolving
// to its successor's record is how a twin ends up describing another object's storage.
template <typename Record>
const Record* PipeRecordAt(const Vector<Record>& table, MG_Pipe::MGPipeHandle handle) {
if (MG_Pipe::MGPipeHandleIsNull(handle)) return nullptr;
if (handle.Slot >= table.size()) return nullptr;
const Record& record = table[handle.Slot];
if (!record.Live || record.Gen != handle.Gen) return nullptr;
return &record;
}
} // namespace
const MG_Pipe::MGPipeResourceRecord* PipeTextureRecordForHandle(MG_Pipe::MGPipeHandle res);
const MG_Pipe::MGPipeSamplerCsoRecord* PipeSamplerCsoRecordForHandle(MG_Pipe::MGPipeHandle cso);
void NotePipeSubDataTargetLowByte(const MG_Pipe::MGPipeResourceRecord& record, Uint16 packedTarget, Uint16 level);
const MG_Pipe::MGPipeResourceRecord::PendingUpload* FindPipeTextureUpload(
const MG_Pipe::MGPipeResourceRecord& record, Uint16 uploadTarget, Uint16 level);
void ConsumePipeTextureUpload(MG_Pipe::MGPipeHandle res, Uint16 uploadTarget, Uint16 level);
Bool RearmPipeTextureLevelUpload(MG_Pipe::MGPipeHandle res, Uint16 packedTarget, Uint16 level,
const MG_Pipe::MGPBox& wholeLevel);
namespace TextureImpl {
// Sets the backend GL unpack state to MobileGL's upload default for the scope,
// then restores it. The previous state is read from a shadow instead of via
// glGetIntegerv - that query forces a driver pipeline sync and, because texture
// uploads run it per dirty texture per frame, it dominated the DirectGLES draw
// path. The backend unpack state is set ONLY by MobileGL's own save/restore
// helpers (this class and, historically, the R32F copy path), all of
// which restore to the resting default, so the shadow stays accurate; a one-time
// forced sync pins the backend to that known default up front. Apply() is
// compare-and-set, so the (now redundant) glPixelStorei calls also usually no-op.
class ScopedDefaultUnpackState {
public:
ScopedDefaultUnpackState() {
EnsureShadowSynced();
m_prevAlignment = s_alignment;
m_prevRowLength = s_rowLength;
m_prevSkipRows = s_skipRows;
m_prevSkipPixels = s_skipPixels;
m_prevImageHeight = s_imageHeight;
m_prevSkipImages = s_skipImages;
// Shadow mip data is tightly packed (ProcessTexturePixelsDataUnpack emits
// width * bpp rows with no padding), so uploads must use UNPACK_ALIGNMENT = 1.
// Alignment 4 made the driver read e.g. 7-byte R8 rows at an 8-byte stride,
// shifting every row of a non-multiple-of-4 upload by one pixel.
Apply(1, 0, 0, 0, 0, 0);
}
~ScopedDefaultUnpackState() {
Apply(m_prevAlignment, m_prevRowLength, m_prevSkipRows, m_prevSkipPixels, m_prevImageHeight,
m_prevSkipImages);
}
private:
static void EnsureShadowSynced() {
#if MOBILEGL_BUILD_DISAGGREGATED
const auto epoch = CurrentContextEpoch();
if (s_epoch != epoch) {
s_synced = false;
s_epoch = epoch;
}
#endif
if (s_synced) {
return;
}
s_synced = true;
g_GLESFuncs.glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
g_GLESFuncs.glPixelStorei(GL_UNPACK_ROW_LENGTH, 0);
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_ROWS, 0);
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_PIXELS, 0);
g_GLESFuncs.glPixelStorei(GL_UNPACK_IMAGE_HEIGHT, 0);
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_IMAGES, 0);
s_alignment = 4;
s_rowLength = 0;
s_skipRows = 0;
s_skipPixels = 0;
s_imageHeight = 0;
s_skipImages = 0;
}
static void Apply(GLint alignment, GLint rowLength, GLint skipRows, GLint skipPixels, GLint imageHeight,
GLint skipImages) {
if (alignment != s_alignment) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_ALIGNMENT, alignment);
s_alignment = alignment;
}
if (rowLength != s_rowLength) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_ROW_LENGTH, rowLength);
s_rowLength = rowLength;
}
if (skipRows != s_skipRows) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_ROWS, skipRows);
s_skipRows = skipRows;
}
if (skipPixels != s_skipPixels) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_PIXELS, skipPixels);
s_skipPixels = skipPixels;
}
if (imageHeight != s_imageHeight) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_IMAGE_HEIGHT, imageHeight);
s_imageHeight = imageHeight;
}
if (skipImages != s_skipImages) {
g_GLESFuncs.glPixelStorei(GL_UNPACK_SKIP_IMAGES, skipImages);
s_skipImages = skipImages;
}
}
GLint m_prevAlignment = 4;
GLint m_prevRowLength = 0;
GLint m_prevSkipRows = 0;
GLint m_prevSkipPixels = 0;
GLint m_prevImageHeight = 0;
GLint m_prevSkipImages = 0;
// Shadow of the backend GL unpack state (GL defaults). See class comment.
static inline Bool s_synced = false;
#if MOBILEGL_BUILD_DISAGGREGATED
static inline ContextEpoch s_epoch{};
#endif
static inline GLint s_alignment = 4;
static inline GLint s_rowLength = 0;
static inline GLint s_skipRows = 0;
static inline GLint s_skipPixels = 0;
static inline GLint s_imageHeight = 0;
static inline GLint s_skipImages = 0;
};
void NoteInPlaceTextureUpload();
Uint GetWidenableClientComponentCount(TextureInternalFormat format);
Bool IsIntegerWidenableFormat(TextureInternalFormat format);
// P5e (tx2): the GL name a log line wants, from whichever side of the arm can answer it.
// Desc.GlNameForDiag is the declared diagnostics carrier (MGPipeTypes.h) and it is the
// ONLY thing the by-handle arm may say about a texture's frontend name - a GL name is
// never an identity, never a memo key and never part of a hash (section 4.2.1). The
// twenty-odd GetExternalIndex() log reads this family carried (scout G-S2-4) all become
// this call.
static Uint32 TextureDiagName(const SharedPtr<MG_State::GLState::ITextureObject>& stateTextureObject,
const MG_Pipe::MGPipeResourceRecord* record) {
if (stateTextureObject) return static_cast<Uint32>(stateTextureObject->GetExternalIndex());
return record != nullptr ? record->Desc.GlNameForDiag : 0u;
}
// P5e (fix1, ID-81 / CONTRACT-P5E §5.8): THE NULL FRONTEND OBJECT IS A CONTRACT AND THIS
// IS WHERE IT IS CHECKED, ONCE, FOR ALL THREE SYNC BODIES.
//
// tx2's three sync bodies accept a null `stateTextureObject` because the RECORD answers
// every read they make - but only where the record arm is actually SELECTED. Two reads
// inside SyncMipmapsToBackend are gated on `Transport != Monolith` (the texture-VIEW test
// and RequireImageBindableStorage's re-dirty transition, §5.2), and every read in all
// three is gated on TextureResourceSubsystemEnabled() through `pushedStorage`. So a null
// object is servable exactly when BOTH hold, and the by-handle entries beside them
// (SyncTextureToBackendByHandle / SyncMipmapsToBackendByHandle) pass null on the strength
// of a NOTED HANDLE, which says nothing about either.
//
// The device found the gap: with MOBILEGL_TRANSPORT=monolith the push build still drives
// its framebuffer attachments from the record (P4a, D-C2), fb's SyncAttachmentSurface
// called the by-handle storage sync from that arm, and `stateTextureObject->IsTextureView()`
// dereferenced the null SharedPtr in Lightmap.<init> -> clearColorTexture -> glClear.
// The arm selection is repaired at that call site; this refusal is what makes the shape
// UNREPEATABLE rather than merely repaired - a null frontend object reaching a body that
// can still read one aborts BY NAME instead of taking a SIGSEGV three frames deep.
//
// Returns true when the caller must decline (no handle noted: the caller's bug it always
// was, named by the caller's own log line), false when the record arm can serve the call,
// and never returns at all when a handle was noted on an arm that cannot serve it.
static Bool RefuseNullFrontendTextureOffTheHandleArm(const char* entry, MG_Pipe::MGPipeHandle notedHandle) {
if (MG_Pipe::MGPipeHandleIsNull(notedHandle)) return true;
return false;
}
// The guard the three sync bodies spell. In the PULL build it folds to the constant the
// pre-fix1 text folded to, so that build's preprocessed text gains no call (D-P).
#define MGB_TEXTURE_NULL_FRONTEND_REFUSED(twin, entry) \
(RefuseNullFrontendTextureOffTheHandleArm((entry), (twin).PushedSyncHandle()))
extern Array<Array<BackendTextureObject*, (SizeT)TextureTarget::TextureTargetCount>,
MG_State::GLState::TextureState::MAX_TEXTURE_IMAGE_UNITS>
g_boundTexturesCache;
void NoteDriverSideTextureWriteByHandle(MG_Pipe::MGPipeHandle res, Bool storeFollowsTheWrite);
} // namespace TextureImpl
namespace ScratchFBOImpl {
void NoteTextureIdDeleted(Uint textureId);
} // namespace ScratchFBOImpl
} // namespace MobileGL::MG_Backend::DirectGLES
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,267 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/ManagersSamplers.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ManagersInternal.h"
namespace MobileGL::MG_Backend::DirectGLES {
static Bool ShouldAvoidSamplerMipmapMinFilterOnAngleLlvmpipe() {
// IsAngleLlvmpipeRenderer combined with the
// MOBILEGL_ESPRYT_AVOID_SAMPLER_MIPMAP_MIN_FILTER feature toggle,
// both resolved in FillInGLESCapabilities.
return g_GLESCapabilities.AvoidSamplerMipmapMinFilter;
}
namespace SamplerImpl {
BackendSamplerObject::BackendSamplerObject() {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
g_GLESFuncs.glGenSamplers(1, &m_backendSamplerId);
m_contextGeneration = g_backendContextGeneration;
if (m_backendSamplerId == 0) {
MGLOG_E_ONCE("Failed to generate sampler object.");
MGLOG_E_ONCE("ES glGetError(): %s", MG_Util::ConvertGLEnumToString(g_GLESFuncs.glGetError()).c_str());
} else {
MGLOG_D("Generated sampler object with ID: %u.", m_backendSamplerId);
}
}
BackendSamplerObject::~BackendSamplerObject() {
if (InProcessTeardown()) {
return; // see InProcessTeardown(): the driver may be unloaded already
}
if (m_backendSamplerId == 0) {
return;
}
// Scrub the unit shadow whether or not the id can still be deleted - the next
// twin can land on this heap address and would otherwise false-skip its Bind.
for (auto& boundSampler : g_boundSamplersCache) {
if (boundSampler == this) {
boundSampler = nullptr; // glDeleteSamplers unbinds from every unit
}
}
if (m_contextGeneration == g_backendContextGeneration && g_GLESFuncs.glDeleteSamplers) {
g_GLESFuncs.glDeleteSamplers(1, &m_backendSamplerId);
}
m_backendSamplerId = 0;
}
void BackendSamplerObject::SyncToBackend(const SharedPtr<MG_State::GLState::SamplerObject>& stateSamplerObject,
MG_Pipe::MGPipeHandle pushedCso) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
if (!stateSamplerObject) {
// A CSO TWIN HAS NO FRONTEND OBJECT (P4a fable seam F-4, ResolveSamplerCsoTwin):
// the content-addressed handle is its identity and the applier's record its
// only authority, so a null object beside a live handle is the record arm and
// not the pre-P4a error. Everything below that names the object is guarded on
// it; the pull build's text is the three lines the #if brackets.
if (MG_Pipe::MGPipeHandleIsNull(pushedCso)) {
MGLOG_E_ONCE("State sampler object is null, cannot sync to backend.");
return;
}
}
// P4a (D-F1): a SamplerObject is a pure 100-byte value with no driver-side per-object
// binding state, so its CSO is CONTENT-ADDRESSED on the client at capacity 256 and
// two identical samplers share one record. What that means here is that the record's
// server-owned Serial, not the frontend object's version, is what says the values
// moved - and that the values themselves cross byte for byte INCLUDING
// borderColorForm, which is why all four border comparands below are still compared.
//
// The whole arm choice is a preprocessor #if/#else so the PULL build's text is the
// pre-P4a text token for token (D-P).
const SamplerParameters* pushedParams = nullptr;
// The GL name for the two log lines below, or 0 for a CSO twin, which has no object
// to name (F-4): the handle in the same line is its name.
const Uint samplerName = stateSamplerObject ? stateSamplerObject->GetExternalIndex() : 0u;
// THE HANDLE COMES FROM THE CALLER, NOT FROM THIS TWIN'S REGISTRY, and the
// difference is the seam D's verification round found on the integrated tree.
//
// v2 asked g_backendSamplerObjects.HandleOf(object) - the twin's own identity
// handle, minted off the frontend lifetime id - and looked the record up by it.
// That can only ever miss: a SamplerCso is CONTENT-ADDRESSED on the client
// (D-F1, ID-14/ID-17), its handles come out of MGPipeSlots().Allocate keyed on a
// parameter hash, and nothing ever emits a create_sampler_state at an identity
// handle. On the integrated tree every glBindSampler-driven parameter set was
// therefore refused and the driver sampler kept its defaults - two integration
// scenarios red, and the twin's own comment below (content-addressed, the
// record's Serial is the authority) already said why it could not work.
//
// The carried fact is MGPipeApplier().BoundSamplerStates[unit], written by the
// client at bind_sampler_states; the caller that knows the unit passes it as
// `pushedCso`. See the declaration for why the parameter is push-only.
if (!MG_Pipe::MGPipeHandleIsNull(pushedCso)) {
const auto* record = PipeSamplerCsoRecordForHandle(pushedCso);
if (record == nullptr) {
MGLOG_E_ONCE("MGPipe: sampler %u has no applier record on the handle arm, so its "
"parameters cannot be pushed (handle {%u, %u})",
samplerName, pushedCso.Slot, pushedCso.Gen);
return;
}
if (m_isInitialized && m_syncedSamplerSerial != 0 && m_syncedSamplerSerial == record->Serial) {
MGLOG_D("Sampler parameters have not changed for sampler ID: %u, skipping sync.", samplerName);
return;
}
m_syncedSamplerSerial = record->Serial;
pushedParams = &record->Params;
} else {
// P13: every sampler twin is a CSO twin, synced with its handle
// (ResolveSamplerCsoTwin); a sampler with no record has nothing to push.
MGLOG_E_ONCE("MGPipe: sampler %u was synced without a SamplerCso handle", samplerName);
return;
}
MGLOG_D("Syncing sampler with backend ID %u to backend for state ID %u", m_backendSamplerId, samplerName);
const SamplerParameters& samplerParams = *pushedParams;
#define SYNC_SAMPLER_PARAM_IF_CHANGED(internalName, glName, type) \
if (m_cacheSamplerParameters.internalName != samplerParams.internalName) { \
g_GLESFuncs.glSamplerParameteri(m_backendSamplerId, glName, \
(GLint)MG_Util::ConvertSampler##type##ToGLEnum(samplerParams.internalName)); \
m_cacheSamplerParameters.internalName = samplerParams.internalName; \
}
if (m_cacheSamplerParameters.minFilter != samplerParams.minFilter ||
m_cacheSamplerParameters.mipmapMode != samplerParams.mipmapMode) {
g_GLESFuncs.glSamplerParameteri(
m_backendSamplerId, GL_TEXTURE_MIN_FILTER,
(GLint)ResolveBackendMinFilter(samplerParams, ShouldAvoidSamplerMipmapMinFilterOnAngleLlvmpipe()));
m_cacheSamplerParameters.minFilter = samplerParams.minFilter;
m_cacheSamplerParameters.mipmapMode = samplerParams.mipmapMode;
}
if (m_cacheSamplerParameters.magFilter != samplerParams.magFilter) {
g_GLESFuncs.glSamplerParameteri(
m_backendSamplerId, GL_TEXTURE_MAG_FILTER,
(GLint)MG_Util::ConvertSamplerFilterModeToGLEnum(samplerParams.magFilter, SamplerMipmapMode::None));
m_cacheSamplerParameters.magFilter = samplerParams.magFilter;
}
SYNC_SAMPLER_PARAM_IF_CHANGED(wrapS, GL_TEXTURE_WRAP_S, WrapMode)
SYNC_SAMPLER_PARAM_IF_CHANGED(wrapT, GL_TEXTURE_WRAP_T, WrapMode)
SYNC_SAMPLER_PARAM_IF_CHANGED(wrapR, GL_TEXTURE_WRAP_R, WrapMode)
SYNC_SAMPLER_PARAM_IF_CHANGED(compareFunc, GL_TEXTURE_COMPARE_FUNC, CompareFunc)
SYNC_SAMPLER_PARAM_IF_CHANGED(compareMode, GL_TEXTURE_COMPARE_MODE, CompareMode)
if (m_cacheSamplerParameters.minLod != samplerParams.minLod) {
g_GLESFuncs.glSamplerParameterf(m_backendSamplerId, GL_TEXTURE_MIN_LOD, samplerParams.minLod);
m_cacheSamplerParameters.minLod = samplerParams.minLod;
}
if (m_cacheSamplerParameters.maxLod != samplerParams.maxLod) {
g_GLESFuncs.glSamplerParameterf(m_backendSamplerId, GL_TEXTURE_MAX_LOD, samplerParams.maxLod);
m_cacheSamplerParameters.maxLod = samplerParams.maxLod;
}
if (m_cacheSamplerParameters.maxAnisotropy != samplerParams.maxAnisotropy) {
if (g_GLESCapabilities.SupportsTextureFilterAnisotropy) {
g_GLESFuncs.glSamplerParameterf(m_backendSamplerId, GL_TEXTURE_MAX_ANISOTROPY_EXT,
samplerParams.maxAnisotropy);
}
m_cacheSamplerParameters.maxAnisotropy = samplerParams.maxAnisotropy;
}
if (m_cacheSamplerParameters.borderColor != samplerParams.borderColor ||
m_cacheSamplerParameters.borderColorI != samplerParams.borderColorI ||
m_cacheSamplerParameters.borderColorUI != samplerParams.borderColorUI ||
m_cacheSamplerParameters.borderColorForm != samplerParams.borderColorForm) {
// Same gate as the texture-side border colour above, and the same reason for
// branching on the form: an integer border colour must reach the driver through
// glSamplerParameterIiv/Iuiv or an integer sampler reads the float's bit pattern
// back instead of the value.
if (g_GLESCapabilities.SupportsTextureBorderClamp) {
if (samplerParams.borderColorForm == BorderColorForm::Int && g_GLESFuncs.glSamplerParameterIiv) {
const GLint borderColorArray[4] = {
samplerParams.borderColorI.x(), samplerParams.borderColorI.y(),
samplerParams.borderColorI.z(), samplerParams.borderColorI.w()};
g_GLESFuncs.glSamplerParameterIiv(m_backendSamplerId, GL_TEXTURE_BORDER_COLOR,
borderColorArray);
} else if (samplerParams.borderColorForm == BorderColorForm::Uint &&
g_GLESFuncs.glSamplerParameterIuiv) {
const GLuint borderColorArray[4] = {
samplerParams.borderColorUI.x(), samplerParams.borderColorUI.y(),
samplerParams.borderColorUI.z(), samplerParams.borderColorUI.w()};
g_GLESFuncs.glSamplerParameterIuiv(m_backendSamplerId, GL_TEXTURE_BORDER_COLOR,
borderColorArray);
} else if (g_GLESFuncs.glSamplerParameterfv) {
const GLfloat borderColorArray[4] = {
samplerParams.borderColor.x(), samplerParams.borderColor.y(), samplerParams.borderColor.z(),
samplerParams.borderColor.w()};
g_GLESFuncs.glSamplerParameterfv(m_backendSamplerId, GL_TEXTURE_BORDER_COLOR, borderColorArray);
}
}
m_cacheSamplerParameters.borderColor = samplerParams.borderColor;
m_cacheSamplerParameters.borderColorI = samplerParams.borderColorI;
m_cacheSamplerParameters.borderColorUI = samplerParams.borderColorUI;
m_cacheSamplerParameters.borderColorForm = samplerParams.borderColorForm;
}
#undef SYNC_SAMPLER_PARAM_IF_CHANGED
m_isInitialized = true;
}
void BackendSamplerObject::Bind(Uint unit) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
if (g_boundSamplersCache[unit] == this) return;
g_GLESFuncs.glBindSampler(static_cast<GLenum>(unit), m_backendSamplerId);
g_boundSamplersCache[unit] = this;
}
Uint BackendSamplerObject::GetBackendSamplerId() const {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
return m_backendSamplerId;
}
void UnbindSampler(Uint unit) {
if (g_boundSamplersCache[unit] == nullptr) return;
g_GLESFuncs.glBindSampler(static_cast<GLenum>(unit), 0);
g_boundSamplersCache[unit] = nullptr;
}
Array<BackendSamplerObject*, MG_State::GLState::TextureState::MAX_TEXTURE_IMAGE_UNITS> g_boundSamplersCache;
TwinRegistry<MG_State::GLState::SamplerObject, BackendSamplerObject, MG_Pipe::MGPipeKind::SamplerCso>
g_backendSamplerObjects;
// P4a fable seam F-4. THE TWIN FOR A CONTENT-ADDRESSED SamplerCso HANDLE, keyed by that
// handle and synced from its record - see the declaration for why the identity-keyed
// lookup it replaces could never hit.
BackendSamplerObject* ResolveSamplerCsoTwin(MG_Pipe::MGPipeHandle cso) {
if (MG_Pipe::MGPipeHandleIsNull(cso)) return nullptr;
// THE RECORD FIRST, before the table is touched: a handle with no record is a seam
// defect (the client minted and named a CSO it never described, or evicted one a
// standing set still names), and adopting a slot for it would leave a twin that
// syncs nothing. The census stem is the sampler family's.
const auto* record = PipeSamplerCsoRecordForHandle(cso);
if (record == nullptr) {
MGLOG_E_ONCE("MGPipe: sampler CSO {%u, %u} has no applier record on the handle arm, so no "
"driver sampler can be built for it and the unit keeps what it holds",
cso.Slot, cso.Gen);
return nullptr;
}
// The same slot table the identity twins live in: one allocator serves both handle
// families of this kind, so a content-addressed slot and an identity-minted slot can
// never coincide, and the generation discipline (forward = recycle, backward =
// refused) is what retires a twin whose CSO the client's LRU evicted and re-minted.
auto* slot = g_backendSamplerObjects.GetOrCreateByHandle(cso);
if (slot == nullptr) {
MGLOG_E_ONCE("MGPipe: sampler CSO {%u, %u} cannot be adopted on the handle arm (the slot's "
"live generation is %u), so no driver sampler is built for it",
cso.Slot, cso.Gen, g_backendSamplerObjects.LiveGenAt(cso.Slot));
return nullptr;
}
if (!*slot) *slot = MakeShared<BackendSamplerObject>();
// Serial-gated inside: a CSO whose record did not move since this twin last synced
// costs the record lookup above and one compare.
(*slot)->SyncToBackend(nullptr, cso);
return slot->get();
}
} // namespace SamplerImpl
} // namespace MobileGL::MG_Backend::DirectGLES
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+71 -53
View File
@@ -25,10 +25,14 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
SizeT IndexTypeSize(GLenum type) {
switch (type) {
case GL_UNSIGNED_BYTE: return 1;
case GL_UNSIGNED_SHORT: return 2;
case GL_UNSIGNED_INT: return 4;
default: return 0;
case GL_UNSIGNED_BYTE:
return 1;
case GL_UNSIGNED_SHORT:
return 2;
case GL_UNSIGNED_INT:
return 4;
default:
return 0;
}
}
@@ -60,12 +64,18 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// first) and for GL_PATCHES, whose primitive size is dynamic tessellation state.
Uint32 ConcatenablePrimitiveSize(GLenum mode) {
switch (mode) {
case GL_POINTS: return 1;
case GL_LINES: return 2;
case GL_TRIANGLES: return 3;
case GL_LINES_ADJACENCY: return 4;
case GL_TRIANGLES_ADJACENCY: return 6;
default: return 0;
case GL_POINTS:
return 1;
case GL_LINES:
return 2;
case GL_TRIANGLES:
return 3;
case GL_LINES_ADJACENCY:
return 4;
case GL_TRIANGLES_ADJACENCY:
return 6;
default:
return 0;
}
}
@@ -112,10 +122,10 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// value is exactly the work its original call site did, in its original order, so the
// monolith arm makes the same calls in the same sequence it made before this package.
enum class IndexBufferQuestion {
Presence, // is an element array buffer bound at all
DriverName, // + the GL name currently bound to GL_ELEMENT_ARRAY_BUFFER
DriverNameAndSize,// + the store's size in bytes (the compute tier's source)
HostBytes, // a CPU-readable copy of the whole store, and its size
Presence, // is an element array buffer bound at all
DriverName, // + the GL name currently bound to GL_ELEMENT_ARRAY_BUFFER
DriverNameAndSize, // + the store's size in bytes (the compute tier's source)
HostBytes, // a CPU-readable copy of the whole store, and its size
};
struct BoundIndexBufferView {
@@ -150,8 +160,7 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// the other buffer target: a missing record on the handle arm aborts by name and never
// falls back to the frontend binding slot, which answers for whatever the CLIENT has
// bound NOW - a later verb than the one being applied.
[[noreturn]] void RefuseMissingIndirectBufferRecord(const char* entry,
MG_Pipe::MGPipeHandle res) {
[[noreturn]] void RefuseMissingIndirectBufferRecord(const char* entry, MG_Pipe::MGPipeHandle res) {
MGLOG_F("MGPipe: Fatal{RoleViolation, \"multidraw-indirect-buffer-arm\"} - %s found no "
"backend resource for the indirect buffer {%u, %u} this verb's record named. "
"The indirect buffer of this family is MGPipeApplier().VerbIndirectBuffer "
@@ -167,8 +176,7 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// server shadow is a missing record and Fatal by name; for one the application ORPHANED
// the gap is its own undefined content, and reading the shadow's zero-fill is exactly
// what the monolith arm's MappedData() hands back.
Bool IndexBufferRecordHasDefinedContent(const MG_Pipe::MGPipeApplierState& st,
MG_Pipe::MGPipeHandle res) {
Bool IndexBufferRecordHasDefinedContent(const MG_Pipe::MGPipeApplierState& st, MG_Pipe::MGPipeHandle res) {
if (res.Slot >= st.Resources.size()) return false;
const auto& candidate = st.Resources[res.Slot];
if (!candidate.Live || candidate.Gen != res.Gen) return false;
@@ -323,7 +331,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
BufferImpl::BindBufferId(BufferImpl::TempBufferTarget, buffer.id);
// Grow in powers of two so a batch that creeps up in size stops respecifying.
SizeT capacity = buffer.capacity == 0 ? bytes : buffer.capacity;
while (capacity < bytes) capacity *= 2;
while (capacity < bytes)
capacity *= 2;
g_GLESFuncs.glBufferData(BufferImpl::TempBufferTarget, static_cast<GLsizeiptr>(capacity), nullptr,
GL_STREAM_DRAW);
buffer.capacity = capacity;
@@ -357,7 +366,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
const SizeT aligned = (bytes + kRingAlignment - 1) & ~(kRingAlignment - 1);
if (buffer.capacity < aligned) {
SizeT capacity = buffer.capacity == 0 ? kMinRingBytes : buffer.capacity;
while (capacity < aligned) capacity *= 2;
while (capacity < aligned)
capacity *= 2;
g_GLESFuncs.glBufferData(BufferImpl::TempBufferTarget, static_cast<GLsizeiptr>(capacity), nullptr,
GL_STREAM_DRAW);
buffer.capacity = capacity;
@@ -422,9 +432,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
void ResolveTierOnce() {
if (g_tierResolved) return;
g_tierResolved = true;
g_resolvedTier =
ResolveTier(g_GLESCapabilities, g_GLESFuncs, MG_Config::Features.EsprytMultiDrawMode,
&g_tierResolution);
g_resolvedTier = ResolveTier(g_GLESCapabilities, g_GLESFuncs, MG_Config::Features.EsprytMultiDrawMode,
&g_tierResolution);
MGLOG_D("DirectGLES multi-draw: %s", g_tierResolution.c_str());
}
@@ -447,8 +456,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// the batch's own shape - not the driver - rules it out; the compute tier keeps
// its remaining feasibility checks inside its implementation, where the data it
// has to walk is already in hand.
GLESMultiDrawMode ResolveTierForBatch(Bool programReadsDrawID, Bool perSubDrawBaseVertex,
Bool hasIndexBuffer, Bool arbitraryRestart) {
GLESMultiDrawMode ResolveTierForBatch(Bool programReadsDrawID, Bool perSubDrawBaseVertex, Bool hasIndexBuffer,
Bool arbitraryRestart) {
ResolveTierOnce();
GLESMultiDrawMode tier = g_resolvedTier;
@@ -480,8 +489,7 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// The indirect tiers describe each sub-draw as an element offset into the
// bound element array buffer. A client-memory index array has no such buffer,
// and indirect draws are not defined without one.
if (!hasIndexBuffer &&
(tier == GLESMultiDrawMode::MultiIndirect || tier == GLESMultiDrawMode::Indirect)) {
if (!hasIndexBuffer && (tier == GLESMultiDrawMode::MultiIndirect || tier == GLESMultiDrawMode::Indirect)) {
tier = SupportsTier(GLESMultiDrawMode::BaseVertex) ? GLESMultiDrawMode::BaseVertex
: GLESMultiDrawMode::DrawElements;
}
@@ -503,14 +511,18 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
for (SizeT i = 0; i < sourceIndexCount; ++i) {
Uint32 value = 0;
switch (indexSize) {
case 1: value = source[i]; break;
case 1:
value = source[i];
break;
case 2: {
Uint16 narrow = 0;
std::memcpy(&narrow, source + i * 2, sizeof(narrow));
value = narrow;
break;
}
default: std::memcpy(&value, source + i * 4, sizeof(value)); break;
default:
std::memcpy(&value, source + i * 4, sizeof(value));
break;
}
// Unsigned wraparound is the defined behaviour for a negative base vertex.
out[i] = (restartActive && value == restartSentinel) ? 0xFFFFFFFFu : value + baseVertexBits;
@@ -525,9 +537,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// P5e (mv): `hasIndexBuffer` is a Bool and not the frontend object it used to be,
// because presence is the only thing this function ever asked of it - and on the record
// arm there is no such object on this side to hand it.
const Uint8* ResolveSubDrawIndices(Bool hasIndexBuffer, const Uint8* indexBufferBytes,
SizeT indexBufferSize, const void* indices, SizeT indexCount,
SizeT indexSize) {
const Uint8* ResolveSubDrawIndices(Bool hasIndexBuffer, const Uint8* indexBufferBytes, SizeT indexBufferSize,
const void* indices, SizeT indexCount, SizeT indexSize) {
if (!hasIndexBuffer) {
return static_cast<const Uint8*>(indices);
}
@@ -610,7 +621,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
for (GLsizei i = 0; i < drawcount; ++i) {
if (feedDrawID) SetCurrentDrawID(static_cast<Uint32>(i));
if (feedBaseVertex) SetCurrentBaseVertex(basevertex ? basevertex[i] : 0);
const SizeT commandOffset = commandBase + static_cast<SizeT>(i) * sizeof(DrawElementsIndirectCommand);
const SizeT commandOffset =
commandBase + static_cast<SizeT>(i) * sizeof(DrawElementsIndirectCommand);
ForEachViewportRoutingPass([&] {
g_GLESFuncs.glDrawElementsIndirect(mode, type, reinterpret_cast<const void*>(commandOffset));
});
@@ -650,8 +662,7 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
// ---------------------------------------------------------------------------
Bool RunRebasedDrawElements(GLenum mode, const GLsizei* count, GLenum type, const GLvoid* const* indices,
GLsizei drawcount, const GLint* basevertex, Bool feedDrawID,
Bool feedBaseVertex) {
GLsizei drawcount, const GLint* basevertex, Bool feedDrawID, Bool feedBaseVertex) {
const SizeT indexSize = IndexTypeSize(type);
if (indexSize == 0) return false;
@@ -688,8 +699,8 @@ namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl {
indices[i], subDrawCount, indexSize);
if (!source) {
MGLOG_E_ONCE("DirectGLES multi-draw (drawelements tier): sub-draw %d reads outside the bound index "
"buffer; skipping the batch",
i);
"buffer; skipping the batch",
i);
return false;
}
RebaseIndices(source, subDrawCount, indexSize, basevertex ? basevertex[i] : 0, restartActive,
@@ -807,7 +818,8 @@ void main() {
if (status != GL_TRUE) {
char log[1024] = {};
g_GLESFuncs.glGetShaderInfoLog(shader, sizeof(log) - 1, nullptr, log);
MGLOG_E_ONCE("DirectGLES multi-draw (compute tier): index-flattening shader failed to compile: %s", log);
MGLOG_E_ONCE("DirectGLES multi-draw (compute tier): index-flattening shader failed to compile: %s",
log);
g_GLESFuncs.glDeleteShader(shader);
return false;
}
@@ -1016,13 +1028,20 @@ void main() {
const char* TierName(GLESMultiDrawMode tier) {
switch (tier) {
case GLESMultiDrawMode::Auto: return "auto";
case GLESMultiDrawMode::Ext: return "ext";
case GLESMultiDrawMode::MultiIndirect: return "multiindirect";
case GLESMultiDrawMode::Indirect: return "indirect";
case GLESMultiDrawMode::BaseVertex: return "basevertex";
case GLESMultiDrawMode::DrawElements: return "drawelements";
case GLESMultiDrawMode::Compute: return "compute";
case GLESMultiDrawMode::Auto:
return "auto";
case GLESMultiDrawMode::Ext:
return "ext";
case GLESMultiDrawMode::MultiIndirect:
return "multiindirect";
case GLESMultiDrawMode::Indirect:
return "indirect";
case GLESMultiDrawMode::BaseVertex:
return "basevertex";
case GLESMultiDrawMode::DrawElements:
return "drawelements";
case GLESMultiDrawMode::Compute:
return "compute";
}
return "unknown";
}
@@ -1061,8 +1080,7 @@ void main() {
const Bool arbitraryRestart = restartKind == RestartSubstitutionKind::RewriteIndices;
const ScopedSuppressedPrimitiveRestart restartCapOverride(restartKind);
const Bool hasIndexBuffer =
ResolveBoundIndexBuffer(IndexBufferQuestion::Presence, "DrawElementsBatch").Present;
const Bool hasIndexBuffer = ResolveBoundIndexBuffer(IndexBufferQuestion::Presence, "DrawElementsBatch").Present;
// The compute tier dispatches BEFORE the draw state is established: doing it
// afterwards would mean unpicking the program, SSBO and index bindings
@@ -1116,8 +1134,8 @@ void main() {
drawn = RunBaseVertexLoop(mode, count, type, indices, drawcount, basevertex, feedDrawID, feedBaseVertex);
break;
case GLESMultiDrawMode::DrawElements:
drawn = RunRebasedDrawElements(mode, count, type, indices, drawcount, basevertex, feedDrawID,
feedBaseVertex);
drawn =
RunRebasedDrawElements(mode, count, type, indices, drawcount, basevertex, feedDrawID, feedBaseVertex);
break;
case GLESMultiDrawMode::Compute:
// Its pre-pass ran above; reaching here means it declined this batch's shape.
@@ -1136,13 +1154,13 @@ void main() {
drawn = RunBaseVertexLoop(mode, count, type, indices, drawcount, basevertex, feedDrawID, feedBaseVertex);
}
if (!drawn) {
drawn = RunRebasedDrawElements(mode, count, type, indices, drawcount, basevertex, feedDrawID,
feedBaseVertex);
drawn =
RunRebasedDrawElements(mode, count, type, indices, drawcount, basevertex, feedDrawID, feedBaseVertex);
}
if (!drawn) {
MGLOG_E_ONCE("DirectGLES multi-draw: no usable tier for a %d sub-draw batch (mode 0x%x, type 0x%x); "
"the batch was dropped",
drawcount, mode, type);
"the batch was dropped",
drawcount, mode, type);
}
}
} // namespace MobileGL::MG_Backend::DirectGLES::MultiDrawImpl
+30 -50
View File
@@ -70,7 +70,6 @@
// slot allocator (the link ratchet's P13 exit gate pins that).
namespace MobileGL::MG_Backend::DirectGLES {
// Declared in Managers.h as well; repeated here because this header is included from it
// before that declaration, and the table below is the arming site on this arm (D13: "the
// arming site moves to the slot table's first insertion").
@@ -218,7 +217,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
Vector<SharedPtr<Table>> Snapshot() const {
Vector<SharedPtr<Table>> all;
all.reserve(m_buckets.size());
for (const auto& bucket : m_buckets) all.push_back(bucket.second);
for (const auto& bucket : m_buckets)
all.push_back(bucket.second);
return all;
}
@@ -271,7 +271,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_buckets.clear();
m_memoKey = Key{};
m_memoTable = nullptr;
for (const SharedPtr<Table>& table : all) table->Reset();
for (const SharedPtr<Table>& table : all)
table->Reset();
}
// ---- the calling thread's bucket, member for member ---------------------------------
@@ -282,12 +283,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// - unchanged source - answer from the right context's object namespace. A caller that
// must name a bucket OTHER than its own (a death walk, a case) takes Shared()/Snapshot()
// and calls the table directly.
BackendPtr& GetOrCreate(MG_Pipe::MGPipeHandle handle) {
return ForCallingThread().GetOrCreate(handle);
}
BackendPtr* FindByHandle(MG_Pipe::MGPipeHandle handle) {
return ForCallingThread().FindByHandle(handle);
}
BackendPtr& GetOrCreate(MG_Pipe::MGPipeHandle handle) { return ForCallingThread().GetOrCreate(handle); }
BackendPtr* FindByHandle(MG_Pipe::MGPipeHandle handle) { return ForCallingThread().FindByHandle(handle); }
const BackendPtr* FindByHandle(MG_Pipe::MGPipeHandle handle) const {
return ForCallingThread().FindByHandle(handle);
}
@@ -321,17 +318,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
// The process-wide answer for the two statics: a notice is about an OBJECT, so it must
// reach the twin in EVERY bucket of EVERY holder. They forward to the table's own
// statics, which walk the holder list - each SHARED holder of it is one bucket.
static Bool ReleaseTwinByHandle(MG_Pipe::MGPipeHandle handle) {
return Table::ReleaseTwinByHandle(handle);
}
static Bool ReleaseTwinByHandle(MG_Pipe::MGPipeHandle handle) { return Table::ReleaseTwinByHandle(handle); }
// The instance spelling, and it HANDS THE TWIN OUT rather than answering a Bool: the two
// tables that ARE a bucket registry (buffers, sampler views) have always been called as
// `g_backendBufferResources.ReleaseByHandle(res)` and their callers still have to decide
// what happens to the driver id the twin owns. The bucket is the CALLING THREAD'S, which
// is the bucket that twin was created in.
BackendPtr ReleaseByHandle(MG_Pipe::MGPipeHandle handle) {
return ForCallingThread().ReleaseByHandle(handle);
}
BackendPtr ReleaseByHandle(MG_Pipe::MGPipeHandle handle) { return ForCallingThread().ReleaseByHandle(handle); }
static Uint32 HolderCount() { return Table::HolderCount(); }
Uint32 LiveCount() const { return ForCallingThread().LiveCount(); }
Uint32 CompositeLiveCount() const { return ForCallingThread().CompositeLiveCount(); }
@@ -405,18 +398,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
// registry) is a holder for exactly as long as it exists. Copy and move carry the
// ENTRIES; the links are the table's own and are never copied.
BackendSlotTable() { LinkHolder(); }
BackendSlotTable(const BackendSlotTable& other):
m_ownerSession(other.m_ownerSession),
m_slots(other.m_slots),
m_band(other.m_band),
m_nullTwin(other.m_nullTwin) {
BackendSlotTable(const BackendSlotTable& other)
: m_ownerSession(other.m_ownerSession), m_slots(other.m_slots), m_band(other.m_band),
m_nullTwin(other.m_nullTwin) {
LinkHolder();
}
BackendSlotTable(BackendSlotTable&& other) noexcept:
m_ownerSession(other.m_ownerSession),
m_slots(std::move(other.m_slots)),
m_band(std::move(other.m_band)),
m_nullTwin(std::move(other.m_nullTwin)) {
BackendSlotTable(BackendSlotTable&& other) noexcept
: m_ownerSession(other.m_ownerSession), m_slots(std::move(other.m_slots)), m_band(std::move(other.m_band)),
m_nullTwin(std::move(other.m_nullTwin)) {
other.m_slots.clear();
other.m_band.clear();
LinkHolder();
@@ -457,8 +446,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// releases the twin by handle, and the slot is freed by the CLIENT after that returns.
//
BackendPtr& GetOrCreate(MG_Pipe::MGPipeHandle handle) {
MOBILEGL_ASSERT(!MG_Pipe::MGPipeHandleIsNull(handle),
"GetOrCreate(handle) named the reserved null handle");
MOBILEGL_ASSERT(!MG_Pipe::MGPipeHandleIsNull(handle), "GetOrCreate(handle) named the reserved null handle");
if (MG_Pipe::MGPipeHandleIsNull(handle)) return m_nullTwin;
// A slot index that ARRIVED in a payload indexes a vector this call would RESIZE,
@@ -479,8 +467,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
"GetOrCreate(handle) named slot %u, past this table's %u bound",
handle.Slot, kMaxHandleSlot);
#else
MOBILEGL_ASSERT(false, "GetOrCreate(handle) named slot %u, past this table's %u bound",
handle.Slot, kMaxHandleSlot);
MOBILEGL_ASSERT(false, "GetOrCreate(handle) named slot %u, past this table's %u bound", handle.Slot,
kMaxHandleSlot);
return m_nullTwin;
#endif
}
@@ -489,15 +477,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
EnsureProcessTeardownSentinel();
// THE TWO DIRECTIONS ARE NOT SYMMETRIC. The handle arrived in a payload, so
// `handle.Gen < entry.Gen` is a reachable input, and adopting it would destroy the INCUMBENT LIVE twin - a driver buffer id,
// a persistent map, a pooled store, released by a defaulted destructor that issues
// no glDeleteBuffers and no pool enrolment - and then stamp the slot back to the
// dead resource's generation, after which the incumbent's own FindByHandle refuses
// it and it is silently handed a fresh, empty twin. That is a leak AND a resource
// that loses its storage with no diagnostic, i.e. the shape commit d7655247 fixed
// and the thing MGPipeHandle::Gen exists to prevent. So: forward is a recycle and
// resets the twin, BACKWARD is refused - which is the same answer FindByHandle
// below already gives the same input.
// `handle.Gen < entry.Gen` is a reachable input, and adopting it would destroy the INCUMBENT LIVE twin - a
// driver buffer id, a persistent map, a pooled store, released by a defaulted destructor that issues no
// glDeleteBuffers and no pool enrolment - and then stamp the slot back to the dead resource's generation,
// after which the incumbent's own FindByHandle refuses it and it is silently handed a fresh, empty twin.
// That is a leak AND a resource that loses its storage with no diagnostic, i.e. the shape commit d7655247
// fixed and the thing MGPipeHandle::Gen exists to prevent. So: forward is a recycle and resets the twin,
// BACKWARD is refused - which is the same answer FindByHandle below already gives the same input.
Entry& entry = EntryAt(handle.Slot);
if (entry.Live && entry.Gen > handle.Gen) {
#if MOBILEGL_BUILD_DISAGGREGATED
@@ -599,8 +585,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// list; nothing on a shipping path asks.
static Uint32 HolderCount() {
Uint32 count = 0;
for (const BackendSlotTable* holder = s_firstHolder; holder != nullptr;
holder = holder->m_nextHolder) {
for (const BackendSlotTable* holder = s_firstHolder; holder != nullptr; holder = holder->m_nextHolder) {
++count;
}
return count;
@@ -632,9 +617,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
const Entry& entry = m_band[index];
if (!entry.Live || !entry.backend) continue;
const BackendPtr twin = entry.backend;
fn(MG_Pipe::MGPipeHandle{
static_cast<Uint32>(index) + MG_Pipe::kMGPipeShaderCsoCompositeSlotBase,
entry.Gen},
fn(MG_Pipe::MGPipeHandle{static_cast<Uint32>(index) + MG_Pipe::kMGPipeShaderCsoCompositeSlotBase,
entry.Gen},
twin);
}
}
@@ -714,9 +698,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
return kHasCompositeBand && MG_Pipe::MGPipeIsCompositeShaderSlot(slot);
}
static constexpr SizeT IndexOfSlot(Uint32 slot) {
return SlotIsBanded(slot)
? static_cast<SizeT>(slot - MG_Pipe::kMGPipeShaderCsoCompositeSlotBase)
: static_cast<SizeT>(slot);
return SlotIsBanded(slot) ? static_cast<SizeT>(slot - MG_Pipe::kMGPipeShaderCsoCompositeSlotBase)
: static_cast<SizeT>(slot);
}
// The entry a slot names, or null when its space has never grown that far. NEVER grows:
@@ -728,9 +711,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (index >= table.size()) return nullptr;
return &table[index];
}
const Entry* EntryOrNull(Uint32 slot) const {
return const_cast<BackendSlotTable*>(this)->EntryOrNull(slot);
}
const Entry* EntryOrNull(Uint32 slot) const { return const_cast<BackendSlotTable*>(this)->EntryOrNull(slot); }
// Grows the right space to hold `slot`. Every caller bounds `slot` first - against
// kMaxHandleSlot - because this is the one place a client-supplied number decides an
@@ -779,7 +760,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
Vector<Entry> m_band;
// Handed back by GetOrCreate for a null state object. Never live, never handed a handle.
BackendPtr m_nullTwin;
};
} // namespace MobileGL::MG_Backend::DirectGLES
+103 -548
View File
@@ -37,13 +37,6 @@
#include <regex>
namespace MobileGL::MG_Backend::DirectGLES {
// read_pixels writes server reply scratch. The client refuses PACK_BUFFER before
// emission, so no frontend pack binding can affect this destination.
static const SharedPtr<MG_State::GLState::BufferObject>& SplitReadbackPackBuffer() {
static const SharedPtr<MG_State::GLState::BufferObject> none;
return none;
}
const FormatCapabilityCache* ActiveBackendFormatCaps() {
// Under a live split session the SERVER's private backend is what owns the context on the
// apply thread (and these reads all run there), so its cache is the authoritative one.
@@ -89,22 +82,19 @@ namespace MobileGL::MG_Backend::DirectGLES {
return options;
}
Flags<PixelFormatNormalizeOptionBit>
GetRuntimeFallbackNormalizeOptions(GLenum requestedInternalFormat,
Flags<PixelFormatNormalizeOptionBit> extraOptions) {
Flags<PixelFormatNormalizeOptionBit> GetRuntimeFallbackNormalizeOptions(
GLenum requestedInternalFormat, Flags<PixelFormatNormalizeOptionBit> extraOptions) {
using namespace MG_Util::TextureFormatProcessor;
const Flags<PixelFormatNormalizeOptionBit> forcedOptions = GetApplicablePixelFormatNormalizeOptions(
requestedInternalFormat, GetForcedPixelFormatNormalizeOptions() | extraOptions);
if (forcedOptions) {
return forcedOptions;
}
return GetApplicablePixelFormatNormalizeOptions(
requestedInternalFormat, GetDriverPixelFormatNormalizeOptions() | extraOptions);
return GetApplicablePixelFormatNormalizeOptions(requestedInternalFormat,
GetDriverPixelFormatNormalizeOptions() | extraOptions);
}
Bool HasCachedFormatCapability(TextureInternalFormat internalFormat,
SizeT targetIndex,
Bool caveat,
Bool HasCachedFormatCapability(TextureInternalFormat internalFormat, SizeT targetIndex, Bool caveat,
FormatCapability capability) {
const FormatCapabilityCache* activeCaps = ActiveBackendFormatCaps();
if (activeCaps == nullptr || targetIndex >= kFormatCapabilityTargetCount) {
@@ -121,8 +111,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
return HasFormatCapability(caps, capability);
}
Bool HasAnyCachedFormatCapability(TextureInternalFormat internalFormat,
Bool caveat,
Bool HasAnyCachedFormatCapability(TextureInternalFormat internalFormat, Bool caveat,
FormatCapability capability) {
for (SizeT targetIndex = 0; targetIndex < kFormatCapabilityTargetCount; ++targetIndex) {
if (HasCachedFormatCapability(internalFormat, targetIndex, caveat, capability)) {
@@ -138,10 +127,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
HasCachedFormatCapability(internalFormat, targetIndex, false, FormatCapability::Creatable);
const Bool caveatCreatable =
HasCachedFormatCapability(internalFormat, targetIndex, true, FormatCapability::Creatable);
const Bool fullRenderable =
HasCachedFormatCapability(internalFormat, targetIndex, false, FormatCapability::FramebufferRenderable);
const Bool caveatRenderable =
HasCachedFormatCapability(internalFormat, targetIndex, true, FormatCapability::FramebufferRenderable);
const Bool fullRenderable = HasCachedFormatCapability(internalFormat, targetIndex, false,
FormatCapability::FramebufferRenderable);
const Bool caveatRenderable = HasCachedFormatCapability(internalFormat, targetIndex, true,
FormatCapability::FramebufferRenderable);
return (!fullCreatable && caveatCreatable) || (!fullRenderable && caveatRenderable);
}
@@ -151,11 +140,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
return HasAnyCachedFormatCapability(internalFormat, true, FormatCapability::Creatable);
}
void GenerateFormatInfo(TextureInternalFormat internalFormat,
SizeT targetIndex,
GLenum* outInternalFormat,
GLenum* outFormat,
GLenum* outType) {
void GenerateFormatInfo(TextureInternalFormat internalFormat, SizeT targetIndex, GLenum* outInternalFormat,
GLenum* outFormat, GLenum* outType) {
using namespace MobileGL::MG_Util::TextureFormatProcessor;
const GLenum requestedInternalFormat = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat);
Flags<PixelFormatNormalizeOptionBit> options;
@@ -264,8 +250,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
const SizeT targetIndex =
target == TextureTarget::Unknown ? kFormatCapabilityTargetCount : GetFormatCapabilityTargetIndex(target);
const SizeT targetIndex = target == TextureTarget::Unknown ? kFormatCapabilityTargetCount
: GetFormatCapabilityTargetIndex(target);
GenerateFormatInfo(internalFormat, targetIndex, outInternalFormat, outFormat, outType);
}
@@ -279,8 +265,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
Bool ShouldUseCaveatTextureFormat(TextureInternalFormat internalFormat, TextureTarget target) {
const SizeT targetIndex =
target == TextureTarget::Unknown ? kFormatCapabilityTargetCount : GetFormatCapabilityTargetIndex(target);
const SizeT targetIndex = target == TextureTarget::Unknown ? kFormatCapabilityTargetCount
: GetFormatCapabilityTargetIndex(target);
return ShouldUseCaveatFormat(internalFormat, targetIndex);
}
@@ -304,8 +290,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
} // namespace
Bool BackendTextureFormatAddsAlpha(TextureInternalFormat internalFormat, TextureTarget target) {
const SizeT targetIndex =
target == TextureTarget::Unknown ? kFormatCapabilityTargetCount : GetFormatCapabilityTargetIndex(target);
const SizeT targetIndex = target == TextureTarget::Unknown ? kFormatCapabilityTargetCount
: GetFormatCapabilityTargetIndex(target);
return BackendFormatAddsAlpha(internalFormat, targetIndex);
}
@@ -315,8 +301,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
ImageBindableStorageWidening GetImageBindableStorageWidening(TextureInternalFormat internalFormat) {
const GLenum requested = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat);
const auto carrier = static_cast<GLenum>(
MG_Util::ShaderTranspiler::ShaderCompiler::WidenedCoreEsslImageFormat(requested));
const auto carrier =
static_cast<GLenum>(MG_Util::ShaderTranspiler::ShaderCompiler::WidenedCoreEsslImageFormat(requested));
if (carrier == 0) {
return {};
}
@@ -336,8 +322,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
ImageBindableStorageWidening widening;
widening.InternalFormat = carrier;
widening.SourceChannels =
MG_Util::ShaderTranspiler::ShaderCompiler::ImageFormatChannelCount(requested);
widening.SourceChannels = MG_Util::ShaderTranspiler::ShaderCompiler::ImageFormatChannelCount(requested);
switch (carrier) {
case GL_RGBA32UI:
case GL_RGBA16UI:
@@ -503,7 +488,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
for (; outputIt != outputEnd; ++outputIt) {
const Uint location = static_cast<Uint>(std::stoul((*outputIt)[1].str()));
if (location < 32 && (outputMask & (1u << location))) {
outputClamps.push_back({(*outputIt)[2].str(), static_cast<Bool>(snormOutputMask & (1u << location))});
outputClamps.push_back(
{(*outputIt)[2].str(), static_cast<Bool>(snormOutputMask & (1u << location))});
}
}
if (outputClamps.empty()) {
@@ -527,8 +513,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
String clampLine;
for (const OutputClamp& outputClamp : outputClamps) {
const String minValue = outputClamp.Signed ? "-1.0" : "0.0";
clampLine += "\n " + outputClamp.Name + " = clamp(" + outputClamp.Name +
", vec4(" + minValue + "), vec4(1.0));";
clampLine += "\n " + outputClamp.Name + " = clamp(" + outputClamp.Name + ", vec4(" +
minValue + "), vec4(1.0));";
}
clampLine += "\n";
glslCode.insert(pos, clampLine);
@@ -564,8 +550,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
String replicaCopies;
for (Uint location = 1; location < drawBufferCount; ++location) {
const String name = String(kLoweredName) + "_" + std::to_string(location);
replicaDecls += "\nlayout(location = " + std::to_string(location) + ") out " + precision + "vec4 " +
name + ";";
replicaDecls +=
"\nlayout(location = " + std::to_string(location) + ") out " + precision + "vec4 " + name + ";";
replicaCopies += "\n " + name + " = " + kLoweredName + ";";
}
@@ -639,8 +625,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
return result;
}
String RetargetTextureBufferExtension(String glslCode,
MG_External::GLESCapabilities::TextureBufferTier tier) {
String RetargetTextureBufferExtension(String glslCode, MG_External::GLESCapabilities::TextureBufferTier tier) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
@@ -796,12 +781,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
const char* PointSizeExtensionName(MG_External::GLESCapabilities::PointSizeTier tier, Bool tessellation) {
using Tier = MG_External::GLESCapabilities::PointSizeTier;
switch (tier) {
case Tier::ExtensionEXT:
return tessellation ? "GL_EXT_tessellation_point_size" : "GL_EXT_geometry_point_size";
case Tier::ExtensionOES:
return tessellation ? "GL_OES_tessellation_point_size" : "GL_OES_geometry_point_size";
default:
return nullptr;
case Tier::ExtensionEXT:
return tessellation ? "GL_EXT_tessellation_point_size" : "GL_EXT_geometry_point_size";
case Tier::ExtensionOES:
return tessellation ? "GL_OES_tessellation_point_size" : "GL_OES_geometry_point_size";
default:
return nullptr;
}
}
@@ -834,8 +819,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
return glslCode;
}
String BakeImageFormatQualifiers(String glslCode,
const UnorderedMap<String, String>& esslFormatByUniformName) {
String BakeImageFormatQualifiers(String glslCode, const UnorderedMap<String, String>& esslFormatByUniformName) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
@@ -938,11 +922,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
while ((pos = essl.find("gl_PerVertex", pos)) != String::npos) {
// Walk back over whitespace to the direction keyword.
SizeT before = pos;
while (before > 0 && std::isspace(static_cast<unsigned char>(essl[before - 1]))) --before;
while (before > 0 && std::isspace(static_cast<unsigned char>(essl[before - 1])))
--before;
const Bool matches = before >= keyword.size() &&
essl.compare(before - keyword.size(), keyword.size(), keyword) == 0 &&
(before == keyword.size() ||
!isIdentifierChar(essl[before - keyword.size() - 1]));
(before == keyword.size() || !isIdentifierChar(essl[before - keyword.size() - 1]));
if (!matches) {
pos += 1;
continue;
@@ -1017,7 +1001,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
continue;
}
SizeT before = pos;
while (before > 0 && std::isspace(static_cast<unsigned char>(glslCode[before - 1]))) --before;
while (before > 0 && std::isspace(static_cast<unsigned char>(glslCode[before - 1])))
--before;
if (before > 0 && (glslCode[before - 1] == '.' || isIdentifierChar(glslCode[before - 1]))) {
continue;
}
@@ -1077,16 +1062,17 @@ namespace MobileGL::MG_Backend::DirectGLES {
for (SizeT pos = members.find("invariant "); pos != String::npos; pos = members.find("invariant ")) {
members.erase(pos, std::strlen("invariant "));
}
while (!members.empty() && std::isspace(static_cast<unsigned char>(members.back()))) members.pop_back();
while (!members.empty() && (members.front() == '\n' || members.front() == '\r')) members.erase(0, 1);
while (!members.empty() && std::isspace(static_cast<unsigned char>(members.back())))
members.pop_back();
while (!members.empty() && (members.front() == '\n' || members.front() == '\r'))
members.erase(0, 1);
String block = "in gl_PerVertex\n{\n" + members + "\n} gl_in[";
block += tessellation ? "gl_MaxPatchVertices" : "";
block += "];\n";
// After the header (#directives, precision statements and the stage's layout-only
// in/out declarations, which a geometry stage's unsized gl_in[] depends on).
static const std::regex headerRegex(
R"(^\s*(#.*|precision\s.*|layout\s*\(.*\)\s*(in|out)\s*;\s*)?$)");
static const std::regex headerRegex(R"(^\s*(#.*|precision\s.*|layout\s*\(.*\)\s*(in|out)\s*;\s*)?$)");
SizeT insertAt = 0;
SizeT lineStart = 0;
while (lineStart < glslCode.size()) {
@@ -1104,7 +1090,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
String RequestClipCullDistanceExtension(String glslCode) {
static constexpr const char* kDirective = "#extension GL_EXT_clip_cull_distance : require\n";
if (glslCode.find("GL_EXT_clip_cull_distance") != String::npos ||
(glslCode.find("gl_ClipDistance") == String::npos && glslCode.find("gl_CullDistance") == String::npos)) {
(glslCode.find("gl_ClipDistance") == String::npos &&
glslCode.find("gl_CullDistance") == String::npos)) {
return glslCode;
}
// Right after the #version line, like every other directive this backend adds.
@@ -1121,10 +1108,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
String BuildPassthroughTessControlEssl(const Uint esslVersion, const Uint patchVertices,
const String& inPerVertexMembers,
const String& outPerVertexMembers,
const FloatVec4& defaultOuterLevel,
const FloatVec2& defaultInnerLevel) {
const String& inPerVertexMembers, const String& outPerVertexMembers,
const FloatVec4& defaultOuterLevel, const FloatVec2& defaultInnerLevel) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
@@ -1176,8 +1161,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
SizeT CountIdentifierOccurrences(const String& code, const String& identifier) {
if (identifier.empty()) return 0;
SizeT count = 0;
for (SizeT pos = code.find(identifier); pos != String::npos;
pos = code.find(identifier, pos + 1)) {
for (SizeT pos = code.find(identifier); pos != String::npos; pos = code.find(identifier, pos + 1)) {
if (pos > 0 && IsImagePassIdentifierChar(code[pos - 1])) continue;
const SizeT after = pos + identifier.size();
if (after < code.size() && IsImagePassIdentifierChar(code[after])) continue;
@@ -1195,14 +1179,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
// readonly/writeonly rule only applies to a declaration that carries one of them.
Bool IsImageFormatQualifier(const String& token) {
static constexpr StringView FORMATS[] = {
"rgba32f", "rgba16f", "rg32f", "rg16f", "r11f_g11f_b10f",
"r32f", "r16f", "rgba16", "rgb10_a2", "rgba8",
"rg16", "rg8", "r16", "r8", "rgba16_snorm",
"rgba8_snorm", "rg16_snorm", "rg8_snorm", "r16_snorm", "r8_snorm",
"rgba32i", "rgba16i", "rgba8i", "rg32i", "rg16i",
"rg8i", "r32i", "r16i", "r8i", "rgba32ui",
"rgba16ui", "rgb10_a2ui", "rgba8ui", "rg32ui", "rg16ui",
"rg8ui", "r32ui", "r16ui", "r8ui",
"rgba32f", "rgba16f", "rg32f", "rg16f", "r11f_g11f_b10f",
"r32f", "r16f", "rgba16", "rgb10_a2", "rgba8",
"rg16", "rg8", "r16", "r8", "rgba16_snorm",
"rgba8_snorm", "rg16_snorm", "rg8_snorm", "r16_snorm", "r8_snorm",
"rgba32i", "rgba16i", "rgba8i", "rg32i", "rg16i",
"rg8i", "r32i", "r16i", "r8i", "rgba32ui",
"rgba16ui", "rgb10_a2ui", "rgba8ui", "rg32ui", "rg16ui",
"rg8ui", "r32ui", "r16ui", "r8ui",
};
for (const StringView format : FORMATS) {
if (token == format) return true;
@@ -1258,7 +1242,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
// How an image builtin touches the image it is handed.
enum class ImageBuiltinAccess { None, Load, Store, Unknown };
enum class ImageBuiltinAccess {
None,
Load,
Store,
Unknown
};
ImageBuiltinAccess ClassifyImageBuiltin(const String& name) {
if (name == "imageStore") return ImageBuiltinAccess::Store;
@@ -1286,7 +1275,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
String arraySuffix; // "" or "[7]"
SizeT declStart = 0;
SizeT declLength = 0;
SizeT nameStart = 0; // the name token alone, for a rename that edits nothing else
SizeT nameStart = 0; // the name token alone, for a rename that edits nothing else
SizeT nameLength = 0;
SizeT referenceCount = 0; // uses this pass recognized and accounted for
Bool loaded = false;
@@ -1347,7 +1336,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
return false;
};
while (isTaken(candidate)) candidate += 'X';
while (isTaken(candidate))
candidate += 'X';
return candidate;
}
@@ -1505,8 +1495,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// stage that was never going to compile, and guessing which element it meant
// would only change which unit it addressed wrongly.
struct ElementUse {
SizeT start; // the first character of the name
SizeT length; // through the closing ']'
SizeT start; // the first character of the name
SizeT length; // through the closing ']'
SizeT element;
};
Vector<ElementUse> uses;
@@ -1566,9 +1556,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
Vector<String> elementNames;
String replacement;
for (SizeT element = 0; element < elementCount; ++element) {
const String elementName =
MakeImageAliasName(IMAGE_ARRAY_ELEMENT_PREFIX,
plan.name + "_" + std::to_string(element), glslCode, takenNames);
const String elementName = MakeImageAliasName(
IMAGE_ARRAY_ELEMENT_PREFIX, plan.name + "_" + std::to_string(element), glslCode, takenNames);
takenNames.push_back(elementName);
elementNames.push_back(elementName);
@@ -1694,7 +1683,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
for (SizeT pos = glslCode.find("image"); pos != String::npos; pos = glslCode.find("image", pos + 1)) {
if (pos > 0 && IsImagePassIdentifierChar(glslCode[pos - 1])) continue; // uimage2D, myimageFoo
SizeT tokenEnd = pos;
while (tokenEnd < glslCode.size() && IsImagePassIdentifierChar(glslCode[tokenEnd])) ++tokenEnd;
while (tokenEnd < glslCode.size() && IsImagePassIdentifierChar(glslCode[tokenEnd]))
++tokenEnd;
const String builtin = glslCode.substr(pos, tokenEnd - pos);
const SizeT openParen = glslCode.find_first_not_of(" \t\r\n", tokenEnd);
@@ -1706,7 +1696,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
continue; // an expression, not a bare variable - it names no image of ours
}
SizeT argEnd = argStart;
while (argEnd < glslCode.size() && IsImagePassIdentifierChar(glslCode[argEnd])) ++argEnd;
while (argEnd < glslCode.size() && IsImagePassIdentifierChar(glslCode[argEnd]))
++argEnd;
const SizeT declIndex = findDecl(glslCode.substr(argStart, argEnd - argStart));
if (declIndex == decls.size()) continue;
@@ -1720,11 +1711,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
Int depth = 0;
SizeT scan = after;
for (; scan < glslCode.size(); ++scan) {
if (glslCode[scan] == '[') ++depth;
else if (glslCode[scan] == ']' && --depth == 0) break;
if (glslCode[scan] == '[')
++depth;
else if (glslCode[scan] == ']' && --depth == 0)
break;
}
after = scan >= glslCode.size() ? String::npos
: glslCode.find_first_not_of(" \t\r\n", scan + 1);
after = scan >= glslCode.size() ? String::npos : glslCode.find_first_not_of(" \t\r\n", scan + 1);
}
const char nextChar = after == String::npos ? '\0' : glslCode[after];
if (nextChar != ',' && nextChar != ')') {
@@ -1818,16 +1810,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
const char* aliasPrefix = decl.loaded && decl.stored ? IMAGE_SPLIT_READ_ALIAS_PREFIX
: decl.stored ? IMAGE_WRITEONLY_ALIAS_PREFIX
: IMAGE_READONLY_ALIAS_PREFIX;
: decl.stored ? IMAGE_WRITEONLY_ALIAS_PREFIX
: IMAGE_READONLY_ALIAS_PREFIX;
decl.aliasName = MakeImageAliasName(aliasPrefix, decl.name, glslCode, takenNames);
takenNames.push_back(decl.aliasName);
if (decl.loaded && decl.stored) {
// Minted from the ALREADY access-tagged name, so the write half of a split
// can never collide with the single declaration another stage's repair mints
// for the same image.
decl.writeName =
MakeImageAliasName(IMAGE_WRITE_ALIAS_PREFIX, decl.aliasName, glslCode, takenNames);
decl.writeName = MakeImageAliasName(IMAGE_WRITE_ALIAS_PREFIX, decl.aliasName, glslCode, takenNames);
takenNames.push_back(decl.writeName);
decl.split = true;
if (outSplitCount != nullptr) ++*outSplitCount;
@@ -1842,13 +1833,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
BuildImageDeclaration(decl, "writeonly", decl.writeName,
/*forceCoherent=*/true)});
} else if (decl.stored) {
edits.push_back({decl.declStart, decl.declLength,
BuildImageDeclaration(decl, "writeonly", decl.aliasName)});
edits.push_back(
{decl.declStart, decl.declLength, BuildImageDeclaration(decl, "writeonly", decl.aliasName)});
} else {
// Loaded only, or only ever handed to imageSize (or unused): readonly is
// the qualifier that keeps every one of those legal.
edits.push_back({decl.declStart, decl.declLength,
BuildImageDeclaration(decl, "readonly", decl.aliasName)});
edits.push_back(
{decl.declStart, decl.declLength, BuildImageDeclaration(decl, "readonly", decl.aliasName)});
}
}
for (const ImageUseSite& site : useSites) {
@@ -1856,8 +1847,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Empty exactly when the declaration was poisoned above and left untouched; its
// uses must keep naming the variable that is still called that.
if (decl.aliasName.empty()) continue;
edits.push_back(
{site.start, site.length, decl.split && site.stores ? decl.writeName : decl.aliasName});
edits.push_back({site.start, site.length, decl.split && site.stores ? decl.writeName : decl.aliasName});
if (!decl.split || !site.stores) continue;
// ...and an explicit barrier behind it. `coherent` on both halves is what makes
// the store VISIBLE to a load through the other variable, but it says nothing
@@ -1897,7 +1887,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// before the optional bias.
struct LodLookupForm {
const char* name;
Int requiredArgs; // arguments before the optional bias (implicit form)
Int requiredArgs; // arguments before the optional bias (implicit form)
Int explicitLodArg; // index of the explicit LOD argument, -1 for implicit
};
@@ -1914,15 +1904,17 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Sampler types with no mip chain, or whose GLSL lookups have no bias overload
// at all (the array-shadow forms), so nothing can or should be folded in.
Bool IsBiasableSamplerType(const String& samplerType) {
if (samplerType.find("MS") != String::npos) return false; // multisample
if (samplerType.find("Buffer") != String::npos) return false; // texture buffer
if (samplerType.find("Rect") != String::npos) return false; // rectangle: no mips
if (samplerType.find("MS") != String::npos) return false; // multisample
if (samplerType.find("Buffer") != String::npos) return false; // texture buffer
if (samplerType.find("Rect") != String::npos) return false; // rectangle: no mips
if (samplerType == "sampler2DArrayShadow") return false;
if (samplerType == "samplerCubeArrayShadow") return false;
return true;
}
Bool IsIdentifierChar(char c) { return std::isalnum(static_cast<unsigned char>(c)) || c == '_'; }
Bool IsIdentifierChar(char c) {
return std::isalnum(static_cast<unsigned char>(c)) || c == '_';
}
// Byte offsets of the top-level argument separators and of the closing paren,
// starting from the '(' at openParen. Empty when the parentheses do not balance.
@@ -2001,7 +1993,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// First argument must be one of our samplers.
const SizeT firstArgStart = result.find_first_not_of(" \t", openParen + 1);
SizeT firstArgEnd = marks.front();
while (firstArgEnd > firstArgStart && (result[firstArgEnd - 1] == ' ' || result[firstArgEnd - 1] == '\t')) {
while (firstArgEnd > firstArgStart &&
(result[firstArgEnd - 1] == ' ' || result[firstArgEnd - 1] == '\t')) {
--firstArgEnd;
}
if (firstArgStart == String::npos || firstArgEnd <= firstArgStart) continue;
@@ -2047,8 +2040,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
for (const auto& samplerName : usedSamplers) {
const String& biasName = samplerNames[samplerName];
if (result.find(String("float ") + biasName + ";") != String::npos) continue;
const std::regex declRegex(
R"(uniform\s+(?:(?:highp|mediump|lowp)\s+)?[iu]?sampler[A-Za-z0-9]*\s+)" + samplerName + R"(\s*;)");
const std::regex declRegex(R"(uniform\s+(?:(?:highp|mediump|lowp)\s+)?[iu]?sampler[A-Za-z0-9]*\s+)" +
samplerName + R"(\s*;)");
std::smatch match;
if (!std::regex_search(result, match, declRegex)) continue;
const SizeT declEnd = static_cast<SizeT>(match.position(0)) + match[0].str().size();
@@ -2089,13 +2082,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (marks.size() != 5) continue;
SizeT nameStart = result.find_first_not_of(" \t", openParen + 1);
SizeT nameEnd = nameStart;
while (nameEnd < marks[0] && IsIdentifierChar(result[nameEnd])) ++nameEnd;
while (nameEnd < marks[0] && IsIdentifierChar(result[nameEnd]))
++nameEnd;
if (names.find(result.substr(nameStart, nameEnd - nameStart)) == names.end()) continue;
const auto argument = [&](SizeT index) {
const SizeT start = marks[index - 1] + 1;
String text = result.substr(start, marks[index] - start);
text.erase(std::remove_if(text.begin(), text.end(),
[](char c) { return c == ' ' || c == '\t'; }),
text.erase(std::remove_if(text.begin(), text.end(), [](char c) { return c == ' ' || c == '\t'; }),
text.end());
return text;
};
@@ -2119,7 +2112,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// deferred, and the only ones a lookup can name directly as its first argument.
static const std::regex declRegex(
R"(uniform\s+(?:(?:highp|mediump|lowp)\s+)?usampler2DMS(?:Array)?\s+([A-Za-z_][A-Za-z0-9_]*)\s*;)");
UnorderedMap<String, String> clampNames; // sampler name -> sample-max uniform name
UnorderedMap<String, String> clampNames; // sampler name -> sample-max uniform name
Vector<std::pair<SizeT, String>> declEnds; // where each declaration ends, and its uniform
for (std::sregex_iterator it(glslCode.begin(), glslCode.end(), declRegex), end; it != end; ++it) {
const String name = (*it)[1].str();
@@ -2285,442 +2278,4 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
} // namespace Utils
// ---- Client-format readback conversion helpers -------------------------------------------------
// ReadPixels/GetTexImage read a guaranteed wide RGBA(_INTEGER) layout from the ES driver and repack
// it on the CPU into the client's (format, type) layout. Everything here is pure byte shuffling so
// unit tests can assert the exact packed words; field positions follow GL 3.3 table 3.6 and mirror
// the GL CTS packed_pixels oracle (glcPackedPixelsTests.cpp pack_UNSIGNED_* helpers).
namespace ReadbackImpl {
using MG_Util::DecodeHalfBitsToFloat;
using MG_Util::EncodeFloatToHalfBits;
Bool GetReadbackChannelMapping(GLenum format, ReadbackChannelMapping& outMapping) {
switch (format) {
case GL_RED: outMapping = {{0, 0, 0, 0}, 1, false}; return true;
case GL_RED_INTEGER: outMapping = {{0, 0, 0, 0}, 1, true}; return true;
// Desktop-GL single-channel client formats (GL CTS packed_pixels rgba8_format_green/blue):
// the destination holds one component sourced from the named channel of the wide RGBA read.
// GL_ALPHA is mapped here from the raw enum because the state layer folds it into Red for the
// legacy alpha-texture upload hack.
case GL_GREEN: outMapping = {{1, 0, 0, 0}, 1, false}; return true;
case GL_GREEN_INTEGER: outMapping = {{1, 0, 0, 0}, 1, true}; return true;
case GL_BLUE: outMapping = {{2, 0, 0, 0}, 1, false}; return true;
case GL_BLUE_INTEGER: outMapping = {{2, 0, 0, 0}, 1, true}; return true;
case GL_ALPHA: outMapping = {{3, 0, 0, 0}, 1, false}; return true;
case GL_ALPHA_INTEGER: outMapping = {{3, 0, 0, 0}, 1, true}; return true;
case GL_RG: outMapping = {{0, 1, 0, 0}, 2, false}; return true;
case GL_RG_INTEGER: outMapping = {{0, 1, 0, 0}, 2, true}; return true;
case GL_RGB: outMapping = {{0, 1, 2, 0}, 3, false}; return true;
case GL_RGB_INTEGER: outMapping = {{0, 1, 2, 0}, 3, true}; return true;
case GL_BGR: outMapping = {{2, 1, 0, 0}, 3, false}; return true;
case GL_BGR_INTEGER: outMapping = {{2, 1, 0, 0}, 3, true}; return true;
case GL_RGBA: outMapping = {{0, 1, 2, 3}, 4, false}; return true;
case GL_RGBA_INTEGER: outMapping = {{0, 1, 2, 3}, 4, true}; return true;
case GL_BGRA: outMapping = {{2, 1, 0, 3}, 4, false}; return true;
case GL_BGRA_INTEGER: outMapping = {{2, 1, 0, 3}, 4, true}; return true;
default:
return false;
}
}
Bool GetPackedReadbackLayout(GLenum type, PackedReadbackLayout& out) {
switch (type) {
// Non-REV types pack the first format component starting at the most significant bit,
// *_REV types starting at the least significant bit (GL CTS pack_UNSIGNED_SHORT_5_6_5:
// R bits 15-11; pack_UNSIGNED_SHORT_1_5_5_5_REV: R bits 4-0, A bit 15).
case GL_UNSIGNED_BYTE_3_3_2: out = {3, {3, 3, 2, 0}, {5, 2, 0, 0}, 1, false}; return true;
case GL_UNSIGNED_BYTE_2_3_3_REV: out = {3, {3, 3, 2, 0}, {0, 3, 6, 0}, 1, false}; return true;
case GL_UNSIGNED_SHORT_5_6_5: out = {3, {5, 6, 5, 0}, {11, 5, 0, 0}, 2, false}; return true;
case GL_UNSIGNED_SHORT_5_6_5_REV: out = {3, {5, 6, 5, 0}, {0, 5, 11, 0}, 2, false}; return true;
case GL_UNSIGNED_SHORT_4_4_4_4: out = {4, {4, 4, 4, 4}, {12, 8, 4, 0}, 2, false}; return true;
case GL_UNSIGNED_SHORT_4_4_4_4_REV: out = {4, {4, 4, 4, 4}, {0, 4, 8, 12}, 2, false}; return true;
case GL_UNSIGNED_SHORT_5_5_5_1: out = {4, {5, 5, 5, 1}, {11, 6, 1, 0}, 2, false}; return true;
case GL_UNSIGNED_SHORT_1_5_5_5_REV: out = {4, {5, 5, 5, 1}, {0, 5, 10, 15}, 2, false}; return true;
case GL_UNSIGNED_INT_8_8_8_8: out = {4, {8, 8, 8, 8}, {24, 16, 8, 0}, 4, false}; return true;
case GL_UNSIGNED_INT_8_8_8_8_REV: out = {4, {8, 8, 8, 8}, {0, 8, 16, 24}, 4, false}; return true;
case GL_UNSIGNED_INT_10_10_10_2: out = {4, {10, 10, 10, 2}, {22, 12, 2, 0}, 4, false}; return true;
case GL_UNSIGNED_INT_2_10_10_10_REV: out = {4, {10, 10, 10, 2}, {0, 10, 20, 30}, 4, false}; return true;
// Packed-float RGB types: fields hold unsigned small floats; 5_9_9_9_REV's shared 5-bit
// exponent (bits 31-27) is emitted by EncodeSharedExponentRGB9E5, not a component field.
case GL_UNSIGNED_INT_10F_11F_11F_REV: out = {3, {11, 11, 10, 0}, {0, 11, 22, 0}, 4, true}; return true;
case GL_UNSIGNED_INT_5_9_9_9_REV: out = {3, {9, 9, 9, 0}, {0, 9, 18, 0}, 4, true}; return true;
default:
return false;
}
}
SizeT GetReadbackComponentSize(GLenum type) {
PackedReadbackLayout packedLayout{};
if (GetPackedReadbackLayout(type, packedLayout)) {
return packedLayout.byteSize;
}
switch (type) {
case GL_UNSIGNED_BYTE:
case GL_BYTE:
return 1;
case GL_UNSIGNED_SHORT:
case GL_SHORT:
case GL_HALF_FLOAT:
return 2;
case GL_UNSIGNED_INT:
case GL_INT:
case GL_FLOAT:
return 4;
default:
return 0;
}
}
SizeT GetReadbackDstPixelSize(const ReadbackChannelMapping& mapping, GLenum type) {
PackedReadbackLayout packedLayout{};
if (GetPackedReadbackLayout(type, packedLayout)) {
if (packedLayout.fieldCount != mapping.channelCount) {
return 0; // 3-field packed types pair with 3-component formats only, 4 with 4
}
if (mapping.isInteger && packedLayout.isFloatPacked) {
return 0; // packed-float RGB types never pair with integer formats
}
return packedLayout.byteSize;
}
if (mapping.isInteger && (type == GL_FLOAT || type == GL_HALF_FLOAT)) {
return 0;
}
const SizeT componentSize = GetReadbackComponentSize(type);
return componentSize == 0 ? 0 : static_cast<SizeT>(mapping.channelCount) * componentSize;
}
namespace {
void WritePackedReadbackWord(Uint8* dst, Uint32 word, SizeT byteSize) {
switch (byteSize) {
case 1: {
const auto out = static_cast<Uint8>(word);
Memcpy(dst, &out, sizeof(out));
break;
}
case 2: {
const auto out = static_cast<Uint16>(word);
Memcpy(dst, &out, sizeof(out));
break;
}
default:
Memcpy(dst, &word, sizeof(word));
break;
}
}
} // namespace
// Shared encoders live in MG_Util/Math/SmallFloat.h so the upload conversion
// (PixelStoreProcessor) uses byte-identical packing; kept exported here for unit tests.
Uint32 EncodeFloatToUnsignedF11(Float value) { return MG_Util::EncodeFloatToUnsignedF11(value); }
Uint32 EncodeFloatToUnsignedF10(Float value) { return MG_Util::EncodeFloatToUnsignedF10(value); }
Uint32 EncodeSharedExponentRGB9E5(const Float rgb[3]) { return MG_Util::EncodeSharedExponentRGB9E5(rgb); }
void ConvertWideReadbackRow(const Uint8* src, Uint8* dst, SizeT width, GLenum wideType,
const ReadbackChannelMapping& mapping, GLenum type) {
PackedReadbackLayout packedLayout{};
const Bool isPacked = GetPackedReadbackLayout(type, packedLayout);
const SizeT dstComponentSize = GetReadbackComponentSize(type);
const SizeT dstPixelBytes = GetReadbackDstPixelSize(mapping, type);
const SizeT srcPixelBytes = 4 * GetReadbackComponentSize(wideType);
for (SizeT col = 0; col < width; ++col) {
const Uint8* srcPixel = src + col * srcPixelBytes;
Uint8* dstPixel = dst + col * dstPixelBytes;
if (mapping.isInteger) {
Int64 srcValues[4];
for (Int c = 0; c < 4; ++c) {
srcValues[c] = wideType == GL_INT
? static_cast<Int64>(reinterpret_cast<const Int32*>(srcPixel)[c])
: static_cast<Int64>(reinterpret_cast<const Uint32*>(srcPixel)[c]);
}
if (isPacked) {
// Integer sources clamp each component to the unsigned range of its field
// (GL 3.3 section 4.3.1 final conversion).
Uint32 word = 0;
for (Int ch = 0; ch < packedLayout.fieldCount; ++ch) {
const Int64 fieldMax = (Int64{1} << packedLayout.width[ch]) - 1;
const auto v = static_cast<Uint32>(
std::clamp<Int64>(srcValues[mapping.sourceChannel[ch]], 0, fieldMax));
word |= v << packedLayout.shift[ch];
}
WritePackedReadbackWord(dstPixel, word, packedLayout.byteSize);
} else {
for (Int ch = 0; ch < mapping.channelCount; ++ch) {
const Int64 v = srcValues[mapping.sourceChannel[ch]];
Uint8* dstComponent = dstPixel + static_cast<SizeT>(ch) * dstComponentSize;
switch (type) {
case GL_UNSIGNED_BYTE:
*dstComponent = static_cast<Uint8>(std::clamp<Int64>(v, 0, 255));
break;
case GL_BYTE: {
const auto out = static_cast<Int8>(std::clamp<Int64>(v, -128, 127));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_SHORT: {
const auto out = static_cast<Uint16>(std::clamp<Int64>(v, 0, 65535));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_SHORT: {
const auto out = static_cast<Int16>(std::clamp<Int64>(v, -32768, 32767));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_INT: {
const auto out = static_cast<Uint32>(std::clamp<Int64>(v, 0, 4294967295LL));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_INT: {
const auto out =
static_cast<Int32>(std::clamp<Int64>(v, -2147483648LL, 2147483647LL));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
default:
break;
}
}
}
} else {
Float srcValues[4];
switch (wideType) {
case GL_UNSIGNED_BYTE:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = static_cast<Float>(srcPixel[c]) / 255.0f;
}
break;
case GL_BYTE:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = std::max(
static_cast<Float>(reinterpret_cast<const Int8*>(srcPixel)[c]) / 127.0f, -1.0f);
}
break;
case GL_UNSIGNED_SHORT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] =
static_cast<Float>(reinterpret_cast<const Uint16*>(srcPixel)[c]) / 65535.0f;
}
break;
case GL_SHORT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = std::max(
static_cast<Float>(reinterpret_cast<const Int16*>(srcPixel)[c]) / 32767.0f, -1.0f);
}
break;
case GL_HALF_FLOAT:
for (Int c = 0; c < 4; ++c) {
srcValues[c] = DecodeHalfBitsToFloat(reinterpret_cast<const Uint16*>(srcPixel)[c]);
}
break;
default: // GL_FLOAT
for (Int c = 0; c < 4; ++c) {
srcValues[c] = reinterpret_cast<const Float*>(srcPixel)[c];
}
break;
}
if (isPacked) {
Uint32 word = 0;
if (packedLayout.isFloatPacked) {
const Float fields[3] = {srcValues[mapping.sourceChannel[0]],
srcValues[mapping.sourceChannel[1]],
srcValues[mapping.sourceChannel[2]]};
word = type == GL_UNSIGNED_INT_5_9_9_9_REV
? EncodeSharedExponentRGB9E5(fields)
: (EncodeFloatToUnsignedF11(fields[0]) << packedLayout.shift[0]) |
(EncodeFloatToUnsignedF11(fields[1]) << packedLayout.shift[1]) |
(EncodeFloatToUnsignedF10(fields[2]) << packedLayout.shift[2]);
} else {
// Normalized encode: round(clamp(v, 0, 1) * (2^bits - 1)) into each field.
for (Int ch = 0; ch < packedLayout.fieldCount; ++ch) {
const auto fieldMax = static_cast<Float>((1u << packedLayout.width[ch]) - 1u);
const auto v = static_cast<Uint32>(std::llround(
std::clamp(srcValues[mapping.sourceChannel[ch]], 0.0f, 1.0f) * fieldMax));
word |= v << packedLayout.shift[ch];
}
}
WritePackedReadbackWord(dstPixel, word, packedLayout.byteSize);
} else {
for (Int ch = 0; ch < mapping.channelCount; ++ch) {
const Float v = srcValues[mapping.sourceChannel[ch]];
Uint8* dstComponent = dstPixel + static_cast<SizeT>(ch) * dstComponentSize;
switch (type) {
case GL_UNSIGNED_BYTE:
*dstComponent =
static_cast<Uint8>(std::llround(std::clamp(v, 0.0f, 1.0f) * 255.0));
break;
case GL_BYTE: {
const auto out =
static_cast<Int8>(std::llround(std::clamp(v, -1.0f, 1.0f) * 127.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_SHORT: {
const auto out =
static_cast<Uint16>(std::llround(std::clamp(v, 0.0f, 1.0f) * 65535.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_SHORT: {
const auto out =
static_cast<Int16>(std::llround(std::clamp(v, -1.0f, 1.0f) * 32767.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_UNSIGNED_INT: {
const auto out = static_cast<Uint32>(
std::llround(static_cast<Double>(std::clamp(v, 0.0f, 1.0f)) * 4294967295.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_INT: {
const auto out = static_cast<Int32>(
std::llround(static_cast<Double>(std::clamp(v, -1.0f, 1.0f)) * 2147483647.0));
Memcpy(dstComponent, &out, sizeof(out));
break;
}
case GL_FLOAT:
Memcpy(dstComponent, &v, sizeof(v));
break;
case GL_HALF_FLOAT: {
const Uint16 out = EncodeFloatToHalfBits(v);
Memcpy(dstComponent, &out, sizeof(out));
break;
}
default:
break;
}
}
}
}
}
}
static SizeT AlignReadbackRow(SizeT rowBytes, Int alignment) {
const SizeT align = alignment > 0 ? static_cast<SizeT>(alignment) : 1;
return (rowBytes + align - 1) / align * align;
}
// Walks the client-side destination the PACK parameters describe and hands each row to
// `fillRow(slice, row, dstRow)`, which writes width * dstPixelBytes bytes of finished client
// texels. Shared by the converting and the raw-word stores so both address the destination -
// and feed the bound pixel-pack buffer - identically.
// applyPackImageParams: GL_PACK_IMAGE_HEIGHT / GL_PACK_SKIP_IMAGES apply only to GetTexImage
// of 3D/array images; ReadPixels and 2D GetTexImage ignore them (GL 3.3 sections 4.3.1, 6.1.4).
// Per the GL addressing rules, slice k row j lands at
// SKIP_IMAGES*imageStride + SKIP_ROWS*rowStride + SKIP_PIXELS*pixelBytes
// + k*imageStride + j*rowStride, with imageStride = max(IMAGE_HEIGHT, sliceHeight)*rowStride.
template <typename FillRow>
static Bool StoreClientRows(SizeT dstPixelBytes, SizeT swapGroupSize, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, void* pixels, Bool applyPackImageParams, FillRow&& fillRow) {
const auto& pixelPackBufferObject =
SplitReadbackPackBuffer();
// Destination layout is computed from the client-side PACK parameters; only the actual pixel
// rows are written so skip regions of the destination stay untouched.
const auto packParams = MG_Pipe::gPipeInputs->GetPixelStoreParameters(false);
const SizeT rowPixels = static_cast<SizeT>(packParams.RowLength > 0 ? packParams.RowLength : width);
const SizeT dstRowStride = AlignReadbackRow(rowPixels * dstPixelBytes, packParams.Alignment);
const SizeT imageRows =
applyPackImageParams && packParams.ImageHeight > 0
? static_cast<SizeT>(packParams.ImageHeight)
: static_cast<SizeT>(sliceHeight);
const SizeT dstImageStride = imageRows * dstRowStride;
const SizeT skipImages =
applyPackImageParams ? static_cast<SizeT>(std::max(packParams.SkipImages, 0)) : SizeT{0};
const SizeT dstSkipOffset = skipImages * dstImageStride +
static_cast<SizeT>(std::max(packParams.SkipRows, 0)) * dstRowStride +
static_cast<SizeT>(std::max(packParams.SkipPixels, 0)) * dstPixelBytes;
const SizeT dstRowBytes = static_cast<SizeT>(width) * dstPixelBytes;
const SizeT pboBaseOffset = reinterpret_cast<SizeT>(pixels); // with a PBO, `pixels` is an offset
if (pixelPackBufferObject) {
const SizeT requiredSize = pboBaseOffset + dstSkipOffset +
static_cast<SizeT>(sliceCount - 1) * dstImageStride +
static_cast<SizeT>(sliceHeight - 1) * dstRowStride + dstRowBytes;
if (requiredSize > pixelPackBufferObject->GetSize()) {
MGLOG_E_ONCE("Readback conversion: pixel pack buffer is too small");
return true;
}
}
Vector<Uint8> convertedRow(dstRowBytes);
for (GLsizei slice = 0; slice < sliceCount; ++slice) {
for (GLsizei row = 0; row < sliceHeight; ++row) {
fillRow(slice, row, convertedRow.data());
if (packParams.SwapBytes && swapGroupSize > 1) {
for (SizeT offset = 0; offset + swapGroupSize <= dstRowBytes; offset += swapGroupSize) {
std::reverse(convertedRow.data() + offset, convertedRow.data() + offset + swapGroupSize);
}
}
const SizeT dstOffset = dstSkipOffset + static_cast<SizeT>(slice) * dstImageStride +
static_cast<SizeT>(row) * dstRowStride;
if (pixelPackBufferObject) {
pixelPackBufferObject->WritebackFromBackend({convertedRow.data(), dstRowBytes},
pboBaseOffset + dstOffset);
} else {
Memcpy(static_cast<Uint8*>(pixels) + dstOffset, convertedRow.data(), dstRowBytes);
}
}
}
if (pixelPackBufferObject) {
// WritebackFromBackend bumps change serials with no backend op; re-open
// the buffer draw-clean memos (once for the whole row loop).
BufferImpl::BumpBufferMutationEpoch();
}
return true;
}
// Repacks wide RGBA(_INTEGER) rows into the client's (format, type) layout, honoring the
// client-side PACK parameters and the bound pixel-pack buffer. `wide` holds
// `sliceHeight * sliceCount` rows of `width` texels (slice-major, tightly stacked),
// 4 components x GetReadbackComponentSize(wideType) bytes each.
Bool StoreWideRowsToClient(const Uint8* wide, GLenum wideType, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, const ReadbackChannelMapping& mapping, GLenum type,
void* pixels, Bool applyPackImageParams) {
const SizeT dstPixelBytes = GetReadbackDstPixelSize(mapping, type);
if (dstPixelBytes == 0) {
return false;
}
PackedReadbackLayout packedLayout{};
const Bool isPackedType = GetPackedReadbackLayout(type, packedLayout);
const SizeT swapGroupSize = isPackedType ? packedLayout.byteSize : GetReadbackComponentSize(type);
const SizeT srcPixelBytes = 4 * GetReadbackComponentSize(wideType);
return StoreClientRows(dstPixelBytes, swapGroupSize, width, sliceHeight, sliceCount, pixels,
applyPackImageParams,
[&](GLsizei slice, GLsizei row, Uint8* dstRow) {
const SizeT flatRow = static_cast<SizeT>(slice) *
static_cast<SizeT>(sliceHeight) +
static_cast<SizeT>(row);
const Uint8* srcRow =
wide + flatRow * static_cast<SizeT>(width) * srcPixelBytes;
ConvertWideReadbackRow(srcRow, dstRow, static_cast<SizeT>(width), wideType,
mapping, type);
});
}
Bool StorePackedWordsToClient(const Uint8* srcWords, GLsizei width, GLsizei sliceHeight, GLsizei sliceCount,
GLenum type, void* pixels, Bool applyPackImageParams) {
PackedReadbackLayout packedLayout{};
if (!GetPackedReadbackLayout(type, packedLayout) || packedLayout.byteSize != 4) {
return false;
}
const SizeT srcRowBytes = static_cast<SizeT>(width) * 4;
return StoreClientRows(4, packedLayout.byteSize, width, sliceHeight, sliceCount, pixels,
applyPackImageParams,
[&](GLsizei slice, GLsizei row, Uint8* dstRow) {
const SizeT flatRow = static_cast<SizeT>(slice) *
static_cast<SizeT>(sliceHeight) +
static_cast<SizeT>(row);
Memcpy(dstRow, srcWords + flatRow * srcRowBytes, srcRowBytes);
});
}
} // namespace ReadbackImpl
} // namespace MobileGL::MG_Backend::DirectGLES
+3 -69
View File
@@ -193,69 +193,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
namespace FramebufferImpl {} // namespace FramebufferImpl
// Pure CPU helpers of the client-format readback conversion (ReadPixels/GetTexImage repack a wide
// RGBA(_INTEGER) read into the caller's (format, type) layout). Kept context-free so unit tests can
// exercise the exact packing the GL CTS packed_pixels oracle compares against.
namespace ReadbackImpl {
struct ReadbackChannelMapping {
Int sourceChannel[4]; // RGBA source channel feeding each destination component
Int channelCount; // destination component count
Bool isInteger;
};
Bool GetReadbackChannelMapping(GLenum format, ReadbackChannelMapping& outMapping);
// Byte size of one destination component of `type`; packed types report the packed word size.
// 0 = type not supported by the conversion path.
SizeT GetReadbackComponentSize(GLenum type);
// Bit-field layout of a GL packed pixel type. width/shift are indexed in the client format's
// component order (matching ReadbackChannelMapping); shift is the LSB position of the field in
// the packed word: non-REV types pack the first component from the MSB, *_REV types from the
// LSB (GL 3.3 table 3.6; field positions mirror the GL CTS glcPackedPixelsTests pack_* oracle).
struct PackedReadbackLayout {
Int fieldCount; // format components stored in the packed word
Int width[4]; // bit width of each component's field
Int shift[4]; // LSB bit position of each component's field
SizeT byteSize; // packed word size in bytes (1, 2 or 4)
Bool isFloatPacked; // 10F_11F_11F_REV / 5_9_9_9_REV: fields hold unsigned small floats
};
Bool GetPackedReadbackLayout(GLenum type, PackedReadbackLayout& out);
// Unsigned small-float encoders (EXT_packed_float / EXT_texture_shared_exponent semantics).
Uint32 EncodeFloatToUnsignedF11(Float value);
Uint32 EncodeFloatToUnsignedF10(Float value);
Uint32 EncodeSharedExponentRGB9E5(const Float rgb[3]);
// Destination bytes per pixel for a (format mapping, type) readback pair; 0 when the pair is
// not convertible (unknown type, packed field count != format component count, floating-point
// or packed-float type with an integer format).
SizeT GetReadbackDstPixelSize(const ReadbackChannelMapping& mapping, GLenum type);
// Repacks one row of wide RGBA(_INTEGER) texels (4 components of wideType each) into the
// client's (format, type) layout. src holds width * 4 * GetReadbackComponentSize(wideType)
// bytes, dst receives width * GetReadbackDstPixelSize(mapping, type) bytes.
void ConvertWideReadbackRow(const Uint8* src, Uint8* dst, SizeT width, GLenum wideType,
const ReadbackChannelMapping& mapping, GLenum type);
// Stores wide RGBA(_INTEGER) rows into the client pointer or the bound PACK pixel buffer,
// honoring the client-side PACK pixel-store parameters (row length, alignment, skips,
// swap-bytes, and - when applyPackImageParams - image height/skip images). Shared by the
// DirectGLES and DirectVulkan readback conversion paths.
Bool StoreWideRowsToClient(const Uint8* wide, GLenum wideType, GLsizei width, GLsizei sliceHeight,
GLsizei sliceCount, const ReadbackChannelMapping& mapping, GLenum type,
void* pixels, Bool applyPackImageParams);
// Stores packed 32-bit source words verbatim, with the same destination addressing, PACK
// parameters and pixel-pack-buffer handling as StoreWideRowsToClient. For the sources whose
// storage word already IS the client word (MG_Util::IsRawPackedPixelTransfer): routing those
// through the wide float intermediate re-encodes them, and the RGB9_E5 encoder canonicalizes
// the shared exponent, so glGetTexImage would answer with different bits than were stored.
// `srcWords` holds sliceHeight * sliceCount tightly stacked rows of `width` 32-bit words.
// False when `type` is not a 4-byte packed type.
Bool StorePackedWordsToClient(const Uint8* srcWords, GLsizei width, GLsizei sliceHeight, GLsizei sliceCount,
GLenum type, void* pixels, Bool applyPackImageParams);
} // namespace ReadbackImpl
namespace PrgramImpl {
String ProcessOutColorLocations(const String& glslCode);
String ForceSupporterOutput(const String& glslCode);
@@ -274,8 +211,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// On a driver that advertises only GL_OES_texture_buffer that directive is a compile
// error, so the name is retargeted in the emitted source. A no-op on every other tier:
// ES 3.2 needs no directive at all and an EXT driver already has the right one.
String RetargetTextureBufferExtension(String glslCode,
MG_External::GLESCapabilities::TextureBufferTier tier);
String RetargetTextureBufferExtension(String glslCode, MG_External::GLESCapabilities::TextureBufferTier tier);
// Adds `#extension GL_NV_image_formats : require` when the shader carries an image
// format qualifier GLSL ES has no core spelling for. SPIRV-Cross prints the format and
// asks for nothing, so the request has to be made here. `needed` is the caller's answer,
@@ -447,10 +383,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// disagree on everything but the algorithm: desktop GLSL 450 against ESSL, a fixed
// gl_PerVertex shape that Vulkan matches structurally against a mirrored one, and a
// VkShaderModule against a driver shader object.
String BuildPassthroughTessControlEssl(Uint esslVersion, Uint patchVertices,
const String& inPerVertexMembers,
const String& outPerVertexMembers,
const FloatVec4& defaultOuterLevel,
String BuildPassthroughTessControlEssl(Uint esslVersion, Uint patchVertices, const String& inPerVertexMembers,
const String& outPerVertexMembers, const FloatVec4& defaultOuterLevel,
const FloatVec2& defaultInnerLevel);
// Prefix of the writeonly half a read+write image uniform is split into (see
// SplitReadWriteImageUniforms); the suffix is the image's own (already access-tagged) name.
@@ -1,469 +0,0 @@
// MobileGL - MobileGL/MG_Backend/DirectGLES/WireTextureReadback.inc
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// Included in DirectGLES.cpp after the native packed-word readback helper.
// A storage promotion needs the driver's own texels as owned tight rows, not
// client PACK layout or a fallback into a frontend texture's potentially stale
// shadow.
class ScopedWireTextureReadLevel {
public:
ScopedWireTextureReadLevel(GLuint texture, GLenum target, GLint level)
: m_texture(texture), m_target(target) {
switch (target) {
case GL_TEXTURE_2D: m_binding = GL_TEXTURE_BINDING_2D; break;
case GL_TEXTURE_3D: m_binding = GL_TEXTURE_BINDING_3D; break;
case GL_TEXTURE_2D_ARRAY: m_binding = GL_TEXTURE_BINDING_2D_ARRAY; break;
case GL_TEXTURE_CUBE_MAP: m_binding = GL_TEXTURE_BINDING_CUBE_MAP; break;
case GL_TEXTURE_CUBE_MAP_ARRAY: m_binding = GL_TEXTURE_BINDING_CUBE_MAP_ARRAY; break;
default: return;
}
GLint previous = 0;
DrainESErrors();
g_GLESFuncs.glGetIntegerv(m_binding, &previous);
g_GLESFuncs.glBindTexture(m_target, m_texture);
g_GLESFuncs.glGetTexParameteriv(m_target, GL_TEXTURE_BASE_LEVEL, &m_baseLevel);
g_GLESFuncs.glGetTexParameteriv(m_target, GL_TEXTURE_MAX_LEVEL, &m_maxLevel);
m_saved = g_GLESFuncs.glGetError() == GL_NO_ERROR;
if (m_saved) {
// ES framebuffer completeness can reject a defined level whose
// base level or intervening mip chain is missing/inconsistent.
// Read this existing native image as the sole base level. The
// logical GL parameters and every cached copy remain unchanged.
g_GLESFuncs.glTexParameteri(m_target, GL_TEXTURE_BASE_LEVEL, level);
g_GLESFuncs.glTexParameteri(m_target, GL_TEXTURE_MAX_LEVEL, level);
m_valid = g_GLESFuncs.glGetError() == GL_NO_ERROR;
}
g_GLESFuncs.glBindTexture(m_target, static_cast<GLuint>(previous));
}
~ScopedWireTextureReadLevel() {
if (!m_saved) return;
// The packed-word path can use its own texture scratch binding.
// Save the binding at restoration time too; do not assume an active
// texture unit or make the backend's binding cache forget a bind.
GLint previous = 0;
g_GLESFuncs.glGetIntegerv(m_binding, &previous);
g_GLESFuncs.glBindTexture(m_target, m_texture);
g_GLESFuncs.glTexParameteri(m_target, GL_TEXTURE_BASE_LEVEL, m_baseLevel);
g_GLESFuncs.glTexParameteri(m_target, GL_TEXTURE_MAX_LEVEL, m_maxLevel);
g_GLESFuncs.glBindTexture(m_target, static_cast<GLuint>(previous));
}
Bool Valid() const { return m_valid; }
private:
GLuint m_texture;
GLenum m_target;
GLenum m_binding = GL_NONE;
GLint m_baseLevel = 0, m_maxLevel = 0;
Bool m_saved = false, m_valid = false;
};
static Bool ReadWireColorRowsTight(GLsizei width, GLsizei height, GLenum format, GLenum type,
Bool forceOpaqueAlpha, const NormalizedImageCarrierRead& carrier,
Vector<Uint8>& bytes) {
ReadbackChannelMapping mapping{};
if (!GetReadbackChannelMapping(format, mapping)) return false;
const SizeT pixelBytes = GetReadbackDstPixelSize(mapping, type);
if (!pixelBytes || width <= 0 || height <= 0) return false;
const GLenum componentType = QueryReadAttachmentComponentType();
const Bool integerAttachment = componentType == GL_INT || componentType == GL_UNSIGNED_INT;
if (carrier.Active() ? (mapping.isInteger || !integerAttachment)
: mapping.isInteger != integerAttachment) return false;
GLint implFormat = 0, implType = 0;
g_GLESFuncs.glGetIntegerv(GL_IMPLEMENTATION_COLOR_READ_FORMAT, &implFormat);
g_GLESFuncs.glGetIntegerv(GL_IMPLEMENTATION_COLOR_READ_TYPE, &implType);
struct Candidate { GLenum format, type; };
Vector<Candidate> candidates;
if (carrier.Active()) {
candidates.push_back({GL_RGBA_INTEGER, GL_UNSIGNED_INT});
} else if (mapping.isInteger) {
if (GetWideReadChannelCount(implFormat) > 0 && IsIntegerReadFormat(implFormat) &&
(implType == GL_INT || implType == GL_UNSIGNED_INT))
candidates.push_back({static_cast<GLenum>(implFormat), static_cast<GLenum>(implType)});
candidates.push_back({GL_RGBA_INTEGER, componentType == GL_INT ? GLenum(GL_INT) : GLenum(GL_UNSIGNED_INT)});
} else {
if (GetWideReadChannelCount(implFormat) > 0 && !IsIntegerReadFormat(implFormat) &&
(CanDecodeWideSourceType(implType) ||
(implFormat == GL_RGBA && implType == GL_UNSIGNED_INT_2_10_10_10_REV)))
candidates.push_back({static_cast<GLenum>(implFormat), static_cast<GLenum>(implType)});
if (componentType == GL_FLOAT) candidates.push_back({GL_RGBA, GL_FLOAT});
if (componentType == GL_SIGNED_NORMALIZED) {
candidates.push_back({GL_RGBA, GL_SHORT});
candidates.push_back({GL_RGBA, GL_BYTE});
} else candidates.push_back({GL_RGBA, GL_UNSIGNED_BYTE});
}
ScopedPixelPackBuffer packBuffer(0);
ScopedPackState packState(PixelStoreImpl::PackState{1, 0, 0, 0});
const SizeT pixels = static_cast<SizeT>(width) * static_cast<SizeT>(height);
Vector<Uint8> wide;
GLenum wideType = GL_NONE;
Int channels = 0;
DrainESErrors();
for (const auto candidate : candidates) {
channels = GetWideReadChannelCount(candidate.format);
wide.resize(pixels * static_cast<SizeT>(channels) * GetReadbackComponentSize(candidate.type));
g_GLESFuncs.glReadPixels(0, 0, width, height, candidate.format, candidate.type, wide.data());
if (g_GLESFuncs.glGetError() == GL_NO_ERROR) { wideType = candidate.type; break; }
}
if (wideType == GL_NONE) return false;
if (carrier.Active()) {
Vector<Uint8> decoded(pixels * 4 * sizeof(Float));
auto* dst = reinterpret_cast<Float*>(decoded.data());
const auto* src = reinterpret_cast<const Uint32*>(wide.data());
for (SizeT i = 0; i < pixels; ++i) {
for (SizeT c = 0; c < 4; ++c) {
const Float divisor = static_cast<Float>(carrier.ChannelMax[c]);
dst[i * 4 + c] = carrier.SignedNormalized
? std::max(static_cast<Float>(static_cast<Int16>(static_cast<Uint16>(src[i * 4 + c]))) / divisor, -1.0f)
: static_cast<Float>(src[i * 4 + c]) / divisor;
}
}
wide = Move(decoded); wideType = GL_FLOAT; channels = 4;
} else if (wideType == GL_UNSIGNED_INT_2_10_10_10_REV) {
Vector<Uint8> decoded(pixels * 4 * sizeof(Float));
auto* dst = reinterpret_cast<Float*>(decoded.data());
for (SizeT i = 0; i < pixels; ++i) {
Uint32 word = 0;
Memcpy(&word, wide.data() + i * sizeof(word), sizeof(word));
dst[i * 4] = static_cast<Float>(word & 1023u) / 1023.0f;
dst[i * 4 + 1] = static_cast<Float>((word >> 10) & 1023u) / 1023.0f;
dst[i * 4 + 2] = static_cast<Float>((word >> 20) & 1023u) / 1023.0f;
dst[i * 4 + 3] = static_cast<Float>((word >> 30) & 3u) / 3.0f;
}
wide = Move(decoded); wideType = GL_FLOAT; channels = 4;
}
if (channels < 4) ExpandNarrowWideRead(wide, pixels, channels, wideType);
if (forceOpaqueAlpha) ForceWideReadAlphaToOne(wide, pixels, wideType);
Vector<Uint8> converted(pixels * pixelBytes);
const SizeT sourceRowBytes = static_cast<SizeT>(width) * 4 * GetReadbackComponentSize(wideType);
for (GLsizei row = 0; row < height; ++row)
ReadbackImpl::ConvertWideReadbackRow(wide.data() + static_cast<SizeT>(row) * sourceRowBytes,
converted.data() + static_cast<SizeT>(row) * width * pixelBytes, width, wideType, mapping, type);
bytes = Move(converted);
return true;
}
static Bool ReadWireDepthStencilLevelTight(GLuint texture, TextureTarget target,
TextureUploadTarget uploadTarget, GLint level,
const IntVec3& extent, GLenum format, GLenum type,
Vector<Uint8>& bytes) {
const SizeT pixelBytes = MG_Util::GetInputBytesPerPixel(
MG_Util::ConvertGLEnumToTextureInputFormat(format), MG_Util::ConvertGLEnumToTexturePixelDataType(type));
if (!pixelBytes) return false;
const Bool needDepth = format == GL_DEPTH_COMPONENT || format == GL_DEPTH_STENCIL;
const Bool needStencil = format == GL_STENCIL_INDEX || format == GL_DEPTH_STENCIL;
ScopedFramebufferBinding bindings(true, false);
GLuint fbo = 0;
g_GLESFuncs.glGenFramebuffers(1, &fbo);
struct DeleteFbo { GLuint id; ~DeleteFbo() { g_GLESFuncs.glDeleteFramebuffers(1, &id); } } cleanup{fbo};
FramebufferImpl::BindFramebufferId(GL_READ_FRAMEBUFFER, fbo);
ScopedPixelPackBuffer packBuffer(0);
ScopedPackState packState(PixelStoreImpl::PackState{1, 0, 0, 0});
const GLenum nativeTarget = TextureImpl::ConvertTextureTargetToBackendGLEnum(target);
const GLenum point = needDepth ? (needStencil ? GL_DEPTH_STENCIL_ATTACHMENT : GL_DEPTH_ATTACHMENT)
: GL_STENCIL_ATTACHMENT;
const SizeT count = static_cast<SizeT>(extent.x()) * extent.y();
Vector<Uint8> result(count * extent.z() * pixelBytes);
if (g_GLESFuncs.glMemoryBarrier) g_GLESFuncs.glMemoryBarrier(GL_ALL_BARRIER_BITS);
g_GLESFuncs.glFinish();
for (GLint z = 0; z < extent.z(); ++z) {
if (nativeTarget == GL_TEXTURE_3D || nativeTarget == GL_TEXTURE_2D_ARRAY ||
nativeTarget == GL_TEXTURE_CUBE_MAP_ARRAY)
g_GLESFuncs.glFramebufferTextureLayer(GL_READ_FRAMEBUFFER, point, texture, level, z);
else
g_GLESFuncs.glFramebufferTexture2D(GL_READ_FRAMEBUFFER, point,
TextureImpl::ConvertTextureUploadTargetToBackendGLEnum(uploadTarget), texture, level);
g_GLESFuncs.glReadBuffer(GL_NONE);
if (g_GLESFuncs.glCheckFramebufferStatus(GL_READ_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) return false;
Vector<Float> depth;
Vector<Uint8> stencil;
if (needDepth && !ReadDepthValuesNative(0, 0, extent.x(), extent.y(), depth)) return false;
if (needStencil && !ReadStencilBytesNative(0, 0, extent.x(), extent.y(), stencil)) return false;
auto* dst = result.data() + count * z * pixelBytes;
for (SizeT i = 0; i < count; ++i) {
if (format == GL_DEPTH_STENCIL) {
const Uint32 s = stencil[i];
if (type == GL_UNSIGNED_INT_24_8) {
const Uint32 packed = (NormalizedDepthToUnsigned(depth[i], 16777215.0) << 8) | s;
Memcpy(dst + i * pixelBytes, &packed, sizeof(packed));
} else if (type == GL_FLOAT_32_UNSIGNED_INT_24_8_REV) {
Memcpy(dst + i * pixelBytes, &depth[i], sizeof(Float));
Memcpy(dst + i * pixelBytes + sizeof(Float), &s, sizeof(s));
} else return false;
} else if (type == GL_FLOAT) {
const Float value = needDepth ? depth[i] : static_cast<Float>(stencil[i]);
Memcpy(dst + i * pixelBytes, &value, sizeof(value));
} else if (type == GL_HALF_FLOAT) {
const Uint16 value = MG_Util::EncodeFloatToHalfBits(
needDepth ? depth[i] : static_cast<Float>(stencil[i]));
Memcpy(dst + i * pixelBytes, &value, sizeof(value));
} else {
const Bool signedType = type == GL_BYTE || type == GL_SHORT || type == GL_INT;
const Double maximum = signedType
? (pixelBytes == 1 ? 127.0 : pixelBytes == 2 ? 32767.0 : 2147483647.0)
: (pixelBytes == 1 ? 255.0 : pixelBytes == 2 ? 65535.0 : 4294967295.0);
const Uint32 value = needDepth ? NormalizedDepthToUnsigned(depth[i], maximum) : stencil[i];
if (pixelBytes == 1) dst[i] = static_cast<Uint8>(value);
else if (pixelBytes == 2) { const Uint16 v = static_cast<Uint16>(value); Memcpy(dst + i * 2, &v, 2); }
else if (pixelBytes == 4) Memcpy(dst + i * 4, &value, 4);
else return false;
}
}
}
bytes = Move(result);
return true;
}
Bool ReadTextureLevelTight(GLuint texture, TextureTarget target, TextureUploadTarget uploadTarget,
TextureInternalFormat logicalFormat, GLint level, const IntVec3& logicalExtent,
Bool sourceUsesImageCarrier, GLenum format, GLenum type, Vector<Uint8>& bytes) {
if (!texture || level < 0 || TextureImpl::IsMultisampleTextureTarget(target)) return false;
const IntVec3 extent = TextureImpl::GetBackendUploadSize(target, logicalExtent);
const GLenum nativeTarget = TextureImpl::ConvertTextureTargetToBackendGLEnum(target);
ScopedWireTextureReadLevel readLevel(texture, nativeTarget, level);
if (!readLevel.Valid()) return false;
if (format == GL_DEPTH_COMPONENT || format == GL_DEPTH_STENCIL || format == GL_STENCIL_INDEX)
return ReadWireDepthStencilLevelTight(texture, target, uploadTarget, level, extent, format, type, bytes);
ReadbackChannelMapping mapping{};
if (!GetReadbackChannelMapping(format, mapping)) return false;
const SizeT pixelBytes = GetReadbackDstPixelSize(mapping, type);
if (!pixelBytes || extent.x() <= 0 || extent.y() <= 0 || extent.z() <= 0) return false;
const GLenum faceTarget = TextureImpl::ConvertTextureUploadTargetToBackendGLEnum(uploadTarget);
const auto widening = sourceUsesImageCarrier
? TextureImpl::GetImageBindableStorageWidening(logicalFormat) : TextureImpl::ImageBindableStorageWidening{};
const SizeT sliceBytes = static_cast<SizeT>(extent.x()) * extent.y() * pixelBytes;
Vector<Uint8> result(sliceBytes * static_cast<SizeT>(extent.z()));
if (g_GLESFuncs.glMemoryBarrier) g_GLESFuncs.glMemoryBarrier(GL_ALL_BARRIER_BITS);
g_GLESFuncs.glFinish();
// A raw packed word may have redundant encodings or NaN payloads. Read
// it through R32UI when the source really still uses that native layout.
const Bool rawPacked = !widening && MG_Util::PixelStoreProcessor::IsRawPackedPixelTransfer(
logicalFormat, MG_Util::ConvertGLEnumToTextureInputFormat(format),
MG_Util::ConvertGLEnumToTexturePixelDataType(type));
if (rawPacked && pixelBytes == sizeof(Uint32)) {
const GLint face = uploadTarget >= TextureUploadTarget::CubeMapPositiveX &&
uploadTarget <= TextureUploadTarget::CubeMapNegativeZ
? static_cast<GLint>(uploadTarget) - static_cast<GLint>(TextureUploadTarget::CubeMapPositiveX) : 0;
for (GLint layer = 0; layer < extent.z(); ++layer)
if (!ReadPackedLevelWordsViaScratch(texture, nativeTarget, level, face + layer, extent.x(), extent.y(),
reinterpret_cast<Uint32*>(result.data() + static_cast<SizeT>(layer) * sliceBytes))) return false;
bytes = Move(result);
return true;
}
NormalizedImageCarrierRead carrier{};
if (widening && widening.CarriesNormalizedCodes()) {
for (SizeT c = 0; c < 4; ++c) carrier.ChannelMax[c] = widening.ChannelMax[c];
carrier.SignedNormalized = widening.SignedNormalized;
}
TempFBOBinder binder(true);
auto& framebuffer = binder.Framebuffer();
for (GLint layer = 0; layer < extent.z(); ++layer) {
if (nativeTarget == GL_TEXTURE_3D || nativeTarget == GL_TEXTURE_2D_ARRAY || nativeTarget == GL_TEXTURE_CUBE_MAP_ARRAY)
ScratchFBOImpl::EnsureColorAttachmentLayer(framebuffer, GL_READ_FRAMEBUFFER, texture, level, layer);
else
ScratchFBOImpl::EnsureColorAttachment2D(framebuffer, GL_READ_FRAMEBUFFER, texture, faceTarget, level);
ScratchFBOImpl::EnsureReadBuffer(framebuffer, GL_COLOR_ATTACHMENT0);
const GLenum status = g_GLESFuncs.glCheckFramebufferStatus(GL_READ_FRAMEBUFFER);
if (status != GL_FRAMEBUFFER_COMPLETE) {
MGLOG_E_ONCE("Espryt wire texture readback: framebuffer status=0x%x texture=%u level=%d target=0x%x",
status, texture, level, nativeTarget);
return false;
}
Vector<Uint8> slice;
if (!ReadWireColorRowsTight(extent.x(), extent.y(), format, type,
TextureImpl::BackendTextureFormatAddsAlpha(logicalFormat, target), carrier, slice) ||
slice.size() != sliceBytes) return false;
Memcpy(result.data() + static_cast<SizeT>(layer) * sliceBytes, slice.data(), sliceBytes);
}
bytes = Move(result);
return true;
}
// P9 W2-a. The store's Defined-ness follows the client's respecifies; the driver's follows the
// twin's last sync. A re-mint asks this before reading a level back, because a level defined
// after that sync (null data, or null data plus a partial upload) is not on the driver yet and
// its FBO attachment is incomplete - which used to end the session at
// Fatal{ResourceUnavailable, "image-promotion-readback"} on legal GL.
Bool NativeTextureLevelHasExtent(GLuint texture, TextureTarget target, TextureUploadTarget uploadTarget,
GLint level, const IntVec3& logicalExtent) {
if (!texture || level < 0 || !g_GLESFuncs.glGetTexLevelParameteriv) return true;
const IntVec3 extent = TextureImpl::GetBackendUploadSize(target, logicalExtent);
const GLenum nativeTarget = TextureImpl::ConvertTextureTargetToBackendGLEnum(target);
GLenum binding = GL_NONE;
switch (nativeTarget) {
case GL_TEXTURE_2D: binding = GL_TEXTURE_BINDING_2D; break;
case GL_TEXTURE_3D: binding = GL_TEXTURE_BINDING_3D; break;
case GL_TEXTURE_2D_ARRAY: binding = GL_TEXTURE_BINDING_2D_ARRAY; break;
case GL_TEXTURE_CUBE_MAP: binding = GL_TEXTURE_BINDING_CUBE_MAP; break;
case GL_TEXTURE_CUBE_MAP_ARRAY: binding = GL_TEXTURE_BINDING_CUBE_MAP_ARRAY; break;
default: return true;
}
// A cube's levels are per face; every other target answers for the whole level.
const GLenum queryTarget = nativeTarget == GL_TEXTURE_CUBE_MAP
? TextureImpl::ConvertTextureUploadTargetToBackendGLEnum(uploadTarget) : nativeTarget;
DrainESErrors();
GLint previous = 0;
g_GLESFuncs.glGetIntegerv(binding, &previous);
g_GLESFuncs.glBindTexture(nativeTarget, texture);
GLint width = 0, height = 0, depth = 0;
g_GLESFuncs.glGetTexLevelParameteriv(queryTarget, level, GL_TEXTURE_WIDTH, &width);
g_GLESFuncs.glGetTexLevelParameteriv(queryTarget, level, GL_TEXTURE_HEIGHT, &height);
g_GLESFuncs.glGetTexLevelParameteriv(queryTarget, level, GL_TEXTURE_DEPTH, &depth);
const Bool answered = g_GLESFuncs.glGetError() == GL_NO_ERROR;
g_GLESFuncs.glBindTexture(nativeTarget, static_cast<GLuint>(previous));
if (!answered) return true;
const Bool layered = nativeTarget == GL_TEXTURE_3D || nativeTarget == GL_TEXTURE_2D_ARRAY ||
nativeTarget == GL_TEXTURE_CUBE_MAP_ARRAY;
return width == extent.x() && height == extent.y() && (!layered || depth == extent.z());
}
// P8-E (docs/archive/notes/p8/E.md). MGITEST_ESPRYT_REFUSE_TEXTURE_READBACK_EXTENT=<w>x<h>, READ BY THE SERVER:
// a colour level of exactly that extent is treated as one the driver refused to attach - what an
// Adreno does for RGB16 and the 16-bit SNORM formats, and what no host driver can be made to do.
// Scoped by the extent so the tcp lane's shared supervisor can carry it for the whole lane: only
// the scenario that builds that extent (CopyImageStoreReadbackScenario) ever meets it.
static Bool WireTextureReadbackRefusedForTesting(const IntVec3& extent) {
static const IntVec3 s_refused = [] {
const char* value = std::getenv("MGITEST_ESPRYT_REFUSE_TEXTURE_READBACK_EXTENT");
if (value == nullptr) return IntVec3{0, 0, 0};
char* end = nullptr;
const long width = std::strtol(value, &end, 10);
if (end == value || *end != 'x') return IntVec3{0, 0, 0};
const long height = std::strtol(end + 1, nullptr, 10);
if (width <= 0 || height <= 0) return IntVec3{0, 0, 0};
return IntVec3{static_cast<Int>(width), static_cast<Int>(height), 1};
}();
return s_refused.x() > 0 && extent == s_refused;
}
// P8-E: THE STORE ROUTE, for a level the driver refused to read back. The server's staged store
// is the only CPU copy of the level on this side, and it answers only while its bytes ARE the
// level: every byte crossed (IsCovered), at the extent asked for, and the texture was never
// written by the driver except by copies the store followed (!IsDriverWritten - see
// StagedTextureStore's P8-E block and FollowCopyImageInStagedStore). The conversion is
// GetTexImageViaShadowConversion's, into tight rows: a packed format read with its own type
// takes the stored words verbatim, everything else decodes to wide RGBA and converts row by row.
// Anything else DECLINES BY NAME: the reply is an error, and the client's
// Fatal{ReplyError, "ReadPixels"} ends the session instead of handing back bytes the driver has
// since overwritten.
static Bool ReadWireLevelFromStagedStore(MG_Pipe::MGPipeHandle res, TextureInternalFormat logicalFormat,
TextureUploadTarget uploadTarget, GLint level, const IntVec3& extent,
GLenum format, GLenum type, Vector<Uint8>& bytes) {
auto& store = MG_Record::ServerStagedTexture();
const Uint64 key = MG_Record::StagedTextureStore::KeyForHandle(res);
const auto target = static_cast<Uint16>(uploadTarget);
const auto mip = static_cast<Uint16>(level);
ReadbackChannelMapping mapping{};
const SizeT pixelBytes = GetReadbackChannelMapping(format, mapping) ? GetReadbackDstPixelSize(mapping, type) : 0;
const SizeT texels = static_cast<SizeT>(std::max(extent.x(), 0)) * static_cast<SizeT>(std::max(extent.y(), 0)) *
static_cast<SizeT>(std::max(extent.z(), 0));
const char* declined = nullptr;
if (!store.Holds()) {
declined = "this arm keeps no staged store";
} else if (store.IsDriverWritten(key)) {
declined = "the driver has written the texture since its bytes crossed";
} else if (!store.IsCovered(key, target, mip)) {
declined = "not every byte of the level crossed";
} else if (store.LevelExtentOrUndefined(key, target, mip) != extent) {
declined = "the staged level is not the extent asked for";
} else if (pixelBytes == 0 || texels == 0) {
declined = "the format/type pair has no readback mapping";
}
Vector<Uint8> out;
if (declined == nullptr) {
const Uint8* staged = store.RequireLevelBytes(key, target, mip, "texture-readback-from-store");
if (MG_Util::PixelStoreProcessor::IsRawPackedPixelTransfer(
logicalFormat, MG_Util::ConvertGLEnumToTextureInputFormat(format),
MG_Util::ConvertGLEnumToTexturePixelDataType(type))) {
if (store.LevelByteSize(key, target, mip) != texels * pixelBytes) {
declined = "the staged words are not the client's word size";
} else {
out.assign(staged, staged + texels * pixelBytes);
}
} else {
Vector<Uint8> wide;
Bool isInteger = false;
Bool isSigned = false;
if (!MG_Util::PixelStoreProcessor::DecodeShadowDataToWideRGBA(logicalFormat, staged, texels, wide,
isInteger, isSigned)) {
declined = "the staged format has no wide decoding";
} else if (mapping.isInteger != isInteger) {
declined = "an integer read of a normalized level, or the reverse";
} else {
const GLenum wideType = isInteger ? (isSigned ? GL_INT : GL_UNSIGNED_INT) : GL_FLOAT;
const SizeT rowTexels = static_cast<SizeT>(extent.x());
const SizeT rows = texels / rowTexels;
const SizeT wideRowBytes = rowTexels * 4 * GetReadbackComponentSize(wideType);
out.resize(texels * pixelBytes);
for (SizeT row = 0; row < rows; ++row) {
ReadbackImpl::ConvertWideReadbackRow(wide.data() + row * wideRowBytes,
out.data() + row * rowTexels * pixelBytes, rowTexels,
wideType, mapping, type);
}
}
}
}
if (declined != nullptr) {
MGLOG_E_ONCE("MGPipe: texture {%u,%u} level %d: the driver refused the readback and the staged store "
"cannot answer it (%s) - DECLINED (texture-readback-store-declined)",
res.Slot, res.Gen, level, declined);
return false;
}
MGLOG_W_ONCE("MGPipe: texture {%u,%u} level %d: the driver refused the readback (internal format %u); "
"answered from the server's staged store (texture-readback-from-store)",
res.Slot, res.Gen, level, static_cast<unsigned>(logicalFormat));
bytes = Move(out);
return true;
}
Bool ReadTextureImageWire(const MG_Pipe::MGPReadbackInfo& info, Vector<Uint8>& bytes) {
const auto* record = PipeTextureRecordForHandle(info.Res);
if (!record) return false;
TextureTarget target = TextureTarget::Unknown;
for (Int i = 0; i < static_cast<Int>(TextureTarget::TextureTargetCount); ++i)
if (MG_Pipe::MGPipeResourceTargetForTextureTarget(static_cast<TextureTarget>(i)) == record->Desc.Target)
target = static_cast<TextureTarget>(i);
if (target == TextureTarget::Unknown) return false;
const auto twin = TextureImpl::SyncTextureToBackendByHandle(info.Res);
if (!twin) return false;
const Bool wholeCube = info.Target == GL_TEXTURE_CUBE_MAP;
const IntVec3 extent{static_cast<Int>(info.Box.W), static_cast<Int>(info.Box.H),
wholeCube ? 1 : static_cast<Int>(info.Box.D)};
const auto logicalFormat = static_cast<TextureInternalFormat>(record->Desc.InternalFormat);
// The R32UI word leg never goes through a framebuffer, so the knob leaves it alone (it is
// ReadTextureLevelTight's first choice for exactly these reads).
const Bool packedWordRead =
!twin->RequiresImageBindableStorage() &&
MG_Util::PixelStoreProcessor::IsRawPackedPixelTransfer(
logicalFormat, MG_Util::ConvertGLEnumToTextureInputFormat(info.Format),
MG_Util::ConvertGLEnumToTexturePixelDataType(info.Type)) &&
MG_Util::GetInputBytesPerPixel(MG_Util::ConvertGLEnumToTextureInputFormat(info.Format),
MG_Util::ConvertGLEnumToTexturePixelDataType(info.Type)) == sizeof(Uint32);
const Bool refusedForTesting = !packedWordRead && WireTextureReadbackRefusedForTesting(extent);
Vector<Uint8> result;
for (Int face = 0; face < (wholeCube ? 6 : 1); ++face) {
const auto uploadTarget = wholeCube
? static_cast<TextureUploadTarget>(static_cast<Int>(TextureUploadTarget::CubeMapPositiveX) + face)
: MG_Util::ConvertGLEnumToTextureUploadTarget(info.Target);
Vector<Uint8> slice;
if (refusedForTesting ||
!ReadTextureLevelTight(twin->GetBackendTextureId(), target, uploadTarget, logicalFormat, info.Level,
extent, twin->RequiresImageBindableStorage(), info.Format, info.Type, slice)) {
if (!ReadWireLevelFromStagedStore(info.Res, logicalFormat, uploadTarget, info.Level, extent,
info.Format, info.Type, slice)) return false;
}
result.insert(result.end(), slice.begin(), slice.end());
}
bytes = Move(result);
return true;
}
@@ -528,8 +528,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const std::lock_guard<std::recursive_mutex> lock(m_eglStateMutex);
const Bool ok = BackendObject::MakeEGLCurrent(dpy, draw, read, ctx);
if (ok && UsesSurfaceTargets() && ctx != EGL_NO_CONTEXT && draw != EGL_NO_SURFACE) {
m_contextBindings[ctx] = EGLCurrentState{.Display = dpy, .DrawSurface = draw, .ReadSurface = read,
.Context = ctx};
m_contextBindings[ctx] =
EGLCurrentState{.Display = dpy, .DrawSurface = draw, .ReadSurface = read, .Context = ctx};
}
return ok;
}
@@ -791,26 +791,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<GLExtension> BuildAdvertisedExtensions(Bool shaderSubgroupSupported, Bool timerQueriesSupported,
Bool anisotropicFilteringSupported,
Bool nonZeroIndirectBaseInstanceSupported,
Bool cubeMapArraySupported,
Bool nonZeroIndirectBaseInstanceSupported, Bool cubeMapArraySupported,
const AdvertisedFeatureSupport& features) {
Vector<GLExtension> extensions = {
// The version tokens have to reach the version the backend actually claims:
// TargetGLVersion is {4,6,0}, and a list that stopped at OpenGL40 told an
// application feature-detecting off these tokens the opposite of what
// GL_MAJOR_VERSION / GL_MINOR_VERSION told it.
V_OpenGL30, V_OpenGL31, V_OpenGL32, V_OpenGL33, V_OpenGL40, V_OpenGL41, V_OpenGL42, V_OpenGL43,
V_OpenGL44, V_OpenGL45, V_OpenGL46,
E_GL_ARB_draw_buffers_blend,
V_OpenGL30, V_OpenGL31, V_OpenGL32, V_OpenGL33, V_OpenGL40, V_OpenGL41, V_OpenGL42, V_OpenGL43, V_OpenGL44,
V_OpenGL45, V_OpenGL46, E_GL_ARB_draw_buffers_blend,
// The GLSL 1.00/1.10-era ARB trio plus NPOT textures: all core since GL 2.0 and
// trivially true for every context this backend claims (4.6). KWin 6's
// EglContext::checkSupported() gates desktop-GL compositing on exactly these four
// strings, so withholding them fails its renderer bring-up. Same block as DirectGLES.
E_GL_ARB_shader_objects, E_GL_ARB_vertex_shader, E_GL_ARB_fragment_shader,
E_GL_ARB_texture_non_power_of_two,
E_GL_ARB_compute_shader, E_GL_ARB_shader_storage_buffer_object, E_GL_ARB_shader_image_load_store,
E_GL_ARB_clear_buffer_object, E_GL_ARB_program_interface_query, E_GL_ARB_framebuffer_object, E_GL_ARB_draw_indirect,
E_GL_ARB_multi_draw_indirect,
E_GL_ARB_texture_non_power_of_two, E_GL_ARB_compute_shader, E_GL_ARB_shader_storage_buffer_object,
E_GL_ARB_shader_image_load_store, E_GL_ARB_clear_buffer_object, E_GL_ARB_program_interface_query,
E_GL_ARB_framebuffer_object, E_GL_ARB_draw_indirect, E_GL_ARB_multi_draw_indirect,
E_GL_ARB_indirect_parameters, E_GL_EXT_framebuffer_object, E_GL_ARB_depth_texture, E_GL_ARB_buffer_storage,
E_GL_ARB_texture_storage, E_GL_ARB_texture_storage_multisample, E_GL_ARB_texture_multisample,
E_GL_ARB_clear_texture, E_GL_ARB_direct_state_access, E_GL_ARB_shader_draw_parameters,
@@ -1116,8 +1113,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
pVulkanRenderer && pVulkanRenderer->IsNonZeroIndirectBaseInstanceSupported(),
m_vulkanCaps.SupportsImageCubeArray,
AdvertisedFeatureSupport{.TessellationShader = m_vulkanCaps.SupportsTessellationShader,
.CullDistance = m_vulkanCaps.SupportsShaderCullDistance &&
m_vulkanCaps.MaxCullDistances > 0});
.CullDistance =
m_vulkanCaps.SupportsShaderCullDistance && m_vulkanCaps.MaxCullDistances > 0});
// GL_EXT_fragment_shading_rate rides on VK_KHR_fragment_shading_rate's pipeline rate, and
// its _primitive companion on the primitive rate; both only when the renderer enabled them.
// _attachment is not advertised: no framebuffer carries a rate attachment to this backend.
@@ -1214,9 +1211,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
0.3125f, 0.1875f, 0.1875f, 0.8125f, 0.0625f, 0.4375f,
0.6875f, 0.9375f, 0.9375f, 0.0625f};
static constexpr Float kStandard16[] = {
0.5625f, 0.5625f, 0.4375f, 0.3125f, 0.3125f, 0.625f, 0.75f, 0.4375f, 0.1875f, 0.375f, 0.625f,
0.8125f, 0.8125f, 0.6875f, 0.6875f, 0.1875f, 0.375f, 0.875f, 0.5f, 0.0625f, 0.25f, 0.125f,
0.125f, 0.75f, 0.0f, 0.5f, 0.9375f, 0.25f, 0.875f, 0.9375f, 0.0625f, 0.0f};
0.5625f, 0.5625f, 0.4375f, 0.3125f, 0.3125f, 0.625f, 0.75f, 0.4375f, 0.1875f, 0.375f, 0.625f,
0.8125f, 0.8125f, 0.6875f, 0.6875f, 0.1875f, 0.375f, 0.875f, 0.5f, 0.0625f, 0.25f, 0.125f,
0.125f, 0.75f, 0.0f, 0.5f, 0.9375f, 0.25f, 0.875f, 0.9375f, 0.0625f, 0.0f};
const Float* const tables[] = {kStandard2, kStandard4, kStandard8, kStandard16};
for (Int samples = 2, row = 0; samples <= decltype(m_dynamicParameters)::kMaxSamplePositionCount &&
samples <= m_vulkanCaps.MaxSamples;
@@ -1275,9 +1272,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_dynamicParameters.MaxCombinedShaderStorageBlocks =
clampLimit("GL_MAX_COMBINED_SHADER_STORAGE_BLOCKS", m_vulkanCaps.MaxCombinedShaderStorageBlocks,
kMaxAdvertisedBufferBlocks);
m_dynamicParameters.MaxComputeUniformBlocks =
clampLimit("GL_MAX_COMPUTE_UNIFORM_BLOCKS", m_vulkanCaps.MaxComputeUniformBlocks,
kMaxAdvertisedBufferBlocks);
m_dynamicParameters.MaxComputeUniformBlocks = clampLimit(
"GL_MAX_COMPUTE_UNIFORM_BLOCKS", m_vulkanCaps.MaxComputeUniformBlocks, kMaxAdvertisedBufferBlocks);
m_dynamicParameters.MaxComputeWorkGroupInvocations = m_vulkanCaps.MaxComputeWorkGroupInvocations;
// The six per-axis compute limits, from the same VkPhysicalDeviceLimits fields
// GLFunctionsTable::GetIntegeri_v (DirectVulkan.cpp) reads live. Carried here so that
@@ -1544,8 +1540,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// The attachment rate stays off (see UpdateAdvertisedExtensions), and with it every limit
// that only describes the attachment.
{
const FragmentShadingRateSupport fsr =
pVulkanRenderer != nullptr ? pVulkanRenderer->GetFragmentShadingRateSupport() : FragmentShadingRateSupport{};
const FragmentShadingRateSupport fsr = pVulkanRenderer != nullptr
? pVulkanRenderer->GetFragmentShadingRateSupport()
: FragmentShadingRateSupport{};
m_dynamicParameters.SupportsFragmentShadingRate = fsr.Pipeline;
m_dynamicParameters.SupportsPrimitiveFragmentShadingRate = fsr.Pipeline && fsr.Primitive;
m_dynamicParameters.SupportsAttachmentFragmentShadingRate = false;
@@ -18,8 +18,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// initialization; the physical device must remain valid for the duration of the call.
void PopulateFormatCapabilities(VkPhysicalDevice physicalDevice,
PFN_vkGetPhysicalDeviceFormatProperties getFormatProperties,
const MG_External::VulkanCapabilities& capabilities,
FormatCapabilityCache& cache);
const MG_External::VulkanCapabilities& capabilities, FormatCapabilityCache& cache);
class BackendObject_DirectVulkan : public BackendObject {
public:
@@ -115,8 +114,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// GL_KHR_shader_subgroup is offered at all, natively or emulated.
Vector<GLExtension> BuildAdvertisedExtensions(Bool shaderSubgroupSupported, Bool timerQueriesSupported,
Bool anisotropicFilteringSupported,
Bool nonZeroIndirectBaseInstanceSupported,
Bool cubeMapArraySupported,
Bool nonZeroIndirectBaseInstanceSupported, Bool cubeMapArraySupported,
const AdvertisedFeatureSupport& features = {});
// Format: <GPU Name>, Vulkan <Vulkan Version>, Driver <Driver Version> — the exact
@@ -16,13 +16,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// headers. They link the static library on desktop; the shipping shared library is built with
// hidden visibility, so on Android nothing outside the library can call GetDescriptorPoolCensus.
struct DescriptorPoolCensus {
bool available = false; // false when no Magma renderer is live (e.g. the DirectGLES backend)
bool available = false; // false when no Magma renderer is live (e.g. the DirectGLES backend)
uint32_t frameSlots = 0;
uint32_t pools = 0; // descriptor pools across every frame slot
uint32_t pools = 0; // descriptor pools across every frame slot
uint32_t maxPoolsInOneSlot = 0;
uint64_t capacitySets = 0; // sum of maxSets over those pools
uint64_t cachedSets = 0; // sets allocated from them and held in the per-layout caches
uint64_t slotTrims = 0; // whole-slot trims since the renderer came up
uint64_t capacitySets = 0; // sum of maxSets over those pools
uint64_t cachedSets = 0; // sets allocated from them and held in the per-layout caches
uint64_t slotTrims = 0; // whole-slot trims since the renderer came up
};
DescriptorPoolCensus GetDescriptorPoolCensus();
@@ -61,7 +61,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#if MOBILEGL_BUILD_DISAGGREGATED
namespace {
// P7 wave 2 package C, CONTRACT-P7 §5.5: MAGMA'S MIRROR OF g_glesStateObjectDeathOps
// (DirectGLES/Managers.cpp:335), and the one arm of that table Magma has any work in.
// (in DirectGLES/Managers.cpp), and the one arm of that table Magma has any work in.
//
// WHAT THE NOTICE IS FOR HERE, and it is not what it is for on Espryt. Espryt's arm
// destroys a BACKEND TWIN keyed by lifetime id; Magma keeps no such twin - its two
@@ -145,21 +145,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 baseInstance = 0;
};
// The verb's handles identify server stores; never inspect a client binding.
//
// P8-D: THE GPU READS THE WORDS (VulkanRenderer::DrawWireIndirectNative, WireDraw.inc), as
// P8-D: THE GPU READS THE WORDS (VulkanRenderer::DrawWireIndirectNative, WireDraw.cpp), as
// on the monolith arm. The CPU expansion below is kept for the one shape the monolith arm
// also expands - a COUNT form the device cannot issue natively (no VK_KHR_draw_indirect_count,
// or maxdrawcount > 1 without multiDrawIndirect: VulkanRenderer.cpp's
// MultiDrawElementsIndirectCount) - where ReadWireBuffer orders GPU-produced command/count
// bytes before the expansion.
void DrawWireIndirect(GLenum mode, GLenum type, const void* indirect, GLsizei drawcount,
GLsizei stride, Bool indexed, Bool counted = false, GLintptr countOffset = 0) {
void DrawWireIndirect(GLenum mode, GLenum type, const void* indirect, GLsizei drawcount, GLsizei stride,
Bool indexed, Bool counted = false, GLintptr countOffset = 0) {
if (drawcount <= 0 || countOffset < 0) return;
if (pVulkanRenderer->DrawWireIndirectNative(mode, type, reinterpret_cast<Uint64>(indirect), drawcount,
stride, indexed, counted, static_cast<Uint64>(countOffset)))
stride, indexed, counted, static_cast<Uint64>(countOffset)))
return;
auto& buffers = pVulkanRenderer->GetWireBufferManager();
const auto& state = MG_Pipe::MGPipeApplier();
@@ -176,8 +174,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} expansionCount{buffers};
if (counted) {
Uint32 count = 0;
if (!buffers.ReadWireBuffer(state.VerbIndirectParameterBuffer,
static_cast<Uint64>(countOffset), sizeof(count), &count)) return;
if (!buffers.ReadWireBuffer(state.VerbIndirectParameterBuffer, static_cast<Uint64>(countOffset),
sizeof(count), &count))
return;
drawcount = static_cast<GLsizei>(std::min<Uint32>(count, static_cast<Uint32>(drawcount)));
if (!drawcount) return;
}
@@ -187,8 +186,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Uint64 byteCount = static_cast<Uint64>(stride) * (drawcount - 1) + commandSize;
if (byteCount > std::numeric_limits<SizeT>::max()) return;
Vector<Uint8> bytes(static_cast<SizeT>(byteCount));
if (!buffers.ReadWireBuffer(state.VerbIndirectBuffer, reinterpret_cast<Uint64>(indirect),
byteCount, bytes.data())) return;
if (!buffers.ReadWireBuffer(state.VerbIndirectBuffer, reinterpret_cast<Uint64>(indirect), byteCount,
bytes.data()))
return;
if (indexed) {
const SizeT indexSize = MG_Util::GetGLTypeSize(type);
if (indexSize != 1 && indexSize != 2 && indexSize != 4) return;
@@ -201,12 +201,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (GLsizei i = 0; i < drawcount; ++i) {
DrawElementsIndirectCommand command{};
Memcpy(&command, bytes.data() + static_cast<SizeT>(i) * stride, sizeof(command));
params[i] = {command.count, command.instanceCount, command.firstIndex,
command.baseVertex, std::bit_cast<Int32>(command.baseInstance)};
params[i] = {command.count, command.instanceCount, command.firstIndex, command.baseVertex,
std::bit_cast<Int32>(command.baseInstance)};
const Uint64 end = (static_cast<Uint64>(command.firstIndex) + command.count) * indexSize;
if (end > std::numeric_limits<SizeT>::max()) return;
payload.indexBufferView.indexByteSize = std::max<SizeT>(payload.indexBufferView.indexByteSize,
static_cast<SizeT>(end));
payload.indexBufferView.indexByteSize =
std::max<SizeT>(payload.indexBufferView.indexByteSize, static_cast<SizeT>(end));
}
pVulkanRenderer->MultiDrawElements(payload);
} else {
@@ -257,7 +257,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
GLuint GetShaderStorageBlockIndex(const MG_State::GLState::ProgramObject& program, const String& name) {
const auto& blocks = program.GetLinkReflection().storageBlocks;
const auto find = [&blocks](const String& key) {
return std::find_if(blocks.begin(), blocks.end(),
return std::find_if(
blocks.begin(), blocks.end(),
[&](const MG_State::GLState::StorageBlockReflection& block) { return block.name == key; });
};
auto it = find(name);
@@ -326,8 +327,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
DrawWireIndirect(mode, type, indirect, maxdrawcount, stride, true, true, drawcount);
return;
}
void MultiDrawArraysIndirectCount(GLenum mode, const void* indirect, GLintptr drawcount,
GLsizei maxdrawcount, GLsizei stride) {
void MultiDrawArraysIndirectCount(GLenum mode, const void* indirect, GLintptr drawcount, GLsizei maxdrawcount,
GLsizei stride) {
DrawWireIndirect(mode, 0, indirect, maxdrawcount, stride, false, true, drawcount);
return;
}
@@ -344,8 +345,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
void DrawElementsInstancedBaseVertexBaseInstance(GLenum mode, GLsizei count, GLenum type, const void* indices,
GLsizei instancecount, GLint basevertex, GLuint baseinstance) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::DrawElementsInstancedBaseVertexBaseInstance called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::DrawElementsInstancedBaseVertexBaseInstance called with null GL context");
MOBILEGL_ASSERT(pVulkanRenderer,
"DirectVulkan::DrawElementsInstancedBaseVertexBaseInstance called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(),
"DirectVulkan::DrawElementsInstancedBaseVertexBaseInstance called with null GL context");
DrawIndexedCmd payload{};
payload.mode = mode;
@@ -376,8 +379,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
void DrawArraysInstancedBaseInstance(GLenum mode, GLint first, GLsizei count, GLsizei instancecount,
GLuint baseinstance) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::DrawArraysInstancedBaseInstance called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::DrawArraysInstancedBaseInstance called with null GL context");
MOBILEGL_ASSERT(pVulkanRenderer,
"DirectVulkan::DrawArraysInstancedBaseInstance called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(),
"DirectVulkan::DrawArraysInstancedBaseInstance called with null GL context");
DrawCmd payload{};
payload.mode = mode;
@@ -406,16 +411,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::CopyTexSubImage2D called with null GL context");
pVulkanRenderer->CopyTexSubImage2D(target, level, xoffset, yoffset, x, y, width, height);
}
void CopyImageSubData(const CopyImageEndpoint& src,
GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY, GLint srcZ,
const CopyImageEndpoint& dst,
GLenum dstTarget, GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ,
GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth) {
void CopyImageSubData(const CopyImageEndpoint& src, GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY,
GLint srcZ, const CopyImageEndpoint& dst, GLenum dstTarget, GLint dstLevel, GLint dstX,
GLint dstY, GLint dstZ, GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::CopyImageSubData called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::CopyImageSubData called with null GL context");
pVulkanRenderer->CopyImageSubData(src, srcTarget, srcLevel, srcX, srcY, srcZ,
dst, dstTarget, dstLevel, dstX, dstY, dstZ,
srcWidth, srcHeight, srcDepth);
pVulkanRenderer->CopyImageSubData(src, srcTarget, srcLevel, srcX, srcY, srcZ, dst, dstTarget, dstLevel, dstX,
dstY, dstZ, srcWidth, srcHeight, srcDepth);
}
void GenerateMipmap(GLenum target) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::GenerateMipmap called with null VulkanRenderer");
@@ -432,7 +434,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void DispatchComputeIndirect(GLintptr indirect) {
// P8-SV: THE GPU READS THE GROUP COUNTS on the wire arm too - VulkanRenderer's
// DispatchComputeIndirect takes its transport branch into DispatchWireComputeIndirect
// (WireDraw.inc), which issues vkCmdDispatchIndirect from the verb's store. The CPU read
// (WireDraw.cpp), which issues vkCmdDispatchIndirect from the verb's store. The CPU read
// that was here (ReadWireBuffer, a whole-GPU wait once a shader had written the store) is gone.
pVulkanRenderer->DispatchComputeIndirect(indirect);
return;
@@ -503,10 +505,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
auto& record = state.ShaderCsos[handle.Slot];
auto found = std::find_if(record.StorageOverrides.begin(), record.StorageOverrides.end(),
[&](const auto& entry) { return entry.Name == storageBlockName; });
[&](const auto& entry) { return entry.Name == storageBlockName; });
if (found == record.StorageOverrides.end())
record.StorageOverrides.push_back({storageBlockName, static_cast<Int32>(storageBlockBinding)});
else found->Binding = static_cast<Int32>(storageBlockBinding);
else
found->Binding = static_cast<Int32>(storageBlockBinding);
++record.BindingsSerial;
return;
// P7 wave 2 package C, OQ-8: NOTHING LEFT TO PATCH, and the hazard goes with it.
@@ -566,8 +569,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!MG_Pipe::MGPipeHandleIsNull(bound.Res)) {
owned.resize(static_cast<SizeT>(count) * width);
const Uint64 offset = bound.Offset + reinterpret_cast<Uint64>(indices);
if (offset < bound.Offset || !pVulkanRenderer->GetWireBufferManager().ReadWireBuffer(
bound.Res, offset, owned.size(), owned.data())) return false;
if (offset < bound.Offset ||
!pVulkanRenderer->GetWireBufferManager().ReadWireBuffer(bound.Res, offset, owned.size(), owned.data()))
return false;
bytes = owned.data();
}
if (!bytes) return false;
@@ -689,8 +693,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (count[i] <= 0) {
continue;
}
DrawElementsBaseVertex(mode, count[i], type, indices[i],
basevertex != nullptr ? basevertex[i] : 0);
DrawElementsBaseVertex(mode, count[i], type, indices[i], basevertex != nullptr ? basevertex[i] : 0);
}
return;
}
@@ -724,9 +727,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// TODO: this index view needs a redesign, now there's a lotta redundant uploads
payload.indexBufferView.indexByteOffset = 0;
payload.indexBufferView.indexByteSize =
std::max(reinterpret_cast<SizeT>(indices[i]) + count[i] * indexSize,
payload.indexBufferView.indexByteSize);
payload.indexBufferView.indexByteSize = std::max(reinterpret_cast<SizeT>(indices[i]) + count[i] * indexSize,
payload.indexBufferView.indexByteSize);
auto& param = params[i];
@@ -743,7 +745,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void DrawElementsBaseVertex(GLenum mode, GLsizei count, GLenum type, const GLvoid* indices, GLint basevertex) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::DrawElementsBaseVertex called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::DrawElementsBaseVertex called with null GL context");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(),
"DirectVulkan::DrawElementsBaseVertex called with null GL context");
if (mode == GL_LINE_LOOP) {
Vector<Uint32> closedIndices;
if (BuildClosedLineLoopIndices(count, type, indices, closedIndices)) {
@@ -767,7 +770,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void MultiDrawElementsBaseVertex(GLenum mode, const GLsizei* count, GLenum type, const GLvoid* const* indices,
GLsizei drawcount, const GLint* basevertex) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::MultiDrawElementsBaseVertex called with null VulkanRenderer");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(), "DirectVulkan::MultiDrawElementsBaseVertex called with null GL context");
MOBILEGL_ASSERT(MG_Pipe::gPipeInputs->IsLive(),
"DirectVulkan::MultiDrawElementsBaseVertex called with null GL context");
MultiDrawElementsImpl(mode, count, type, indices, drawcount, basevertex);
}
@@ -821,8 +825,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
pVulkanRenderer->FlushForSyncPoint(sync->submitIndex);
}
if (timeout == 0) {
return pVulkanRenderer->IsSubmitIndexComplete(sync->submitIndex) ? GL_ALREADY_SIGNALED
: GL_TIMEOUT_EXPIRED;
return pVulkanRenderer->IsSubmitIndexComplete(sync->submitIndex) ? GL_ALREADY_SIGNALED : GL_TIMEOUT_EXPIRED;
}
// Blocking wait: flush even without the flush bit - the sync's batch
// can only be submitted from this thread, so waiting on an unflushed
@@ -876,7 +879,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// records are shared (SharedPtr) with the owning pool's pending list,
// so deleting the query while results are still in flight is safe.
struct VulkanTimerQuery {
enum class Kind : Uint8 { Timer, Occlusion, XfbWritten, XfbGenerated };
enum class Kind : Uint8 {
Timer,
Occlusion,
XfbWritten,
XfbGenerated
};
Kind kind = Kind::Timer;
SharedPtr<VkTimerQueryManager::TimestampRecord> begin;
SharedPtr<VkTimerQueryManager::TimestampRecord> end;
@@ -1005,22 +1013,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
*outNanoseconds = samples;
return true;
}
if (query->kind == VulkanTimerQuery::Kind::XfbWritten ||
query->kind == VulkanTimerQuery::Kind::XfbGenerated) {
if (query->kind == VulkanTimerQuery::Kind::XfbWritten || query->kind == VulkanTimerQuery::Kind::XfbGenerated) {
Uint64 primitives = 0;
if (!pVulkanRenderer->ResolveXfbQueryResult(query->occlusionSlots, query->rerouteSlots,
query->kind == VulkanTimerQuery::Kind::XfbGenerated,
primitives)) {
return false;
}
if (query->kind == VulkanTimerQuery::Kind::XfbGenerated &&
!query->pausedPrimitivesCountedByGpu &&
if (query->kind == VulkanTimerQuery::Kind::XfbGenerated && !query->pausedPrimitivesCountedByGpu &&
#if MOBILEGL_BUILD_DISAGGREGATED
MG_Config::Transport == MG_Config::TransportMode::Monolith &&
#endif
MG_Pipe::gPipeInputs->IsLive()) {
primitives += MG_Pipe::gPipeInputs->GetTransformFeedbackPausedPrimitiveCounter() -
query->pausedPrimitiveSnapshot;
primitives +=
MG_Pipe::gPipeInputs->GetTransformFeedbackPausedPrimitiveCounter() - query->pausedPrimitiveSnapshot;
}
*outNanoseconds = primitives;
return true;
@@ -1069,7 +1075,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#if MOBILEGL_BUILD_DISAGGREGATED
MG_Config::Transport == MG_Config::TransportMode::Monolith &&
#endif
MG_Pipe::gPipeInputs->IsLive() ? MG_Pipe::gPipeInputs->GetTransformFeedbackPausedPrimitiveCounter() : 0;
MG_Pipe::gPipeInputs->IsLive()
? MG_Pipe::gPipeInputs->GetTransformFeedbackPausedPrimitiveCounter()
: 0;
// Read AFTER StartXfbQueryCapture, which is where a failed reroute-pool creation
// disarms: the answer is then what this span will actually do for every draw.
query->pausedPrimitivesCountedByGpu = generated && pVulkanRenderer->ArePausedDrawsGpuCounted();
@@ -1082,9 +1090,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (query == nullptr || query->rendererGeneration != GetRendererGeneration()) {
return;
}
pVulkanRenderer->StopXfbQueryCapture(
query->kind == VulkanTimerQuery::Kind::XfbGenerated ? 1u : 0u, query->occlusionSlots,
query->rerouteSlots);
pVulkanRenderer->StopXfbQueryCapture(query->kind == VulkanTimerQuery::Kind::XfbGenerated ? 1u : 0u,
query->occlusionSlots, query->rerouteSlots);
}
BackendQueryHandle BeginOcclusionQuery() {
@@ -76,8 +76,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void MultiDrawArraysIndirect(GLenum mode, const void* indirect, GLsizei drawcount, GLsizei stride);
void MultiDrawElementsIndirectCount(GLenum mode, GLenum type, const void* indirect, GLintptr drawcount,
GLsizei maxdrawcount, GLsizei stride);
void MultiDrawArraysIndirectCount(GLenum mode, const void* indirect, GLintptr drawcount,
GLsizei maxdrawcount, GLsizei stride);
void MultiDrawArraysIndirectCount(GLenum mode, const void* indirect, GLintptr drawcount, GLsizei maxdrawcount,
GLsizei stride);
void DrawRangeElementsBaseVertex(GLenum mode, GLuint start, GLuint end, GLsizei count, GLenum type,
const void* indices, GLint basevertex);
void DrawRangeElements(GLenum mode, GLuint start, GLuint end, GLsizei count, GLenum type, const void* indices);
@@ -99,11 +99,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
GLsizei height, GLint border);
void CopyTexSubImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y, GLsizei width,
GLsizei height);
void CopyImageSubData(const CopyImageEndpoint& src,
GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY, GLint srcZ,
const CopyImageEndpoint& dst,
GLenum dstTarget, GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ,
GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void CopyImageSubData(const CopyImageEndpoint& src, GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY,
GLint srcZ, const CopyImageEndpoint& dst, GLenum dstTarget, GLint dstLevel, GLint dstX,
GLint dstY, GLint dstZ, GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void GenerateMipmap(GLenum target);
void DispatchCompute(GLuint numGroupsX, GLuint numGroupsY, GLuint numGroupsZ);
void DispatchComputeIndirect(GLintptr indirect);
@@ -12,9 +12,9 @@
namespace MobileGL::MG_State::GLState {
class ProgramObject;
}
} // namespace MobileGL::MG_State::GLState
namespace MobileGL::MG_Backend::DirectVulkan {
GLuint GetShaderStorageBlockIndex(const MG_State::GLState::ProgramObject& program, const String& name);
GLuint GetShaderStorageBlockBinding(const MG_State::GLState::ProgramObject& program, GLuint blockIndex);
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -40,7 +40,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return static_cast<Uint32>(registry.size() - 1);
}
MagmaSession& Detail::ProcessMagmaSession() { return ProcessSessionInstance(); }
MagmaSession& Detail::ProcessMagmaSession() {
return ProcessSessionInstance();
}
void* Detail::SessionLocalSlotSlow(MagmaSession& session, Uint32 index) {
if (index >= session.m_slots.size()) session.m_slots.resize(Registry().size(), nullptr);
@@ -54,7 +56,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the app renders on, and a lazily built slot would be a first-touch race between two.
const auto& registry = Registry();
m_slots.resize(registry.size(), nullptr);
for (SizeT i = 0; i < registry.size(); ++i) m_slots[i] = registry[i].construct();
for (SizeT i = 0; i < registry.size(); ++i)
m_slots[i] = registry[i].construct();
}
MagmaSession::~MagmaSession() {
@@ -73,10 +76,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Detail::t_boundMagmaSession = session;
}
MagmaSessionScope::~MagmaSessionScope() { Detail::t_boundMagmaSession = m_previous; }
MagmaSessionScope::~MagmaSessionScope() {
Detail::t_boundMagmaSession = m_previous;
}
void BindMagmaSessionToThisThread(MagmaSession* session) { Detail::t_boundMagmaSession = session; }
void BindMagmaSessionToThisThread(MagmaSession* session) {
Detail::t_boundMagmaSession = session;
}
MagmaSession* BoundMagmaSession() { return Detail::t_boundMagmaSession; }
MagmaSession* BoundMagmaSession() {
return Detail::t_boundMagmaSession;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -104,9 +104,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
SessionLocal(const SessionLocal&) = delete;
SessionLocal& operator=(const SessionLocal&) = delete;
T& Get() const {
return *static_cast<T*>(Detail::SessionLocalSlot(Detail::CurrentMagmaSession(), m_index));
}
T& Get() const { return *static_cast<T*>(Detail::SessionLocalSlot(Detail::CurrentMagmaSession(), m_index)); }
T& operator*() const { return Get(); }
T* operator->() const { return &Get(); }
@@ -35,7 +35,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void CollectDeferredReleases(Uint32 frameIndex);
Bool Allocate(Uint32 frameIndex, VkDeviceSize size, VkDeviceSize alignment, BufferSlice& outSlice);
Bool Upload(Uint32 frameIndex, const void* data, VkDeviceSize size, VkDeviceSize alignment, BufferSlice& outSlice);
Bool Upload(Uint32 frameIndex, const void* data, VkDeviceSize size, VkDeviceSize alignment,
BufferSlice& outSlice);
VkDeviceSize GetWriteCursor(Uint32 frameIndex) const;
Uint32 GetFrameCount() const;
@@ -101,7 +101,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer& FrameContext::BeginCommandRecording(VkCommandBufferUsageFlags flags,
const VkCommandBufferInheritanceInfo* pInheritanceInfo) {
auto& frame = GetCurrent();
MOBILEGL_ASSERT(!frame.isCommandRecording, "BeginCommandRecording called while command buffer is already recording");
MOBILEGL_ASSERT(!frame.isCommandRecording,
"BeginCommandRecording called while command buffer is already recording");
frame.hasCommandBufferRecorded = false;
VK_VERIFY(vkResetCommandBuffer(frame.commandBuffer, 0), "BeginCommandRecording, vkResetCommandBuffer");
@@ -174,8 +175,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_swapchainImageRenderFinishedSemaphores.assign(swapchainImageCount, VK_NULL_HANDLE);
VkSemaphoreCreateInfo semaphoreInfo{VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO};
for (Uint32 imageIndex = 0; imageIndex < swapchainImageCount; ++imageIndex) {
VkResult result =
vkCreateSemaphore(device, &semaphoreInfo, nullptr, &m_swapchainImageRenderFinishedSemaphores[imageIndex]);
VkResult result = vkCreateSemaphore(device, &semaphoreInfo, nullptr,
&m_swapchainImageRenderFinishedSemaphores[imageIndex]);
if (result != VK_SUCCESS) {
DestroySwapchainSemaphores(device);
return result;
@@ -240,10 +241,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
presentBarrier.subresourceRange.levelCount = 1;
presentBarrier.subresourceRange.baseArrayLayer = 0;
presentBarrier.subresourceRange.layerCount = 1;
vkCmdPipelineBarrier(commandBuffer,
VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, 0,
nullptr, 0, nullptr, 1, &presentBarrier);
vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0,
0, nullptr, 0, nullptr, 1, &presentBarrier);
if (openedRecording) {
EndCommandRecording();
@@ -254,7 +253,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
FrameContext::SubmitInfoPacket FrameContext::GetSubmitInfo(Bool shouldSubmitCommandBuffer,
Uint32 swapchainImageIndex) const {
const auto& frame = GetCurrent();
MOBILEGL_ASSERT(!frame.isCommandRecording, "GetSubmitInfo called while command buffer recording is still active");
MOBILEGL_ASSERT(!frame.isCommandRecording,
"GetSubmitInfo called while command buffer recording is still active");
MOBILEGL_ASSERT(!frame.isPreCommandRecording,
"GetSubmitInfo called while the pre-pass stream is still recording");
AssertValidSwapchainImageIndex(swapchainImageIndex);
@@ -273,7 +273,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
packet.submitInfo.waitSemaphoreCount = frame.imageAvailableSemaphoreConsumed ? 0U : 1U;
packet.submitInfo.pWaitSemaphores = frame.imageAvailableSemaphoreConsumed ? nullptr : &packet.waitSemaphore;
packet.submitInfo.pWaitDstStageMask = frame.imageAvailableSemaphoreConsumed ? nullptr : &packet.waitDstStageMask;
packet.submitInfo.pWaitDstStageMask =
frame.imageAvailableSemaphoreConsumed ? nullptr : &packet.waitDstStageMask;
packet.submitInfo.commandBufferCount = commandBufferCount;
packet.submitInfo.pCommandBuffers = commandBufferCount > 0 ? packet.commandBuffers : nullptr;
packet.submitInfo.signalSemaphoreCount = 1;
@@ -464,8 +465,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
DestroySyncObjectsForFrame(device, frameIndex);
auto& frame = m_frames[frameIndex];
VkResult result =
vkCreateSemaphore(device, &semaphoreInfo, nullptr, &frame.imageAvailableSemaphore);
VkResult result = vkCreateSemaphore(device, &semaphoreInfo, nullptr, &frame.imageAvailableSemaphore);
if (result != VK_SUCCESS) {
return result;
}
@@ -120,7 +120,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkResult AcquireNextImageInCurrentSlot(VkDevice device, VkSwapchainKHR swapchain, Uint32& outImageIndex);
// The per-swapchain-image render-finished semaphores belong to one swapchain; a renderer
// that parks a surface parks them with it.
void ExchangeSwapchainSemaphores(Vector<VkSemaphore>& other) { std::swap(m_swapchainImageRenderFinishedSemaphores, other); }
void ExchangeSwapchainSemaphores(Vector<VkSemaphore>& other) {
std::swap(m_swapchainImageRenderFinishedSemaphores, other);
}
VkResult WaitAndAcquireNextImage(VkDevice device, VkSwapchainKHR swapchain, Uint32& outImageIndex,
Uint64 timeout = UINT64_MAX, VkFence acquireFence = VK_NULL_HANDLE);
@@ -189,7 +189,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer buffers[kMaxOwn + 2];
Uint32 count = 0;
buffers[count++] = bracket.Begin;
for (Uint32 i = 0; i < info.commandBufferCount; ++i) buffers[count++] = info.pCommandBuffers[i];
for (Uint32 i = 0; i < info.commandBufferCount; ++i)
buffers[count++] = info.pCommandBuffers[i];
buffers[count++] = bracket.End;
const VkCommandBuffer* own = info.pCommandBuffers;
const Uint32 ownCount = info.commandBufferCount;
@@ -80,9 +80,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer Begin = VK_NULL_HANDLE;
VkCommandBuffer End = VK_NULL_HANDLE;
Uint64 Id = 0;
Bool InFlight = false; // submitted and not yet seen finished
Bool Claimed = false; // handed out by Acquire, not yet submitted or abandoned
Bool Dirty = false; // its events were set by a past submission: reset before reuse
Bool InFlight = false; // submitted and not yet seen finished
Bool Claimed = false; // handed out by Acquire, not yet submitted or abandoned
Bool Dirty = false; // its events were set by a past submission: reset before reuse
Uint64 SubmitIndex = 0; // the renderer submit index of the submission it last bracketed
};
static constexpr Uint32 kMaxSlots = 16;
@@ -81,26 +81,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// longer does it SILENTLY: the combination is named once, at startup, right here.
inline void MagmaPipeValidateSubsystemConfiguration() {
if (!MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemRenderState)) {
MGLOG_W("MGPipe: MOBILEGL_PIPE_LEGACY_MEMOS=0 with kMGPipeSubsystemRenderState (bit 0 "
"of MOBILEGL_PIPE_PUSH) clear - Magma's pipeline memo has no CSO handle to key "
"on, so every draw whose pipeline-state version moved runs the pre-handle STATE "
"HASH instead. That is not a Track-H subsystem and not fatal, but it is not the "
"handle arm either: set bit 0 (MOBILEGL_PIPE_PUSH=0x%llx) if this run was meant "
"to measure it.",
static_cast<unsigned long long>(MG_Config::Features.PipePush |
MG_Pipe::kMGPipeSubsystemRenderState));
MGLOG_W(
"MGPipe: MOBILEGL_PIPE_LEGACY_MEMOS=0 with kMGPipeSubsystemRenderState (bit 0 "
"of MOBILEGL_PIPE_PUSH) clear - Magma's pipeline memo has no CSO handle to key "
"on, so every draw whose pipeline-state version moved runs the pre-handle STATE "
"HASH instead. That is not a Track-H subsystem and not fatal, but it is not the "
"handle arm either: set bit 0 (MOBILEGL_PIPE_PUSH=0x%llx) if this run was meant "
"to measure it.",
static_cast<unsigned long long>(MG_Config::Features.PipePush | MG_Pipe::kMGPipeSubsystemRenderState));
}
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) return;
const char* const why =
"this build has cmake -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF, which compiles no such arm";
MGLOG_F("MGPipe: Fatal{PipeLegacyMemosDisabled} Magma's Track-H subsystem "
"(kMGPipeSubsystemMagmaVertexInput, bit 6 of MOBILEGL_PIPE_PUSH) is clear, so the "
"vertex-input cache and the VAO draw memo want the pre-handle arm - but %s. Set "
"bit 6 (MOBILEGL_PIPE_PUSH=0x%llx, or the default 0x%llx), or allow the legacy arm.",
why,
static_cast<unsigned long long>(MG_Config::Features.PipePush |
MG_Pipe::kMGPipeSubsystemMagmaVertexInput),
static_cast<unsigned long long>(MG_Pipe::kMGPipeSubsystemsMigratedAtP2));
const char* const why = "this build has cmake -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF, which compiles no such arm";
MGLOG_F(
"MGPipe: Fatal{PipeLegacyMemosDisabled} Magma's Track-H subsystem "
"(kMGPipeSubsystemMagmaVertexInput, bit 6 of MOBILEGL_PIPE_PUSH) is clear, so the "
"vertex-input cache and the VAO draw memo want the pre-handle arm - but %s. Set "
"bit 6 (MOBILEGL_PIPE_PUSH=0x%llx, or the default 0x%llx), or allow the legacy arm.",
why,
static_cast<unsigned long long>(MG_Config::Features.PipePush | MG_Pipe::kMGPipeSubsystemMagmaVertexInput),
static_cast<unsigned long long>(MG_Pipe::kMGPipeSubsystemsMigratedAtP2));
std::abort();
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -34,21 +34,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MG_Pipe::MGPipeHandle Handle() const { return m_handle; }
Uint GetExternalIndex() const { return m_handle.Slot; }
Uint64 GetLifetimeId() const {
return (Uint64(m_handle.Gen) << 32) | m_handle.Slot;
}
Uint64 GetLifetimeId() const { return (Uint64(m_handle.Gen) << 32) | m_handle.Slot; }
Bool GetLinkStatus() const { return m_record->Desc.LinkStatus != 0; }
Bool GetSpirvStatus() const { return m_record->Desc.SpirvStatus != 0; }
Bool PointSizeDemoted() const { return m_record->Desc.PointSizeDemoted != 0; }
Bool GetSpirvValidationEnabled() const {
return m_record->Desc.EnableSpirvValidation != 0;
}
const Vector<Vector<Uint>>& GetGeneratedSpirv() const {
return m_record->Archive->Spirv.generatedSpirv;
}
Bool GetSpirvValidationEnabled() const { return m_record->Desc.EnableSpirvValidation != 0; }
const Vector<Vector<Uint>>& GetGeneratedSpirv() const { return m_record->Archive->Spirv.generatedSpirv; }
Vector<ShaderStage> GetLinkedShaderStages() const {
Vector<ShaderStage> result;
for (const auto stage : m_record->Archive->LinkedStages) result.push_back(static_cast<ShaderStage>(stage));
for (const auto stage : m_record->Archive->LinkedStages)
result.push_back(static_cast<ShaderStage>(stage));
return result;
}
Bool HasLinkedShaderStage(ShaderStage stage) const {
@@ -85,12 +80,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const void* GetUBOData() const { return m_record->GlobalConstants.data(); }
Uint GetUBOSize() const { return static_cast<Uint>(m_record->GlobalConstants.size()); }
Uint32 GetUBOContentVersion() const { return m_record->GlobalConstantsVersion; }
Int GetActiveUniformBlocksCount() const {
return static_cast<Int>(Link().glBlockIndexToTProgram.size());
}
Int GetActiveUniformBlocksCount() const { return static_cast<Int>(Link().glBlockIndexToTProgram.size()); }
Uint GetUniformBlockIndex(const char* name) const {
auto found = Link().uniformBlockIndexByName.find(name);
if (found == Link().uniformBlockIndexByName.end()) found = Link().uniformBlockIndexByName.find(String(name) + "[0]");
if (found == Link().uniformBlockIndexByName.end())
found = Link().uniformBlockIndexByName.find(String(name) + "[0]");
return found == Link().uniformBlockIndexByName.end() ? GL_INVALID_INDEX : found->second;
}
Uint GetUniformBlockBinding(Uint index) const {
@@ -105,20 +99,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto* block = Block(index);
return block ? (static_cast<Uint>(block->size) + 15u) & ~15u : 0;
}
Bool IsValidUniformLocation(Int location) const {
return Program::IsValidUniformLocation(Link(), location);
}
Bool IsValidUniformLocation(Int location) const { return Program::IsValidUniformLocation(Link(), location); }
Bool UniformLocationsAliasSameUniform(Int a, Int b) const {
return IsValidUniformLocation(a) && IsValidUniformLocation(b) &&
Link().uniformIndexInTProgram[a] == Link().uniformIndexInTProgram[b];
Link().uniformIndexInTProgram[a] == Link().uniformIndexInTProgram[b];
}
GLenum GetUniformType(Uint location) const {
return IsValidUniformLocation(static_cast<Int>(location))
? Program::UniformAtIn(Link(), Link().uniformIndexInTProgram[location]).glDefineType : 0;
? Program::UniformAtIn(Link(), Link().uniformIndexInTProgram[location]).glDefineType
: 0;
}
Int GetUniformSamplerOrImageUnitIndex(Uint location) const {
const auto& units = m_record->SamplerUnits;
const auto found = std::lower_bound(units.begin(), units.end(), location,
const auto found = std::lower_bound(
units.begin(), units.end(), location,
[](const MG_Pipe::MGPProgramSamplerUnit& unit, Uint value) { return unit.Location < value; });
return found != units.end() && found->Location == location ? found->Unit : -1;
}
@@ -145,27 +139,24 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!IsValidUniformLocation(base)) return -1;
const Int tIndex = Link().uniformIndexInTProgram[base];
if (!Program::UniformAtIn(Link(), tIndex).type.isArray ||
static_cast<GLint>(element) >= Program::GetUniformArraySizeByTIndex(Link(), tIndex)) return -1;
static_cast<GLint>(element) >= Program::GetUniformArraySizeByTIndex(Link(), tIndex))
return -1;
const Int location = base + static_cast<Int>(element);
return UniformLocationsAliasSameUniform(base, location) ? location : -1;
}
SizeT GetTransformFeedbackVaryingCount() const { return Link().xfbVaryings.size(); }
const Vector<MG_State::GLState::XfbVarying>& GetTransformFeedbackVaryings() const {
return Link().xfbVaryings;
}
const Vector<MG_State::GLState::XfbVarying>& GetTransformFeedbackVaryings() const { return Link().xfbVaryings; }
SizeT GetTransformFeedbackBufferCount() const { return Link().xfbStrides.size(); }
Uint32 GetTransformFeedbackStride(Uint index) const {
return index < Link().xfbStrides.size() ? Link().xfbStrides[index] : 0;
}
GLenum GetTransformFeedbackBufferMode() const {
return Link().xfbBufferMode;
}
GLenum GetTransformFeedbackBufferMode() const { return Link().xfbBufferMode; }
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): READ OUT OF THE ARCHIVE.
//
// What stood here re-ran spvReflectCreateShaderModule over EVERY stage module of the
// program, and it did so PER DRAW: a MagmaProgramSource is a borrowed view that
// WireDraw.inc's WireProgramSource constructs afresh for each wire draw, so the lazy
// WireDraw.cpp's WireProgramSource constructs afresh for each wire draw, so the lazy
// `m_storageBlocksReady` latch could never outlive one. The list it rebuilt is a pure
// function of the linked program, so it is now built once at link time and travels in
// the archive (LinkArtifacts::storageBlocks), which takes the SPIR-V reflector off the
@@ -177,18 +168,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// ProgramFactory::ReflectLayout looks up. A caller that still holds a GL-style
// "Foo[2]" gets the same second chance it always had.
const auto find = [&blocks](const String& key) {
return std::find_if(blocks.begin(), blocks.end(),
[&](const MG_State::GLState::StorageBlockReflection& block) {
return block.name == key;
});
return std::find_if(
blocks.begin(), blocks.end(),
[&](const MG_State::GLState::StorageBlockReflection& block) { return block.name == key; });
};
auto found = find(name);
if (found == blocks.end() && !name.empty() && name.back() == ']') {
const auto bracket = name.rfind('[');
if (bracket != String::npos) found = find(name.substr(0, bracket));
}
return found == blocks.end() ? GL_INVALID_INDEX
: static_cast<Uint>(found - blocks.begin());
return found == blocks.end() ? GL_INVALID_INDEX : static_cast<Uint>(found - blocks.begin());
}
Uint GetShaderStorageBlockBinding(Uint index) const {
const auto& blocks = Link().storageBlocks;
@@ -210,7 +199,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (index >= Link().glBlockIndexToTProgram.size()) return nullptr;
const Int tIndex = Link().glBlockIndexToTProgram[index];
return tIndex >= 0 && static_cast<SizeT>(tIndex) < Link().blockReflection.size()
? &Link().blockReflection[tIndex] : nullptr;
? &Link().blockReflection[tIndex]
: nullptr;
}
MG_Pipe::MGPipeHandle m_handle = MG_Pipe::kMGPipeNullHandle;
const MG_Pipe::MGPipeShaderCsoRecord* m_record = nullptr;
@@ -219,4 +209,4 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// also why they never helped: a view lives for one draw, so the latch was cold every
// time it was read.
};
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -13,17 +13,17 @@
namespace MobileGL::MG_Backend::DirectVulkan {
static const char* PrimitiveTopologyToString(VkPrimitiveTopology topology) {
switch (topology) {
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_POINT_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_STRIP)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_FAN)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_LIST_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_STRIP_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_PATCH_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_POINT_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_STRIP)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_FAN)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_LIST_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_LINE_STRIP_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP_WITH_ADJACENCY)
ENUM_STR_CASE(VK_PRIMITIVE_TOPOLOGY_PATCH_LIST)
default:
return "VK_PRIMITIVE_TOPOLOGY_UNKNOWN";
}
@@ -31,13 +31,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static const char* SampleCountToString(VkSampleCountFlagBits sampleCount) {
switch (sampleCount) {
ENUM_STR_CASE(VK_SAMPLE_COUNT_1_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_2_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_4_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_8_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_16_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_32_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_64_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_1_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_2_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_4_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_8_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_16_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_32_BIT)
ENUM_STR_CASE(VK_SAMPLE_COUNT_64_BIT)
default:
return "VK_SAMPLE_COUNT_UNKNOWN";
}
@@ -60,14 +60,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static const char* CompareOpToString(VkCompareOp compareOp) {
switch (compareOp) {
ENUM_STR_CASE(VK_COMPARE_OP_NEVER)
ENUM_STR_CASE(VK_COMPARE_OP_LESS)
ENUM_STR_CASE(VK_COMPARE_OP_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_LESS_OR_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_GREATER)
ENUM_STR_CASE(VK_COMPARE_OP_NOT_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_GREATER_OR_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_ALWAYS)
ENUM_STR_CASE(VK_COMPARE_OP_NEVER)
ENUM_STR_CASE(VK_COMPARE_OP_LESS)
ENUM_STR_CASE(VK_COMPARE_OP_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_LESS_OR_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_GREATER)
ENUM_STR_CASE(VK_COMPARE_OP_NOT_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_GREATER_OR_EQUAL)
ENUM_STR_CASE(VK_COMPARE_OP_ALWAYS)
default:
return "VK_COMPARE_OP_UNKNOWN";
}
@@ -75,29 +75,29 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static const char* LogicOpToString(VkLogicOp logicOp) {
switch (logicOp) {
ENUM_STR_CASE(VK_LOGIC_OP_CLEAR)
ENUM_STR_CASE(VK_LOGIC_OP_AND)
ENUM_STR_CASE(VK_LOGIC_OP_AND_REVERSE)
ENUM_STR_CASE(VK_LOGIC_OP_COPY)
ENUM_STR_CASE(VK_LOGIC_OP_AND_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_NO_OP)
ENUM_STR_CASE(VK_LOGIC_OP_XOR)
ENUM_STR_CASE(VK_LOGIC_OP_OR)
ENUM_STR_CASE(VK_LOGIC_OP_NOR)
ENUM_STR_CASE(VK_LOGIC_OP_EQUIVALENT)
ENUM_STR_CASE(VK_LOGIC_OP_INVERT)
ENUM_STR_CASE(VK_LOGIC_OP_OR_REVERSE)
ENUM_STR_CASE(VK_LOGIC_OP_COPY_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_OR_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_NAND)
ENUM_STR_CASE(VK_LOGIC_OP_SET)
ENUM_STR_CASE(VK_LOGIC_OP_CLEAR)
ENUM_STR_CASE(VK_LOGIC_OP_AND)
ENUM_STR_CASE(VK_LOGIC_OP_AND_REVERSE)
ENUM_STR_CASE(VK_LOGIC_OP_COPY)
ENUM_STR_CASE(VK_LOGIC_OP_AND_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_NO_OP)
ENUM_STR_CASE(VK_LOGIC_OP_XOR)
ENUM_STR_CASE(VK_LOGIC_OP_OR)
ENUM_STR_CASE(VK_LOGIC_OP_NOR)
ENUM_STR_CASE(VK_LOGIC_OP_EQUIVALENT)
ENUM_STR_CASE(VK_LOGIC_OP_INVERT)
ENUM_STR_CASE(VK_LOGIC_OP_OR_REVERSE)
ENUM_STR_CASE(VK_LOGIC_OP_COPY_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_OR_INVERTED)
ENUM_STR_CASE(VK_LOGIC_OP_NAND)
ENUM_STR_CASE(VK_LOGIC_OP_SET)
default:
return "VK_LOGIC_OP_UNKNOWN";
}
}
PipelineFactory::PipelineFactory(VkDevice device, const VulkanRendererConfig& config):
m_device(device), m_config(config) {
PipelineFactory::PipelineFactory(VkDevice device, const VulkanRendererConfig& config)
: m_device(device), m_config(config) {
MOBILEGL_ASSERT(m_device != VK_NULL_HANDLE, "PipelineFactory: device is null");
if (m_config.DisablePipelineCache) {
@@ -149,8 +149,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// If a future workload pairs another blend shape with an equality chain, widen
// this with that evidence in hand rather than pre-emptively.
Bool IsAccumulationBlend(const VkPipelineColorBlendAttachmentState& attachment) {
return attachment.colorBlendOp == VK_BLEND_OP_MIN ||
attachment.colorBlendOp == VK_BLEND_OP_MAX;
return attachment.colorBlendOp == VK_BLEND_OP_MIN || attachment.colorBlendOp == VK_BLEND_OP_MAX;
}
} // namespace
@@ -204,9 +203,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
constexpr Uint64 wireDomain = 0x5749524552504b59ull;
XXHASH_VERIFY(XXH64_update(hashState, &wireDomain, sizeof(wireDomain)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.wireRenderPassCompatibilityId,
sizeof(payload.wireRenderPassCompatibilityId)));
} else
{
sizeof(payload.wireRenderPassCompatibilityId)));
} else {
XXHASH_VERIFY(XXH64_update(hashState, &payload.renderPass, sizeof(payload.renderPass)));
}
XXHASH_VERIFY(XXH64_update(hashState, &payload.colorAttachmentCount, sizeof(payload.colorAttachmentCount)));
@@ -216,11 +214,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(hashState, &payload.sampleMask, sizeof(payload.sampleMask)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.subpass, sizeof(payload.subpass)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.topology, sizeof(payload.topology)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.primitiveRestartEnable, sizeof(payload.primitiveRestartEnable)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.primitiveRestartEnable, sizeof(payload.primitiveRestartEnable)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.patchControlPoints, sizeof(payload.patchControlPoints)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.passthroughTessControlKey,
sizeof(payload.passthroughTessControlKey)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.passthroughTessControlKey, sizeof(payload.passthroughTessControlKey)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.viewportCount, sizeof(payload.viewportCount)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.nonLayeredStages, sizeof(payload.nonLayeredStages)));
const Bool fullCoverageSampleMask = payload.fullCoverageFragmentModule != VK_NULL_HANDLE;
@@ -228,8 +225,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(hashState, &payload.polygonMode, sizeof(payload.polygonMode)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.cullMode, sizeof(payload.cullMode)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.frontFace, sizeof(payload.frontFace)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.provokingVertexMode, sizeof(payload.provokingVertexMode)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.provokingVertexMode, sizeof(payload.provokingVertexMode)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.depthTestEnable, sizeof(payload.depthTestEnable)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.depthWriteEnable, sizeof(payload.depthWriteEnable)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.depthBiasEnable, sizeof(payload.depthBiasEnable)));
@@ -243,21 +239,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(hashState, &payload.frontStencilPassOp, sizeof(payload.frontStencilPassOp)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.frontStencilDepthFailOp, sizeof(payload.frontStencilDepthFailOp)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.frontStencilCompareOp, sizeof(payload.frontStencilCompareOp)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.frontStencilCompareOp, sizeof(payload.frontStencilCompareOp)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.backStencilFailOp, sizeof(payload.backStencilFailOp)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.backStencilPassOp, sizeof(payload.backStencilPassOp)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.backStencilDepthFailOp, sizeof(payload.backStencilDepthFailOp)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.backStencilCompareOp, sizeof(payload.backStencilCompareOp)));
XXHASH_VERIFY(
XXH64_update(hashState, &payload.fragmentReplacesDepth, sizeof(payload.fragmentReplacesDepth)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.backStencilDepthFailOp, sizeof(payload.backStencilDepthFailOp)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.backStencilCompareOp, sizeof(payload.backStencilCompareOp)));
XXHASH_VERIFY(XXH64_update(hashState, &payload.fragmentReplacesDepth, sizeof(payload.fragmentReplacesDepth)));
if (payload.colorAttachmentCount > 0) {
XXHASH_VERIFY(XXH64_update(
hashState,
payload.colorBlendAttachments.data(),
sizeof(payload.colorBlendAttachments[0]) * payload.colorAttachmentCount));
XXHASH_VERIFY(XXH64_update(hashState, payload.colorBlendAttachments.data(),
sizeof(payload.colorBlendAttachments[0]) * payload.colorAttachmentCount));
}
return XXH64_digest(hashState);
}
@@ -284,17 +274,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Log.h ordering fix made MGLOG_E live in INFO builds.
MGLOG_E("PipelineFactory::GetOrCreatePipeline: creation failed for hash=0x%llx "
"programHash=0x%llx; not caching the failure",
static_cast<unsigned long long>(hash),
static_cast<unsigned long long>(payload.programHash));
static_cast<unsigned long long>(hash), static_cast<unsigned long long>(payload.programHash));
return VK_NULL_HANDLE;
}
m_cache.emplace(hash, PipelineCacheEntry{pipeline, payload.programHash,
// Wire pipelines outlive their creator pass and
// may serve another compatible pass in flight.
// Never evict them by a recycled native handle.
payload.wireRenderPassCompatibilityId != 0 ? VK_NULL_HANDLE :
payload.renderPass,
m_frameCounter});
m_cache.emplace(
hash, PipelineCacheEntry{pipeline, payload.programHash,
// Wire pipelines outlive their creator pass and
// may serve another compatible pass in flight.
// Never evict them by a recycled native handle.
payload.wireRenderPassCompatibilityId != 0 ? VK_NULL_HANDLE : payload.renderPass,
m_frameCounter});
return pipeline;
}
@@ -356,8 +345,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
if (evicted > 0) {
MGLOG_D("PipelineFactory::EvictByProgramHash: evicted %u pipelines for program hash 0x%llx",
evicted, static_cast<unsigned long long>(programHash));
MGLOG_D("PipelineFactory::EvictByProgramHash: evicted %u pipelines for program hash 0x%llx", evicted,
static_cast<unsigned long long>(programHash));
}
return evicted;
}
@@ -377,27 +366,24 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(payload.pipelineLayout != VK_NULL_HANDLE, "PipelineFactory: pipelineLayout is null");
MOBILEGL_ASSERT(payload.renderPass != VK_NULL_HANDLE, "PipelineFactory: renderPass is null");
MOBILEGL_ASSERT(payload.colorAttachmentCount <= PipelineCreatePayload::kMaxColorAttachments,
"PipelineFactory: colorAttachmentCount=%u is unexpectedly large",
payload.colorAttachmentCount);
MGLOG_D("PipelineFactory::CreatePipeline: programHash=0x%llx vertexInputHash=0x%llx colorAttachmentCount=%u subpass=%u",
static_cast<unsigned long long>(payload.programHash),
static_cast<unsigned long long>(payload.vertexInputHash),
payload.colorAttachmentCount,
payload.subpass);
"PipelineFactory: colorAttachmentCount=%u is unexpectedly large", payload.colorAttachmentCount);
MGLOG_D("PipelineFactory::CreatePipeline: programHash=0x%llx vertexInputHash=0x%llx colorAttachmentCount=%u "
"subpass=%u",
static_cast<unsigned long long>(payload.programHash),
static_cast<unsigned long long>(payload.vertexInputHash), payload.colorAttachmentCount,
payload.subpass);
// The fragment shading rate goes last so the common prefix is the eight-state list every
// pipeline shares; it is only present when the device enabled the pipeline rate.
static constexpr VkDynamicState kDynamicStates[] = {
VK_DYNAMIC_STATE_VIEWPORT,
VK_DYNAMIC_STATE_SCISSOR,
VK_DYNAMIC_STATE_BLEND_CONSTANTS,
VK_DYNAMIC_STATE_DEPTH_BIAS,
VK_DYNAMIC_STATE_LINE_WIDTH,
VK_DYNAMIC_STATE_STENCIL_COMPARE_MASK,
VK_DYNAMIC_STATE_STENCIL_WRITE_MASK,
VK_DYNAMIC_STATE_STENCIL_REFERENCE,
VK_DYNAMIC_STATE_FRAGMENT_SHADING_RATE_KHR
};
static constexpr VkDynamicState kDynamicStates[] = {VK_DYNAMIC_STATE_VIEWPORT,
VK_DYNAMIC_STATE_SCISSOR,
VK_DYNAMIC_STATE_BLEND_CONSTANTS,
VK_DYNAMIC_STATE_DEPTH_BIAS,
VK_DYNAMIC_STATE_LINE_WIDTH,
VK_DYNAMIC_STATE_STENCIL_COMPARE_MASK,
VK_DYNAMIC_STATE_STENCIL_WRITE_MASK,
VK_DYNAMIC_STATE_STENCIL_REFERENCE,
VK_DYNAMIC_STATE_FRAGMENT_SHADING_RATE_KHR};
VkPipelineDynamicStateCreateInfo dynamicState{};
dynamicState.sType = VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO;
@@ -552,12 +538,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!s_warnedHalfTessellatedPipeline) {
s_warnedHalfTessellatedPipeline = true;
MGLOG_E_ONCE("PipelineFactory::CreatePipeline: refusing a pipeline with %s tessellation stage and "
"no %s stage (VUID-VkGraphicsPipelineCreateInfo-pStages-00730). programHash=0x%llx "
"patchControlPoints=%u. Its draws are skipped; logged once.",
hasTessEval ? "an evaluation" : "a control",
hasTessEval ? "control" : "evaluation",
static_cast<unsigned long long>(payload.programHash),
payload.patchControlPoints);
"no %s stage (VUID-VkGraphicsPipelineCreateInfo-pStages-00730). programHash=0x%llx "
"patchControlPoints=%u. Its draws are skipped; logged once.",
hasTessEval ? "an evaluation" : "a control", hasTessEval ? "control" : "evaluation",
static_cast<unsigned long long>(payload.programHash), payload.patchControlPoints);
}
return VK_NULL_HANDLE;
}
@@ -568,8 +552,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
gpi.pStages = effectiveStages->data();
gpi.pVertexInputState = payload.vertexInputState;
gpi.pInputAssemblyState = &ia;
gpi.pTessellationState =
payload.topology == VK_PRIMITIVE_TOPOLOGY_PATCH_LIST ? &tessellation : nullptr;
gpi.pTessellationState = payload.topology == VK_PRIMITIVE_TOPOLOGY_PATCH_LIST ? &tessellation : nullptr;
gpi.pViewportState = &vpci;
gpi.pRasterizationState = &raster;
gpi.pMultisampleState = &ms;
@@ -590,32 +573,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// GetOrCreatePipeline deliberately does not cache the failure, which is what makes that
// repetition happen; if the repetition ever needs to stop, fix the pipeline, not the log.
if (result != VK_SUCCESS) {
MGLOG_F("PipelineFactory::CreatePipeline failed: result=%s (%d) programHash=0x%llx vertexInputHash=0x%llx stageCount=%u topology=%s(%d) colorAttachmentCount=%u samples=%s(%d) subpass=%u",
VkResultToString(result),
result,
static_cast<unsigned long long>(payload.programHash),
static_cast<unsigned long long>(payload.vertexInputHash),
gpi.stageCount,
PrimitiveTopologyToString(payload.topology),
payload.topology,
payload.colorAttachmentCount,
SampleCountToString(payload.rasterizationSamples),
payload.rasterizationSamples,
payload.subpass);
MGLOG_F("PipelineFactory::CreatePipeline state: cullMode=%s(0x%x) frontFace=%d depthTest=%d depthWrite=%d depthCompare=%s(%d) depthBias=%d rasterizerDiscard=%d stencilTest=%d logicOpEnable=%d logicOp=%s(%d)",
CullModeToString(payload.cullMode),
static_cast<Uint32>(payload.cullMode),
payload.frontFace,
payload.depthTestEnable ? 1 : 0,
payload.depthWriteEnable ? 1 : 0,
CompareOpToString(payload.depthCompareOp),
payload.depthCompareOp,
payload.depthBiasEnable ? 1 : 0,
payload.rasterizerDiscardEnable ? 1 : 0,
payload.stencilTestEnable ? 1 : 0,
payload.logicOpEnable ? 1 : 0,
LogicOpToString(payload.logicOp),
payload.logicOp);
MGLOG_F("PipelineFactory::CreatePipeline failed: result=%s (%d) programHash=0x%llx vertexInputHash=0x%llx "
"stageCount=%u topology=%s(%d) colorAttachmentCount=%u samples=%s(%d) subpass=%u",
VkResultToString(result), result, static_cast<unsigned long long>(payload.programHash),
static_cast<unsigned long long>(payload.vertexInputHash), gpi.stageCount,
PrimitiveTopologyToString(payload.topology), payload.topology, payload.colorAttachmentCount,
SampleCountToString(payload.rasterizationSamples), payload.rasterizationSamples, payload.subpass);
MGLOG_F(
"PipelineFactory::CreatePipeline state: cullMode=%s(0x%x) frontFace=%d depthTest=%d depthWrite=%d "
"depthCompare=%s(%d) depthBias=%d rasterizerDiscard=%d stencilTest=%d logicOpEnable=%d logicOp=%s(%d)",
CullModeToString(payload.cullMode), static_cast<Uint32>(payload.cullMode), payload.frontFace,
payload.depthTestEnable ? 1 : 0, payload.depthWriteEnable ? 1 : 0,
CompareOpToString(payload.depthCompareOp), payload.depthCompareOp, payload.depthBiasEnable ? 1 : 0,
payload.rasterizerDiscardEnable ? 1 : 0, payload.stencilTestEnable ? 1 : 0,
payload.logicOpEnable ? 1 : 0, LogicOpToString(payload.logicOp), payload.logicOp);
MGLOG_F("PipelineFactory::CreatePipeline vertex input: bindingCount=%u attributeCount=%u",
payload.vertexInputState->vertexBindingDescriptionCount,
payload.vertexInputState->vertexAttributeDescriptionCount);
@@ -629,8 +600,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto& digest = (*payload.stageSpirvDigests)[i];
MGLOG_I("PipelineFactory::CreatePipeline spirv[%zu]: stage=0x%x words=%u bytes=%zu "
"hash=0x%llx",
i, digest.stage, digest.wordCount,
static_cast<SizeT>(digest.wordCount) * sizeof(Uint32),
i, digest.stage, digest.wordCount, static_cast<SizeT>(digest.wordCount) * sizeof(Uint32),
static_cast<unsigned long long>(digest.hash));
}
} else {
@@ -651,16 +621,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
for (Uint32 i = 0; i < payload.colorAttachmentCount; ++i) {
const auto& attachment = payload.colorBlendAttachments[i];
MGLOG_F("PipelineFactory::CreatePipeline colorAttachment[%u]: blend=%d colorWriteMask=0x%x srcColor=%d dstColor=%d colorOp=%d srcAlpha=%d dstAlpha=%d alphaOp=%d",
i,
attachment.blendEnable == VK_TRUE ? 1 : 0,
static_cast<Uint32>(attachment.colorWriteMask),
attachment.srcColorBlendFactor,
attachment.dstColorBlendFactor,
attachment.colorBlendOp,
attachment.srcAlphaBlendFactor,
attachment.dstAlphaBlendFactor,
attachment.alphaBlendOp);
MGLOG_F("PipelineFactory::CreatePipeline colorAttachment[%u]: blend=%d colorWriteMask=0x%x srcColor=%d "
"dstColor=%d colorOp=%d srcAlpha=%d dstAlpha=%d alphaOp=%d",
i, attachment.blendEnable == VK_TRUE ? 1 : 0, static_cast<Uint32>(attachment.colorWriteMask),
attachment.srcColorBlendFactor, attachment.dstColorBlendFactor, attachment.colorBlendOp,
attachment.srcAlphaBlendFactor, attachment.dstAlphaBlendFactor, attachment.alphaBlendOp);
}
}
VK_VERIFY(result, "vkCreateGraphicsPipelines");
@@ -174,8 +174,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static Bool IsSuppressBlendedDepthWriteEnabled() { return s_suppressBlendedDepthWrite; }
// Device gate for the quirk: ForceOn/ForceOff bypass detection, Auto enables it on
// the known-affected vendor (Qualcomm).
static Bool ShouldSuppressBlendedDepthWriteForDevice(MG_Config::QuirkOverride quirkOverride,
Uint32 vendorId);
static Bool ShouldSuppressBlendedDepthWriteForDevice(MG_Config::QuirkOverride quirkOverride, Uint32 vendorId);
// Pure per-pipeline strip decision (exempts gl_FragDepth writers, masked-out and
// non-accumulation blends); combined with the device flag in CreatePipeline. Static
// and payload-only so tests can pin the contract without a VkDevice.
File diff suppressed because it is too large Load Diff
@@ -102,9 +102,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
};
// What a program parsed below #version 450 carries, and the fallback when a module's
// block cannot be read.
static constexpr Uint32 kDefaultPerVertexMembers =
static_cast<Uint32>(PerVertexMemberBit::Position) | static_cast<Uint32>(PerVertexMemberBit::PointSize) |
static_cast<Uint32>(PerVertexMemberBit::ClipDistance);
static constexpr Uint32 kDefaultPerVertexMembers = static_cast<Uint32>(PerVertexMemberBit::Position) |
static_cast<Uint32>(PerVertexMemberBit::PointSize) |
static_cast<Uint32>(PerVertexMemberBit::ClipDistance);
struct UpdateAfterBindLimits {
Bool enabled = false;
@@ -281,7 +281,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// entry pointer re-stamps use through a const reference (StampProgramUse).
mutable Uint64 lastUsedFrame = 0;
static inline SessionLocal<VkDevice> s_device; // per Magma session (MagmaSession.h)
static inline SessionLocal<VkDevice> s_device; // per Magma session (MagmaSession.h)
VkProgramObject() = default;
VkProgramObject(const VkProgramObject&) = delete;
@@ -315,8 +315,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerTextureTargetByBinding = std::move(other.samplerTextureTargetByBinding);
samplerNumericDomainByBinding = std::move(other.samplerNumericDomainByBinding);
storageImageFormatByBinding = std::move(other.storageImageFormatByBinding);
storageImageUsesBindingFormatByBinding =
std::move(other.storageImageUsesBindingFormatByBinding);
storageImageUsesBindingFormatByBinding = std::move(other.storageImageUsesBindingFormatByBinding);
storageBlockNameByBinding = std::move(other.storageBlockNameByBinding);
storageBlockIndexByBinding = std::move(other.storageBlockIndexByBinding);
hasStorageImages = other.hasStorageImages;
@@ -390,8 +389,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerTextureTargetByBinding = std::move(other.samplerTextureTargetByBinding);
samplerNumericDomainByBinding = std::move(other.samplerNumericDomainByBinding);
storageImageFormatByBinding = std::move(other.storageImageFormatByBinding);
storageImageUsesBindingFormatByBinding =
std::move(other.storageImageUsesBindingFormatByBinding);
storageImageUsesBindingFormatByBinding = std::move(other.storageImageUsesBindingFormatByBinding);
storageBlockNameByBinding = std::move(other.storageBlockNameByBinding);
storageBlockIndexByBinding = std::move(other.storageBlockIndexByBinding);
hasStorageImages = other.hasStorageImages;
@@ -441,9 +439,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return *this;
}
~VkProgramObject() {
Destroy();
}
~VkProgramObject() { Destroy(); }
private:
void Destroy() {
@@ -489,11 +485,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// repair passes patch modules AROUND them, and the emulation only replaces them
// on opted-in devices with no subgroup support at all.
struct SubgroupLoweringPolicy {
Bool emulateSubgroups = false; // no-native-support devices, or MOBILEGL_SUBGROUP_EMULATION
Bool emulateSubgroups = false; // no-native-support devices, or MOBILEGL_SUBGROUP_EMULATION
Bool fixIterationRPSubgroupScratch = false; // patch iterationRP's under-declared scratch
Bool fixIterationRPBarrier = false; // repair Program 203's shared-scratch race
Bool deriveNumSubgroups = false; // repair the NumSubgroups builtin
Bool requireFullSubgroups = false; // computeFullSubgroups enabled on the device
Bool fixIterationRPBarrier = false; // repair Program 203's shared-scratch race
Bool deriveNumSubgroups = false; // repair the NumSubgroups builtin
Bool requireFullSubgroups = false; // computeFullSubgroups enabled on the device
Uint32 nativeSubgroupSize = 0;
// Full-subgroup launches are bounded by this device limit; a dispatch whose
// workgroup needs more subgroups than this cannot request the flag.
@@ -504,18 +500,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
};
explicit ProgramFactory(VkDevice device, const VulkanRendererConfig& config, Uint32 maxBindings,
Bool shaderDrawParametersEnabled,
Bool unformattedFloatStorageImagesEnabled,
Bool tessellationAndGeometryPointSizeEnabled,
Bool enableSpirvValidation,
UpdateAfterBindLimits updateAfterBindLimits,
SubgroupLoweringPolicy subgroupPolicy)
Bool shaderDrawParametersEnabled, Bool unformattedFloatStorageImagesEnabled,
Bool tessellationAndGeometryPointSizeEnabled, Bool enableSpirvValidation,
UpdateAfterBindLimits updateAfterBindLimits, SubgroupLoweringPolicy subgroupPolicy)
: m_device(device), m_maxBindings(maxBindings), m_config(config),
m_shaderDrawParametersEnabled(shaderDrawParametersEnabled),
m_unformattedFloatStorageImagesEnabled(unformattedFloatStorageImagesEnabled),
m_tessellationAndGeometryPointSizeEnabled(tessellationAndGeometryPointSizeEnabled),
m_enableSpirvValidation(enableSpirvValidation),
m_updateAfterBindLimits(updateAfterBindLimits),
m_enableSpirvValidation(enableSpirvValidation), m_updateAfterBindLimits(updateAfterBindLimits),
m_subgroupPolicy(subgroupPolicy) {
VkProgramObject::s_device.Get() = device;
}
@@ -526,8 +518,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ProgramFactory(const ProgramFactory&) = delete;
HashType ComputeHash(const MagmaProgramSource& program, CompileOptionFlags flags) const;
const VkProgramObject& GetOrCreateProgram(
const MagmaProgramSource& program, CompileOptionFlags flags);
const VkProgramObject& GetOrCreateProgram(const MagmaProgramSource& program, CompileOptionFlags flags);
// The default framebuffer's current image height, baked as a literal into every
// FragCoordYFlip variant (there is no push-constant or specialization channel here, and
@@ -631,9 +622,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// (size, levels, members) combinations a program draws with - one or two in practice -
// and only ever happens for the rare program that has no control stage at all.
VkPipelineShaderStageCreateInfo GetOrCreatePassthroughTessControlStage(Uint32 patchVertices,
const FloatVec4& defaultOuterLevel,
const FloatVec2& defaultInnerLevel,
Uint32 perVertexMembers);
const FloatVec4& defaultOuterLevel,
const FloatVec2& defaultInnerLevel,
Uint32 perVertexMembers);
// Source of the module above. Exposed for tests: the generated GLSL is the whole
// contract with the evaluation stage, so it is worth pinning independently of a device.
@@ -668,22 +659,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// `spirv`, at the same index. Taking the stages rather than the shader objects is what
// keeps the program's live attach list, which is a longer and differently-indexed list
// the moment a glAttachShader lands after the link, from being passed here by mistake.
void ReflectVertexInputs(const Vector<ShaderStage>& stages,
const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
void ReflectViewportIndexUsage(const Vector<ShaderStage>& stages,
const Vector<Vector<Uint>>& spirv,
void ReflectVertexInputs(const Vector<ShaderStage>& stages, const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
void ReflectViewportIndexUsage(const Vector<ShaderStage>& stages, const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
void ReflectFragmentOutputs(const Vector<ShaderStage>& stages,
const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
void ReflectFragmentOutputs(const Vector<ShaderStage>& stages, const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
void ReflectLayout(const MagmaProgramSource& program, const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
// Fills needsPassthroughTessControl / passthroughTessControlEmulatable off the linked
// modules. Const and reflection-only: it decides nothing about the pipeline, it only
// records what the evaluation stage's input interface is made of.
void ReflectPassthroughTessControlNeed(const Vector<ShaderStage>& stages,
const Vector<Vector<Uint>>& spirv,
void ReflectPassthroughTessControlNeed(const Vector<ShaderStage>& stages, const Vector<Vector<Uint>>& spirv,
VkProgramObject& entry) const;
VkDevice m_device = VK_NULL_HANDLE;
@@ -13,29 +13,29 @@
namespace MobileGL::MG_Backend::DirectVulkan {
// Collect every fence in the ordered submission prefix. Waiting on only the last matching
// frame-serial record is insufficient: the same serial may have an earlier in-flight submit.
template <typename Records, typename Fence>
bool CollectSubmitFencePrefix(const Records& records, std::uint64_t submitIndex,
Fence nullFence, std::vector<Fence>& fences) {
fences.clear();
bool foundTarget = false;
for (const auto& record : records) {
if (record.submitIndex > submitIndex) {
break;
// Collect every fence in the ordered submission prefix. Waiting on only the last matching
// frame-serial record is insufficient: the same serial may have an earlier in-flight submit.
template <typename Records, typename Fence>
bool CollectSubmitFencePrefix(const Records& records, std::uint64_t submitIndex, Fence nullFence,
std::vector<Fence>& fences) {
fences.clear();
bool foundTarget = false;
for (const auto& record : records) {
if (record.submitIndex > submitIndex) {
break;
}
if (record.fence == nullFence) {
fences.clear();
return false;
}
fences.push_back(record.fence);
foundTarget = foundTarget || record.submitIndex == submitIndex;
}
if (record.fence == nullFence) {
if (!foundTarget) {
fences.clear();
return false;
}
fences.push_back(record.fence);
foundTarget = foundTarget || record.submitIndex == submitIndex;
return true;
}
if (!foundTarget) {
fences.clear();
return false;
}
return true;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -92,7 +92,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} // namespace
SwapchainObject::SwapchainCapabilities SwapchainObject::GetSwapchainCapabilities(VkPhysicalDevice physicalDevice,
VkSurfaceKHR surface) {
VkSurfaceKHR surface) {
SwapchainCapabilities swapchainCapabilities{};
VK_VERIFY(
@@ -102,17 +102,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VK_VERIFY(vkGetPhysicalDeviceSurfaceFormatsKHR(physicalDevice, surface, &formatCount, nullptr));
if (formatCount != 0) {
swapchainCapabilities.surfaceFormats.resize(formatCount);
VK_VERIFY(vkGetPhysicalDeviceSurfaceFormatsKHR(
physicalDevice, surface, &formatCount, swapchainCapabilities.surfaceFormats.data()));
VK_VERIFY(vkGetPhysicalDeviceSurfaceFormatsKHR(physicalDevice, surface, &formatCount,
swapchainCapabilities.surfaceFormats.data()));
}
Uint32 presentModeCount = 0;
VK_VERIFY(
vkGetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, &presentModeCount, nullptr));
VK_VERIFY(vkGetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, &presentModeCount, nullptr));
if (presentModeCount != 0) {
swapchainCapabilities.presentModes.resize(presentModeCount);
VK_VERIFY(vkGetPhysicalDeviceSurfacePresentModesKHR(
physicalDevice, surface, &presentModeCount, swapchainCapabilities.presentModes.data()));
VK_VERIFY(vkGetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, &presentModeCount,
swapchainCapabilities.presentModes.data()));
}
return swapchainCapabilities;
@@ -139,13 +138,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return availableFormats[0];
}
VkPresentModeKHR SwapchainObject::ChooseSwapchainPresentMode(
const Vector<VkPresentModeKHR>& availablePresentModes, Optional<Int> swapInterval) {
VkPresentModeKHR SwapchainObject::ChooseSwapchainPresentMode(const Vector<VkPresentModeKHR>& availablePresentModes,
Optional<Int> swapInterval) {
static constexpr VkPresentModeKHR s_unthrottledPresentModes[]{
VK_PRESENT_MODE_IMMEDIATE_KHR, VK_PRESENT_MODE_MAILBOX_KHR, VK_PRESENT_MODE_FIFO_KHR};
static constexpr VkPresentModeKHR s_vsyncPresentModes[]{VK_PRESENT_MODE_FIFO_KHR};
static constexpr VkPresentModeKHR s_adaptiveVsyncPresentModes[]{VK_PRESENT_MODE_FIFO_RELAXED_KHR,
VK_PRESENT_MODE_FIFO_KHR};
VK_PRESENT_MODE_FIFO_KHR};
std::span<const VkPresentModeKHR> ranked = s_desiredPresentModes;
if (swapInterval) {
if (*swapInterval == 0) {
@@ -200,8 +199,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto& swapchainCaps = swapchainCapabilities.capabilities;
// The extra images count from the surface's own minimum: that is how many the platform
// keeps for itself and the application, so only images beyond it delay a buffer's reuse.
Uint32 targetImageCount =
std::max<Uint32>(minImageCountHint, swapchainCaps.minImageCount + extraImageCount);
Uint32 targetImageCount = std::max<Uint32>(minImageCountHint, swapchainCaps.minImageCount + extraImageCount);
if (swapchainCaps.maxImageCount != 0) {
targetImageCount = std::min(targetImageCount, swapchainCaps.maxImageCount);
}
@@ -217,11 +215,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
createInfo.imageColorSpace = pickedSurfaceFormat.colorSpace;
createInfo.imageExtent = swapchainCaps.currentExtent;
if (createInfo.imageExtent.width == UINT32_MAX || createInfo.imageExtent.height == UINT32_MAX) {
createInfo.imageExtent.width = std::clamp(desiredExtent.width,
swapchainCaps.minImageExtent.width,
swapchainCaps.maxImageExtent.width);
createInfo.imageExtent.height = std::clamp(desiredExtent.height,
swapchainCaps.minImageExtent.height,
createInfo.imageExtent.width =
std::clamp(desiredExtent.width, swapchainCaps.minImageExtent.width, swapchainCaps.maxImageExtent.width);
createInfo.imageExtent.height = std::clamp(desiredExtent.height, swapchainCaps.minImageExtent.height,
swapchainCaps.maxImageExtent.height);
}
const VkExtent2D defaultFramebufferExtent = createInfo.imageExtent;
@@ -258,11 +254,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
createInfo.preTransform = swapchainCaps.currentTransform;
MGLOG_I("Set swapchain preTransform = %s", string_VkSurfaceTransformFlagBitsKHR(createInfo.preTransform));
const VkCompositeAlphaFlagBitsKHR compositeAlphaCandidates[] = {
VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR,
VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR,
VK_COMPOSITE_ALPHA_POST_MULTIPLIED_BIT_KHR,
VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR
};
VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR, VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR,
VK_COMPOSITE_ALPHA_POST_MULTIPLIED_BIT_KHR, VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR};
createInfo.compositeAlpha = VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR;
for (auto candidate : compositeAlphaCandidates) {
if ((swapchainCaps.supportedCompositeAlpha & candidate) != 0) {
@@ -312,24 +305,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto& defaultFBOInfo = MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo;
const Int extentWidth = static_cast<Int>(defaultFramebufferExtent.width);
const Int extentHeight = static_cast<Int>(defaultFramebufferExtent.height);
const SizeT defaultAttachmentByteSize =
static_cast<SizeT>(defaultFramebufferExtent.width) *
static_cast<SizeT>(defaultFramebufferExtent.height) * 4;
const SizeT defaultAttachmentByteSize = static_cast<SizeT>(defaultFramebufferExtent.width) *
static_cast<SizeT>(defaultFramebufferExtent.height) * 4;
TextureInternalFormat depthFormat = TextureInternalFormat::Depth24Stencil8;
switch (m_depthStencilFormat) {
case VK_FORMAT_D24_UNORM_S8_UINT:
depthFormat = TextureInternalFormat::Depth24Stencil8;
break;
case VK_FORMAT_D32_SFLOAT_S8_UINT:
depthFormat = TextureInternalFormat::Depth32FStencil8;
break;
case VK_FORMAT_D32_SFLOAT:
depthFormat = TextureInternalFormat::DepthComponent32F;
break;
default:
depthFormat = TextureInternalFormat::Depth24Stencil8;
break;
case VK_FORMAT_D24_UNORM_S8_UINT:
depthFormat = TextureInternalFormat::Depth24Stencil8;
break;
case VK_FORMAT_D32_SFLOAT_S8_UINT:
depthFormat = TextureInternalFormat::Depth32FStencil8;
break;
case VK_FORMAT_D32_SFLOAT:
depthFormat = TextureInternalFormat::DepthComponent32F;
break;
default:
depthFormat = TextureInternalFormat::Depth24Stencil8;
break;
}
// The default FBO's stencil attachment must track the swapchain extent:
@@ -339,16 +331,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// GL_INVALID_FRAMEBUFFER_OPERATION on DirectVulkan.
TextureInternalFormat stencilFormat = TextureInternalFormat::Depth24Stencil8;
switch (m_depthStencilFormat) {
case VK_FORMAT_D32_SFLOAT_S8_UINT:
stencilFormat = TextureInternalFormat::Depth32FStencil8;
break;
case VK_FORMAT_D24_UNORM_S8_UINT:
stencilFormat = TextureInternalFormat::Depth24Stencil8;
break;
default:
// No stencil plane; mirror the depth format for consistency.
stencilFormat = depthFormat;
break;
case VK_FORMAT_D32_SFLOAT_S8_UINT:
stencilFormat = TextureInternalFormat::Depth32FStencil8;
break;
case VK_FORMAT_D24_UNORM_S8_UINT:
stencilFormat = TextureInternalFormat::Depth24Stencil8;
break;
default:
// No stencil plane; mirror the depth format for consistency.
stencilFormat = depthFormat;
break;
}
if (MG_Pipe::gMGPipeCallbacks.OnSurfaceChanged != nullptr) {
@@ -367,26 +359,24 @@ namespace MobileGL::MG_Backend::DirectVulkan {
info.Layers = 1;
info.IsDefault = 1;
MG_Pipe::gMGPipeCallbacks.OnSurfaceChanged(&info);
} else
{
} else {
auto* colorTex = static_cast<MG_State::GLState::TextureObject2D*>(defaultFBOInfo->colorAttachment.get());
colorTex->AllocateStorage(
TextureUploadTarget::Texture2D, 0, {
{extentWidth, extentHeight, 1},
defaultAttachmentByteSize}); // TODO: 4 is format size
TextureUploadTarget::Texture2D, 0,
{{extentWidth, extentHeight, 1}, defaultAttachmentByteSize}); // TODO: 4 is format size
auto* depthTex = static_cast<MG_State::GLState::TextureObject2D*>(defaultFBOInfo->depthAttachment.get());
depthTex->SetInternalFormat(depthFormat);
depthTex->AllocateStorage(TextureUploadTarget::Texture2D, 0, {
{extentWidth, extentHeight, 1},
defaultAttachmentByteSize}); // TODO: 4 is format size
depthTex->AllocateStorage(
TextureUploadTarget::Texture2D, 0,
{{extentWidth, extentHeight, 1}, defaultAttachmentByteSize}); // TODO: 4 is format size
auto* stencilTex = static_cast<MG_State::GLState::TextureObject2D*>(defaultFBOInfo->stencilAttachment.get());
auto* stencilTex =
static_cast<MG_State::GLState::TextureObject2D*>(defaultFBOInfo->stencilAttachment.get());
stencilTex->SetInternalFormat(stencilFormat);
stencilTex->AllocateStorage(TextureUploadTarget::Texture2D, 0, {
{extentWidth, extentHeight, 1},
defaultAttachmentByteSize}); // TODO: 4 is format size
stencilTex->AllocateStorage(
TextureUploadTarget::Texture2D, 0,
{{extentWidth, extentHeight, 1}, defaultAttachmentByteSize}); // TODO: 4 is format size
}
}
void SwapchainObject::CreateDepthStencilResources(VkDevice device, VkPhysicalDevice physicalDevice) {
@@ -20,9 +20,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<VkSurfaceFormatKHR> surfaceFormats;
Vector<VkPresentModeKHR> presentModes;
Bool IsComplete() const {
return !surfaceFormats.empty() && !presentModes.empty();
}
Bool IsComplete() const { return !surfaceFormats.empty() && !presentModes.empty(); }
};
static SwapchainCapabilities GetSwapchainCapabilities(VkPhysicalDevice physicalDevice, VkSurfaceKHR surface);
@@ -97,12 +95,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void CreateImageViews(VkDevice device);
void CreateDepthStencilResources(VkDevice device, VkPhysicalDevice physicalDevice);
void DestroyDepthStencilResources(VkDevice device);
static constexpr VkPresentModeKHR s_desiredPresentModes[] {
VK_PRESENT_MODE_MAILBOX_KHR,
VK_PRESENT_MODE_IMMEDIATE_KHR,
VK_PRESENT_MODE_FIFO_RELAXED_KHR,
VK_PRESENT_MODE_FIFO_KHR
};
static constexpr VkPresentModeKHR s_desiredPresentModes[]{
VK_PRESENT_MODE_MAILBOX_KHR, VK_PRESENT_MODE_IMMEDIATE_KHR, VK_PRESENT_MODE_FIFO_RELAXED_KHR,
VK_PRESENT_MODE_FIFO_KHR};
VkSwapchainKHR m_swapchain = VK_NULL_HANDLE;
VkPresentModeKHR m_presentMode = VK_PRESENT_MODE_FIFO_KHR;
@@ -44,8 +44,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
SizeT UniformManager::WireImageViewKeyHash::operator()(const WireImageViewKey& key) const {
SizeT hash = std::hash<VkImage>{}(key.image);
const auto mix = [&hash](Uint64 value) {
hash ^= std::hash<Uint64>{}(value) + static_cast<SizeT>(0x9e3779b97f4a7c15ULL) +
(hash << 6) + (hash >> 2);
hash ^= std::hash<Uint64>{}(value) + static_cast<SizeT>(0x9e3779b97f4a7c15ULL) + (hash << 6) + (hash >> 2);
};
mix(key.root.Slot);
mix(key.root.Gen);
@@ -65,33 +64,41 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return hash;
}
// P7 wave 0, the descriptor half of WireFramebuffer.inc's MagmaWireFatal - same message,
// P7 wave 0, the descriptor half of WireFramebuffer.cpp's MagmaWireFatal - same message,
// same seam, same reason. Four of the fifteen P7-marked refusals die here.
[[noreturn]] static void WireDescriptorFatal(const char* detail) {
MG_Pipe::MGPipeSessionFail(MG_Pipe::MGPipeFatalFamily::UnmigratedVerb,
"MGPipe: Fatal{UnmigratedVerb, \"Magma:%s\"}", detail);
}
static Bool ResolveWireRange(Uint64 offset, Uint64 declaredSize, VkDeviceSize bufferSize,
VkDeviceSize& start, VkDeviceSize& size) {
static Bool ResolveWireRange(Uint64 offset, Uint64 declaredSize, VkDeviceSize bufferSize, VkDeviceSize& start,
VkDeviceSize& size) {
if (offset >= bufferSize) return false;
start = static_cast<VkDeviceSize>(offset);
const VkDeviceSize remaining = bufferSize - start;
size = declaredSize == MG_Pipe::kMGPipeWholeBuffer
? remaining : std::min<VkDeviceSize>(declaredSize, remaining);
size =
declaredSize == MG_Pipe::kMGPipeWholeBuffer ? remaining : std::min<VkDeviceSize>(declaredSize, remaining);
return size != 0;
}
static VkImageViewType WireViewType(MG_Pipe::MGPipeResourceTarget target) {
using T = MG_Pipe::MGPipeResourceTarget;
switch (target) {
case T::Tex1D: return VK_IMAGE_VIEW_TYPE_1D;
case T::Tex1DArray: return VK_IMAGE_VIEW_TYPE_1D_ARRAY;
case T::Tex3D: return VK_IMAGE_VIEW_TYPE_3D;
case T::TexCube: return VK_IMAGE_VIEW_TYPE_CUBE;
case T::TexCubeArray: return VK_IMAGE_VIEW_TYPE_CUBE_ARRAY;
case T::Tex2DArray: case T::Tex2DMSArray: return VK_IMAGE_VIEW_TYPE_2D_ARRAY;
default: return VK_IMAGE_VIEW_TYPE_2D;
case T::Tex1D:
return VK_IMAGE_VIEW_TYPE_1D;
case T::Tex1DArray:
return VK_IMAGE_VIEW_TYPE_1D_ARRAY;
case T::Tex3D:
return VK_IMAGE_VIEW_TYPE_3D;
case T::TexCube:
return VK_IMAGE_VIEW_TYPE_CUBE;
case T::TexCubeArray:
return VK_IMAGE_VIEW_TYPE_CUBE_ARRAY;
case T::Tex2DArray:
case T::Tex2DMSArray:
return VK_IMAGE_VIEW_TYPE_2D_ARRAY;
default:
return VK_IMAGE_VIEW_TYPE_2D;
}
}
@@ -99,8 +106,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// view, and an ignored nonzero Layer must not address another slice.
static Bool WireTargetHasLayers(MG_Pipe::MGPipeResourceTarget target) {
using T = MG_Pipe::MGPipeResourceTarget;
return target == T::Tex1DArray || target == T::Tex2DArray || target == T::Tex2DMSArray ||
target == T::Tex3D || target == T::TexCube || target == T::TexCubeArray;
return target == T::Tex1DArray || target == T::Tex2DArray || target == T::Tex2DMSArray || target == T::Tex3D ||
target == T::TexCube || target == T::TexCubeArray;
}
// GL 4.6 core 8.26: an access through an image unit whose texture does not HAVE the
@@ -140,20 +147,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static VkComponentSwizzle WireSwizzle(Uint8 value) {
switch (static_cast<TextureSwizzleParam>(value)) {
case TextureSwizzleParam::Red: return VK_COMPONENT_SWIZZLE_R;
case TextureSwizzleParam::Green: return VK_COMPONENT_SWIZZLE_G;
case TextureSwizzleParam::Blue: return VK_COMPONENT_SWIZZLE_B;
case TextureSwizzleParam::Alpha: return VK_COMPONENT_SWIZZLE_A;
case TextureSwizzleParam::Zero: return VK_COMPONENT_SWIZZLE_ZERO;
case TextureSwizzleParam::One: return VK_COMPONENT_SWIZZLE_ONE;
default: WireDescriptorFatal("sampler-swizzle");
case TextureSwizzleParam::Red:
return VK_COMPONENT_SWIZZLE_R;
case TextureSwizzleParam::Green:
return VK_COMPONENT_SWIZZLE_G;
case TextureSwizzleParam::Blue:
return VK_COMPONENT_SWIZZLE_B;
case TextureSwizzleParam::Alpha:
return VK_COMPONENT_SWIZZLE_A;
case TextureSwizzleParam::Zero:
return VK_COMPONENT_SWIZZLE_ZERO;
case TextureSwizzleParam::One:
return VK_COMPONENT_SWIZZLE_ONE;
default:
WireDescriptorFatal("sampler-swizzle");
}
}
Bool UniformManager::ResolveWireImageDescriptor(
VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding, Uint32 element,
Bool storage, VkDescriptorImageInfo& out, VkTextureManager::TextureResource** outResource) const {
Bool UniformManager::ResolveWireImageDescriptor(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 element, Bool storage, VkDescriptorImageInfo& out,
VkTextureManager::TextureResource** outResource) const {
out = {};
if (outResource != nullptr) *outResource = nullptr;
auto& state = MG_Pipe::MGPipeApplier();
@@ -161,8 +175,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Int location = baseLocation + static_cast<Int>(element);
if (baseLocation < 0 || !program.UniformLocationsAliasSameUniform(baseLocation, location)) return false;
const Int unit = program.GetUniformSamplerOrImageUnitIndex(static_cast<Uint>(location));
if (unit < 0 || static_cast<Uint32>(unit) >=
(storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits)) return false;
if (unit < 0 ||
static_cast<Uint32>(unit) >= (storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits))
return false;
const auto handle = storage ? state.BoundShaderImages[unit].Res : state.BoundSamplerViews[unit].Texture;
// An EMPTY image unit is the same 8.26 invalid access as the named-but-missing texel
// below, so it takes the same answer: a null storage descriptor where the device has
@@ -190,8 +205,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Bool layerable = WireTargetHasLayers(target);
Uint32 level = storage ? state.BoundShaderImages[unit].Level : record.Params.BaseLevel;
// GL ignores Layer/Layered for non-layerable targets (WireTargetHasLayers).
Uint32 layer = storage && layerable && !state.BoundShaderImages[unit].Layered
? state.BoundShaderImages[unit].Layer : 0;
Uint32 layer =
storage && layerable && !state.BoundShaderImages[unit].Layered ? state.BoundShaderImages[unit].Layer : 0;
const Uint32 localLevel = level;
const Uint32 localLayer = layer;
Uint32 layers = 0;
@@ -207,8 +222,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 levels = 1;
if (!storage) {
if (record.Params.MaxLevel < localLevel || record.Desc.Levels <= localLevel) return false;
levels = std::min<Uint32>(record.Desc.Levels - localLevel,
record.Params.MaxLevel - localLevel + 1);
levels = std::min<Uint32>(record.Desc.Levels - localLevel, record.Params.MaxLevel - localLevel + 1);
const auto viewHandle = state.BoundSamplerViews[unit].View;
if (!MG_Pipe::MGPipeHandleIsNull(viewHandle) && viewHandle.Slot < state.SamplerViewCsos.size()) {
const auto& view = state.SamplerViewCsos[viewHandle.Slot];
@@ -229,8 +243,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto domain = programObj.samplerNumericDomainByBinding[binding];
if (storage) {
format = ResolveStorageImageViewFormat(programObj.storageImageFormatByBinding[binding],
state.BoundShaderImages[unit].InternalFormat, format,
programObj.storageImageUsesBindingFormatByBinding[binding]);
state.BoundShaderImages[unit].InternalFormat, format,
programObj.storageImageUsesBindingFormatByBinding[binding]);
} else if (resource->aspect == VK_IMAGE_ASPECT_COLOR_BIT) {
format = VkTextureManager::ResolveSampledImageViewFormat(format, domain);
}
@@ -238,14 +252,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkImageAspectFlags aspect = resource->aspect;
if (!storage && (aspect & VK_IMAGE_ASPECT_DEPTH_BIT))
aspect = record.Params.DepthStencilMode == MG_Pipe::kMGPipeDepthStencilModeStencil
? VK_IMAGE_ASPECT_STENCIL_BIT : VK_IMAGE_ASPECT_DEPTH_BIT;
? VK_IMAGE_ASPECT_STENCIL_BIT
: VK_IMAGE_ASPECT_DEPTH_BIT;
// All wire image descriptors use GENERAL. A texture sampled and image-bound in
// the same shader then has one truthful layout, including views of the same owner.
if (!VkTextureManager::TransitionImageLayout(commandBuffer, resource->image, resource->layout,
VK_IMAGE_LAYOUT_GENERAL, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
resource->layout == VK_IMAGE_LAYOUT_UNDEFINED ? 0 : VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT,
VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT, resource->aspect, 0, resource->mipLevels))
if (!VkTextureManager::TransitionImageLayout(
commandBuffer, resource->image, resource->layout, VK_IMAGE_LAYOUT_GENERAL,
VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
resource->layout == VK_IMAGE_LAYOUT_UNDEFINED ? 0
: VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT,
VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT, resource->aspect, 0, resource->mipLevels))
WireDescriptorFatal("image-descriptor-transition");
// A second descriptor/draw can keep GENERAL after a shader write. A layout
// equality fast return is not a memory dependency for that prior write.
@@ -268,18 +285,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
info.subresourceRange = {aspect, level, levels, layer, layers};
if (!storage) {
// An X-format shared image carries undefined alpha bytes, exactly like an expanded RGB.
const Bool alphaIsOne = resource->sharedImageAlphaOne || ResolveTextureFormatInfo(
static_cast<TextureInternalFormat>(record.Desc.InternalFormat)).expandRgbToRgba;
const Bool alphaIsOne =
resource->sharedImageAlphaOne ||
ResolveTextureFormatInfo(static_cast<TextureInternalFormat>(record.Desc.InternalFormat))
.expandRgbToRgba;
const auto swizzle = [&](Uint32 channel) {
const auto value = record.Params.Swizzle[channel];
return alphaIsOne && static_cast<TextureSwizzleParam>(value) == TextureSwizzleParam::Alpha
? VK_COMPONENT_SWIZZLE_ONE : WireSwizzle(value);
? VK_COMPONENT_SWIZZLE_ONE
: WireSwizzle(value);
};
info.components = {swizzle(0), swizzle(1), swizzle(2), swizzle(3)};
}
VkImageView view = VK_NULL_HANDLE;
const WireImageViewKey key{root, info.image, m_textureManager->GetTextureImageEpoch(),
info.flags, info.viewType, info.format, info.components, info.subresourceRange};
const WireImageViewKey key{root,
info.image,
m_textureManager->GetTextureImageEpoch(),
info.flags,
info.viewType,
info.format,
info.components,
info.subresourceRange};
const auto& frame = m_frames[m_wireFrameIndex];
const auto cached = frame.wireImageViewCache.find(key);
if (cached != frame.wireImageViewCache.end()) {
@@ -306,26 +332,26 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// session - the DirectGLES arm declines the push in both cases.
using SamplerCsoRecord = std::decay_t<decltype(state.SamplerCsos[0])>;
static const decltype(SamplerCsoRecord::Params) kDefaultSamplerParams{};
const Bool known = !MG_Pipe::MGPipeHandleIsNull(samplerHandle) &&
samplerHandle.Slot < state.SamplerCsos.size();
const Bool known =
!MG_Pipe::MGPipeHandleIsNull(samplerHandle) && samplerHandle.Slot < state.SamplerCsos.size();
const SamplerCsoRecord* sampler = known ? &state.SamplerCsos[samplerHandle.Slot] : nullptr;
const Bool stale = sampler == nullptr || !sampler->Live || sampler->Gen != samplerHandle.Gen;
if (stale) {
MGLOG_E_ONCE("Magma wire: sampler CSO {%u, %u} named by texture unit %u is missing or no longer "
"live; sampling with default parameters", samplerHandle.Slot, samplerHandle.Gen, unit);
"live; sampling with default parameters",
samplerHandle.Slot, samplerHandle.Gen, unit);
}
const Bool forceNearest =
domain == SamplerNumericDomain::SignedInteger || domain == SamplerNumericDomain::UnsignedInteger;
static_assert(kWireSamplerMemoUnits == MG_Pipe::kMGPipeMaxTextureUnits,
"the wire sampler memo has one entry per texture unit");
auto& memo = m_wireSamplerMemo[static_cast<SizeT>(unit)];
const Bool memoHit = !stale && memo.valid && memo.slot == samplerHandle.Slot &&
memo.gen == samplerHandle.Gen && memo.serial == sampler->Serial &&
memo.contextSerial == state.ContextSerial &&
memo.format == record.Desc.InternalFormat && memo.viewLevelCount == levels &&
memo.forceNearest == forceNearest &&
memo.frameBoundary == m_samplerManager->FrameBoundaryCount() &&
memo.destroyEpoch == m_samplerManager->DestroyEpoch();
const Bool memoHit =
!stale && memo.valid && memo.slot == samplerHandle.Slot && memo.gen == samplerHandle.Gen &&
memo.serial == sampler->Serial && memo.contextSerial == state.ContextSerial &&
memo.format == record.Desc.InternalFormat && memo.viewLevelCount == levels &&
memo.forceNearest == forceNearest && memo.frameBoundary == m_samplerManager->FrameBoundaryCount() &&
memo.destroyEpoch == m_samplerManager->DestroyEpoch();
if (memoHit) {
out.sampler = memo.sampler;
} else {
@@ -379,7 +405,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return (properties.bufferFeatures & requiredFeature) == requiredFeature;
}
}
} // namespace
#include "WirePlaceholderImages.inc"
@@ -402,12 +428,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return reflectedFormat != VK_FORMAT_UNDEFINED ? reflectedFormat : resourceFormat;
}
Bool UniformManager::Initialize(VkDevice device, VkPhysicalDevice physicalDevice,
VkBufferManager* bufferManager,
ProgramFactory* programFactory,
VkDeviceSize minUniformBufferOffsetAlignment, Uint32 frameCount,
Uint32 maxBindings, Uint32 setsPerFrame,
VkTextureManager* textureManager, VkSamplerManager* samplerManager) {
Bool UniformManager::Initialize(VkDevice device, VkPhysicalDevice physicalDevice, VkBufferManager* bufferManager,
ProgramFactory* programFactory, VkDeviceSize minUniformBufferOffsetAlignment,
Uint32 frameCount, Uint32 maxBindings, Uint32 setsPerFrame,
VkTextureManager* textureManager, VkSamplerManager* samplerManager) {
Shutdown();
MOBILEGL_ASSERT(device != VK_NULL_HANDLE, "UniformDescriptorBinder::Initialize requires valid VkDevice");
@@ -465,7 +489,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkDescriptorPool initialPool = VK_NULL_HANDLE;
if (!CreateDescriptorPool(m_setsPerFrame, false, initialPool)) {
MGLOG_E_ONCE("UniformDescriptorBinder::Initialize failed: cannot create frame descriptor pool %u",
frameIndex);
frameIndex);
Shutdown();
return false;
}
@@ -491,7 +515,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (auto& frame : m_frames) {
if (m_device != VK_NULL_HANDLE) {
frame.wireImageViewCache.clear();
for (const auto view : frame.wireImageViews) vkDestroyImageView(m_device, view, nullptr);
for (const auto view : frame.wireImageViews)
vkDestroyImageView(m_device, view, nullptr);
frame.wireImageViews.clear();
for (auto& view : frame.texelBufferViews) {
if (view != VK_NULL_HANDLE) {
@@ -515,7 +540,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frame.quietEpochs = 0;
}
m_frames.clear();
for (auto& entry : m_wirePlaceholderImages) DestroyWirePlaceholderImage(entry.second);
for (auto& entry : m_wirePlaceholderImages)
DestroyWirePlaceholderImage(entry.second);
m_wirePlaceholderImages.clear();
m_wirePrivateStoragePlaceholders = 0;
m_wireInvalidStorageImagesBindNull = false;
@@ -542,7 +568,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto& frame = m_frames[frameIndex];
m_wireFrameIndex = frameIndex;
frame.wireImageViewCache.clear();
for (const auto view : frame.wireImageViews) vkDestroyImageView(m_device, view, nullptr);
for (const auto view : frame.wireImageViews)
vkDestroyImageView(m_device, view, nullptr);
frame.wireImageViews.clear();
for (auto& view : frame.texelBufferViews) {
if (view != VK_NULL_HANDLE) {
@@ -552,9 +579,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frame.texelBufferViews.clear();
if (frame.peakAllocatedSetsThisFrame > m_peakDescriptorSetsObserved) {
m_peakDescriptorSetsObserved = frame.peakAllocatedSetsThisFrame;
MGLOG_D(
"UniformDescriptorBinder: new descriptor set peak observed=%u (base setsPerFrame=%u, frame=%u, pools=%zu)",
m_peakDescriptorSetsObserved, m_setsPerFrame, frameIndex, frame.descriptorPools.size());
MGLOG_D("UniformDescriptorBinder: new descriptor set peak observed=%u (base setsPerFrame=%u, frame=%u, "
"pools=%zu)",
m_peakDescriptorSetsObserved, m_setsPerFrame, frameIndex, frame.descriptorPools.size());
}
// The epoch that just ended is the one of the slot begun last - this slot again on the
// drain path, the previous frame's slot on the Present path. Its peak is still unreset.
@@ -725,22 +752,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
Bool UniformManager::ResolveWireTexelBufferDescriptor(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding, Uint32 frameIndex,
Bool storage, VkBufferView& out) {
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, Uint32 frameIndex, Bool storage,
VkBufferView& out) {
out = VK_NULL_HANDLE;
if (binding >= programObj.samplerUniformLocationByBinding.size() ||
binding >= programObj.samplerNumericDomainByBinding.size() || frameIndex >= m_frames.size()) return false;
binding >= programObj.samplerNumericDomainByBinding.size() || frameIndex >= m_frames.size())
return false;
const Int location = programObj.samplerUniformLocationByBinding[binding];
if (location < 0) return false;
const Int unit = program.GetUniformSamplerOrImageUnitIndex(static_cast<Uint>(location));
const auto& state = MG_Pipe::MGPipeApplier();
if (unit < 0 || static_cast<Uint32>(unit) >=
(storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits)) return false;
if (unit < 0 ||
static_cast<Uint32>(unit) >= (storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits))
return false;
if (storage && binding >= programObj.storageImageFormatByBinding.size()) return false;
const VkFormat declaredFormat = storage ? programObj.storageImageFormatByBinding[binding] : VK_FORMAT_UNDEFINED;
const auto placeholder = [&] {
out = AcquireUnboundTexelBufferView(declaredFormat,
programObj.samplerNumericDomainByBinding[binding], storage);
out = AcquireUnboundTexelBufferView(declaredFormat, programObj.samplerNumericDomainByBinding[binding],
storage);
return out != VK_NULL_HANDLE;
};
const auto texture = storage ? state.BoundShaderImages[unit].Res : state.BoundSamplerViews[unit].Texture;
@@ -754,14 +784,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (MG_Pipe::MGPipeHandleIsNull(buffer)) return placeholder();
BufferSlice slice{};
if (!m_bufferManager->AcquireWireSlice(BufferKind::TextureBuffer, buffer, slice) || !slice.IsValid()) return false;
if (!m_bufferManager->AcquireWireSlice(BufferKind::TextureBuffer, buffer, slice) || !slice.IsValid())
return false;
const auto internalFormat = static_cast<TextureInternalFormat>(record.Desc.InternalFormat);
VkFormat format = declaredFormat;
if (storage && format == VK_FORMAT_UNDEFINED && state.BoundShaderImages[unit].InternalFormat != 0)
format = MG_Util::ConvertTextureInternalFormatToVkEnum(MG_Util::ConvertGLEnumToTextureInternalFormat(
state.BoundShaderImages[unit].InternalFormat));
format = MG_Util::ConvertTextureInternalFormatToVkEnum(
MG_Util::ConvertGLEnumToTextureInternalFormat(state.BoundShaderImages[unit].InternalFormat));
if (format == VK_FORMAT_UNDEFINED) format = MG_Util::ConvertTextureInternalFormatToVkEnum(internalFormat);
const auto feature = storage ? VK_FORMAT_FEATURE_STORAGE_TEXEL_BUFFER_BIT : VK_FORMAT_FEATURE_UNIFORM_TEXEL_BUFFER_BIT;
const auto feature =
storage ? VK_FORMAT_FEATURE_STORAGE_TEXEL_BUFFER_BIT : VK_FORMAT_FEATURE_UNIFORM_TEXEL_BUFFER_BIT;
if (format == VK_FORMAT_UNDEFINED || !BufferFormatSupportsFeature(m_physicalDevice, format, feature)) {
// P7 A.5, a decline (rule I (a)). GL's buffer-texture format list is wider than what
// Vulkan mandates for texel buffers - GL_RGB32{F,I,UI} and the 16-bit UNORM members
@@ -785,7 +817,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const VkDeviceSize texelSize = MG_Util::GetSizedInternalFormatSizeInBytes(internalFormat);
VkDeviceSize start = 0, size = 0;
if (texelSize == 0 || !ResolveWireRange(record.Desc.BufOffset, record.Desc.BufSize, slice.size, start, size)) return false;
if (texelSize == 0 || !ResolveWireRange(record.Desc.BufOffset, record.Desc.BufSize, slice.size, start, size))
return false;
size = std::min(size / texelSize, static_cast<VkDeviceSize>(m_wireMaxTexelElements)) * texelSize;
if (size == 0) return false;
if (start > std::numeric_limits<VkDeviceSize>::max() - slice.offset)
@@ -799,8 +832,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Bool writable = storage && state.BoundShaderImages[unit].Access != kMGPipeImageAccessReadOnly;
MGLOG_E_ONCE("ResolveWireTexelBufferDescriptor: binding %u is bound at offset %llu, which is not a "
"multiple of this device's minTexelBufferOffsetAlignment (%llu); the %s view is DECLINED "
"(unaligned-texel-buffer-range)", binding,
static_cast<unsigned long long>(slice.offset + start),
"(unaligned-texel-buffer-range)",
binding, static_cast<unsigned long long>(slice.offset + start),
static_cast<unsigned long long>(m_wireTexelOffsetAlignment),
writable ? "writable" : "read-only");
// A read-only fetch can still be served the way an unbound buffer texture is -
@@ -827,9 +860,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
Bool UniformManager::ResolveTexelBufferDescriptor(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, Uint32 frameIndex,
VkBufferView& outBufferView) {
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 frameIndex, VkBufferView& outBufferView) {
return ResolveWireTexelBufferDescriptor(program, programObj, binding, frameIndex, false, outBufferView);
}
@@ -862,12 +894,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(binding < programObj.storageBlockIndexByBinding.size(),
"ResolveStorageBufferDescriptor: binding %u out of range", binding);
if ((binding >= programObj.storageBlockIndexByBinding.size() ||
binding >= programObj.storageBlockNameByBinding.size()))
binding >= programObj.storageBlockNameByBinding.size()))
WireDescriptorFatal("storage-buffer-reflection-binding");
const Int blockIndex = programObj.storageBlockIndexByBinding[binding];
MOBILEGL_ASSERT(blockIndex >= 0, "ResolveStorageBufferDescriptor: no SSBO block mapped to binding %u",
binding);
MOBILEGL_ASSERT(blockIndex >= 0, "ResolveStorageBufferDescriptor: no SSBO block mapped to binding %u", binding);
if (blockIndex < 0) WireDescriptorFatal("storage-buffer-reflection-block");
// An atomic counter is not an SSBO the application ever declared: glslang lowers every
// atomic_uint onto a synthesized gl_AtomicCounterBlock_<N> storage block, where N is the
@@ -895,12 +926,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// synthesizes one counter block per GL binding, so a counter block is never an instance
// array and `element` is always 0 there; the +element rule stays with the SSBO case.
const GLuint frontendBinding =
isAtomicCounterBlock
? static_cast<GLuint>(atomicCounterBinding)
: program.GetShaderStorageBlockBinding(static_cast<GLuint>(blockIndex)) + element;
isAtomicCounterBlock ? static_cast<GLuint>(atomicCounterBinding)
: program.GetShaderStorageBlockBinding(static_cast<GLuint>(blockIndex)) + element;
const auto& state = MG_Pipe::MGPipeApplier();
const Uint32 cls = isAtomicCounterBlock ? MG_Pipe::kMGPipeShaderBufferClassAtomicCounter
: MG_Pipe::kMGPipeShaderBufferClassShaderStorage;
: MG_Pipe::kMGPipeShaderBufferClassShaderStorage;
if (frontendBinding >= MG_Pipe::kMGPipeMaxBufferBindingPoints)
WireDescriptorFatal("storage-buffer-binding-point");
// The applier retains entries outside the last partial update window. Count
@@ -913,7 +943,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
BufferSlice slice{};
if (!m_bufferManager->AcquireWireSlice(BufferKind::ShaderStorage, range.Res, slice) || !slice.IsValid()) return false;
if (!m_bufferManager->AcquireWireSlice(BufferKind::ShaderStorage, range.Res, slice) || !slice.IsValid())
return false;
VkDeviceSize start = 0, size = 0;
if (!ResolveWireRange(range.Offset, range.Size, slice.size, start, size)) return false;
if (start > std::numeric_limits<VkDeviceSize>::max() - slice.offset)
@@ -939,7 +970,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// MarkWireBufferGpuWritten below already assumes any binding here may be
// written. A correct copy needs a post-dispatch copy-back from the transient
// slice to the unaligned source, ordered with a barrier - and the place to
// record that is the draw/dispatch tail in VulkanRenderer.cpp / WireDraw.inc,
// record that is the draw/dispatch tail in VulkanRenderer.cpp / WireDraw.cpp,
// which belong to wave-2 packages B and C, not to this file. Losing the
// dispatch with a named line in the role log beats returning stale counters
// with no error at all. docs/archive/notes/p7/magma-a.md carries the recipe.
@@ -973,8 +1004,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
Bool UniformManager::ResolveStorageImageDescriptor(VkCommandBuffer commandBuffer,
const MagmaProgramSource& program,
Bool UniformManager::ResolveStorageImageDescriptor(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, Uint32 element,
VkDescriptorImageInfo& outImageInfo) const {
@@ -984,8 +1014,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBufferView UniformManager::AcquireUnboundTexelBufferView(VkFormat declaredFormat,
SamplerNumericDomain numericDomain, Bool storage) {
MOBILEGL_ASSERT(m_bufferManager != nullptr, "AcquireUnboundTexelBufferView: buffer manager is null");
const VkFormatFeatureFlags requiredFeature = storage ? VK_FORMAT_FEATURE_STORAGE_TEXEL_BUFFER_BIT
: VK_FORMAT_FEATURE_UNIFORM_TEXEL_BUFFER_BIT;
const VkFormatFeatureFlags requiredFeature =
storage ? VK_FORMAT_FEATURE_STORAGE_TEXEL_BUFFER_BIT : VK_FORMAT_FEATURE_UNIFORM_TEXEL_BUFFER_BIT;
const VkFormat fallbackFormat = PlaceholderFormatForNumericDomain(numericDomain);
VkFormat format = declaredFormat;
@@ -1054,10 +1084,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return imageAccess != GL_READ_ONLY && imageLevel >= samplerBaseLevel && imageLevel <= samplerMaxLevel;
}
Bool UniformManager::ResolveWireUniformBufferPayload(const MagmaProgramSource& program,
Uint32 blockIndex, Uint32 bindingPoint, UboBindResult& out) const {
if (bindingPoint >= MG_Pipe::kMGPipeMaxBufferBindingPoints)
WireDescriptorFatal("uniform-buffer-binding-point");
Bool UniformManager::ResolveWireUniformBufferPayload(const MagmaProgramSource& program, Uint32 blockIndex,
Uint32 bindingPoint, UboBindResult& out) const {
if (bindingPoint >= MG_Pipe::kMGPipeMaxBufferBindingPoints) WireDescriptorFatal("uniform-buffer-binding-point");
VkDeviceSize blockSize = program.GetUBOSizeAt(blockIndex);
if (blockSize == 0) return false;
if (blockSize > m_wireMaxUniformRange) {
@@ -1077,13 +1106,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static_cast<unsigned long long>(m_wireMaxUniformRange));
blockSize = m_wireMaxUniformRange;
}
const auto& range = MG_Pipe::MGPipeApplier().BoundShaderBuffers[
MG_Pipe::kMGPipeShaderBufferClassUniform][bindingPoint];
const auto& range =
MG_Pipe::MGPipeApplier().BoundShaderBuffers[MG_Pipe::kMGPipeShaderBufferClassUniform][bindingPoint];
BufferSlice source{};
VkDeviceSize start = 0, available = 0;
const Bool bound = !MG_Pipe::MGPipeHandleIsNull(range.Res);
if (bound) {
if (!m_bufferManager->AcquireWireSlice(BufferKind::Uniform, range.Res, source) || !source.IsValid()) return false;
if (!m_bufferManager->AcquireWireSlice(BufferKind::Uniform, range.Res, source) || !source.IsValid())
return false;
if (!ResolveWireRange(range.Offset, range.Size, source.size, start, available)) return false;
if (start > std::numeric_limits<VkDeviceSize>::max() - source.offset)
WireDescriptorFatal("uniform-buffer-offset-overflow");
@@ -1107,7 +1137,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
zero.assign(static_cast<SizeT>(blockSize), 0);
BufferSlice padded{};
if (!m_bufferManager->UploadTransient(BufferKind::Uniform, m_wireFrameIndex, zero.data(), blockSize,
std::max<VkDeviceSize>(4, m_minDynamicOffsetAlignment), padded) || !padded.IsValid()) return false;
std::max<VkDeviceSize>(4, m_minDynamicOffsetAlignment), padded) ||
!padded.IsValid())
return false;
if (bound && copied != 0) {
// P7 A.1 retires `uniform-buffer-byte-tail`. The window is word-aligned at both
// ends on nearly every bind - glBindBufferRange already forces the offset onto
@@ -1121,10 +1153,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// slice and one extra region copy, and nothing outside [start, start + copied)
// reaches the block, which is what makes the padded zeros the visible tail.
const Bool wordWindow = ((start | copied) & 3u) == 0;
if (!(wordWindow
? m_bufferManager->CopyWireBufferRangeToSlice(range.Res, start, copied, padded)
: m_bufferManager->CopyWireBufferSubWordRangeToSlice(range.Res, start, copied,
m_wireFrameIndex, padded, 0)))
if (!(wordWindow ? m_bufferManager->CopyWireBufferRangeToSlice(range.Res, start, copied, padded)
: m_bufferManager->CopyWireBufferSubWordRangeToSlice(range.Res, start, copied,
m_wireFrameIndex, padded, 0)))
return false;
}
if (padded.offset > std::numeric_limits<Uint32>::max()) {
@@ -1184,9 +1215,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto arrayIt = programObj.arrayedUniformBlockIndicesByBinding.find(binding);
const Bool elementValid = arrayIt != programObj.arrayedUniformBlockIndicesByBinding.end() &&
arrayElement < arrayIt->second.size();
MOBILEGL_ASSERT(elementValid,
"ResolveUniformBufferPayload: UBO binding %u has no array element %u", binding,
arrayElement);
MOBILEGL_ASSERT(elementValid, "ResolveUniformBufferPayload: UBO binding %u has no array element %u",
binding, arrayElement);
if (!elementValid) {
return false;
}
@@ -1258,7 +1288,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VkResult result = vkCreateDescriptorPool(m_device, &poolInfo, nullptr, &outPool);
if (result != VK_SUCCESS) {
MGLOG_E_ONCE("UniformDescriptorBinder::CreateDescriptorPool failed: vkCreateDescriptorPool returned %d",
result);
result);
return false;
}
return true;
@@ -1292,8 +1322,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// rather than failing the draw - halving down to the base size, where the old policy lived.
while (!CreateDescriptorPool(grownMaxSets, updateAfterBind, grownPool)) {
if (grownMaxSets <= m_setsPerFrame) {
MGLOG_E_ONCE("UniformDescriptorBinder::GrowFrameDescriptorPool failed: cannot create grown pool (%u sets)",
grownMaxSets);
MGLOG_E_ONCE(
"UniformDescriptorBinder::GrowFrameDescriptorPool failed: cannot create grown pool (%u sets)",
grownMaxSets);
return false;
}
grownMaxSets = std::max(m_setsPerFrame, grownMaxSets / 2);
@@ -1310,18 +1341,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
VkResult UniformManager::AllocateDescriptorSetsFromActivePool(Uint32 frameIndex, const ProgramFactory::VkProgramObject& programObj, VkDescriptorSet& outDescriptorSet) {
VkResult UniformManager::AllocateDescriptorSetsFromActivePool(Uint32 frameIndex,
const ProgramFactory::VkProgramObject& programObj,
VkDescriptorSet& outDescriptorSet) {
auto& frame = m_frames[frameIndex];
const Bool updateAfterBind = programObj.usesUpdateAfterBind;
if (frame.activeDescriptorPoolIndex >= frame.descriptorPools.size() ||
frame.descriptorPools[frame.activeDescriptorPoolIndex].updateAfterBind != updateAfterBind ||
frame.descriptorPools[frame.activeDescriptorPoolIndex].allocatedSets >=
frame.descriptorPools[frame.activeDescriptorPoolIndex].maxSets) {
const auto availableBucket = std::find_if(
frame.descriptorPools.begin(), frame.descriptorPools.end(),
[updateAfterBind](const DescriptorPoolBucket& candidate) {
return candidate.updateAfterBind == updateAfterBind && candidate.allocatedSets < candidate.maxSets;
});
const auto availableBucket = std::find_if(frame.descriptorPools.begin(), frame.descriptorPools.end(),
[updateAfterBind](const DescriptorPoolBucket& candidate) {
return candidate.updateAfterBind == updateAfterBind &&
candidate.allocatedSets < candidate.maxSets;
});
if (availableBucket == frame.descriptorPools.end()) {
outDescriptorSet = VK_NULL_HANDLE;
return VK_ERROR_OUT_OF_POOL_MEMORY;
@@ -1347,8 +1380,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return result;
}
VkResult UniformManager::AcquireDescriptorSet(Uint32 frameIndex,
const ProgramFactory::VkProgramObject& programObj,
VkResult UniformManager::AcquireDescriptorSet(Uint32 frameIndex, const ProgramFactory::VkProgramObject& programObj,
VkDescriptorSet& outDescriptorSet) {
auto& frame = m_frames[frameIndex];
auto& cache = frame.descriptorSetCacheByLayout[programObj.descriptorSetLayout];
@@ -1376,22 +1408,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
++frame.allocatedSetsThisFrame;
frame.peakAllocatedSetsThisFrame =
std::max(frame.peakAllocatedSetsThisFrame, frame.allocatedSetsThisFrame);
frame.peakAllocatedSetsThisFrame = std::max(frame.peakAllocatedSetsThisFrame, frame.allocatedSetsThisFrame);
return VK_SUCCESS;
}
Bool UniformManager::ResolveDynamicUboDescriptor(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, Uint32 arrayElement, Uint32 frameIndex,
VkBuffer& outBuffer, VkDeviceSize& outRange,
Uint32& outDynamicOffset) {
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 arrayElement, Uint32 frameIndex, VkBuffer& outBuffer,
VkDeviceSize& outRange, Uint32& outDynamicOffset) {
UboBindResult ubo{};
const Bool hasPayload = ResolveUniformBufferPayload(program, programObj, binding, arrayElement, ubo);
if (!hasPayload) return false;
MOBILEGL_ASSERT(hasPayload && (ubo.directBindable || (ubo.payload != nullptr && ubo.payloadSize > 0)),
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: missing UBO payload on binding %u element %u",
binding, arrayElement);
MOBILEGL_ASSERT(
hasPayload && (ubo.directBindable || (ubo.payload != nullptr && ubo.payloadSize > 0)),
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: missing UBO payload on binding %u element %u",
binding, arrayElement);
if (ubo.directBindable) {
// Zero-copy: bind the app's resident VkBuffer directly, no per-draw memcpy.
outBuffer = ubo.buffer;
@@ -1421,7 +1452,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!m_bufferManager->UploadTransient(BufferKind::Uniform, frameIndex, ubo.payload, ubo.payloadSize,
m_minDynamicOffsetAlignment, slice)) {
MOBILEGL_ASSERT(false,
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: UBO upload failed on binding %u element %u",
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: UBO upload failed on binding "
"%u element %u",
binding, arrayElement);
return false;
}
@@ -1438,7 +1470,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (isGlobalUbo) {
m_globalUboMemo[m_globalUboMemoNext] =
GlobalUboSliceMemo{uboProgramLifetimeId, uboFrameSerial, uboContentVersion,
slice.buffer, slice.offset, static_cast<VkDeviceSize>(ubo.payloadSize)};
slice.buffer, slice.offset, static_cast<VkDeviceSize>(ubo.payloadSize)};
m_globalUboMemoNext = (m_globalUboMemoNext + 1) % kGlobalUboMemoSize;
}
return true;
@@ -1450,9 +1482,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Skip the driver call when this exact binding is already live on the
// command buffer (see the bind-dedup shadow in the header).
const Uint32 offsetCount = static_cast<Uint32>(dynamicOffsets.size());
Bool identicalBind = m_lastBindValid && m_lastBindSet == descriptorSet &&
m_lastBindLayout == pipelineLayout && m_lastBindPoint == bindPoint &&
m_lastBindOffsetCount == offsetCount && offsetCount <= kMaxShadowedDynamicOffsets;
Bool identicalBind = m_lastBindValid && m_lastBindSet == descriptorSet && m_lastBindLayout == pipelineLayout &&
m_lastBindPoint == bindPoint && m_lastBindOffsetCount == offsetCount &&
offsetCount <= kMaxShadowedDynamicOffsets;
if (identicalBind) {
for (Uint32 i = 0; i < offsetCount; ++i) {
if (m_lastBindOffsets[i] != dynamicOffsets[i]) {
@@ -1462,8 +1494,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
if (!identicalBind) {
vkCmdBindDescriptorSets(commandBuffer, bindPoint, pipelineLayout, 0, 1,
&descriptorSet, offsetCount, dynamicOffsets.data());
vkCmdBindDescriptorSets(commandBuffer, bindPoint, pipelineLayout, 0, 1, &descriptorSet, offsetCount,
dynamicOffsets.data());
if (offsetCount <= kMaxShadowedDynamicOffsets) {
m_lastBindValid = true;
m_lastBindSet = descriptorSet;
@@ -1478,7 +1510,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
Bool UniformManager::PrepareWireTextureResources(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj) {
const ProgramFactory::VkProgramObject& programObj) {
if (programObj.declinedDescriptors || m_textureManager == nullptr) {
MGL_WIRE_DECLINE_AT(DescriptorDeclined, "the program carries a descriptor MobileGL could not resolve");
return false;
@@ -1504,12 +1536,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const Int unit = program.GetUniformSamplerOrImageUnitIndex(static_cast<Uint>(location));
if (unit < 0 || static_cast<Uint32>(unit) >=
(storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits)) {
(storage ? MG_Pipe::kMGPipeMaxImageUnits : MG_Pipe::kMGPipeMaxTextureUnits)) {
MGL_WIRE_DECLINE_AT(SamplerUnitRange, "sampler/image unit %d is out of range", unit);
return false;
}
const auto handle = storage ? state.BoundShaderImages[unit].Res
: state.BoundSamplerViews[unit].Texture;
const auto handle =
storage ? state.BoundShaderImages[unit].Res : state.BoundSamplerViews[unit].Texture;
// Null is the wire's unbound/incomplete binding. Its native
// placeholder needs no resource upload or frontend allocation.
if (MG_Pipe::MGPipeHandleIsNull(handle)) continue;
@@ -1530,11 +1562,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
Bool UniformManager::BindProgramUniformBuffers(VkCommandBuffer commandBuffer,
const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 frameIndex,
VkPipelineBindPoint bindPoint) {
Bool UniformManager::BindProgramUniformBuffers(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 frameIndex,
VkPipelineBindPoint bindPoint) {
// This program has a descriptor MobileGL could not resolve (see
// VkProgramObject::declinedDescriptors). Refusing here is the whole of the decline: the
// binding is still declared in the layout, so the pipeline is consistent with the shader
@@ -1548,7 +1578,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
auto& frame = m_frames[frameIndex];
if (frame.descriptorPools.empty()) {
MGLOG_E_ONCE("UniformDescriptorBinder::BindProgramUniformBuffers failed: frame descriptor pools are invalid");
MGLOG_E_ONCE(
"UniformDescriptorBinder::BindProgramUniformBuffers failed: frame descriptor pools are invalid");
return false;
}
if (frame.activeDescriptorPoolIndex >= frame.descriptorPools.size()) {
@@ -1610,7 +1641,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
texelBufferViews.reserve(m_maxBindings);
dynamicOffsets.reserve(programObj.dynamicBindings.size() + uboArrayExtra);
// Iterate only the bindings this program declares. The old walk covered all 256 slots of
// bindingKinds on every draw to find the 1-8 a real program uses.
for (const Uint32 binding : programObj.activeBindings) {
@@ -1657,9 +1687,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBufferView bufferView = VK_NULL_HANDLE;
if (!ResolveTexelBufferDescriptor(program, programObj, binding, frameIndex, bufferView) ||
bufferView == VK_NULL_HANDLE) {
MGLOG_E_ONCE(
"UniformDescriptorBinder::BindProgramUniformBuffers failed: texture buffer binding %u has no valid descriptor",
binding);
MGLOG_E_ONCE("UniformDescriptorBinder::BindProgramUniformBuffers failed: texture buffer binding %u "
"has no valid descriptor",
binding);
return false;
}
@@ -1676,8 +1706,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!ResolveStorageTexelBufferDescriptor(program, programObj, binding, frameIndex, bufferView) ||
bufferView == VK_NULL_HANDLE) {
MGLOG_E_ONCE("UniformDescriptorBinder::BindProgramUniformBuffers failed: image buffer binding %u "
"has no valid descriptor",
binding);
"has no valid descriptor",
binding);
return false;
}
@@ -1746,8 +1776,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Bool hasImage = ResolveWireImageDescriptor(commandBuffer, program, programObj, binding,
element, false, imageInfo, &sampledResource);
// A placeholder (an empty unit) has no resource to re-check: full resolve.
if (sampledResource == nullptr) rebindEligible = false;
else rebindResources.push_back(sampledResource);
if (sampledResource == nullptr)
rebindEligible = false;
else
rebindResources.push_back(sampledResource);
if (!hasImage) {
MGLOG_E_ONCE(
"UniformDescriptorBinder::BindProgramUniformBuffers failed: sampler binding %u element %u "
@@ -1783,9 +1815,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// bypass and invalidate the cache.
Uint64 signature = 0xcbf29ce484222325ULL;
{
const auto mix64 = [&signature](Uint64 word) {
signature = (signature ^ word) * 0x100000001b3ULL;
};
const auto mix64 = [&signature](Uint64 word) { signature = (signature ^ word) * 0x100000001b3ULL; };
// The hashed descriptor payloads (VkDescriptorBufferInfo=24B,
// VkDescriptorImageInfo=24B, VkBufferView=8B) are all 8-byte-multiple sized
// and value-initialized (padding is zero), so hashing 64-bit words at a time
@@ -1807,8 +1837,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the memo miss on every draw.
mix64(m_bufferManager->GetWireStoreDestroyEpoch());
for (const auto& write : writes) {
mix64((static_cast<Uint64>(write.dstBinding) << 40) ^
(static_cast<Uint64>(write.descriptorType) << 8) ^
mix64((static_cast<Uint64>(write.dstBinding) << 40) ^ (static_cast<Uint64>(write.descriptorType) << 8) ^
static_cast<Uint64>(write.descriptorCount));
}
mixWords(bufferInfos.data(), bufferInfos.size() * sizeof(VkDescriptorBufferInfo));
@@ -1828,8 +1857,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} else {
VkResult allocResult = AcquireDescriptorSet(frameIndex, programObj, descriptorSet);
if (allocResult != VK_SUCCESS || descriptorSet == VK_NULL_HANDLE) {
MGLOG_E_ONCE("UniformDescriptorBinder::BindProgramUniformBuffers failed: descriptor set acquire returned %d",
allocResult);
MGLOG_E_ONCE(
"UniformDescriptorBinder::BindProgramUniformBuffers failed: descriptor set acquire returned %d",
allocResult);
return false;
}
for (auto& write : writes) {
@@ -1838,19 +1868,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!writes.empty()) {
vkUpdateDescriptorSets(m_device, static_cast<Uint32>(writes.size()), writes.data(), 0, nullptr);
}
m_descriptorReuseMemo[m_descriptorReuseMemoNext] =
DescriptorReuseEntry{signature, descriptorSet, true};
m_descriptorReuseMemo[m_descriptorReuseMemoNext] = DescriptorReuseEntry{signature, descriptorSet, true};
m_descriptorReuseMemoNext = (m_descriptorReuseMemoNext + 1) % kDescriptorReuseMemoSize;
}
BindDescriptorSetDeduped(commandBuffer, bindPoint, programObj.pipelineLayout, descriptorSet,
dynamicOffsets);
BindDescriptorSetDeduped(commandBuffer, bindPoint, programObj.pipelineLayout, descriptorSet, dynamicOffsets);
// Stamp what this full resolve depended on, for TryRebindWireDescriptorSet. Taken after
// the walk: the walk's own syncs and barriers are part of what it proved.
auto& rebind = m_wireDescriptorRebind;
rebind.valid = rebindEligible && descriptorSet != VK_NULL_HANDLE &&
WireImageWriteEpoch() == m_wireImageBarrierEpoch;
rebind.valid =
rebindEligible && descriptorSet != VK_NULL_HANDLE && WireImageWriteEpoch() == m_wireImageBarrierEpoch;
if (rebind.valid) {
const auto& st = MG_Pipe::MGPipeApplier();
rebind.frameIndex = frameIndex;
@@ -1891,8 +1919,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return false;
const auto& st = MG_Pipe::MGPipeApplier();
if (rebind.contextSerial != st.ContextSerial || rebind.samplerViewsSerial != st.SamplerViewsSerial ||
rebind.samplerStatesSerial != st.SamplerStatesSerial || rebind.shaderImagesSerial != st.ShaderImagesSerial ||
rebind.shaderBuffersSerial != st.ShaderBuffersSerial || rebind.textureShutterSerial != st.TextureShutterSerial)
rebind.samplerStatesSerial != st.SamplerStatesSerial ||
rebind.shaderImagesSerial != st.ShaderImagesSerial ||
rebind.shaderBuffersSerial != st.ShaderBuffersSerial ||
rebind.textureShutterSerial != st.TextureShutterSerial)
return false;
if (rebind.storeDestroyEpoch != m_bufferManager->GetWireStoreDestroyEpoch() ||
rebind.textureImageEpoch != m_textureManager->GetTextureImageEpoch() ||
@@ -23,7 +23,7 @@ namespace MobileGL::MG_State::GLState {
class ITextureObject;
class ProgramObject;
class SamplerObject;
}
} // namespace MobileGL::MG_State::GLState
namespace MobileGL::MG_Backend::DirectVulkan {
class UniformManager {
@@ -40,10 +40,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// an unbound texel-buffer binding has to be built from a format the DEVICE accepts as a
// texel buffer, and there is no other route to that answer from here.
Bool Initialize(VkDevice device, VkPhysicalDevice physicalDevice, VkBufferManager* bufferManager,
ProgramFactory* programFactory,
VkDeviceSize minUniformBufferOffsetAlignment, Uint32 frameCount,
Uint32 maxBindings = 16, Uint32 setsPerFrame = 64,
VkTextureManager* textureManager = nullptr, VkSamplerManager* samplerManager = nullptr);
ProgramFactory* programFactory, VkDeviceSize minUniformBufferOffsetAlignment, Uint32 frameCount,
Uint32 maxBindings = 16, Uint32 setsPerFrame = 64, VkTextureManager* textureManager = nullptr,
VkSamplerManager* samplerManager = nullptr);
void Shutdown();
void BeginFrame(Uint32 frameIndex);
@@ -95,17 +94,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 textureLifetimeId = 0;
Uint64 samplerLifetimeId = 0;
};
static Bool SamplerOverlapsWritableImageSubresource(Int samplerBaseLevel, Int samplerMaxLevel,
GLint imageLevel, GLenum imageAccess);
static Bool SamplerOverlapsWritableImageSubresource(Int samplerBaseLevel, Int samplerMaxLevel, GLint imageLevel,
GLenum imageAccess);
// Resolve lazy texture uploads/promotions before the caller captures the
// command buffer passed to BindProgramUniformBuffers. Preparation may
// submit older work, but descriptor recording must never rotate it.
Bool PrepareWireTextureResources(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj);
Bool BindProgramUniformBuffers(VkCommandBuffer commandBuffer,
const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 frameIndex,
const ProgramFactory::VkProgramObject& programObj);
Bool BindProgramUniformBuffers(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 frameIndex,
VkPipelineBindPoint bindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS);
// Pure format-policy helper kept public for host regression tests. Formatted storage
@@ -154,7 +151,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
DescriptorPoolCensus GetDescriptorPoolCensus() const;
private:
// A cached descriptor set together with the pool it was allocated from, so a
// layout-destroyed purge can vkFreeDescriptorSets it back and credit the
// owning bucket's accounting.
@@ -188,8 +184,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
components.r == other.components.r && components.g == other.components.g &&
components.b == other.components.b && components.a == other.components.a &&
range.aspectMask == other.range.aspectMask && range.baseMipLevel == other.range.baseMipLevel &&
range.levelCount == other.range.levelCount && range.baseArrayLayer == other.range.baseArrayLayer &&
range.layerCount == other.range.layerCount;
range.levelCount == other.range.levelCount &&
range.baseArrayLayer == other.range.baseArrayLayer && range.layerCount == other.range.layerCount;
}
};
@@ -222,8 +218,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// (GL 4.6 core 8.26); UNDEFINED keeps the numeric-domain R32 default. `unit` is the image
// unit of a storage binding (what its private placeholder is private to); unused for a sampler.
Bool ResolveWirePlaceholderImage(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding, Bool storage,
VkDescriptorImageInfo& out, VkFormat boundStorageFormat = VK_FORMAT_UNDEFINED, Uint32 unit = ~0u) const;
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Bool storage, VkDescriptorImageInfo& out,
VkFormat boundStorageFormat = VK_FORMAT_UNDEFINED, Uint32 unit = ~0u) const;
void DestroyWirePlaceholderImage(WirePlaceholderImage& image) const;
// descriptorSetCacheByLayout is the per-frame set cache: BeginFrame rewinds every cursor, so
@@ -264,9 +261,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBufferView AcquireUnboundTexelBufferView(VkFormat declaredFormat, SamplerNumericDomain numericDomain,
Bool storage);
Bool ResolveWireImageDescriptor(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 element, Bool storage, VkDescriptorImageInfo& out,
VkTextureManager::TextureResource** outResource = nullptr) const;
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 element, Bool storage, VkDescriptorImageInfo& out,
VkTextureManager::TextureResource** outResource = nullptr) const;
// P14: THE WIRE DESCRIPTOR REBIND. A draw whose image descriptors cannot have changed since
// the last full resolve of the same program in the same frame - same applier context, same
// unit / sampler / image / buffer-binding serials, same texture shutter, no texture image
@@ -301,8 +298,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const ProgramFactory::VkProgramObject& programObj, Uint32 frameIndex,
VkPipelineBindPoint bindPoint);
Bool ResolveWireTexelBufferDescriptor(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 frameIndex, Bool storage, VkBufferView& out);
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 frameIndex, Bool storage, VkBufferView& out);
Uint32 m_wireFrameIndex = 0;
VkDeviceSize m_wireStorageOffsetAlignment = 1;
VkDeviceSize m_wireTexelOffsetAlignment = 1;
@@ -328,8 +325,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 element, VkDescriptorBufferInfo& outBufferInfo) const;
// `element` indexes an image ARRAY inside one binding; each element carries its own
// independently assigned GL image unit.
Bool ResolveStorageImageDescriptor(VkCommandBuffer commandBuffer,
const MagmaProgramSource& program,
Bool ResolveStorageImageDescriptor(VkCommandBuffer commandBuffer, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 element, VkDescriptorImageInfo& outImageInfo) const;
// Result of resolving a UBO binding: either a zero-copy direct bind to the app's resident
@@ -342,8 +338,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const void* payload = nullptr; // fallback UploadTransient path
VkDeviceSize payloadSize = 0;
};
Bool ResolveWireUniformBufferPayload(const MagmaProgramSource& program, Uint32 blockIndex,
Uint32 bindingPoint, UboBindResult& out) const;
Bool ResolveWireUniformBufferPayload(const MagmaProgramSource& program, Uint32 blockIndex, Uint32 bindingPoint,
UboBindResult& out) const;
Bool ResolveUniformBufferPayload(const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 arrayElement, UboBindResult& out) const;
@@ -386,10 +382,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void TrimFrameDescriptorPools(FrameResources& frame, Uint32 frameIndex);
// Drops every handle this class holds to a descriptor set outside the per-layout caches.
void InvalidateDescriptorSetMemos();
VkResult AllocateDescriptorSetsFromActivePool(
Uint32 frameIndex, const ProgramFactory::VkProgramObject& programObj, VkDescriptorSet& outDescriptorSet);
VkResult AcquireDescriptorSet(Uint32 frameIndex,
const ProgramFactory::VkProgramObject& programObj,
VkResult AllocateDescriptorSetsFromActivePool(Uint32 frameIndex,
const ProgramFactory::VkProgramObject& programObj,
VkDescriptorSet& outDescriptorSet);
VkResult AcquireDescriptorSet(Uint32 frameIndex, const ProgramFactory::VkProgramObject& programObj,
VkDescriptorSet& outDescriptorSet);
VkDevice m_device = VK_NULL_HANDLE;
@@ -25,8 +25,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_state.pVertexAttributeDescriptions = nullptr;
}
VertexInputStateBuilder& VertexInputStateBuilder::AddBinding(
Uint32 binding, Uint32 stride, VkVertexInputRate inputRate) {
VertexInputStateBuilder& VertexInputStateBuilder::AddBinding(Uint32 binding, Uint32 stride,
VkVertexInputRate inputRate) {
VkVertexInputBindingDescription desc{};
desc.binding = binding;
desc.stride = stride;
@@ -35,8 +35,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return *this;
}
VertexInputStateBuilder& VertexInputStateBuilder::AddAttribute(
Uint32 location, Uint32 binding, VkFormat format, Uint32 offset) {
VertexInputStateBuilder& VertexInputStateBuilder::AddAttribute(Uint32 location, Uint32 binding, VkFormat format,
Uint32 offset) {
VkVertexInputAttributeDescription desc{};
desc.location = location;
desc.binding = binding;
@@ -15,8 +15,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VertexInputStateBuilder();
void Reset();
VertexInputStateBuilder& AddBinding(
Uint32 binding, Uint32 stride, VkVertexInputRate inputRate = VK_VERTEX_INPUT_RATE_VERTEX);
VertexInputStateBuilder& AddBinding(Uint32 binding, Uint32 stride,
VkVertexInputRate inputRate = VK_VERTEX_INPUT_RATE_VERTEX);
VertexInputStateBuilder& AddAttribute(Uint32 location, Uint32 binding, VkFormat format, Uint32 offset);
const VkPipelineVertexInputStateCreateInfo& Build();
@@ -37,8 +37,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} // namespace
const VertexInputStateFactory::BackendVertexInputState* VertexInputStateFactory::ResolveWireVertexInput(
const MG_Pipe::MGPipeVertexElementsRecord& elements, const MG_Pipe::MGPipeApplierState& state,
Uint32 activeMask) const {
const MG_Pipe::MGPipeVertexElementsRecord& elements, const MG_Pipe::MGPipeApplierState& state,
Uint32 activeMask) const {
auto* memo = static_cast<WireVertexInputMemo*>(elements.BackendMemo.get());
if (memo == nullptr) {
auto created = std::make_shared<WireVertexInputMemo>();
@@ -77,8 +77,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
Bool VertexInputStateFactory::BuildWireVertexInput(const MG_Pipe::MGPipeVertexElementsRecord& elements,
const MG_Pipe::MGPipeApplierState& state, Uint32 activeMask, BackendVertexInputState& out) const {
// The caller's state is a per-draw scratch (WireDraw.inc): reset it without freeing.
const MG_Pipe::MGPipeApplierState& state, Uint32 activeMask,
BackendVertexInputState& out) const {
// The caller's state is a per-draw scratch (WireDraw.cpp): reset it without freeing.
out.ResetKeepingCapacity();
for (Uint32 location = 0; location < elements.AttributeCount; ++location) {
const auto& attr = elements.Attributes[location];
@@ -117,11 +118,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// a Fatal for a number no application can reach on purpose. Recorded in
// docs/archive/notes/p7/magma-c.md rather than silently reconciled.
const auto describe = [&](const char* reason, VkFormat format) {
MGLOG_E_ONCE("Magma wire vertex layout: %s location=%u type=%u size=%u normalized=%u integer=%u long=%u bgra=%u stride=%d offset=%llu format=%d bufferWindow=%u+%u activeMask=0x%x",
reason, location, attr.Type, static_cast<Uint32>(attr.Size), static_cast<Uint32>(attr.Normalized),
static_cast<Uint32>(attr.IsInteger), static_cast<Uint32>(attr.IsLong), static_cast<Uint32>(attr.IsBgra),
attr.Stride, static_cast<unsigned long long>(attr.Offset), static_cast<Int>(format),
state.VertexBufferStart, state.VertexBufferCount, activeMask);
MGLOG_E_ONCE("Magma wire vertex layout: %s location=%u type=%u size=%u normalized=%u integer=%u "
"long=%u bgra=%u stride=%d offset=%llu format=%d bufferWindow=%u+%u activeMask=0x%x",
reason, location, attr.Type, static_cast<Uint32>(attr.Size),
static_cast<Uint32>(attr.Normalized), static_cast<Uint32>(attr.IsInteger),
static_cast<Uint32>(attr.IsLong), static_cast<Uint32>(attr.IsBgra), attr.Stride,
static_cast<unsigned long long>(attr.Offset), static_cast<Int>(format),
state.VertexBufferStart, state.VertexBufferCount, activeMask);
};
// The protocol verdict: the caller turns a false into the named Fatal.
const auto reject = [&](const char* reason, VkFormat format = VK_FORMAT_UNDEFINED) {
@@ -138,13 +141,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
};
// set_vertex_buffers is flattened PER ATTRIBUTE (VertexInputEmit.h), not
// indexed by the original ARB binding point in attr.BindingIndex.
if (location < state.VertexBufferStart ||
location - state.VertexBufferStart >= state.VertexBufferCount) return reject("buffer-window");
if (location < state.VertexBufferStart || location - state.VertexBufferStart >= state.VertexBufferCount)
return reject("buffer-window");
const auto& buffer = state.VertexBuffers[location];
const auto type = static_cast<DataType>(attr.Type);
if (attr.Stride < 0 || attr.Size < 1 || attr.Size > 4) { maskOut("attribute-shape"); continue; }
VkFormat format = ToVkVertexFormat(type, attr.Size, attr.Normalized, attr.IsInteger,
attr.IsBgra, attr.IsLong);
if (attr.Stride < 0 || attr.Size < 1 || attr.Size > 4) {
maskOut("attribute-shape");
continue;
}
VkFormat format =
ToVkVertexFormat(type, attr.Size, attr.Normalized, attr.IsInteger, attr.IsBgra, attr.IsLong);
auto conversion = VertexStreamConversion::None;
if (format == VK_FORMAT_UNDEFINED && type == DataType::Float64) {
const auto* backend = MG_Backend::ApplyRoleBackend();
@@ -168,29 +174,45 @@ namespace MobileGL::MG_Backend::DirectVulkan {
format = ToFloat32VertexFormat(attr.Size);
conversion = VertexStreamConversion::ScaledIntegerToFloat32;
}
if (format == VK_FORMAT_UNDEFINED) { maskOut("format-map"); continue; }
if (format == VK_FORMAT_UNDEFINED) {
maskOut("format-map");
continue;
}
if (!SupportsVertexBufferFormat(format)) {
if (!IsScaledIntegerVertexFormat(format)) { maskOut("native-format-feature", format); continue; }
if (!IsScaledIntegerVertexFormat(format)) {
maskOut("native-format-feature", format);
continue;
}
format = ToFloat32VertexFormat(attr.Size);
conversion = VertexStreamConversion::ScaledIntegerToFloat32;
if (!SupportsVertexBufferFormat(format)) { maskOut("converted-format-feature", format); continue; }
if (!SupportsVertexBufferFormat(format)) {
maskOut("converted-format-feature", format);
continue;
}
}
const SizeT elementSize = GetAttributeByteSize(type, attr.Size, attr.IsBgra);
if (!elementSize) { maskOut("element-size", format); continue; }
if (!elementSize) {
maskOut("element-size", format);
continue;
}
const Uint64 offset = attr.Offset + buffer.Offset;
if (offset < attr.Offset) { maskOut("offset-overflow", format); continue; }
if (offset < attr.Offset) {
maskOut("offset-overflow", format);
continue;
}
const SizeT alignment = (type == DataType::Int2101010Rev || type == DataType::Uint2101010Rev)
? elementSize : GetComponentSize(type);
? elementSize
: GetComponentSize(type);
if (conversion == VertexStreamConversion::None && alignment > 1 &&
(offset % alignment || static_cast<Uint32>(attr.Stride) % alignment))
conversion = VertexStreamConversion::Repack;
Uint32 stride = static_cast<Uint32>(attr.Stride);
if (stride && conversion != VertexStreamConversion::None)
stride = conversion == VertexStreamConversion::Repack ? static_cast<Uint32>(elementSize)
: static_cast<Uint32>(attr.Size) * sizeof(Float);
: static_cast<Uint32>(attr.Size) * sizeof(Float);
const Uint32 binding = static_cast<Uint32>(out.bindings.size());
out.bindings.push_back({binding, stride,
buffer.Divisor ? VK_VERTEX_INPUT_RATE_INSTANCE : VK_VERTEX_INPUT_RATE_VERTEX});
out.bindings.push_back(
{binding, stride, buffer.Divisor ? VK_VERTEX_INPUT_RATE_INSTANCE : VK_VERTEX_INPUT_RATE_VERTEX});
out.attributes.push_back({location, binding, format, 0});
out.bindingAttributeLocations.push_back(location);
out.bindingBaseOffsets.push_back(static_cast<SizeT>(offset));
@@ -202,8 +224,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// resolved binding/attribute/divisor values, including a legal zero stride.
Uint64 hash = XXH64(out.bindings.data(), out.bindings.size() * sizeof(out.bindings[0]), 0);
hash = XXH64(out.attributes.data(), out.attributes.size() * sizeof(out.attributes[0]), hash);
hash = XXH64(out.bindingDivisors.data(),
out.bindingDivisors.size() * sizeof(out.bindingDivisors[0]), hash);
hash = XXH64(out.bindingDivisors.data(), out.bindingDivisors.size() * sizeof(out.bindingDivisors[0]), hash);
// THE MASK IS PART OF THE LAYOUT NOW, for the same reason the monolith entry's hash
// carries it (the XXH64_update over unsupportedAttribMask in GetOrCreateVertexInputState
// above). Before this package the mask was always zero here, so leaving it out of the
@@ -271,60 +292,84 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// P5c (hd, CONTRACT-P5C §3.7): see the narrowFloat64Arrays site above.
MG_Config::Transport != MG_Config::TransportMode::Monolith
? (MG_Remote::Server::ServerLoopInstance().Backend() == nullptr ||
!MG_Remote::Server::ServerLoopInstance().Backend()
!MG_Remote::Server::ServerLoopInstance()
.Backend()
->GetDynamicParameters()
.SupportsFloat64VertexAttributes)
:
#endif
MG_Backend::pActiveBackendObject == nullptr ||
!MG_Backend::pActiveBackendObject->GetDynamicParameters().SupportsFloat64VertexAttributes) {
MG_Backend::pActiveBackendObject == nullptr ||
!MG_Backend::pActiveBackendObject->GetDynamicParameters().SupportsFloat64VertexAttributes) {
return VK_FORMAT_UNDEFINED;
}
if (!isLong || isInteger || normalized) return VK_FORMAT_UNDEFINED;
switch (size) {
case 1: return VK_FORMAT_R32G32_UINT;
case 2: return VK_FORMAT_R32G32B32A32_UINT;
case 1:
return VK_FORMAT_R32G32_UINT;
case 2:
return VK_FORMAT_R32G32B32A32_UINT;
// A dvec3/dvec4 input is 6/8 uint32 components: no single VkFormat, and GL spreads it
// over two attribute locations, which the location-per-VAO-index model here does not
// express. Declined rather than fetched wrong.
default: return VK_FORMAT_UNDEFINED;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Float32:
switch (size) {
case 1: return VK_FORMAT_R32_SFLOAT;
case 2: return VK_FORMAT_R32G32_SFLOAT;
case 3: return VK_FORMAT_R32G32B32_SFLOAT;
case 4: return VK_FORMAT_R32G32B32A32_SFLOAT;
default: return VK_FORMAT_UNDEFINED;
case 1:
return VK_FORMAT_R32_SFLOAT;
case 2:
return VK_FORMAT_R32G32_SFLOAT;
case 3:
return VK_FORMAT_R32G32B32_SFLOAT;
case 4:
return VK_FORMAT_R32G32B32A32_SFLOAT;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Float16:
// GL_HALF_FLOAT is a floating-point array type: it is never an integer attribute, and
// GL_TRUE for `normalized` is ignored for float types rather than selecting a *NORM format.
if (isInteger) return VK_FORMAT_UNDEFINED;
switch (size) {
case 1: return VK_FORMAT_R16_SFLOAT;
case 2: return VK_FORMAT_R16G16_SFLOAT;
case 3: return VK_FORMAT_R16G16B16_SFLOAT;
case 4: return VK_FORMAT_R16G16B16A16_SFLOAT;
default: return VK_FORMAT_UNDEFINED;
case 1:
return VK_FORMAT_R16_SFLOAT;
case 2:
return VK_FORMAT_R16G16_SFLOAT;
case 3:
return VK_FORMAT_R16G16B16_SFLOAT;
case 4:
return VK_FORMAT_R16G16B16A16_SFLOAT;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Int32:
if (!isInteger || normalized) return VK_FORMAT_UNDEFINED;
switch (size) {
case 1: return VK_FORMAT_R32_SINT;
case 2: return VK_FORMAT_R32G32_SINT;
case 3: return VK_FORMAT_R32G32B32_SINT;
case 4: return VK_FORMAT_R32G32B32A32_SINT;
default: return VK_FORMAT_UNDEFINED;
case 1:
return VK_FORMAT_R32_SINT;
case 2:
return VK_FORMAT_R32G32_SINT;
case 3:
return VK_FORMAT_R32G32B32_SINT;
case 4:
return VK_FORMAT_R32G32B32A32_SINT;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Uint32:
if (!isInteger || normalized) return VK_FORMAT_UNDEFINED;
switch (size) {
case 1: return VK_FORMAT_R32_UINT;
case 2: return VK_FORMAT_R32G32_UINT;
case 3: return VK_FORMAT_R32G32B32_UINT;
case 4: return VK_FORMAT_R32G32B32A32_UINT;
default: return VK_FORMAT_UNDEFINED;
case 1:
return VK_FORMAT_R32_UINT;
case 2:
return VK_FORMAT_R32G32_UINT;
case 3:
return VK_FORMAT_R32G32B32_UINT;
case 4:
return VK_FORMAT_R32G32B32A32_UINT;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Int16:
switch (size) {
@@ -339,7 +384,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
case 4:
return isInteger ? VK_FORMAT_R16G16B16A16_SINT
: (normalized ? VK_FORMAT_R16G16B16A16_SNORM : VK_FORMAT_R16G16B16A16_SSCALED);
default: return VK_FORMAT_UNDEFINED;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Uint16:
switch (size) {
@@ -354,37 +400,38 @@ namespace MobileGL::MG_Backend::DirectVulkan {
case 4:
return isInteger ? VK_FORMAT_R16G16B16A16_UINT
: (normalized ? VK_FORMAT_R16G16B16A16_UNORM : VK_FORMAT_R16G16B16A16_USCALED);
default: return VK_FORMAT_UNDEFINED;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Int8:
switch (size) {
case 1:
return isInteger ? VK_FORMAT_R8_SINT : (normalized ? VK_FORMAT_R8_SNORM : VK_FORMAT_R8_SSCALED);
case 2:
return isInteger ? VK_FORMAT_R8G8_SINT
: (normalized ? VK_FORMAT_R8G8_SNORM : VK_FORMAT_R8G8_SSCALED);
return isInteger ? VK_FORMAT_R8G8_SINT : (normalized ? VK_FORMAT_R8G8_SNORM : VK_FORMAT_R8G8_SSCALED);
case 3:
return isInteger ? VK_FORMAT_R8G8B8_SINT
: (normalized ? VK_FORMAT_R8G8B8_SNORM : VK_FORMAT_R8G8B8_SSCALED);
case 4:
return isInteger ? VK_FORMAT_R8G8B8A8_SINT
: (normalized ? VK_FORMAT_R8G8B8A8_SNORM : VK_FORMAT_R8G8B8A8_SSCALED);
default: return VK_FORMAT_UNDEFINED;
default:
return VK_FORMAT_UNDEFINED;
}
case DataType::Uint8:
switch (size) {
case 1:
return isInteger ? VK_FORMAT_R8_UINT : (normalized ? VK_FORMAT_R8_UNORM : VK_FORMAT_R8_USCALED);
case 2:
return isInteger ? VK_FORMAT_R8G8_UINT
: (normalized ? VK_FORMAT_R8G8_UNORM : VK_FORMAT_R8G8_USCALED);
return isInteger ? VK_FORMAT_R8G8_UINT : (normalized ? VK_FORMAT_R8G8_UNORM : VK_FORMAT_R8G8_USCALED);
case 3:
return isInteger ? VK_FORMAT_R8G8B8_UINT
: (normalized ? VK_FORMAT_R8G8B8_UNORM : VK_FORMAT_R8G8B8_USCALED);
case 4:
return isInteger ? VK_FORMAT_R8G8B8A8_UINT
: (normalized ? VK_FORMAT_R8G8B8A8_UNORM : VK_FORMAT_R8G8B8A8_USCALED);
default: return VK_FORMAT_UNDEFINED;
default:
return VK_FORMAT_UNDEFINED;
}
default:
return VK_FORMAT_UNDEFINED;
@@ -454,11 +501,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkFormat VertexInputStateFactory::ToFloat32VertexFormat(Int componentCount) {
switch (componentCount) {
case 1: return VK_FORMAT_R32_SFLOAT;
case 2: return VK_FORMAT_R32G32_SFLOAT;
case 3: return VK_FORMAT_R32G32B32_SFLOAT;
case 4: return VK_FORMAT_R32G32B32A32_SFLOAT;
default: return VK_FORMAT_UNDEFINED;
case 1:
return VK_FORMAT_R32_SFLOAT;
case 2:
return VK_FORMAT_R32G32_SFLOAT;
case 3:
return VK_FORMAT_R32G32B32_SFLOAT;
case 4:
return VK_FORMAT_R32G32B32A32_SFLOAT;
default:
return VK_FORMAT_UNDEFINED;
}
}
@@ -69,11 +69,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// binding uses divisor 1, which is what the plain input rate already means.
Vector<VkVertexInputBindingDivisorDescriptionEXT> bindingDivisors;
VkPipelineVertexInputDivisorStateCreateInfoEXT divisorState{
VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_DIVISOR_STATE_CREATE_INFO_EXT
};
VkPipelineVertexInputStateCreateInfo state{
VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO
};
VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_DIVISOR_STATE_CREATE_INFO_EXT};
VkPipelineVertexInputStateCreateInfo state{VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO};
// Back to a default-constructed state while keeping the vectors' storage, so a
// per-draw scratch rebuilds without touching the heap. Every member is reset by the
@@ -104,8 +101,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
};
VertexInputStateFactory(const VulkanRendererConfig& config, VkPhysicalDevice physicalDevice):
m_config(config), m_physicalDevice(physicalDevice) {}
VertexInputStateFactory(const VulkanRendererConfig& config, VkPhysicalDevice physicalDevice)
: m_config(config), m_physicalDevice(physicalDevice) {}
~VertexInputStateFactory() = default;
VertexInputStateFactory(const VertexInputStateFactory&) = delete;
@@ -88,9 +88,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// its mapped pointer), and may be bound to any role, so it carries every usage.
// TRANSFER_DST is added by CreateResidentStorage.
constexpr VkBufferUsageFlags kPersistentBackedUsage =
VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_INDEX_BUFFER_BIT |
VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT |
VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT | VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT |
VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_INDEX_BUFFER_BIT | VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT |
VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT |
VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT |
// "Every usage" has to mean every usage: a buffer texture reached through an IMAGE
// unit takes a VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER descriptor, and the write is
// invalid unless the buffer was created with this bit. Nothing asked for it until
@@ -98,8 +98,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VK_BUFFER_USAGE_STORAGE_TEXEL_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
// Appended to kPersistentBackedUsage when VK_EXT_transform_feedback is enabled
// (see VkBufferManagerInitInfo::transformFeedbackUsageEnabled).
constexpr VkBufferUsageFlags kTransformFeedbackUsage =
VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_BUFFER_BIT_EXT;
constexpr VkBufferUsageFlags kTransformFeedbackUsage = VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_BUFFER_BIT_EXT;
// The app writes into the persistent map with no explicit flush, so its memory must
// be host-coherent (Adreno host-visible memory is; requiring it keeps us portable).
constexpr VkMemoryPropertyFlags kPersistentBackedRequiredFlags =
@@ -108,7 +107,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
using MG_State::GLState::BufferObject;
// The manager owned by the active VulkanRenderer; immediate ops route here.
SessionLocal<VkBufferManager*> g_activeBufferManager; // per Magma session (MagmaSession.h)
SessionLocal<VkBufferManager*> g_activeBufferManager; // per Magma session (MagmaSession.h)
// The session's manager, or null once its renderer is gone - a client's records can still
// arrive after its eglTerminate released it (or, surfaceless, before its first surface built
@@ -122,27 +121,38 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const MG_Pipe::MGPipeResourceOps g_vulkanWireResourceOps = {
.Create = [](auto res, const auto& desc) { if (auto* m = WireManager("create")) m->CreateWireBuffer(res, desc); },
.Respecify = [](auto res, const auto& desc, const void* bytes) {
if (auto* m = WireManager("respecify")) m->RespecifyWireBuffer(res, desc, bytes);
},
.SubData = [](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("write"))
m->WriteWireBuffer(res, MG_Pipe::MGPipeSubDataBufferOffset(record),
MG_Pipe::MGPipeSubDataBufferSize(record), bytes);
},
.SubDataResident = [](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("write"))
m->WriteWireBuffer(res, MG_Pipe::MGPipeSubDataBufferOffset(record),
MG_Pipe::MGPipeSubDataBufferSize(record), bytes);
},
.FlushRange = [](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("flush")) m->FlushWireBuffer(res, record.Offset, record.Size, bytes);
},
.Readback = [](auto res, const auto& record) {
if (auto* m = WireManager("readback")) m->ReadbackWireBuffer(res, record.Offset, record.Size);
},
.Destroy = [](auto res) { if (auto* m = WireManager("destroy")) m->DestroyWireBuffer(res); },
.Create =
[](auto res, const auto& desc) {
if (auto* m = WireManager("create")) m->CreateWireBuffer(res, desc);
},
.Respecify =
[](auto res, const auto& desc, const void* bytes) {
if (auto* m = WireManager("respecify")) m->RespecifyWireBuffer(res, desc, bytes);
},
.SubData =
[](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("write"))
m->WriteWireBuffer(res, MG_Pipe::MGPipeSubDataBufferOffset(record),
MG_Pipe::MGPipeSubDataBufferSize(record), bytes);
},
.SubDataResident =
[](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("write"))
m->WriteWireBuffer(res, MG_Pipe::MGPipeSubDataBufferOffset(record),
MG_Pipe::MGPipeSubDataBufferSize(record), bytes);
},
.FlushRange =
[](auto res, const auto& record, const void* bytes) {
if (auto* m = WireManager("flush")) m->FlushWireBuffer(res, record.Offset, record.Size, bytes);
},
.Readback =
[](auto res, const auto& record) {
if (auto* m = WireManager("readback")) m->ReadbackWireBuffer(res, record.Offset, record.Size);
},
.Destroy =
[](auto res) {
if (auto* m = WireManager("destroy")) m->DestroyWireBuffer(res);
},
// Split mapping remains T2: the client owns its map and pushes exact
// modified ranges. A server pointer is never donated across the wire. Monolith's record
// arm (P13 W4) is one process: there the store IS handed over, as the legacy arm's
@@ -206,8 +216,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void VkBufferManager::CreateWireBuffer(MG_Pipe::MGPipeHandle res, const MG_Pipe::MGPResourceDesc& desc) {
if (FindWireBuffer(res) != nullptr) {
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"duplicate-buffer-create\"} {slot=%u, gen=%u}",
res.Slot, res.Gen);
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"duplicate-buffer-create\"} {slot=%u, gen=%u}", res.Slot,
res.Gen);
std::abort();
}
m_wireBuffers.try_emplace(WireBufferKey(res));
@@ -219,11 +229,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
void VkBufferManager::RespecifyWireBuffer(MG_Pipe::MGPipeHandle res, const MG_Pipe::MGPResourceDesc& desc,
const void* initialBytes) {
const void* initialBytes) {
auto* resource = FindWireBuffer(res);
if (!resource) {
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"unknown-buffer-respecify\"} {slot=%u, gen=%u}",
res.Slot, res.Gen);
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"unknown-buffer-respecify\"} {slot=%u, gen=%u}", res.Slot,
res.Gen);
std::abort();
}
// THE SERIAL IS READ BEFORE THE RESET BELOW ZEROES IT. Whether it is zero - whether any
@@ -240,7 +250,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource->imported = false;
resource->size = desc.Width;
resource->lastUseSerial = 0;
resource->lastUseSubmitIndex = 0; // M2 r2: the new store carries no submission yet (B3's stamp is per store)
resource->lastUseSubmitIndex = 0; // M2 r2: the new store carries no submission yet (B3's stamp is per store)
resource->gpuWritesPending = false;
resource->indirectReadBarrierPending = false;
resource->stagedCoverage.clear();
@@ -249,8 +259,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (resource->size == 0) return;
if (!CreateWireStoreBuffer(*resource)) {
MGLOG_F("Magma: Fatal{ResourceUnavailable, \"buffer-storage\"} {slot=%u, gen=%u, size=%llu}",
res.Slot, res.Gen, static_cast<unsigned long long>(resource->size));
MGLOG_F("Magma: Fatal{ResourceUnavailable, \"buffer-storage\"} {slot=%u, gen=%u, size=%llu}", res.Slot,
res.Gen, static_cast<unsigned long long>(resource->size));
std::abort();
}
++m_wireStoreCount;
@@ -378,8 +388,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
barrier.srcAccessMask = VK_ACCESS_MEMORY_WRITE_BIT;
barrier.dstAccessMask = VK_ACCESS_HOST_READ_BIT | VK_ACCESS_HOST_WRITE_BIT;
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_HOST_BIT,
0, 1, &barrier, 0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &barrier,
0, nullptr, 0, nullptr);
// P8-D: counted here, past the idle early-out, so `whw` is waits TAKEN, not calls.
++m_wireHostWaits;
const Bool timed = MG_Util::PipeStats::Enabled();
@@ -389,8 +399,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (timed) {
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireHostWaits, 1);
MG_Util::PipeStats::AddCalls(MG_Util::PipeStats::CallClass::WireHostWaitMicros,
static_cast<Uint64>(std::chrono::duration_cast<std::chrono::microseconds>(
std::chrono::steady_clock::now() - waitStart).count()));
static_cast<Uint64>(std::chrono::duration_cast<std::chrono::microseconds>(
std::chrono::steady_clock::now() - waitStart)
.count()));
}
if (!waited) {
return false;
@@ -403,18 +414,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
void VkBufferManager::WriteWireBuffer(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
const void* bytes) {
void VkBufferManager::WriteWireBuffer(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size, const void* bytes) {
auto* resource = FindWireBuffer(res);
if (!resource || offset > resource->size || size > resource->size - offset || (size && !bytes)) {
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"buffer-write-range\"} {slot=%u, gen=%u}",
res.Slot, res.Gen);
MGLOG_F("Magma: Fatal{ProtocolCorruption, \"buffer-write-range\"} {slot=%u, gen=%u}", res.Slot, res.Gen);
std::abort();
}
if (size == 0) return;
resource->contentSerial = ++m_wireContentSerial;
if (!resource->shadow.empty())
Memcpy(resource->shadow.data() + offset, bytes, static_cast<SizeT>(size));
if (!resource->shadow.empty()) Memcpy(resource->shadow.data() + offset, bytes, static_cast<SizeT>(size));
Bool uploaded = false;
const Uint64 probeLastUse = resource->lastUseSerial;
const Uint64 probeCompleted = GetCompletedSerial();
@@ -468,8 +476,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// byte update waits, then writes only the exact range; rounding it
// from a CPU shadow could overwrite neighbouring GPU-written bytes.
if (((offset | size) & 3u) == 0) {
uploaded = StagedWireRangeCopy(*resource, bytes, static_cast<SizeT>(offset),
static_cast<SizeT>(size));
uploaded = StagedWireRangeCopy(*resource, bytes, static_cast<SizeT>(offset), static_cast<SizeT>(size));
}
if (!uploaded && !WaitForWireBufferHostAccess(*resource)) {
if (LatchedOnDeviceLoss("buffer-host-write")) return;
@@ -490,32 +497,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (WireBufProbeEnabled()) {
MGLOG_I("WBUF write slot=%u gen=%u off=%llu size=%llu branch=%s bySerial=%d bySubmit=%d "
"lastUse=%llu completed=%llu frameSerial=%llu floor=%llu lastUseSubmit=%llu",
res.Slot, res.Gen, static_cast<unsigned long long>(offset),
static_cast<unsigned long long>(size),
(busyBySerial || busyBySubmission) ? "STAGED-or-WAIT" : "HOST-IMMEDIATE",
busyBySerial ? 1 : 0, busyBySubmission ? 1 : 0,
static_cast<unsigned long long>(probeLastUse), static_cast<unsigned long long>(probeCompleted),
static_cast<unsigned long long>(m_frameSerial),
res.Slot, res.Gen, static_cast<unsigned long long>(offset), static_cast<unsigned long long>(size),
(busyBySerial || busyBySubmission) ? "STAGED-or-WAIT" : "HOST-IMMEDIATE", busyBySerial ? 1 : 0,
busyBySubmission ? 1 : 0, static_cast<unsigned long long>(probeLastUse),
static_cast<unsigned long long>(probeCompleted), static_cast<unsigned long long>(m_frameSerial),
static_cast<unsigned long long>(m_completedSerialFloor),
static_cast<unsigned long long>(resource->lastUseSubmitIndex));
}
MG_Record::StagedShadowStore::CoverageAdd(resource->stagedCoverage,
static_cast<SizeT>(offset),
static_cast<SizeT>(offset + size));
MG_Record::StagedShadowStore::CoverageAdd(resource->stagedCoverage, static_cast<SizeT>(offset),
static_cast<SizeT>(offset + size));
}
void VkBufferManager::FlushWireBuffer(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
const void* bytes) {
void VkBufferManager::FlushWireBuffer(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size, const void* bytes) {
if (bytes != nullptr) {
WriteWireBuffer(res, offset, size, bytes);
return;
}
const auto* resource = FindWireBuffer(res);
if (!resource || offset > resource->size || size > resource->size - offset ||
!MG_Record::StagedShadowStore::CoverageHas(resource->stagedCoverage,
static_cast<SizeT>(offset), static_cast<SizeT>(offset + size))) {
MGLOG_F("Magma: Fatal{StageSnapshotTooNarrow, \"buffer-flush\"} {slot=%u, gen=%u}",
res.Slot, res.Gen);
!MG_Record::StagedShadowStore::CoverageHas(resource->stagedCoverage, static_cast<SizeT>(offset),
static_cast<SizeT>(offset + size))) {
MGLOG_F("Magma: Fatal{StageSnapshotTooNarrow, \"buffer-flush\"} {slot=%u, gen=%u}", res.Slot, res.Gen);
std::abort();
}
// SubData has already copied the owned record bytes into GPU storage,
@@ -567,7 +569,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
while (!m_convertedStreams.empty() && m_convertedStreamBytes + size > kConvertedStreamBudget) {
SizeT oldest = 0;
for (SizeT i = 1; i < m_convertedStreams.size(); ++i)
if (m_convertedStreams[i].lastUseFrameSerial < m_convertedStreams[oldest].lastUseFrameSerial) oldest = i;
if (m_convertedStreams[i].lastUseFrameSerial < m_convertedStreams[oldest].lastUseFrameSerial)
oldest = i;
RetireConvertedStream(oldest);
}
ConvertedStream entry;
@@ -582,7 +585,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.memoryUsage = VMA_MEMORY_USAGE_AUTO,
.allocationFlags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT,
.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,
}) || entry.buffer.Map() == nullptr || !entry.buffer.Upload(bytes, static_cast<VkDeviceSize>(size), 0))
}) ||
entry.buffer.Map() == nullptr || !entry.buffer.Upload(bytes, static_cast<VkDeviceSize>(size), 0))
return false;
m_convertedStreamBytes += size;
outSlice = entry.buffer.GetSlice();
@@ -614,8 +618,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// path) can submit this index without the draw. This stamp is the defence term's input
// (see WriteWireBuffer); the floor behind lastUseSerial is the guarantee. Read off the
// owning renderer's counter (its GetWireNextSubmitIndex()) when it was handed one.
if (m_submitCounter != nullptr) resource->lastUseSubmitIndex = *m_submitCounter + 1;
else if (pVulkanRenderer) resource->lastUseSubmitIndex = pVulkanRenderer->GetWireNextSubmitIndex();
if (m_submitCounter != nullptr)
resource->lastUseSubmitIndex = *m_submitCounter + 1;
else if (pVulkanRenderer)
resource->lastUseSubmitIndex = pVulkanRenderer->GetWireNextSubmitIndex();
outSlice = resource->buffer.GetSlice(0, resource->size);
outSlice.mapped = nullptr;
if (WireBufProbeEnabled()) {
@@ -674,15 +680,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
after.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
after.dstAccessMask = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT;
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
0, 1, &after, 0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, 0, 1, &after,
0, nullptr, 0, nullptr);
resource->lastUseSerial = m_frameSerial;
return true;
}
Bool VkBufferManager::CopyWireBufferSubWordRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset,
Uint64 size, Uint32 frameIndex,
const BufferSlice& dst, Uint64 dstSkip) {
Bool VkBufferManager::CopyWireBufferSubWordRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
Uint32 frameIndex, const BufferSlice& dst, Uint64 dstSkip) {
auto* resource = FindWireBuffer(res);
if (!resource || offset > resource->size || size > resource->size - offset) return false;
if (dstSkip > dst.size || size > dst.size - dstSkip) return false;
@@ -703,8 +708,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// already name - see BufferArena::EnsureCapacity - so the two slices are allowed to
// sit in different VkBuffers and the copy names each slice's own handle.
BufferSlice staging{};
if (!m_transientUploadArena.Allocate(frameIndex, stagedSize, 4, staging) || !staging.IsValid())
return false;
if (!m_transientUploadArena.Allocate(frameIndex, stagedSize, 4, staging) || !staging.IsValid()) return false;
const VkCommandBuffer commands = m_copyProvider->AcquireBufferCopyCommandBuffer();
if (commands == VK_NULL_HANDLE) return false;
@@ -723,8 +727,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
staged.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
staged.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT;
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
0, 1, &staged, 0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 1, &staged, 0,
nullptr, 0, nullptr);
const VkBufferCopy shift{staging.offset + headPad, dst.offset + dstSkip, size};
EndActiveRenderPassOn(commands);
vkCmdCopyBuffer(commands, staging.buffer, dst.buffer, 1, &shift);
@@ -732,8 +736,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
after.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
after.dstAccessMask = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT;
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
0, 1, &after, 0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, 0, 1, &after,
0, nullptr, 0, nullptr);
resource->lastUseSerial = m_frameSerial;
return true;
}
@@ -763,8 +767,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
// The reverse producer synchronously copies into SEG_EVENT before
// this owned vector is released; no mapped Vulkan pointer crosses roles.
MG_Pipe::gMGPipeCallbacks.OnBufferWriteback(res, offset,
{reinterpret_cast<Uint64>(bytes.data()), size, MG_Pipe::kMGHostSpanSegNone, 0});
MG_Pipe::gMGPipeCallbacks.OnBufferWriteback(
res, offset, {reinterpret_cast<Uint64>(bytes.data()), size, MG_Pipe::kMGHostSpanSegNone, 0});
}
void VkBufferManager::MarkWireBufferGpuWritten(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size) {
@@ -809,8 +813,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#if defined(__ANDROID__)
namespace {
Int32 T0PickMemoryType(const VkPhysicalDeviceMemoryProperties& props, Uint32 bits,
VkMemoryPropertyFlags want) {
Int32 T0PickMemoryType(const VkPhysicalDeviceMemoryProperties& props, Uint32 bits, VkMemoryPropertyFlags want) {
for (Uint32 i = 0; i < props.memoryTypeCount; ++i) {
if ((bits & (1u << i)) != 0 && (props.memoryTypes[i].propertyFlags & want) == want)
return static_cast<Int32>(i);
@@ -874,7 +877,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
VkMemoryDedicatedAllocateInfo dedicated{VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO};
dedicated.buffer = out.buffer;
VkImportAndroidHardwareBufferInfoANDROID import{VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID};
VkImportAndroidHardwareBufferInfoANDROID import{
VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID};
import.buffer = static_cast<AHardwareBuffer*>(ahb);
import.pNext = &dedicated;
VkMemoryAllocateInfo allocateInfo{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO};
@@ -991,11 +995,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (pVulkanRenderer == nullptr || !pVulkanRenderer->GetWireAhbImport(ctx))
return answer(false, "the device did not take VK_ANDROID_external_memory_android_hardware_buffer");
constexpr Uint64 kBytes = 64 * 1024;
constexpr Uint32 kWords = 1024; // one 4 KiB region
constexpr Uint64 kFillAt = 4 * 4096; // region 4
constexpr Uint32 kWords = 1024; // one 4 KiB region
constexpr Uint64 kFillAt = 4 * 4096; // region 4
constexpr Uint32 kFillWord = 0x6C0FFEE0u;
Vector<Uint32> pattern(kBytes / 4);
for (SizeT i = 0; i < pattern.size(); ++i) pattern[i] = 0x5A000000u ^ static_cast<Uint32>(i * 2654435761u);
for (SizeT i = 0; i < pattern.size(); ++i)
pattern[i] = 0x5A000000u ^ static_cast<Uint32>(i * 2654435761u);
MG_Remote::Transport::AdoptT0::HeldStore held;
String why;
if (!MG_Remote::Transport::AdoptT0::AllocateHeld(kBytes, pattern.data(), held, why))
@@ -1035,12 +1040,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
VkMemoryRequirements requirements{};
vkGetBufferMemoryRequirements(ctx.device, staging.buffer, &requirements);
const Int32 stagingType = T0PickMemoryType(ctx.memory, requirements.memoryTypeBits,
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
const Int32 stagingType =
T0PickMemoryType(ctx.memory, requirements.memoryTypeBits,
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
VkMemoryAllocateInfo stagingAlloc{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO};
stagingAlloc.allocationSize = requirements.size;
stagingAlloc.memoryTypeIndex = static_cast<Uint32>(stagingType < 0 ? 0 : stagingType);
if (stagingType < 0 || vkAllocateMemory(ctx.device, &stagingAlloc, nullptr, &staging.memory) != VK_SUCCESS ||
if (stagingType < 0 ||
vkAllocateMemory(ctx.device, &stagingAlloc, nullptr, &staging.memory) != VK_SUCCESS ||
vkBindBufferMemory(ctx.device, staging.buffer, staging.memory, 0) != VK_SUCCESS ||
vkMapMemory(ctx.device, staging.memory, 0, VK_WHOLE_SIZE, 0, &staging.mapped) != VK_SUCCESS) {
verdict = "staging memory";
@@ -1084,8 +1091,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
toHost.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
toHost.dstAccessMask = VK_ACCESS_HOST_READ_BIT;
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &toHost,
0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &toHost, 0,
nullptr, 0, nullptr);
vkEndCommandBuffer(commands);
VkSubmitInfo submit{VK_STRUCTURE_TYPE_SUBMIT_INFO};
submit.commandBufferCount = 1;
@@ -1105,8 +1112,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ok = gpuRead == kWords && heldSaw == kWords;
verdict = "64 KiB AHB, lock held throughout: host map read ok (" +
String(imported.coherent ? "coherent" : "non-coherent") + " memory), GPU read " +
std::to_string(gpuRead) + "/1024 words of the CPU's pattern, GPU write seen through the "
"held lock " + std::to_string(heldSaw) + "/1024 words";
std::to_string(gpuRead) +
"/1024 words of the CPU's pattern, GPU write seen through the "
"held lock " +
std::to_string(heldSaw) + "/1024 words";
if (!ok) break;
// STEADY STATE. The round above reads bytes the CPU wrote BEFORE the import, and an import
@@ -1142,8 +1151,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
EndActiveRenderPassOn(commands);
vkCmdFillBuffer(commands, imported.buffer, kFillAt, kWords * 4, kSteadyFillWord);
EndActiveRenderPassOn(commands);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &toHost,
0, nullptr, 0, nullptr);
vkCmdPipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &toHost, 0,
nullptr, 0, nullptr);
vkEndCommandBuffer(commands);
if (vkQueueSubmit(ctx.queue, 1, &submit, fence) != VK_SUCCESS ||
vkWaitForFences(ctx.device, 1, &fence, VK_TRUE, 2'000'000'000ull) != VK_SUCCESS) {
@@ -1172,17 +1181,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#endif
}
void VkBufferManager::DescribeWireBufferHistory(MG_Pipe::MGPipeHandle res) const {
const auto& history = *g_wireBufferHistory;
Int createdAgo = -1, destroyedAgo = -1;
for (SizeT i = 0; i < std::min(history.createdCount, WireBufferHistory::kDepth); ++i) {
const auto& h = history.created[(history.createdCount - 1 - i) % WireBufferHistory::kDepth];
if (h.Slot == res.Slot && h.Gen == res.Gen) { createdAgo = static_cast<Int>(i); break; }
if (h.Slot == res.Slot && h.Gen == res.Gen) {
createdAgo = static_cast<Int>(i);
break;
}
}
for (SizeT i = 0; i < std::min(history.destroyedCount, WireBufferHistory::kDepth); ++i) {
const auto& h = history.destroyed[(history.destroyedCount - 1 - i) % WireBufferHistory::kDepth];
if (h.Slot == res.Slot && h.Gen == res.Gen) { destroyedAgo = static_cast<Int>(i); break; }
if (h.Slot == res.Slot && h.Gen == res.Gen) {
destroyedAgo = static_cast<Int>(i);
break;
}
}
MGLOG_E("Magma wire buffer {slot=%u, gen=%u}: live stores %zu; created %d creates ago, destroyed %d "
"destroys ago (-1 = not in the last %zu); %zu creates, %zu destroys in this session",
@@ -1466,8 +1480,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return std::max(completed, m_completedSerialFloor);
}
Bool VkBufferManager::UploadTransient(BufferKind kind, Uint32 frameIndex, const void* data,
VkDeviceSize size, VkDeviceSize alignment, BufferSlice& outSlice) {
Bool VkBufferManager::UploadTransient(BufferKind kind, Uint32 frameIndex, const void* data, VkDeviceSize size,
VkDeviceSize alignment, BufferSlice& outSlice) {
if (!m_transientUploadArena.Upload(frameIndex, data, size, alignment, outSlice)) {
return false;
}
@@ -1492,8 +1506,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
break;
case BufferKind::TextureBuffer:
case BufferKind::ShaderStorage:
MG_Util::PipeStats::AddBytes(MG_Util::PipeStats::ByteClass::StageBuffer,
static_cast<Uint64>(size));
MG_Util::PipeStats::AddBytes(MG_Util::PipeStats::ByteClass::StageBuffer, static_cast<Uint64>(size));
break;
case BufferKind::Uniform:
break;
@@ -1508,8 +1521,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.frameCount = m_initInfo.frameCount,
.usage = VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_INDEX_BUFFER_BIT |
VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT |
(VkBufferUsageFlags{VK_BUFFER_USAGE_TRANSFER_DST_BIT}) |
VK_BUFFER_USAGE_TRANSFER_SRC_BIT,
(VkBufferUsageFlags{VK_BUFFER_USAGE_TRANSFER_DST_BIT}) | VK_BUFFER_USAGE_TRANSFER_SRC_BIT,
.memoryUsage = m_initInfo.transientMemoryUsage,
.allocationFlags = m_initInfo.transientAllocationFlags,
.minBufferSize = m_initInfo.minUploadBytes,
@@ -1517,14 +1529,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
});
}
Bool VkBufferManager::StagedWireRangeCopy(WireBufferResource& resource, const void* data,
SizeT offset, SizeT size) {
Bool VkBufferManager::StagedWireRangeCopy(WireBufferResource& resource, const void* data, SizeT offset,
SizeT size) {
if (!m_copyProvider) {
return false;
}
BufferSlice staging{};
if (!m_transientUploadArena.Upload(m_currentFrameIndex, data,
static_cast<VkDeviceSize>(size), 16, staging)) {
if (!m_transientUploadArena.Upload(m_currentFrameIndex, data, static_cast<VkDeviceSize>(size), 16, staging)) {
return false;
}
if (MG_Util::PipeStats::Enabled()) {
@@ -1569,7 +1580,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
void VkBufferManager::CollectDeferredReleases(Uint32 frameIndex) {
MOBILEGL_ASSERT(frameIndex < m_deferredBufferReleases.size(),
"VkBufferManager::CollectDeferredReleases frame index out of range");
@@ -1595,8 +1605,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.size = kUnboundStorageDescriptorBytes,
.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT,
.memoryUsage = VMA_MEMORY_USAGE_AUTO,
.allocationFlags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
VMA_ALLOCATION_CREATE_MAPPED_BIT,
.allocationFlags =
VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_MAPPED_BIT,
.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,
});
if (!created) {
@@ -1628,8 +1638,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.usage = VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_TEXEL_BUFFER_BIT |
VK_BUFFER_USAGE_TRANSFER_DST_BIT,
.memoryUsage = VMA_MEMORY_USAGE_AUTO,
.allocationFlags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
VMA_ALLOCATION_CREATE_MAPPED_BIT,
.allocationFlags =
VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_MAPPED_BIT,
.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,
});
if (!created) {
@@ -1647,7 +1657,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void VkBufferManager::DestroyAllDeferredReleases() {
// Both callers (Shutdown, RecreateTransientArenas) have proven the device idle.
DestroyAllDeferredWireReleases();
for (auto& stream : m_convertedStreams) stream.buffer.Destroy();
for (auto& stream : m_convertedStreams)
stream.buffer.Destroy();
m_convertedStreams.clear();
m_convertedStreamBytes = 0;
for (auto& releases : m_deferredBufferReleases) {
@@ -62,8 +62,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Names, for a Fatal about an unknown handle, whether this manager ever saw it.
void DescribeWireBufferHistory(MG_Pipe::MGPipeHandle res) const;
Bool ReadWireBuffer(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size, void* dst);
Bool CopyWireBufferRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
const BufferSlice& dst);
Bool CopyWireBufferRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size, const BufferSlice& dst);
// P7 A.1, the sub-word half of the one above: the same copy for a window whose start
// or end does not land on a four-byte boundary. The read of the APPLICATION's store is
// rounded OUT to whole words and clamped to the store's end, so the widened window can
@@ -73,8 +72,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// on a region's offsets or size (unlike vkCmdUpdateBuffer / vkCmdFillBuffer), so the
// shift is a plain legal copy. `dstSkip` is the byte inside `dst` the window starts at.
// This retires `uniform-buffer-byte-tail`.
Bool CopyWireBufferSubWordRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size,
Uint32 frameIndex, const BufferSlice& dst, Uint64 dstSkip);
Bool CopyWireBufferSubWordRangeToSlice(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size, Uint32 frameIndex,
const BufferSlice& dst, Uint64 dstSkip);
void MarkWireBufferGpuWritten(MG_Pipe::MGPipeHandle res, Uint64 offset, Uint64 size);
// P8-D: whether a shader write was marked on this store since the last indirect draw
// read it natively, clearing the mark. The caller records the INDIRECT_COMMAND_READ
@@ -186,6 +185,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// below it may be considered signaled. Drives IsResourceBusy and the
// backend GL fence objects.
Uint64 GetCompletedSerial() const;
private:
struct WireBufferResource {
VkBufferObject buffer;
@@ -36,9 +36,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBufferObject& operator=(VkBufferObject&& other) noexcept;
Bool Create(const VkBufferObjectDesc& desc);
Bool Create(VmaAllocator allocator, VkDeviceSize size, VkBufferUsageFlags usage,
VmaMemoryUsage memoryUsage, VmaAllocationCreateFlags allocationFlags = 0,
VkMemoryPropertyFlags requiredFlags = 0);
Bool Create(VmaAllocator allocator, VkDeviceSize size, VkBufferUsageFlags usage, VmaMemoryUsage memoryUsage,
VmaAllocationCreateFlags allocationFlags = 0, VkMemoryPropertyFlags requiredFlags = 0);
void Destroy();
void* Map();
@@ -76,8 +75,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the memory and drops `ahb` - which is why the store goes through the wire arm's
// serial-gated deferred release like every other: the AHardwareBuffer is let go only once
// no submission can still name it. `coherent` = the memory type is HOST_COHERENT.
void AdoptExternal(VkDevice device, VkBuffer buffer, VkDeviceMemory memory, VkDeviceSize size,
void* mapped, Bool coherent, void* ahb);
void AdoptExternal(VkDevice device, VkBuffer buffer, VkDeviceMemory memory, VkDeviceSize size, void* mapped,
Bool coherent, void* ahb);
Bool IsExternal() const { return m_externalMemory != VK_NULL_HANDLE; }
private:
@@ -22,8 +22,7 @@
namespace MobileGL::MG_Backend::DirectVulkan {
static Bool IsCubeMapFaceUploadTarget(TextureUploadTarget target) {
return target >= TextureUploadTarget::CubeMapPositiveX &&
target <= TextureUploadTarget::CubeMapNegativeZ;
return target >= TextureUploadTarget::CubeMapPositiveX && target <= TextureUploadTarget::CubeMapNegativeZ;
}
VkClearColorValue MakeVkClearColorValue(const ClearAttachmentPayload& payload, Bool formatLacksAlpha) {
@@ -64,8 +63,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Float value = std::clamp(encoded, 0.0f, 1.0f);
return value <= 0.04045f ? value / 12.92f : std::pow((value + 0.055f) / 1.055f, 2.4f);
};
payload.color = FloatVec4(toLinear(payload.color.x()), toLinear(payload.color.y()),
toLinear(payload.color.z()), payload.color.w());
payload.color = FloatVec4(toLinear(payload.color.x()), toLinear(payload.color.y()), toLinear(payload.color.z()),
payload.color.w());
}
// ResolveAttachmentLayerCount used to be duplicated here, reading attachment.GetSize().z()
@@ -29,7 +29,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// image's format rather than converting between the members - a float written where an
// integer format is expected is reinterpreted bit for bit, not rounded. Remember which entry
// point supplied the value so the member written when the clear is materialized matches.
enum class ClearColorEncoding : Uint8 { Float, Int, Uint };
enum class ClearColorEncoding : Uint8 {
Float,
Int,
Uint
};
struct ClearAttachmentPayload {
GLbitfield mask = 0;
@@ -18,7 +18,8 @@
#include <MG_Backend/MGPipe/PipeInputs.h>
namespace MobileGL::MG_Backend::DirectVulkan {
[[maybe_unused]] static Float ResolveColorClearAlpha(const MG_State::GLState::ITextureObject* texture, Float requestedAlpha) {
[[maybe_unused]] static Float ResolveColorClearAlpha(const MG_State::GLState::ITextureObject* texture,
Float requestedAlpha) {
if (texture != nullptr && MG_Util::GetBaseInternalFormatComponentCount(texture->GetFormat()) == 3) {
return 1.0f;
}
@@ -29,20 +30,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// VkClearManager needs the SAME answer: its pending-clear key's layerCount becomes a real
// VkImageSubresourceRange when a clear is materialised outside a render pass. See the header.
DepthStencilAttachmentLoadInfo ResolveDepthStencilAttachmentLoadInfo(
VkImageLayout trackedLayout, Bool clearDepth, Bool clearStencil) {
DepthStencilAttachmentLoadInfo ResolveDepthStencilAttachmentLoadInfo(VkImageLayout trackedLayout, Bool clearDepth,
Bool clearStencil) {
DepthStencilAttachmentLoadInfo info{};
info.depthLoadOp = clearDepth
? VK_ATTACHMENT_LOAD_OP_CLEAR
: (trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED ? VK_ATTACHMENT_LOAD_OP_DONT_CARE
: VK_ATTACHMENT_LOAD_OP_LOAD);
info.depthLoadOp = clearDepth ? VK_ATTACHMENT_LOAD_OP_CLEAR
: (trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED ? VK_ATTACHMENT_LOAD_OP_DONT_CARE
: VK_ATTACHMENT_LOAD_OP_LOAD);
info.stencilLoadOp = clearStencil
? VK_ATTACHMENT_LOAD_OP_CLEAR
: (trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED ? VK_ATTACHMENT_LOAD_OP_DONT_CARE
: VK_ATTACHMENT_LOAD_OP_LOAD);
info.initialLayout =
(trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED || (clearDepth && clearStencil)) ? VK_IMAGE_LAYOUT_UNDEFINED
: trackedLayout;
? VK_ATTACHMENT_LOAD_OP_CLEAR
: (trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED ? VK_ATTACHMENT_LOAD_OP_DONT_CARE
: VK_ATTACHMENT_LOAD_OP_LOAD);
info.initialLayout = (trackedLayout == VK_IMAGE_LAYOUT_UNDEFINED || (clearDepth && clearStencil))
? VK_IMAGE_LAYOUT_UNDEFINED
: trackedLayout;
return info;
}
@@ -54,11 +54,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return {attachmentExtent.x(), attachmentExtent.y()};
}
VkRenderPassManager::VkRenderPassManager(VkDevice device,
VkPhysicalDevice physicalDevice, VmaAllocator allocator, const VulkanRendererConfig& config,
VkTextureManager& textureManager, SwapchainObject& swapchainObject):
m_device(device), m_physicalDevice(physicalDevice), m_allocator(allocator), m_config(config),
m_textureManager(textureManager), m_swapchainObject(swapchainObject) {
VkRenderPassManager::VkRenderPassManager(VkDevice device, VkPhysicalDevice physicalDevice, VmaAllocator allocator,
const VulkanRendererConfig& config, VkTextureManager& textureManager,
SwapchainObject& swapchainObject)
: m_device(device), m_physicalDevice(physicalDevice), m_allocator(allocator), m_config(config),
m_textureManager(textureManager), m_swapchainObject(swapchainObject) {
RenderPassEntry::s_device.Get() = m_device;
s_textureManager.Get() = &m_textureManager;
s_swapchainObject.Get() = &m_swapchainObject;
@@ -83,12 +83,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
renderPassBeginInfo.pNext = nullptr;
renderPassBeginInfo.renderPass = renderPassEntry.renderPass;
renderPassBeginInfo.framebuffer = renderPassEntry.framebuffer;
renderPassBeginInfo.renderArea.offset = { 0, 0 };
renderPassBeginInfo.renderArea.extent = {
(Uint32)renderPassEntry.extent.x(), (Uint32)renderPassEntry.extent.y() };
renderPassBeginInfo.renderArea.offset = {0, 0};
renderPassBeginInfo.renderArea.extent = {(Uint32)renderPassEntry.extent.x(),
(Uint32)renderPassEntry.extent.y()};
Vector<VkClearValue> clearValues(renderPassEntry.attachmentCount);
for (auto& clearValue: clearValues) {
for (auto& clearValue : clearValues) {
clearValue.color = {0.0f, 0.0f, 0.0f, 1.0f};
clearValue.depthStencil = {1.0f, 0};
}
@@ -111,8 +111,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkRenderPassBeginInfo begin{VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO};
begin.renderPass = renderPass;
begin.framebuffer = renderPassEntry.framebuffer;
begin.renderArea = {{0, 0}, {static_cast<Uint32>(renderPassEntry.extent.x()),
static_cast<Uint32>(renderPassEntry.extent.y())}};
begin.renderArea = {
{0, 0}, {static_cast<Uint32>(renderPassEntry.extent.x()), static_cast<Uint32>(renderPassEntry.extent.y())}};
begin.clearValueCount = clearValueCount;
begin.pClearValues = clearValues;
vkCmdBeginRenderPass(commandBuffer, &begin, VK_SUBPASS_CONTENTS_INLINE);
@@ -137,9 +137,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
namespace {
std::atomic<Uint64> g_wireImageWriteEpoch{1};
} // namespace
Uint64 WireImageWriteEpoch() {
return g_wireImageWriteEpoch.load(std::memory_order_relaxed);
}
void BumpWireImageWriteEpoch() {
g_wireImageWriteEpoch.fetch_add(1, std::memory_order_relaxed);
}
Uint64 WireImageWriteEpoch() { return g_wireImageWriteEpoch.load(std::memory_order_relaxed); }
void BumpWireImageWriteEpoch() { g_wireImageWriteEpoch.fetch_add(1, std::memory_order_relaxed); }
void EndActiveRenderPassOn(VkCommandBuffer commandBuffer) {
BumpWireImageWriteEpoch();
@@ -26,8 +26,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkImageLayout initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
};
DepthStencilAttachmentLoadInfo ResolveDepthStencilAttachmentLoadInfo(
VkImageLayout trackedLayout, Bool clearDepth, Bool clearStencil);
DepthStencilAttachmentLoadInfo ResolveDepthStencilAttachmentLoadInfo(VkImageLayout trackedLayout, Bool clearDepth,
Bool clearStencil);
IntVec2 ResolveRenderPassFramebufferExtent(Bool isDefaultFbo, const TextureSize& attachmentExtent,
VkExtent2D swapchainExtent);
@@ -128,9 +128,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return this->compatibilityHash == that.compatibilityHash;
}
Bool CompatibleWith(Uint64 compatibilityHash) const {
return this->compatibilityHash == compatibilityHash;
}
Bool CompatibleWith(Uint64 compatibilityHash) const { return this->compatibilityHash == compatibilityHash; }
};
struct ActiveRenderPassInfo {
@@ -139,31 +137,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
IntVec2 extent = {0, 0};
Bool CompatibleWith(const RenderPassEntry& that) const {
return compatibilityHash == that.compatibilityHash;
}
Bool CompatibleWith(const RenderPassEntry& that) const { return compatibilityHash == that.compatibilityHash; }
Bool CompatibleWith(Uint64 thatCompatibilityHash) const {
return compatibilityHash == thatCompatibilityHash;
}
Bool CompatibleWith(Uint64 thatCompatibilityHash) const { return compatibilityHash == thatCompatibilityHash; }
};
class VkRenderPassManager {
public:
using HashType = Uint64;
VkRenderPassManager(VkDevice device,
VkPhysicalDevice physicalDevice, VmaAllocator allocator, const VulkanRendererConfig& config,
VkTextureManager& textureManager, SwapchainObject& swapchainObject);
VkRenderPassManager(VkDevice device, VkPhysicalDevice physicalDevice, VmaAllocator allocator,
const VulkanRendererConfig& config, VkTextureManager& textureManager,
SwapchainObject& swapchainObject);
~VkRenderPassManager();
Bool Initialize();
void Shutdown();
// The surface target the default framebuffer currently names (VulkanRenderer's surface
// targets): a default-framebuffer pass of one surface is not another's.
void SetDefaultFramebufferTarget(Uint64 serial) {
m_defaultFramebufferTarget = serial;
}
void SetDefaultFramebufferTarget(Uint64 serial) { m_defaultFramebufferTarget = serial; }
static Bool BeginRenderPass(VkCommandBuffer commandBuffer, RenderPassEntry& renderPassEntry);
// P15 S0: begins `renderPassEntry` through `renderPass` (one of its clear-load variants)
// with these clear values; it is active as the entry itself, so a draw continues it.
@@ -171,6 +163,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkRenderPass renderPass, const VkClearValue* clearValues, Uint32 clearValueCount);
static Bool EndRenderPass(VkCommandBuffer commandBuffer);
static ActiveRenderPassInfo* GetActiveRenderPass();
private:
VkDevice m_device = VK_NULL_HANDLE;
VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE;
@@ -190,14 +183,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 GetRenderbufferImageEpoch() const { return m_renderbufferImageEpoch; }
private:
Uint64 m_defaultFramebufferTarget = 0;
// Supported sample counts per attachment format, so per-draw resource lookups
// do not repeat vkGetPhysicalDeviceImageFormatProperties.
UnorderedMap<VkFormat, VkSampleCountFlags> m_attachmentSampleCountsByFormat;
static inline thread_local XXH64_state_t* m_hashState = XXH64_createState();
// Per Magma session (MagmaSession.h): one session's open render pass and managers are not
// another's.
@@ -261,7 +261,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
template <class SamplerSource>
Float VkSamplerManager::ResolveEffectiveMaxAnisotropy(const SamplerSource& sampler,
Bool forceNearestFiltering) const {
Bool forceNearestFiltering) const {
if (!m_samplerAnisotropySupported) return 1.0f;
if (forceNearestFiltering) return 1.0f;
// VUID-VkSamplerCreateInfo-anisotropyEnable-01071/01072: anisotropy requires both filters to
@@ -322,9 +322,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
template <class SamplerSource>
Uint64 VkSamplerManager::BuildSamplerKey(const SamplerSource& sampler,
Bool forceNearestFiltering, Bool singleLevelView,
const ResolvedBorderColor& borderColor) const {
Uint64 VkSamplerManager::BuildSamplerKey(const SamplerSource& sampler, Bool forceNearestFiltering,
Bool singleLevelView, const ResolvedBorderColor& borderColor) const {
MOBILEGL_ASSERT(m_config != nullptr, "VkSamplerManager::BuildSamplerKey: m_config is null");
// P15: one read of the thread_local per hash (emulated TLS: every read is a call).
XXH64_state_t* const hashState = m_hashState;
@@ -371,9 +370,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
template <class SamplerSource>
VkSampler VkSamplerManager::GetOrCreateSamplerImpl(const SamplerSource& sampler,
TextureInternalFormat format,
Bool forceNearestFiltering, Uint32 viewLevelCount) {
VkSampler VkSamplerManager::GetOrCreateSamplerImpl(const SamplerSource& sampler, TextureInternalFormat format,
Bool forceNearestFiltering, Uint32 viewLevelCount) {
// A view that exposes a single mip level has no second level to blend with, so GL's
// *_MIPMAP_* minification filters degenerate to plain filtering on the base level -
// sampling is unchanged by pinning the Vulkan sampler to NEAREST mip mode at LOD 0.
@@ -396,9 +394,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerInfo.sType = VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO;
samplerInfo.magFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMagFilter());
samplerInfo.minFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMinFilter());
samplerInfo.mipmapMode = (forceNearestFiltering || singleLevelView)
? VK_SAMPLER_MIPMAP_MODE_NEAREST
: ToVkMipmapMode(sampler.GetMipmapMode());
samplerInfo.mipmapMode = (forceNearestFiltering || singleLevelView) ? VK_SAMPLER_MIPMAP_MODE_NEAREST
: ToVkMipmapMode(sampler.GetMipmapMode());
samplerInfo.addressModeU = ToVkAddressMode(sampler.GetWrapS());
samplerInfo.addressModeV = ToVkAddressMode(sampler.GetWrapT());
samplerInfo.addressModeW = ToVkAddressMode(sampler.GetWrapR());
@@ -408,7 +405,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Float maxAnisotropy = ResolveEffectiveMaxAnisotropy(sampler, forceNearestFiltering);
samplerInfo.anisotropyEnable = maxAnisotropy > 1.0f ? VK_TRUE : VK_FALSE;
samplerInfo.maxAnisotropy = maxAnisotropy;
samplerInfo.compareEnable = sampler.GetCompareMode() == SamplerCompareMode::CompareToTexture ? VK_TRUE : VK_FALSE;
samplerInfo.compareEnable =
sampler.GetCompareMode() == SamplerCompareMode::CompareToTexture ? VK_TRUE : VK_FALSE;
samplerInfo.compareOp = ToVkCompareOp(sampler.GetSamplerCompareFunc());
// Must match BuildSamplerKey's resolution exactly.
samplerInfo.maxLod = ResolveSingleLevelMaxLod(sampler, singleLevelView);
@@ -445,9 +443,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
VkSampler VkSamplerManager::GetOrCreateSamplerFromParameters(const SamplerParameters& parameters,
TextureInternalFormat format, Bool forceNearestFiltering, Uint32 viewLevelCount) {
return GetOrCreateSamplerImpl(SamplerParametersSource{parameters}, format,
forceNearestFiltering, viewLevelCount);
TextureInternalFormat format,
Bool forceNearestFiltering, Uint32 viewLevelCount) {
return GetOrCreateSamplerImpl(SamplerParametersSource{parameters}, format, forceNearestFiltering,
viewLevelCount);
}
VkFilter VkSamplerManager::ToVkFilter(SamplerFilterMode mode) {
@@ -506,8 +505,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
template <class SamplerSource>
VkSamplerManager::ResolvedBorderColor VkSamplerManager::ResolveBorderColor(
const SamplerSource& sampler, TextureInternalFormat format) const {
VkSamplerManager::ResolvedBorderColor VkSamplerManager::ResolveBorderColor(const SamplerSource& sampler,
TextureInternalFormat format) const {
ResolvedBorderColor resolved{};
if (!UsesBorderColor(sampler)) {
return resolved; // FLOAT_TRANSPARENT_BLACK, never sampled
@@ -14,117 +14,116 @@
#include <MG_State/GLState/SamplerState/SamplerObject.h>
namespace MobileGL::MG_State::GLState {
class SamplerObject;
class ITextureObject;
}
class SamplerObject;
class ITextureObject;
} // namespace MobileGL::MG_State::GLState
namespace MobileGL::MG_Backend::DirectVulkan {
class VkSamplerManager {
public:
struct InitInfo {
VkDevice device = VK_NULL_HANDLE;
const VulkanRendererConfig* config = nullptr;
// The samplerAnisotropy device feature was requested and granted at vkCreateDevice.
Bool samplerAnisotropySupported = false;
// VkPhysicalDeviceLimits::maxSamplerAnisotropy.
Float maxSamplerAnisotropy = 1.0f;
// VK_EXT_custom_border_color was enabled with BOTH customBorderColors and
// customBorderColorWithoutFormat; see VulkanRenderer::m_customBorderColorFeatureEnabled.
Bool customBorderColorSupported = false;
// VkPhysicalDeviceCustomBorderColorPropertiesEXT::maxCustomBorderColorSamplers. A hard device
// limit on how many LIVE samplers may carry a custom border colour, so the cache counts them
// and falls back to the snapped predefined value once it is reached.
Uint32 maxCustomBorderColorSamplers = 0;
class VkSamplerManager {
public:
struct InitInfo {
VkDevice device = VK_NULL_HANDLE;
const VulkanRendererConfig* config = nullptr;
// The samplerAnisotropy device feature was requested and granted at vkCreateDevice.
Bool samplerAnisotropySupported = false;
// VkPhysicalDeviceLimits::maxSamplerAnisotropy.
Float maxSamplerAnisotropy = 1.0f;
// VK_EXT_custom_border_color was enabled with BOTH customBorderColors and
// customBorderColorWithoutFormat; see VulkanRenderer::m_customBorderColorFeatureEnabled.
Bool customBorderColorSupported = false;
// VkPhysicalDeviceCustomBorderColorPropertiesEXT::maxCustomBorderColorSamplers. A hard device
// limit on how many LIVE samplers may carry a custom border colour, so the cache counts them
// and falls back to the snapped predefined value once it is reached.
Uint32 maxCustomBorderColorSamplers = 0;
};
Bool Initialize(const InitInfo& initInfo);
void Shutdown();
// The applier owns these values: effective bound/builtin sampler parameters and
// the sampled texture view's format/range. No frontend object is constructed.
VkSampler GetOrCreateSamplerFromParameters(const SamplerParameters& parameters, TextureInternalFormat format,
Bool forceNearestFiltering = false, Uint32 viewLevelCount = 0);
// Frame boundary hook: ages the sampler cache and destroys samplers not used
// for many frames. The key hashes continuous float state (lodBias, LOD clamps,
// anisotropy), so an app animating those would otherwise mint an unbounded
// stream of never-destroyed VkSamplers and eventually exhaust the device's
// maxSamplerAllocationCount. A sampler idle for over a thousand frame
// boundaries cannot be referenced by any in-flight command buffer (frames in
// flight are single digits), and every descriptor set the GPU consumes is
// written that same frame with live handles (the per-binding resolve memo and
// descriptor-set reuse are both frame-reset), so destruction here needs no
// fence wait. Self-gated: one counter bump and compare except on sweep
// boundaries.
void OnFrameBoundary();
// For a caller that memoises a returned VkSampler (UniformManager's wire sampler memo): the
// frame-boundary count a GetOrCreate call stamps, and a counter that moves whenever any cached
// sampler is destroyed. A memo may reuse a handle only while the destroy epoch is unchanged,
// and must still call GetOrCreate once per frame boundary so the entry stays stamped as used.
Uint64 FrameBoundaryCount() const { return m_frameBoundaryCounter; }
Uint64 DestroyEpoch() const { return m_destroyEpoch; }
// What GL_TEXTURE_BORDER_COLOR resolves to for one (sampler, texture) pair. `color` is always a
// legal VkBorderColor; when `isCustom` it is one of the *_CUSTOM_EXT values and `customValue`
// carries the actual components in a VkSamplerCustomBorderColorCreateInfoEXT.
//
// Resolved ONCE per GetOrCreateSampler call and threaded into both the cache key and the
// create-info, so the two cannot disagree - the same discipline the resolved anisotropy needs,
// and here it also makes the maxCustomBorderColorSamplers fallback deterministic: whether a
// custom colour was affordable is decided before the key is built, not twice with a budget
// change in between.
struct ResolvedBorderColor {
VkBorderColor color = VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK;
VkClearColorValue customValue{};
Bool isCustom = false;
};
private:
struct SamplerCacheEntry {
VkSampler handle = VK_NULL_HANDLE;
Uint externalIndex = 0;
Uint16 version = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age have their VkSampler destroyed.
Uint64 lastUsedFrameBoundary = 0;
// Counted against maxCustomBorderColorSamplers for as long as this entry lives.
Bool usesCustomBorderColor = false;
};
template <class SamplerSource>
VkSampler GetOrCreateSamplerImpl(const SamplerSource& sampler, TextureInternalFormat format,
Bool forceNearestFiltering, Uint32 viewLevelCount);
template <class SamplerSource>
Uint64 BuildSamplerKey(const SamplerSource& sampler, Bool forceNearestFiltering, Bool singleLevelView,
const ResolvedBorderColor& borderColor) const;
static VkFilter ToVkFilter(SamplerFilterMode mode);
static VkSamplerMipmapMode ToVkMipmapMode(SamplerMipmapMode mode);
static VkSamplerAddressMode ToVkAddressMode(SamplerWrapMode mode);
static VkCompareOp ToVkCompareOp(SamplerCompareFunc func);
template <class SamplerSource>
ResolvedBorderColor ResolveBorderColor(const SamplerSource& sampler, TextureInternalFormat format) const;
// The anisotropy Vulkan will actually apply: 1.0 (i.e. disabled) unless the feature is on and
// the sampler filters linearly both ways, otherwise the GL request clamped to the device limit.
// GL happily carries GL_TEXTURE_MAX_ANISOTROPY on a NEAREST sampler (Blaze3D's blocks do exactly
// that) while Vulkan forbids anisotropyEnable there, so the GL value must never be forwarded raw.
template <class SamplerSource>
Float ResolveEffectiveMaxAnisotropy(const SamplerSource& sampler, Bool forceNearestFiltering) const;
VkDevice m_device = VK_NULL_HANDLE;
const VulkanRendererConfig* m_config = nullptr;
Bool m_samplerAnisotropySupported = false;
Float m_maxSamplerAnisotropy = 1.0f;
Bool m_customBorderColorSupported = false;
Uint32 m_maxCustomBorderColorSamplers = 0;
// Live cache entries carrying a custom border colour. Kept in step with the entries themselves
// in exactly the three places one can appear or disappear: creation, the OnFrameBoundary sweep,
// and Shutdown.
Uint32 m_customBorderColorSamplerCount = 0;
UnorderedMap<Uint64, SamplerCacheEntry> m_samplers;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameBoundaryCounter = 0;
// Moves on every vkDestroySampler of a cached entry (sweep and Shutdown).
Uint64 m_destroyEpoch = 0;
static inline thread_local XXH64_state_t* m_hashState = XXH64_createState();
};
Bool Initialize(const InitInfo& initInfo);
void Shutdown();
// The applier owns these values: effective bound/builtin sampler parameters and
// the sampled texture view's format/range. No frontend object is constructed.
VkSampler GetOrCreateSamplerFromParameters(const SamplerParameters& parameters,
TextureInternalFormat format, Bool forceNearestFiltering = false, Uint32 viewLevelCount = 0);
// Frame boundary hook: ages the sampler cache and destroys samplers not used
// for many frames. The key hashes continuous float state (lodBias, LOD clamps,
// anisotropy), so an app animating those would otherwise mint an unbounded
// stream of never-destroyed VkSamplers and eventually exhaust the device's
// maxSamplerAllocationCount. A sampler idle for over a thousand frame
// boundaries cannot be referenced by any in-flight command buffer (frames in
// flight are single digits), and every descriptor set the GPU consumes is
// written that same frame with live handles (the per-binding resolve memo and
// descriptor-set reuse are both frame-reset), so destruction here needs no
// fence wait. Self-gated: one counter bump and compare except on sweep
// boundaries.
void OnFrameBoundary();
// For a caller that memoises a returned VkSampler (UniformManager's wire sampler memo): the
// frame-boundary count a GetOrCreate call stamps, and a counter that moves whenever any cached
// sampler is destroyed. A memo may reuse a handle only while the destroy epoch is unchanged,
// and must still call GetOrCreate once per frame boundary so the entry stays stamped as used.
Uint64 FrameBoundaryCount() const { return m_frameBoundaryCounter; }
Uint64 DestroyEpoch() const { return m_destroyEpoch; }
// What GL_TEXTURE_BORDER_COLOR resolves to for one (sampler, texture) pair. `color` is always a
// legal VkBorderColor; when `isCustom` it is one of the *_CUSTOM_EXT values and `customValue`
// carries the actual components in a VkSamplerCustomBorderColorCreateInfoEXT.
//
// Resolved ONCE per GetOrCreateSampler call and threaded into both the cache key and the
// create-info, so the two cannot disagree - the same discipline the resolved anisotropy needs,
// and here it also makes the maxCustomBorderColorSamplers fallback deterministic: whether a
// custom colour was affordable is decided before the key is built, not twice with a budget
// change in between.
struct ResolvedBorderColor {
VkBorderColor color = VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK;
VkClearColorValue customValue{};
Bool isCustom = false;
};
private:
struct SamplerCacheEntry {
VkSampler handle = VK_NULL_HANDLE;
Uint externalIndex = 0;
Uint16 version = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age have their VkSampler destroyed.
Uint64 lastUsedFrameBoundary = 0;
// Counted against maxCustomBorderColorSamplers for as long as this entry lives.
Bool usesCustomBorderColor = false;
};
template <class SamplerSource>
VkSampler GetOrCreateSamplerImpl(const SamplerSource& sampler, TextureInternalFormat format,
Bool forceNearestFiltering, Uint32 viewLevelCount);
template <class SamplerSource>
Uint64 BuildSamplerKey(const SamplerSource& sampler, Bool forceNearestFiltering,
Bool singleLevelView, const ResolvedBorderColor& borderColor) const;
static VkFilter ToVkFilter(SamplerFilterMode mode);
static VkSamplerMipmapMode ToVkMipmapMode(SamplerMipmapMode mode);
static VkSamplerAddressMode ToVkAddressMode(SamplerWrapMode mode);
static VkCompareOp ToVkCompareOp(SamplerCompareFunc func);
template <class SamplerSource>
ResolvedBorderColor ResolveBorderColor(const SamplerSource& sampler, TextureInternalFormat format) const;
// The anisotropy Vulkan will actually apply: 1.0 (i.e. disabled) unless the feature is on and
// the sampler filters linearly both ways, otherwise the GL request clamped to the device limit.
// GL happily carries GL_TEXTURE_MAX_ANISOTROPY on a NEAREST sampler (Blaze3D's blocks do exactly
// that) while Vulkan forbids anisotropyEnable there, so the GL value must never be forwarded raw.
template <class SamplerSource>
Float ResolveEffectiveMaxAnisotropy(const SamplerSource& sampler,
Bool forceNearestFiltering) const;
VkDevice m_device = VK_NULL_HANDLE;
const VulkanRendererConfig* m_config = nullptr;
Bool m_samplerAnisotropySupported = false;
Float m_maxSamplerAnisotropy = 1.0f;
Bool m_customBorderColorSupported = false;
Uint32 m_maxCustomBorderColorSamplers = 0;
// Live cache entries carrying a custom border colour. Kept in step with the entries themselves
// in exactly the three places one can appear or disappear: creation, the OnFrameBoundary sweep,
// and Shutdown.
Uint32 m_customBorderColorSamplerCount = 0;
UnorderedMap<Uint64, SamplerCacheEntry> m_samplers;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameBoundaryCounter = 0;
// Moves on every vkDestroySampler of a cached entry (sweep and Shutdown).
Uint64 m_destroyEpoch = 0;
static inline thread_local XXH64_state_t* m_hashState = XXH64_createState();
};
} // namespace MobileGL::MG_Backend::DirectVulkan
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -16,15 +16,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(initInfo.frameCount > 0, "VkTimerQueryManager::Initialize requires non-zero frame count");
if (initInfo.timestampValidBits == 0 || initInfo.timestampPeriodNs <= 0.0f || initInfo.slotsPerPool == 0) {
MGLOG_W_ONCE("VkTimerQueryManager: timestamps unsupported (validBits=%u, period=%f, slots=%u)",
initInfo.timestampValidBits, initInfo.timestampPeriodNs, initInfo.slotsPerPool);
initInfo.timestampValidBits, initInfo.timestampPeriodNs, initInfo.slotsPerPool);
return false;
}
m_device = initInfo.device;
m_timestampPeriodNs = initInfo.timestampPeriodNs;
m_validBitsMask = initInfo.timestampValidBits >= 64
? ~0ull
: ((1ull << initInfo.timestampValidBits) - 1ull);
m_validBitsMask = initInfo.timestampValidBits >= 64 ? ~0ull : ((1ull << initInfo.timestampValidBits) - 1ull);
m_slotsPerPool = initInfo.slotsPerPool;
m_pools.resize(initInfo.frameCount);
@@ -91,8 +89,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (poolState.cursor >= m_slotsPerPool) {
if (!poolState.exhaustionWarned) {
MGLOG_W_ONCE("VkTimerQueryManager: frame %u timestamp pool exhausted (%u slots); further timer queries "
"this frame fall back to the frontend path",
frameIndex, m_slotsPerPool);
"this frame fall back to the frontend path",
frameIndex, m_slotsPerPool);
poolState.exhaustionWarned = true;
}
return nullptr;
@@ -64,8 +64,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// vkCmdWriteTimestamp (valid both inside and outside a render pass).
// Returns null on pool exhaustion, with one warning per pool cycle; the
// frontend falls back gracefully on a null handle.
SharedPtr<TimestampRecord> WriteTimestamp(VkCommandBuffer commandBuffer, Uint32 frameIndex,
Uint64 frameSerial);
SharedPtr<TimestampRecord> WriteTimestamp(VkCommandBuffer commandBuffer, Uint32 frameIndex, Uint64 frameSerial);
// Non-blocking single-slot read (WITH_AVAILABILITY, no WAIT). Returns
// true once the record holds its raw ticks. Callers gate this on the
File diff suppressed because it is too large Load Diff
@@ -48,15 +48,15 @@ namespace MobileGL::MG_State::GLState {
} // namespace MobileGL::MG_State::GLState
namespace MobileGL::MG_Backend::DirectVulkan {
enum class DrawSetupAspect: Uint8 {
FramebufferObject = 1 << 0,
VertexArrayObject = 1 << 1,
UniformBuffer = 1 << 2,
VertexBuffer = 1 << 3,
IndexBuffer = 1 << 4,
enum class DrawSetupAspect : Uint8 {
FramebufferObject = 1 << 0,
VertexArrayObject = 1 << 1,
UniformBuffer = 1 << 2,
VertexBuffer = 1 << 3,
IndexBuffer = 1 << 4,
IndirectDrawBuffer = 1 << 5,
Viewport = 1 << 6,
Scissor = 1 << 7,
Viewport = 1 << 6,
Scissor = 1 << 7,
};
struct DrawCmdParam {
@@ -185,13 +185,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// queue submission order already puts every earlier draw's access to the old texels
// ahead of the copy. The preserve arm keeps the wait: it retires the old image.
// (rd12 Magma: the wait was ~1.8 ms/frame of fence stall, one per lightmap update.)
// P15 (WireFramebuffer.inc): runs an image's deferred clears now, outside any pass.
// P15 (WireFramebuffer.cpp): runs an image's deferred clears now, outside any pass.
void MaterializeWirePendingClears(VkTextureManager::TextureResource& resource);
Bool FlushWirePendingCommandsForTextureUpload() {
return !HasPendingRecordedWork() || FlushPendingCommands();
}
Bool FlushWirePendingCommandsForTextureUpload() { return !HasPendingRecordedWork() || FlushPendingCommands(); }
// P7 wave 2 package B3: the submission that will carry whatever is recorded NEXT.
// RetireWireObjects (WireDraw.inc) already tags future objects with exactly this, and
// RetireWireObjects (WireDraw.cpp) already tags future objects with exactly this, and
// for the same reason: a draw being set up now is not in any submission yet, so the
// index that names it is one past the counter. Deliberately NOT
// GetSyncPointSubmitIndex(), which answers m_submitCounter when nothing is recorded -
@@ -201,7 +199,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// from the wire stores MGPipeApplier().VerbIndirectBuffer / VerbIndirectParameterBuffer
// name. `offset` / `countOffset` are byte offsets into those stores; `stride` 0 means
// tightly packed. FALSE = this device cannot issue the COUNT form natively (the monolith
// arm's own condition, see the definition in WireDraw.inc) and the caller must read the
// arm's own condition, see the definition in WireDraw.cpp) and the caller must read the
// words on the CPU; TRUE = issued, or declined by name.
Bool DrawWireIndirectNative(GLenum mode, GLenum type, Uint64 offset, GLsizei drawcount, GLsizei stride,
Bool indexed, Bool counted, Uint64 countOffset);
@@ -236,7 +234,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// submission's sync_file in `*fence` (-1: none, and the work is waited out here), and - for
// a session that writes images - the boundary PublishSharedImageAccesses makes.
Bool ExportNativeFence(int* fence);
// The reading side's per-use hook (WireSharedImage.inc): every shared-image texture the
// The reading side's per-use hook (WireSharedImage.cpp): every shared-image texture the
// wire draw or dispatch being set up samples (VkTextureManager::NoteSharedImageUse) is
// acquired from the foreign family before it, when it moved or is not held.
void AcquireNotedSharedImages() {
@@ -265,32 +263,28 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// idleness guarantee - they are only bound through draws/dispatches that
// stamp the program entry) and purge the descriptor-set cache entries
// keyed by its now-recyclable VkDescriptorSetLayout handle.
void OnProgramEvicted(ProgramFactory::HashType programHash,
VkDescriptorSetLayout descriptorSetLayout) override;
void OnProgramEvicted(ProgramFactory::HashType programHash, VkDescriptorSetLayout descriptorSetLayout) override;
Bool SetupDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView = nullptr);
const DrawCmdParam& drawParams, const IndexBufferView* pIndexBufferView = nullptr);
enum class ScissoredClearPrep {
NotNeeded, // scissor covers the whole target — take the deferred whole-surface path instead
NoOp, // nothing to clear (degenerate target or empty scissor rect)
Ready, // a render pass is active; record vkCmdClearAttachments with the returned rect
NotNeeded, // scissor covers the whole target — take the deferred whole-surface path instead
NoOp, // nothing to clear (degenerate target or empty scissor rect)
Ready, // a render pass is active; record vkCmdClearAttachments with the returned rect
};
void Clear(GLbitfield mask);
void ClearBufferfi(GLenum buffer, GLint drawbuffer, GLfloat depth, GLint stencil);
void ClearBufferfv(GLenum buffer, GLint drawbuffer, const GLfloat* value);
void ClearBufferuiv(GLenum buffer, GLint drawbuffer, const GLuint* value);
void ClearBufferiv(GLenum buffer, GLint drawbuffer, const GLint* value);
void BlitFramebuffer(GLint srcX0, GLint srcY0, GLint srcX1, GLint srcY1,
GLint dstX0, GLint dstY0, GLint dstX1, GLint dstY1,
GLbitfield mask, GLenum filter);
void CopyTexSubImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset,
GLint x, GLint y, GLsizei width, GLsizei height);
void CopyImageSubData(const CopyImageEndpoint& srcEndpoint,
GLenum srcTarget, GLint srcLevel, GLint srcX, GLint srcY, GLint srcZ,
const CopyImageEndpoint& dstEndpoint,
GLenum dstTarget, GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ,
GLsizei srcWidth, GLsizei srcHeight, GLsizei srcDepth);
void BlitFramebuffer(GLint srcX0, GLint srcY0, GLint srcX1, GLint srcY1, GLint dstX0, GLint dstY0, GLint dstX1,
GLint dstY1, GLbitfield mask, GLenum filter);
void CopyTexSubImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y,
GLsizei width, GLsizei height);
void CopyImageSubData(const CopyImageEndpoint& srcEndpoint, GLenum srcTarget, GLint srcLevel, GLint srcX,
GLint srcY, GLint srcZ, const CopyImageEndpoint& dstEndpoint, GLenum dstTarget,
GLint dstLevel, GLint dstX, GLint dstY, GLint dstZ, GLsizei srcWidth, GLsizei srcHeight,
GLsizei srcDepth);
void GenerateMipmap(GLenum target);
void ReadPixels(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels);
// Server resource handle in, tightly packed owned bytes out. No frontend
@@ -329,14 +323,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkOffset2D* imageOffset, VkExtent2D* imageCopyExtent);
// Reorder a tightly packed block copied with MapDefaultFramebufferReadbackRect back into
// GL row order. The input block has swapped dimensions for 90/270 degree transforms.
static Bool RemapDefaultFramebufferReadback(const Uint8* rawPixels, Uint32 logicalWidth,
Uint32 logicalHeight,
VkSurfaceTransformFlagBitsKHR preTransform,
SizeT texelSize, Uint8* outPixels);
static Bool ConvertReadbackPixels(const Uint8* sourcePixels, VkFormat sourceFormat,
GLsizei width, GLsizei height, GLenum destinationFormat,
GLenum destinationType, SizeT destinationRowStride,
Uint8* destinationPixels);
static Bool RemapDefaultFramebufferReadback(const Uint8* rawPixels, Uint32 logicalWidth, Uint32 logicalHeight,
VkSurfaceTransformFlagBitsKHR preTransform, SizeT texelSize,
Uint8* outPixels);
static Bool ConvertReadbackPixels(const Uint8* sourcePixels, VkFormat sourceFormat, GLsizei width,
GLsizei height, GLenum destinationFormat, GLenum destinationType,
SizeT destinationRowStride, Uint8* destinationPixels);
void DispatchCompute(GLuint numGroupsX, GLuint numGroupsY, GLuint numGroupsZ);
void DispatchComputeIndirect(GLintptr indirect);
void MemoryBarrier(GLbitfield barriers);
@@ -538,7 +530,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// P7 wave 2-B2, CONTRACT-P7 §5.2 (B'): THE FRONTEND MEMBERS ARE PULL-BUILD ONLY. In a
// disaggregated build the monolith arm blits and generates depth mips with the baked
// modules (WireColorBlit.inc, WireDepthMipmap.inc), so no DirectVulkan object holds a
// modules (WireColorBlit.cpp, WireDepthMipmap.cpp), so no DirectVulkan object holds a
// ProgramObject, a ShaderObject or a SamplerObject any more - which is what takes
// ProgramObject::{AllocateLifetimeId, AttachShader, Link, ~ProgramObject},
// ShaderObject::{SetShaderSource, Compile, JoinPendingCompile, ReleaseCompileNode,
@@ -548,11 +540,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
//
// The structs stay, empty, so ShutdownBlitResources and the two `= {}` assignments read
// the same on both builds.
struct BlitResources {
};
struct BlitResources {};
struct DepthMipmapResources {
};
struct DepthMipmapResources {};
// A single-sample staging image for multisample-resolve blits that also have to change
// orientation. vkCmdResolveImage cannot flip (it takes one offset per side, not the
@@ -610,11 +600,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// `isWriteTarget` selects which side of the default framebuffer this resolves: a write
// re-points it at the image Present() acquired, a read keeps the current one (see
// m_defaultFramebufferImageIndex). Reads must pass false.
WireImage ResolveWireImage(const MG_Pipe::MGPFramebufferState& fbo,
const MG_Pipe::MGPSurface& surface, VkImageAspectFlags aspect,
Bool isWriteTarget = false);
WireImage ResolveWireImage(const MG_Pipe::MGPFramebufferState& fbo, const MG_Pipe::MGPSurface& surface,
VkImageAspectFlags aspect, Bool isWriteTarget = false);
void TransitionWireImage(WireImage& image, VkImageLayout layout, Bool coalesceWithLastFullBarrier = false);
// P15: WIRE DEFERRED CLEARS (WireFramebuffer.inc). A clear of images the open pass
// P15: WIRE DEFERRED CLEARS (WireFramebuffer.cpp). A clear of images the open pass
// holds is recorded inside it; a whole-image clear of others waits on the image
// (WirePendingClear) for the next pass that draws into it (LOAD_OP_CLEAR) or the first
// other use (MaterializeWirePendingClears).
@@ -637,26 +626,24 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool BeginWireDrawPassFolding(FrameContext::FrameData& frame, RenderPassEntry& begun, Uint64 variantKey,
const VkClearValue* values);
void ApplyWireFoldClearsInPass(FrameContext::FrameData& frame, RenderPassEntry& pass);
void ClearWireFramebuffer(const MG_Pipe::MGPFramebufferState& fbo,
const ClearAttachmentPayload& payload, GLint drawbuffer = -1);
void BlitWireFramebuffers(GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLbitfield mask, GLenum filter);
void ResolveWireDepthStencil(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1);
void ClearWireFramebuffer(const MG_Pipe::MGPFramebufferState& fbo, const ClearAttachmentPayload& payload,
GLint drawbuffer = -1);
void BlitWireFramebuffers(GLint sx0, GLint sy0, GLint sx1, GLint sy1, GLint dx0, GLint dy0, GLint dx1,
GLint dy1, GLbitfield mask, GLenum filter);
void ResolveWireDepthStencil(WireImage source, WireImage destination, GLint sx0, GLint sy0, GLint sx1,
GLint sy1, GLint dx0, GLint dy0, GLint dx1, GLint dy1);
// One aspect of a depth/stencil image through a buffer (source in TRANSFER_SRC_OPTIMAL,
// destination in TRANSFER_DST_OPTIMAL, one format). The returned buffer is named by the
// recorded commands and must be retired with them.
UniquePtr<VkBufferObject> CopyWireAspectThroughBuffer(VkImage srcImage, Uint32 srcLevel, Uint32 srcLayer,
GLint sx, GLint sy, const WireImage& destination,
GLint dx, GLint dy, Uint32 width, Uint32 height,
VkImageAspectFlags aspect, VkFormat format,
Bool mirrorY, const char* fatalDetail);
VkImageAspectFlags aspect, VkFormat format, Bool mirrorY,
const char* fatalDetail);
// A single-sample depth/stencil blit whose rectangles make it a plain copy, performed
// without vkCmdBlitImage (P7 gate 5). False when the shape is not a plain copy.
Bool CopyWireDepthStencilBlit(WireImage& source, WireImage& destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, VkImageAspectFlags aspect);
Bool CopyWireDepthStencilBlit(WireImage& source, WireImage& destination, GLint sx0, GLint sy0, GLint sx1,
GLint sy1, GLint dx0, GLint dy0, GLint dx1, GLint dy1, VkImageAspectFlags aspect);
// True when the rectangles give a multisample depth/stencil resolve a shape it declines
// (a scale, an X mirror, a Y mirror across two formats); the decline has then been
// logged and its INVALID_OPERATION recorded. BlitWireFramebuffers asks it BEFORE any
@@ -665,20 +652,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// ResolveWireDepthStencil (a stencil-only resolve without VK_EXT_shader_stencil_export
// on a device without VK_KHR_depth_stencil_resolve) is still decided per aspect, after
// colour ran; rule I (a) accepts that (docs/archive/notes/p7/magma-b2.md §2.10).
Bool DeclineWireDepthStencilResolveShape(const WireImage& source, const WireImage& destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1);
Bool DeclineWireDepthStencilResolveShape(const WireImage& source, const WireImage& destination, GLint sx0,
GLint sy0, GLint sx1, GLint sy1, GLint dx0, GLint dy0, GLint dx1,
GLint dy1);
PFN_vkCreateRenderPass2 m_wireCreateRenderPass2 = nullptr;
VkResolveModeFlags m_wireDepthResolveModes = 0;
VkResolveModeFlags m_wireStencilResolveModes = 0;
// P7 wave 2-B2, CONTRACT-P7 §3.2 (`multisample-blit-aspect`): the SECOND arm of the
// multisample depth/stencil resolve, for the device that does not carry
// VK_KHR_depth_stencil_resolve. One baked pass per aspect, sample zero, no frontend
// object - see WireMultisampleResolve.inc. `resolved` must be a single-sample image of
// object - see WireMultisampleResolve.cpp. `resolved` must be a single-sample image of
// the source's format and extent; the rect is expressed by the scissor.
Bool ResolveWireDepthStencilWithShader(WireImage source, WireImage resolved,
VkImageAspectFlags aspect, GLint sx0, GLint sy0,
Uint32 width, Uint32 height);
Bool ResolveWireDepthStencilWithShader(WireImage source, WireImage resolved, VkImageAspectFlags aspect,
GLint sx0, GLint sy0, Uint32 width, Uint32 height);
struct WireMultisampleResolveResources;
WireMultisampleResolveResources* m_wireMultisampleResolveResources = nullptr;
void DestroyWireMultisampleResolveResources();
@@ -695,7 +681,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// one entry point the import needs (PFN_vkGetAndroidHardwareBufferPropertiesANDROID).
Bool m_wireAhbImport = false;
void* m_wireGetAhbProperties = nullptr;
// YUV shared images (WireYuvImage.inc): the device enabled samplerYcbcrConversion, and the
// YUV shared images (WireYuvImage.cpp): the device enabled samplerYcbcrConversion, and the
// conversion pipelines made so far (one per buffer format and colour conversion).
Bool m_samplerYcbcrConversion = false;
struct WireYuvResources;
@@ -718,13 +704,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
UnorderedMap<Uint64, SharedImagePresentTarget> m_sharedImagePresentTargets;
void DestroySharedImagePresentTargets(Bool onlyDead);
// SHARED-IMAGE SYNCHRONISATION (WireSharedImage.inc). The sync_file fences other sessions
// SHARED-IMAGE SYNCHRONISATION (WireSharedImage.cpp). The sync_file fences other sessions
// publish become binary semaphores: a temporary SYNC_FD import is a WAIT, an exportable
// semaphore a SIGNAL, of the next frame submission - whichever of SubmitPendingCommandBuffer
// and Present's submit comes first, which is the one carrying what was recorded since.
// Each semaphore is recycled once the submission that used it has completed.
// m_sharedImageSyncFd: the device imports and exports SYNC_FD binary semaphores; without
// it the waits happen on the CPU and nothing is exported (see WireSharedImage.inc).
// it the waits happen on the CPU and nothing is exported (see WireSharedImage.cpp).
Bool m_sharedImageSyncFd = false;
// The session writes images - it ends frames at publishing flushes
// (PublishSharedImageAccesses), or it reached one through a framebuffer (ResolveWireImage):
@@ -806,33 +792,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void ArmWireDiscardStateReemit();
void ForgetDynamicStateAfterDiscardDraw();
void ArmWireE5RawReads();
public:
// The kit when the route is armed, else nullptr. VkTextureManager asks before a level copy.
const WireE5RawCopyKit* GetWireE5RawCopyKit() const {
return m_wireE5RawReads ? &m_wireE5RawCopyKit : nullptr;
}
const WireE5RawCopyKit* GetWireE5RawCopyKit() const { return m_wireE5RawReads ? &m_wireE5RawCopyKit : nullptr; }
private:
Bool BlitWireColorToDefault(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter);
Bool BlitWireColorImage(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter, Bool mipmap);
Bool BlitWireColorToDefault(WireImage source, WireImage destination, GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter);
Bool BlitWireColorImage(WireImage source, WireImage destination, GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter, Bool mipmap);
// P7 wave 2-B, CONTRACT-P7 §3.2 (`default-color-blit-shape`): the rotated arm. Moves the
// source region into an owned single-sample 2D image of a sampled-float format in the
// same size-compatibility class, then shader-blits THAT to the rotated default
// framebuffer - the shapes BlitWireColorImage itself cannot sample (an integer format, a
// 3D or array source) become shapes it can.
Bool BlitWireColorToDefaultThroughScratch(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1,
Bool BlitWireColorToDefaultThroughScratch(WireImage source, WireImage destination, GLint sx0, GLint sy0,
GLint sx1, GLint sy1, GLint dx0, GLint dy0, GLint dx1, GLint dy1,
GLenum filter);
struct WireColorBlitResources;
WireColorBlitResources* m_wireColorBlitResources = nullptr;
void DestroyWireColorBlitResources();
void ResetWireColorBlitFramePool(Uint32 frameIndex);
void RetireWireColorBlitTargets(Bool all);
// P7 wave 2-B, CONTRACT-P7 §5.1 (A): the baked depth-mip program (WireDepthMipmap.inc).
// P7 wave 2-B, CONTRACT-P7 §5.1 (A): the baked depth-mip program (WireDepthMipmap.cpp).
// One level of a depth chain, source and destination being two depth subresources of
// the same image; the caller loops. It owns no frontend object, which is what the
// monolith arm's GenerateDepthMipmapWithShader cannot say.
@@ -841,10 +824,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
WireDepthMipmapResources* m_wireDepthMipmapResources = nullptr;
void DestroyWireDepthMipmapResources();
// P7 wave 2-B (exit gate 3): the readback's own write-visibility barrier, recorded before
// the copy instead of inferred from the image's tracked layout. See WireFramebuffer.inc.
// the copy instead of inferred from the image's tracked layout. See WireFramebuffer.cpp.
void RecordWireReadbackWriteBarrier();
void ReadWirePixels(GLint x, GLint y, GLsizei width, GLsizei height,
GLenum format, GLenum type, void* pixels);
void ReadWirePixels(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels);
void GenerateWireMipmap();
Bool SetupWireDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams, const IndexBufferView* indices);
@@ -905,7 +887,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void DestroyWireDrawPassCacheEntry(WireDrawPassCacheEntry& entry);
Vector<WireDrawPassCacheEntry> m_wireDrawPassCache;
WireDrawPassKey m_wireDrawPassKey;
// P15: the open pass's attachment that is `image` (WireFramebuffer.inc's deferred clears).
// P15: the open pass's attachment that is `image` (WireFramebuffer.cpp's deferred clears).
static Uint32 OpenPassAttachmentOf(const WireImage& image, const Vector<WireDrawAttachmentKey>& attachments);
WireRenderPassCompatibilityTable m_wireRenderPassCompatibility;
// The interned compatibility id of m_wireDrawPass's attachments, or 0 when not known yet.
@@ -914,7 +896,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 m_wireDrawPassCompatibilityId = 0;
// P14: SetupWireDraw's per-draw containers, cleared and refilled every draw instead of
// allocated and freed (the heap was ~10% of a Minecraft frame on the wire arm). A nested
// SetupWireDraw - nothing does that today - would use a local set (WireDraw.inc).
// SetupWireDraw - nothing does that today - would use a local set (WireDraw.cpp).
struct WireDrawScratch {
VertexInputStateFactory::BackendVertexInputState input;
Vector<BufferSlice> vertexSlices;
@@ -989,19 +971,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool m_wireNativeIndirectDraw = false;
void DispatchWireCompute(GLuint x, GLuint y, GLuint z);
// P8-SV: glDispatchComputeIndirect on the wire arm - vkCmdDispatchIndirect from the
// dispatch-indirect store, as the monolith arm issues it (WireDraw.inc).
// dispatch-indirect store, as the monolith arm issues it (WireDraw.cpp).
void DispatchWireComputeIndirect(GLintptr offset);
// False: the device was lost and the session latched; the caller drops its draw/dispatch.
Bool RewindWireDescriptorSetsIfDue();
UniquePtr<RenderPassEntry> m_wireDrawPass;
Vector<VkImageView> m_wireDrawViews;
VkPipeline GetOrCreatePipelineWithInput(GLenum mode, const MagmaProgramSource& program,
const ProgramFactory::VkProgramObject& programObj, ProgramFactory::CompileOptionFlags transformFlags,
const VertexInputStateFactory::BackendVertexInputState& vis, const RenderPassEntry& renderPassEntry,
Bool primitiveRestartEnable, Bool layeredFramebuffer, Uint64 wireRenderPassCompatibilityId = 0);
const ProgramFactory::VkProgramObject& programObj,
ProgramFactory::CompileOptionFlags transformFlags,
const VertexInputStateFactory::BackendVertexInputState& vis,
const RenderPassEntry& renderPassEntry, Bool primitiveRestartEnable,
Bool layeredFramebuffer, Uint64 wireRenderPassCompatibilityId = 0);
void CopyWireFramebufferToTexture(GLenum target, GLint level, GLint xoffset, GLint yoffset,
GLint x, GLint y, GLsizei width, GLsizei height);
void CopyWireFramebufferToTexture(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y,
GLsizei width, GLsizei height);
// ---- Submission fence tracking (GL sync objects) ----
// One record per vkQueueSubmit still in flight, in ascending submit
// order. Present/readback submissions reference the frame slot's
@@ -1276,7 +1260,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the pipeline-state value hash, neither of which moves when glBeginTransformFeedback is
// called, so a dynamic input here would hand back a stale VkPipeline.
VkProvokingVertexModeEXT SelectProvokingVertexMode(VkPrimitiveTopology topology,
Bool capturesXfbFromGeometryStage) const;
Bool capturesXfbFromGeometryStage) const;
// VK_EXT_vertex_attribute_divisor: without it every non-zero glVertexAttribDivisor
// behaves as 1, because that is all Vulkan's instance input rate can express.
Bool m_vertexAttributeDivisorEnabled = false;
@@ -1310,9 +1294,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Wraps a recorded draw with BeginTransformFeedbackEXT/EndTransformFeedbackEXT
// when GL transform feedback is active; binds capture buffers on demand.
Bool BeginXfbCaptureForDraw(FrameContext::FrameData& frame);
Bool BeginXfbCaptureWithBuffers(FrameContext::FrameData& frame, Uint32 count,
const VkBuffer* buffers, const VkDeviceSize* offsets,
const VkDeviceSize* sizes);
Bool BeginXfbCaptureWithBuffers(FrameContext::FrameData& frame, Uint32 count, const VkBuffer* buffers,
const VkDeviceSize* offsets, const VkDeviceSize* sizes);
Bool BeginWireXfbCaptureForDraw(FrameContext::FrameData& frame);
Uint32 m_currentDrawXfbBufferCount = 0;
Uint32 m_currentDrawXfbBufferMask = 0;
@@ -1378,8 +1361,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// tessellationShader was enabled at device creation (it is taken whenever the
// device advertises it); gates the probe's PATCHES shape.
Bool m_tessellationShaderFeatureEnabled = false;
MG_Util::SelfTest::PrimGenRerouteKind m_primGenRerouteKind =
MG_Util::SelfTest::PrimGenRerouteKind::None;
MG_Util::SelfTest::PrimGenRerouteKind m_primGenRerouteKind = MG_Util::SelfTest::PrimGenRerouteKind::None;
// The bring-up probe measured this device's stream query as counting draws made
// with no capture span open (the StreamCounts verdict) - so it counts the
// PAUSED-span ones too, through the stream slot they take when nothing is
@@ -1411,8 +1393,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// kind: 0 = PRIMITIVES_WRITTEN, 1 = PRIMITIVES_GENERATED.
Bool StartXfbQueryCapture(Uint32 kind);
void StopXfbQueryCapture(Uint32 kind, Vector<Uint32>& outSlots, Vector<Uint32>& outRerouteSlots);
Bool ResolveXfbQueryResult(const Vector<Uint32>& slots, const Vector<Uint32>& rerouteSlots,
Bool wantGenerated, Uint64& outPrimitives);
Bool ResolveXfbQueryResult(const Vector<Uint32>& slots, const Vector<Uint32>& rerouteSlots, Bool wantGenerated,
Uint64& outPrimitives);
private:
void BeginXfbQueryForDraw(VkCommandBuffer commandBuffer, Bool xfbActive);
@@ -1632,8 +1614,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// pass renders unflipped). Keyed on BOTH inputs; missing the FBO bit shipped
// an upside-down default-framebuffer pass after any render-to-texture
// (minecraft-1.17-main-menu retrace, whole frame flipped).
VkSurfaceTransformFlagBitsKHR m_baseTransformFlagsPreTransform =
VK_SURFACE_TRANSFORM_FLAG_BITS_MAX_ENUM_KHR;
VkSurfaceTransformFlagBitsKHR m_baseTransformFlagsPreTransform = VK_SURFACE_TRANSFORM_FLAG_BITS_MAX_ENUM_KHR;
Bool m_baseTransformFlagsIsDefaultFbo = false;
Bool m_baseTransformFlagsKeyValid = false;
Uint32 m_baseTransformFlagsCache = 0;
@@ -1671,7 +1652,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// must not allocate.
Vector<VkVertexInputAttributeDescription> m_patchedAttributesScratch;
void CreateInstance();
VkResult SetupDebugMessenger();
VkResult DestroyDebugMessenger();
@@ -1708,8 +1688,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 viewportCount = 1);
void ApplyMultiViewportDynamicState(VkCommandBuffer commandBuffer, Uint32 viewportCount, const IntVec2& extent,
VkSurfaceTransformFlagBitsKHR preTransform, Bool isDefaultFbo);
VkRect2D ComputeGLScissorRect(Uint32 index, const IntVec2& extent,
VkSurfaceTransformFlagBitsKHR preTransform, Bool isDefaultFbo) const;
VkRect2D ComputeGLScissorRect(Uint32 index, const IntVec2& extent, VkSurfaceTransformFlagBitsKHR preTransform,
Bool isDefaultFbo) const;
// How many viewports a draw with this program rasterizes into: 1 unless the program
// assigns gl_ViewportIndex AND the device enabled multiViewport. Both the pipeline's
// baked viewportCount and the dynamic arrays come from this one answer, so they cannot
@@ -1728,7 +1708,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
public:
private:
void ShutdownSwapchain();
// Static functions
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,831 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererDraw.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "VulkanRendererInternal.h"
namespace MobileGL::MG_Backend::DirectVulkan {
// Keyed on the frontend's never-reused lifetime id, NOT on the GL name. The name is
// recycled the moment glDeleteTransformFeedbacks gives it back, so a name-keyed slot
// handed a brand-new object the counter group - and the m_xfbCountersValid /
// m_xfbLastSeenGeneration entries - of the object that died under that name.
//
// Slots are never handed back (there is no backend entry telling this renderer that a span
// closed - registering the EndTransformFeedback one would flip the "captures through its own
// driver" test FixupGsStripCaptureOrder makes of it), so once all sixteen are owned a new
// object has to take one over. The victim is chosen among owners with NO OPEN SPAN: an object
// whose span is closed, or which no longer exists at all, can never resume, so its counter
// bytes are dead. Least-recently-used ALONE would be exactly the wrong rule - GL only permits
// another object to capture while this one is PAUSED, so the paused span whose counters the
// slots exist to protect is by construction the least recently used entry. Taking a group over
// resets its counter state, because those bytes describe the previous owner's span.
Uint32 VulkanRenderer::CurrentXfbCounterSlot() {
constexpr Uint32 kNoSlot = static_cast<Uint32>(kXfbCounterObjectSlots);
const Uint64 identity = MG_Pipe::gPipeInputs->GetBoundTransformFeedbackLifetimeId();
MOBILEGL_ASSERT(identity != 0,
"transform feedback object reported the free-slot sentinel (0) as its identity - "
"every slot would then read as 'mine' without ever being claimed");
Uint32 freeSlot = kNoSlot;
for (Uint32 slot = 0; slot < kNoSlot; ++slot) {
if (m_xfbCounterSlotOwner[slot] == identity) {
m_xfbCounterSlotLastUse[slot] = ++m_xfbCounterSlotUseSerial;
return slot;
}
if (m_xfbCounterSlotOwner[slot] == 0 && freeSlot == kNoSlot) {
freeSlot = slot;
}
}
Uint32 slot = freeSlot;
if (slot == kNoSlot) {
for (Uint32 candidate = 0; candidate < kNoSlot; ++candidate) {
if (m_xfbCounterSlotOwner[candidate] != 0 &&
MG_Pipe::MGPipeApplier().StreamOutputSpans.count(m_xfbCounterSlotOwner[candidate]) != 0) {
continue;
}
if (slot == kNoSlot || m_xfbCounterSlotLastUse[candidate] < m_xfbCounterSlotLastUse[slot]) {
slot = candidate;
}
}
}
if (slot == kNoSlot) {
// Sixteen capture spans open at once. Whatever is taken loses its resume offset and
// restarts at byte 0 of its capture buffers, which is a wrong picture rather than a
// slow one - hence a report rather than a silent choice.
MGLOG_E_ONCE("CurrentXfbCounterSlot: all %zu counter groups belong to transform feedback objects "
"with an open capture span; the least recently used one is taken over and that span "
"will restart at offset 0 instead of appending",
kXfbCounterObjectSlots);
slot = 0;
for (Uint32 candidate = 1; candidate < kNoSlot; ++candidate) {
if (m_xfbCounterSlotLastUse[candidate] < m_xfbCounterSlotLastUse[slot]) {
slot = candidate;
}
}
}
m_xfbCounterSlotOwner[slot] = identity;
m_xfbCounterSlotLastUse[slot] = ++m_xfbCounterSlotUseSerial;
m_xfbCountersValid[slot] = false;
m_xfbLastSeenGeneration[slot] = 0;
return slot;
}
Bool VulkanRenderer::BeginXfbCaptureForDraw(FrameContext::FrameData& frame) {
if (!m_transformFeedbackFeatureEnabled || !MG_Pipe::gPipeInputs->IsLive() ||
!MG_Pipe::gPipeInputs->IsTransformFeedbackActive()) {
return false;
}
// A paused span captures nothing, and the counter buffers keep their values, so the
// next resumed draw appends exactly where the last captured one stopped - which is
// what pause/resume means (ARB_transform_feedback2).
if (MG_Pipe::gPipeInputs->IsTransformFeedbackPaused()) {
return false;
}
// The bound pipeline's last pre-rasterization stage has to have been declared with Xfb
// (VUID-vkCmdBeginTransformFeedbackEXT-None-04128). Everything above this line reads GL
// state, which cannot answer that: a program can be built as a capture variant and still
// end up with a module carrying no Xfb mode - the clip/XFB validation backstop rewinding
// past the decoration, or XfbCaptureDecoratePass resolving none of the requested varyings
// and changing nothing. Declining the span leaves the capture buffers untouched, which is
// the same nothing the driver would have written, without the undefined behaviour.
if (m_currentDrawXfbCaptureDeclined) {
MGLOG_E_ONCE("BeginXfbCaptureForDraw: declining the capture span - the bound program's last "
"pre-rasterization stage carries no Xfb execution mode, so recording one would be "
"undefined behaviour rather than a capture");
return false;
}
return BeginWireXfbCaptureForDraw(frame);
}
Bool VulkanRenderer::BeginXfbCaptureWithBuffers(FrameContext::FrameData& frame, Uint32 bufferCount,
const VkBuffer* buffers, const VkDeviceSize* offsets,
const VkDeviceSize* sizes) {
if (!m_xfbCounterBuffer.IsValid()) {
if (!m_xfbCounterBuffer.Create({
.allocator = m_allocator,
.size = 16 * kXfbCounterObjectSlots,
.usage =
VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_COUNTER_BUFFER_BIT_EXT | VK_BUFFER_USAGE_TRANSFER_DST_BIT,
.memoryUsage = VMA_MEMORY_USAGE_AUTO,
})) {
MGLOG_E_ONCE("BeginXfbCaptureForDraw: failed to create the counter buffer");
return false;
}
}
m_currentDrawXfbBufferCount = bufferCount;
m_currentDrawXfbBufferMask = 0;
for (Uint32 i = 0; i < bufferCount; ++i) {
if (buffers[i] == VK_NULL_HANDLE) continue;
m_currentDrawXfbBufferMask |= 1u << i;
s_vkCmdBindTransformFeedbackBuffersEXT(frame.commandBuffer, i, 1, buffers + i, offsets + i, sizes + i);
}
const Uint32 counterSlot = CurrentXfbCounterSlot();
const Uint64 generation = MG_Pipe::gPipeInputs->GetTransformFeedbackGeneration();
const Bool resume = m_xfbCountersValid[counterSlot] && m_xfbLastSeenGeneration[counterSlot] == generation;
m_xfbLastSeenGeneration[counterSlot] = generation;
VkBuffer counterBuffers[4] = {};
VkDeviceSize counterOffsets[4] = {};
for (SizeT i = 0; i < bufferCount; ++i) {
if (!(m_currentDrawXfbBufferMask & (1u << i))) continue;
counterBuffers[i] = m_xfbCounterBuffer.GetHandle();
counterOffsets[i] = static_cast<VkDeviceSize>(counterSlot) * 16 + static_cast<VkDeviceSize>(i) * 4;
}
if (resume) {
s_vkCmdBeginTransformFeedbackEXT(frame.commandBuffer, 0, static_cast<Uint32>(bufferCount), counterBuffers,
counterOffsets);
} else {
s_vkCmdBeginTransformFeedbackEXT(frame.commandBuffer, 0, 0, nullptr, nullptr);
}
return true;
}
void VulkanRenderer::EndXfbCaptureForDraw(FrameContext::FrameData& frame, Bool began) {
if (!began) {
return;
}
// Exactly the targets used by this draw's Begin, independent of frontend
// objects and of whether the span came from the wire or monolith state.
const Uint32 bufferCount = m_currentDrawXfbBufferCount;
const Uint32 counterSlot = CurrentXfbCounterSlot();
VkBuffer counterBuffers[4] = {};
VkDeviceSize counterOffsets[4] = {};
for (SizeT i = 0; i < bufferCount; ++i) {
if (!(m_currentDrawXfbBufferMask & (1u << i))) continue;
counterBuffers[i] = m_xfbCounterBuffer.GetHandle();
counterOffsets[i] = static_cast<VkDeviceSize>(counterSlot) * 16 + static_cast<VkDeviceSize>(i) * 4;
}
s_vkCmdEndTransformFeedbackEXT(frame.commandBuffer, 0, static_cast<Uint32>(bufferCount), counterBuffers,
counterOffsets);
m_xfbCountersValid[counterSlot] = true;
m_xfbWritesPendingVisibility = true;
}
// GL makes transform feedback results visible to every later command on their own, with no
// glMemoryBarrier in between - unlike shader storage writes, which is why the barrier the
// Vulkan memory model requires has to be supplied here rather than by the application. It
// cannot be recorded where the write happens (inside the capturing draw's render pass, which
// declares no self-dependency), so it is emitted at the next point that could read the
// captured buffer: the following draw, or a readback.
void VulkanRenderer::MakeXfbWritesVisible() {
if (!m_xfbWritesPendingVisibility) {
return;
}
m_xfbWritesPendingVisibility = false;
auto& frame = m_frameContext.GetCurrent();
if (!frame.isCommandRecording) {
m_frameContext.BeginCommandRecording();
}
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
VkMemoryBarrier memoryBarrier{};
memoryBarrier.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER;
memoryBarrier.srcAccessMask =
VK_ACCESS_TRANSFORM_FEEDBACK_WRITE_BIT_EXT | VK_ACCESS_TRANSFORM_FEEDBACK_COUNTER_WRITE_BIT_EXT;
// Every way a captured buffer can be read back: replayed as vertex attributes or indices
// by glDrawTransformFeedback, sampled through a uniform or storage binding, sourced as an
// indirect command, copied out, or mapped.
memoryBarrier.dstAccessMask =
VK_ACCESS_VERTEX_ATTRIBUTE_READ_BIT | VK_ACCESS_INDEX_READ_BIT | VK_ACCESS_UNIFORM_READ_BIT |
VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_INDIRECT_COMMAND_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT |
VK_ACCESS_HOST_READ_BIT | VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_TRANSFORM_FEEDBACK_COUNTER_READ_BIT_EXT;
vkCmdPipelineBarrier(frame.commandBuffer, VK_PIPELINE_STAGE_TRANSFORM_FEEDBACK_BIT_EXT,
VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, 0, 1, &memoryBarrier, 0, nullptr, 0, nullptr);
}
void VulkanRenderer::DrawArrays(const DrawCmd& payload) {
auto& frame = m_frameContext.GetCurrent();
if (!SetupDraw(frame, payload.mode, 0, payload.params)) {
return;
}
MOBILEGL_ASSERT(frame.isCommandRecording, "%s: frame recording was not started", __func__);
VkCommandBuffer& commandBuffer = frame.commandBuffer;
const Bool xfbActive = BeginXfbCaptureForDraw(frame);
BeginXfbQueryForDraw(commandBuffer, xfbActive);
const Bool occlusionActive = BeginOcclusionForDraw(commandBuffer);
vkCmdDraw(commandBuffer, payload.params.vertexCount, payload.params.instanceCount, payload.params.firstVertex,
payload.params.firstInstance);
EndOcclusionForDraw(commandBuffer, occlusionActive);
EndXfbCaptureForDraw(frame, xfbActive);
EndXfbQueryForDraw(commandBuffer);
}
Bool VulkanRenderer::StartOcclusionQueryCapture() {
if (!m_hostQueryResetEnabled || s_vkResetQueryPool == nullptr) {
return false;
}
if (m_occlusionQueryPool == VK_NULL_HANDLE) {
VkQueryPoolCreateInfo poolInfo{};
poolInfo.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO;
poolInfo.queryType = VK_QUERY_TYPE_OCCLUSION;
poolInfo.queryCount = kOcclusionQuerySlots;
if (vkCreateQueryPool(m_device, &poolInfo, nullptr, &m_occlusionQueryPool) != VK_SUCCESS) {
MGLOG_E_ONCE("StartOcclusionQueryCapture: vkCreateQueryPool failed");
m_occlusionQueryPool = VK_NULL_HANDLE;
return false;
}
s_vkResetQueryPool(m_device, m_occlusionQueryPool, 0, kOcclusionQuerySlots);
}
m_occlusionActiveSlots.clear();
m_occlusionCaptureActive = true;
return true;
}
void VulkanRenderer::StopOcclusionQueryCapture(Vector<Uint32>& outSlots) {
outSlots = Move(m_occlusionActiveSlots);
m_occlusionActiveSlots.clear();
m_occlusionCaptureActive = false;
}
Bool VulkanRenderer::ResolveOcclusionQueryResult(const Vector<Uint32>& slots, Uint64& outSamples) {
outSamples = 0;
if (slots.empty()) {
return true;
}
if (m_occlusionQueryPool == VK_NULL_HANDLE) {
return true;
}
auto& frame = m_frameContext.GetCurrent();
if (frame.isCommandRecording) {
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
if (!SubmitReadbackCommandsAndWait(frame)) {
return false;
}
}
for (const Uint32 slot : slots) {
Uint64 value = 0;
const VkResult result =
vkGetQueryPoolResults(m_device, m_occlusionQueryPool, slot, 1, sizeof(value), &value, sizeof(value),
VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WAIT_BIT);
if (result == VK_SUCCESS) {
outSamples += value;
}
s_vkResetQueryPool(m_device, m_occlusionQueryPool, slot, 1);
}
return true;
}
Bool VulkanRenderer::StartXfbQueryCapture(Uint32 kind) {
if (!m_xfbQueriesSupported || !m_hostQueryResetEnabled || s_vkResetQueryPool == nullptr ||
s_vkCmdBeginQueryIndexedEXT == nullptr || kind > 1) {
return false;
}
if (m_xfbQueryPool == VK_NULL_HANDLE) {
VkQueryPoolCreateInfo poolInfo{};
poolInfo.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO;
poolInfo.queryType = VK_QUERY_TYPE_TRANSFORM_FEEDBACK_STREAM_EXT;
poolInfo.queryCount = kXfbQuerySlots;
if (vkCreateQueryPool(m_device, &poolInfo, nullptr, &m_xfbQueryPool) != VK_SUCCESS) {
MGLOG_E_ONCE("StartXfbQueryCapture: vkCreateQueryPool failed");
m_xfbQueryPool = VK_NULL_HANDLE;
return false;
}
s_vkResetQueryPool(m_device, m_xfbQueryPool, 0, kXfbQuerySlots);
}
// The reroute pool, on the first GENERATED span that needs it. A creation
// failure disarms rather than failing the capture: the stream path still
// answers (with the driver's defect), which beats answering nothing.
if (kind == 1 && m_primGenRerouteKind != MG_Util::SelfTest::PrimGenRerouteKind::None &&
m_primGenReroutePool == VK_NULL_HANDLE) {
VkQueryPoolCreateInfo poolInfo{};
poolInfo.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO;
poolInfo.queryCount = kXfbQuerySlots;
if (m_primGenRerouteKind == MG_Util::SelfTest::PrimGenRerouteKind::PrimitivesGeneratedExt) {
// The query Vulkan defines for this GL target; counts vertex stream 0
// when begun with plain vkCmdBeginQuery.
poolInfo.queryType = VK_QUERY_TYPE_PRIMITIVES_GENERATED_EXT;
} else {
poolInfo.queryType = VK_QUERY_TYPE_PIPELINE_STATISTICS;
// The clipping-stage INVOCATION counter: one per primitive reaching
// primitive clipping (GL's CLIPPING_INPUT_PRIMITIVES) - post-tess/GS,
// pre-clip, and per spec still counted under rasterizer discard, which
// is exactly the set GL_PRIMITIVES_GENERATED is defined over. The
// stage's OUTPUT count (CLIPPING_PRIMITIVES_BIT) would be wrong:
// clipping may drop or split primitives.
poolInfo.pipelineStatistics = VK_QUERY_PIPELINE_STATISTIC_CLIPPING_INVOCATIONS_BIT;
}
if (vkCreateQueryPool(m_device, &poolInfo, nullptr, &m_primGenReroutePool) != VK_SUCCESS) {
MGLOG_E_ONCE("StartXfbQueryCapture: reroute pool creation failed; the "
"PRIMITIVES_GENERATED reroute is disarmed and XFB-inactive draws keep "
"the stream query");
m_primGenReroutePool = VK_NULL_HANDLE;
m_primGenRerouteKind = MG_Util::SelfTest::PrimGenRerouteKind::None;
} else {
s_vkResetQueryPool(m_device, m_primGenReroutePool, 0, kXfbQuerySlots);
}
}
m_xfbQueryActiveSlots[kind].clear();
m_xfbQueryCaptureActive[kind] = true;
if (kind == 1) {
m_primGenRerouteActiveSlots.clear();
}
return true;
}
Bool VulkanRenderer::ArePausedDrawsGpuCounted() const {
// Exactly the gate BeginXfbQueryForDraw applies per draw, so a span told "armed"
// really does get a reroute slot for every draw with no open capture - a paused
// span's draws included.
const Bool rerouteArmed = m_primGenRerouteKind != MG_Util::SelfTest::PrimGenRerouteKind::None &&
m_primGenReroutePool != VK_NULL_HANDLE;
// Otherwise the paused draw takes a stream slot, which is an exact count of it
// on a driver the probe measured as counting capture-less draws.
return rerouteArmed || m_primGenStreamCountsXfbInactiveDraws;
}
void VulkanRenderer::StopXfbQueryCapture(Uint32 kind, Vector<Uint32>& outSlots, Vector<Uint32>& outRerouteSlots) {
if (kind > 1) {
return;
}
outSlots = Move(m_xfbQueryActiveSlots[kind]);
m_xfbQueryActiveSlots[kind].clear();
m_xfbQueryCaptureActive[kind] = false;
outRerouteSlots.clear();
if (kind == 1) {
outRerouteSlots = Move(m_primGenRerouteActiveSlots);
m_primGenRerouteActiveSlots.clear();
}
}
Bool VulkanRenderer::ResolveXfbQueryResult(const Vector<Uint32>& slots, const Vector<Uint32>& rerouteSlots,
Bool wantGenerated, Uint64& outPrimitives) {
outPrimitives = 0;
const Bool haveStreamSlots = !slots.empty() && m_xfbQueryPool != VK_NULL_HANDLE;
// Reroute slots only ever accumulate the GENERATED target (see
// BeginXfbQueryForDraw); WRITTEN never opens one.
const Bool haveRerouteSlots = wantGenerated && !rerouteSlots.empty() && m_primGenReroutePool != VK_NULL_HANDLE;
if (!haveStreamSlots && !haveRerouteSlots) {
return true;
}
auto& frame = m_frameContext.GetCurrent();
if (frame.isCommandRecording) {
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
if (!SubmitReadbackCommandsAndWait(frame)) {
return false;
}
}
if (haveStreamSlots) {
for (const Uint32 slot : slots) {
Uint64 pair[2] = {0, 0}; // {primitivesWritten, primitivesNeeded}
const VkResult result =
vkGetQueryPoolResults(m_device, m_xfbQueryPool, slot, 1, sizeof(pair), pair, sizeof(pair),
VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WAIT_BIT);
if (result == VK_SUCCESS) {
outPrimitives += pair[wantGenerated ? 1 : 0];
}
}
}
if (haveRerouteSlots) {
for (const Uint32 slot : rerouteSlots) {
// Both reroute pool kinds answer one 64-bit primitive count per slot.
Uint64 generated = 0;
const VkResult result =
vkGetQueryPoolResults(m_device, m_primGenReroutePool, slot, 1, sizeof(generated), &generated,
sizeof(generated), VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WAIT_BIT);
if (result == VK_SUCCESS) {
outPrimitives += generated;
}
}
}
return true;
}
void VulkanRenderer::BeginXfbQueryForDraw(VkCommandBuffer commandBuffer, Bool xfbActive) {
m_xfbQuerySlotOpen = false;
m_primGenRerouteSlotOpen = false;
if ((!m_xfbQueryCaptureActive[0] && !m_xfbQueryCaptureActive[1]) || m_xfbQueryPool == VK_NULL_HANDLE) {
return;
}
// Every draw with no OPEN capture is the stream query's silent case, and that
// includes a draw made while the GL span is merely PAUSED (the pause closes the
// capture, so BeginXfbCaptureForDraw already answered false for it). Paused
// draws are rerouted like any other: the frontend's CPU paused-primitive
// counter cannot stand in for them - it is written by only 3 of the ~15 draw
// entry points (never the instanced, indirect or multi-draw ones) and answers 0
// for GL_PATCHES by design, since the tessellator's amplification is not
// knowable on the CPU - which is exactly the CTS's shape. Double counting is
// prevented on the other side instead: a GENERATED span opened while this
// reroute is armed ignores that CPU counter entirely (see
// ArePausedDrawsGpuCounted and DirectVulkan.cpp's XfbGenerated resolve), so
// every XFB-inactive draw in the span is priced exactly once, by this pool.
const Bool rerouteGenerated = m_xfbQueryCaptureActive[1] &&
m_primGenRerouteKind != MG_Util::SelfTest::PrimGenRerouteKind::None &&
m_primGenReroutePool != VK_NULL_HANDLE && !xfbActive;
// The stream slot stays for WRITTEN whatever the reroute does (with capture
// inactive its primitivesWritten is 0, which is the correct WRITTEN answer),
// and for GENERATED wherever this draw is not rerouted - so one GL query span
// may accumulate stream slots (XFB-active draws) and reroute slots
// (XFB-inactive draws) side by side.
const Bool wantStreamSlot = m_xfbQueryCaptureActive[0] || (m_xfbQueryCaptureActive[1] && !rerouteGenerated);
if (wantStreamSlot) {
const Uint32 slot = m_xfbQuerySlotCursor;
m_xfbQuerySlotCursor = (m_xfbQuerySlotCursor + 1) % kXfbQuerySlots;
// Slots are never host-reset at read time (both GL targets may reference one
// slot); recycle them here instead.
s_vkResetQueryPool(m_device, m_xfbQueryPool, slot, 1);
s_vkCmdBeginQueryIndexedEXT(commandBuffer, m_xfbQueryPool, slot, 0, 0);
if (m_xfbQueryCaptureActive[0]) {
m_xfbQueryActiveSlots[0].push_back(slot);
}
if (m_xfbQueryCaptureActive[1] && !rerouteGenerated) {
m_xfbQueryActiveSlots[1].push_back(slot);
}
m_xfbQuerySlotOpen = true;
m_xfbQueryOpenSlot = slot;
}
if (rerouteGenerated) {
// Latched at INFO on purpose: it is the pinned integration lane's arming
// observable (the shape UnlocatedIoBlockScenario asserts), and the builds
// CI runs compile INFO in.
MGLOG_I_ONCE("PRIMITIVES_GENERATED reroute engaged: an XFB-inactive draw accumulates "
"through the %s pool",
m_primGenRerouteKind == MG_Util::SelfTest::PrimGenRerouteKind::PrimitivesGeneratedExt
? "VK_QUERY_TYPE_PRIMITIVES_GENERATED_EXT"
: "clipping-invocations statistics");
const Uint32 slot = m_primGenRerouteSlotCursor;
m_primGenRerouteSlotCursor = (m_primGenRerouteSlotCursor + 1) % kXfbQuerySlots;
// Same recycle-at-begin discipline as the stream pool. Both pool kinds
// are begun with plain vkCmdBeginQuery (a PRIMITIVES_GENERATED_EXT
// query begun this way counts vertex stream 0).
s_vkResetQueryPool(m_device, m_primGenReroutePool, slot, 1);
vkCmdBeginQuery(commandBuffer, m_primGenReroutePool, slot, 0);
m_primGenRerouteActiveSlots.push_back(slot);
m_primGenRerouteSlotOpen = true;
m_primGenRerouteOpenSlot = slot;
}
}
void VulkanRenderer::EndXfbQueryForDraw(VkCommandBuffer commandBuffer) {
if (m_xfbQuerySlotOpen) {
s_vkCmdEndQueryIndexedEXT(commandBuffer, m_xfbQueryPool, m_xfbQueryOpenSlot, 0);
m_xfbQuerySlotOpen = false;
}
if (m_primGenRerouteSlotOpen) {
vkCmdEndQuery(commandBuffer, m_primGenReroutePool, m_primGenRerouteOpenSlot);
m_primGenRerouteSlotOpen = false;
}
}
Bool VulkanRenderer::BeginOcclusionForDraw(VkCommandBuffer commandBuffer) {
if (!m_occlusionCaptureActive || m_occlusionQueryPool == VK_NULL_HANDLE) {
return false;
}
const Uint32 slot = m_occlusionSlotCursor;
m_occlusionSlotCursor = (m_occlusionSlotCursor + 1) % kOcclusionQuerySlots;
// Slots recycle after their read; a wrapped-past unread slot is stale, so
// reset it here (host reset - the slot's prior GPU use has long retired).
s_vkResetQueryPool(m_device, m_occlusionQueryPool, slot, 1);
vkCmdBeginQuery(commandBuffer, m_occlusionQueryPool, slot,
m_occlusionQueryPreciseEnabled ? VK_QUERY_CONTROL_PRECISE_BIT : 0);
m_occlusionActiveSlots.push_back(slot);
return true;
}
void VulkanRenderer::EndOcclusionForDraw(VkCommandBuffer commandBuffer, Bool began) {
if (!began) {
return;
}
vkCmdEndQuery(commandBuffer, m_occlusionQueryPool, m_occlusionActiveSlots.back());
}
void VulkanRenderer::DrawElements(const DrawIndexedCmd& payload) {
auto& frame = m_frameContext.GetCurrent();
DrawCmdParam vertexRange{};
vertexRange.vertexCount =
payload.params.indexCount +
(payload.params.vertexOffset > 0 ? static_cast<Uint32>(payload.params.vertexOffset) : 0);
vertexRange.instanceCount = payload.params.instanceCount;
vertexRange.firstVertex = 0;
vertexRange.firstInstance = static_cast<Uint32>(payload.params.firstInstance);
vertexRange.baseVertex = payload.params.vertexOffset;
// Direct DrawElements fetches exactly the indices in its view, so vertex-stream
// conversion may bound its work by scanning them.
vertexRange.indexRangeIsExactView = true;
if (!SetupDraw(frame, payload.mode, DrawSetupAspect::IndexBuffer, vertexRange, &payload.indexBufferView)) {
return;
}
MOBILEGL_ASSERT(frame.isCommandRecording, "%s: frame recording was not started", __func__);
VkCommandBuffer& commandBuffer = frame.commandBuffer;
const Bool xfbActive = BeginXfbCaptureForDraw(frame);
BeginXfbQueryForDraw(commandBuffer, xfbActive);
const Bool occlusionActive = BeginOcclusionForDraw(commandBuffer);
vkCmdDrawIndexed(commandBuffer, payload.params.indexCount, payload.params.instanceCount,
payload.params.firstIndex, payload.params.vertexOffset, payload.params.firstInstance);
EndOcclusionForDraw(commandBuffer, occlusionActive);
EndXfbCaptureForDraw(frame, xfbActive);
EndXfbQueryForDraw(commandBuffer);
}
void VulkanRenderer::MultiDrawArrays(const MultiDrawCmd& payload) {
auto& frame = m_frameContext.GetCurrent();
// One state/pipeline setup covering the union of all sub-draw vertex ranges, then a vkCmdDraw
// per range -- mirrors MultiDrawElements.
DrawCmdParam vertexRange{};
for (Uint32 idraw = 0; idraw < payload.drawCount; ++idraw) {
vertexRange.vertexCount = std::max(vertexRange.vertexCount,
payload.pParams[idraw].firstVertex + payload.pParams[idraw].vertexCount);
vertexRange.instanceCount = std::max(vertexRange.instanceCount, payload.pParams[idraw].instanceCount);
vertexRange.firstInstance = std::max(vertexRange.firstInstance, payload.pParams[idraw].firstInstance);
}
if (!SetupDraw(frame, payload.mode, 0, vertexRange)) {
return;
}
MOBILEGL_ASSERT(frame.isCommandRecording, "%s: frame recording was not started", __func__);
EmitMultiDraw(frame.commandBuffer, payload.pParams, payload.drawCount);
}
// The tier-2 indirect batch uploads the param arrays as-is: the leading members of the
// renderer's draw-parameter structs are exactly Vulkan's indirect command layouts, and
// vkCmdDraw(Indexed)Indirect accepts any 4-aligned stride >= the command size, so the
// trailing CPU-side metadata rides along unread instead of forcing a repack.
static_assert(
sizeof(DrawIndexedCmdParam) == sizeof(VkDrawIndexedIndirectCommand) &&
offsetof(DrawIndexedCmdParam, indexCount) == offsetof(VkDrawIndexedIndirectCommand, indexCount) &&
offsetof(DrawIndexedCmdParam, instanceCount) == offsetof(VkDrawIndexedIndirectCommand, instanceCount) &&
offsetof(DrawIndexedCmdParam, firstIndex) == offsetof(VkDrawIndexedIndirectCommand, firstIndex) &&
offsetof(DrawIndexedCmdParam, vertexOffset) == offsetof(VkDrawIndexedIndirectCommand, vertexOffset) &&
offsetof(DrawIndexedCmdParam, firstInstance) == offsetof(VkDrawIndexedIndirectCommand, firstInstance),
"DrawIndexedCmdParam must alias VkDrawIndexedIndirectCommand for the tier-2 multi-draw upload");
static_assert(sizeof(DrawCmdParam) % 4 == 0 && sizeof(DrawCmdParam) >= sizeof(VkDrawIndirectCommand) &&
offsetof(DrawCmdParam, vertexCount) == offsetof(VkDrawIndirectCommand, vertexCount) &&
offsetof(DrawCmdParam, instanceCount) == offsetof(VkDrawIndirectCommand, instanceCount) &&
offsetof(DrawCmdParam, firstVertex) == offsetof(VkDrawIndirectCommand, firstVertex) &&
offsetof(DrawCmdParam, firstInstance) == offsetof(VkDrawIndirectCommand, firstInstance),
"DrawCmdParam must lead with VkDrawIndirectCommand for the tier-2 multi-draw upload");
void VulkanRenderer::EmitMultiDraw(VkCommandBuffer commandBuffer, const DrawCmdParam* pParams, Uint32 drawCount) {
if (drawCount == 0) {
return;
}
if (drawCount == 1) {
vkCmdDraw(commandBuffer, pParams[0].vertexCount, pParams[0].instanceCount, pParams[0].firstVertex,
pParams[0].firstInstance);
return;
}
// Tier 1: VK_EXT_multi_draw. vkCmdDrawMultiEXT shares one instanceCount/firstInstance
// across the whole batch, so the batch must be uniform in both (GL's glMultiDrawArrays
// always is: 1/0).
if (m_multiDrawAllowExt) {
Bool uniformInstances = true;
for (Uint32 idraw = 1; idraw < drawCount; ++idraw) {
if (pParams[idraw].instanceCount != pParams[0].instanceCount ||
pParams[idraw].firstInstance != pParams[0].firstInstance) {
uniformInstances = false;
break;
}
}
if (uniformInstances) {
static thread_local Vector<VkMultiDrawInfoEXT> infos;
infos.resize(drawCount);
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
infos[idraw].firstVertex = pParams[idraw].firstVertex;
infos[idraw].vertexCount = pParams[idraw].vertexCount;
}
for (Uint32 base = 0; base < drawCount; base += m_maxMultiDrawCount) {
const Uint32 chunk = std::min(drawCount - base, m_maxMultiDrawCount);
s_vkCmdDrawMultiEXT(commandBuffer, chunk, infos.data() + base, pParams[0].instanceCount,
pParams[0].firstInstance, sizeof(VkMultiDrawInfoEXT));
}
return;
}
}
// Tier 2: multiDrawIndirect - one vkCmdDrawIndirect over a transient command array.
// A sub-draw with firstInstance != 0 is illegal in an indirect command without the
// drawIndirectFirstInstance feature; such a batch falls to the unrolled tier.
if (m_multiDrawAllowIndirect) {
Bool firstInstanceLegal = m_drawIndirectFirstInstanceFeatureEnabled;
if (!firstInstanceLegal) {
firstInstanceLegal = true;
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
if (pParams[idraw].firstInstance != 0) {
firstInstanceLegal = false;
break;
}
}
}
const Uint32 maxIndirectCount = m_physicalDevice.properties.limits.maxDrawIndirectCount;
if (firstInstanceLegal && maxIndirectCount > 0) {
BufferSlice commandSlice{};
if (m_bufferManager.UploadTransient(
BufferKind::Indirect, m_frameContext.GetCurrentFrameIndex(), pParams,
static_cast<VkDeviceSize>(drawCount) * sizeof(DrawCmdParam), sizeof(Uint32), commandSlice)) {
for (Uint32 base = 0; base < drawCount; base += maxIndirectCount) {
const Uint32 chunk = std::min(drawCount - base, maxIndirectCount);
vkCmdDrawIndirect(commandBuffer, commandSlice.buffer,
commandSlice.offset + static_cast<VkDeviceSize>(base) * sizeof(DrawCmdParam),
chunk, sizeof(DrawCmdParam));
}
return;
}
// Transient arena refused the upload: fall through to the unrolled tier.
}
}
// Tier 3: unrolled loop, byte-identical fallback (and the only tier where a SPIR-V
// DrawIndex consumer sees 0 for every sub-draw instead of the sub-draw index).
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
vkCmdDraw(commandBuffer, pParams[idraw].vertexCount, pParams[idraw].instanceCount,
pParams[idraw].firstVertex, pParams[idraw].firstInstance);
}
}
void VulkanRenderer::EmitMultiDrawIndexed(VkCommandBuffer commandBuffer, const DrawIndexedCmdParam* pParams,
Uint32 drawCount) {
if (drawCount == 0) {
return;
}
if (drawCount == 1) {
vkCmdDrawIndexed(commandBuffer, pParams[0].indexCount, pParams[0].instanceCount, pParams[0].firstIndex,
pParams[0].vertexOffset, pParams[0].firstInstance);
return;
}
// Tier 1: VK_EXT_multi_draw. VkMultiDrawIndexedInfoEXT carries per-draw
// firstIndex/indexCount/vertexOffset (pVertexOffset = nullptr keeps the per-draw
// offsets), but instanceCount/firstInstance are batch-wide, so the batch must be
// uniform in both (GL's glMultiDrawElements* always is: 1/0).
if (m_multiDrawAllowExt) {
Bool uniformInstances = true;
for (Uint32 idraw = 1; idraw < drawCount; ++idraw) {
if (pParams[idraw].instanceCount != pParams[0].instanceCount ||
pParams[idraw].firstInstance != pParams[0].firstInstance) {
uniformInstances = false;
break;
}
}
if (uniformInstances) {
static thread_local Vector<VkMultiDrawIndexedInfoEXT> infos;
infos.resize(drawCount);
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
infos[idraw].firstIndex = pParams[idraw].firstIndex;
infos[idraw].indexCount = pParams[idraw].indexCount;
infos[idraw].vertexOffset = pParams[idraw].vertexOffset;
}
for (Uint32 base = 0; base < drawCount; base += m_maxMultiDrawCount) {
const Uint32 chunk = std::min(drawCount - base, m_maxMultiDrawCount);
s_vkCmdDrawMultiIndexedEXT(commandBuffer, chunk, infos.data() + base, pParams[0].instanceCount,
static_cast<Uint32>(pParams[0].firstInstance),
sizeof(VkMultiDrawIndexedInfoEXT), nullptr);
}
return;
}
}
// Tier 2: multiDrawIndirect - one vkCmdDrawIndexedIndirect over a transient command
// array (DrawIndexedCmdParam aliases VkDrawIndexedIndirectCommand, see static_assert).
if (m_multiDrawAllowIndirect) {
Bool firstInstanceLegal = m_drawIndirectFirstInstanceFeatureEnabled;
if (!firstInstanceLegal) {
firstInstanceLegal = true;
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
if (pParams[idraw].firstInstance != 0) {
firstInstanceLegal = false;
break;
}
}
}
const Uint32 maxIndirectCount = m_physicalDevice.properties.limits.maxDrawIndirectCount;
if (firstInstanceLegal && maxIndirectCount > 0) {
BufferSlice commandSlice{};
if (m_bufferManager.UploadTransient(BufferKind::Indirect, m_frameContext.GetCurrentFrameIndex(),
pParams,
static_cast<VkDeviceSize>(drawCount) * sizeof(DrawIndexedCmdParam),
sizeof(Uint32), commandSlice)) {
for (Uint32 base = 0; base < drawCount; base += maxIndirectCount) {
const Uint32 chunk = std::min(drawCount - base, maxIndirectCount);
vkCmdDrawIndexedIndirect(commandBuffer, commandSlice.buffer,
commandSlice.offset +
static_cast<VkDeviceSize>(base) * sizeof(DrawIndexedCmdParam),
chunk, sizeof(DrawIndexedCmdParam));
}
return;
}
}
}
// Tier 3: unrolled loop, byte-identical fallback (and the only tier where a SPIR-V
// DrawIndex consumer sees 0 for every sub-draw instead of the sub-draw index).
for (Uint32 idraw = 0; idraw < drawCount; ++idraw) {
vkCmdDrawIndexed(commandBuffer, pParams[idraw].indexCount, pParams[idraw].instanceCount,
pParams[idraw].firstIndex, pParams[idraw].vertexOffset, pParams[idraw].firstInstance);
}
}
void VulkanRenderer::MultiDrawElements(const MultiDrawIndexedCmd& payload) {
auto& frame = m_frameContext.GetCurrent();
DrawCmdParam vertexRange{};
for (Uint32 idraw = 0; idraw < payload.drawCount; ++idraw) {
vertexRange.vertexCount = std::max(vertexRange.vertexCount, payload.pParams[idraw].indexCount);
vertexRange.instanceCount = std::max(vertexRange.instanceCount, payload.pParams[idraw].instanceCount);
vertexRange.firstInstance =
std::max(vertexRange.firstInstance, static_cast<Uint32>(payload.pParams[idraw].firstInstance));
}
if (!SetupDraw(frame, payload.mode, DrawSetupAspect::IndexBuffer, vertexRange, &payload.indexBufferView)) {
return;
}
MOBILEGL_ASSERT(frame.isCommandRecording, "%s: frame recording was not started", __func__);
// Collapse contiguous sub-draw runs BEFORE tier dispatch: merging shrinks the
// param span every tier consumes (fewer VkMultiDrawIndexedInfoEXT entries, a
// smaller transient command array, fewer unrolled vkCmdDrawIndexed). Per-sub-draw
// command emission in the driver dominates a Sodium-shaped multi-draw
// (steady-state profile: >60% of the case inside the Vulkan driver's
// vkCmdDrawIndexed encoding for 132x32 sub-draws/frame), and a chunk
// renderer's sub-draws are runs of adjacent index ranges over one buffer.
// Two draws are one iff they concatenate to an identical index stream:
// - a LIST topology (points/lines/triangles). Strips/fans/loops would
// weld primitives across the seam.
// - the accumulated count ends on a primitive boundary, otherwise GL
// discards the dangling indices at the sub-draw's end but the merged
// stream would assemble them with the next sub-draw's indices.
// - primitive restart is off: with restart on, a sentinel mid-stream
// resets assembly, so a partial primitive before the seam would
// otherwise be discarded per sub-draw (same dangling-index argument).
// - identical baseVertex/instancing and firstIndex adjacency, so the
// merged range fetches exactly the two sub-draws' indices in order.
Uint32 mergeGranularity = 0;
switch (payload.mode) {
case GL_POINTS:
mergeGranularity = 1;
break;
case GL_LINES:
mergeGranularity = 2;
break;
case GL_TRIANGLES:
mergeGranularity = 3;
break;
default:
break;
}
if (mergeGranularity != 0) {
const RenderStateParameters& rsp = MG_Pipe::gPipeInputs->GetRenderStateParameters();
if (rsp.PrimitiveRestartEnabled || rsp.PrimitiveRestartFixedIndexEnabled) {
mergeGranularity = 0;
}
}
const DrawIndexedCmdParam* pParams = payload.pParams;
Uint32 drawCount = payload.drawCount;
static thread_local Vector<DrawIndexedCmdParam> mergedParams;
if (mergeGranularity != 0) {
mergedParams.clear();
mergedParams.reserve(drawCount);
Uint32 idraw = 0;
while (idraw < drawCount) {
DrawIndexedCmdParam head = pParams[idraw];
++idraw;
if (head.indexCount == 0) {
continue; // draws nothing, contributes nothing to a run
}
if (head.instanceCount == 1) {
while (idraw < drawCount) {
const DrawIndexedCmdParam& next = pParams[idraw];
if (next.indexCount == 0) {
++idraw;
continue;
}
if (head.indexCount % mergeGranularity != 0 || next.instanceCount != 1 ||
next.vertexOffset != head.vertexOffset || next.firstInstance != head.firstInstance ||
next.firstIndex != head.firstIndex + head.indexCount ||
head.indexCount + next.indexCount < head.indexCount) {
break;
}
head.indexCount += next.indexCount;
++idraw;
}
}
mergedParams.push_back(head);
}
pParams = mergedParams.data();
drawCount = static_cast<Uint32>(mergedParams.size());
}
EmitMultiDrawIndexed(frame.commandBuffer, pParams, drawCount);
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -0,0 +1,437 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererInternal.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "RenderPassGuard.h"
#include "VulkanRenderer.h"
#include "SubmitFencePrefix.h"
#include "MG_Util/X11/DisplayGuard.h"
#include "OffscreenSurfaceRoute.h"
#include "MG_Backend/DirectVulkan/SubgroupSupportPolicy.h"
#include "MG_Backend/ClientReadback.h"
#include "VertexInputStateFactory.h"
#include "VertexInputStateBuilder.h"
#include "MG_State/GLState/Core.h"
#include <MG_Backend/MGPipe/PipeInputs.h>
// P5c ev: the GPU-write announcement routes through the reverse channel (R2).
#include <MG_Impl/Pipe/ResourceTracker.h>
#include "MG_State/GLState/ProgramState/ProgramObject.h"
#include "MG_State/GLState/ProgramState/ShaderObject.h"
#include "MG_State/GLState/SamplerState/SamplerObject.h"
#include "MG_State/GLState/TextureState/TextureObject.h"
#include "MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h"
#include "MG_Impl/GLImpl/Texture/GL_Texture.h"
// The generation window GenerateMipmap defines its chain in: one definition, shared with the
// frontend arm and with the wire carrier that publishes it (MGPMipPlan::LevelCount).
#include "MG_Impl/GLImpl/Texture/MipmapGenerationPlan.h"
#include "MG_Util/Converters/GLToMG/TextureEnumConverter.h"
// Only reached from an MGLOG_W, which the shipping INFO log level compiles out - so the
// missing include never broke a default build and did break every WARN/DEBUG-level one.
#include "MG_Util/Converters/MGToStr/TextureEnumConverter.h"
#include "MG_Util/Converters/MGToVk/RenderStateEnumConverter.h"
#include "MG_Util/Converters/MGToVk/TextureEnumConverter.h"
#include "MG_Util/Math/HalfFloat.h"
#include "MG_Util/Metrics/PipeStats.h"
#include "MG_Util/Metrics/TextureMetrics.h"
#include "MG_Util/SelfTest/PrimitivesGeneratedNoXfbProbe.h"
#include "MG_Util/Texture/PixelStoreProcessor.h"
#include <Config.h>
#include <MG_Backend/BackendObject.h>
// P7 wave 0: the seam WireFramebuffer.cpp's MagmaWireFatal and WireDraw.cpp's
// WireBufferLegacyFatal die through. Declared by MG_Pipe on purpose - see PipeSessionFail.h.
#include <MG_Pipe/PipeSessionFail.h>
// P5c (T5 / tx): the server's staged-texture shadow GenerateMipmap defines its chain on.
#include <MG_Backend/Record/StagedTextureStore.h>
// P7 wave 2 package B3: rule I's tally for WireDraw.cpp's silent draw drops.
#include "WireDeclineTally.h"
#include "WireColorBlitFilter.h"
#include "WireDepthResolveArm.h"
#include "WireCopySliceProbe.h"
#include "WireDiscardStateProbe.h"
#include "WireDepthResolveProbe.h"
#if MOBILEGL_BUILD_DISAGGREGATED
#include <MG_Remote/Server/ServerLoop.h>
// Shared images: the present side resolves the image it imports (WireSharedImage.cpp).
#include <MG_Remote/Server/SharedImageRegistry.h>
#endif
#include <algorithm>
#include <bit>
#include <cstdlib>
#include <cstring>
#include <vulkan/utility/vk_format_utils.h>
#include <vulkan/vulkan_core.h>
#ifdef __ANDROID__
#include <sys/system_properties.h>
#endif
#if MOBILEGL_BUILD_DISAGGREGATED && !defined(_WIN32)
#include <unistd.h> // close(): shared-image sync_file descriptors (WireSharedImage.cpp)
#endif
#if defined(__APPLE__)
#include <CoreGraphics/CoreGraphics.h>
#include <objc/message.h>
#include <objc/objc.h>
#include <objc/runtime.h>
#endif
namespace MobileGL::MG_Backend::DirectVulkan {
#if defined(__APPLE__)
namespace {
template <typename Fn>
Fn ObjcMsgSend() {
return reinterpret_cast<Fn>(objc_msgSend);
}
} // namespace
#endif
static Bool IsQuarterTurnPreTransform(VkSurfaceTransformFlagBitsKHR preTransform) {
return preTransform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR ||
preTransform == VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR;
}
// Redundant dynamic-state elimination for the per-draw hot path: within one
// command-buffer recording, a vkCmdSet* whose values already match what the
// command buffer holds is skipped. Valid because every PipelineFactory
// pipeline declares the same eight dynamic states, so the values persist
// across those pipeline binds; the shadow resets whenever a recording
// (re)begins, and whenever an auxiliary pipeline with a narrower dynamic
// set (blit, depth-mipmap) binds - their static state makes the
// corresponding dynamic values undefined per the spec.
struct DynamicStateShadow {
// Last graphics pipeline bound on the frame command buffer. Pipeline
// binds are command-buffer state (they survive render-pass boundaries),
// so the same reset points that invalidate dynamic state - recording
// (re)begin and the aux blit pipelines' raw binds - are exactly the
// points where this becomes unknown.
Bool graphicsPipelineValid = false;
VkPipeline graphicsPipeline = VK_NULL_HANDLE;
// Index/vertex buffer binds are command-buffer state too. Terrain
// sections and GUI quads share one sequential index buffer, and GUI
// batches often reuse a vertex arena buffer, so skipping identical
// rebinds removes a large share of per-draw driver calls.
Bool indexBindValid = false;
VkBuffer indexBuffer = VK_NULL_HANDLE;
VkDeviceSize indexOffset = 0;
VkIndexType indexType = VK_INDEX_TYPE_MAX_ENUM;
static constexpr Uint32 kMaxShadowedVertexBindings = 8;
// The wire buffer manager's store-destroy epoch the vertex/index halves were stamped at:
// a wire store can be destroyed mid-recording and its VkBuffer handle value re-minted, so
// an equal handle is only the same buffer while this epoch has not moved.
Uint64 bindStoreEpoch = 0;
Bool vertexBindValid = false;
Uint32 vertexBindingCount = 0;
VkBuffer vertexBuffers[kMaxShadowedVertexBindings] = {};
VkDeviceSize vertexOffsets[kMaxShadowedVertexBindings] = {};
Bool viewportValid = false;
VkViewport viewport{};
Bool scissorValid = false;
VkRect2D scissor{};
Bool blendConstantsValid = false;
Float blendConstants[4] = {0.0f, 0.0f, 0.0f, 0.0f};
Bool depthBiasValid = false;
Float depthBiasConstantFactor = 0.0f;
Float depthBiasSlopeFactor = 0.0f;
Float depthBiasClamp = 0.0f;
Bool lineWidthValid = false;
Float lineWidth = 0.0f;
Bool stencilValid = false;
Uint32 stencilFrontCompareMask = 0;
Uint32 stencilBackCompareMask = 0;
Uint32 stencilFrontWriteMask = 0;
Uint32 stencilBackWriteMask = 0;
Uint32 stencilFrontReference = 0;
Uint32 stencilBackReference = 0;
// glShadingRateEXT / glShadingRateCombinerOpsEXT, in RenderStateParameters' packed form
// (MGPackShadingRate). Only pipelines built with fragmentShadingRateDynamic carry the state.
Bool fragmentShadingRateValid = false;
Uint8 fragmentShadingRate = 0;
Uint8 fragmentShadingRateCombinerOp0 = 0;
Uint8 fragmentShadingRateCombinerOp1 = 0;
// Gate over the whole per-draw dynamic-state tail (viewport, scissor, blend
// constants, depth bias, line width, stencil) - see ApplyDynamicDrawStateTail.
// Every GL input of that tail lives in RenderState's value-shadowed parameters:
// each setter early-outs on an equal value and bumps the parameters version
// otherwise, and capability toggles (scissor test) bump it too. So an unchanged
// version + unchanged pass geometry means re-running the tail could only
// re-derive the exact values already applied on this command buffer. The
// remaining input, the swapchain pre-transform, cannot change mid-recording
// (a swapchain recreate retires the command buffer, and recording begin resets
// this whole shadow); the value key below pins it anyway.
Bool dynamicTailValid = false;
Uint dynamicTailParamsVersion = 0;
Int dynamicTailExtentX = 0;
Int dynamicTailExtentY = 0;
Bool dynamicTailIsDefaultFbo = false;
// VALUE key over the tail's inputs, as a second-level gate behind the version.
// The parameters version is ONE counter for all of RenderState, so anything that
// is not tail input - a GL_BLEND toggle, a glBlendFuncSeparate, a glColorMask -
// moves it and forced a full tail re-run. Blaze3D toggles blend around every
// batch, so that was a per-draw re-derivation of six dynamic states that could
// not have changed. Equal key => the six Apply* below would each re-derive the
// value their shadow already holds and emit nothing, so the tail is skippable.
//
// Complete input inventory of ApplyDynamicDrawStateTail, one line per reader
// (each accessor it replaces is a verified plain field read of the same
// RenderStateParameters field - RenderState.cpp):
// ApplyGLViewportState : Viewports[0], DepthRanges[0], + extent/isDefaultFbo/preTransform
// ApplyBlendConstants : BlendColor
// ApplyPolygonOffsetState : PolygonOffsetUnits, PolygonOffsetFactor, PolygonOffsetClamp
// ApplyLineWidthState : LineWidth (see the caveat below)
// ApplyStencilState : StencilStates[0..1].{ValueMask, WriteMask, Ref}
// scissor rect : ScissorTestEnabledMask bit 0, ScissorBoxes[0],
// + extent/isDefaultFbo/preTransform
// Caveat, unchanged from the version-only gate: ApplyLineWidthState also clamps
// to the ACTIVE BACKEND OBJECT's aliased line-width range. Those are device
// limits queried once at backend init and constant for the renderer's lifetime,
// so they are not part of the key (the version gate never covered them either).
struct DynamicTailKey {
Float viewport[4] = {0.0f, 0.0f, 0.0f, 0.0f};
Float depthRange[2] = {0.0f, 0.0f};
Float blendColor[4] = {0.0f, 0.0f, 0.0f, 0.0f};
Float polygonOffsetFactor = 0.0f;
Float polygonOffsetUnits = 0.0f;
Float polygonOffsetClamp = 0.0f;
Float lineWidth = 0.0f;
Uint32 stencilValueMask[2] = {0, 0};
Uint32 stencilWriteMask[2] = {0, 0};
Int stencilRef[2] = {0, 0};
Int scissorBox[4] = {0, 0, 0, 0};
Int extentX = 0;
Int extentY = 0;
Uint32 preTransform = 0;
Bool scissorEnabled = false;
Bool isDefaultFbo = false;
Bool operator==(const DynamicTailKey& other) const {
// NaN in any float input makes this false, which only costs a redundant
// tail run - never a skipped one.
for (Uint32 i = 0; i < 4; ++i) {
if (viewport[i] != other.viewport[i] || blendColor[i] != other.blendColor[i] ||
scissorBox[i] != other.scissorBox[i]) {
return false;
}
}
for (Uint32 i = 0; i < 2; ++i) {
if (depthRange[i] != other.depthRange[i] || stencilValueMask[i] != other.stencilValueMask[i] ||
stencilWriteMask[i] != other.stencilWriteMask[i] || stencilRef[i] != other.stencilRef[i]) {
return false;
}
}
return polygonOffsetFactor == other.polygonOffsetFactor &&
polygonOffsetUnits == other.polygonOffsetUnits &&
polygonOffsetClamp == other.polygonOffsetClamp && lineWidth == other.lineWidth &&
extentX == other.extentX && extentY == other.extentY && preTransform == other.preTransform &&
scissorEnabled == other.scissorEnabled && isDefaultFbo == other.isDefaultFbo;
}
};
DynamicTailKey dynamicTailKey{};
};
// Per Magma session (MagmaSession.h): it shadows the session's own command buffer. Inline,
// so every file of the renderer shares the one instance.
inline SessionLocal<DynamicStateShadow> g_dynamicStateShadow;
static void ResetDynamicStateShadow() {
*g_dynamicStateShadow = {};
}
// Whether the GL scissor test clips viewport index `index`. The scissor rectangles start
// all-zero, and all-zero there means "never written" (RenderState's constructor), not an
// empty box: GL's initial box is the whole window, so until the application's first
// glScissor an enabled test clips nothing. Read as a box it would clip EVERYTHING, so every
// reader of the box asks this first. The resolution matches the viewport's (ComputeGLViewport
// resolves a never-written viewport to the current framebuffer's extent).
static Bool GLScissorTestClips(Uint32 index = 0) {
const RenderStateParameters& parameters = MG_Pipe::gPipeInputs->GetRenderStateParameters();
const Uint32 bit = 1u << index;
return (parameters.ScissorTestEnabledMask & bit) != 0 && (parameters.ScissorBoxWrittenMask & bit) != 0;
}
// Vertex attribute locations are tracked in Uint32 bitmasks, so MAX_VERTEX_ATTRIBS is both the
// state-layer storage bound and the width of every mask below. Keep them in lockstep.
static constexpr Uint32 kMaxVertexAttribs =
static_cast<Uint32>(MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS);
static_assert(kMaxVertexAttribs <= 32, "Vertex attribute masks are Uint32");
// The loops below walk locations [0, kMaxVertexAttribs) and index programObj.vertexInputTypes with
// each one, so that array must be at least as wide.
static_assert(kMaxVertexAttribs <= ProgramFactory::VkProgramObject::kMaxVertexInputLocations,
"vertexInputTypes is indexed by vertex attribute location");
namespace {
static void GetImageTransitionSourceState(VkImageLayout oldLayout, VkPipelineStageFlags& outSrcStageMask,
VkAccessFlags& outSrcAccessMask) {
switch (oldLayout) {
case VK_IMAGE_LAYOUT_UNDEFINED:
case VK_IMAGE_LAYOUT_PRESENT_SRC_KHR:
outSrcStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
outSrcAccessMask = 0;
break;
case VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL:
outSrcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
outSrcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
break;
case VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL:
outSrcStageMask =
VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT;
outSrcAccessMask =
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
break;
case VK_IMAGE_LAYOUT_DEPTH_STENCIL_READ_ONLY_OPTIMAL:
case VK_IMAGE_LAYOUT_DEPTH_READ_ONLY_STENCIL_ATTACHMENT_OPTIMAL:
case VK_IMAGE_LAYOUT_DEPTH_ATTACHMENT_STENCIL_READ_ONLY_OPTIMAL:
outSrcStageMask = VK_PIPELINE_STAGE_ALL_GRAPHICS_BIT;
outSrcAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT;
break;
case VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL:
outSrcStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT;
outSrcAccessMask = VK_ACCESS_TRANSFER_READ_BIT;
break;
case VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL:
outSrcStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT;
outSrcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
break;
case VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL:
outSrcStageMask = VK_PIPELINE_STAGE_ALL_GRAPHICS_BIT;
outSrcAccessMask = VK_ACCESS_SHADER_READ_BIT;
break;
default:
outSrcStageMask = VK_PIPELINE_STAGE_ALL_COMMANDS_BIT;
outSrcAccessMask = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT;
break;
}
}
} // namespace
// A tightly packed depth/stencil readback, one aspect or both, in the client's own width.
// The two arms that read depth or stencil off a Vulkan image - the monolith
// ReadDepthStencilImageToClient and the wire ReadTextureImageWire - copy their aspect into a
// host-visible buffer and both owe the application the same bytes, so they share this one
// encoding rather than each carrying a copy of it. A copy is exactly the kind of pair where
// a fix landed on one side only: the signed ranges and GL_HALF_FLOAT were added to the wire
// caller and the monolith caller kept scaling a GL_SHORT depth into 65535, which put a 0.5
// depth at 0x8000 - a negative index.
static Bool PackDepthStencilTight(const Uint8* depth, const Uint8* stencil, VkFormat sourceFormat, SizeT count,
GLenum format, GLenum type, Vector<Uint8>& result) {
const Bool stencilOnly = format == GL_STENCIL_INDEX;
const auto readDepth = [&](SizeT i) -> Float {
if (sourceFormat == VK_FORMAT_D16_UNORM) {
Uint16 word{};
Memcpy(&word, depth + i * 2, sizeof(word));
return static_cast<Float>(word) / 65535.0f;
}
if (sourceFormat == VK_FORMAT_X8_D24_UNORM_PACK32 || sourceFormat == VK_FORMAT_D24_UNORM_S8_UINT) {
Uint32 word{};
Memcpy(&word, depth + i * 4, sizeof(word));
return static_cast<Float>(word & 0xffffffu) / 16777215.0f;
}
Float value{};
Memcpy(&value, depth + i * sizeof(value), sizeof(value));
return value;
};
SizeT size = 0;
switch (type) {
case GL_BYTE:
case GL_UNSIGNED_BYTE:
size = 1;
break;
case GL_SHORT:
case GL_UNSIGNED_SHORT:
case GL_HALF_FLOAT:
size = 2;
break;
case GL_INT:
case GL_UNSIGNED_INT:
case GL_FLOAT:
size = 4;
break;
case GL_UNSIGNED_INT_24_8:
size = 4;
break;
case GL_FLOAT_32_UNSIGNED_INT_24_8_REV:
size = 8;
break;
default:
return false;
}
const Bool packed = type == GL_UNSIGNED_INT_24_8 || type == GL_FLOAT_32_UNSIGNED_INT_24_8_REV;
if ((format == GL_DEPTH_STENCIL) != packed) return false;
result.resize(count * size);
for (SizeT i = 0; i < count; ++i) {
Uint8* dst = result.data() + i * size;
const Uint32 s = stencil ? stencil[i] : 0;
const Float d = stencilOnly ? 0.0f : readDepth(i);
switch (type) {
case GL_FLOAT: {
const Float value = stencilOnly ? static_cast<Float>(s) : d;
Memcpy(dst, &value, sizeof(value));
break;
}
case GL_HALF_FLOAT: {
const Uint16 value = MG_Util::EncodeFloatToHalfBits(stencilOnly ? static_cast<Float>(s) : d);
Memcpy(dst, &value, sizeof(value));
break;
}
case GL_UNSIGNED_INT_24_8: {
Uint32 depth24 = 0;
if (sourceFormat == VK_FORMAT_D24_UNORM_S8_UINT) {
Memcpy(&depth24, depth + i * 4, sizeof(depth24));
depth24 &= 0xffffffu;
} else {
depth24 = static_cast<Uint32>(std::llround(std::clamp<double>(d, 0.0, 1.0) * 16777215.0));
}
const Uint32 value = (depth24 << 8) | s;
Memcpy(dst, &value, sizeof(value));
break;
}
case GL_FLOAT_32_UNSIGNED_INT_24_8_REV:
Memcpy(dst, &d, sizeof(d));
Memcpy(dst + 4, &s, sizeof(s));
break;
default: {
const Bool signedType = type == GL_BYTE || type == GL_SHORT || type == GL_INT;
const Uint64 maximum =
signedType ? ((Uint64{1} << (size * 8 - 1)) - 1) : ((Uint64{1} << (size * 8)) - 1);
const Uint32 value = stencilOnly ? s
: static_cast<Uint32>(std::llround(std::clamp<double>(d, 0.0, 1.0) *
static_cast<double>(maximum)));
Memcpy(dst, &value, size);
break;
}
}
}
return true;
}
// P7 wave 0. THE MESSAGE IS UNCHANGED; where it goes is not. This funnel is how eleven of
// the fifteen P7-marked refusals die (the other four are WireDescriptorFatal's), and until now
// every one of them raised its own abort: no SessionFault frame for the peer, no
// SessionFaultCount() for exit gate S8, and nothing tying the hand-typed word to
// FatalFamilies.def. MGPipeSessionFail hands the line to the hook MG_Remote installs at
// server-role init, whose arm for UnmigratedVerb is SessionFail(MGFatalFamily::
// UnmigratedVerb, ...) - and with no hook installed it logs and aborts exactly as this did.
[[noreturn]] static void MagmaWireFatal(const char* detail) {
MG_Pipe::MGPipeSessionFail(MG_Pipe::MGPipeFatalFamily::UnmigratedVerb,
"MGPipe: Fatal{UnmigratedVerb, \"Magma:%s\"}", detail);
}
// AND THE SAME PROBLEM AGAIN FOR THE SHADER MULTISAMPLE RESOLVE (P7 wave 2-B2). lavapipe
// carries VK_KHR_create_renderpass2 AND VK_KHR_depth_stencil_resolve, so the extension arm
// is taken for every depth/stencil resolve this host can run and the shader arm - the one
// that exists for the device WITHOUT them, which is where the session used to die at
// `Magma:multisample-depth-resolve-capability` - would never execute under any lane. The
// knob drops the extension arm. It does not pretend the extension is missing anywhere else:
// nothing but the arm choice reads it. Inline, so the renderer reads the variable once.
inline Bool MagmaWireForcedShaderDepthResolve() {
static const Bool forced = [] {
const char* value = std::getenv("MGITEST_MAGMA_FORCE_SHADER_DEPTH_RESOLVE");
return value && *value && value[0] != '0';
}();
return forced;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,635 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererPresent.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "VulkanRendererInternal.h"
namespace MobileGL::MG_Backend::DirectVulkan {
void VulkanRenderer::Present() {
#if MOBILEGL_BUILD_DISAGGREGATED
// A lost device presents nothing (its session is latched and closing).
if (m_deviceLost && LatchWireDeviceLoss("present")) return;
// Present has its own submit path. Tag its last wire pass before that
// submission, rather than retaining it until a draw in the next frame.
RetireWireDrawPass();
#endif
if (m_swapchainObject.GetHandle() == VK_NULL_HANDLE || m_presentSuspended) {
// No usable swapchain: the window was zero-area at initialization, or
// presentation was suspended when the window minimized. Try to bring a
// swapchain up now that the window may have a real size; until then, drop
// this frame's recording instead of submitting - a submit would wait on a
// never-signaled acquire semaphore and reuse a still-signaled fence.
if (!RecreateSwapchain()) {
auto& suspendedFrame = m_frameContext.GetCurrent();
if (VkRenderPassManager::GetActiveRenderPass()) {
VkRenderPassManager::EndRenderPass(suspendedFrame.commandBuffer);
}
if (suspendedFrame.isCommandRecording) {
m_frameContext.EndCommandRecording();
}
m_frameContext.AbandonPreCommandRecording();
suspendedFrame.isCommandRecording = false;
suspendedFrame.hasCommandBufferRecorded = false;
InvalidatePipelineMemo();
#if MOBILEGL_BUILD_DISAGGREGATED
AbandonSharedImageReads();
#endif
// The dropped recording is never submitted, so once the fence
// poll shows the pre-suspension submissions complete the frame
// transients (descriptor sets, transient arenas, deferred
// releases, conversion caches) can rewind; without this a
// minimized-window app accumulates them for the whole
// suspension.
TryDrainFrameTransients();
// The application counted this swap; no image did.
m_swapchainObject.OnFrameDropped();
MGLOG_D("Present skipped: no usable swapchain (zero-area window)");
return;
}
m_presentSuspended = false;
// A slot's Present fence alone does not prove its older pooled
// flushes retired. Wait the full prefix before FrameContext may
// free its retired command buffers and reset the slot fence.
const Uint64 slotSubmit = m_frameContext.GetCurrent().lastSubmitIndex;
if (slotSubmit > m_completedSubmitCounter && !WaitForSubmitsUpTo(slotSubmit, UINT64_MAX)) {
if (LatchWireDeviceLoss("deferred-acquire-submit-wait")) return;
MagmaWireFatal("deferred-acquire-submit-wait");
}
const VkResult acquireResult =
m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
if (acquireResult == VK_SUBOPTIMAL_KHR) {
// Usable image with its acquire signal already armed; a rebuild is scheduled
// only if the surface genuinely no longer matches (see step 4 of Present).
m_swapchainResizeRequested = m_swapchainResizeRequested || SwapchainIsOutOfDate();
} else {
VK_VERIFY(acquireResult, "Present, deferred first WaitAndAcquireNextImage");
}
}
MOBILEGL_ASSERT(m_imageIndexAcquired < m_swapchainObject.GetImageCount(),
"Present, acquired image index out of range");
// A frame that never drew into the window still presents its image: what it recorded
// goes out first, unwaited, and the present's own submission waits for the release.
SubmitAheadOfAcquiredImage();
// A real presented frame is the canonical aging cadence; mid-frame drains
// count against this and only age when presents stop coming.
m_drainsSinceLastPresent = 0;
// Age the content-addressed caches on the same frame-boundary cadence. Each
// keeps its own internal 256-sweep gate, so the per-frame cost is one counter
// increment and compare per cache; entries used by this frame's still-
// unsubmitted recording were stamped this boundary (every command-buffer
// boundary drops the pipeline memo, so the first draw of each recording
// performs a real, stamping lookup) and can never age out.
m_programFactory->OnFrameBoundary();
if (m_pipelineFactory->OnFrameBoundary() > 0) {
InvalidatePipelineMemo(); // an aged-out pipeline may still be memoized
// A recreated pipeline could reuse a freed handle value and alias
// the bind-dedup shadow; force the next draw to re-bind.
g_dynamicStateShadow->graphicsPipelineValid = false;
}
m_samplerManager->OnFrameBoundary();
auto& frame = m_frameContext.GetCurrent();
auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass();
if (activeRenderPass) VkRenderPassManager::EndRenderPass(frame.commandBuffer);
#if MOBILEGL_BUILD_DISAGGREGATED
// The frame boundary of the shared images this frame read (WireSharedImage.cpp): handed
// back to the foreign family in this recording, and this submission signals the read
// fence their next writer waits for.
ReleaseHeldSharedImages();
if (m_sharedImageSyncFd && m_sharedImageReads.Pending() > 0 && m_sharedImageSignal == VK_NULL_HANDLE)
m_sharedImageSignal = TakeSharedImageSemaphore(/*exportable=*/true);
#endif
// Transition while this frame's recording is still open. A frame that
// rendered only into FBOs has no default-framebuffer render pass, and that
// pass's finalLayout is the only other thing that carries the swapchain
// image to PRESENT_SRC_KHR - so closing the buffer first, which made
// TransitionToPresent refuse to record, handed the image to
// vkQueuePresentKHR in the layout it was acquired in (UNDEFINED on a fresh
// swapchain). The SetImageLayout below then made the tracker's
// disagreement with reality permanent for that image index.
const auto acquiredImageLayout = m_swapchainObject.GetImageLayout(m_imageIndexAcquired);
m_frameContext.TransitionToPresent(m_swapchainObject.GetImage(m_imageIndexAcquired), acquiredImageLayout);
if (frame.isCommandRecording) {
m_frameContext.EndCommandRecording();
frame.hasCommandBufferRecorded = true;
InvalidatePipelineMemo(); // command-buffer boundary: drop the pipeline memo
}
m_frameContext.EndPreCommandRecordingIfOpen();
const Bool shouldSubmitCommandBuffer = frame.hasCommandBufferRecorded;
// 1) Submit current frame work (the pre-pass stream, when recorded,
// rides the same submission strictly ahead of the frame commands).
// Batched texture uploads go first: the frame's commands may sample
// images whose texels only exist in the open upload batch, and
// flushing here also bounds upload latency to one frame.
if (m_textureManager) {
m_textureManager->FlushPendingUploads();
}
auto submitPacket = m_frameContext.GetSubmitInfo(shouldSubmitCommandBuffer, m_imageIndexAcquired);
#if MOBILEGL_BUILD_DISAGGREGATED
if (LatchIfGpuHung("present-submit")) return;
SharedImageSubmitSync sharedImageSync;
AttachSharedImageSync(submitPacket.submitInfo, sharedImageSync);
const GpuProgressMarkers::Bracket bracket = m_progressMarkers.Acquire(m_completedSubmitCounter);
const VkResult presentSubmit = GpuProgressMarkers::SubmitBracketed(m_graphicsQueue, submitPacket.submitInfo,
frame.imageInFlightFence, bracket);
if (presentSubmit == VK_SUCCESS)
m_progressMarkers.Submitted(bracket, m_submitCounter + 1); // RegisterSubmit below
else
m_progressMarkers.Abandon(bracket);
#else
const VkResult presentSubmit =
vkQueueSubmit(m_graphicsQueue, 1, &submitPacket.submitInfo, frame.imageInFlightFence);
#endif
NoteDeviceLoss(presentSubmit, "Present vkQueueSubmit");
if (presentSubmit != VK_SUCCESS && LatchWireDeviceLoss("present-submit")) return;
VK_VERIFY(presentSubmit);
RegisterSubmit(frame.imageInFlightFence, /*pooledFence=*/false);
frame.lastSubmitIndex = m_submitCounter;
#if MOBILEGL_BUILD_DISAGGREGATED
CommitSharedImageSync(sharedImageSync);
PublishSharedImageReads();
#endif
frame.isCommandRecording = false;
frame.hasCommandBufferRecorded = false;
frame.hasPreCommandBufferRecorded = false;
m_swapchainObject.SetImageLayout(m_imageIndexAcquired, VK_IMAGE_LAYOUT_PRESENT_SRC_KHR);
// 2) Present current frame.
auto presentPacket = m_frameContext.GetPresentInfo(m_swapchainObject.GetHandle(), m_imageIndexAcquired);
auto result = vkQueuePresentKHR(m_presentQueue, &presentPacket.presentInfo);
if (result == VK_SUBOPTIMAL_KHR) {
// Suboptimal is not a reason to rebuild on its own: a driver may report it for a
// surface whose size and orientation still match what we built from (Android does
// this routinely), and rebuilding on it alone destroys every pipeline and
// reallocates the default framebuffer once per frame - flicker, then garbage.
// Defer to the surface-capabilities comparison below.
result = VK_SUCCESS;
}
if (result == VK_ERROR_OUT_OF_DATE_KHR) {
MGLOG_D("Present, vkQueuePresentKHR got %d, recreating swapchain", result);
if (!RecreateSwapchain()) {
// Window went zero-area (minimize) with the swapchain out of date:
// stop submitting/acquiring until it has a size again.
m_presentSuspended = true;
m_swapchainResizeRequested = false;
MGLOG_D("Present, zero-area window with out-of-date swapchain; suspending presentation");
return;
}
m_swapchainResizeRequested = false;
result = VK_SUCCESS;
}
NoteDeviceLoss(result, "vkQueuePresentKHR");
if (result != VK_SUCCESS && LatchWireDeviceLoss("present-queue")) return;
VK_VERIFY(result, "Present, vkQueuePresentKHR");
// EGL swap semantics: the presented color buffer's content is undefined the
// next time this image is acquired (EGL_BUFFER_DESTROYED, the default swap
// behaviour), and EVERY ancillary depth/stencil buffer's content is
// undefined after any swap. The render-pass manager turns the undefined
// attachments' next tile loads into LOAD_OP_DONT_CARE.
//
// EGL_EXT_buffer_age changes the first half: once the client has asked for an age it
// repaints only what changed since the image it is given was last shown, so a presented
// image keeps its content (the wire passes load it; Vulkan preserves an image's content
// through a present and a layout transition out of PRESENT_SRC) and its age is counted.
m_swapchainObject.OnImagePresented(m_imageIndexAcquired);
if (!m_swapchainObject.BufferAgeAsked()) m_swapchainObject.SetImageContentDefined(m_imageIndexAcquired, false);
m_swapchainObject.SetAllDepthStencilContentUndefined();
// The authoritative check, done here - after the frame is presented, before the next
// acquire. This is what makes a launcher-side resolution change take effect: shrinking
// the window's buffer (SurfaceHolder.setFixedSize) moves currentExtent, the swapchain
// follows, and the compositor scales the smaller image up to the view for free.
if (!m_swapchainResizeRequested && SwapchainIsOutOfDate()) {
m_swapchainResizeRequested = true;
}
// A new eglSwapInterval lands here too, between frames: this one is on screen and the
// next has recorded nothing. Only a different present mode is worth the rebuild (it
// drops every pipeline); otherwise the live swapchain already paces as asked.
if (m_config.SwapInterval != m_swapchainSwapInterval) {
if (SwapchainObject::ChooseSwapchainPresentMode(m_swapchainObject.GetSupportedPresentModes(),
m_config.SwapInterval) !=
m_swapchainObject.GetPresentMode()) {
m_swapchainResizeRequested = true;
} else {
MGLOG_D("DirectVulkan: swap interval %d -> %s", *m_config.SwapInterval,
SwapchainObject::GetPresentModeName(m_swapchainObject.GetPresentMode()));
m_swapchainSwapInterval = m_config.SwapInterval;
}
}
if (m_swapchainResizeRequested) {
MGLOG_D("Present, processing requested swapchain rebuild (resize or swap interval)");
if (!RecreateSwapchain()) {
m_presentSuspended = true;
m_swapchainResizeRequested = false;
MGLOG_D("Present, zero-area window on requested resize; suspending presentation");
return;
}
m_swapchainResizeRequested = false;
}
// 3) Advance frame slot.
m_frameContext.AdvanceToNext();
// FrameContext waits then resets the slot fence and frees retired
// command buffers. A later Present fence is not aggregate proof for
// earlier pooled flushes, so wait every registered submission in this
// slot's prefix BEFORE the reset/free happens.
const Uint64 slotSubmit = m_frameContext.GetCurrent().lastSubmitIndex;
if (slotSubmit > m_completedSubmitCounter && !WaitForSubmitsUpTo(slotSubmit, UINT64_MAX)) {
if (LatchWireDeviceLoss("present-slot-submit-wait")) return;
MagmaWireFatal("present-slot-submit-wait");
}
// 4) Wait/reset/acquire for next frame.
result = m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
if (result == VK_SUBOPTIMAL_KHR) {
// An image WAS acquired and its signal is armed on this slot's
// imageAvailableSemaphore, so the frame proceeds normally. Whether a rebuild is
// actually needed is decided by the surface-capabilities comparison at the next
// Present - suboptimal alone must not schedule one, or a driver that reports it
// every frame would rebuild every frame.
m_swapchainResizeRequested = m_swapchainResizeRequested || SwapchainIsOutOfDate();
result = VK_SUCCESS;
} else if (result == VK_ERROR_OUT_OF_DATE_KHR) {
// Nothing acquired, nothing signaled: safe to rebuild and re-acquire.
MGLOG_D("Present, vkAcquireNextImageKHR got %d, recreating swapchain", result);
if (!RecreateSwapchain()) {
m_presentSuspended = true;
m_swapchainResizeRequested = false;
MGLOG_D("Present, zero-area window on next-frame acquire; suspending presentation");
return;
}
m_swapchainResizeRequested = false;
result =
m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
}
NoteDeviceLoss(result, "vkAcquireNextImageKHR");
if (result != VK_SUCCESS && LatchWireDeviceLoss("present-acquire")) return;
VK_VERIFY(result, "Present, vkAcquireNextImageKHR");
// Pull keeps its historical slot-fence inference. In a disaggregated
// build the aggregate wait above already retired the registered prefix
// before FrameContext reset the slot fence; this repeat is idempotent.
OnSubmitsCompletedUpTo(m_frameContext.GetCurrent().lastSubmitIndex);
CollectDeferredDepthMipmapCleanup(m_frameContext.GetCurrentFrameIndex());
// Its previous recording has completed. Drop cached attachment views
// before the texture manager can release their retired images.
ClearWireDrawPassCache(m_frameContext.GetCurrentFrameIndex());
m_textureManager->BeginFrame(m_frameContext.GetCurrentFrameIndex());
m_bufferManager.BeginFrame(m_frameContext.GetCurrentFrameIndex());
// Descriptor-set reuse cursors rewind exactly once per frame, here,
// after the slot's fence wait proved its previous sets GPU-idle. (The
// per-draw-path lazy rewind missed frames whose recording was opened
// by a staged buffer copy or timer-query timestamp, leaking a fresh
// descriptor set per draw for the whole frame; it would also be unsafe
// after a mid-frame FlushPendingCommands, which does not wait.)
m_uniformManager->BeginFrame(m_frameContext.GetCurrentFrameIndex());
}
Int32 VulkanRenderer::CurrentDrawBufferAge() {
if (m_surface == VK_NULL_HANDLE || m_swapchainObject.GetHandle() == VK_NULL_HANDLE || m_presentSuspended)
return 0;
m_swapchainObject.NoteBufferAgeAsked();
if (m_imageIndexAcquired >= m_swapchainObject.GetImageCount()) return 0;
// The image the next frame's first draw makes the default framebuffer's (it was acquired at
// the end of the last present). Its content is a frame only if nothing undefined it since.
if (!m_swapchainObject.IsImageContentDefined(m_imageIndexAcquired)) return 0;
return m_swapchainObject.BufferAgeOf(m_imageIndexAcquired);
}
void VulkanRenderer::ShutdownSwapchain() {
MOBILEGL_ASSERT(m_renderPassManager != nullptr, "ShutdownSwapchain: render pass manager is null");
m_renderPassManager->Shutdown();
m_swapchainObject.Shutdown(m_device);
}
Bool VulkanRenderer::RecreateSwapchain() {
// No surface target is active (its surface was released): nothing to build on.
if (m_surface == VK_NULL_HANDLE) return false;
// Handle cases like minimize on Windows, where swapchain could return a 0x0 extent
const auto swapchainCapabilities =
SwapchainObject::GetSwapchainCapabilities(m_physicalDevice.handle, m_surface);
if (swapchainCapabilities.capabilities.currentExtent.width == 0 ||
swapchainCapabilities.capabilities.currentExtent.height == 0) {
return false;
}
vkDeviceWaitIdle(m_device);
OnSubmitsCompletedUpTo(m_submitCounter);
// The old swapchain will be destroyed below, including recordings that
// are being abandoned instead of submitted. Release all their views now.
DestroyWireDrawPass();
CollectWireObjects(m_submitCounter, true);
ClearAllWireDrawPassCaches();
if (m_timerQueryManager) {
// The in-progress command buffer is abandoned below (its recording
// flags are force-cleared), so timestamp writes recorded into it
// will never execute; resolve or invalidate all pending records now
// to keep later waits from hanging on never-available queries.
m_timerQueryManager->InvalidatePendingRecords();
}
DestroyDeferredDepthMipmapCleanup();
m_deferredDepthMipmapCleanup.assign(m_frameContext.GetFrameCount(), {});
const VkExtent2D previousExtent = m_swapchainObject.GetSurfaceExtent();
ShutdownSwapchain();
// The new swapchain publishes its extent (SwapchainObject::Create), which is how a client
// sizing its viewport from eglQuerySurface learns that the window moved.
CreateSwapchain();
const VkExtent2D extent = m_swapchainObject.GetSurfaceExtent();
if (extent.width != previousExtent.width || extent.height != previousExtent.height) {
MGLOG_I("DirectVulkan: swapchain rebuilt at %ux%u (was %ux%u); the new extent is published", extent.width,
extent.height, previousExtent.width, previousExtent.height);
}
// Every image of the fresh swapchain holds garbage, so the default framebuffer restarts
// at index 0: the next write into it re-points the GL-visible index at whatever the next
// acquire returns (see m_defaultFramebufferImageIndex). Without this the index could name
// an image of the swapchain just destroyed.
m_defaultFramebufferImageIndex = 0;
VK_VERIFY(m_frameContext.InitializeSwapchainSemaphores(m_device,
static_cast<Uint32>(m_swapchainObject.GetImageCount())),
"RecreateSwapchain, InitializeSwapchainSemaphores");
MOBILEGL_ASSERT(m_renderPassManager != nullptr, "RecreateSwapchain: render pass manager is null");
Bool ok = m_renderPassManager->Initialize();
MOBILEGL_ASSERT(ok, "RecreateSwapchain: render pass manager initialization failed");
if (m_pipelineFactory) {
m_pipelineFactory->DestroyAll();
}
InvalidatePipelineMemo(); // pipelines freed -> the memoized handle would dangle
g_dynamicStateShadow->graphicsPipelineValid = false;
DestroyComputePipelines();
if (m_frameContext.GetFrameCount() > 0) {
m_frameContext.GetCurrent().isCommandRecording = false;
m_frameContext.GetCurrent().hasCommandBufferRecorded = false;
// The pre-pass stream paired with the abandoned recording is
// dropped with it (its next Begin resets the buffer).
m_frameContext.GetCurrent().isPreCommandRecording = false;
m_frameContext.GetCurrent().hasPreCommandBufferRecorded = false;
}
const Bool okArena = m_bufferManager.RecreateTransientArenas(m_frameContext.GetFrameCount());
MOBILEGL_ASSERT(okArena, "RecreateSwapchain: buffer manager transient arena initialization failed");
if (m_frameContext.GetFrameCount() > 0) {
if (m_textureManager) {
m_textureManager->BeginFrame(m_frameContext.GetCurrentFrameIndex());
}
m_bufferManager.BeginFrame(m_frameContext.GetCurrentFrameIndex());
}
return true;
}
const PhysicalDevice& VulkanRenderer::GetPhysicalDevice() const {
return m_physicalDevice;
}
Bool VulkanRenderer::SwapchainIsOutOfDate() {
if (m_surface == VK_NULL_HANDLE || m_swapchainObject.GetHandle() == VK_NULL_HANDLE) {
return false;
}
VkSurfaceCapabilitiesKHR surfaceCaps{};
if (vkGetPhysicalDeviceSurfaceCapabilitiesKHR(m_physicalDevice.handle, m_surface, &surfaceCaps) != VK_SUCCESS) {
return false;
}
// A driver-defined currentExtent (UINT32_MAX) means the surface takes its size from the
// swapchain, so there is nothing to compare against - the app's requested size wins and
// only an explicit RequestSwapchainResize can change it.
if (surfaceCaps.currentExtent.width == UINT32_MAX || surfaceCaps.currentExtent.height == UINT32_MAX) {
return false;
}
// Compare in SURFACE space against the extent the live swapchain was created from. Using
// the swapchain's own (quarter-turn swapped) extent here would report a difference on
// every rotated frame and rebuild forever.
const VkExtent2D builtFrom = m_swapchainObject.GetSurfaceExtent();
const Bool extentChanged =
surfaceCaps.currentExtent.width != builtFrom.width || surfaceCaps.currentExtent.height != builtFrom.height;
const Bool transformChanged = surfaceCaps.currentTransform != m_swapchainObject.GetPreTransform();
if (!extentChanged && !transformChanged) {
return false;
}
MGLOG_D("Swapchain out of date: surface %ux%u transform %u -> %ux%u transform %u", builtFrom.width,
builtFrom.height, static_cast<Uint32>(m_swapchainObject.GetPreTransform()),
surfaceCaps.currentExtent.width, surfaceCaps.currentExtent.height,
static_cast<Uint32>(surfaceCaps.currentTransform));
return true;
}
void VulkanRenderer::RequestSwapchainResize(Uint32 width, Uint32 height) {
width = std::max<Uint32>(width, 1);
height = std::max<Uint32>(height, 1);
if (m_config.SurfaceWidth == width && m_config.SurfaceHeight == height) {
return;
}
m_config.SurfaceWidth = width;
m_config.SurfaceHeight = height;
m_swapchainResizeRequested = true;
}
void VulkanRenderer::SetSwapInterval(Int interval) {
if (m_presentsToAppWindow) {
m_config.SwapInterval = interval;
}
}
void VulkanRenderer::ExchangeActiveTarget(SurfaceTarget& target) {
std::swap(m_activeTargetSerial, target.serial);
std::swap(m_window, target.window);
std::swap(m_platformDisplay, target.platformDisplay);
std::swap(m_platformLibrary, target.platformLibrary);
std::swap(m_platformCloseDisplay, target.platformCloseDisplay);
std::swap(m_fallbackImageReader, target.fallbackImageReader);
std::swap(m_config.SurfaceWidth, target.surfaceWidth);
std::swap(m_config.SurfaceHeight, target.surfaceHeight);
std::swap(m_config.SwapInterval, target.swapInterval);
std::swap(m_presentsToAppWindow, target.presentsToAppWindow);
std::swap(m_swapchainSwapInterval, target.swapchainSwapInterval);
std::swap(m_swapchainResizeRequested, target.swapchainResizeRequested);
std::swap(m_presentSuspended, target.presentSuspended);
std::swap(m_surface, target.surface);
std::swap(m_swapchainObject, target.swapchain);
m_frameContext.ExchangeSwapchainSemaphores(target.renderFinishedSemaphores);
std::swap(m_imageIndexAcquired, target.imageIndexAcquired);
std::swap(m_defaultFramebufferImageIndex, target.defaultFramebufferImageIndex);
}
void VulkanRenderer::ParkActiveTarget() {
if (m_surface == VK_NULL_HANDLE) return;
// The active surface's recorded work goes to the queue now, while its images are still
// the default framebuffer's.
FlushPendingCommands();
auto& frame = m_frameContext.GetCurrent();
// Its acquired image keeps its acquire for when the surface comes back, but the signal
// lives on THIS slot's semaphore, which the next surface acquires with. Wait it out with
// an empty submission; the image is then simply held, and its next submit waits nothing.
if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE && !m_presentSuspended &&
!frame.imageAvailableSemaphoreConsumed) {
const VkFence fence = AcquirePooledSubmitFence();
if (fence == VK_NULL_HANDLE ||
!SubmitPendingCommandBuffer(frame, fence, /*pooledFence=*/true, /*waitAcquire=*/true)) {
MGLOG_E("DirectVulkan: a parked surface's pending acquire could not be waited out");
}
}
SurfaceTarget parked;
ExchangeActiveTarget(parked);
m_parkedTargets[m_activeTargetKey] = std::move(parked);
m_activeTargetKey = 0;
}
void VulkanRenderer::OnActiveTargetChanged() {
if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE) {
m_swapchainObject.PublishDefaultFramebufferInfo();
if (m_programFactory) m_programFactory->SetDefaultFramebufferHeight(m_swapchainObject.GetExtent().height);
}
if (m_renderPassManager) m_renderPassManager->SetDefaultFramebufferTarget(m_activeTargetSerial);
InvalidatePipelineMemo();
ResetDynamicStateShadow();
}
Bool VulkanRenderer::ActivateSurfaceTarget(Uint64 key, NativeWindowType window,
const VulkanRendererConfig& surfaceConfig) {
if (key == m_activeTargetKey && m_surface != VK_NULL_HANDLE) return true;
ParkActiveTarget();
auto parked = m_parkedTargets.find(key);
if (parked != m_parkedTargets.end()) {
ExchangeActiveTarget(parked->second);
m_parkedTargets.erase(parked);
// Its acquire was waited when it was parked (ParkActiveTarget).
m_frameContext.GetCurrent().imageAvailableSemaphoreConsumed = true;
} else {
m_activeTargetSerial = m_nextTargetSerial++;
m_window = window;
m_presentsToAppWindow = window != NativeWindowType{};
m_config.SurfaceWidth = surfaceConfig.SurfaceWidth;
m_config.SurfaceHeight = surfaceConfig.SurfaceHeight;
m_config.SwapInterval = m_presentsToAppWindow ? surfaceConfig.SwapInterval : Nullopt;
m_swapchainSwapInterval.reset();
m_swapchainResizeRequested = false;
m_presentSuspended = false;
if (!CreateSurface()) {
// Nothing of the half-built target stays active: its image reader and library go
// with it, and the renderer has no target until the caller activates another.
SurfaceTarget failed;
ExchangeActiveTarget(failed);
DestroyParkedTarget(failed);
m_activeTargetKey = 0;
MGLOG_E("DirectVulkan: surface target for key %llu could not be built (%s)",
static_cast<unsigned long long>(key), window != NativeWindowType{} ? "window" : "pbuffer");
return false;
}
CreateSwapchain();
m_defaultFramebufferImageIndex = 0;
VK_VERIFY(m_frameContext.InitializeSwapchainSemaphores(
m_device, static_cast<Uint32>(m_swapchainObject.GetImageCount())),
"ActivateSurfaceTarget, InitializeSwapchainSemaphores");
if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE) {
VkResult result = m_frameContext.AcquireNextImageInCurrentSlot(m_device, m_swapchainObject.GetHandle(),
m_imageIndexAcquired);
if (result == VK_SUBOPTIMAL_KHR) result = VK_SUCCESS;
VK_VERIFY(result, "ActivateSurfaceTarget, AcquireNextImageInCurrentSlot");
m_defaultFramebufferImageIndex = m_imageIndexAcquired;
} else {
m_presentSuspended = true;
}
MGLOG_I("DirectVulkan: surface target %llu built (%ux%u, %s)",
static_cast<unsigned long long>(m_activeTargetSerial), m_swapchainObject.GetExtent().width,
m_swapchainObject.GetExtent().height, m_presentsToAppWindow ? "window" : "pbuffer");
}
m_activeTargetKey = key;
OnActiveTargetChanged();
return true;
}
void VulkanRenderer::DestroyParkedTarget(SurfaceTarget& target) {
target.swapchain.Shutdown(m_device);
for (VkSemaphore semaphore : target.renderFinishedSemaphores) {
if (semaphore != VK_NULL_HANDLE) vkDestroySemaphore(m_device, semaphore, nullptr);
}
target.renderFinishedSemaphores.clear();
if (target.surface != VK_NULL_HANDLE) {
vkDestroySurfaceKHR(m_instance, target.surface, nullptr);
target.surface = VK_NULL_HANDLE;
}
#if defined(VK_USE_PLATFORM_ANDROID_KHR)
if (target.fallbackImageReader != nullptr && target.platformLibrary != nullptr) {
using AImageReaderDeleteFn = void (*)(void*);
auto* imageReaderDelete =
reinterpret_cast<AImageReaderDeleteFn>(dlsym(target.platformLibrary, "AImageReader_delete"));
if (imageReaderDelete) imageReaderDelete(target.fallbackImageReader);
target.fallbackImageReader = nullptr;
dlclose(target.platformLibrary);
target.platformLibrary = nullptr;
}
#elif defined(VK_USE_PLATFORM_XLIB_KHR)
if (target.platformDisplay != nullptr) {
using XCloseDisplayFn = int (*)(Display*);
auto* closeDisplay = reinterpret_cast<XCloseDisplayFn>(target.platformCloseDisplay);
if (closeDisplay) closeDisplay(static_cast<Display*>(target.platformDisplay));
target.platformDisplay = nullptr;
}
if (target.platformLibrary != nullptr) {
dlclose(target.platformLibrary);
target.platformLibrary = nullptr;
}
#endif
}
void VulkanRenderer::DestroySurfaceTarget(Uint64 key) {
if (key == m_activeTargetKey && m_surface != VK_NULL_HANDLE) ParkActiveTarget();
auto parked = m_parkedTargets.find(key);
if (parked == m_parkedTargets.end()) return;
// Whatever is recorded may name the target's images (a blit out of it, say): submit it,
// then let the GPU finish with everything before the views go.
FlushPendingCommands();
VK_VERIFY(vkDeviceWaitIdle(m_device));
OnSubmitsCompletedUpTo(m_submitCounter);
DestroyWireDrawPass();
CollectWireObjects(m_submitCounter, true);
ClearAllWireDrawPassCaches();
InvalidatePipelineMemo();
ResetDynamicStateShadow();
DestroyParkedTarget(parked->second);
m_parkedTargets.erase(parked);
}
Bool VulkanRenderer::FollowActiveWindowResize() {
if (m_surface == VK_NULL_HANDLE || !m_presentsToAppWindow) return false;
const auto& frame = m_frameContext.GetCurrent();
if (frame.isCommandRecording || frame.hasCommandBufferRecorded || frame.isPreCommandRecording ||
frame.hasPreCommandBufferRecorded) {
m_swapchainResizeRequested = true;
MGLOG_I("DirectVulkan: the window resized mid-frame; its swapchain is rebuilt at this frame's present");
return true;
}
// The surface already reports the swapchain's extent: nothing to rebuild.
if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE && !SwapchainIsOutOfDate()) return true;
// The target's own rebuild - the path a surface's re-activation takes: its pending acquire waited
// out, a new VkSurfaceKHR and swapchain at the surface's current extent and pre-transform, a new
// target serial for the render passes, and the extent published.
const Uint64 key = m_activeTargetKey;
const NativeWindowType window = m_window;
VulkanRendererConfig config;
config.SurfaceWidth = m_config.SurfaceWidth;
config.SurfaceHeight = m_config.SurfaceHeight;
config.SwapInterval = m_config.SwapInterval;
DestroySurfaceTarget(key);
return ActivateSurfaceTarget(key, window, config);
}
VkExtent2D VulkanRenderer::SurfaceTargetExtent(Uint64 key) const {
if (key == m_activeTargetKey && m_surface != VK_NULL_HANDLE) {
return m_swapchainObject.GetHandle() != VK_NULL_HANDLE ? m_swapchainObject.GetSurfaceExtent()
: VkExtent2D{0, 0};
}
const auto parked = m_parkedTargets.find(key);
if (parked == m_parkedTargets.end() || parked->second.swapchain.GetHandle() == VK_NULL_HANDLE) {
return VkExtent2D{0, 0};
}
return parked->second.swapchain.GetSurfaceExtent();
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -0,0 +1,814 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/VulkanRendererSubmit.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "VulkanRendererInternal.h"
namespace MobileGL::MG_Backend::DirectVulkan {
VkCommandBuffer VulkanRenderer::AcquireBufferCopyCommandBuffer() {
if (m_device == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return VK_NULL_HANDLE;
}
auto& frame = m_frameContext.GetCurrent();
if (!frame.isCommandRecording) {
m_frameContext.BeginCommandRecording();
}
// vkCmdCopyBuffer must be recorded outside a render pass; draws re-begin
// their render pass lazily, matching the existing blit/clear pattern.
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
return frame.commandBuffer;
}
Bool VulkanRenderer::IsFrameSerialComplete(Uint64 serial) const {
return serial <= m_bufferManager.GetCompletedSerial();
}
Bool VulkanRenderer::WaitForFrameSerial(Uint64 serial, Uint64 timeoutNs) {
(void)timeoutNs;
if (IsFrameSerialComplete(serial)) {
return true;
}
// Work recorded under the current serial has not been submitted yet
// (submission happens in Present, on this same thread), so blocking
// can never make progress; the caller reports a timeout instead.
if (serial >= m_bufferManager.GetFrameSerial()) {
return false;
}
if (m_device == VK_NULL_HANDLE || m_graphicsQueue == VK_NULL_HANDLE) {
return IsFrameSerialComplete(serial);
}
// More than one submit can carry the same frame serial (mid-frame flush,
// then Present). The completed floor is clamped below any serial still
// held by an in-flight record; the first matching fence is not proof.
// OnSubmitsCompletedUpTo erases records, so copy fields and restart the
// search after every wait instead of continuing an invalidated iterator.
for (;;) {
const auto record =
std::find_if(m_inFlightSubmits.begin(), m_inFlightSubmits.end(), [serial](const auto& candidate) {
return candidate.frameSerial >= serial && candidate.fence != VK_NULL_HANDLE;
});
if (record == m_inFlightSubmits.end()) break;
const Uint64 submitIndex = record->submitIndex;
if (!WaitForSubmitsUpTo(submitIndex, UINT64_MAX)) break; // fall through to the queue drain below
TryDrainFrameTransients();
if (IsFrameSerialComplete(serial)) return true;
}
// No usable record - fall back to draining the graphics queue. This over-waits (bounded by
// the in-flight frame count) but never deadlocks.
const VkResult result = vkQueueWaitIdle(m_graphicsQueue);
if (result != VK_SUCCESS) {
NoteDeviceLoss(result, "WaitForFrameSerial vkQueueWaitIdle");
MGLOG_E_ONCE("WaitForFrameSerial: vkQueueWaitIdle returned %d", result);
return false;
}
m_bufferManager.NotifyDeviceIdle();
OnSubmitsCompletedUpTo(m_submitCounter);
// The queue was just drained; take the free frame-boundary drain when
// nothing is recorded (present-less timer-query loops). No-op otherwise.
TryDrainFrameTransients();
return IsFrameSerialComplete(serial);
}
Uint64 VulkanRenderer::GetSyncPointSubmitIndex() const {
// Commands recorded (or still recording) since the last submission are
// carried by the NEXT submission; a fence created now must wait for it.
return m_submitCounter + (HasPendingRecordedWork() ? 1 : 0);
}
Bool VulkanRenderer::HasPendingRecordedWork() const {
if (m_frameContext.GetFrameCount() == 0) {
return false;
}
const auto& frame = m_frameContext.GetCurrent();
return frame.isCommandRecording || frame.hasCommandBufferRecorded || frame.isPreCommandRecording ||
frame.hasPreCommandBufferRecorded;
}
Bool VulkanRenderer::RewindWireDescriptorSetsIfDue() {
if (!m_uniformManager || m_frameContext.GetFrameCount() == 0) return true;
const Uint32 frameIndex = m_frameContext.GetCurrentFrameIndex();
if (!m_uniformManager->WireDescriptorSetBudgetReached(frameIndex)) return true;
// A set cannot be rewritten while a recorded or submitted command
// buffer might still read it. This wire-only boundary first submits
// the current recording, then idles the graphics queue before reuse.
if (HasPendingRecordedWork() && !FlushPendingCommands()) {
if (LatchWireDeviceLoss("descriptor-rewind-flush")) return false;
MagmaWireFatal("descriptor-rewind-flush");
}
const Uint64 through = m_submitCounter;
if (through > m_completedSubmitCounter) {
// B4 has not yet made every old retirement site aggregate-safe.
// Queue idle proves even a submission a previous single-fence path
// prematurely removed from m_inFlightSubmits has finished; only
// then may any layout cursor be rewound and its sets rewritten.
if (const VkResult idle = vkQueueWaitIdle(m_graphicsQueue); idle != VK_SUCCESS) {
NoteDeviceLoss(idle, "descriptor rewind vkQueueWaitIdle");
if (LatchWireDeviceLoss("descriptor-rewind-wait")) return false;
MagmaWireFatal("descriptor-rewind-wait");
}
OnSubmitsCompletedUpTo(through);
}
if (HasPendingRecordedWork() || m_completedSubmitCounter < through) MagmaWireFatal("descriptor-rewind-proof");
const SizeT cachedSets = m_uniformManager->RewindWireDescriptorSets(frameIndex);
MGLOG_I("Magma descriptor rewind: frame=%u retiredSubmit=%llu budget=%u cached=%zu", frameIndex,
static_cast<unsigned long long>(through), UniformManager::kWireDescriptorSetBudget, cachedSets);
return true;
}
Bool VulkanRenderer::IsSubmitIndexComplete(Uint64 submitIndex) {
if (submitIndex <= m_completedSubmitCounter) {
return true;
}
if (submitIndex > m_submitCounter) {
return false; // not even submitted; no point polling fences
}
// P15: a fence just seen unsignaled is not polled again for a short while. Callers ask per
// buffer write (FCL MC 1.21.5: vkGetFenceStatus was 0.020 ms/frame), and every one of them
// treats "not yet" as "still busy" - rename, defer, report unsignaled - so a late "yes" costs
// at most kUnsignaledPollQuietNs of extra caution, never correctness.
constexpr Uint64 kUnsignaledPollQuietNs = 50000;
const Uint64 now = static_cast<Uint64>(
std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now().time_since_epoch())
.count());
if (!m_inFlightSubmits.empty() && m_unsignaledPollFront == m_inFlightSubmits.front().submitIndex &&
now - m_unsignaledPollNs < kUnsignaledPollQuietNs) {
return false;
}
RefreshCompletedSubmits();
if (submitIndex <= m_completedSubmitCounter) return true;
if (!m_inFlightSubmits.empty()) {
m_unsignaledPollFront = m_inFlightSubmits.front().submitIndex;
m_unsignaledPollNs = now;
}
return false;
}
void VulkanRenderer::RegisterSubmit(VkFence fence, Bool pooledFence) {
++m_submitCounter;
m_inFlightSubmits.push_back({m_submitCounter, m_bufferManager.GetFrameSerial(), fence, pooledFence});
}
void VulkanRenderer::RefreshCompletedSubmits() {
if (m_device == VK_NULL_HANDLE) {
return;
}
// Prefix-only scan: submissions to a single queue complete in order,
// and stopping at the first unsignaled fence stays conservative even
// if they did not.
while (!m_inFlightSubmits.empty()) {
// Copy before OnSubmitsCompletedUpTo erases the front record.
const Uint64 frontIndex = m_inFlightSubmits.front().submitIndex;
if (vkGetFenceStatus(m_device, m_inFlightSubmits.front().fence) != VK_SUCCESS) {
break;
}
OnSubmitsCompletedUpTo(frontIndex);
}
}
Bool VulkanRenderer::WaitForSubmitsUpTo(Uint64 submitIndex, Uint64 timeoutNs) {
if (submitIndex <= m_completedSubmitCounter) return true;
if (submitIndex > m_submitCounter || m_device == VK_NULL_HANDLE) return false;
Vector<VkFence> fences;
if (!CollectSubmitFencePrefix(m_inFlightSubmits, submitIndex, VkFence{}, fences)) return false;
const VkResult result =
vkWaitForFences(m_device, static_cast<Uint32>(fences.size()), fences.data(), VK_TRUE, timeoutNs);
if (result == VK_SUCCESS) {
OnSubmitsCompletedUpTo(submitIndex);
return true;
}
NoteDeviceLoss(result, "WaitForSubmitsUpTo vkWaitForFences");
if (result != VK_TIMEOUT) MGLOG_E_ONCE("WaitForSubmitsUpTo: vkWaitForFences returned %d", result);
return false;
}
void VulkanRenderer::OnSubmitsCompletedUpTo(Uint64 submitIndex) {
m_completedSubmitCounter = std::max(m_completedSubmitCounter, submitIndex);
// ---- P7 wave 2 package B3: THE COMPLETED-FRAME-SERIAL FLOOR MUST BE PROVABLE --------
//
// "Frame-serial completion piggybacks on submission completion" holds only while a
// frame serial has ONE submission. On the wire arm it has at least two: the per-frame
// palette glTexImage2D reaches UploadPendingWireLevels ->
// FlushWirePendingCommandsForTextureUpdate -> FlushPendingCommands at the first
// palette-sampling draw and submits S1 under a POOLED fence, and the frame's draws
// (and the barriered vkCmdCopyBuffer that StagedWireRangeCopy records for every
// streamed glBufferSubData) then go out in the Present submission S2.
//
// S1 is tiny and signals almost immediately. Retiring it used to advance the floor to
// ITS frameSerial - which is also S2's - so VkBufferManager was told "frame N-1 is
// complete" while S2(N-1), the submission actually carrying frame N-1's buffer copies,
// was still executing. The next frame's first write to that buffer then failed
// WriteWireBuffer's busy predicate and took the unordered host memcpy, and S2(N-1)'s
// queued copies landed on top of it afterwards: a PREFIX TEAR of one draw's vertex
// range, no log line, no GL error, no Fatal.
//
// The floor now advances only to a serial NO remaining in-flight submission still
// carries. NotifyFrameSerialComplete is monotone and already refuses the still-
// recording serial, so this can only ever advance the floor later, never further.
// On the disaggregated build's MONOLITH arm this is behaviour-identical (one
// submission per serial). The #else branch below is the pull build's statement,
// unchanged, which is what keeps G1's .text byte-identical.
Bool retiredAny = false;
Uint64 advanceTo = 0;
while (!m_inFlightSubmits.empty() && m_inFlightSubmits.front().submitIndex <= submitIndex) {
SubmitRecord record = m_inFlightSubmits.front();
m_inFlightSubmits.erase(m_inFlightSubmits.begin());
retiredAny = true;
advanceTo = std::max(advanceTo, record.frameSerial);
if (!record.pooledFence || m_device == VK_NULL_HANDLE) {
continue; // frame-slot fences are reset/destroyed by FrameContext
}
if (vkResetFences(m_device, 1, &record.fence) == VK_SUCCESS) {
m_freeSubmitFences.push_back(record.fence);
} else {
vkDestroyFence(m_device, record.fence, nullptr);
}
}
if (retiredAny) {
// Clamp to one below the lowest serial still in flight.
for (const auto& remaining : m_inFlightSubmits) {
advanceTo = std::min(advanceTo, remaining.frameSerial > 0 ? remaining.frameSerial - 1 : Uint64{0});
}
// Lens C's probe, and the red-once's reading: an advance that names a serial some
// remaining submission still carries. Unreachable with the clamp above; the R-16
// revert is to delete the clamp loop, and then this counts.
for (const auto& remaining : m_inFlightSubmits) {
if (remaining.frameSerial <= advanceTo) {
WireDeclineTally::CountUnsoundSerialComplete();
// Logged per event, not once: this is the red-once's reading and the log is
// the only place it can be read from: the renderer this trace builds is
// never Shutdown() (the process exits), so a teardown dump reports the
// start-up context's zero instead of the replay's count.
MGLOG_W("MGWIRE-FLOOR unsound-serial-complete #%llu: advancing the completed "
"frame-serial floor to %llu, which submission %llu (serial %llu) still carries",
static_cast<unsigned long long>(WireDeclineTally::UnsoundSerialCompleteEvents()),
static_cast<unsigned long long>(advanceTo),
static_cast<unsigned long long>(remaining.submitIndex),
static_cast<unsigned long long>(remaining.frameSerial));
break;
}
}
{
static const Bool probe = [] {
const char* value = std::getenv("MOBILEGL_MAGMA_WIREBUF_PROBE");
return value && value[0] == '1';
}();
if (probe) {
Uint64 minRemaining = ~Uint64{0};
for (const auto& r : m_inFlightSubmits)
minRemaining = std::min(minRemaining, r.frameSerial);
MGLOG_I("FLOOR retire upTo=%llu advanceTo=%llu remaining=%zu minRemainingSerial=%lld "
"frameSerial=%llu",
static_cast<unsigned long long>(submitIndex), static_cast<unsigned long long>(advanceTo),
m_inFlightSubmits.size(),
m_inFlightSubmits.empty() ? -1LL : static_cast<long long>(minRemaining),
static_cast<unsigned long long>(m_bufferManager.GetFrameSerial()));
}
}
m_bufferManager.NotifyFrameSerialComplete(advanceTo);
}
// Mid-frame-flushed command buffers whose submission just completed can
// be freed now; present-less flush loops have no other reclaim point.
m_frameContext.FreeRetiredCommandBuffersCompletedUpTo(m_completedSubmitCounter);
CollectWireObjects(m_completedSubmitCounter);
#if MOBILEGL_BUILD_DISAGGREGATED
RecycleSharedImageSemaphores(m_completedSubmitCounter);
#endif
}
Bool VulkanRenderer::TryDrainFrameTransients() {
if (m_wirePreparationDepth != 0) return false;
if (m_device == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return false;
}
if (m_completedSubmitCounter != m_submitCounter) {
RefreshCompletedSubmits();
if (m_completedSubmitCounter != m_submitCounter) {
return false;
}
}
if (HasPendingRecordedWork()) {
return false;
}
// Texture uploads submit on this queue with their own fences. The
// renderer watermark alone cannot prove images/views are idle.
if (m_textureManager && !m_textureManager->WireUploadsAreIdle()) return false;
// No preparation, recording or GPU work remains. A minimized Present
// may have abandoned a recording tagged for a submission that will
// never occur, so reclaim those future-tagged objects as well.
CollectWireObjects(m_completedSubmitCounter, true);
ClearAllWireDrawPassCaches();
// Every submission is complete and nothing recorded references the
// per-frame transients. Pure-reclaim work runs on every drain: it only
// releases memory that is provably dead, never invalidates anything a
// later draw would have to rebuild. Raise the buffer manager's
// completed floor first so busy-tracking reflects the proven idleness.
m_bufferManager.NotifyDeviceIdle();
const Uint32 frameIndex = m_frameContext.GetCurrentFrameIndex();
m_frameContext.FreeAllRetiredCommandBuffers();
for (Uint32 slot = 0; slot < m_deferredDepthMipmapCleanup.size(); ++slot) {
CollectDeferredDepthMipmapCleanup(slot);
}
if (m_textureManager) {
m_textureManager->CollectAllDeferredReleases();
}
m_bufferManager.CollectAllDeferredReleases();
// Descriptor cursors rewind on every drain (the pre-drain readback path
// already did exactly this), keeping fence/readback loops' set usage bounded.
if (m_uniformManager) {
m_uniformManager->BeginFrame(frameIndex);
}
// Frame-boundary-equivalent work - transient arena rewind (which invalidates
// the conversion cache) and the cache-aging clocks - is gated to every 8th
// drain since the last Present: a presenting app's mid-frame readbacks/waits
// must neither force re-conversion/re-upload churn for the rest of the frame
// nor multiply the aging rate (which would shrink the 1024-boundary retire
// window and thrash periodically-used pipelines/programs), while present-less
// loops still rewind the arena and age their caches every 8 iterations -
// bounded by 8 iterations' transient usage.
++m_drainsSinceLastPresent;
// B3 probe: a drain that reaches here found the GPU caught up and nothing recording.
// Every 8th one is treated as a FRAME BOUNDARY - mid-frame. This probe is what REFUTED
// that as the OpenRA mechanism (one successful drain, zero boundary works across the
// replay; magma-b3.md §2.4): the cause was the floor in OnSubmitsCompletedUpTo.
{
static const Bool probe = [] {
const char* value = std::getenv("MOBILEGL_MAGMA_WIREBUF_PROBE");
return value && value[0] == '1';
}();
if (probe) {
MGLOG_I("WBUF drain#%llu%s submit=%llu completed=%llu",
static_cast<unsigned long long>(m_drainsSinceLastPresent),
(m_drainsSinceLastPresent % 8) == 0 ? " ***FRAME-BOUNDARY-WORK***" : "",
static_cast<unsigned long long>(m_submitCounter),
static_cast<unsigned long long>(m_completedSubmitCounter));
}
}
if ((m_drainsSinceLastPresent % 8) != 0) {
return true;
}
if (m_textureManager) {
m_textureManager->BeginFrame(frameIndex);
}
m_bufferManager.BeginFrame(frameIndex);
// The pipeline memo can survive across these boundaries (no per-frame reset
// on this path), so it must drop whenever the sweep destroys anything.
if (m_programFactory) {
m_programFactory->OnFrameBoundary();
}
if (m_pipelineFactory && m_pipelineFactory->OnFrameBoundary() > 0) {
InvalidatePipelineMemo();
}
if (m_samplerManager) {
m_samplerManager->OnFrameBoundary();
}
return true;
}
VkFence VulkanRenderer::AcquirePooledSubmitFence() {
if (!m_freeSubmitFences.empty()) {
VkFence fence = m_freeSubmitFences.back();
m_freeSubmitFences.pop_back();
return fence;
}
VkFenceCreateInfo fenceInfo{VK_STRUCTURE_TYPE_FENCE_CREATE_INFO};
VkFence fence = VK_NULL_HANDLE;
const VkResult result = vkCreateFence(m_device, &fenceInfo, nullptr, &fence);
if (result != VK_SUCCESS) {
MGLOG_E_ONCE("AcquirePooledSubmitFence: vkCreateFence returned %d", result);
return VK_NULL_HANDLE;
}
return fence;
}
void VulkanRenderer::DestroySubmitFencePool() {
// Callers guarantee device idle, so in-flight fences are inert.
for (const auto& record : m_inFlightSubmits) {
if (record.pooledFence && m_device != VK_NULL_HANDLE) {
vkDestroyFence(m_device, record.fence, nullptr);
}
}
m_inFlightSubmits.clear();
for (auto fence : m_freeSubmitFences) {
if (m_device != VK_NULL_HANDLE) {
vkDestroyFence(m_device, fence, nullptr);
}
}
m_freeSubmitFences.clear();
m_completedSubmitCounter = m_submitCounter;
}
Bool VulkanRenderer::SubmitPendingCommandBuffer(FrameContext::FrameData& frame, VkFence fence, Bool pooledFence,
Bool waitAcquire) {
RetireWireDrawPass();
// Batched texture uploads must reach the queue before the frame's
// commands: the recording being submitted may sample images whose
// texels only exist in the texture manager's open upload batch.
if (m_textureManager) {
m_textureManager->FlushPendingUploads();
}
VkPipelineStageFlags waitDstStageMask = FrameContext::kAcquireWaitStages;
VkSemaphore waitSemaphore = frame.imageAvailableSemaphore;
VkSubmitInfo submitInfo{VK_STRUCTURE_TYPE_SUBMIT_INFO};
// A recording that never touched the acquired image does not wait for its release: on a
// queue that runs submissions in order, that wait would hold this work, and everything
// after it, until the display lets go of the image (SubmitAheadOfAcquiredImage).
const Bool waitsAcquire =
!frame.imageAvailableSemaphoreConsumed && (frame.acquiredImageReferenced || waitAcquire);
if (waitsAcquire) {
submitInfo.waitSemaphoreCount = 1;
submitInfo.pWaitSemaphores = &waitSemaphore;
submitInfo.pWaitDstStageMask = &waitDstStageMask;
}
// The pre-pass stream, when recorded, executes strictly before the
// frame's commands within the same submission.
VkCommandBuffer commandBuffers[2] = {VK_NULL_HANDLE, VK_NULL_HANDLE};
Uint32 commandBufferCount = 0;
if (frame.hasPreCommandBufferRecorded) {
commandBuffers[commandBufferCount++] = frame.preCommandBuffer;
}
if (frame.hasCommandBufferRecorded) {
commandBuffers[commandBufferCount++] = frame.commandBuffer;
}
submitInfo.commandBufferCount = commandBufferCount;
submitInfo.pCommandBuffers = commandBuffers;
#if MOBILEGL_BUILD_DISAGGREGATED
// Shared-image fences staged for the work recorded since the last submission.
if (LatchIfGpuHung("flush-submit")) return false;
SharedImageSubmitSync sharedImageSync;
AttachSharedImageSync(submitInfo, sharedImageSync);
const GpuProgressMarkers::Bracket bracket = m_progressMarkers.Acquire(m_completedSubmitCounter);
const VkResult result = GpuProgressMarkers::SubmitBracketed(m_graphicsQueue, submitInfo, fence, bracket);
if (result == VK_SUCCESS)
m_progressMarkers.Submitted(bracket, m_submitCounter + 1); // RegisterSubmit below
else
m_progressMarkers.Abandon(bracket);
#else
const VkResult result = vkQueueSubmit(m_graphicsQueue, 1, &submitInfo, fence);
#endif
if (result != VK_SUCCESS) {
NoteDeviceLoss(result, "SubmitPendingCommandBuffer vkQueueSubmit");
MGLOG_E_ONCE("SubmitPendingCommandBuffer: vkQueueSubmit returned %d", result);
return false;
}
if (waitsAcquire) frame.imageAvailableSemaphoreConsumed = true;
frame.hasCommandBufferRecorded = false;
frame.hasPreCommandBufferRecorded = false;
RegisterSubmit(fence, pooledFence);
frame.lastSubmitIndex = m_submitCounter;
#if MOBILEGL_BUILD_DISAGGREGATED
CommitSharedImageSync(sharedImageSync);
#endif
return true;
}
Bool VulkanRenderer::FlushPendingCommands() {
if (m_device == VK_NULL_HANDLE || m_graphicsQueue == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return false;
}
// Non-blocking completion poll: gives flush-only workloads (no sync
// objects, no present) a point where finished submissions retire their
// pooled fences and mid-frame command buffers.
RefreshCompletedSubmits();
auto& frame = m_frameContext.GetCurrent();
if (!frame.isCommandRecording && !frame.hasCommandBufferRecorded && !frame.isPreCommandRecording &&
!frame.hasPreCommandBufferRecorded) {
// GL flush semantics still demand batched texture uploads start
// executing in finite time even when no draw was recorded.
if (m_textureManager) {
m_textureManager->FlushPendingUploads();
}
return false;
}
// Acquire the fence while recording is still open: failing here must
// not end recording, or the next draw's BeginCommandRecording would
// reset the command buffer and silently drop the frame's commands.
VkFence fence = AcquirePooledSubmitFence();
if (fence == VK_NULL_HANDLE) {
return false;
}
if (frame.isCommandRecording) {
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
m_frameContext.EndCommandRecording();
}
m_frameContext.EndPreCommandRecordingIfOpen();
const Bool submittingPreCommandBuffer = frame.hasPreCommandBufferRecorded;
if (!SubmitPendingCommandBuffer(frame, fence, /*pooledFence=*/true)) {
// Submit failure (device loss regime): the ended command buffer
// stays marked recorded so Present can still try to submit it.
m_freeSubmitFences.push_back(fence); // still unsignaled, reusable
return false;
}
// Command-buffer boundary: the pipeline memo must not survive it, or a
// pipeline bound only through memo hits is never re-stamped in the factory
// cache and the aging sweep could destroy it while the flushed submission
// still references it. Mirrors the drops at the readback and Present
// boundaries; costs one full pipeline lookup on the next draw.
InvalidatePipelineMemo();
// The submitted command buffer may still be executing; recording must
// restart on a fresh one. If none can be allocated, fall back to
// draining this submission so reusing the buffer stays legal.
const VkResult retireResult = m_frameContext.RetireCurrentCommandBuffer(submittingPreCommandBuffer);
if (retireResult != VK_SUCCESS) {
MGLOG_E_ONCE("FlushPendingCommands: RetireCurrentCommandBuffer returned %d; draining submission",
retireResult);
if (WaitForSubmitsUpTo(m_submitCounter, UINT64_MAX)) {
// Every registered fence through this submission was waited.
} else if (vkQueueWaitIdle(m_graphicsQueue) == VK_SUCCESS) {
m_bufferManager.NotifyDeviceIdle();
OnSubmitsCompletedUpTo(m_submitCounter);
} else {
// Device is effectively lost; the command buffer may still be
// pending, but no recovery can make reuse legal.
MGLOG_E_ONCE("FlushPendingCommands: drain failed; command buffer reuse is unsafe");
}
}
return true;
}
void VulkanRenderer::SubmitAheadOfAcquiredImage() {
if (m_frameContext.GetFrameCount() == 0) return;
auto& frame = m_frameContext.GetCurrent();
if (frame.imageAvailableSemaphoreConsumed || frame.acquiredImageReferenced) return;
if (m_swapchainObject.GetHandle() == VK_NULL_HANDLE || m_presentSuspended) return;
// The frame's work so far renders off screen. Submitted together with the window pass it
// would wait behind the acquire semaphore, which signals only when the display releases
// the image - about every other frame that is the image it has just stopped scanning out,
// so the whole frame's GPU work started after the next display commit. Submitted now it
// runs while the display still holds the image. A failed flush leaves the work recorded,
// and it goes out later with the wait, as before. Inside a draw's preparation nothing is
// submitted (the draw may already hold state of the open recording); the image is only
// marked, so the submission that carries the use still waits.
if (m_wirePreparationDepth == 0 && HasPendingRecordedWork() && FlushPendingCommands())
MGLOG_I_ONCE("DirectVulkan: a frame's off-screen work goes to the queue ahead of its window pass; only "
"the submission that uses the swapchain image waits for its acquire");
frame.acquiredImageReferenced = true;
}
void VulkanRenderer::NoteDeviceLoss(VkResult result, const char* where) {
if (result != VK_ERROR_DEVICE_LOST || m_deviceLost) return;
m_deviceLost = true;
MGLOG_E("DirectVulkan: the VkDevice is LOST - %s returned VK_ERROR_DEVICE_LOST (the driver reset a GPU "
"fault or hang); nothing this renderer recorded or submits from here on will execute",
where != nullptr ? where : "a Vulkan call");
#if MOBILEGL_BUILD_DISAGGREGATED
// Latch NOW where a session latch exists, so the session stops at its next record however
// the verb in hand unwinds - including through a caller that only logs a failed submit.
if (MG_Pipe::MGPipeSessionLatchArmed()) (void)LatchWireDeviceLoss(where);
#endif
}
Bool VulkanRenderer::IsDeviceLost() {
// Only failure paths ask, so the probe's wait never lands on a healthy frame: a device
// that is merely slow answers VK_SUCCESS once its work drains, a lost one answers at once.
if (!m_deviceLost && m_device != VK_NULL_HANDLE) {
NoteDeviceLoss(vkDeviceWaitIdle(m_device), "the device-loss probe's vkDeviceWaitIdle");
}
return m_deviceLost;
}
Bool VulkanRenderer::LatchWireDeviceLoss(const char* site) {
if (!IsDeviceLost()) return false;
if (!m_deviceLossLatched) {
m_deviceLossLatched = true;
// Returns (latched) on a served session; dies, as the site's own Fatal would have,
// where no session latch is armed.
(void)MG_Pipe::MGPipeSessionLatch(MG_Pipe::MGPipeFatalFamily::DeviceLost,
"MGPipe: Fatal{BackendDeviceLost, \"Magma:%s\"} - this session's "
"VkDevice was lost to a GPU fault or hang; the session ends and its "
"client reads a lost context, every other session keeps running",
site != nullptr ? site : "?");
}
return true;
}
Bool VulkanRenderer::LatchIfGpuHung(const char* site) {
if (!m_progressMarkers.Hung()) return false;
if (!m_deviceLost) {
// NOT LOST TO THE DRIVER: a submission that a failed preemption leaves running is reset
// without a device loss, and the next one would stop the GPU again. Treated as lost from
// here on - nothing more is submitted - and the session ends as one whose device was.
m_deviceLost = true;
MGLOG_E("DirectVulkan: the GPU hang watch named this session's VkDevice (a submission ran %llu ms "
"without finishing, at %s); the device is treated as LOST and nothing more is submitted",
static_cast<unsigned long long>(m_progressMarkers.HungRunningMs()), site != nullptr ? site : "?");
}
// Where no session latch is armed (no served session) the device only stays "lost".
if (!m_deviceLossLatched && MG_Pipe::MGPipeSessionLatchArmed()) {
m_deviceLossLatched = true;
(void)MG_Pipe::MGPipeSessionLatch(MG_Pipe::MGPipeFatalFamily::DeviceLost,
"MGPipe: Fatal{BackendDeviceLost, \"Magma:gpu-hang:%s\"} - this "
"session's GPU work ran past the hang budget and stopped the GPU for "
"every process; the session ends and its client reads a lost context, "
"every other session keeps running",
site != nullptr ? site : "?");
}
return true;
}
Bool VulkanRenderer::FlushForSyncPoint(Uint64 submitIndex) {
// A flush only helps a sync point whose commands are not submitted
// yet; for an already-submitted index it would just split the frame's
// render pass (a full tile load/store on TBDR GPUs) without advancing
// the fence.
if (submitIndex <= m_submitCounter) {
return false;
}
return FlushPendingCommands();
}
void VulkanRenderer::SplitOversizedRecording() {
const Uint32 maxDraws = MG_Config::Features.MagmaMaxDrawsPerCommandBuffer;
if (maxDraws == 0) {
return;
}
// Counted before the check, so a buffer carries at most maxDraws calls before the one
// that splits it. OnFrameCommandRecordingBegan resets the count on every new recording.
if (++m_drawsInRecording <= maxDraws || !HasPendingRecordedWork()) {
return;
}
// Nothing of the current call is recorded yet, so the split falls between two GL calls,
// exactly where a glFlush would put it: the render pass ends with STORE, the next draw
// re-begins it with LOAD, and queries and transform feedback are bracketed per draw.
if (!FlushPendingCommands()) {
return;
}
// Fresh command buffer: the same reset every other mid-frame flush site applies.
++m_oversizedRecordingSplits;
MGLOG_I_ONCE("DirectVulkan: a frame recorded more than %u draws into one command buffer; submitting it and "
"continuing on a fresh one (MOBILEGL_MAGMA_MAX_DRAWS_PER_COMMAND_BUFFER, 0 = unbounded)",
maxDraws);
MGLOG_D("DirectVulkan: split an oversized recording (%llu so far, submit index %llu)",
static_cast<unsigned long long>(m_oversizedRecordingSplits),
static_cast<unsigned long long>(m_submitCounter));
// Splitting only bounds the driver's command memory if the submitted buffers are freed
// as fast as they are produced. A GPU that falls behind would park them all on the
// retired list, which is the same growth spread over more buffers - so hold the queue
// to frames-in-flight + 1 outstanding submissions, the steady state of a pipelined
// frame plus this split. Wait on the fence directly rather than through
// WaitForSubmitIndex: its opportunistic TryDrainFrameTransients may rewind the transient
// arena, and this runs at the start of a draw whose caller may already hold slices of it.
const SizeT maxOutstanding = static_cast<SizeT>(m_frameContext.GetFrameCount()) + 1;
while (m_inFlightSubmits.size() > maxOutstanding) {
const SubmitRecord oldest = m_inFlightSubmits.front();
if (oldest.fence == VK_NULL_HANDLE) {
break;
}
const VkResult waitResult = vkWaitForFences(m_device, 1, &oldest.fence, VK_TRUE, UINT64_MAX);
if (waitResult != VK_SUCCESS) {
NoteDeviceLoss(waitResult, "SplitOversizedRecording vkWaitForFences");
MGLOG_E_ONCE("SplitOversizedRecording: vkWaitForFences returned %d", waitResult);
break;
}
OnSubmitsCompletedUpTo(oldest.submitIndex);
}
}
Bool VulkanRenderer::WaitForSubmitIndex(Uint64 submitIndex, Uint64 timeoutNs, Bool flushIfPending) {
if (IsSubmitIndexComplete(submitIndex)) {
return true;
}
if (submitIndex > m_submitCounter) {
if (!flushIfPending) {
return false;
}
FlushPendingCommands();
if (submitIndex > m_submitCounter) {
// Nothing could be submitted (empty batch or submit failure);
// the index cannot complete yet.
return false;
}
}
if (!WaitForSubmitsUpTo(submitIndex, timeoutNs)) return false;
// A blocking wait can drain a present-less loop's frame transients.
TryDrainFrameTransients();
return true;
}
void VulkanRenderer::OnFrameCommandRecordingBegan(VkCommandBuffer commandBuffer) {
m_drawsInRecording = 0;
// Dynamic state does not survive a command-buffer boundary.
ResetDynamicStateShadow();
if (m_uniformManager) {
m_uniformManager->OnCommandBufferBoundary();
}
// Pre-pass stream bookkeeping: a fresh frame recording references no
// textures yet.
if (m_textureManager) {
m_textureManager->AdvanceRecordingGeneration();
}
if (m_timerQueryManager) {
m_timerQueryManager->OnFrameCommandRecordingBegan(commandBuffer, m_frameContext.GetCurrentFrameIndex(),
m_bufferManager.GetFrameSerial());
}
}
VkProvokingVertexModeEXT VulkanRenderer::SelectProvokingVertexMode(VkPrimitiveTopology topology,
Bool capturesXfbFromGeometryStage) const {
if (!m_provokingVertexLastEnabled) {
return VK_PROVOKING_VERTEX_MODE_FIRST_VERTEX_EXT;
}
// Measured, and identical on lavapipe and on the NVIDIA Vulkan driver: a geometry shader's
// emitted triangle strip is already recorded in GL's provoking-last vertex order, so asking
// for LAST rotates it a second time. The input-assembler path has the opposite problem, and
// the mode is a single pipeline bit, so the two cannot be satisfied at once: a program that
// both runs a geometry shader and captures transform feedback keeps Vulkan's own convention,
// and pays for it with a GL-wrong flat vertex in that one case. Deliberately a link-time
// program property, not IsTransformFeedbackActive() - see the memo note in the header.
if (capturesXfbFromGeometryStage) {
return VK_PROVOKING_VERTEX_MODE_FIRST_VERTEX_EXT;
}
// VUID-VkGraphicsPipelineCreateInfo-topology-04884 only bites when
// transformFeedbackPreservesProvokingVertex is enabled; when it is not, a fan may take LAST.
if (m_provokingVertexXfbPreserveEnabled && topology == VK_PRIMITIVE_TOPOLOGY_TRIANGLE_FAN &&
!m_provokingVertexFanPreserved) {
return VK_PROVOKING_VERTEX_MODE_FIRST_VERTEX_EXT;
}
// Only provokingVertexModePerPipeline lets modes differ inside one render pass instance;
// elsewhere every pipeline takes GL's default so the render pass stays self-consistent, and
// glProvokingVertex(GL_FIRST_VERTEX_CONVENTION) goes unhonoured. Honouring it there would
// mean ending the render pass on every glProvokingVertex change; not worth it until a target
// device actually lacks the property.
if (!m_provokingVertexModePerPipeline) {
return VK_PROVOKING_VERTEX_MODE_LAST_VERTEX_EXT;
}
return (MG_Pipe::gPipeInputs->IsLive() &&
MG_Pipe::gPipeInputs->GetProvokingVertexMode() == ProvokingVertexMode::FirstVertex)
? VK_PROVOKING_VERTEX_MODE_FIRST_VERTEX_EXT
: VK_PROVOKING_VERTEX_MODE_LAST_VERTEX_EXT;
}
Bool VulkanRenderer::IsTimerQuerySupported() const {
return m_timerQuerySupported && m_timerQueryManager != nullptr;
}
SharedPtr<VkTimerQueryManager::TimestampRecord> VulkanRenderer::WriteTimerQueryTimestamp() {
if (!IsTimerQuerySupported() || m_device == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return nullptr;
}
auto& frame = m_frameContext.GetCurrent();
if (!frame.isCommandRecording) {
m_frameContext.BeginCommandRecording();
}
// vkCmdWriteTimestamp is valid both inside and outside a render pass,
// so any active render pass is left untouched.
return m_timerQueryManager->WriteTimestamp(frame.commandBuffer, m_frameContext.GetCurrentFrameIndex(),
m_bufferManager.GetFrameSerial());
}
Bool VulkanRenderer::IsTimerQueryResultReady(VkTimerQueryManager::TimestampRecord& record) {
if (record.harvested) {
return true;
}
if (!m_timerQueryManager) {
return false;
}
// Ask the pool first. It polls with VK_QUERY_RESULT_WITH_AVAILABILITY_BIT and is the
// authority on whether the timestamp has landed; the frame serial is not, because it only
// advances at Present and neither completion notifier will mark the CURRENT serial done - so
// a timestamp written and fence-waited inside one GL frame could never be read back in it.
if (m_timerQueryManager->TryHarvest(record)) {
return true;
}
return IsFrameSerialComplete(record.frameSerial) && m_timerQueryManager->TryHarvest(record);
}
Bool VulkanRenderer::WaitForTimerQueryResult(VkTimerQueryManager::TimestampRecord& record) {
if (IsTimerQueryResultReady(record)) {
return true;
}
// WaitForFrameSerial refuses serials that cannot complete without
// further submissions (a timestamp written this frame only executes
// once Present submits the command buffer), so this returns false
// instead of deadlocking; the record resolves after a later Present.
if (!WaitForFrameSerial(record.frameSerial, UINT64_MAX)) {
return false;
}
return IsTimerQueryResultReady(record);
}
Uint64 VulkanRenderer::GetTimerQueryElapsedNs(const VkTimerQueryManager::TimestampRecord& begin,
const VkTimerQueryManager::TimestampRecord& end) const {
return m_timerQueryManager ? m_timerQueryManager->ElapsedNs(begin, end) : 0;
}
Uint64 VulkanRenderer::GetTimerQueryTimestampNs(const VkTimerQueryManager::TimestampRecord& record) const {
return m_timerQueryManager ? m_timerQueryManager->TimestampNs(record) : 0;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -1,4 +1,4 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/WireColorBlit.inc
// MobileGL - MobileGL/MG_Backend/DirectVulkan/Renderer/WireColorBlit.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
@@ -6,7 +6,31 @@
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// Included by VulkanRenderer.cpp inside the backend namespace, after WireFramebuffer.inc.
#include "VulkanRendererInternal.h"
namespace MobileGL::MG_Backend::DirectVulkan {
namespace {
enum class BlitSurfaceTransform : Uint32 {
Identity = 0,
Rotate90 = 1,
Rotate180 = 2,
Rotate270 = 3,
};
static BlitSurfaceTransform ToBlitSurfaceTransform(VkSurfaceTransformFlagBitsKHR preTransform) {
switch (preTransform) {
case VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR:
return BlitSurfaceTransform::Rotate90;
case VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR:
return BlitSurfaceTransform::Rotate180;
case VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR:
return BlitSurfaceTransform::Rotate270;
default:
return BlitSurfaceTransform::Identity;
}
}
} // namespace
// The native blit owns no frontend object and never invokes the GL shader compiler.
#include "WireColorBlitSpirv.h"
@@ -106,9 +130,9 @@
m_wireColorBlitResources = nullptr;
}
Bool VulkanRenderer::BlitWireColorToDefault(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter) {
Bool VulkanRenderer::BlitWireColorToDefault(WireImage source, WireImage destination, GLint sx0, GLint sy0,
GLint sx1, GLint sy1, GLint dx0, GLint dy0, GLint dx1, GLint dy1,
GLenum filter) {
if (!destination.isDefault) return false;
return BlitWireColorImage(source, destination, sx0, sy0, sx1, sy1, dx0, dy0, dx1, dy1, filter, false);
}
@@ -126,19 +150,19 @@
// there is no defined result to be wrong about - only a shape that used to end the session
// and now produces the source's bits. For a float source of the wrong VIEW TYPE the copy is
// a plain slice extraction and the result is exactly right.
Bool VulkanRenderer::BlitWireColorToDefaultThroughScratch(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter) {
Bool VulkanRenderer::BlitWireColorToDefaultThroughScratch(WireImage source, WireImage destination, GLint sx0,
GLint sy0, GLint sx1, GLint sy1, GLint dx0, GLint dy0,
GLint dx1, GLint dy1, GLenum filter) {
if (!destination.isDefault || source.isDefault) return false;
if (source.samples != VK_SAMPLE_COUNT_1_BIT || destination.samples != VK_SAMPLE_COUNT_1_BIT)
return false;
if (source.samples != VK_SAMPLE_COUNT_1_BIT || destination.samples != VK_SAMPLE_COUNT_1_BIT) return false;
if (!source.image || !destination.image || source.image == destination.image) return false;
const Int64 width = std::abs(static_cast<Int64>(sx1) - sx0);
const Int64 height = std::abs(static_cast<Int64>(sy1) - sy0);
if (!width || !height || dx0 == dx1 || dy0 == dy1) return true;
if (std::min(sx0, sx1) < 0 || std::min(sy0, sy1) < 0 ||
static_cast<Uint64>(std::max(sx0, sx1)) > source.extent.width ||
static_cast<Uint64>(std::max(sy0, sy1)) > source.extent.height) return false;
static_cast<Uint64>(std::max(sy0, sy1)) > source.extent.height)
return false;
// A compressed source cannot be a colour attachment, so it cannot be a framebuffer blit's
// source either - but the block arithmetic below would silently produce nonsense for one,
// and "cannot happen" is the reason the guards this package retires were written as
@@ -147,16 +171,28 @@
if (vkuFormatIsCompressed(source.format)) return false;
VkFormat scratchFormat = VK_FORMAT_UNDEFINED;
switch (vkuFormatTexelBlockSize(source.format)) {
case 1: scratchFormat = VK_FORMAT_R8_UNORM; break;
case 2: scratchFormat = VK_FORMAT_R8G8_UNORM; break;
case 4: scratchFormat = VK_FORMAT_R8G8B8A8_UNORM; break;
case 8: scratchFormat = VK_FORMAT_R16G16B16A16_SFLOAT; break;
case 16: scratchFormat = VK_FORMAT_R32G32B32A32_SFLOAT; break;
default: return false;
case 1:
scratchFormat = VK_FORMAT_R8_UNORM;
break;
case 2:
scratchFormat = VK_FORMAT_R8G8_UNORM;
break;
case 4:
scratchFormat = VK_FORMAT_R8G8B8A8_UNORM;
break;
case 8:
scratchFormat = VK_FORMAT_R16G16B16A16_SFLOAT;
break;
case 16:
scratchFormat = VK_FORMAT_R32G32B32A32_SFLOAT;
break;
default:
return false;
}
VkFormatProperties scratchProperties{};
vkGetPhysicalDeviceFormatProperties(m_physicalDevice.handle, scratchFormat, &scratchProperties);
const VkFormatFeatureFlags scratchRequired = VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT |
const VkFormatFeatureFlags scratchRequired =
VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT |
(filter == GL_LINEAR ? VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_LINEAR_BIT : 0);
if ((scratchProperties.optimalTilingFeatures & scratchRequired) != scratchRequired) return false;
@@ -173,8 +209,8 @@
VmaAllocationCreateInfo allocation{};
allocation.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
allocation.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
if (vmaCreateImage(m_allocator, &info, &allocation, &scratch->image, &scratch->allocation,
nullptr) != VK_SUCCESS)
if (vmaCreateImage(m_allocator, &info, &allocation, &scratch->image, &scratch->allocation, nullptr) !=
VK_SUCCESS)
return false;
scratch->format = scratchFormat;
scratch->extent = {info.extent.width, info.extent.height};
@@ -194,14 +230,12 @@
// A 3D source carries its slice on z and always names layer 0; every other source
// carries it on the layer axis, which is the same split ResolveWireImage records.
copy.srcSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, source.level, source.is3D ? 0u : source.layer, 1};
copy.srcOffset = {std::min(sx0, sx1), std::min(sy0, sy1),
source.is3D ? static_cast<Int32>(source.layer) : 0};
copy.srcOffset = {std::min(sx0, sx1), std::min(sy0, sy1), source.is3D ? static_cast<Int32>(source.layer) : 0};
copy.dstSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1};
copy.dstOffset = {0, 0, 0};
copy.extent = {static_cast<Uint32>(width), static_cast<Uint32>(height), 1};
vkCmdCopyImage(m_frameContext.GetCurrent().commandBuffer, source.image,
VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, staged.image,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &copy);
vkCmdCopyImage(m_frameContext.GetCurrent().commandBuffer, source.image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
staged.image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &copy);
staged.layout = scratch->layout;
// The scratch holds the region at its own origin, so a reversed source rectangle
// becomes a reversed rectangle OVER THE SCRATCH - the multisample resolve arm's shape.
@@ -209,8 +243,8 @@
const GLint rx1 = sx1 > sx0 ? static_cast<GLint>(width) : 0;
const GLint ry0 = sy1 > sy0 ? 0 : static_cast<GLint>(height);
const GLint ry1 = sy1 > sy0 ? static_cast<GLint>(height) : 0;
const Bool blitted = BlitWireColorImage(staged, destination, rx0, ry0, rx1, ry1,
dx0, dy0, dx1, dy1, filter, false);
const Bool blitted =
BlitWireColorImage(staged, destination, rx0, ry0, rx1, ry1, dx0, dy0, dx1, dy1, filter, false);
// Retired whether or not the blit took: the copy above is already recorded against this
// image, so the submission that consumes it owns the storage either way.
WireRetiredObjects retired{};
@@ -219,39 +253,44 @@
return blitted;
}
Bool VulkanRenderer::BlitWireColorImage(WireImage source, WireImage destination,
GLint sx0, GLint sy0, GLint sx1, GLint sy1,
GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter, Bool mipmap) {
Bool VulkanRenderer::BlitWireColorImage(WireImage source, WireImage destination, GLint sx0, GLint sy0, GLint sx1,
GLint sy1, GLint dx0, GLint dy0, GLint dx1, GLint dy1, GLenum filter,
Bool mipmap) {
const Bool sameImage = source.image == destination.image;
if (source.isDefault || (sameImage && source.level == destination.level && source.layer == destination.layer) ||
source.samples != VK_SAMPLE_COUNT_1_BIT || destination.samples != VK_SAMPLE_COUNT_1_BIT ||
source.is3D || destination.is3D || (source.viewType != VK_IMAGE_VIEW_TYPE_2D &&
source.viewType != VK_IMAGE_VIEW_TYPE_2D_ARRAY) ||
source.samples != VK_SAMPLE_COUNT_1_BIT || destination.samples != VK_SAMPLE_COUNT_1_BIT || source.is3D ||
destination.is3D ||
(source.viewType != VK_IMAGE_VIEW_TYPE_2D && source.viewType != VK_IMAGE_VIEW_TYPE_2D_ARRAY) ||
(destination.viewType != VK_IMAGE_VIEW_TYPE_2D && destination.viewType != VK_IMAGE_VIEW_TYPE_2D_ARRAY) ||
!vkuFormatIsSampledFloat(source.format)) return false;
!vkuFormatIsSampledFloat(source.format))
return false;
if (sx0 == sx1 || sy0 == sy1 || dx0 == dx1 || dy0 == dy1) return true;
if (!source.extent.width || !source.extent.height ||
!destination.extent.width || !destination.extent.height) return false;
const auto transform = destination.isDefault ? m_swapchainObject.GetPreTransform() :
VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR;
if (transform != VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR &&
transform != VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR &&
if (!source.extent.width || !source.extent.height || !destination.extent.width || !destination.extent.height)
return false;
const auto transform =
destination.isDefault ? m_swapchainObject.GetPreTransform() : VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR;
if (transform != VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR && transform != VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR &&
transform != VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR &&
transform != VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR) return false;
transform != VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR)
return false;
VkFormatProperties sourceProperties{};
vkGetPhysicalDeviceFormatProperties(m_physicalDevice.handle, source.format, &sourceProperties);
const VkFormatFeatureFlags required = VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT |
const VkFormatFeatureFlags required =
VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT |
(filter == GL_LINEAR ? VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_LINEAR_BIT : 0);
if ((sourceProperties.optimalTilingFeatures & required) != required) return false;
// Mip generation samples and writes the texture's own color space;
// GL_FRAMEBUFFER_SRGB only controls framebuffer operations.
if (!mipmap) destination.format = ResolveSrgbAttachmentWriteFormat(destination.format,
MG_Pipe::gPipeInputs->IsCapabilityEnabled(CapabilityInput::FramebufferSrgb));
if (!mipmap)
destination.format = ResolveSrgbAttachmentWriteFormat(
destination.format, MG_Pipe::gPipeInputs->IsCapabilityEnabled(CapabilityInput::FramebufferSrgb));
VkFormatProperties destinationProperties{};
vkGetPhysicalDeviceFormatProperties(m_physicalDevice.handle, destination.format, &destinationProperties);
if (!(destinationProperties.optimalTilingFeatures & VK_FORMAT_FEATURE_COLOR_ATTACHMENT_BIT)) return false;
const VkImageLayout sourceLayout = sameImage ? VK_IMAGE_LAYOUT_GENERAL : VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
const VkImageLayout destinationLayout = sameImage ? VK_IMAGE_LAYOUT_GENERAL : VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
const VkImageLayout sourceLayout =
sameImage ? VK_IMAGE_LAYOUT_GENERAL : VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
const VkImageLayout destinationLayout =
sameImage ? VK_IMAGE_LAYOUT_GENERAL : VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
struct Constants {
float srcRect[4];
@@ -270,8 +309,8 @@
};
shader(kWireColorBlitVertexSpirv, sizeof(kWireColorBlitVertexSpirv), cache.vertex);
shader(kWireColorBlitFragmentSpirv, sizeof(kWireColorBlitFragmentSpirv), cache.fragment);
const VkDescriptorSetLayoutBinding binding{0, VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,
1, VK_SHADER_STAGE_FRAGMENT_BIT, nullptr};
const VkDescriptorSetLayoutBinding binding{0, VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 1,
VK_SHADER_STAGE_FRAGMENT_BIT, nullptr};
VkDescriptorSetLayoutCreateInfo layout{VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO};
layout.bindingCount = 1;
layout.pBindings = &binding;
@@ -295,7 +334,10 @@
auto& cache = *m_wireColorBlitResources;
WireColorBlitResources::Target* target = nullptr;
for (auto& entry : cache.targets) {
if (entry.format == destination.format && entry.layout == destinationLayout) { target = &entry; break; }
if (entry.format == destination.format && entry.layout == destinationLayout) {
target = &entry;
break;
}
}
if (!target) {
cache.targets.emplace_back();
@@ -330,8 +372,10 @@
stages[0].module = cache.vertex;
stages[1].stage = VK_SHADER_STAGE_FRAGMENT_BIT;
stages[1].module = cache.fragment;
const VkPipelineVertexInputStateCreateInfo vertex{VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO};
VkPipelineInputAssemblyStateCreateInfo assembly{VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO};
const VkPipelineVertexInputStateCreateInfo vertex{
VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO};
VkPipelineInputAssemblyStateCreateInfo assembly{
VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO};
assembly.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST;
VkPipelineViewportStateCreateInfo viewport{VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO};
viewport.viewportCount = viewport.scissorCount = 1;
@@ -343,8 +387,8 @@
VkPipelineMultisampleStateCreateInfo samples{VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO};
samples.rasterizationSamples = VK_SAMPLE_COUNT_1_BIT;
VkPipelineColorBlendAttachmentState colorMask{};
colorMask.colorWriteMask = VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT |
VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT;
colorMask.colorWriteMask = VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT |
VK_COLOR_COMPONENT_A_BIT;
VkPipelineColorBlendStateCreateInfo blend{VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO};
blend.attachmentCount = 1;
blend.pAttachments = &colorMask;
@@ -372,8 +416,10 @@
// output. GENERAL keeps the whole-image layout tracker truthful; the
// transition also orders the previous mip's attachment write.
TransitionWireImage(source, sourceLayout);
if (sameImage) destination.layout = source.layout;
else TransitionWireImage(destination, destinationLayout);
if (sameImage)
destination.layout = source.layout;
else
TransitionWireImage(destination, destinationLayout);
const auto makeView = [&](const WireImage& image) {
VkImageViewCreateInfo view{VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO};
view.image = image.image;
@@ -416,7 +462,10 @@
auto& cachedTargets = cache.cachedTargets;
WireColorBlitResources::CachedTarget* hit = nullptr;
for (auto& entry : cachedTargets)
if (sameTarget(entry)) { hit = &entry; break; }
if (sameTarget(entry)) {
hit = &entry;
break;
}
if (hit == nullptr) {
if (cachedTargets.size() >= WireColorBlitResources::kCachedTargets) {
WireRetiredObjects evicted{};
@@ -463,7 +512,8 @@
VK_VERIFY(vkCreateDescriptorPool(m_device, &pool, nullptr, &cache.framePools[frameIndex]));
}
alloc.descriptorPool = cache.framePools[frameIndex];
if (vkAllocateDescriptorSets(m_device, &alloc, &descriptorSet) != VK_SUCCESS) descriptorSet = VK_NULL_HANDLE;
if (vkAllocateDescriptorSets(m_device, &alloc, &descriptorSet) != VK_SUCCESS)
descriptorSet = VK_NULL_HANDLE;
}
if (descriptorSet == VK_NULL_HANDLE) {
const VkDescriptorPoolSize poolSize{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 1};
@@ -480,10 +530,10 @@
// at a quantised coordinate is not (WireColorBlitFilter.h). The format check above still
// asks for SAMPLED_IMAGE_FILTER_LINEAR whenever GL said GL_LINEAR, so which blits reach
// this pass is unchanged.
const VkFilter samplerFilter = WireColorBlitFilterFor({sx0, sy0, sx1, sy1}, {dx0, dy0, dx1, dy1},
filter, transform);
const VkFilter samplerFilter =
WireColorBlitFilterFor({sx0, sy0, sx1, sy1}, {dx0, dy0, dx1, dy1}, filter, transform);
const VkDescriptorImageInfo sampled{samplerFilter == VK_FILTER_LINEAR ? cache.linear : cache.nearest,
sourceView, sourceLayout};
sourceView, sourceLayout};
VkWriteDescriptorSet write{VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET};
write.dstSet = descriptorSet;
write.dstBinding = 0;
@@ -493,8 +543,8 @@
vkUpdateDescriptorSets(m_device, 1, &write, 0, nullptr);
const float sw = static_cast<float>(source.extent.width), sh = static_cast<float>(source.extent.height);
const Bool quarterTurn = transform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR ||
transform == VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR;
const Bool quarterTurn =
transform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR || transform == VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR;
const float dw = static_cast<float>(quarterTurn ? destination.extent.height : destination.extent.width);
const float dh = static_cast<float>(quarterTurn ? destination.extent.width : destination.extent.height);
// The shared vertex shader flips window Y. Offscreen images keep the
@@ -503,11 +553,10 @@
dy0 = static_cast<GLint>(destination.extent.height) - dy0;
dy1 = static_cast<GLint>(destination.extent.height) - dy1;
}
const Constants constants{{sx0 / sw, sy0 / sh, (static_cast<float>(sx1) - sx0) / sw,
(static_cast<float>(sy1) - sy0) / sh},
{dx0 / dw, dy0 / dh, (static_cast<float>(dx1) - dx0) / dw,
(static_cast<float>(dy1) - dy0) / dh},
static_cast<Int32>(ToBlitSurfaceTransform(transform))};
const Constants constants{
{sx0 / sw, sy0 / sh, (static_cast<float>(sx1) - sx0) / sw, (static_cast<float>(sy1) - sy0) / sh},
{dx0 / dw, dy0 / dh, (static_cast<float>(dx1) - dx0) / dw, (static_cast<float>(dy1) - dy0) / dh},
static_cast<Int32>(ToBlitSurfaceTransform(transform))};
auto& frame = m_frameContext.GetCurrent();
VkRenderPassBeginInfo begin{VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO};
begin.renderPass = target->renderPass;
@@ -516,15 +565,16 @@
EndActiveRenderPassOn(frame.commandBuffer);
vkCmdBeginRenderPass(frame.commandBuffer, &begin, VK_SUBPASS_CONTENTS_INLINE);
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, target->pipeline);
const VkViewport viewport{0.0f, 0.0f, static_cast<float>(destination.extent.width),
static_cast<float>(destination.extent.height), 0.0f, 1.0f};
const VkViewport viewport{
0.0f, 0.0f, static_cast<float>(destination.extent.width), static_cast<float>(destination.extent.height),
0.0f, 1.0f};
const VkRect2D scissor{{0, 0}, destination.extent};
vkCmdSetViewport(frame.commandBuffer, 0, 1, &viewport);
vkCmdSetScissor(frame.commandBuffer, 0, 1, &scissor);
vkCmdBindDescriptorSets(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS,
cache.pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);
vkCmdPushConstants(frame.commandBuffer, cache.pipelineLayout, VK_SHADER_STAGE_VERTEX_BIT,
0, sizeof(constants), &constants);
vkCmdBindDescriptorSets(frame.commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, cache.pipelineLayout, 0, 1,
&descriptorSet, 0, nullptr);
vkCmdPushConstants(frame.commandBuffer, cache.pipelineLayout, VK_SHADER_STAGE_VERTEX_BIT, 0, sizeof(constants),
&constants);
vkCmdDraw(frame.commandBuffer, 6, 1, 0, 0);
vkCmdEndRenderPass(frame.commandBuffer);
ResetDynamicStateShadow();
@@ -540,3 +590,4 @@
if (destination.isDefault) m_swapchainObject.SetImageContentDefined(destination.swapchainImageIndex, true);
return true;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -15,7 +15,7 @@
//
// GL applies a blit's `filter` only where the image is STRETCHED (GL 4.6 core 18.3.1). A blit whose
// source and destination rectangles are the same size is a copy, whatever `filter` says, and so is
// a mirrored one. The wire arm's shader blit (WireColorBlit.inc, BlitWireColorImage) draws a quad
// a mirrored one. The wire arm's shader blit (WireColorBlit.cpp, BlitWireColorImage) draws a quad
// and samples `textureLod(sourceImage, texCoord, 0)` at interpolated coordinates, and it used the
// LINEAR sampler for every GL_LINEAR blit. At 1:1 every fragment centre maps to a texel centre, so
// in exact arithmetic linear filtering reads the same single texel. Adreno 830 quantises the
@@ -44,15 +44,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Same extent on both axes, orientation aside. The extents are taken in 64 bits: GL accepts any
// GLint corner, and INT_MAX - INT_MIN does not fit in one.
inline Bool WireBlitIsUnscaled(const WireBlitRect& source, const WireBlitRect& destination) {
const auto extent = [](GLint a, GLint b) {
return std::abs(static_cast<Int64>(b) - static_cast<Int64>(a));
};
const auto extent = [](GLint a, GLint b) { return std::abs(static_cast<Int64>(b) - static_cast<Int64>(a)); };
return extent(source.x0, source.x1) == extent(destination.x0, destination.x1) &&
extent(source.y0, source.y1) == extent(destination.y0, destination.y1);
}
inline VkFilter WireColorBlitFilterFor(const WireBlitRect& source, const WireBlitRect& destination,
GLenum filter, VkSurfaceTransformFlagBitsKHR transform) {
inline VkFilter WireColorBlitFilterFor(const WireBlitRect& source, const WireBlitRect& destination, GLenum filter,
VkSurfaceTransformFlagBitsKHR transform) {
if (filter != GL_LINEAR) return VK_FILTER_NEAREST;
const Bool expressed = transform == VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR ||
transform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR ||
@@ -17,124 +17,111 @@
#pragma once
static constexpr Uint32 kWireColorBlitVertexSpirv[] = {
0x07230203u, 0x00010000u, 0x0008000bu, 0x00000070u, 0x00000000u, 0x00020011u, 0x00000001u, 0x0006000bu,
0x00000001u, 0x4c534c47u, 0x6474732eu, 0x3035342eu, 0x00000000u, 0x0003000eu, 0x00000000u, 0x00000001u,
0x0008000fu, 0x00000000u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x00000016u, 0x0000005du, 0x00000066u,
0x00030003u, 0x00000002u, 0x000001c2u, 0x00040005u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x00030005u,
0x00000009u, 0x00007675u, 0x00060005u, 0x00000016u, 0x565f6c67u, 0x65747265u, 0x646e4978u, 0x00007865u,
0x00050005u, 0x00000019u, 0x65646e69u, 0x6c626178u, 0x00000065u, 0x00030005u, 0x0000001cu, 0x00000070u,
0x00050005u, 0x0000001eu, 0x74696c42u, 0x74636552u, 0x00000000u, 0x00050006u, 0x0000001eu, 0x00000000u,
0x52637273u, 0x00746365u, 0x00050006u, 0x0000001eu, 0x00000001u, 0x52747364u, 0x00746365u, 0x00080006u,
0x0000001eu, 0x00000002u, 0x66727573u, 0x54656361u, 0x736e6172u, 0x6d726f66u, 0x00000000u, 0x00040005u,
0x00000020u, 0x74636572u, 0x00000000u, 0x00060005u, 0x0000005bu, 0x505f6c67u, 0x65567265u, 0x78657472u,
0x00000000u, 0x00060006u, 0x0000005bu, 0x00000000u, 0x505f6c67u, 0x7469736fu, 0x006e6f69u, 0x00070006u,
0x0000005bu, 0x00000001u, 0x505f6c67u, 0x746e696fu, 0x657a6953u, 0x00000000u, 0x00070006u, 0x0000005bu,
0x00000002u, 0x435f6c67u, 0x4470696cu, 0x61747369u, 0x0065636eu, 0x00070006u, 0x0000005bu, 0x00000003u,
0x435f6c67u, 0x446c6c75u, 0x61747369u, 0x0065636eu, 0x00030005u, 0x0000005du, 0x00000000u, 0x00050005u,
0x00000066u, 0x43786574u, 0x64726f6fu, 0x00000000u, 0x00040047u, 0x00000016u, 0x0000000bu, 0x0000002au,
0x00030047u, 0x0000001eu, 0x00000002u, 0x00050048u, 0x0000001eu, 0x00000000u, 0x00000023u, 0x00000000u,
0x00050048u, 0x0000001eu, 0x00000001u, 0x00000023u, 0x00000010u, 0x00050048u, 0x0000001eu, 0x00000002u,
0x00000023u, 0x00000020u, 0x00030047u, 0x0000005bu, 0x00000002u, 0x00050048u, 0x0000005bu, 0x00000000u,
0x0000000bu, 0x00000000u, 0x00050048u, 0x0000005bu, 0x00000001u, 0x0000000bu, 0x00000001u, 0x00050048u,
0x0000005bu, 0x00000002u, 0x0000000bu, 0x00000003u, 0x00050048u, 0x0000005bu, 0x00000003u, 0x0000000bu,
0x00000004u, 0x00040047u, 0x00000066u, 0x0000001eu, 0x00000000u, 0x00020013u, 0x00000002u, 0x00030021u,
0x00000003u, 0x00000002u, 0x00030016u, 0x00000006u, 0x00000020u, 0x00040017u, 0x00000007u, 0x00000006u,
0x00000002u, 0x00040020u, 0x00000008u, 0x00000007u, 0x00000007u, 0x00040015u, 0x0000000au, 0x00000020u,
0x00000000u, 0x0004002bu, 0x0000000au, 0x0000000bu, 0x00000006u, 0x0004001cu, 0x0000000cu, 0x00000007u,
0x0000000bu, 0x0004002bu, 0x00000006u, 0x0000000du, 0x00000000u, 0x0005002cu, 0x00000007u, 0x0000000eu,
0x0000000du, 0x0000000du, 0x0004002bu, 0x00000006u, 0x0000000fu, 0x3f800000u, 0x0005002cu, 0x00000007u,
0x00000010u, 0x0000000fu, 0x0000000du, 0x0005002cu, 0x00000007u, 0x00000011u, 0x0000000du, 0x0000000fu,
0x0005002cu, 0x00000007u, 0x00000012u, 0x0000000fu, 0x0000000fu, 0x0009002cu, 0x0000000cu, 0x00000013u,
0x0000000eu, 0x00000010u, 0x00000011u, 0x00000011u, 0x00000010u, 0x00000012u, 0x00040015u, 0x00000014u,
0x00000020u, 0x00000001u, 0x00040020u, 0x00000015u, 0x00000001u, 0x00000014u, 0x0004003bu, 0x00000015u,
0x00000016u, 0x00000001u, 0x00040020u, 0x00000018u, 0x00000007u, 0x0000000cu, 0x00040017u, 0x0000001du,
0x00000006u, 0x00000004u, 0x0005001eu, 0x0000001eu, 0x0000001du, 0x0000001du, 0x00000014u, 0x00040020u,
0x0000001fu, 0x00000009u, 0x0000001eu, 0x0004003bu, 0x0000001fu, 0x00000020u, 0x00000009u, 0x0004002bu,
0x00000014u, 0x00000021u, 0x00000001u, 0x00040020u, 0x00000022u, 0x00000009u, 0x0000001du, 0x0004002bu,
0x00000006u, 0x0000002cu, 0x40000000u, 0x0004002bu, 0x0000000au, 0x00000030u, 0x00000001u, 0x00040020u,
0x00000031u, 0x00000007u, 0x00000006u, 0x0004002bu, 0x00000014u, 0x00000036u, 0x00000002u, 0x00040020u,
0x00000037u, 0x00000009u, 0x00000014u, 0x00020014u, 0x0000003au, 0x0004002bu, 0x0000000au, 0x00000041u,
0x00000000u, 0x0004002bu, 0x00000014u, 0x00000050u, 0x00000003u, 0x0004001cu, 0x0000005au, 0x00000006u,
0x00000030u, 0x0006001eu, 0x0000005bu, 0x0000001du, 0x00000006u, 0x0000005au, 0x0000005au, 0x00040020u,
0x0000005cu, 0x00000003u, 0x0000005bu, 0x0004003bu, 0x0000005cu, 0x0000005du, 0x00000003u, 0x0004002bu,
0x00000014u, 0x0000005eu, 0x00000000u, 0x00040020u, 0x00000063u, 0x00000003u, 0x0000001du, 0x00040020u,
0x00000065u, 0x00000003u, 0x00000007u, 0x0004003bu, 0x00000065u, 0x00000066u, 0x00000003u, 0x00050036u,
0x00000002u, 0x00000004u, 0x00000000u, 0x00000003u, 0x000200f8u, 0x00000005u, 0x0004003bu, 0x00000008u,
0x00000009u, 0x00000007u, 0x0004003bu, 0x00000018u, 0x00000019u, 0x00000007u, 0x0004003bu, 0x00000008u,
0x0000001cu, 0x00000007u, 0x0004003du, 0x00000014u, 0x00000017u, 0x00000016u, 0x0003003eu, 0x00000019u,
0x00000013u, 0x00050041u, 0x00000008u, 0x0000001au, 0x00000019u, 0x00000017u, 0x0004003du, 0x00000007u,
0x0000001bu, 0x0000001au, 0x0003003eu, 0x00000009u, 0x0000001bu, 0x00050041u, 0x00000022u, 0x00000023u,
0x00000020u, 0x00000021u, 0x0004003du, 0x0000001du, 0x00000024u, 0x00000023u, 0x0007004fu, 0x00000007u,
0x00000025u, 0x00000024u, 0x00000024u, 0x00000000u, 0x00000001u, 0x0004003du, 0x00000007u, 0x00000026u,
0x00000009u, 0x00050041u, 0x00000022u, 0x00000027u, 0x00000020u, 0x00000021u, 0x0004003du, 0x0000001du,
0x00000028u, 0x00000027u, 0x0007004fu, 0x00000007u, 0x00000029u, 0x00000028u, 0x00000028u, 0x00000002u,
0x00000003u, 0x00050085u, 0x00000007u, 0x0000002au, 0x00000026u, 0x00000029u, 0x00050081u, 0x00000007u,
0x0000002bu, 0x00000025u, 0x0000002au, 0x0005008eu, 0x00000007u, 0x0000002du, 0x0000002bu, 0x0000002cu,
0x00050050u, 0x00000007u, 0x0000002eu, 0x0000000fu, 0x0000000fu, 0x00050083u, 0x00000007u, 0x0000002fu,
0x0000002du, 0x0000002eu, 0x0003003eu, 0x0000001cu, 0x0000002fu, 0x00050041u, 0x00000031u, 0x00000032u,
0x0000001cu, 0x00000030u, 0x0004003du, 0x00000006u, 0x00000033u, 0x00000032u, 0x0004007fu, 0x00000006u,
0x00000034u, 0x00000033u, 0x00050041u, 0x00000031u, 0x00000035u, 0x0000001cu, 0x00000030u, 0x0003003eu,
0x00000035u, 0x00000034u, 0x00050041u, 0x00000037u, 0x00000038u, 0x00000020u, 0x00000036u, 0x0004003du,
0x00000014u, 0x00000039u, 0x00000038u, 0x000500aau, 0x0000003au, 0x0000003bu, 0x00000039u, 0x00000021u,
0x000300f7u, 0x0000003du, 0x00000000u, 0x000400fau, 0x0000003bu, 0x0000003cu, 0x00000045u, 0x000200f8u,
0x0000003cu, 0x00050041u, 0x00000031u, 0x0000003eu, 0x0000001cu, 0x00000030u, 0x0004003du, 0x00000006u,
0x0000003fu, 0x0000003eu, 0x0004007fu, 0x00000006u, 0x00000040u, 0x0000003fu, 0x00050041u, 0x00000031u,
0x00000042u, 0x0000001cu, 0x00000041u, 0x0004003du, 0x00000006u, 0x00000043u, 0x00000042u, 0x00050050u,
0x00000007u, 0x00000044u, 0x00000040u, 0x00000043u, 0x0003003eu, 0x0000001cu, 0x00000044u, 0x000200f9u,
0x0000003du, 0x000200f8u, 0x00000045u, 0x00050041u, 0x00000037u, 0x00000046u, 0x00000020u, 0x00000036u,
0x0004003du, 0x00000014u, 0x00000047u, 0x00000046u, 0x000500aau, 0x0000003au, 0x00000048u, 0x00000047u,
0x00000036u, 0x000300f7u, 0x0000004au, 0x00000000u, 0x000400fau, 0x00000048u, 0x00000049u, 0x0000004du,
0x000200f8u, 0x00000049u, 0x0004003du, 0x00000007u, 0x0000004bu, 0x0000001cu, 0x0004007fu, 0x00000007u,
0x0000004cu, 0x0000004bu, 0x0003003eu, 0x0000001cu, 0x0000004cu, 0x000200f9u, 0x0000004au, 0x000200f8u,
0x0000004du, 0x00050041u, 0x00000037u, 0x0000004eu, 0x00000020u, 0x00000036u, 0x0004003du, 0x00000014u,
0x0000004fu, 0x0000004eu, 0x000500aau, 0x0000003au, 0x00000051u, 0x0000004fu, 0x00000050u, 0x000300f7u,
0x00000053u, 0x00000000u, 0x000400fau, 0x00000051u, 0x00000052u, 0x00000053u, 0x000200f8u, 0x00000052u,
0x00050041u, 0x00000031u, 0x00000054u, 0x0000001cu, 0x00000030u, 0x0004003du, 0x00000006u, 0x00000055u,
0x00000054u, 0x00050041u, 0x00000031u, 0x00000056u, 0x0000001cu, 0x00000041u, 0x0004003du, 0x00000006u,
0x00000057u, 0x00000056u, 0x0004007fu, 0x00000006u, 0x00000058u, 0x00000057u, 0x00050050u, 0x00000007u,
0x00000059u, 0x00000055u, 0x00000058u, 0x0003003eu, 0x0000001cu, 0x00000059u, 0x000200f9u, 0x00000053u,
0x000200f8u, 0x00000053u, 0x000200f9u, 0x0000004au, 0x000200f8u, 0x0000004au, 0x000200f9u, 0x0000003du,
0x000200f8u, 0x0000003du, 0x0004003du, 0x00000007u, 0x0000005fu, 0x0000001cu, 0x00050051u, 0x00000006u,
0x00000060u, 0x0000005fu, 0x00000000u, 0x00050051u, 0x00000006u, 0x00000061u, 0x0000005fu, 0x00000001u,
0x00070050u, 0x0000001du, 0x00000062u, 0x00000060u, 0x00000061u, 0x0000000du, 0x0000000fu, 0x00050041u,
0x00000063u, 0x00000064u, 0x0000005du, 0x0000005eu, 0x0003003eu, 0x00000064u, 0x00000062u, 0x00050041u,
0x00000022u, 0x00000067u, 0x00000020u, 0x0000005eu, 0x0004003du, 0x0000001du, 0x00000068u, 0x00000067u,
0x0007004fu, 0x00000007u, 0x00000069u, 0x00000068u, 0x00000068u, 0x00000000u, 0x00000001u, 0x0004003du,
0x00000007u, 0x0000006au, 0x00000009u, 0x00050041u, 0x00000022u, 0x0000006bu, 0x00000020u, 0x0000005eu,
0x0004003du, 0x0000001du, 0x0000006cu, 0x0000006bu, 0x0007004fu, 0x00000007u, 0x0000006du, 0x0000006cu,
0x0000006cu, 0x00000002u, 0x00000003u, 0x00050085u, 0x00000007u, 0x0000006eu, 0x0000006au, 0x0000006du,
0x00050081u, 0x00000007u, 0x0000006fu, 0x00000069u, 0x0000006eu, 0x0003003eu, 0x00000066u, 0x0000006fu,
0x000100fdu, 0x00010038u,
0x07230203u, 0x00010000u, 0x0008000bu, 0x00000070u, 0x00000000u, 0x00020011u, 0x00000001u, 0x0006000bu, 0x00000001u,
0x4c534c47u, 0x6474732eu, 0x3035342eu, 0x00000000u, 0x0003000eu, 0x00000000u, 0x00000001u, 0x0008000fu, 0x00000000u,
0x00000004u, 0x6e69616du, 0x00000000u, 0x00000016u, 0x0000005du, 0x00000066u, 0x00030003u, 0x00000002u, 0x000001c2u,
0x00040005u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x00030005u, 0x00000009u, 0x00007675u, 0x00060005u, 0x00000016u,
0x565f6c67u, 0x65747265u, 0x646e4978u, 0x00007865u, 0x00050005u, 0x00000019u, 0x65646e69u, 0x6c626178u, 0x00000065u,
0x00030005u, 0x0000001cu, 0x00000070u, 0x00050005u, 0x0000001eu, 0x74696c42u, 0x74636552u, 0x00000000u, 0x00050006u,
0x0000001eu, 0x00000000u, 0x52637273u, 0x00746365u, 0x00050006u, 0x0000001eu, 0x00000001u, 0x52747364u, 0x00746365u,
0x00080006u, 0x0000001eu, 0x00000002u, 0x66727573u, 0x54656361u, 0x736e6172u, 0x6d726f66u, 0x00000000u, 0x00040005u,
0x00000020u, 0x74636572u, 0x00000000u, 0x00060005u, 0x0000005bu, 0x505f6c67u, 0x65567265u, 0x78657472u, 0x00000000u,
0x00060006u, 0x0000005bu, 0x00000000u, 0x505f6c67u, 0x7469736fu, 0x006e6f69u, 0x00070006u, 0x0000005bu, 0x00000001u,
0x505f6c67u, 0x746e696fu, 0x657a6953u, 0x00000000u, 0x00070006u, 0x0000005bu, 0x00000002u, 0x435f6c67u, 0x4470696cu,
0x61747369u, 0x0065636eu, 0x00070006u, 0x0000005bu, 0x00000003u, 0x435f6c67u, 0x446c6c75u, 0x61747369u, 0x0065636eu,
0x00030005u, 0x0000005du, 0x00000000u, 0x00050005u, 0x00000066u, 0x43786574u, 0x64726f6fu, 0x00000000u, 0x00040047u,
0x00000016u, 0x0000000bu, 0x0000002au, 0x00030047u, 0x0000001eu, 0x00000002u, 0x00050048u, 0x0000001eu, 0x00000000u,
0x00000023u, 0x00000000u, 0x00050048u, 0x0000001eu, 0x00000001u, 0x00000023u, 0x00000010u, 0x00050048u, 0x0000001eu,
0x00000002u, 0x00000023u, 0x00000020u, 0x00030047u, 0x0000005bu, 0x00000002u, 0x00050048u, 0x0000005bu, 0x00000000u,
0x0000000bu, 0x00000000u, 0x00050048u, 0x0000005bu, 0x00000001u, 0x0000000bu, 0x00000001u, 0x00050048u, 0x0000005bu,
0x00000002u, 0x0000000bu, 0x00000003u, 0x00050048u, 0x0000005bu, 0x00000003u, 0x0000000bu, 0x00000004u, 0x00040047u,
0x00000066u, 0x0000001eu, 0x00000000u, 0x00020013u, 0x00000002u, 0x00030021u, 0x00000003u, 0x00000002u, 0x00030016u,
0x00000006u, 0x00000020u, 0x00040017u, 0x00000007u, 0x00000006u, 0x00000002u, 0x00040020u, 0x00000008u, 0x00000007u,
0x00000007u, 0x00040015u, 0x0000000au, 0x00000020u, 0x00000000u, 0x0004002bu, 0x0000000au, 0x0000000bu, 0x00000006u,
0x0004001cu, 0x0000000cu, 0x00000007u, 0x0000000bu, 0x0004002bu, 0x00000006u, 0x0000000du, 0x00000000u, 0x0005002cu,
0x00000007u, 0x0000000eu, 0x0000000du, 0x0000000du, 0x0004002bu, 0x00000006u, 0x0000000fu, 0x3f800000u, 0x0005002cu,
0x00000007u, 0x00000010u, 0x0000000fu, 0x0000000du, 0x0005002cu, 0x00000007u, 0x00000011u, 0x0000000du, 0x0000000fu,
0x0005002cu, 0x00000007u, 0x00000012u, 0x0000000fu, 0x0000000fu, 0x0009002cu, 0x0000000cu, 0x00000013u, 0x0000000eu,
0x00000010u, 0x00000011u, 0x00000011u, 0x00000010u, 0x00000012u, 0x00040015u, 0x00000014u, 0x00000020u, 0x00000001u,
0x00040020u, 0x00000015u, 0x00000001u, 0x00000014u, 0x0004003bu, 0x00000015u, 0x00000016u, 0x00000001u, 0x00040020u,
0x00000018u, 0x00000007u, 0x0000000cu, 0x00040017u, 0x0000001du, 0x00000006u, 0x00000004u, 0x0005001eu, 0x0000001eu,
0x0000001du, 0x0000001du, 0x00000014u, 0x00040020u, 0x0000001fu, 0x00000009u, 0x0000001eu, 0x0004003bu, 0x0000001fu,
0x00000020u, 0x00000009u, 0x0004002bu, 0x00000014u, 0x00000021u, 0x00000001u, 0x00040020u, 0x00000022u, 0x00000009u,
0x0000001du, 0x0004002bu, 0x00000006u, 0x0000002cu, 0x40000000u, 0x0004002bu, 0x0000000au, 0x00000030u, 0x00000001u,
0x00040020u, 0x00000031u, 0x00000007u, 0x00000006u, 0x0004002bu, 0x00000014u, 0x00000036u, 0x00000002u, 0x00040020u,
0x00000037u, 0x00000009u, 0x00000014u, 0x00020014u, 0x0000003au, 0x0004002bu, 0x0000000au, 0x00000041u, 0x00000000u,
0x0004002bu, 0x00000014u, 0x00000050u, 0x00000003u, 0x0004001cu, 0x0000005au, 0x00000006u, 0x00000030u, 0x0006001eu,
0x0000005bu, 0x0000001du, 0x00000006u, 0x0000005au, 0x0000005au, 0x00040020u, 0x0000005cu, 0x00000003u, 0x0000005bu,
0x0004003bu, 0x0000005cu, 0x0000005du, 0x00000003u, 0x0004002bu, 0x00000014u, 0x0000005eu, 0x00000000u, 0x00040020u,
0x00000063u, 0x00000003u, 0x0000001du, 0x00040020u, 0x00000065u, 0x00000003u, 0x00000007u, 0x0004003bu, 0x00000065u,
0x00000066u, 0x00000003u, 0x00050036u, 0x00000002u, 0x00000004u, 0x00000000u, 0x00000003u, 0x000200f8u, 0x00000005u,
0x0004003bu, 0x00000008u, 0x00000009u, 0x00000007u, 0x0004003bu, 0x00000018u, 0x00000019u, 0x00000007u, 0x0004003bu,
0x00000008u, 0x0000001cu, 0x00000007u, 0x0004003du, 0x00000014u, 0x00000017u, 0x00000016u, 0x0003003eu, 0x00000019u,
0x00000013u, 0x00050041u, 0x00000008u, 0x0000001au, 0x00000019u, 0x00000017u, 0x0004003du, 0x00000007u, 0x0000001bu,
0x0000001au, 0x0003003eu, 0x00000009u, 0x0000001bu, 0x00050041u, 0x00000022u, 0x00000023u, 0x00000020u, 0x00000021u,
0x0004003du, 0x0000001du, 0x00000024u, 0x00000023u, 0x0007004fu, 0x00000007u, 0x00000025u, 0x00000024u, 0x00000024u,
0x00000000u, 0x00000001u, 0x0004003du, 0x00000007u, 0x00000026u, 0x00000009u, 0x00050041u, 0x00000022u, 0x00000027u,
0x00000020u, 0x00000021u, 0x0004003du, 0x0000001du, 0x00000028u, 0x00000027u, 0x0007004fu, 0x00000007u, 0x00000029u,
0x00000028u, 0x00000028u, 0x00000002u, 0x00000003u, 0x00050085u, 0x00000007u, 0x0000002au, 0x00000026u, 0x00000029u,
0x00050081u, 0x00000007u, 0x0000002bu, 0x00000025u, 0x0000002au, 0x0005008eu, 0x00000007u, 0x0000002du, 0x0000002bu,
0x0000002cu, 0x00050050u, 0x00000007u, 0x0000002eu, 0x0000000fu, 0x0000000fu, 0x00050083u, 0x00000007u, 0x0000002fu,
0x0000002du, 0x0000002eu, 0x0003003eu, 0x0000001cu, 0x0000002fu, 0x00050041u, 0x00000031u, 0x00000032u, 0x0000001cu,
0x00000030u, 0x0004003du, 0x00000006u, 0x00000033u, 0x00000032u, 0x0004007fu, 0x00000006u, 0x00000034u, 0x00000033u,
0x00050041u, 0x00000031u, 0x00000035u, 0x0000001cu, 0x00000030u, 0x0003003eu, 0x00000035u, 0x00000034u, 0x00050041u,
0x00000037u, 0x00000038u, 0x00000020u, 0x00000036u, 0x0004003du, 0x00000014u, 0x00000039u, 0x00000038u, 0x000500aau,
0x0000003au, 0x0000003bu, 0x00000039u, 0x00000021u, 0x000300f7u, 0x0000003du, 0x00000000u, 0x000400fau, 0x0000003bu,
0x0000003cu, 0x00000045u, 0x000200f8u, 0x0000003cu, 0x00050041u, 0x00000031u, 0x0000003eu, 0x0000001cu, 0x00000030u,
0x0004003du, 0x00000006u, 0x0000003fu, 0x0000003eu, 0x0004007fu, 0x00000006u, 0x00000040u, 0x0000003fu, 0x00050041u,
0x00000031u, 0x00000042u, 0x0000001cu, 0x00000041u, 0x0004003du, 0x00000006u, 0x00000043u, 0x00000042u, 0x00050050u,
0x00000007u, 0x00000044u, 0x00000040u, 0x00000043u, 0x0003003eu, 0x0000001cu, 0x00000044u, 0x000200f9u, 0x0000003du,
0x000200f8u, 0x00000045u, 0x00050041u, 0x00000037u, 0x00000046u, 0x00000020u, 0x00000036u, 0x0004003du, 0x00000014u,
0x00000047u, 0x00000046u, 0x000500aau, 0x0000003au, 0x00000048u, 0x00000047u, 0x00000036u, 0x000300f7u, 0x0000004au,
0x00000000u, 0x000400fau, 0x00000048u, 0x00000049u, 0x0000004du, 0x000200f8u, 0x00000049u, 0x0004003du, 0x00000007u,
0x0000004bu, 0x0000001cu, 0x0004007fu, 0x00000007u, 0x0000004cu, 0x0000004bu, 0x0003003eu, 0x0000001cu, 0x0000004cu,
0x000200f9u, 0x0000004au, 0x000200f8u, 0x0000004du, 0x00050041u, 0x00000037u, 0x0000004eu, 0x00000020u, 0x00000036u,
0x0004003du, 0x00000014u, 0x0000004fu, 0x0000004eu, 0x000500aau, 0x0000003au, 0x00000051u, 0x0000004fu, 0x00000050u,
0x000300f7u, 0x00000053u, 0x00000000u, 0x000400fau, 0x00000051u, 0x00000052u, 0x00000053u, 0x000200f8u, 0x00000052u,
0x00050041u, 0x00000031u, 0x00000054u, 0x0000001cu, 0x00000030u, 0x0004003du, 0x00000006u, 0x00000055u, 0x00000054u,
0x00050041u, 0x00000031u, 0x00000056u, 0x0000001cu, 0x00000041u, 0x0004003du, 0x00000006u, 0x00000057u, 0x00000056u,
0x0004007fu, 0x00000006u, 0x00000058u, 0x00000057u, 0x00050050u, 0x00000007u, 0x00000059u, 0x00000055u, 0x00000058u,
0x0003003eu, 0x0000001cu, 0x00000059u, 0x000200f9u, 0x00000053u, 0x000200f8u, 0x00000053u, 0x000200f9u, 0x0000004au,
0x000200f8u, 0x0000004au, 0x000200f9u, 0x0000003du, 0x000200f8u, 0x0000003du, 0x0004003du, 0x00000007u, 0x0000005fu,
0x0000001cu, 0x00050051u, 0x00000006u, 0x00000060u, 0x0000005fu, 0x00000000u, 0x00050051u, 0x00000006u, 0x00000061u,
0x0000005fu, 0x00000001u, 0x00070050u, 0x0000001du, 0x00000062u, 0x00000060u, 0x00000061u, 0x0000000du, 0x0000000fu,
0x00050041u, 0x00000063u, 0x00000064u, 0x0000005du, 0x0000005eu, 0x0003003eu, 0x00000064u, 0x00000062u, 0x00050041u,
0x00000022u, 0x00000067u, 0x00000020u, 0x0000005eu, 0x0004003du, 0x0000001du, 0x00000068u, 0x00000067u, 0x0007004fu,
0x00000007u, 0x00000069u, 0x00000068u, 0x00000068u, 0x00000000u, 0x00000001u, 0x0004003du, 0x00000007u, 0x0000006au,
0x00000009u, 0x00050041u, 0x00000022u, 0x0000006bu, 0x00000020u, 0x0000005eu, 0x0004003du, 0x0000001du, 0x0000006cu,
0x0000006bu, 0x0007004fu, 0x00000007u, 0x0000006du, 0x0000006cu, 0x0000006cu, 0x00000002u, 0x00000003u, 0x00050085u,
0x00000007u, 0x0000006eu, 0x0000006au, 0x0000006du, 0x00050081u, 0x00000007u, 0x0000006fu, 0x00000069u, 0x0000006eu,
0x0003003eu, 0x00000066u, 0x0000006fu, 0x000100fdu, 0x00010038u,
};
static constexpr Uint32 kWireColorBlitFragmentSpirv[] = {
0x07230203u, 0x00010000u, 0x0008000bu, 0x00000027u, 0x00000000u, 0x00020011u, 0x00000001u, 0x0006000bu,
0x00000001u, 0x4c534c47u, 0x6474732eu, 0x3035342eu, 0x00000000u, 0x0003000eu, 0x00000000u, 0x00000001u,
0x0007000fu, 0x00000004u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x0000000au, 0x0000001fu, 0x00030010u,
0x00000004u, 0x00000007u, 0x00030003u, 0x00000002u, 0x000001c2u, 0x00040005u, 0x00000004u, 0x6e69616du,
0x00000000u, 0x00050005u, 0x0000000au, 0x43786574u, 0x64726f6fu, 0x00000000u, 0x00050005u, 0x0000001fu,
0x4374756fu, 0x726f6c6fu, 0x00000000u, 0x00050005u, 0x00000023u, 0x72756f73u, 0x6d496563u, 0x00656761u,
0x00040047u, 0x0000000au, 0x0000001eu, 0x00000000u, 0x00040047u, 0x0000001fu, 0x0000001eu, 0x00000000u,
0x00040047u, 0x00000023u, 0x00000021u, 0x00000000u, 0x00040047u, 0x00000023u, 0x00000022u, 0x00000000u,
0x00020013u, 0x00000002u, 0x00030021u, 0x00000003u, 0x00000002u, 0x00020014u, 0x00000006u, 0x00030016u,
0x00000007u, 0x00000020u, 0x00040017u, 0x00000008u, 0x00000007u, 0x00000002u, 0x00040020u, 0x00000009u,
0x00000001u, 0x00000008u, 0x0004003bu, 0x00000009u, 0x0000000au, 0x00000001u, 0x0004002bu, 0x00000007u,
0x0000000cu, 0x00000000u, 0x0005002cu, 0x00000008u, 0x0000000du, 0x0000000cu, 0x0000000cu, 0x00040017u,
0x0000000eu, 0x00000006u, 0x00000002u, 0x0004002bu, 0x00000007u, 0x00000015u, 0x3f800000u, 0x0005002cu,
0x00000008u, 0x00000016u, 0x00000015u, 0x00000015u, 0x00040017u, 0x0000001du, 0x00000007u, 0x00000004u,
0x00040020u, 0x0000001eu, 0x00000003u, 0x0000001du, 0x0004003bu, 0x0000001eu, 0x0000001fu, 0x00000003u,
0x00090019u, 0x00000020u, 0x00000007u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000001u,
0x00000000u, 0x0003001bu, 0x00000021u, 0x00000020u, 0x00040020u, 0x00000022u, 0x00000000u, 0x00000021u,
0x0004003bu, 0x00000022u, 0x00000023u, 0x00000000u, 0x00050036u, 0x00000002u, 0x00000004u, 0x00000000u,
0x00000003u, 0x000200f8u, 0x00000005u, 0x0004003du, 0x00000008u, 0x0000000bu, 0x0000000au, 0x000500b8u,
0x0000000eu, 0x0000000fu, 0x0000000bu, 0x0000000du, 0x0004009au, 0x00000006u, 0x00000010u, 0x0000000fu,
0x000400a8u, 0x00000006u, 0x00000011u, 0x00000010u, 0x000300f7u, 0x00000013u, 0x00000000u, 0x000400fau,
0x00000011u, 0x00000012u, 0x00000013u, 0x000200f8u, 0x00000012u, 0x0004003du, 0x00000008u, 0x00000014u,
0x0000000au, 0x000500bau, 0x0000000eu, 0x00000017u, 0x00000014u, 0x00000016u, 0x0004009au, 0x00000006u,
0x00000018u, 0x00000017u, 0x000200f9u, 0x00000013u, 0x000200f8u, 0x00000013u, 0x000700f5u, 0x00000006u,
0x00000019u, 0x00000010u, 0x00000005u, 0x00000018u, 0x00000012u, 0x000300f7u, 0x0000001bu, 0x00000000u,
0x000400fau, 0x00000019u, 0x0000001au, 0x0000001bu, 0x000200f8u, 0x0000001au, 0x000100fcu, 0x000200f8u,
0x0000001bu, 0x0004003du, 0x00000021u, 0x00000024u, 0x00000023u, 0x0004003du, 0x00000008u, 0x00000025u,
0x0000000au, 0x00070058u, 0x0000001du, 0x00000026u, 0x00000024u, 0x00000025u, 0x00000002u, 0x0000000cu,
0x0003003eu, 0x0000001fu, 0x00000026u, 0x000100fdu, 0x00010038u,
0x07230203u, 0x00010000u, 0x0008000bu, 0x00000027u, 0x00000000u, 0x00020011u, 0x00000001u, 0x0006000bu, 0x00000001u,
0x4c534c47u, 0x6474732eu, 0x3035342eu, 0x00000000u, 0x0003000eu, 0x00000000u, 0x00000001u, 0x0007000fu, 0x00000004u,
0x00000004u, 0x6e69616du, 0x00000000u, 0x0000000au, 0x0000001fu, 0x00030010u, 0x00000004u, 0x00000007u, 0x00030003u,
0x00000002u, 0x000001c2u, 0x00040005u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x00050005u, 0x0000000au, 0x43786574u,
0x64726f6fu, 0x00000000u, 0x00050005u, 0x0000001fu, 0x4374756fu, 0x726f6c6fu, 0x00000000u, 0x00050005u, 0x00000023u,
0x72756f73u, 0x6d496563u, 0x00656761u, 0x00040047u, 0x0000000au, 0x0000001eu, 0x00000000u, 0x00040047u, 0x0000001fu,
0x0000001eu, 0x00000000u, 0x00040047u, 0x00000023u, 0x00000021u, 0x00000000u, 0x00040047u, 0x00000023u, 0x00000022u,
0x00000000u, 0x00020013u, 0x00000002u, 0x00030021u, 0x00000003u, 0x00000002u, 0x00020014u, 0x00000006u, 0x00030016u,
0x00000007u, 0x00000020u, 0x00040017u, 0x00000008u, 0x00000007u, 0x00000002u, 0x00040020u, 0x00000009u, 0x00000001u,
0x00000008u, 0x0004003bu, 0x00000009u, 0x0000000au, 0x00000001u, 0x0004002bu, 0x00000007u, 0x0000000cu, 0x00000000u,
0x0005002cu, 0x00000008u, 0x0000000du, 0x0000000cu, 0x0000000cu, 0x00040017u, 0x0000000eu, 0x00000006u, 0x00000002u,
0x0004002bu, 0x00000007u, 0x00000015u, 0x3f800000u, 0x0005002cu, 0x00000008u, 0x00000016u, 0x00000015u, 0x00000015u,
0x00040017u, 0x0000001du, 0x00000007u, 0x00000004u, 0x00040020u, 0x0000001eu, 0x00000003u, 0x0000001du, 0x0004003bu,
0x0000001eu, 0x0000001fu, 0x00000003u, 0x00090019u, 0x00000020u, 0x00000007u, 0x00000001u, 0x00000000u, 0x00000000u,
0x00000000u, 0x00000001u, 0x00000000u, 0x0003001bu, 0x00000021u, 0x00000020u, 0x00040020u, 0x00000022u, 0x00000000u,
0x00000021u, 0x0004003bu, 0x00000022u, 0x00000023u, 0x00000000u, 0x00050036u, 0x00000002u, 0x00000004u, 0x00000000u,
0x00000003u, 0x000200f8u, 0x00000005u, 0x0004003du, 0x00000008u, 0x0000000bu, 0x0000000au, 0x000500b8u, 0x0000000eu,
0x0000000fu, 0x0000000bu, 0x0000000du, 0x0004009au, 0x00000006u, 0x00000010u, 0x0000000fu, 0x000400a8u, 0x00000006u,
0x00000011u, 0x00000010u, 0x000300f7u, 0x00000013u, 0x00000000u, 0x000400fau, 0x00000011u, 0x00000012u, 0x00000013u,
0x000200f8u, 0x00000012u, 0x0004003du, 0x00000008u, 0x00000014u, 0x0000000au, 0x000500bau, 0x0000000eu, 0x00000017u,
0x00000014u, 0x00000016u, 0x0004009au, 0x00000006u, 0x00000018u, 0x00000017u, 0x000200f9u, 0x00000013u, 0x000200f8u,
0x00000013u, 0x000700f5u, 0x00000006u, 0x00000019u, 0x00000010u, 0x00000005u, 0x00000018u, 0x00000012u, 0x000300f7u,
0x0000001bu, 0x00000000u, 0x000400fau, 0x00000019u, 0x0000001au, 0x0000001bu, 0x000200f8u, 0x0000001au, 0x000100fcu,
0x000200f8u, 0x0000001bu, 0x0004003du, 0x00000021u, 0x00000024u, 0x00000023u, 0x0004003du, 0x00000008u, 0x00000025u,
0x0000000au, 0x00070058u, 0x0000001du, 0x00000026u, 0x00000024u, 0x00000025u, 0x00000002u, 0x0000000cu, 0x0003003eu,
0x0000001fu, 0x00000026u, 0x000100fdu, 0x00010038u,
};

Some files were not shown because too many files have changed in this diff Show More