[Feat] (Metrics): land the MGPipe boundary counters - bytes, dynamic accessor calls and the six memo gates, off unless MOBILEGL_PIPE_STATS is set

- Plan B section 11 P0 asks for TracyPlot per-frame counters on both sides of the
  boundary, and section 2.3.1 adds the deliverable v1 did not have: DYNAMIC call
  counters. The static call-site counts everyone quotes (Espryt 124 / Magma 169) are
  not the per-draw cost, because every one of those paths is memo-gated; without a
  dynamic counter P2's verdict stays a guess. The tree had no per-frame byte or call
  measurement at all - MG_Util/Metrics is format arithmetic, and Tracy has zones but
  no plots.
- Cost when off: g_pipeStatsEnabled is a plain global Bool latched once in
  Initialize() right after MG_ConfigLoader::Init(), and every counting site is
  `if (PipeStats::Enabled()) ...` - one load of a hot global and one never-taken
  branch. The counters are relaxed atomics because buffer and texture staging are
  reachable from more than one thread; the off path never touches them.
- Eight byte classes (stage-buffer, stage-texture, stage-ubo-global, stage-ubo-named,
  stage-vertex-client, stage-index-client, persistent-map-push and the
  residual-value-block placeholder of section 6.3), six call classes and the six memo
  gates of section 2.3.1, each as a hit/miss pair. Names are minted now, including the
  two that stay 0 in P0, so no recorded baseline is invalidated by a later rename -
  which is why the name set is pinned by a test.
- Reporting: TracyPlot per counter per frame when TRACY_ENABLE; with
  MOBILEGL_PIPE_STATS=1 one MGLOG_I summary line every 120 frames and at teardown, plus
  a JSON dump to MOBILEGL_PIPE_STATS_FILE when that is set. MGLOG_I against the usual
  "MGLOG_D for non-critical" rule on purpose: the line has to survive an INFO build -
  the only build a device runs - it is at most one line per 120 frames, and it exists
  only when the operator asked for it.
- Summary lines report disjoint WINDOWS, not run totals: a run total over a workload
  that changes shape (load, then steady state) averages away the very number section
  2.3.1 wants an absolute value for.
- MOBILEGL_PIPE_STATS_FILE joins the six switches parsed in e48a3582; like them it
  needs no allow-list entry because InitializeAcceptedEnvVariables accepts every
  MOBILEGL_-prefixed variable by construction.
This commit is contained in:
2026-09-05 20:16:49 -04:00
parent 9bbf71990c
commit 42e0f47ebb
8 changed files with 823 additions and 0 deletions
+2
View File
@@ -238,6 +238,8 @@ set(SOURCE_FILES
MobileGL/MG_Util/Metrics/BufferMetrics.cpp
MobileGL/MG_Util/Metrics/PipeStats.cpp
MobileGL/MG_Util/Converters/GLToStr/GLEnumConverter.cpp
MobileGL/MG_Util/Converters/EGLToStr/EGLEnumConverter.cpp
MobileGL/MG_Util/Converters/MGToStr/DataTypeConverter.cpp
+4
View File
@@ -347,6 +347,10 @@ namespace MobileGL::MG_Config {
// the server without shipping index bytes per draw. Over budget it degrades to
// per-draw staging, counted separately in the stats.
Uint32 PipeIndexMirrorMb = 64;
// MOBILEGL_PIPE_STATS_FILE: where the boundary counters' teardown JSON dump goes.
// Empty (the default) means no dump; the per-120-frame summary line still goes to
// the log whenever PipeStats is on, so a device run needs no writable path.
String PipeStatsFile;
};
extern FeaturesTable Features;
} // namespace MobileGL::MG_Config
+1
View File
@@ -242,6 +242,7 @@ namespace MobileGL::MG_ConfigLoader {
QueryEnvQuirkOverride("MOBILEGL_PIPE_LEGACY_MEMOS") != MG_Config::QuirkOverride::ForceOff;
features.PipeTexelRetainMb = QueryEnvUint32("MOBILEGL_PIPE_TEXEL_RETAIN_MB", 0, 0, 4096);
features.PipeIndexMirrorMb = QueryEnvUint32("MOBILEGL_PIPE_INDEX_MIRROR_MB", 64, 0, 4096);
QueryEnvVariable("MOBILEGL_PIPE_STATS_FILE", features.PipeStatsFile, "");
}
inline void InitBackendType() {
+10
View File
@@ -17,6 +17,7 @@
#include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <MG_Impl/GLImpl/Query/GL_Query.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Util/Metrics/PipeStats.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_State/GLState/ProgramState/ProgramTranslationCache.h>
#include <MG_Util/ShaderTranspiler/TranslationCache.h>
@@ -42,6 +43,11 @@ namespace MobileGL {
if (logLifecycle) {
MGLOG_I("MobileGL closing...");
}
// Before any subsystem the counters name goes away, and before the last frame's
// numbers can be lost: emits the final summary line and, when
// MOBILEGL_PIPE_STATS_FILE is set, the JSON dump. A no-op when the counters are
// off, and idempotent.
MG_Util::PipeStats::Shutdown();
// First, before anything else is torn down. In-flight compile/link jobs own
// their own inputs and are safe against everything below EXCEPT glslang's
// process globals and the TShader/TProgram objects hanging off pGLContext,
@@ -102,6 +108,10 @@ namespace MobileGL {
MGLOG_I("Initializing MobileGL...");
MG_ConfigLoader::Init();
MGLOG_I("Config loaded");
// Immediately after the config load and before anything can count: the MGPipe
// boundary counters latch their enable flag here, so every counting site in the
// two backends is a load of an already-settled global for the rest of the run.
MG_Util::PipeStats::Init();
MG_State::Init();
MGLOG_D("MG_State initialized");
MG_Backend::Init();
+19
View File
@@ -35,6 +35,25 @@ target_link_libraries(
${LINK_LIBRARIES}
)
# The MGPipe boundary counters: the enable latch, the byte/call/gate arithmetic, the
# payload histogram's bucketing and the two report formats. No GL context, no driver.
add_executable(
PipeStatsTest
PipeStatsTest.cpp
)
target_include_directories(PipeStatsTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
PipeStatsTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
include(GoogleTest)
gtest_discover_tests(JobNodeTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
gtest_discover_tests(LogLevelTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
gtest_discover_tests(PipeStatsTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
+213
View File
@@ -0,0 +1,213 @@
// MobileGL - MobileGL/MG_Test/Util/PipeStatsTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// The MGPipe boundary counters (plan B section 11 P0, corollary in section 2.3.1).
// No GL context and no driver: the module is arithmetic over a fixed set of counters,
// which is exactly what has to be pinned before anyone reads a number off a device.
#include <gtest/gtest.h>
#include <MG_Util/Metrics/PipeStats.h>
#include <cstdio>
#include <fstream>
#include <sstream>
namespace {
namespace PS = MobileGL::MG_Util::PipeStats;
using MobileGL::String;
using MobileGL::Uint32;
using MobileGL::Uint64;
class PipeStatsTest : public ::testing::Test {
protected:
void SetUp() override {
PS::ResetForTesting();
PS::SetEnabledForTesting(true);
}
void TearDown() override {
PS::SetEnabledForTesting(false);
PS::ResetForTesting();
}
};
// The off latch is the whole cost argument: every counting site in the two backends is
// written as `if (Enabled()) ...`, so a false latch has to mean "nothing is counted".
TEST_F(PipeStatsTest, EnabledLatchIsTheOnlyGate) {
PS::SetEnabledForTesting(false);
EXPECT_FALSE(PS::Enabled());
PS::SetEnabledForTesting(true);
EXPECT_TRUE(PS::Enabled());
}
TEST_F(PipeStatsTest, ByteClassesAccumulateIndependently) {
PS::AddBytes(PS::ByteClass::StageBuffer, 100);
PS::AddBytes(PS::ByteClass::StageBuffer, 40);
PS::AddBytes(PS::ByteClass::StageTexture, 7);
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::StageBuffer), 140u);
EXPECT_EQ(PS::FrameBytes(PS::ByteClass::StageBuffer), 140u);
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::StageTexture), 7u);
// Every other class untouched, the residual-value-block placeholder included.
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::StageUboGlobal), 0u);
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::StageUboNamed), 0u);
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::ResidualValueBlock), 0u);
}
// The frame accumulator is what feeds TracyPlot; the run total is what feeds the JSON
// dump. A present must clear the first and keep the second.
TEST_F(PipeStatsTest, PresentClearsTheFrameButKeepsTheTotal) {
PS::AddBytes(PS::ByteClass::StageTexture, 512);
PS::AddCalls(PS::CallClass::Draws, 3);
PS::CountGate(PS::Gate::EsprytRenderState, /*hit=*/true);
PS::OnPresent();
EXPECT_EQ(PS::FrameBytes(PS::ByteClass::StageTexture), 0u);
EXPECT_EQ(PS::FrameCalls(PS::CallClass::Draws), 0u);
EXPECT_EQ(PS::TotalBytes(PS::ByteClass::StageTexture), 512u);
EXPECT_EQ(PS::TotalCalls(PS::CallClass::Draws), 3u);
EXPECT_EQ(PS::TotalGateHits(PS::Gate::EsprytRenderState), 1u);
EXPECT_EQ(PS::FrameCount(), 1u);
}
TEST_F(PipeStatsTest, GateHitsAndMissesAreSeparateCounters) {
for (Uint32 i = 0; i < 5; ++i) {
PS::CountGate(PS::Gate::MagmaPipelineMemo, /*hit=*/true);
}
PS::CountGate(PS::Gate::MagmaPipelineMemo, /*hit=*/false);
PS::CountGate(PS::Gate::MagmaDrawFastPath, /*hit=*/false);
EXPECT_EQ(PS::TotalGateHits(PS::Gate::MagmaPipelineMemo), 5u);
EXPECT_EQ(PS::TotalGateMisses(PS::Gate::MagmaPipelineMemo), 1u);
EXPECT_EQ(PS::TotalGateHits(PS::Gate::MagmaDrawFastPath), 0u);
EXPECT_EQ(PS::TotalGateMisses(PS::Gate::MagmaDrawFastPath), 1u);
}
// Bucket 0 is "no payload"; bucket n>0 is [2^(n-1), 2^n). The placeholder histogram is
// the SEG_CMD sizing input (section 4.5.7), so its bucketing is pinned now rather than
// when a generator first calls it.
TEST_F(PipeStatsTest, PayloadHistogramBucketsByPowerOfTwo) {
PS::RecordDrawPayloadBytes(0);
PS::RecordDrawPayloadBytes(1); // [1, 2) -> bucket 1
PS::RecordDrawPayloadBytes(2); // [2, 4) -> bucket 2
PS::RecordDrawPayloadBytes(3); // [2, 4) -> bucket 2
PS::RecordDrawPayloadBytes(48); // [32, 64) -> bucket 6
PS::RecordDrawPayloadBytes(64); // [64, 128)-> bucket 7
EXPECT_EQ(PS::TotalPayloadBucket(0), 1u);
EXPECT_EQ(PS::TotalPayloadBucket(1), 1u);
EXPECT_EQ(PS::TotalPayloadBucket(2), 2u);
EXPECT_EQ(PS::TotalPayloadBucket(6), 1u);
EXPECT_EQ(PS::TotalPayloadBucket(7), 1u);
}
// A record far larger than the last bucket must land in the last bucket, not past the
// end of the array.
TEST_F(PipeStatsTest, PayloadHistogramSaturatesInsteadOfOverflowing) {
PS::RecordDrawPayloadBytes(~Uint64{0});
EXPECT_EQ(PS::TotalPayloadBucket(PS::kPayloadHistogramBuckets - 1), 1u);
EXPECT_EQ(PS::TotalPayloadBucket(PS::kPayloadHistogramBuckets), 0u);
}
// The summary line's shape is what an operator greps and what the smoke check in this
// package matches, so it is pinned here rather than left to the log reader's memory.
TEST_F(PipeStatsTest, SummaryLineCarriesEveryClassAndGate) {
PS::AddCalls(PS::CallClass::Draws, 4);
PS::AddCalls(PS::CallClass::AccessorCalls, 50);
PS::AddBytes(PS::ByteClass::StageBuffer, 4096);
PS::OnPresent();
const String line = PS::FormatSummaryLine();
EXPECT_NE(line.find("MGPipe stats:"), String::npos) << line;
EXPECT_NE(line.find("draws=4"), String::npos) << line;
// 50 accessor calls over 4 draws, two decimals, no <iomanip>.
EXPECT_NE(line.find("acc/draw=12.50"), String::npos) << line;
EXPECT_NE(line.find("buf=4096"), String::npos) << line;
for (Uint32 i = 0; i < static_cast<Uint32>(PS::Gate::Count); ++i) {
EXPECT_NE(line.find("="), String::npos);
}
EXPECT_NE(line.find("gates["), String::npos) << line;
EXPECT_NE(line.find("tex[emit="), String::npos) << line;
}
// Successive summaries report WINDOWS, not run totals: a run total over a workload that
// changes shape (load, then steady state) averages away the very number section 2.3.1
// wants.
TEST_F(PipeStatsTest, SummaryLinesReportDisjointWindows) {
PS::AddCalls(PS::CallClass::Draws, 10);
PS::OnPresent();
const String first = PS::FormatSummaryLine();
EXPECT_NE(first.find("draws=10"), String::npos) << first;
PS::AddCalls(PS::CallClass::Draws, 3);
PS::OnPresent();
const String second = PS::FormatSummaryLine();
EXPECT_NE(second.find("draws=3"), String::npos) << second;
EXPECT_NE(second.find("frames=2"), String::npos) << second;
}
TEST_F(PipeStatsTest, SummaryLineSurvivesZeroDraws) {
PS::OnPresent();
const String line = PS::FormatSummaryLine();
EXPECT_NE(line.find("acc/draw=0.00"), String::npos) << line;
}
TEST_F(PipeStatsTest, JsonDumpNamesEveryCounter) {
PS::AddBytes(PS::ByteClass::StageUboNamed, 256);
PS::CountGate(PS::Gate::MagmaDynamicTail, /*hit=*/false);
PS::RecordDrawPayloadBytes(9);
PS::OnPresent();
const String json = PS::FormatJson();
for (Uint32 i = 0; i < static_cast<Uint32>(PS::ByteClass::Count); ++i) {
const String name = PS::NameOf(static_cast<PS::ByteClass>(i));
EXPECT_NE(json.find("\"" + name + "\""), String::npos) << name << " missing from " << json;
}
for (Uint32 i = 0; i < static_cast<Uint32>(PS::CallClass::Count); ++i) {
const String name = PS::NameOf(static_cast<PS::CallClass>(i));
EXPECT_NE(json.find("\"" + name + "\""), String::npos) << name << " missing from " << json;
}
for (Uint32 i = 0; i < static_cast<Uint32>(PS::Gate::Count); ++i) {
const String name = PS::NameOf(static_cast<PS::Gate>(i));
EXPECT_NE(json.find("\"" + name + "\""), String::npos) << name << " missing from " << json;
}
EXPECT_NE(json.find("\"stage-ubo-named\": 256"), String::npos) << json;
EXPECT_NE(json.find("\"frames\": 1"), String::npos) << json;
EXPECT_NE(json.find("cmd-bytes-per-draw-histogram"), String::npos) << json;
}
// The counter names are the TracyPlot series names and the JSON keys; a rename is a
// breaking change for every recorded baseline, so the whole set is pinned.
TEST_F(PipeStatsTest, CounterNamesAreStable) {
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageBuffer), "stage-buffer");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageTexture), "stage-texture");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageUboGlobal), "stage-ubo-global");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageUboNamed), "stage-ubo-named");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageVertexClient), "stage-vertex-client");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::StageIndexClient), "stage-index-client");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::PersistentMapPush), "persistent-map-push");
EXPECT_STREQ(PS::NameOf(PS::ByteClass::ResidualValueBlock), "residual-value-block");
EXPECT_STREQ(PS::NameOf(PS::Gate::EsprytRenderState), "espryt-render-state");
EXPECT_STREQ(PS::NameOf(PS::Gate::EsprytTextureSyncList), "espryt-texture-sync-list");
EXPECT_STREQ(PS::NameOf(PS::Gate::EsprytUnitBindingsEpoch), "espryt-unit-bindings-epoch");
EXPECT_STREQ(PS::NameOf(PS::Gate::MagmaDrawFastPath), "magma-draw-fastpath");
EXPECT_STREQ(PS::NameOf(PS::Gate::MagmaPipelineMemo), "magma-pipeline-memo");
EXPECT_STREQ(PS::NameOf(PS::Gate::MagmaDynamicTail), "magma-dynamic-tail");
}
// A summary is emitted every kSummaryFramePeriod presents. The period is a constant the
// smoke check depends on, so a change to it has to break a test.
TEST_F(PipeStatsTest, SummaryPeriodIsOneHundredAndTwentyFrames) {
EXPECT_EQ(PS::kSummaryFramePeriod, 120u);
for (Uint64 i = 0; i < PS::kSummaryFramePeriod; ++i) {
PS::OnPresent();
}
EXPECT_EQ(PS::FrameCount(), PS::kSummaryFramePeriod);
}
} // namespace
+403
View File
@@ -0,0 +1,403 @@
// MobileGL - MobileGL/MG_Util/Metrics/PipeStats.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "PipeStats.h"
#include <Config.h>
#include <fstream>
// ---------------------------------------------------------------------------------------
// SITE INVENTORY - what these counters DO and DO NOT cover.
//
// Byte classes
// stage-buffer DirectGLES Managers.cpp: RespecifyStorageNow's glBufferData,
// FlushPendingRangesNow's three shapes (map-write, glBufferSubData,
// upload-ring stage). Covers every byte Espryt hands the driver for
// a buffer object's contents.
// NOT covered: DirectVulkan's own buffer staging (its buffer bytes
// reach the GPU through a persistent map the frontend already owns,
// so there is no second copy to count) - see the note on
// persistent-map-push.
// stage-texture DirectGLES Managers.cpp texture upload: the bytes of whichever of
// the three upload shapes ran (rect list / union box / whole level).
// NOT covered: the DirectVulkan texture staging path, and Espryt's
// compressed-texture and readback paths.
// stage-ubo-global DirectGLES.cpp default-uniform-block image, both the UBO-ring
// memcpy and the glBufferSubData fallback.
// stage-ubo-named DirectVulkan UniformManager::ResolveUniformBufferPayload - the
// bytes Magma repacks into its own UBO ring. Espryt contributes
// nothing by construction (D-B8).
// stage-vertex-client DirectGLES BackendVertexArrayObject::SyncClientSideAttributesFor-
// DrawArrays, both the Float64-narrowing and the verbatim shapes.
// NOT covered: the DirectVulkan converted-vertex-stream cache.
// stage-index-client DirectGLES index rewriting (the primitive-restart substitution
// buffer).
// NOT covered: DirectVulkan's index staging.
// persistent-map-push Not wired in P0: today a persistent map is a permanent address
// space donation (D4/D-B4) that survives the whole monolith track,
// so there is no push to count until the IPC track breaks it.
// residual-value-block Placeholder, always 0 until P2 (plan section 6.3).
//
// Call classes
// draws DirectGLES PrepareForDraw and DirectVulkan TrySetupDrawFastPath's
// caller-visible entry. A dispatch is not a draw and is not counted.
// accessor-calls STATIC TALLIES at the instrumented entry points, NOT a wrapper
// around all 293 pGLContext-> sites. Each instrumented function adds
// the number of GLContext accessor calls that its OWN body executed
// on the path taken. Covered: PrepareForDraw's own reads,
// SyncRenderState, CaptureDrawTextureSyncKeys/CurrentUnitBindings-
// Epoch, SyncNeccessaryTextures' walk, TrySetupDrawFastPath,
// GetOrCreatePipeline and ApplyDynamicDrawStateTail. NOT covered:
// the reads inside the callees those functions invoke (buffer/VAO/
// FBO/program sync, the pipeline payload builder's ~40 reads on a
// memo miss), and every non-draw entry point. The number is
// therefore a LOWER BOUND on the per-draw accessor count, and it is
// the bound over exactly the six gates section 2.3.1 tabulates.
// texture-* DirectGLES texture upload, per (target, level) emission.
//
// Gates: the six of section 2.3.1, each counted exactly once per probe.
//
// READING acc/draw. The accessor tally covers the instrumented functions wherever they
// run, and three of them (SyncRenderState, the texture-key capture, SyncNeccessaryTextures)
// are also reached from NON-draw call sites - Clear, readbacks, the DSA by-name entry
// points - which the `draws` counter deliberately does not count. So acc/draw is the
// per-draw steady-state number section 2.3.1 asks for only in a DRAW-DOMINATED window; in a
// window dominated by clears and readbacks it is inflated by exactly those non-draw
// probes, and the gate hit/miss pairs are the honest reading there.
// ---------------------------------------------------------------------------------------
namespace MobileGL::MG_Util::PipeStats {
Bool g_pipeStatsEnabled = false;
namespace {
constexpr Uint32 kByteClassCount = static_cast<Uint32>(ByteClass::Count);
constexpr Uint32 kCallClassCount = static_cast<Uint32>(CallClass::Count);
constexpr Uint32 kGateCount = static_cast<Uint32>(Gate::Count);
using Counter = std::atomic<Uint64>;
Counter g_frameBytes[kByteClassCount];
Counter g_totalBytes[kByteClassCount];
Counter g_frameCalls[kCallClassCount];
Counter g_totalCalls[kCallClassCount];
Counter g_frameGateHit[kGateCount];
Counter g_totalGateHit[kGateCount];
Counter g_frameGateMiss[kGateCount];
Counter g_totalGateMiss[kGateCount];
Counter g_totalPayloadBuckets[kPayloadHistogramBuckets];
Counter g_frameCount{0};
// Window bases: the run totals as of the previous summary line. Only ever touched
// from OnPresent()/Shutdown() (the present thread), so plain integers.
Uint64 g_windowBaseBytes[kByteClassCount] = {};
Uint64 g_windowBaseCalls[kCallClassCount] = {};
Uint64 g_windowBaseGateHit[kGateCount] = {};
Uint64 g_windowBaseGateMiss[kGateCount] = {};
Uint64 g_windowBaseFrames = 0;
Bool g_shutdownDone = false;
inline void Bump(Counter& counter, Uint64 amount) {
counter.fetch_add(amount, std::memory_order_relaxed);
}
inline Uint64 Read(const Counter& counter) { return counter.load(std::memory_order_relaxed); }
// Bucket 0 is "0 bytes", bucket n>0 holds [2^(n-1), 2^n). Saturates at the last
// bucket so a pathological record cannot index out of the array.
Uint32 PayloadBucketOf(Uint64 bytes) {
if (bytes == 0) {
return 0;
}
Uint32 bucket = 1;
while (bucket + 1 < kPayloadHistogramBuckets && bytes >= (Uint64{1} << bucket)) {
++bucket;
}
return bucket;
}
const char* const kByteClassNames[kByteClassCount] = {
"stage-buffer", "stage-texture", "stage-ubo-global", "stage-ubo-named",
"stage-vertex-client", "stage-index-client", "persistent-map-push", "residual-value-block",
};
const char* const kCallClassNames[kCallClassCount] = {
"draws", "accessor-calls", "tex-upload-emissions", "tex-upload-box", "tex-upload-rect",
"tex-upload-jobs",
};
const char* const kGateNames[kGateCount] = {
"espryt-render-state", "espryt-texture-sync-list", "espryt-unit-bindings-epoch",
"magma-draw-fastpath", "magma-pipeline-memo", "magma-dynamic-tail",
};
// Short forms, so the per-120-frame line stays one terminal line wide.
const char* const kByteClassShort[kByteClassCount] = {"buf", "tex", "ubog", "ubon",
"vtxc", "idxc", "pmap", "resid"};
const char* const kGateShort[kGateCount] = {"ers", "etl", "eub", "mfp", "mpm", "mdt"};
void EmitSummaryLine() {
const String line = FormatSummaryLine();
// MGLOG_I on purpose, against the project's usual "MGLOG_D for anything
// non-critical" rule: the line has to survive an INFO build (that is the only
// build a device ever runs), it is emitted at most once per 120 frames, and it
// exists at all only when the operator set MOBILEGL_PIPE_STATS=1. It is an
// opt-in measurement channel, not per-frame noise.
MGLOG_I("%s", line.c_str());
}
void WriteJsonDump() {
const String& path = MG_Config::Features.PipeStatsFile;
if (path.empty()) {
return;
}
std::ofstream out(path, std::ios::out | std::ios::trunc);
if (!out) {
MGLOG_W("PipeStats: could not open MOBILEGL_PIPE_STATS_FILE='%s' for writing", path.c_str());
return;
}
out << FormatJson();
out.flush();
if (!out) {
MGLOG_W("PipeStats: failed writing MOBILEGL_PIPE_STATS_FILE='%s'", path.c_str());
return;
}
MGLOG_I("MGPipe stats: wrote JSON dump to %s", path.c_str());
}
} // namespace
void Init() {
ResetForTesting();
g_shutdownDone = false;
g_pipeStatsEnabled = MG_Config::Features.PipeStats;
if (g_pipeStatsEnabled) {
MGLOG_I("MGPipe stats: counters ON (MOBILEGL_PIPE_STATS), summary every %llu frames%s%s",
static_cast<unsigned long long>(kSummaryFramePeriod),
MG_Config::Features.PipeStatsFile.empty() ? "" : ", JSON dump to ",
MG_Config::Features.PipeStatsFile.c_str());
}
}
void Shutdown() {
if (!g_pipeStatsEnabled || g_shutdownDone) {
return;
}
g_shutdownDone = true;
EmitSummaryLine();
WriteJsonDump();
}
void AddBytes(ByteClass byteClass, Uint64 bytes) {
const Uint32 index = static_cast<Uint32>(byteClass);
Bump(g_frameBytes[index], bytes);
Bump(g_totalBytes[index], bytes);
}
void AddCalls(CallClass callClass, Uint64 count) {
const Uint32 index = static_cast<Uint32>(callClass);
Bump(g_frameCalls[index], count);
Bump(g_totalCalls[index], count);
}
void CountGate(Gate gate, Bool hit) {
const Uint32 index = static_cast<Uint32>(gate);
if (hit) {
Bump(g_frameGateHit[index], 1);
Bump(g_totalGateHit[index], 1);
} else {
Bump(g_frameGateMiss[index], 1);
Bump(g_totalGateMiss[index], 1);
}
}
void RecordDrawPayloadBytes(Uint64 bytes) { Bump(g_totalPayloadBuckets[PayloadBucketOf(bytes)], 1); }
void OnPresent() {
#ifdef TRACY_ENABLE
// One plot per counter, the frame's value. Tracy keeps the series by name, and the
// names are the static literals above, which is what TracyPlot requires.
for (Uint32 i = 0; i < kByteClassCount; ++i) {
TracyPlot(kByteClassNames[i], static_cast<Int64>(Read(g_frameBytes[i])));
}
for (Uint32 i = 0; i < kCallClassCount; ++i) {
TracyPlot(kCallClassNames[i], static_cast<Int64>(Read(g_frameCalls[i])));
}
for (Uint32 i = 0; i < kGateCount; ++i) {
TracyPlot(kGateNames[i], static_cast<Int64>(Read(g_frameGateMiss[i])));
}
#endif
for (Uint32 i = 0; i < kByteClassCount; ++i) {
g_frameBytes[i].store(0, std::memory_order_relaxed);
}
for (Uint32 i = 0; i < kCallClassCount; ++i) {
g_frameCalls[i].store(0, std::memory_order_relaxed);
}
for (Uint32 i = 0; i < kGateCount; ++i) {
g_frameGateHit[i].store(0, std::memory_order_relaxed);
g_frameGateMiss[i].store(0, std::memory_order_relaxed);
}
const Uint64 frames = g_frameCount.fetch_add(1, std::memory_order_relaxed) + 1;
if (frames % kSummaryFramePeriod == 0) {
EmitSummaryLine();
}
}
Uint64 FrameBytes(ByteClass byteClass) { return Read(g_frameBytes[static_cast<Uint32>(byteClass)]); }
Uint64 TotalBytes(ByteClass byteClass) { return Read(g_totalBytes[static_cast<Uint32>(byteClass)]); }
Uint64 FrameCalls(CallClass callClass) { return Read(g_frameCalls[static_cast<Uint32>(callClass)]); }
Uint64 TotalCalls(CallClass callClass) { return Read(g_totalCalls[static_cast<Uint32>(callClass)]); }
Uint64 TotalGateHits(Gate gate) { return Read(g_totalGateHit[static_cast<Uint32>(gate)]); }
Uint64 TotalGateMisses(Gate gate) { return Read(g_totalGateMiss[static_cast<Uint32>(gate)]); }
Uint64 TotalPayloadBucket(Uint32 bucket) {
return bucket < kPayloadHistogramBuckets ? Read(g_totalPayloadBuckets[bucket]) : 0;
}
Uint64 FrameCount() { return Read(g_frameCount); }
const char* NameOf(ByteClass byteClass) { return kByteClassNames[static_cast<Uint32>(byteClass)]; }
const char* NameOf(CallClass callClass) { return kCallClassNames[static_cast<Uint32>(callClass)]; }
const char* NameOf(Gate gate) { return kGateNames[static_cast<Uint32>(gate)]; }
String FormatSummaryLine() {
// Window values: everything since the previous summary. A run total over a workload
// whose shape changes (load, then steady state) hides exactly the number P2 wants.
const Uint64 frames = Read(g_frameCount);
const Uint64 windowFrames = frames - g_windowBaseFrames;
const Uint64 divisorFrames = windowFrames == 0 ? 1 : windowFrames;
Uint64 bytes[kByteClassCount];
for (Uint32 i = 0; i < kByteClassCount; ++i) {
bytes[i] = Read(g_totalBytes[i]) - g_windowBaseBytes[i];
}
Uint64 calls[kCallClassCount];
for (Uint32 i = 0; i < kCallClassCount; ++i) {
calls[i] = Read(g_totalCalls[i]) - g_windowBaseCalls[i];
}
Uint64 gateHit[kGateCount];
Uint64 gateMiss[kGateCount];
for (Uint32 i = 0; i < kGateCount; ++i) {
gateHit[i] = Read(g_totalGateHit[i]) - g_windowBaseGateHit[i];
gateMiss[i] = Read(g_totalGateMiss[i]) - g_windowBaseGateMiss[i];
}
const Uint64 draws = calls[static_cast<Uint32>(CallClass::Draws)];
const Uint64 accessorCalls = calls[static_cast<Uint32>(CallClass::AccessorCalls)];
String line = "MGPipe stats:";
line += " frames=" + std::to_string(frames);
line += " window=" + std::to_string(windowFrames);
line += " draws=" + std::to_string(draws);
line += " draws/f=" + std::to_string(draws / divisorFrames);
line += " acc=" + std::to_string(accessorCalls);
// Two decimals without <iomanip>: the per-draw accessor count is the number section
// 2.3.1 wants to an integer's worth of precision, and it is small (10-25).
const Uint64 accPerDrawHundredths = draws == 0 ? 0 : (accessorCalls * 100 + draws / 2) / draws;
line += " acc/draw=" + std::to_string(accPerDrawHundredths / 100) + "." +
(accPerDrawHundredths % 100 < 10 ? "0" : "") + std::to_string(accPerDrawHundredths % 100);
line += " bytes/f[";
for (Uint32 i = 0; i < kByteClassCount; ++i) {
if (i != 0) {
line += " ";
}
line += kByteClassShort[i];
line += "=";
line += std::to_string(bytes[i] / divisorFrames);
}
line += "] tex[emit=" + std::to_string(calls[static_cast<Uint32>(CallClass::TextureUploadEmissions)]);
line += " box=" + std::to_string(calls[static_cast<Uint32>(CallClass::TextureUploadBoxEmissions)]);
line += " rect=" + std::to_string(calls[static_cast<Uint32>(CallClass::TextureUploadRectEmissions)]);
line += " jobs=" + std::to_string(calls[static_cast<Uint32>(CallClass::TextureUploadJobs)]);
line += "] gates[";
for (Uint32 i = 0; i < kGateCount; ++i) {
if (i != 0) {
line += " ";
}
line += kGateShort[i];
line += "=";
line += std::to_string(gateHit[i]);
line += "/";
line += std::to_string(gateMiss[i]);
}
line += "]";
for (Uint32 i = 0; i < kByteClassCount; ++i) {
g_windowBaseBytes[i] = Read(g_totalBytes[i]);
}
for (Uint32 i = 0; i < kCallClassCount; ++i) {
g_windowBaseCalls[i] = Read(g_totalCalls[i]);
}
for (Uint32 i = 0; i < kGateCount; ++i) {
g_windowBaseGateHit[i] = Read(g_totalGateHit[i]);
g_windowBaseGateMiss[i] = Read(g_totalGateMiss[i]);
}
g_windowBaseFrames = frames;
return line;
}
String FormatJson() {
String json = "{\n";
json += " \"frames\": " + std::to_string(Read(g_frameCount)) + ",\n";
json += " \"bytes\": {\n";
for (Uint32 i = 0; i < kByteClassCount; ++i) {
json += " \"";
json += kByteClassNames[i];
json += "\": " + std::to_string(Read(g_totalBytes[i]));
json += (i + 1 == kByteClassCount) ? "\n" : ",\n";
}
json += " },\n \"calls\": {\n";
for (Uint32 i = 0; i < kCallClassCount; ++i) {
json += " \"";
json += kCallClassNames[i];
json += "\": " + std::to_string(Read(g_totalCalls[i]));
json += (i + 1 == kCallClassCount) ? "\n" : ",\n";
}
json += " },\n \"gates\": {\n";
for (Uint32 i = 0; i < kGateCount; ++i) {
json += " \"";
json += kGateNames[i];
json += "\": {\"hit\": " + std::to_string(Read(g_totalGateHit[i])) +
", \"miss\": " + std::to_string(Read(g_totalGateMiss[i])) + "}";
json += (i + 1 == kGateCount) ? "\n" : ",\n";
}
json += " },\n \"cmd-bytes-per-draw-histogram\": [";
for (Uint32 i = 0; i < kPayloadHistogramBuckets; ++i) {
if (i != 0) {
json += ", ";
}
json += std::to_string(Read(g_totalPayloadBuckets[i]));
}
json += "]\n}\n";
return json;
}
void SetEnabledForTesting(Bool enabled) { g_pipeStatsEnabled = enabled; }
void ResetForTesting() {
for (Uint32 i = 0; i < kByteClassCount; ++i) {
g_frameBytes[i].store(0, std::memory_order_relaxed);
g_totalBytes[i].store(0, std::memory_order_relaxed);
g_windowBaseBytes[i] = 0;
}
for (Uint32 i = 0; i < kCallClassCount; ++i) {
g_frameCalls[i].store(0, std::memory_order_relaxed);
g_totalCalls[i].store(0, std::memory_order_relaxed);
g_windowBaseCalls[i] = 0;
}
for (Uint32 i = 0; i < kGateCount; ++i) {
g_frameGateHit[i].store(0, std::memory_order_relaxed);
g_totalGateHit[i].store(0, std::memory_order_relaxed);
g_frameGateMiss[i].store(0, std::memory_order_relaxed);
g_totalGateMiss[i].store(0, std::memory_order_relaxed);
g_windowBaseGateHit[i] = 0;
g_windowBaseGateMiss[i] = 0;
}
for (Uint32 i = 0; i < kPayloadHistogramBuckets; ++i) {
g_totalPayloadBuckets[i].store(0, std::memory_order_relaxed);
}
g_frameCount.store(0, std::memory_order_relaxed);
g_windowBaseFrames = 0;
}
} // namespace MobileGL::MG_Util::PipeStats
+171
View File
@@ -0,0 +1,171 @@
// MobileGL - MobileGL/MG_Util/Metrics/PipeStats.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
// MGPipe boundary counters (plan B section 11 "P0 - hygiene, measurement, gates and
// skeleton", and the corollary in section 2.3.1).
//
// WHAT THIS IS FOR. The disaggregation plan has to size two things it cannot size by
// reading the tree: how many BYTES cross the frontend/backend boundary per frame (that
// sizes SEG_STAGE and the command segment), and how many accessor CALLS and memo-gate
// probes the backends actually execute per draw (that decides whether pushing state is
// cheaper than pulling it at all). Section 2.3.1 makes the second one the load-bearing
// number: the static call-site counts everyone quoted - Espryt 124 / Magma 169 - are NOT
// the dynamic per-draw cost, because every one of those paths is memo-gated, and the real
// steady state is believed to be 10-25 accessor calls per backend per draw. Without a
// dynamic counter the P2 verdict stays a guess.
//
// COST WHEN OFF. g_pipeStatsEnabled is a plain global Bool latched once at Init() from
// MG_Config::Features.PipeStats (MOBILEGL_PIPE_STATS). Every counting site in the two
// backends is written as
//
// if (MG_Util::PipeStats::Enabled()) MG_Util::PipeStats::Add...(...);
//
// so with the feature off a site costs one load of a hot global plus one never-taken,
// perfectly-predicted branch, and none of the counter state is touched. The counters
// themselves are relaxed atomics rather than plain integers because texture and buffer
// staging can be reached from more than one thread; relaxed adds cost nothing extra on the
// off path, which never reaches them.
//
// WHAT IS COUNTED AND WHAT IS NOT: see the site inventory in PipeStats.cpp.
namespace MobileGL::MG_Util::PipeStats {
// Byte classes. Every one of these names a population of bytes that would have to be
// MOVED across the boundary once the backend no longer shares an address space with
// the frontend, which is why they are grouped this way rather than by call site.
enum class ByteClass : Uint32 {
// Buffer object contents flushed to the driver: glBufferData / glBufferSubData /
// map-write ranges / the persistent upload ring.
StageBuffer = 0,
// Texel bytes handed to glTexSubImage & friends, whichever upload shape was chosen.
StageTexture,
// The default-uniform-block ("global UBO") image, uploaded at most once per program
// per frame.
StageUboGlobal,
// Named uniform-block bytes that a backend has to repack itself, i.e. Magma's UBO
// ring. Espryt binds the frontend buffer straight to the driver and contributes
// nothing here - which is exactly the asymmetry D-B8 is about.
StageUboNamed,
// Client-memory vertex arrays uploaded into a scratch VBO on the draw path.
StageVertexClient,
// Client-memory / rewritten index data staged on the draw path.
StageIndexClient,
// Bytes pushed because a persistently mapped range was published to the backend.
PersistentMapPush,
// PLACEHOLDER (plan section 6.3): the residual value block does not exist yet. The
// class is minted now so the counter names never churn; it stays at 0 until P2.
ResidualValueBlock,
Count
};
// Call classes: the dynamic per-draw cost section 2.3.1 says P2 cannot be decided
// without.
enum class CallClass : Uint32 {
// Draws that reached an instrumented backend draw-preparation entry point. The
// denominator for every "per draw" number below.
Draws = 0,
// GLContext accessor calls actually EXECUTED on the instrumented paths. Counted in
// static tallies at the ~10 hot entry points, not by wrapping all 293 call sites -
// see the inventory in PipeStats.cpp for exactly what is and is not in this number.
AccessorCalls,
// Texture upload emissions: one per (upload target, level) that actually shipped
// texels. The eventual resource_subdata record count.
TextureUploadEmissions,
// Emissions that took the union-box shape (one driver upload job).
TextureUploadBoxEmissions,
// Emissions that took the refined rect-list shape (N driver upload jobs). The
// box/rect split is the thing SSIM cannot see and the +6 ms/frame Mali cliff came
// from, so it is counted separately from the byte total.
TextureUploadRectEmissions,
// Driver upload jobs issued by those emissions: 1 per box emission, N per rect-list
// emission.
TextureUploadJobs,
Count
};
// Memo gates. Each is a place where a backend decides "nothing moved, skip the work".
// Hit == the gate short-circuited; Miss == it fell through and did the work. The six
// are exactly the ones section 2.3.1 tabulates.
enum class Gate : Uint32 {
// DirectGLES.cpp SyncRenderState: the render-state-version early-out.
EsprytRenderState = 0,
// DirectGLES.cpp SyncNeccessaryTextures: the six-value sync-list key compare.
EsprytTextureSyncList,
// DirectGLES.cpp CurrentUnitBindingsEpoch: the (context, max unit, bind generation)
// shutter over the unit walk.
EsprytUnitBindingsEpoch,
// VulkanRenderer.cpp TrySetupDrawFastPath: the whole snapshot fast path.
MagmaDrawFastPath,
// VulkanRenderer.cpp GetOrCreatePipeline: the pipeline memo.
MagmaPipelineMemo,
// VulkanRenderer.cpp ApplyDynamicDrawStateTail: the version+extent tail gate.
MagmaDynamicTail,
Count
};
// Per-draw command payload size histogram (plan section 4.5.7: SEG_CMD has to be sized
// off the DISTRIBUTION, not off a per-frame total). PLACEHOLDER in P0: MGPipe emits no
// records yet, so nothing in the backends calls RecordDrawPayloadBytes. The bucketing
// and the reporting are implemented and unit-tested so that the first generator to
// emit records only has to add the one call.
inline constexpr Uint32 kPayloadHistogramBuckets = 24;
// Frames between two summary lines when MOBILEGL_PIPE_STATS=1.
inline constexpr Uint64 kSummaryFramePeriod = 120;
// The latch. Read directly by Enabled() so the off path is a global load and a
// predicted branch - do not turn this into a function call.
extern Bool g_pipeStatsEnabled;
inline Bool Enabled() { return g_pipeStatsEnabled; }
// Latches g_pipeStatsEnabled from MG_Config::Features.PipeStats and resets every
// counter. Called from MobileGL::Initialize() right after the config load.
void Init();
// Final summary line plus, if MOBILEGL_PIPE_STATS_FILE names a path, the JSON dump.
// Called from MobileGL's teardown. Idempotent.
void Shutdown();
void AddBytes(ByteClass byteClass, Uint64 bytes);
void AddCalls(CallClass callClass, Uint64 count);
void CountGate(Gate gate, Bool hit);
void RecordDrawPayloadBytes(Uint64 bytes);
// Frame boundary: publishes the frame's values to Tracy (when TRACY_ENABLE), folds them
// into the run totals, clears the frame accumulators, and every kSummaryFramePeriod
// frames emits the summary line. Called from each backend's Present().
void OnPresent();
// --- introspection, for the unit test and the JSON dump -------------------------
Uint64 FrameBytes(ByteClass byteClass);
Uint64 TotalBytes(ByteClass byteClass);
Uint64 FrameCalls(CallClass callClass);
Uint64 TotalCalls(CallClass callClass);
Uint64 TotalGateHits(Gate gate);
Uint64 TotalGateMisses(Gate gate);
Uint64 TotalPayloadBucket(Uint32 bucket);
Uint64 FrameCount();
const char* NameOf(ByteClass byteClass);
const char* NameOf(CallClass callClass);
const char* NameOf(Gate gate);
// The compact fixed-format one-liner MGLOG_I prints. Same text in the log and in the
// test, so the format is pinned by a test rather than by the log reader's memory.
String FormatSummaryLine();
// The teardown dump. Run totals only: a per-frame JSON stream is a different tool.
String FormatJson();
// Test hooks. Not used by any shipping path.
void SetEnabledForTesting(Bool enabled);
void ResetForTesting();
} // namespace MobileGL::MG_Util::PipeStats