swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-09-12 06:08:30 +09:00
Code Issues Packages Projects Releases Wiki Activity

[Perf] (MG_State, MG_Util): compile shaders with a single relaxed parse

Browse Source 
glCompileShader used to parse every source twice: once under the GL client
(reflection only) and once under the relaxed Vulkan client (SPIR-V + the
plain-uniform global UBO), with GenerateBinary re-preprocessing, re-parsing
and re-linking every attached shader on every glLinkProgram. The GL-client
pass is gone: Compile() performs the one link-compatible relaxed parse and
the linked TProgram serves reflection and codegen both. Measured on the BSL
shaderpack compile phase: Espryt 2.80s -> 2.14s, Magma 3.78s -> 3.07s.

What the relaxed parse cannot provide is restored explicitly:
- explicit layout(location/binding) qualifiers on default-block uniforms and
  samplers are extracted lexically at Compile() (the relaxed parse strips
  them) and merged per link with cross-stage conflict checks;
- uniforms the relaxed parse sweeps into MGL_GLOBAL_UBO but no stage reads
  are filtered from the GL reflection surface through GL<->TProgram index
  translation maps (dead uniforms stay inactive, the synthesized block stays
  hidden, builtins reflect under their GL spellings);
- SPIR-V is generated BEFORE buildReflection touches the program (its
  live-variable analysis perturbs GlslangToSpv output - generated modules
  stay bit-identical to the old pipeline's), while the glUniform*-to-scratch
  routing tables are built strictly AFTER reflection, whose results size and
  key them;
- a TShader feeds exactly one link (mapIO mutates the intermediate); relinks
  and multi-program attachments re-parse the stored preprocessed source.

Validated: DirectGLES retrace suite green (two pre-existing local-driver
failures unchanged old vs new), KHR-GL30 877/878 on Espryt/NVIDIA (the one
failure pre-exists this change), unit tests green, per-module SPIR-V hashes
identical across a full DirectVulkan replay.
This commit is contained in:
BZLZHH
2026-08-08 01:25:54 -04:00
parent 81bcbd6c14
commit 0d0527192a
7 changed files with 983 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp
+279
View File
@@ -10,6 +10,8 @@
#include <algorithm>
#include <cctype>
#include <climits>
#include <cstdlib>
#include <initializer_list>
#include <utility>
#include <Config.h>
@@ -1441,6 +1443,283 @@ namespace MobileGL {
return std::nullopt;
}
namespace {
bool IsNonLayoutQualifierKeyword(const String& text) {
static const char* kQualifiers[] = {
"highp", "mediump", "lowp", "precise", "const", "flat",
"noperspective", "smooth", "centroid", "sample", "patch", "invariant",
"coherent", "volatile", "restrict", "readonly", "writeonly", "subroutine",
};
for (const char* qualifier : kQualifiers) {
if (text == qualifier) return true;
}
return false;
}
bool IsDecimalIntegerToken(const String& text) {
if (text.empty()) return false;
return std::all_of(text.begin(), text.end(),
[](char ch) { return ch >= '0' && ch <= '9'; });
}
// Parses one brace-free depth-0 statement [begin, end) and records its
// declarators when it is a uniform declaration carrying an integral
// layout(location = N). Multi-declarator statements assign consecutive
// locations, each declarator advancing by its array element count
// (ARB_explicit_uniform_location rules). Anything the narrow grammar does
// not recognize is skipped, never guessed at.
void RecordUniformDeclarationLocations(const Vector<CodeToken>& tokens, SizeT begin, SizeT end,
MobileGL::UnorderedMap<String, MobileGL::Int>& locations) {
using MobileGL::Int;
long long location = -1;
bool sawUniform = false;
SizeT declaratorBegin = end;
for (SizeT k = begin; k < end;) {
const String& text = tokens[k].text;
if (text == "layout" && k + 1 < end && tokens[k + 1].text == "(") {
SizeT j = k + 2;
Int parenDepth = 1;
while (j < end && parenDepth > 0) {
const String& layoutToken = tokens[j].text;
if (layoutToken == "(") {
++parenDepth;
} else if (layoutToken == ")") {
--parenDepth;
} else if (parenDepth == 1 && layoutToken == "location" && j + 2 < end &&
tokens[j + 1].text == "=" && IsDecimalIntegerToken(tokens[j + 2].text)) {
location = std::min(std::strtoll(tokens[j + 2].text.c_str(), nullptr, 10),
static_cast<long long>(INT_MAX / 2));
j += 2;
}
++j;
}
k = j;
continue;
}
if (text == "uniform") {
sawUniform = true;
++k;
continue;
}
if (sawUniform && location >= 0 && IsIdentifierToken(tokens[k]) &&
!IsNonLayoutQualifierKeyword(text)) {
declaratorBegin = k + 1; // 'text' is the type; declarators follow
break;
}
++k;
}
if (!sawUniform || location < 0 || declaratorBegin >= end) return;
long long nextLocation = location;
for (SizeT k = declaratorBegin; k < end;) {
if (!IsIdentifierToken(tokens[k])) return; // malformed; record nothing further
const String& name = tokens[k].text;
++k;
long long span = 1;
while (k < end && tokens[k].text == "[") {
++k;
long long dimension = 1;
if (k < end && IsDecimalIntegerToken(tokens[k].text)) {
dimension = std::strtoll(tokens[k].text.c_str(), nullptr, 10);
++k;
}
if (k >= end || tokens[k].text != "]") return; // sized by expression; bail out
++k;
span *= std::max(1ll, std::min(dimension, static_cast<long long>(INT_MAX / 2)));
}
// Keep the first sighting: a duplicate can only come from alternative
// preprocessor branches declaring the same name.
locations.emplace(name, static_cast<Int>(std::min(
nextLocation, static_cast<long long>(INT_MAX / 2))));
nextLocation += span;
if (k >= end) break;
if (tokens[k].text == "=") { // skip an initializer up to the declarator comma
Int nestingDepth = 0;
++k;
while (k < end) {
const String& initializerToken = tokens[k].text;
if (initializerToken == "(" || initializerToken == "[") {
++nestingDepth;
} else if (initializerToken == ")" || initializerToken == "]") {
--nestingDepth;
} else if (initializerToken == "," && nestingDepth == 0) {
break;
}
++k;
}
}
if (k >= end) break;
if (tokens[k].text != ",") return;
++k;
}
}
// Parses one brace-free depth-0 statement [begin, end) and records its
// declarators when it is a sampler/image uniform declaration carrying an
// integral layout(binding = N). Such a binding is a GL texture/image unit,
// which the Vulkan-client relaxed parse strips before mapIO can observe it
// (it is not a valid descriptor binding there), so it is extracted lexically
// and restored as the uniform's initial unit. Every declarator in the
// statement shares the qualifier's binding, matching what the GL-client
// mapIO used to capture from the shared type qualifier. Anything the narrow
// grammar does not recognize is skipped, never guessed at.
void RecordOpaqueDeclarationBindings(const Vector<CodeToken>& tokens, SizeT begin, SizeT end,
MobileGL::UnorderedMap<String, MobileGL::Uint>& bindings) {
using MobileGL::Int;
long long binding = -1;
bool sawUniform = false;
SizeT declaratorBegin = end;
for (SizeT k = begin; k < end;) {
const String& text = tokens[k].text;
if (text == "layout" && k + 1 < end && tokens[k + 1].text == "(") {
SizeT j = k + 2;
Int parenDepth = 1;
while (j < end && parenDepth > 0) {
const String& layoutToken = tokens[j].text;
if (layoutToken == "(") {
++parenDepth;
} else if (layoutToken == ")") {
--parenDepth;
} else if (parenDepth == 1 && layoutToken == "binding" && j + 2 < end &&
tokens[j + 1].text == "=" && IsDecimalIntegerToken(tokens[j + 2].text)) {
binding = std::min(std::strtoll(tokens[j + 2].text.c_str(), nullptr, 10),
static_cast<long long>(INT_MAX / 2));
j += 2;
}
++j;
}
k = j;
continue;
}
if (text == "uniform") {
sawUniform = true;
++k;
continue;
}
if (sawUniform && binding >= 0 && IsIdentifierToken(tokens[k]) &&
!IsNonLayoutQualifierKeyword(text)) {
// 'text' is the type. Only sampler/image opaques carry unit
// bindings; on anything else (e.g. atomic_uint, whose binding
// is a counter-buffer index) record nothing.
if (text.find("sampler") == String::npos && text.find("image") == String::npos) return;
declaratorBegin = k + 1;
break;
}
++k;
}
if (!sawUniform || binding < 0 || declaratorBegin >= end) return;
for (SizeT k = declaratorBegin; k < end;) {
if (!IsIdentifierToken(tokens[k])) return; // malformed; record nothing further
const String& name = tokens[k].text;
++k;
while (k < end && tokens[k].text == "[") {
++k;
if (k < end && IsDecimalIntegerToken(tokens[k].text)) ++k;
if (k >= end || tokens[k].text != "]") return; // sized by expression; bail out
++k;
}
bindings[name] = static_cast<MobileGL::Uint>(binding);
if (k >= end) break;
if (tokens[k].text != ",") return; // opaque declarators cannot take initializers
++k;
}
}
} // namespace
UnorderedMap<String, Uint> ExtractExplicitOpaqueBindings(const String& source) {
UnorderedMap<String, Uint> bindings;
// Fast path: without the qualifier keyword there is nothing to extract.
if (source.find("binding") == String::npos) return bindings;
const Vector<CodeToken> tokens = TokenizeCode(source);
const SizeT count = tokens.size();
Int braceDepth = 0;
SizeT pos = 0;
while (pos < count) {
const String& text = tokens[pos].text;
if (text == "{") {
++braceDepth;
++pos;
continue;
}
if (text == "}") {
if (braceDepth > 0) --braceDepth;
++pos;
continue;
}
if (braceDepth != 0 || text == ";") {
++pos;
continue;
}
// A depth-0 statement runs to its ';'. One that opens a brace instead is
// a function definition or an interface/uniform block: a block's binding
// is a buffer binding point, not a texture unit, so skip both alike.
SizeT statementEnd = pos;
while (statementEnd < count && tokens[statementEnd].text != ";" &&
tokens[statementEnd].text != "{") {
++statementEnd;
}
if (statementEnd >= count || tokens[statementEnd].text == "{") {
pos = statementEnd;
continue;
}
RecordOpaqueDeclarationBindings(tokens, pos, statementEnd, bindings);
pos = statementEnd + 1;
}
return bindings;
}
UnorderedMap<String, Int> ExtractExplicitUniformLocations(const String& source) {
UnorderedMap<String, Int> locations;
// Fast path: without the qualifier keyword there is nothing to extract.
if (source.find("location") == String::npos) return locations;
const Vector<CodeToken> tokens = TokenizeCode(source);
const SizeT count = tokens.size();
Int braceDepth = 0;
SizeT pos = 0;
while (pos < count) {
const String& text = tokens[pos].text;
if (text == "{") {
++braceDepth;
++pos;
continue;
}
if (text == "}") {
if (braceDepth > 0) --braceDepth;
++pos;
continue;
}
if (braceDepth != 0 || text == ";") {
++pos;
continue;
}
// A depth-0 statement runs to its ';'. One that opens a brace instead is a
// function definition or an interface/uniform block: neither can declare a
// default-block uniform location, so hand the '{' back to the depth tracker.
SizeT statementEnd = pos;
while (statementEnd < count && tokens[statementEnd].text != ";" &&
tokens[statementEnd].text != "{") {
++statementEnd;
}
if (statementEnd >= count || tokens[statementEnd].text == "{") {
pos = statementEnd;
continue;
}
RecordUniformDeclarationLocations(tokens, pos, statementEnd, locations);
pos = statementEnd + 1;
}
return locations;
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.h
+22
View File
@@ -45,6 +45,28 @@ namespace MobileGL {
// "row_major" outside a layout(...) list, the image*Shadow family). Returns the
// compile-error text for the first violation, or nullopt for a clean source.
std::optional<String> FindReservedIdentifierViolation(const String& source);
// Explicit layout(location = N) qualifiers on default-block uniform declarations,
// keyed by declared name (no "[0]" suffix). Multi-declarator statements assign
// consecutive locations, advancing by the array element count.
//
// Exists because the single link-compatible parse runs under relaxed Vulkan rules,
// where glslang's vkRelaxedRemapUniformVariable moves plain uniforms into
// MGL_GLOBAL_UBO and DISCARDS their location qualifiers ("ignoring layout qualifier
// for uniform location"); opaque uniforms keep theirs. This lexical side-channel
// restores the discarded locations to the GL location assigner
// (ProgramObject::DoReflection). It scans preprocessor-visible text, so a
// declaration inside an inactive #if branch is still recorded - harmless unless a
// pack declares the same uniform with different explicit locations in alternative
// branches (none observed; explicit uniform locations have zero incidence in the
// shader-pack corpus, this is an ARB_explicit_uniform_location conformance surface).
UnorderedMap<String, Int> ExtractExplicitUniformLocations(const String& source);
// Explicit layout(binding = N) on sampler/image uniforms, i.e. their initial
// texture/image units. The Vulkan-client relaxed parse strips these before
// mapIO can capture them, so they are recovered lexically (same narrow
// grammar discipline as ExtractExplicitUniformLocations).
UnorderedMap<String, Uint> ExtractExplicitOpaqueBindings(const String& source);
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL