mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-09-11 21:58:31 +09:00
[Feat] (Pipe): map every frontend mutator onto the aggregate generation that publishes it, and make the dirty-surface scanner a gate
- MG_Pipe/DirtySurface.def: 73 rows, one per distinct mutator the scanner finds, each answering
"what publishes this". The answer vocabulary is a MGPipeDirty bit name or one of five
non-bit answers, and each of the five is documented in the file's header rather than left to
be inferred: kImmediate, kReverseChannel, kNoBackendRead, kExplicitDestroy and
kPulledEveryVerb. Where a mutator has more than one true answer the row carries the COARSER
one - the one that cannot under-fire.
- gen_pipe_dirty_surface.py --check is the gate and it fails in BOTH directions: an unmapped
mutator renders stale, and a row naming a mutator the scan no longer finds keeps a real hole
looking covered. It also rejects an answer that is neither a documented non-bit answer nor a
bit name read out of Tracker.h's own kMGPipeDirtyNames, so a renamed bit cannot leave a row
silently pointing at nothing.
- --self-test runs three canned negative controls - a withheld mutator, a stale row, a bad
answer - and each must trip; trips == 0 is itself an error, the shape
check_include_closure.py and gen_pipe.py --self-test already use. ROADMAP.md's rule is that
every gate must be able to go red for the reason it exists.
- --summary keeps working unchanged, because the CI file that still calls it belongs to
another package until it lands.
- The human report prints the mapped answer where it printed UNMAPPED.
- FillPoints.def: the verdict on the eight statically over-approximated rows, recorded per
group in the def's own comment. All eight are KEPT and the reason is the same in all three
groups - each row names a concrete backend path (the depth/stencil read emulation's paused
capture, VkClearManager::PreCompensateSrgbClearColor's GL_FRAMEBUFFER_SRGB read, the shader
blit's viewport / provoking vertex / binding-point reads), and the only evidence that could
retire one is dynamic. A corpus that never reaches a path proves nothing about it, and a row
dropped on that basis turns a rare path into Fatal{UnmigratedPipeInput} in a shipped build.
The contract's new FramebufferSrgb storage in fact makes one of the eight MORE load-bearing
than it was, not less: it used to read a compile-time constant.
This commit is contained in:
@@ -0,0 +1,151 @@
|
|||||||
|
// MobileGL - MobileGL/MG_Pipe/DirtySurface.def
|
||||||
|
// Copyright (c) 2025-2026 MobileGL-Dev
|
||||||
|
// Licensed under the GNU Lesser General Public License v3.0:
|
||||||
|
// https://www.gnu.org/licenses/gpl-3.0.txt
|
||||||
|
// https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||||
|
// SPDX-License-Identifier: LGPL-3.0-only
|
||||||
|
// End of Source File Header
|
||||||
|
|
||||||
|
// The dirty-surface mapping (ARCHITECTURE.md 5.2 corollary 4, P2 brief D16).
|
||||||
|
//
|
||||||
|
// MGPipe replaces "the backend rediscovers what changed" with "the frontend says what
|
||||||
|
// changed", which only works if EVERY frontend mutation a backend can observe has an answer
|
||||||
|
// to "what publishes this". The failure mode is silent and one-directional: a mutation that
|
||||||
|
// forgets to publish renders stale, and no purity gate can see it.
|
||||||
|
//
|
||||||
|
// So the surface is enumerated MECHANICALLY. scripts/gen_pipe_dirty_surface.py scans
|
||||||
|
// MG_Impl/GLImpl for every pGLContext-> mutator call and, with --check, fails if a scanned
|
||||||
|
// mutator has no row here or a row here names a mutator the scan no longer finds. Both
|
||||||
|
// directions, so a deleted mutator cannot leave a stale row behind either.
|
||||||
|
//
|
||||||
|
// ANSWERS. One per row, and where a mutator has more than one true answer the row carries
|
||||||
|
// the COARSER one - the one that cannot under-fire:
|
||||||
|
//
|
||||||
|
// NEW_* a MGPipeDirty bit (MG_Impl/Pipe/Tracker.h). The tracker's shutter for
|
||||||
|
// that bit moves when this mutator runs, so the next verb publishes it.
|
||||||
|
// kImmediate the mutating function also reaches the backend in the same body, so the
|
||||||
|
// mutation is published inline and needs no shutter at all.
|
||||||
|
// kReverseChannel not state: a write INTO the frontend from the backend's side.
|
||||||
|
// kNoBackendRead no backend read point observes this state at all.
|
||||||
|
// kExplicitDestroy published by the delete_* call the Track H slice emits when the object's
|
||||||
|
// last reference drops - an object's DEATH, which no generation shutters
|
||||||
|
// because there is no longer an object to carry one.
|
||||||
|
// kPulledEveryVerb no shutter exists, and none is needed yet: the PipeInputs field this
|
||||||
|
// writes is in its verb class's may-read mask, so the residual fill copies
|
||||||
|
// it at EVERY verb of that class. A shutter here is a P3/P4 optimisation,
|
||||||
|
// not a correctness gap.
|
||||||
|
//
|
||||||
|
// KNOWN BLIND SPOTS OF THE SCANNER, recorded here rather than left implicit
|
||||||
|
// (gen_pipe_dirty_surface.py's own notes plus its scan root):
|
||||||
|
// 1. it matches braced function bodies textually, so a mutator inside a LAMBDA is
|
||||||
|
// attributed to the enclosing function;
|
||||||
|
// 2. a mutation published through a HELPER the entry point calls reads as deferred here;
|
||||||
|
// 3. the scan root is MG_Impl/GLImpl only, so the four MGP_NOTE_MUTATION sites in
|
||||||
|
// MG_State/GLState/TextureState/TextureState.h are outside it entirely.
|
||||||
|
// The gate is therefore a COMPLETENESS gate over what the scanner does see. The semantic
|
||||||
|
// proof stays the MOBILEGL_PIPE_VERIFY lane, which is blind to none of the three.
|
||||||
|
//
|
||||||
|
// clang-format off
|
||||||
|
|
||||||
|
// X(Mutator, Answer)
|
||||||
|
#define MGP_DIRTY_SURFACE_LIST(X) \
|
||||||
|
/* ---- the reverse channel: 836 of the 926 calls, 90% of the surface ---- */ \
|
||||||
|
X(RecordError, kReverseChannel) \
|
||||||
|
/* ---- immediate publish points: the same body reaches the backend ---- */ \
|
||||||
|
X(SetActiveTextureUnit, kImmediate) \
|
||||||
|
X(BeginTransformFeedback, kImmediate) \
|
||||||
|
X(EndTransformFeedback, kImmediate) \
|
||||||
|
X(SetTransformFeedbackPaused, kImmediate) \
|
||||||
|
X(MarkTransformFeedbackObjectForDeletion, kImmediate) \
|
||||||
|
/* ---- the render state: NEW_PIPELINE_STATE when a setter calls */ \
|
||||||
|
/* BumpVersions (P2 brief D6's only rule), NEW_RENDER_STATE otherwise */ \
|
||||||
|
X(SetBlendEquation, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetBlendEquationIndexed, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetBlendFunc, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetBlendFuncIndexed, NEW_PIPELINE_STATE) \
|
||||||
|
/* SetCapability's ClipDistance0..7 arms move only m_version; every other */ \
|
||||||
|
/* arm calls BumpVersions, and the coarser answer is the one that holds. */ \
|
||||||
|
X(SetCapability, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetCapabilityIndexed, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetColorMask, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetColorMaskIndexed, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetCullFaceMode, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetDepthFunc, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetDepthMask, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetFrontFaceMode, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetLogicOp, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetMinSampleShadingValue, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetPolygonMode, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetProvokingVertexMode, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetSampleCoverage, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetSampleMaskValue, NEW_PIPELINE_STATE) \
|
||||||
|
/* SetStencilFunc writes Func (pipeline chunk P2/P3) AND Ref/ValueMask */ \
|
||||||
|
/* (dynamic D3/D4); SetStencilOp is wholly pipeline, SetStencilMask wholly */ \
|
||||||
|
/* dynamic. That split is what keeps a glStencilFunc that moves only the */ \
|
||||||
|
/* reference from evicting a cached pipeline. */ \
|
||||||
|
X(SetStencilFunc, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetStencilOp, NEW_PIPELINE_STATE) \
|
||||||
|
X(SetStencilMask, NEW_RENDER_STATE) \
|
||||||
|
X(SetBlendColor, NEW_RENDER_STATE) \
|
||||||
|
X(SetClampReadColor, NEW_RENDER_STATE) \
|
||||||
|
X(SetClearColor, NEW_RENDER_STATE) \
|
||||||
|
X(SetClearDepth, NEW_RENDER_STATE) \
|
||||||
|
X(SetClearStencil, NEW_RENDER_STATE) \
|
||||||
|
X(SetClipControl, NEW_RENDER_STATE) \
|
||||||
|
X(SetDepthRange, NEW_RENDER_STATE) \
|
||||||
|
X(SetDepthRangeIndexed, NEW_RENDER_STATE) \
|
||||||
|
X(SetHint, NEW_RENDER_STATE) \
|
||||||
|
X(SetLineWidth, NEW_RENDER_STATE) \
|
||||||
|
X(SetPointFadeThresholdSize, NEW_RENDER_STATE) \
|
||||||
|
X(SetPointSize, NEW_RENDER_STATE) \
|
||||||
|
X(SetPointSpriteCoordOrigin, NEW_RENDER_STATE) \
|
||||||
|
X(SetPolygonOffset, NEW_RENDER_STATE) \
|
||||||
|
X(SetPolygonOffsetClamped, NEW_RENDER_STATE) \
|
||||||
|
X(SetPrimitiveRestartIndex, NEW_RENDER_STATE) \
|
||||||
|
X(SetScissorBox, NEW_RENDER_STATE) \
|
||||||
|
X(SetScissorBoxIndexed, NEW_RENDER_STATE) \
|
||||||
|
X(SetViewport, NEW_RENDER_STATE) \
|
||||||
|
X(SetViewportIndexed, NEW_RENDER_STATE) \
|
||||||
|
/* ---- the other value-class bits ---- */ \
|
||||||
|
X(SetPixelStoreParam, NEW_PIXEL_PACK) \
|
||||||
|
X(SetPatchDefaultInnerLevel, NEW_PATCH_STATE) \
|
||||||
|
X(SetPatchDefaultOuterLevel, NEW_PATCH_STATE) \
|
||||||
|
/* Also an immediate publish point, but it has a real bit and the bit is */ \
|
||||||
|
/* the more useful answer: set_patch_state carries it whatever the caller */ \
|
||||||
|
/* does next. */ \
|
||||||
|
X(SetPatchVertices, NEW_PATCH_STATE) \
|
||||||
|
X(SetCurrentVertexAttributeFloat, NEW_VERTEX_ATTRIB_DEFAULTS) \
|
||||||
|
X(SetCurrentVertexAttributeInt, NEW_VERTEX_ATTRIB_DEFAULTS) \
|
||||||
|
X(SetCurrentVertexAttributeUint, NEW_VERTEX_ATTRIB_DEFAULTS) \
|
||||||
|
/* ---- object class ---- */ \
|
||||||
|
X(BumpTextureBindGeneration, NEW_SAMPLER_VIEWS) \
|
||||||
|
X(SetNamedTransformFeedbackBinding, NEW_SO_TARGETS) \
|
||||||
|
/* ---- an object's death: no generation, because there is no longer an */ \
|
||||||
|
/* object to carry one. Espryt 0b's delete_* is what publishes these. */ \
|
||||||
|
X(MarkBufferObjectForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkFramebufferObjectForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkProgramForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkProgramPipelineForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkRenderbufferObjectForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkSamplerObjectForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkShaderForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkTextureObjectForDeletion, kExplicitDestroy) \
|
||||||
|
X(MarkVertexArrayForDeletion, kExplicitDestroy) \
|
||||||
|
/* ---- no backend read point observes these at all ---- */ \
|
||||||
|
/* GL_ANY_SAMPLES_PASSED conditional rendering is resolved wholly in the */ \
|
||||||
|
/* frontend: IsConditionalRenderActive / GetConditionalRenderQuery have no */ \
|
||||||
|
/* reader under MG_Backend and no Coverage.def row. */ \
|
||||||
|
X(BeginConditionalRender, kNoBackendRead) \
|
||||||
|
X(EndConditionalRender, kNoBackendRead) \
|
||||||
|
/* ---- pulled at every verb of the class, so the next verb publishes them */ \
|
||||||
|
/* unconditionally. The transform-feedback accounting counters reach the */ \
|
||||||
|
/* backend through GetTransformFeedbackCapturedVertices and friends, which */ \
|
||||||
|
/* are in the kDraw and kXfbSpan may-read masks. */ \
|
||||||
|
X(AddTransformFeedbackAccountedCaptureDraw, kPulledEveryVerb) \
|
||||||
|
X(AddTransformFeedbackCapturedVertices, kPulledEveryVerb) \
|
||||||
|
X(AddTransformFeedbackGeometryCaptureDraw, kPulledEveryVerb) \
|
||||||
|
X(AddTransformFeedbackInputPrimitives, kPulledEveryVerb) \
|
||||||
|
X(AddTransformFeedbackPausedPrimitives, kPulledEveryVerb) \
|
||||||
|
X(AddTransformFeedbackPrimitives, kPulledEveryVerb)
|
||||||
|
|
||||||
|
// clang-format on
|
||||||
@@ -26,6 +26,44 @@
|
|||||||
// Fatal{UnmigratedPipeInput, "Field@Verb"} found there is fixed by adding the (class, field)
|
// Fatal{UnmigratedPipeInput, "Field@Verb"} found there is fixed by adding the (class, field)
|
||||||
// row, never by marking the field sticky.
|
// row, never by marking the field sticky.
|
||||||
//
|
//
|
||||||
|
// ---------------------------------------------------------------------------------------
|
||||||
|
// THE VERDICT ON THE EIGHT STATICALLY OVER-APPROXIMATED ROWS (P2 brief C.1, MEASUREMENTS.md
|
||||||
|
// section 4). Every one of them is KEPT, and the reason is the same in all three groups: the
|
||||||
|
// row is not a guess, it names a concrete backend path, and the only evidence that could
|
||||||
|
// retire it is DYNAMIC - a corpus that never reaches the path proves nothing, because a row
|
||||||
|
// removed on that basis turns a rare path into Fatal{UnmigratedPipeInput} in a shipped build.
|
||||||
|
//
|
||||||
|
// kReadback + IsTransformFeedbackActive / IsTransformFeedbackPaused
|
||||||
|
// KEPT. The depth/stencil read emulation draws (ScopedEmulationDrawState, DirectGLES.cpp)
|
||||||
|
// and pauses an active capture around its own draw, so a glReadPixels of a depth or
|
||||||
|
// stencil attachment reads the transform-feedback state exactly as a draw does. Reached
|
||||||
|
// only when the emulation is armed, which is a driver-shaped decision, so no desktop
|
||||||
|
// corpus can decide it.
|
||||||
|
//
|
||||||
|
// kTextureOp + IsCapabilityEnabled, kDispatch + IsCapabilityEnabled
|
||||||
|
// KEPT. Magma's GenerateMipmap materialises a texture's queued clear before it blits and
|
||||||
|
// PrepareStorageImageTextures does the same for every storage image a dispatch writes;
|
||||||
|
// both go through VkClearManager::PreCompensateSrgbClearColor, which reads
|
||||||
|
// GL_FRAMEBUFFER_SRGB. The P2 contract gave that capability real storage for the first
|
||||||
|
// time, so this row went from reading a compile-time constant to reading real state -
|
||||||
|
// which is the opposite of a row that could be dropped.
|
||||||
|
//
|
||||||
|
// kBlitOrCopy / kTextureOp + the shader blit's viewport and vertex/buffer bindings
|
||||||
|
// (GetViewportIndexed, GetDepthRangeIndexed, GetProvokingVertexMode, GetBufferBindingPoint)
|
||||||
|
// KEPT. TryBlitToDefaultFramebufferWithShader is a real draw of a backend-owned helper
|
||||||
|
// program: ApplyGLViewportState -> ComputeGLViewport reads viewport 0 and its depth range,
|
||||||
|
// GetOrCreateBlitPipeline -> SelectProvokingVertexMode reads the provoking vertex, and
|
||||||
|
// BindProgramUniformBuffers' block resolvers read the frontend binding points. It is taken
|
||||||
|
// when a blit's destination is the default framebuffer and the driver cannot do it
|
||||||
|
// natively - again a driver-shaped decision.
|
||||||
|
//
|
||||||
|
// What WOULD retire a row: the poison build already answers "was this field read at this
|
||||||
|
// verb" exactly (MOBILEGL_PIPE_POISON_OMIT withholds one field's stamp for one verb and a
|
||||||
|
// read of it aborts naming the pair). Turning that into a retirement gate means running the
|
||||||
|
// omission across the full CTS caselist on both devices, not the desktop corpus, and that is
|
||||||
|
// recorded as P3a work rather than done here on evidence that cannot support it.
|
||||||
|
// ---------------------------------------------------------------------------------------
|
||||||
|
//
|
||||||
// gen_pipe.py's block regexes end at a blank line: keep the empty line after each macro.
|
// gen_pipe.py's block regexes end at a blank line: keep the empty line after each macro.
|
||||||
//
|
//
|
||||||
// clang-format off
|
// clang-format off
|
||||||
|
|||||||
@@ -21,8 +21,10 @@ publish points, the ones that must map onto an aggregate generation.
|
|||||||
P0 is the skeleton: it reports. P1 adds the mapping file and CI regenerates it with
|
P0 is the skeleton: it reports. P1 adds the mapping file and CI regenerates it with
|
||||||
`git diff --exit-code` and zero unmapped mutators, the same shape as gen_pipe.py's G6.
|
`git diff --exit-code` and zero unmapped mutators, the same shape as gen_pipe.py's G6.
|
||||||
|
|
||||||
python3 scripts/gen_pipe_dirty_surface.py # human-readable report
|
python3 scripts/gen_pipe_dirty_surface.py # human-readable report
|
||||||
python3 scripts/gen_pipe_dirty_surface.py --summary # counts only
|
python3 scripts/gen_pipe_dirty_surface.py --summary # counts only
|
||||||
|
python3 scripts/gen_pipe_dirty_surface.py --check # THE GATE: rc 1 on any hole
|
||||||
|
python3 scripts/gen_pipe_dirty_surface.py --self-test # the gate's own negative controls
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
@@ -135,14 +137,76 @@ def scan_file(path):
|
|||||||
return findings, all_mutators
|
return findings, all_mutators
|
||||||
|
|
||||||
|
|
||||||
def main():
|
DEF_PATH = os.path.join(REPO_ROOT, "MobileGL", "MG_Pipe", "DirtySurface.def")
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
TRACKER_PATH = os.path.join(REPO_ROOT, "MobileGL", "MG_Impl", "Pipe", "Tracker.h")
|
||||||
parser.add_argument("--summary", action="store_true", help="print the counts only")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
if not os.path.isdir(SCAN_ROOT):
|
ROW_RE = re.compile(r"^[ \t]*X\((\w+),\s*(\w+)\)\s*\\?\s*$", re.M)
|
||||||
sys.exit("missing %s" % SCAN_ROOT)
|
DIRTY_NAME_RE = re.compile(r'^\s*"(NEW_[A-Z0-9_]+)",\s*$', re.M)
|
||||||
|
|
||||||
|
# The answers that are not a dirty-bit name. Each one is documented in DirtySurface.def's
|
||||||
|
# header; a row that uses anything else is a typo, and a typo that read as "mapped" would be
|
||||||
|
# exactly the silent hole this gate exists to close.
|
||||||
|
NON_BIT_ANSWERS = ("kImmediate", "kReverseChannel", "kNoBackendRead", "kExplicitDestroy",
|
||||||
|
"kPulledEveryVerb")
|
||||||
|
|
||||||
|
|
||||||
|
def dirty_bit_names():
|
||||||
|
"""The MGPipeDirty bit names, read out of Tracker.h's kMGPipeDirtyNames so a row cannot
|
||||||
|
name a bit that does not exist and a bit cannot be renamed out from under a row. Read
|
||||||
|
from the RAW text on purpose: the names are string literals, which is exactly what
|
||||||
|
mask_comments_and_strings blanks."""
|
||||||
|
with open(TRACKER_PATH, "r", encoding="utf-8", errors="replace") as handle:
|
||||||
|
return set(DIRTY_NAME_RE.findall(handle.read()))
|
||||||
|
|
||||||
|
|
||||||
|
def load_mapping(text=None):
|
||||||
|
"""{mutator: answer} from DirtySurface.def, or from `text` for the self-test."""
|
||||||
|
if text is None:
|
||||||
|
with open(DEF_PATH, "r", encoding="utf-8", errors="replace") as handle:
|
||||||
|
text = handle.read()
|
||||||
|
rows = {}
|
||||||
|
duplicates = []
|
||||||
|
for match in ROW_RE.finditer(mask_comments_and_strings(text)):
|
||||||
|
mutator, answer = match.group(1), match.group(2)
|
||||||
|
if mutator in rows:
|
||||||
|
duplicates.append(mutator)
|
||||||
|
rows[mutator] = answer
|
||||||
|
return rows, duplicates
|
||||||
|
|
||||||
|
|
||||||
|
def check_mapping(mapping, duplicates, scanned, bits):
|
||||||
|
"""Every problem the gate fails on, as a list of human-readable lines. BOTH directions:
|
||||||
|
an unmapped mutator renders stale, and a row naming a mutator the scan no longer finds is
|
||||||
|
a stale row that would keep a real hole looking covered."""
|
||||||
|
problems = []
|
||||||
|
for mutator in sorted(set(scanned) - set(mapping)):
|
||||||
|
problems.append("UNMAPPED mutator %s - add a row to MG_Pipe/DirtySurface.def" % mutator)
|
||||||
|
for mutator in sorted(set(mapping) - set(scanned)):
|
||||||
|
problems.append("STALE row %s - the scan no longer finds this mutator; delete the row"
|
||||||
|
% mutator)
|
||||||
|
for mutator in sorted(duplicates):
|
||||||
|
problems.append("DUPLICATE row %s" % mutator)
|
||||||
|
for mutator in sorted(mapping):
|
||||||
|
answer = mapping[mutator]
|
||||||
|
if answer in NON_BIT_ANSWERS:
|
||||||
|
continue
|
||||||
|
if answer in bits:
|
||||||
|
continue
|
||||||
|
problems.append("BAD answer %s for %s - not a MGPipeDirty bit name and not one of %s"
|
||||||
|
% (answer, mutator, ", ".join(NON_BIT_ANSWERS)))
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
SELF_TEST_WITHHELD = """
|
||||||
|
#define MGP_DIRTY_SURFACE_LIST(X) \\
|
||||||
|
X(RecordError, kReverseChannel)
|
||||||
|
"""
|
||||||
|
|
||||||
|
SELF_TEST_STALE = None # built from the real def at run time
|
||||||
|
|
||||||
|
|
||||||
|
def scan_all():
|
||||||
|
"""(findings-per-file, {mutator: call count}) over the whole scan root."""
|
||||||
sources = []
|
sources = []
|
||||||
for root, _, files in os.walk(SCAN_ROOT):
|
for root, _, files in os.walk(SCAN_ROOT):
|
||||||
for name in sorted(files):
|
for name in sorted(files):
|
||||||
@@ -150,15 +214,102 @@ def main():
|
|||||||
sources.append(os.path.join(root, name))
|
sources.append(os.path.join(root, name))
|
||||||
sources.sort()
|
sources.sort()
|
||||||
|
|
||||||
total_functions = 0
|
per_file = []
|
||||||
total_mutators = 0
|
|
||||||
deferred_mutators = 0
|
|
||||||
distinct_mutators = {}
|
|
||||||
distinct_all = {}
|
distinct_all = {}
|
||||||
for path in sources:
|
for path in sources:
|
||||||
findings, all_mutators = scan_file(path)
|
findings, all_mutators = scan_file(path)
|
||||||
for mutator, _ in all_mutators:
|
for mutator, _ in all_mutators:
|
||||||
distinct_all[mutator] = distinct_all.get(mutator, 0) + 1
|
distinct_all[mutator] = distinct_all.get(mutator, 0) + 1
|
||||||
|
per_file.append((path, findings, all_mutators))
|
||||||
|
return sources, per_file, distinct_all
|
||||||
|
|
||||||
|
|
||||||
|
def self_test(scanned, bits):
|
||||||
|
"""Canned negative controls. Each MUST trip; trips == 0 is an error, which is the shape
|
||||||
|
check_include_closure.py and gen_pipe.py --self-test already use."""
|
||||||
|
trips = 0
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
# 1. a mutator withheld from the def.
|
||||||
|
mapping, duplicates = load_mapping(SELF_TEST_WITHHELD)
|
||||||
|
problems = check_mapping(mapping, duplicates, scanned, bits)
|
||||||
|
if any(p.startswith("UNMAPPED") for p in problems):
|
||||||
|
trips += 1
|
||||||
|
else:
|
||||||
|
failures.append("negative control 1 (a withheld mutator) did NOT trip")
|
||||||
|
|
||||||
|
# 2. a row naming a mutator the scan does not find.
|
||||||
|
real, real_duplicates = load_mapping()
|
||||||
|
with_ghost = dict(real)
|
||||||
|
with_ghost["SetSomethingThatDoesNotExist"] = "kImmediate"
|
||||||
|
problems = check_mapping(with_ghost, real_duplicates, scanned, bits)
|
||||||
|
if any(p.startswith("STALE") for p in problems):
|
||||||
|
trips += 1
|
||||||
|
else:
|
||||||
|
failures.append("negative control 2 (a stale row) did NOT trip")
|
||||||
|
|
||||||
|
# 3. a row whose answer is neither a dirty bit nor one of the documented non-bit answers.
|
||||||
|
with_typo = dict(real)
|
||||||
|
with_typo["RecordError"] = "NEW_TYPO_THAT_IS_NOT_A_BIT"
|
||||||
|
problems = check_mapping(with_typo, real_duplicates, scanned, bits)
|
||||||
|
if any(p.startswith("BAD answer") for p in problems):
|
||||||
|
trips += 1
|
||||||
|
else:
|
||||||
|
failures.append("negative control 3 (a bad answer) did NOT trip")
|
||||||
|
|
||||||
|
for failure in failures:
|
||||||
|
print("dirty-surface self-test: %s" % failure)
|
||||||
|
if trips == 0:
|
||||||
|
print("dirty-surface self-test: NOTHING tripped - the gate cannot fail, which is worse "
|
||||||
|
"than a red gate")
|
||||||
|
return 1
|
||||||
|
if failures:
|
||||||
|
return 1
|
||||||
|
print("dirty-surface self-test: %d negative controls, all tripped" % trips)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--summary", action="store_true", help="print the counts only")
|
||||||
|
parser.add_argument("--check", action="store_true",
|
||||||
|
help="fail when a scanned mutator has no row in DirtySurface.def, or a "
|
||||||
|
"row names a mutator the scan no longer finds")
|
||||||
|
parser.add_argument("--self-test", action="store_true",
|
||||||
|
help="run the canned negative controls; each must trip")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if not os.path.isdir(SCAN_ROOT):
|
||||||
|
sys.exit("missing %s" % SCAN_ROOT)
|
||||||
|
if not os.path.isfile(DEF_PATH):
|
||||||
|
sys.exit("missing %s" % DEF_PATH)
|
||||||
|
|
||||||
|
sources, per_file, distinct_all = scan_all()
|
||||||
|
bits = dirty_bit_names()
|
||||||
|
if not bits:
|
||||||
|
sys.exit("could not read the MGPipeDirty bit names out of %s" % TRACKER_PATH)
|
||||||
|
|
||||||
|
if args.self_test:
|
||||||
|
return self_test(distinct_all, bits)
|
||||||
|
|
||||||
|
mapping, duplicates = load_mapping()
|
||||||
|
|
||||||
|
if args.check:
|
||||||
|
problems = check_mapping(mapping, duplicates, distinct_all, bits)
|
||||||
|
for problem in problems:
|
||||||
|
print("dirty-surface: %s" % problem)
|
||||||
|
if problems:
|
||||||
|
print("dirty-surface: %d problem(s); the mapping must cover every mutator the scan "
|
||||||
|
"finds, in both directions" % len(problems))
|
||||||
|
return 1
|
||||||
|
print("dirty-surface: %d mutators, all mapped, no stale rows" % len(mapping))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
total_functions = 0
|
||||||
|
total_mutators = 0
|
||||||
|
deferred_mutators = 0
|
||||||
|
distinct_mutators = {}
|
||||||
|
for path, findings, all_mutators in per_file:
|
||||||
deferred_mutators += len(all_mutators)
|
deferred_mutators += len(all_mutators)
|
||||||
if not findings:
|
if not findings:
|
||||||
continue
|
continue
|
||||||
@@ -175,7 +326,7 @@ def main():
|
|||||||
print(" %s (line %d) -> backend: %s" % (finding["function"], finding["line"],
|
print(" %s (line %d) -> backend: %s" % (finding["function"], finding["line"],
|
||||||
", ".join(finding["backend"][:4])))
|
", ".join(finding["backend"][:4])))
|
||||||
for mutator, line in finding["mutators"]:
|
for mutator, line in finding["mutators"]:
|
||||||
print(" %-44s :%d UNMAPPED" % (mutator, line))
|
print(" %-44s :%d %s" % (mutator, line, mapping.get(mutator, "UNMAPPED")))
|
||||||
|
|
||||||
print("\ndirty-surface: %d files scanned under MG_Impl/GLImpl" % len(sources))
|
print("\ndirty-surface: %d files scanned under MG_Impl/GLImpl" % len(sources))
|
||||||
print("dirty-surface: %d mutator calls in total, %d distinct mutators" % (deferred_mutators,
|
print("dirty-surface: %d mutator calls in total, %d distinct mutators" % (deferred_mutators,
|
||||||
@@ -186,12 +337,17 @@ def main():
|
|||||||
print("dirty-surface: the remaining %d are DEFERRED: nothing reaches the backend in the same "
|
print("dirty-surface: the remaining %d are DEFERRED: nothing reaches the backend in the same "
|
||||||
"function, so the next verb publishes them, and each one needs an aggregate generation"
|
"function, so the next verb publishes them, and each one needs an aggregate generation"
|
||||||
% (deferred_mutators - total_mutators))
|
% (deferred_mutators - total_mutators))
|
||||||
print("dirty-surface: distinct mutators, by call count")
|
print("dirty-surface: distinct mutators, by call count, with what publishes each")
|
||||||
for mutator in sorted(distinct_all, key=lambda k: (-distinct_all[k], k)):
|
for mutator in sorted(distinct_all, key=lambda k: (-distinct_all[k], k)):
|
||||||
print(" %5d %s%s" % (distinct_all[mutator], mutator,
|
print(" %5d %-42s %s%s" % (distinct_all[mutator], mutator,
|
||||||
" (immediate)" if mutator in distinct_mutators else ""))
|
mapping.get(mutator, "UNMAPPED"),
|
||||||
print("dirty-surface: every mutator above is UNMAPPED - the aggregate-generation mapping file "
|
" (immediate)" if mutator in distinct_mutators else ""))
|
||||||
"lands in P1, and this report is what it has to cover.")
|
unmapped = sorted(set(distinct_all) - set(mapping))
|
||||||
|
if unmapped:
|
||||||
|
print("dirty-surface: %d UNMAPPED - run --check, which is a gate since P2" % len(unmapped))
|
||||||
|
else:
|
||||||
|
print("dirty-surface: every mutator above is mapped (MG_Pipe/DirtySurface.def); --check "
|
||||||
|
"is a gate and --self-test proves it can fail")
|
||||||
print("dirty-surface: known limits of this scanner - it matches braced function bodies "
|
print("dirty-surface: known limits of this scanner - it matches braced function bodies "
|
||||||
"textually, so a mutator inside a lambda is attributed to the enclosing function, and a "
|
"textually, so a mutator inside a lambda is attributed to the enclosing function, and a "
|
||||||
"mutation published through a helper the entry point calls reads as deferred here.")
|
"mutation published through a helper the entry point calls reads as deferred here.")
|
||||||
|
|||||||
Reference in New Issue
Block a user