docs(perf,benchmark-infra): authorize realtime audio benchmark + advisory CI (SysRS-307..309 / SysDes-156..158 / SRS-216..219 / SAD-088..091 / SDD-120)

Author the full SysRS -> SysDes -> SRS -> SAD -> SDD chain for the benchmark infrastructure authorized by the Option B product decision (Dimensions 1 + 2-advisory; Dimension 3 telemetry export deferred to P1; build-failing hard CI gate deferred until baseline maturity).

SysRS v0.9.10 adds:
- SysRS-307: maintained numeric performance baselines for the realtime audio path (allocations per callback after warmup, callback wall-clock, Opus encode/decode latency, resampler throughput).
- SysRS-308: advisory CI regression reporting on PR + merge to default; non-blocking semantics.
- SysRS-309: explicit declared tolerance window.

SysDes v0.9.8 adds:
- SysDes-156: benchmark coverage allocated to SE-13 (Audio Subsystem).
- SysDes-157: CI advisory-reporting integration allocated to SE-18 (Deployment).
- SysDes-158: per-metric tolerance table (zero / +20% p95 / +15% mean / -10% throughput).

SRS v0.9.9 adds:
- SRS-216: realtime audio benchmark instrumentation in chanora_audio/benches/.
- SRS-217: baseline storage format (JSON with metric/value/unit/host_arch/toolchain/git_sha/timestamp).
- SRS-218: CI advisory workflow with non-blocking semantics.
- SRS-219: tolerance window binding + merge-base comparison methodology.

SAD v0.9.9 adds:
- SAD-088: chanora_audio criterion bench harness (extends SAD-034).
- SAD-089: baseline JSON path pinned to crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json.
- SAD-090: advisory CI workflow file (.github/workflows/bench-advisory.yml).
- SAD-091: manual-trigger baseline-update workflow (sole writer of SAD-089).
- Yellow-marker semantics pinned at SAD: 50%-of-tolerance trending detection.

SDD v0.9.15 adds:
- SDD-120: criterion 0.5 + dhat 0.3 dev-deps; three bench files (realtime_capture, opus_codec, resampler) + common.rs; two post-processor binaries (emit_baseline, compare_baseline); two GitHub Actions workflow YAMLs; simpler-form yellow-marker realization (baseline-only comparator).

Out of scope (deferred):
- Dimension 3 production telemetry export (P1).
- Build-failing hard CI gate (post-baseline-maturity).
- Multi-host benchmarking (Linux x86_64 only).
- Dart-side flutter_rust_bridge round-trip benchmark.

Implementation follows in a separate commit per the no-huge-commit guideline.
This commit is contained in:
EdisonJwa
2026-05-18 13:36:01 +08:00
parent d13b56d379
commit 575a6cbc5c
5 changed files with 309 additions and 4 deletions
+71 -1
View File
@@ -3,7 +3,7 @@
**Product name:** Chanora
**Document type:** SRS / Software Requirements Specification
**ASPICE process alignment:** SWE.1 Software Requirements Analysis
**Version:** 0.9.8
**Version:** 0.9.9
**Status:** Baseline Candidate
**Source baseline:** `chanora_SysDes_ASPICE_SYS3_v0.5.md`
**Architecture baseline:** Flutter + Rust Core + `tsclientlib` protocol adapter
@@ -2743,6 +2743,75 @@ This subsection groups the Android voice-audio-backend software requirements der
| SysDes-135 (Android platform baseline) | Retained as the secondary/context anchor on `SRS-208` through `SRS-215` (the eight Android voice-audio SRS items conceptually rest on the Android platform baseline as their platform context, but no longer take SysDes-135 as the primary derivation). Upstream SysRS items addressed across the eight items are SysRS-055, SysRS-217, SysRS-288, SysRS-305, and SysRS-306. |
## 20. Realtime Audio Benchmark Infrastructure (P0)
This section authors the software-requirements layer of the realtime-audio benchmark infrastructure derived strictly from the SysDes-156 / SysDes-157 / SysDes-158 chain (per SRS-005 strict-layered sourcing). The SysDes anchors are themselves a refinement of SysRS-307 / SysRS-308 / SysRS-309, but the SRS layer does not source upstream of SysDes by policy.
**SRS-216**: The `chanora_audio` software shall expose its realtime capture and playback paths to benchmark instrumentation that covers the five metrics enumerated by SysDes-156: (1) heap allocation count per realtime callback after a warmup window of approximately 100 callbacks (the steady-state expectation is zero allocations on the realtime thread); (2) per-callback wall-clock duration reported at the p50, p95, and p99 percentiles, expressed as a fraction of the active cpal stream period; (3) Opus encode latency measured on the canonical 960-sample / 20 ms voice frame shape; (4) Opus decode latency measured on the same 960-sample / 20 ms frame shape; and (5) resampler throughput measured at the canonical rate pairs 44.1 → 48 kHz, 16 → 48 kHz, and 48 → 48 kHz (the last serving as a passthrough control). The instrumentation shall be authored as Rust benchmarks located in a per-crate `benches/` directory (the choice of benchmark harness crate — `criterion` or equivalent — is a downstream SDD concern and is not authored here). The instrumentation shall consume the existing realtime-path public surface (for example `CaptureState::ingest`, already public at the chanora_audio crate boundary per the Wave 4 Tier A1 work) without introducing new public seams in production code; if any additional seam is required, that decision is delegated to SAD/SDD and is not authored at the SRS layer.
- Status: Baseline Candidate
- Type: Verification Infrastructure Requirement
- Stage: P0 / MVP
- Allocated to: `chanora_audio` crate (realtime capture and playback modules, plus a co-located `benches/` directory)
- Source SysDes: SysDes-156 (realtime-audio benchmark surface allocated to SE-13)
- Verification method: SWE.4 — automated benchmark execution against the instrumentation surface; reviewers confirm the five metric families enumerated in SysDes-156 each have at least one benchmark target.
- Acceptance criteria: A `benches/` directory exists under the `chanora_audio` crate (or the SDD-decided per-crate equivalent); the directory contains benchmark targets that exercise the realtime capture callback, the realtime playback callback, the Opus encoder on a 960-sample / 20 ms frame, the Opus decoder on a 960-sample / 20 ms frame, and the resampler at each of the three canonical rate pairs; each benchmark target reports a numeric value in the unit appropriate to its metric family (count, fraction-of-period, ns/iter, or samples/sec); the benchmarks run to completion on a Linux x86_64 host in under the wall-clock budget allowed by SysDes-157's host scope (see SRS-218).
- Analysis: Feasible. The realtime path already exposes the entry points required for benchmark instrumentation per the chanora_audio crate boundary as of Wave 4 Tier A1; no production-code seam authoring is required at the SRS layer. Verification intent (SWE.6): demonstrate that all five metric families have at least one benchmark target and that the benchmarks execute deterministically on the Linux x86_64 host enumerated by SysDes-157.
- Unresolved assumptions: The exact benchmark harness crate (`criterion` is the conventional choice on the Rust ecosystem but is not authored here) and the exact target-naming convention are SDD concerns. If the realtime path requires a new public seam to be benchmarkable at SRS granularity, that gap will be discovered by the SAD/SDD pass and routed back to a future SRS revision; this clause does not authorise such a seam in production code.
**SRS-217**: Baseline measurements produced by the SRS-216 benchmark instrumentation shall be stored as structured JSON committed to a deterministic location inside the repository, so that the baseline becomes a reviewable, version-controlled artefact rather than an ephemeral CI side effect. Each baseline record shall contain at minimum the following fields: `metric` (string identifier matching the metric name enumerated by SysDes-156), `value` (numeric — the measured value for that metric), `unit` (string — for example `"ns/iter"`, `"samples/sec"`, `"fraction"`, or `"count"`, matching the metric family), `host_arch` (string — the Rust target triple of the measurement host, for example `"x86_64-unknown-linux-gnu"`), `toolchain` (string — the rustc version active at measurement time), `git_sha` (string — the full Git commit SHA at measurement time), and `timestamp` (string — ISO-8601 UTC). The canonical on-disk path for the baseline JSON (for example `benches/baselines/<crate>.json` as a suggestion) is delegated to SDD and is not fixed at the SRS layer; however, the path shall be deterministic (the same crate at the same commit shall write to the same file) and shall be committed to the repository on the default branch.
- Status: Baseline Candidate
- Type: Verification Infrastructure Requirement
- Stage: P0 / MVP
- Allocated to: `chanora_audio` crate (the bench harness publishes the baseline JSON), plus the SDD-decided on-disk path under the repository root.
- Source SysDes: SysDes-156 (baseline storage as structured JSON in a deterministic repo location capturing metric value, host architecture, toolchain version, git SHA, and timestamp).
- Verification method: Review — the JSON record format is review-checkable against this clause by inspecting a sample baseline file.
- Acceptance criteria: A baseline JSON file exists at the SDD-decided deterministic path on the default branch; the file is parseable as JSON; each record carries the seven fields enumerated above; the `metric` field values match the metric names declared by SysDes-156; the `host_arch`, `toolchain`, `git_sha`, and `timestamp` fields are populated with values that correspond to the measurement host and commit; the same crate at the same commit deterministically writes to the same file path.
- Analysis: Feasible. JSON is the conventional serialization format for benchmark-baseline metadata in Rust ecosystem tooling, and the fields enumerated are the minimum needed for the SysDes-157 advisory comparison to be reproducible. Verification intent (SWE.6): inspect a sample baseline file and confirm the seven fields are populated correctly and the file is parseable.
- Unresolved assumptions: The exact on-disk path is delegated to SDD per SysDes-156. The schema may evolve (additional optional fields) without breaking this clause provided the seven enumerated fields remain present.
**SRS-218**: A continuous-integration workflow shall execute the SRS-216 benchmark suite under the constraints authored by SysDes-157, namely: (1) the workflow shall trigger on every pull request opened against the default branch and on every merge to the default branch, and shall not trigger on tag pushes or on pushes to non-default branches; (2) the workflow shall execute on the Linux x86_64 GitHub Actions runner (`ubuntu-latest`) and shall not execute on any other host architecture (multi-host benchmarking is out of scope for this revision per SysDes-157 point 2); (3) the workflow result shall be rendered as a markdown table on the pull-request surface that compares each metric enumerated by SysDes-156 against the baseline-at-merge-base (per SRS-219), with a per-metric visual marker drawn from the set { 🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance } evaluated against the per-metric tolerance window of SysDes-158; (4) the workflow shall report a `success` status check regardless of whether any metric is flagged 🔴, so that the advisory check never blocks merge (this enforces the SysDes-157 point 4 non-blocking semantics at the SRS layer); and (5) the baseline-update operation shall be a separate, manually-invoked workflow (GitHub Actions `workflow_dispatch` or equivalent), and the PR-triggered workflow authored by this clause shall never write to the baseline JSON file authored under SRS-217.
- Status: Baseline Candidate
- Type: Verification Infrastructure Requirement
- Stage: P0 / MVP
- Allocated to: CI tooling under `.github/workflows/` (or the SDD-decided equivalent integration point) — specifically a new advisory benchmark workflow file; the exact filename is delegated to SAD/SDD.
- Source SysDes: SysDes-157 (advisory CI integration allocated to SE-18, with the six design constraints on trigger, host, surface, non-blocking semantics, baseline source, and baseline-update isolation).
- Verification method: Demo — a synthetic pull-request run shall demonstrate the workflow firing, the markdown comparison table rendering, the status check reporting `success` even when a 🔴 marker is present, and the absence of any write to the baseline JSON file under the PR-triggered path. Human reviewers confirm in the PR.
- Acceptance criteria: A CI workflow file exists at the SDD-decided path; the file's trigger configuration matches SysDes-157 point 1 (PR-against-default plus merge-to-default, no tags, no non-default-branch pushes); the workflow's `runs-on` clause is `ubuntu-latest`; a PR run posts a markdown table that lists each SysDes-156 metric with its measured value, the baseline value at the PR's merge-base, the percentage delta, and one of the three visual markers; the workflow's status check reports `success` on every run, including runs that contain at least one 🔴 marker; a separate `workflow_dispatch` workflow file exists for the baseline-update operation, and the PR-triggered workflow does not modify the baseline JSON.
- Analysis: Feasible on GitHub Actions. The advisory-only, never-block-on-failure semantics is the active rule per SysRS-308 (inherited via SysDes-157); escalating this workflow to a hard build-failing gate would require a future SysRS authorisation and is not within the scope of SRS-218. Verification intent (SWE.6): observe a synthetic PR run end-to-end and confirm the five constraints (trigger, host, surface, non-blocking semantics, baseline-update isolation).
- Unresolved assumptions: The exact workflow filename, the exact named status-check string, and the exact markdown table layout are SAD/SDD concerns. The triggering criterion for the 🟡 "trending" state is delegated to SDD per SysDes-157's notes (this clause only requires that the marker set { 🟢, 🟡, 🔴 } is present, not how 🟡 is computed).
**SRS-219**: The advisory comparison performed by the SRS-218 workflow shall apply the per-metric tolerance window and the comparison methodology authored by SysDes-158. The tolerance window shall be evaluated per metric as follows: (a) heap allocation count per realtime callback after warmup — tolerance is zero, i.e. any non-zero allocation count on a PR is flagged 🔴; (b) per-callback wall-clock duration at p95 — tolerance is +20% relative to the baseline (p50 and p99 are advisory data only and do not drive the 🔴 marker); (c) Opus encode latency on a 960-sample / 20 ms frame — tolerance is +15% relative to the baseline on the mean; (d) Opus decode latency on a 960-sample / 20 ms frame — tolerance is +15% relative to the baseline on the mean; (e) resampler throughput on the canonical rate pairs — tolerance is 10% relative to the baseline on samples-per-second (i.e. a throughput drop greater than 10% is flagged 🔴). The comparator on a pull-request run shall be the baseline JSON committed at the pull request's merge-base commit, **not** the baseline at the current default-branch tip; this disambiguates the comparison when a PR is rebased onto a newer default and prevents a default-branch baseline update from retroactively re-classifying an in-flight PR's regression status.
- Status: Baseline Candidate
- Type: Verification Infrastructure Requirement
- Stage: P0 / MVP
- Allocated to: CI tooling co-located with the SRS-218 workflow (the comparison logic resides in the same SE-18 surface).
- Source SysDes: SysDes-158 (per-metric tolerance window and merge-base-snapshot comparison methodology allocated to SE-18).
- Verification method: Review — the comparison logic and the tolerance table are review-checkable against this clause; a small synthetic data set (one baseline, one PR run for each of the five metric families with values at, just inside, and just outside the tolerance window) demonstrates each marker assignment.
- Acceptance criteria: The CI tooling reads the baseline JSON at the PR's merge-base commit (verifiable by observing the SHA the tooling resolves before reading the file); the comparison applies the per-metric thresholds enumerated above (heap = zero, p95 wall-clock = +20%, Opus encode mean = +15%, Opus decode mean = +15%, resampler samples/sec = 10%); the marker assignment for a synthetic value at the threshold, just inside the threshold, and just outside the threshold matches the expected 🟢 / 🟡 (where defined by SDD) / 🔴 result; rebasing a PR onto a newer default branch re-resolves the merge-base and re-reads the baseline at the new merge-base, rather than reading the default-branch-tip baseline.
- Analysis: Feasible. The merge-base resolution is a standard `git merge-base` operation; per-metric thresholds are simple numeric comparisons. The merge-base-snapshot semantics is the SysDes-158 active rule and is preferred over default-branch-tip comparison because it makes the comparator stable across rebases. Verification intent (SWE.6): observe each of the five metric families' threshold behaviour on synthetic data and confirm the merge-base SHA is the comparator anchor.
- Unresolved assumptions: None at the SRS layer. The exact representation of the 🟡 "trending" intermediate state (for example a sub-threshold band, or a multi-run trend) is delegated to SDD per SysDes-157's notes, but does not alter the 🔴 threshold rules authored here.
### 20.1 SRS-216..219 to SysDes Coverage
| SysDes anchor | SRS coverage |
|---|---|
| SysDes-156 (realtime-audio benchmark surface, SE-13; five metric families; baseline JSON in deterministic repo location with metric/value/unit/host_arch/toolchain/git_sha/timestamp fields) | Primary anchor for `SRS-216` (benchmark instrumentation covering the five metric families) and `SRS-217` (baseline JSON storage format). |
| SysDes-157 (advisory CI integration, SE-18; trigger on PR + merge-to-default; host = `ubuntu-latest`; markdown table with 🟢/🟡/🔴 markers; never-fail status check; baseline-update isolation via `workflow_dispatch`) | Primary anchor for `SRS-218` (CI advisory-reporting workflow). |
| SysDes-158 (per-metric tolerance window and merge-base-snapshot comparison methodology, SE-18; heap = 0, p95 wall-clock = +20%, Opus encode/decode mean = +15%, resampler samples/sec = 10%) | Primary anchor for `SRS-219` (tolerance window and comparison methodology binding). |
### 20.2 Intentionally Out of Scope at This Revision
Consistent with the SysRS-307 / SysRS-308 / SysRS-309 deferrals propagated through SysDes-156 / SysDes-157 / SysDes-158, this revision intentionally does **not** author the following SRS clauses, and they shall not be inferred from SRS-216 through SRS-219:
- **Dimension 3 production telemetry export** — the runtime-export-of-metrics-from-shipped-clients dimension is deferred to P1 by SysRS, is not authorised by any SysDes-156/157/158 clause, and remains subject to SysRS-295's no-off-device-transmission rule. No SRS clause in this revision authorises such export.
- **Build-failing hard CI gate** — SysRS-308's advisory-only contract is the active rule, enforced at the SysDes layer by SysDes-157's non-blocking semantics. SRS-218 explicitly inherits that contract by requiring the status check to report `success` on every run. Escalation to a hard gate requires a future SysRS authorisation.
- **Multi-host benchmarking** — SysDes-157 point 2 fixes the host to Linux x86_64 (`ubuntu-latest`). ARM Android, macOS Apple Silicon, Windows x86_64, and any other host runner are explicitly deferred. SRS-218 inherits this constraint.
## Baseline Candidate 0.9.1 Update
| Version | Date | Description |
@@ -2775,6 +2844,7 @@ This subsection groups the Android voice-audio-backend software requirements der
| Version | Date | Description |
|---|---|---|
| 0.9.9 | 2026-05-18 | Added realtime-audio benchmark-infrastructure software requirements SRS-216 through SRS-219 in a new section 20, sourced strictly from SysDes-156 / SysDes-157 / SysDes-158 (the Option B benchmark-infrastructure scope decision propagated through the Wave 1.5 benchmark chain). SRS-216 allocates benchmark instrumentation covering the five SysDes-156 metric families (heap allocation count per realtime callback after ~100-callback warmup, per-callback wall-clock at p50/p95/p99 as a fraction of cpal stream period, Opus encode latency on 960-sample/20 ms frame, Opus decode latency on the same shape, and resampler throughput at 44.1→48 / 16→48 / 48→48 kHz) to the `chanora_audio` crate's `benches/` directory; no new public seam in production code is authored. SRS-217 authors the baseline-JSON storage format (`metric`, `value`, `unit`, `host_arch`, `toolchain`, `git_sha`, `timestamp` fields at minimum) committed to a deterministic SDD-decided repo path. SRS-218 authors the CI advisory-reporting workflow under SysDes-157 (trigger on PR-against-default + merge-to-default, host = `ubuntu-latest`, markdown table with 🟢/🟡/🔴 markers, never-fail status check, baseline-update isolated to a separate `workflow_dispatch` workflow). SRS-219 binds the per-metric tolerance window of SysDes-158 (heap = 0, p95 wall-clock = +20%, Opus encode/decode mean = +15%, resampler samples/sec = 10%) and the merge-base-snapshot comparison methodology. Intentionally NOT authored consistent with the SysRS deferral chain: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-295 still applies); (b) build-failing hard CI gate (SysRS-308 advisory-only rule active, SysDes-157 non-blocking semantics inherited); (c) multi-host benchmarking (SysDes-157 Linux x86_64 host-scope clause active). Strict layered sourcing preserved (`SRS -> SysDes` only per SRS-005); SysRS-307/308/309 are not cited as direct SRS sources. |
| 0.9.8 | 2026-05-18 | Retargeted the `Source SysDes` field on SRS-208, SRS-209, and SRS-210 through SRS-215 from the generic SysDes-135 (Android platform baseline) to the dedicated Android voice-audio derivation items authored by the system-architecture owner: SysDes-152 (Android in-call audio mode subsystem) is now the primary anchor for SRS-208; SysDes-153 (Android RECORD_AUDIO runtime permission acquisition flow as Permission/Voice subsystem partitioning) is now the primary anchor for SRS-209; SysDes-154 (Android voice audio backend subsystem — AAudio low-latency path, voice-communication usage/preset, hardware-effects engagement with software fallback, sharing-mode policy, FGS-hosted background mic lifecycle) is now the primary anchor for SRS-210 (latency), SRS-211 (input preset), SRS-212 (hardware AEC/NS/AGC), SRS-213 (output usage/content-type), SRS-214 (sharing mode), and SRS-215 (microphone-typed foreground service). SRS-213 additionally retains SysDes-152 as a secondary anchor because the output-stream usage is the precondition for in-call-mode routing. SysDes-135 is retained on all eight items as the secondary/context anchor (platform-baseline relationship). Section 19 coverage table updated. Strict layered sourcing preserved (`SRS -> SysDes` only). No technical or behavioural content of SRS-208 through SRS-215 was modified; this is a surgical retargeting of the `Source SysDes` field only. |
| 0.9.7 | 2026-05-18 | Added Android Audio Backend (P0) section 18 hosting SRS-210 through SRS-215 to support the platform-adapter decision to host Android voice capture/playback on a dedicated low-latency native audio backend (mirroring the iOS voice-processing-audio-unit precedent). SRS-210 sets numeric mouth-to-ear latency targets (≤150 ms on devices that return `AAUDIO_PERFORMANCE_MODE_LOW_LATENCY`, ≤250 ms otherwise). SRS-211 mandates `AAUDIO_INPUT_PRESET_VOICE_COMMUNICATION` with `VOICE_PERFORMANCE` fallback. SRS-212 mandates engagement of hardware `AcousticEchoCanceler` / `NoiseSuppressor` / `AutomaticGainControl` on the capture session id with software-AEC fallback (closes the prior silent no-op). SRS-213 mandates `AAUDIO_USAGE_VOICE_COMMUNICATION` + `AAUDIO_CONTENT_TYPE_SPEECH` on output. SRS-214 mandates best-effort `AAUDIO_SHARING_MODE_EXCLUSIVE` with `SHARED` fallback. SRS-215 mandates microphone-typed foreground-service hosting (`foregroundServiceType="microphone"`) for backgrounded voice transmission. Added section 19 coverage statement. Strict SRS→SysDes sourcing preserved: `Source SysDes` traces via SysDes-135 (Android platform baseline) pending a SysDes revision that introduces a dedicated derivation item for the Android voice audio backend; SysRS-055, SysRS-217, SysRS-288, SysRS-305 are cited as the upstream SysRS obligations addressed. |
| 0.9.6 | 2026-05-17 | Propagated SysRS reconciliation (Wave 1.5) into SRS for P0 Android. SRS-187 minimum Android API raised from 24 to 28 per DEC-004 (ID preserved, change record added). SRS-188 verified consistent with SysRS-289 (Google Play upload-date target SDK); no rewrite required. SRS-111 (Android foreground service for active voice session), SRS-119 (Android AAB release build), and SRS-163 (Android back-navigation intent) reviewed and left unchanged — text remains unambiguous and P0/P1 scope is intact. Added SRS-208 (Android in-call audio mode engaged on voice-session connect, released on last disconnect — addresses SysRS-305) and SRS-209 (explicit `RECORD_AUDIO` runtime permission acquisition at or before voice-session activation, fail-safe to listen-only on denial — addresses SysRS-306). Strict layered sourcing preserved (`SRS -> SysDes` only); `Source SysDes` for SRS-208/209 traces via SysDes-135 pending a SysDes revision that introduces dedicated derivation items for SysRS-305/306. |
+23 -1
View File
@@ -2,7 +2,7 @@
**Product name:** Chanora
**Document type:** SysRS / System Requirements Specification
**Version:** 0.9.2
**Version:** 0.9.10
**Status:** Baseline Candidate
**Product category:** Cross-platform voice client application
**Architecture:** Flutter + Rust Core
@@ -1944,6 +1944,21 @@ This section converts the baseline product decisions into auditable system-level
- Priority: P0
- Verification: Platform Test (Android), Functional Test
**SysRS-307**: The Chanora project shall maintain measured numeric performance baselines for the realtime audio capture and playback paths. The baseline set shall include, at minimum: (i) heap allocation count per realtime audio callback measured after warmup, (ii) per-callback wall-clock time expressed as a fraction of (or absolute bound relative to) the audio frame period, (iii) Opus encode latency and Opus decode latency per frame, and (iv) resampler throughput at the common rate-pair conversions exercised by the audio pipeline. Each baseline shall be expressed as a numeric threshold (or numeric range), not as free-form prose, so that regression against the baseline is deterministically detectable by automated comparison. This requirement extends the prescriptive intent of SysRS-180 ("shall minimize local audio pipeline latency") and the directional target of SysRS-181 ("should target local audio pipeline latency under 100 ms") into a measurable, contract-grade obligation, and it is consistent with the underrun-avoidance obligation in SysRS-186. The specific baseline values, the warmup definition, the rate-pair set, and the storage format for the baselines are SysDes/SAD/SDD concerns and are not authored here. This clause does not authorize automatic upload, transmission, or off-device export of any measured baseline data and is therefore consistent with SysRS-295 (no automatic telemetry / diagnostic upload in MVP).
- Priority: P0
- Verification: Test (SWE.4 benchmark assertions, delegated to verification layer; cross-references SysRS-236)
**SysRS-308**: The Chanora project's continuous-integration workflow shall execute the realtime-audio benchmark suite on every pull request against the default branch and on every merge to the default branch, on at least one host architecture (Linux x86_64 on the existing GitHub Actions runner is sufficient to satisfy this clause), and shall report the comparison of the executed run against the maintained baselines (SysRS-307) in the pull request's status-check surface such that human reviewers can see, before approving the change, whether any baseline has regressed beyond the declared tolerance window (SysRS-309). The CI regression check authorized by this clause is **advisory only**: it shall NOT fail the CI build, shall NOT block merge, and shall NOT be treated as a hard quality gate at the P0-MVP stage. Its purpose is to surface evidence for human reviewer judgement and to avoid the failure mode in which a strict build-failing gate is bypassed under release-crunch pressure (e.g., "skip CI"). Escalation of this advisory tier to a build-failing hard gate is explicitly out of scope of SysRS-308 and shall be authorized only by a separate, later SysRS clause after a period of baseline maturity sufficient to establish that the advisory signal is stable and low-noise (indicative target: 46 weeks of clean baseline data on the default branch, but the actual escalation criteria are to be set by the future clause). This requirement is consistent with the CI-as-quality-evidence pattern established by the SysRS-234..239 verification family and does not displace any obligation in that family. This clause does not authorize off-device transmission of baseline measurements beyond the project's existing CI provider surface (i.e., GitHub Actions logs and PR status checks visible to repository collaborators); it is therefore consistent with SysRS-295.
- Priority: P0
- Verification: Demo (CI workflow exercised on a representative PR; advisory status check appears and does not block merge on regression)
**SysRS-309**: The advisory CI regression comparison authorized by SysRS-308 shall use an explicitly declared tolerance window — a numeric percentage (or a numeric per-metric percentage set) above the maintained baseline (SysRS-307) — beyond which the advisory check shall mark the run as a regression. The tolerance shall be a single declared value (or one declared value per metric), not an ad-hoc reviewer judgement, so that the advisory signal is reproducible. The specific numeric value(s) of the tolerance window and the comparison methodology (for example, "comparison against the most recent baseline snapshot on the default branch") shall be authored at the SysDes layer and may be refined at SAD/SDD; a suggested starting value of +20% over baseline is recorded here for downstream traceability but is not binding at the SysRS layer. Cross-references: SysRS-307 (the baselines being compared against), SysRS-308 (the advisory CI surface in which the comparison runs), SysRS-180/SysRS-181 (latency intent the tolerance must not silently erode), SysRS-295 (the tolerance window value is a project-local configuration and does not authorize off-device telemetry).
- Priority: P0
- Verification: Review (SysDes/SAD declaration of tolerance value is present and is referenced by the CI workflow definition)
## 32. Change History Addendum
@@ -1977,3 +1992,10 @@ This section converts the baseline product decisions into auditable system-level
| Version | Date | Description |
|---|---|---|
| 0.9.9 | 2026-05-17 | Reconciled SysRS-288 with DEC-004 (Accepted 2026-05-14): Android minimum runtime baseline raised from API 24 to API 28 (Android 9.0); rationale and decision citation added in-line. Verified SysRS-055, SysRS-161, SysRS-195, SysRS-217, and SysRS-289 remain consistent with DEC-004 (no API-level text in any of these; no rewrite required). Added SysRS-305 (Android in-call audio mode engagement during active voice session — back-fills the v0.9.8 product-decision-register `AudioManager.setMode(MODE_IN_COMMUNICATION)` entry) and SysRS-306 (explicit Android runtime microphone permission acquisition at or before voice session activation, additive to SysRS-055). |
## Baseline Candidate 0.9.10 Update
| Version | Date | Description |
|---|---|---|
| 0.9.10 | 2026-05-18 | Authorized Option B of the benchmark-infrastructure decision for the realtime audio path. Added SysRS-307 (maintained numeric performance baselines for the realtime audio capture and playback paths — heap allocation count per callback after warmup, per-callback wall-clock budget relative to the audio frame period, Opus encode/decode latency, and resampler throughput at common rate-pair conversions; extends SysRS-180/SysRS-181 from prescriptive intent into a measurable contract; consistent with SysRS-186 and SysRS-236). Added SysRS-308 (advisory CI regression reporting executing the benchmark suite on every PR and every merge to the default branch on at least one host architecture, surfacing results in the PR status-check view; explicitly **advisory only** — does not fail the build, does not block merge; escalation to a build-failing hard gate is out of scope and deferred to a future SysRS clause after baseline maturity; consistent with the SysRS-234..239 verification-family pattern). Added SysRS-309 (explicitly declared numeric tolerance window for the advisory comparison; numeric value(s) and comparison methodology delegated to SysDes/SAD; suggested starting value +20% recorded as non-binding downstream guidance). All three new clauses are consistent with SysRS-295 (no automatic telemetry / diagnostic upload in MVP) — they neither authorize nor require off-device transmission of measurement data beyond the existing CI provider surface visible to repository collaborators. Explicitly **NOT** authored in this update: (a) Dimension 3 production telemetry export of timing histograms (deferred to P1; any future opt-in performance-evidence export through the user-initiated diagnostic-export path requires a separate P1 SysRS clause and must be reconciled with SysRS-295 at that time); (b) build-failing hard CI gate (deferred to a future SysRS clause). |