docs(perf,benchmark-infra): authorize realtime audio benchmark + advisory CI (SysRS-307..309 / SysDes-156..158 / SRS-216..219 / SAD-088..091 / SDD-120)

Author the full SysRS -> SysDes -> SRS -> SAD -> SDD chain for the benchmark infrastructure authorized by the Option B product decision (Dimensions 1 + 2-advisory; Dimension 3 telemetry export deferred to P1; build-failing hard CI gate deferred until baseline maturity).

SysRS v0.9.10 adds:
- SysRS-307: maintained numeric performance baselines for the realtime audio path (allocations per callback after warmup, callback wall-clock, Opus encode/decode latency, resampler throughput).
- SysRS-308: advisory CI regression reporting on PR + merge to default; non-blocking semantics.
- SysRS-309: explicit declared tolerance window.

SysDes v0.9.8 adds:
- SysDes-156: benchmark coverage allocated to SE-13 (Audio Subsystem).
- SysDes-157: CI advisory-reporting integration allocated to SE-18 (Deployment).
- SysDes-158: per-metric tolerance table (zero / +20% p95 / +15% mean / -10% throughput).

SRS v0.9.9 adds:
- SRS-216: realtime audio benchmark instrumentation in chanora_audio/benches/.
- SRS-217: baseline storage format (JSON with metric/value/unit/host_arch/toolchain/git_sha/timestamp).
- SRS-218: CI advisory workflow with non-blocking semantics.
- SRS-219: tolerance window binding + merge-base comparison methodology.

SAD v0.9.9 adds:
- SAD-088: chanora_audio criterion bench harness (extends SAD-034).
- SAD-089: baseline JSON path pinned to crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json.
- SAD-090: advisory CI workflow file (.github/workflows/bench-advisory.yml).
- SAD-091: manual-trigger baseline-update workflow (sole writer of SAD-089).
- Yellow-marker semantics pinned at SAD: 50%-of-tolerance trending detection.

SDD v0.9.15 adds:
- SDD-120: criterion 0.5 + dhat 0.3 dev-deps; three bench files (realtime_capture, opus_codec, resampler) + common.rs; two post-processor binaries (emit_baseline, compare_baseline); two GitHub Actions workflow YAMLs; simpler-form yellow-marker realization (baseline-only comparator).

Out of scope (deferred):
- Dimension 3 production telemetry export (P1).
- Build-failing hard CI gate (post-baseline-maturity).
- Multi-host benchmarking (Linux x86_64 only).
- Dart-side flutter_rust_bridge round-trip benchmark.

Implementation follows in a separate commit per the no-huge-commit guideline.
This commit is contained in:
EdisonJwa
2026-05-18 13:36:01 +08:00
parent d13b56d379
commit 575a6cbc5c
5 changed files with 309 additions and 4 deletions
+49
View File
@@ -1434,6 +1434,50 @@ A single SRS item may be intentionally allocated to more than one SAD item when
- Source SysDes: SysDes-155 (macOS runtime baseline — deployment target, universal-binary packaging, `.framework` `Versions/A` layout, CocoaPods podspec automation)
- Verification method: Platform Test (macOS), Release Inspection, Architecture Review
**SAD-088**: The software architecture shall add a `criterion`-based realtime-audio benchmark harness as a first-class architectural element of the `chanora_audio` crate, extending the existing audio-subsystem allocation in SAD-034. The harness lives in `crates/chanora_audio/benches/` as a Cargo-convention `benches/` directory (a sibling of the crate's `src/`, not a public seam of the crate's library API). The harness shall use the `criterion` crate as its measurement framework (selected at this layer; SDD pins the exact version and feature flags). Rationale for `criterion`: it is the de-facto stable-toolchain Rust microbenchmark harness, produces stable statistical output (mean / p50 / p95 / p99) without requiring a nightly compiler, and integrates with `cargo bench` so the same invocation works in developer shells and on CI runners. The harness shall expose one bench function (or one criterion `BenchmarkGroup`) per SysDes-156 metric family, with the following architectural assignment of metric → bench element: (a) `bench_capture_alloc_count` — counts heap allocations on the steady-state `CaptureState::ingest` path after a warmup phase, using a per-bench allocation-counting mechanism (a dev-dependency allocation tracker such as `dhat`, or a custom global-allocator counter — selection deferred to SDD); (b) `bench_capture_callback_wall_clock` — measures wall-clock duration of `CaptureState::ingest` and reports p50 / p95 / p99 from criterion's sample distribution; (c) `bench_opus_encode_latency` — directly times an Opus encoder over a fixed 960-sample / 20 ms input frame; (d) `bench_opus_decode_latency` — directly times the Opus decode path; if the upstream `tsclientlib` `AudioHandler` is opaque to the crate, the bench shall time the smallest decode wrapper the crate owns and the SDD shall name that wrapper; (e) `bench_resampler_throughput` — measures samples-per-second on the three canonical rate pairs (44.1→48, 16→48, 48→48 kHz). The harness shall emit, for each bench function, the SRS-217 baseline JSON record (one record per metric, in the SRS-217 schema); the emission mechanism (a custom `criterion` measurement reporter, a post-bench JSON-aggregation step, or equivalent) is delegated to SDD. The harness shall not introduce any new public seam on the `chanora_audio` crate's library API surface — benches consume the same internal entry points that the realtime callback consumes, so SAD-034's isolation property is preserved.
- Status: Baseline Candidate
- Type: Software Architecture Item
- Stage: P0 / MVP
- Allocated to: Audio Subsystem (`chanora_audio` crate, `benches/` directory; extends SAD-034)
- Source SRS: SRS-216, SRS-217
- Source SysDes: SysDes-156 (realtime-audio benchmark surface, SE-13; five metric families; baseline-JSON storage)
- Verification method: Inspection (bench-function presence and metric coverage), Bench Run (criterion executes on the host runner), Architecture Review
**SAD-089**: The software architecture shall pin the baseline-JSON on-disk path for the SAD-088 harness to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`, a single deterministic file per host architecture under the crate's `benches/` directory. The filename is keyed on Rust target triple (`x86_64-unknown-linux-gnu`) so future multi-host expansion can add sibling files (`aarch64-apple-darwin.json`, `x86_64-pc-windows-msvc.json`, etc.) without restructuring the directory layout; for the present revision, exactly one such file exists because SysDes-157 fixes the host to Linux x86_64 (`ubuntu-latest`). The file shall be committed to the default branch so that the SAD-090 advisory workflow has a deterministic comparison target at every PR's merge-base commit. The file contents shall conform to the SRS-217 record schema (one record per metric, fields `metric`, `value`, `unit`, `host_arch`, `toolchain`, `git_sha`, `timestamp` at minimum). The file is the sole on-disk output sink of the SAD-088 harness; it is written by exactly one workflow (SAD-091) and read by exactly one workflow (SAD-090) — see SAD-090 / SAD-091 for the workflow isolation property. Multi-host baselines (additional sibling files keyed on alternate target triples) are explicitly out of scope for this revision per SysDes-157 point 2.
- Status: Baseline Candidate
- Type: Software Architecture Item
- Stage: P0 / MVP
- Allocated to: Audio Subsystem (`chanora_audio` crate, `benches/baselines/` subdirectory; output sink of SAD-088)
- Source SRS: SRS-217
- Source SysDes: SysDes-156 (baseline-JSON storage; deterministic repo location)
- Verification method: Inspection (file path, schema), Bench Run (file is regenerable end-to-end)
**SAD-090**: The software architecture shall add an advisory CI workflow file `.github/workflows/bench-advisory.yml` to the build/release/operations allocation (extending SAD-037), realizing the SRS-218 advisory-reporting workflow. Triggers: `pull_request` against the default branch, plus `push` to the default branch (matching SRS-218 clause 1; tag pushes and non-default branch pushes are excluded). Runner: `ubuntu-latest` (the sole host in scope per SysDes-157 point 2). Step shape (high-level; SDD authors the exact YAML): (1) checkout the PR head; (2) install the Rust toolchain pinned by the repo's `rust-toolchain.toml`; (3) for each bench function defined under SAD-088, invoke `cargo bench` on the `chanora_audio` crate; (4) read the baseline JSON at SAD-089's path *as of the PR's merge-base commit* (using `git show <merge-base>:crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` or an equivalent git-plumbing read) — never the default-branch-tip baseline, per SRS-219's anti-retroactive-reclassification rule; (5) compare current measurements against that merge-base baseline using the SRS-219 per-metric tolerance window; (6) render the comparison as a markdown table with one row per metric and a 🟢 / 🟡 / 🔴 marker per the SAD-determined semantics (see below); (7) post the table as a PR comment via `actions/github-script@v7` or an equivalent comment-posting action (exact action choice delegated to SDD); (8) the workflow's status check shall always exit `success` (exit code 0) regardless of any 🔴 marker, per SRS-218 clause 4 — regression visibility is delivered through the markdown table, not through the check status. The named status check shall follow existing repo workflow-naming convention (e.g. `bench-advisory`; SDD reads existing workflow files and matches the convention). This workflow shall never write to the baseline JSON file authored under SAD-089 — write authority is exclusively SAD-091's, enforcing the SRS-218 clause 5 workflow-isolation property at the architecture layer. Yellow-marker semantics is pinned at this SAD layer (see §24.1 paragraph below). The workflow is allocated to the CI surface (SE-18) and is the realization of the SE-18 anchor previously named only at the SysDes layer (no prior SAD item exists for the SE-18 surface; this is the first such allocation).
- Status: Baseline Candidate
- Type: Software Architecture Item
- Stage: P0 / MVP
- Allocated to: Build, Release, Operations (`.github/workflows/bench-advisory.yml`; extends SAD-037 — CI surface, SE-18 realization)
- Source SRS: SRS-218, SRS-219 (tolerance binding consumed by comparator step)
- Source SysDes: SysDes-157 (advisory CI integration), SysDes-158 (tolerance window and merge-base-snapshot comparator)
- Verification method: Inspection (workflow trigger / runner / non-blocking exit), CI Dry Run (synthetic PR exercises markdown rendering end-to-end), Architecture Review
**SAD-091**: The software architecture shall add a separate baseline-update CI workflow file `.github/workflows/bench-baseline-update.yml` to the build/release/operations allocation (extending SAD-037), realizing the SRS-218 clause 5 baseline-update-isolation property. Trigger: `workflow_dispatch` only — the workflow is manually invoked from the GitHub Actions tab on the default branch and shall not fire on `pull_request`, `push`, or any tag event. Runner: `ubuntu-latest` (matching SAD-090's host for byte-comparability of measurements). Step shape: (1) checkout the default branch; (2) install the Rust toolchain pinned by `rust-toolchain.toml`; (3) execute the same `cargo bench` invocations as SAD-090 across the SAD-088 bench functions; (4) write the resulting JSON, conformant to SRS-217's schema, to the SAD-089 path `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (overwriting the prior file); (5) commit and push the updated file to the default branch — the commit mechanism (a direct push using a GitHub-bot token, the `peter-evans/create-pull-request` action, or an equivalent push action) is delegated to SDD. This workflow is the **only** writer of the SAD-089 baseline JSON. The SAD-090 PR-triggered workflow is strictly a reader. This single-writer property is the architectural realization of SRS-218 clause 5 and shall be invariant under future revisions (any future revision adding a second writer requires a SAD revision, not an SDD revision).
- Status: Baseline Candidate
- Type: Software Architecture Item
- Stage: P0 / MVP
- Allocated to: Build, Release, Operations (`.github/workflows/bench-baseline-update.yml`; extends SAD-037 — CI surface, SE-18 realization; sole writer of SAD-089)
- Source SRS: SRS-218 (workflow-isolation clause)
- Source SysDes: SysDes-157 (baseline-update isolation via `workflow_dispatch`)
- Verification method: Inspection (trigger is `workflow_dispatch` only; commit destination is SAD-089's path), CI Dry Run (manual invocation regenerates baseline end-to-end), Architecture Review
### Yellow-marker semantics (pinned at SAD)
Pinned at SAD layer to remove a downstream ambiguity that would otherwise surface as an open question in the SAD-090 SDD unit: the 🟡 "within tolerance but trending" marker (SRS-218 clause 3, third element of the marker set) shall be defined as follows: a metric is rendered 🟡 if and only if (a) the metric is within the SRS-219 per-metric tolerance window relative to the merge-base baseline (i.e. it is not 🔴), AND (b) the absolute delta between the PR's current measurement and the **previous run of the SAD-090 workflow on the default branch** (i.e. the most recent default-branch CI bench result, not the baseline JSON itself) is greater than 50% of the SRS-219 tolerance window for that metric. Metrics whose SRS-219 tolerance is exactly zero (the heap-allocation-count metric, SRS-219 clause a) cannot be 🟡 because the 50% trigger is undefined when the window has zero width; for those metrics only 🟢 (zero allocations) and 🔴 (any non-zero allocation count) apply. Rationale for pinning at SAD rather than deferring to SDD: the question "is yellow relative to baseline or relative to previous-run-on-default" is an architectural choice about what state the workflow consumes (baseline JSON only, vs baseline JSON + a previous-runs cache), not an implementation detail; pinning it here fixes the workflow's required inputs at the architecture layer so SDD has a fully determined input set. Operationally, the "previous-run-on-default" datum may be sourced from the SAD-091 baseline (treating the committed baseline as the latest default-branch run, which is the simpler and recommended SDD realization) or from a separate run-history artifact; SDD selects which.
## 27. Updated SRS-to-SAD Coverage Matrix
| SRS Range | SAD Coverage |
@@ -1447,6 +1491,10 @@ A single SRS item may be intentionally allocated to more than one SAD item when
| SRS-209 | Covered by `SAD-085` |
| SRS-111 (Android foreground voice service) | Covered by `SAD-086` (Android-specific allocation; SAD-036 retains the cross-cutting diagnostics allocation for the SRS-111124 range per §24.1 dual-allocation pattern) |
| SRS-013 (multi-platform support — macOS runtime baseline aspect) | Covered by `SAD-087` (macOS runtime baseline; counterpart to SAD-061 / SAD-063) |
| SRS-216 (realtime-audio benchmark instrumentation) | Covered by `SAD-088` (criterion-based bench harness in `chanora_audio/benches/`; extends SAD-034 audio-subsystem allocation). Cross-cutting note: this is a cross-cutting allocation per §24.1 in the sense that the harness sits in the Audio Subsystem (SAD-034 extension) but its CI consumers sit in Build/Release/Operations (SAD-037 extensions); there is no platform-specific variation because SysDes-157 fixes the host to Linux x86_64. |
| SRS-217 (baseline-JSON storage format and on-disk path) | Covered by `SAD-088` (the harness emits the JSON) and `SAD-089` (the on-disk path is pinned to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`) |
| SRS-218 (CI advisory workflow + baseline-update isolation) | Covered by `SAD-090` (`.github/workflows/bench-advisory.yml`, advisory reader, non-blocking exit) and `SAD-091` (`.github/workflows/bench-baseline-update.yml`, `workflow_dispatch`, sole writer of SAD-089); both extend SAD-037 build/release/operations allocation |
| SRS-219 (per-metric tolerance window and merge-base-snapshot comparison methodology) | Covered by `SAD-090` (comparator step consumes the SRS-219 tolerance and the merge-base-snapshot rule; tolerance-window evaluation is the workflow's comparator-step responsibility) |
## Baseline Candidate 0.9.1 Update
@@ -1497,3 +1545,4 @@ A single SRS item may be intentionally allocated to more than one SAD item when
|---|---|---|
| 0.9.7 | 2026-05-18 | Formalized the cross-cutting + platform-specific SAD allocation pattern flagged by the traceability-auditor: added §24.1 narrative subsection describing the deliberate dual-allocation rule (cross-cutting concern + platform-specific realization both recorded), with SRS-111 / SAD-036 / SAD-086 as the worked example, and annotated the §24.2 matrix row for SRS-111..124 accordingly. Added SAD-087 (macOS runtime baseline: deployment target isolation, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, hand-rolled CocoaPods podspec automation), parallel to SAD-061 (iOS runtime) and SAD-063 (Android runtime), closing the macOS-runtime anchor gap flagged by the SDD-119 detailed-designer. SAD-087 traces upward through SRS-013 (multi-platform support); a parallel macOS-runtime SysDes derivation does not yet exist and is recorded as an open follow-up for the system-architect (SysDes-133 covers iOS, SysDes-135 covers Android, macOS has no counterpart yet). §27 matrix updated with the SAD-087 row. Strict layered sourcing preserved (`SAD -> SRS` only). |
| 0.9.8 | 2026-05-18 | Retargeted SAD-087's `Source SysDes` field from the "open follow-up — no dedicated macOS-runtime SysDes anchor exists" placeholder to `SysDes-155` (macOS runtime baseline — deployment target, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, CocoaPods podspec automation), authored by the system-architect in SysDes v0.9.7 as the parallel macOS-runtime anchor to SysDes-133 (iOS) / SysDes-135 (Android). This closes the open follow-up that SAD-087 carried at its 0.9.7 authoring, giving SAD-087 a proper upward SysDes trace. Surgical single-field edit: no behavioral content of SAD-087 changed; `Source SRS` retained as `SRS-013` (multi-platform support — Windows, macOS, Linux, iOS, Android), which remains the correct SRS anchor (the SRS-013 vs alternate-anchor question raised during SysDes-155 authoring referred to SysRS-013 / SysRS-002 at the system-requirements layer, not to SRS at the software-requirements layer). §27 SRS-to-SAD coverage matrix row for SRS-013 unchanged because it already cites SAD-087 and remains textually accurate. Strict layered sourcing preserved (`SAD -> SRS`, with SysDes cross-reference now resolved). |
| 0.9.9 | 2026-05-18 | Added realtime-audio benchmark-infrastructure software-architecture items SAD-088 through SAD-091 sourced from SRS-216 / SRS-217 / SRS-218 / SRS-219 (the Option B benchmark-infrastructure scope decision propagated through the Wave 1.5 benchmark chain SysRS-307/308/309 → SysDes-156/157/158 → SRS-216..219). SAD-088 allocates a `criterion`-based bench harness to the `chanora_audio` crate's `benches/` directory, extending SAD-034 (audio subsystem allocation); the harness covers the five SysDes-156 metric families (heap allocation count, per-callback wall-clock at p50/p95/p99, Opus encode latency on 960-sample/20 ms frame, Opus decode latency on the same shape, resampler throughput at 44.1→48 / 16→48 / 48→48 kHz) and emits SRS-217 baseline records. SAD-089 pins the baseline-JSON on-disk path to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (single deterministic file, single host architecture in scope, target-triple-keyed filename to permit future multi-host expansion without directory restructuring). SAD-090 adds the advisory CI workflow `.github/workflows/bench-advisory.yml` extending SAD-037 (build/release/operations allocation): triggers on PR-against-default + push-to-default, runs on `ubuntu-latest`, reads the merge-base baseline (never the default-tip baseline, per SRS-219 anti-retroactive-reclassification), compares using SRS-219 tolerances, posts a markdown table with 🟢/🟡/🔴 markers, and always exits success (non-blocking per SRS-218 clause 4). SAD-091 adds the separate baseline-update workflow `.github/workflows/bench-baseline-update.yml` extending SAD-037: `workflow_dispatch` trigger only, sole writer of the SAD-089 baseline file, realizing the SRS-218 clause 5 workflow-isolation property at the architecture layer (single-writer invariant). Yellow-marker semantics pinned at this SAD layer in a §26 subsection: 🟡 means "within SRS-219 tolerance relative to merge-base baseline AND delta-from-previous-default-branch-run exceeds 50% of the tolerance window"; metrics with zero-width tolerance (heap allocations) cannot be 🟡. Rationale for pinning at SAD: the choice between baseline-only vs baseline-plus-run-history as workflow input is an architectural input-set decision, not an implementation detail. §27 SRS-to-SAD coverage matrix updated with four new rows (SRS-216, SRS-217, SRS-218, SRS-219) citing SAD-088..091; the SRS-216 row also annotates the cross-cutting allocation pattern per §24.1 (harness sits in Audio Subsystem via SAD-034 extension while CI consumers sit in Build/Release/Operations via SAD-037 extensions, with no platform-specific variation because SysDes-157 fixes the host to Linux x86_64). Intentionally NOT authored consistent with the SysRS/SysDes/SRS deferral chain: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-295 active); (b) build-failing hard CI gate (SysRS-308 advisory-only rule active; SAD-090 status-check always exits success); (c) multi-host benchmarking (SysDes-157 host-scope clause active; SAD-089 holds a single target-triple-keyed file); (d) IDE integration (e.g. `cargo bench` invocation from rust-analyzer in VS Code — tooling polish, not P0 architecture). Strict layered sourcing preserved (`SAD -> SRS` only per the SAD §2 rule, with SysDes cross-references recorded for completeness on each new item). |
+113
View File
@@ -1703,6 +1703,112 @@ Notes:
- This is a back-fill SDD. No new implementation work is created by SDD-119; the code already exists and ships. A builder agent does NOT need to act on SDD-119. The doc-drift defect in item 4 and the deployment-target single-source-of-truth improvement in item 3 are recorded as open follow-ups and are not in scope for this unit.
- SAD-side anchoring of the build-automation pattern at the architectural layer is NOT performed by this SDD unit; a software-architect dispatch is required if the orchestrator wants a SAD-layer build-automation item.
**SDD-120 — Realtime audio benchmark harness and advisory CI infrastructure**
- Source SAD: SAD-088 (criterion-based bench harness allocated to `chanora_audio` `benches/` directory, extending SAD-034 audio-subsystem allocation), SAD-089 (baseline-JSON on-disk path pinned to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`), SAD-090 (advisory CI workflow `.github/workflows/bench-advisory.yml` — PR-triggered, non-blocking, comment-posting reader of SAD-089), SAD-091 (baseline-update CI workflow `.github/workflows/bench-baseline-update.yml` — `workflow_dispatch`-only, sole writer of SAD-089).
- Allocated to: Audio Subsystem (`crates/chanora_audio/benches/`) + Build/Release/Operations (`.github/workflows/bench-advisory.yml`, `.github/workflows/bench-baseline-update.yml`).
- Software units (new): `crates/chanora_audio/benches/realtime_capture.rs`, `crates/chanora_audio/benches/opus_codec.rs`, `crates/chanora_audio/benches/resampler.rs`, `crates/chanora_audio/benches/common.rs` (shared synthetic-input helper module), `crates/chanora_audio/benches/emit_baseline.rs` (post-bench JSON aggregator binary), `crates/chanora_audio/benches/compare_baseline.rs` (PR-vs-baseline comparator binary), `crates/chanora_audio/Cargo.toml` (dev-dependency additions and `[[bench]]` entries), `.github/workflows/bench-advisory.yml`, `.github/workflows/bench-baseline-update.yml`, `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (seeded SAD-089 baseline file).
- Cross-trace: SDD-094 (the `chanora_audio::engine::AudioEngine` capture path that the §3 bench seam exercises is the same audio engine whose `ensure_running` / `shutdown_if_idle` lifecycle is specified by SDD-094; the harness measures the steady-state callback that runs while `ensure_running` has opened the cpal input stream, so the bench seam is part of the engine's verification surface). No cross-trace to SDD-076..SDD-080 / SDD-118: the bench targets the realtime audio path and not the PTT capability surface; the Android-bridge build automation is unrelated to the Linux-host bench workflow.
§1 — Crate selection and version pin.
1. Add `criterion = "0.5"` to `crates/chanora_audio/Cargo.toml` `[dev-dependencies]`. The `0.5` series is the current stable major and is compatible with the workspace `rust-version = "1.95"` pin. Default features are kept; `html_reports` is left at its default (criterion only emits HTML when invoked with `--save-baseline`/`--load-baseline` against the local user, and CI consumes the per-bench `estimates.json` files directly from `target/criterion/`, so an HTML feature toggle is not load-bearing for the workflow). Rationale: criterion is the de-facto statistical benchmark harness for Rust; it emits a stable JSON estimates file under `target/criterion/<bench>/<group>/estimates.json` that the §5 post-processor parses.
2. Add `dhat = "0.3"` to `crates/chanora_audio/Cargo.toml` `[dev-dependencies]`. `dhat` is used inside `bench_capture_alloc_count` to install a `#[global_allocator] static ALLOC: dhat::Alloc = dhat::Alloc;` profiler at bench-binary scope and to compare `dhat::HeapStats::get().total_blocks` (or the equivalent v0.3 API name) before and after the measurement window. Rationale: `dhat` is the de-facto heap profiler for Rust, has zero-cost when disabled, and is the lowest-friction way to obtain a steady-state heap-allocation count for the SRS-219 clause-a zero-allocation invariant (which carries `tolerance = 0`).
3. Add three `[[bench]]` entries to `crates/chanora_audio/Cargo.toml`, one per bench file enumerated in §2. Each entry shall set `harness = false` only if the bench file calls `criterion::criterion_main!` directly (criterion's `main!` macro replaces the cargo test harness); the §3 signatures use `criterion_main!` so `harness = false` is required.
4. Add two `[[bin]]` entries pointing at `benches/emit_baseline.rs` and `benches/compare_baseline.rs` so that `cargo run --bin emit_baseline` and `cargo run --bin compare_baseline` resolve correctly. The §6 / §7 workflow steps invoke these binaries by name; placing the source files under `benches/` keeps the entire benchmark surface in one directory but requires the `path = "benches/<name>.rs"` field on each `[[bin]]` entry.
§2 — Bench file layout.
1. `crates/chanora_audio/benches/realtime_capture.rs` — declares `bench_capture_alloc_count` and `bench_capture_callback_wall_clock`. Uses `criterion_group!(realtime_capture, bench_capture_alloc_count, bench_capture_callback_wall_clock);` and `criterion_main!(realtime_capture);`.
2. `crates/chanora_audio/benches/opus_codec.rs` — declares `bench_opus_encode_latency` and `bench_opus_decode_latency`. Uses `criterion_group!(opus_codec, bench_opus_encode_latency, bench_opus_decode_latency);` and `criterion_main!(opus_codec);`.
3. `crates/chanora_audio/benches/resampler.rs` — declares `bench_resampler_throughput`. Uses `criterion_group!(resampler, bench_resampler_throughput);` and `criterion_main!(resampler);`.
4. `crates/chanora_audio/benches/common.rs` — shared synthetic-input helpers per §4. Included from each bench file with `mod common;` (criterion bench files compile as standalone binaries, so `common.rs` is included by relative module declaration in each `realtime_capture.rs` / `opus_codec.rs` / `resampler.rs`; do NOT add it as a fourth `[[bench]]` entry).
§3 — Bench function signatures and warmup phase.
1. `fn bench_capture_alloc_count(c: &mut Criterion)` — the bench installs a `dhat::Profiler::builder().testing().build()` once at the start of the function, runs the §4 `synthetic_capture_buffer(960, 1)` through `CaptureState::ingest(&buf)` 100 times as a warm-up (this primes any first-call allocations from the cpal capture-state machine — ring-buffer growth, resampler-state init, opus-encoder-handle init via the existing engine code path), then samples `dhat::HeapStats::get()` to capture the `total_blocks` count `B_warm`, runs 1000 additional `ingest` calls inside `c.bench_function("capture_alloc_count", |b| b.iter(|| ingest_one()))`, samples `dhat::HeapStats::get()` again to obtain `B_final`, and the criterion-reported metric value shall be `(B_final - B_warm)` (the post-warmup allocation count). The function additionally `assert_eq!(B_final - B_warm, 0)` so a regression also fails the bench process at the local-developer surface; the CI advisory comparator in `compare_baseline.rs` carries the SRS-219 clause-a `tolerance = 0` zero-allocation rule independently. Bench seam: the function exercises the `CaptureState::ingest<T>(&mut self, buf: &[T])` signature located at `crates/chanora_audio/src/engine.rs:1207` per the §“State” reading; the bench obtains a `CaptureState` instance by calling the engine's existing test-mode constructor (the same path `tests/ptt_privacy.rs` uses today). No new public API surface is created on `CaptureState`.
2. `fn bench_capture_callback_wall_clock(c: &mut Criterion)` — uses criterion's default `c.bench_function("capture_callback_wall_clock", |b| b.iter(|| capture_state.ingest(black_box(&buf))));` over the same `synthetic_capture_buffer(960, 1)` input. Criterion's internal warmup (default 3 s, configurable but not overridden here) precedes the measurement window; the harness emits `mean`, `median`, and the 95 % and 99 % bootstrap-confidence-interval percentiles into `target/criterion/capture_callback_wall_clock/.../estimates.json`. The §5 post-processor extracts p50 (median), p95, and p99 from that file and writes the three values into the SRS-217 record's `value` triplet.
3. `fn bench_opus_encode_latency(c: &mut Criterion)` — constructs an `audiopus::coder::Encoder::new(SampleRate::Hz48000, Channels::Mono, Application::Voip)` once outside the measurement loop (per criterion best practice; encoder construction is dominated by libopus state allocation and is not part of the per-frame latency), pre-builds a `synthetic_opus_frame()` PCM buffer (exactly `FRAME_SAMPLES = 960` samples per SAD-088 / SysDes-156), and measures `encoder.encode_float(&pcm, &mut opus_out)` per iteration. The bench reports p50/p95/p99 as in item 2.
4. `fn bench_opus_decode_latency(c: &mut Criterion)` — same pattern as item 3 but with `audiopus::coder::Decoder::decode_float(&opus_bytes, &mut pcm_out, false)`. The SDD explicitly pins the bench target to the `audiopus` direct decoder call rather than `tsclientlib::AudioHandler::fill_buffer`, because `AudioHandler::fill_buffer` performs jitter-buffer dequeue + Opus decode + PCM mix in a single call and the wall-clock measurement would conflate three distinct concerns. The canonical SRS-217 metric `opus_decode_latency` therefore refers to the codec-level decode of a single 20 ms / 960-sample frame, not the full AudioHandler call. (Future revision may add a separate `audio_handler_fill_buffer_latency` metric covering the composite call; that metric is out of scope at this SDD revision.)
5. `fn bench_resampler_throughput(c: &mut Criterion)` — opens one `criterion::BenchmarkGroup` named `resampler_throughput` and registers three sub-benches: `c.bench_function("44100_to_48000", ...)`, `c.bench_function("16000_to_48000", ...)`, and `c.bench_function("48000_passthrough", ...)`. Each sub-bench feeds a 1-second synthetic-sine input (length is 44100 / 16000 / 48000 frames respectively) through the existing engine's resampler entry point and reports `samples_per_second = input_len / measured_duration_secs`. The bench uses `group.throughput(Throughput::Elements(input_len as u64))` so criterion's report exposes the throughput estimate directly in `estimates.json`.
§4 — Synthetic input generation.
1. `crates/chanora_audio/benches/common.rs` exposes two functions, both deterministic (no RNG, no time-of-day input): `pub fn synthetic_capture_buffer(frames: usize, channels: usize) -> Vec<f32>` and `pub fn synthetic_opus_frame() -> Vec<f32>` (the latter is the special case `synthetic_capture_buffer(960, 1)`).
2. Sample generation: a 440 Hz sine at amplitude 0.5 over the 48 kHz playback rate (matching the audio engine's mixer rate per SAD-034). For frame index `n` the sample value is `0.5 * (2.0 * PI * 440.0 * (n as f32) / 48_000.0).sin()`. For multi-channel buffers the same scalar value is replicated across all channels at the same frame index (interleaved layout, matching cpal's `Stream` data layout). No randomness, no fade-in, no DC offset — the same input produces the same bytes across runs and the same bytes across hosts.
3. Determinism is load-bearing for §5 baseline stability: if the synthetic input drifts between runs the criterion `estimates.json` would also drift and the comparator would emit false-positive 🔴 markers. The §5 emit-baseline binary and the §6 comparator both assume byte-stable input.
§5 — JSON post-processing.
1. After each bench file runs, criterion writes `target/criterion/<bench-name>/<group-name>/estimates.json` per benchmark group. The §5 binary `crates/chanora_audio/benches/emit_baseline.rs` reads the five expected estimates files (one per §3 bench function, with the resampler bench contributing three sub-bench estimate files that are aggregated into the single SRS-217 `resampler_throughput` record by averaging the three throughput estimates into a per-mode triplet, NOT a single scalar — the SRS-217 schema carries a per-mode record).
2. The binary parses each `estimates.json` (a stable criterion-emitted JSON shape: `mean`, `median`, `std_dev`, `confidence_interval`) and projects each metric into an SRS-217 record with fields `{ metric: <name>, value: <f64 or triplet>, unit: <string>, tolerance_pct: <f64 from SRS-219>, host: "x86_64-unknown-linux-gnu", timestamp_utc: <RFC3339> }`. The full output is a single JSON document with one top-level array `metrics`.
3. The binary writes the document to `./current.json` in the workspace root (not the SAD-089 path — that path is the comparator's input on advisory runs and the writer's output target on baseline-update runs; §6 step 7 reads SAD-089 from git, §7 step 5 writes SAD-089 by copy).
4. Rationale for the Rust-binary approach over a `tools/bench-emit.sh` script: same toolchain as the bench files (no separate `jq` / `python` / `bash` dependency on the runner), same `serde_json` already pulled by the workspace, and the binary can share types with `compare_baseline.rs` (both crates use the same SRS-217 struct definitions).
§6 — `.github/workflows/bench-advisory.yml` structure (SAD-090 realization).
1. Triggers: `pull_request: { types: [opened, synchronize, reopened], branches: [main] }` and `push: { branches: [main] }`. (The default branch is `main` per the existing `.github/workflows/ci.yml` convention; if the repository default branch ever renames, both YAML files in this SDD unit must update in lockstep.)
2. Job name: `bench-advisory`. `runs-on: ubuntu-latest`. `permissions: { pull-requests: write, contents: read }` — `pull-requests: write` is required by step 9 (PR-comment posting); `contents: read` is the minimum for `actions/checkout`.
3. Step 1: `actions/checkout@v4` with `fetch-depth: 0`. Full history is required because step 6 resolves the PR merge-base via `git merge-base origin/main HEAD` and step 7 reads the baseline file from that merge-base via `git show`.
4. Step 2: install Rust toolchain via `dtolnay/rust-toolchain@stable` (matching the existing `.github/workflows/ci.yml` pattern). The workspace `rust-version = "1.95"` floor is satisfied by `@stable` at the time of authoring; if a future workspace pin moves above the public stable channel, both this file and `ci.yml` must update.
5. Step 3: `Swatinem/rust-cache@v2` (matching existing `.github/workflows/ci.yml`). Reuses the bench target directory across runs.
6. Step 4: install the same system audio deps `ci.yml` installs (`libasound2-dev libpulse-dev pkg-config libopus-dev`). Required because the `chanora_audio` crate links `libopus` and the cpal Linux backend pulls ALSA headers at build time.
7. Step 5: `cargo bench --bench realtime_capture --bench opus_codec --bench resampler`. (`cargo bench` without `--benches` would build doctests; the explicit `--bench <name>` list bypasses that and matches the three `[[bench]]` entries from §1 item 3.) No `-- --output-format json` flag is required because §5 reads `target/criterion/.../estimates.json` files directly; criterion writes those unconditionally.
8. Step 6: `cargo run --bin emit_baseline` → writes `./current.json` to the workspace root.
9. Step 7: `MERGE_BASE=$(git merge-base origin/main HEAD)` and `git show $MERGE_BASE:crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json > baseline.json`. If the merge-base predates the introduction of SAD-089 (i.e. the file did not exist at that commit), `git show` exits non-zero; the step shall handle the missing-baseline case by writing an empty `baseline.json` placeholder and emitting a single advisory comment `Baseline not yet established at merge-base; first run on default branch will establish.` instead of running the comparator. This is the only branch where the workflow short-circuits before the comparator step. (The `push: branches: [main]` trigger never hits this short-circuit after the first baseline lands.)
10. Step 8: `cargo run --bin compare_baseline -- --current ./current.json --baseline ./baseline.json --output ./report.md`. The binary applies the SRS-219 per-metric tolerance table and the §8 marker logic, then writes a markdown table to `./report.md`. The binary shall always exit 0 (per SRS-218 clause 4); regression detection is rendered as a 🔴 marker inside the comment, not as a non-zero exit code.
11. Step 9: post the markdown as a PR comment via `actions/github-script@v7`, gated on `if: github.event_name == 'pull_request'` (the `push` trigger has no PR to comment on; for `push` runs the workflow still writes `./report.md` as a workflow artifact via `actions/upload-artifact@v4` so the default-branch trend is inspectable from the Actions tab). Comment body is `body: require('fs').readFileSync('report.md', 'utf8')`. The script shall also de-duplicate prior comments by the same `github-actions[bot]` identifier carrying a sentinel marker line (e.g. `<!-- chanora-bench-advisory -->`) at the head of the body — on each run the existing comment is updated in place rather than appended, so a long-lived PR does not accumulate one comment per push. The de-duplication query uses `octokit.rest.issues.listComments` filtered on `user.login === 'github-actions[bot]'` and body-startsWith the sentinel.
12. Step 10: `if: always()` and a final `exit 0` — no step in the job sets `continue-on-error: false` against a regression-detecting predicate, so the SRS-218 clause 4 non-blocking property is realized by absence of a failing step rather than by an explicit override.
13. Status-check name registered with the branch-protection ruleset (if any): `bench-advisory`. The SDD does not mandate adding this to the required-checks list; SRS-218 clause 4 explicitly prohibits making it required.
§7 — `.github/workflows/bench-baseline-update.yml` structure (SAD-091 realization).
1. Trigger: `workflow_dispatch: {}` only. No `pull_request`, no `push`, no `schedule`, no tag triggers. SAD-091 is invariant on this property.
2. Job name: `bench-baseline-update`. `runs-on: ubuntu-latest` (byte-comparable to SAD-090 per SAD-091). `permissions: { contents: write, pull-requests: write }` — write on both because step 6 opens a PR via `peter-evans/create-pull-request@v6`.
3. Step 1: `actions/checkout@v4` with default ref (no `fetch-depth: 0` needed; the workflow does not resolve merge-bases).
4. Step 2: install Rust toolchain (same shape as §6 step 2).
5. Step 3: install audio system deps (same shape as §6 step 4).
6. Step 4: `cargo bench --bench realtime_capture --bench opus_codec --bench resampler` (same shape as §6 step 5).
7. Step 5: `cargo run --bin emit_baseline` (same shape as §6 step 6).
8. Step 6: `cp ./current.json crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`.
9. Step 7: `peter-evans/create-pull-request@v6` with `title: "chore(bench): update baselines on $(date -u +%Y-%m-%d)"`, `branch: bench/baseline-update-$(date -u +%Y%m%d)`, `commit-message: "chore(bench): refresh x86_64-unknown-linux-gnu baseline"`, `body: <generated diff summary>`, and `add-paths: crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`. Rationale for opening a PR rather than direct-pushing to `main`: baseline updates should be reviewable in case of accidental triggering (a contributor clicks the dispatch button by mistake), runner pollution (an unrelated process on the GitHub-hosted runner skewed the measurement), or a legitimate regression that should not silently move the floor. The PR is then merged by a reviewer using the normal branch-protection flow; this preserves the SAD-091 invariant that this workflow is the **sole writer** of the SAD-089 file (the PR-merge commit is also written by this workflow's PR, not by a human directly editing the JSON).
§8 — Yellow-marker realization (simpler-form selection per SAD-090).
1. Per SAD-090's yellow-marker pin, the SDD selects the simpler operationalization: the committed SAD-089 baseline JSON IS treated as the most recent default-branch run, and is the comparator for both the green/red tolerance evaluation AND the yellow trending evaluation. No separate run-history artifact is maintained.
2. `compare_baseline.rs` logic — for each metric `m` in the SRS-217 record set:
- Let `c = current.metrics[m].value`, `b = baseline.metrics[m].value`, `t = SRS-219 tolerance for m`.
- If `m` is a zero-tolerance metric (SRS-219 clause a — `capture_alloc_count`): emit 🔴 if `c != 0`, 🟢 if `c == 0`. Yellow does not apply.
- Otherwise (`t > 0`): compute `delta_pct = (c - b) / b` when `b > 0`, or `delta_pct = 0.0` when `b == 0` (a degenerate baseline). Then:
- 🔴 if `delta_pct > t` (regression beyond tolerance).
- 🟡 if `0.5 * t < delta_pct ≤ t` (within tolerance but trending up; the SAD-090 yellow-marker semantics).
- 🟢 otherwise (`delta_pct ≤ 0.5 * t`, including all improvements where `delta_pct < 0`).
3. The markdown report's table carries columns: `metric | unit | baseline | current | delta_pct | tolerance | marker`. Marker column uses the literal emoji characters 🟢 / 🟡 / 🔴. A header line cites the merge-base SHA from §6 step 7 so the reviewer can confirm which baseline the comparison used.
§9 — Cross-traces (informational, not derivation).
1. SDD-094 (cross-platform audio engine `voice_join` / `voice_leave` lifecycle): the §3 bench seam targets `CaptureState::ingest`, which is part of the engine's running-state surface exposed by `AudioEngine::ensure_running`. The harness measures the steady-state callback path that runs while the engine is in the `ensure_running == true` state; this places the bench inside SDD-094's verification surface. The bench does NOT construct a full `AudioEngine` (no cpal stream is opened — the bench drives `CaptureState::ingest` directly with synthetic input), so the §3 bench seam is isolated from cpal device enumeration and from JNI / oboe-rs concerns.
2. SDD-118 (Android `chanora_bridge` build automation): explicitly NOT cross-traced. The bench harness runs only on the `ubuntu-latest` host with the `x86_64-unknora-linux-gnu` target per SysDes-157; Android cross-compilation is not exercised by either workflow file in this SDD unit.
3. SDD-076..SDD-080 (PTT capability surface): explicitly NOT cross-traced. The realtime callback being benched is downstream of these — a PTT key event ultimately gates the audio engine's transmit path — but the bench specifically targets the audio capture / codec / resampler path, not the PTT event-routing path. Adding PTT events would conflate the measurement.
§10 — Out of scope (deferrals preserved through Wave 1.5).
1. No production telemetry export — the bench is a CI-only artifact; no runtime metric is exported from a shipping `chanora_audio` build.
2. No hard CI gate — SRS-218 clause 4 is invariant; the advisory workflow never fails a check on regression.
3. No multi-host benchmarking — SysDes-157 pins the host to `x86_64-unknown-linux-gnu`; the SAD-089 filename is target-triple-keyed to permit future multi-host expansion without directory restructuring, but adding a second host (e.g. `aarch64-apple-darwin` for macOS measurement) is out of scope at this revision.
4. No IDE integration — no VS Code task, no IntelliJ run configuration, no flutter-tool wrapper is authored.
5. No benchmark of the Dart-side `flutter_rust_bridge` round-trip — deferred (researcher Tier B4). Measuring the FRB serialization overhead would require a separate `package:benchmark_harness` setup on the Flutter side and a synthetic Dart harness; the Rust-side bench in this SDD unit does not exercise that path.
§11 — Verification matrix (forward-allocation, not authoritative).
1. The new SDD unit traces to suggested SWE.4 verification IDs `SWE4-UV-058` through `SWE4-UV-062` covering the five §3 bench functions (`bench_capture_alloc_count`, `bench_capture_callback_wall_clock`, `bench_opus_encode_latency`, `bench_opus_decode_latency`, `bench_resampler_throughput`). These IDs are SUGGESTED forward allocations only; verification-engineer authors the actual SWE.4 entries in a follow-up dispatch per the SAD-090 verification-method field (CI Dry Run + Inspection). The SDD does not author SWE.4 entries.
2. Verification method per SAD-088 / SAD-089 / SAD-090 / SAD-091: Inspection of the bench-file structure and YAML structure against this SDD unit; Test (manual local `cargo bench --bench …` invocation) for the §3 bench functions; CI Dry Run (manual dispatch of `bench-baseline-update.yml`) for the §7 baseline-update path; PR-comment-posting validation via the §6 step 9 sentinel-marker round-trip on a synthetic PR.
Notes:
- This is a forward-looking SDD unit. The bench files, the Cargo.toml additions, the two workflow YAML files, the `emit_baseline.rs` / `compare_baseline.rs` binaries, and the initial seed of `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` are NEW implementation work for a builder agent. The seed baseline file is generated by manually dispatching `bench-baseline-update.yml` once after the builder lands the rest of the unit; that first dispatch is the bootstrap that the §6 step 9 short-circuit branch will then no longer take on subsequent PR runs.
- The `tsclientlib::AudioHandler::fill_buffer` decode-side bench was explicitly NOT selected as the canonical `opus_decode_latency` target (see §3 item 4 rationale). A future SDD revision may add `audio_handler_fill_buffer_latency` as a separate composite-call metric without modifying the canonical Opus codec metric.
## 11. Updated SAD-to-SDD Coverage Matrix
| SAD Range | SDD Coverage |
@@ -1715,6 +1821,7 @@ Notes:
| SAD-077, SAD-081, SAD-084, SAD-085, SAD-086 | Covered by `SDD-111` through `SDD-116` (Android voice audio backend — SRS-210..SRS-215). No dedicated SAD anchor exists for the platform-abstracted mobile-voice-audio-backend trait itself; SDD-111 is sourced from SAD-077 / SAD-081 pending a SAD revision that adds a dedicated allocation item. |
| SAD-063 | Covered by `SDD-118` (Android `chanora_bridge` cdylib build automation — Gradle + `cargo-ndk` + per-ABI jniLibs staging). Cross-trace: SDD-073, SDD-105, SDD-109. |
| SAD-061, SAD-062, SAD-087 | Covered by `SDD-119` (iOS / macOS `chanora_bridge` cdylib build automation — CocoaPods podspec + `cargo` + `lipo` + framework-layout shell, back-fill of existing code). Source SAD allocation: iOS half anchored to SAD-061 (iOS runtime) and SAD-062 (App Store / packaging); macOS half anchored to SAD-087 (macOS runtime baseline — deployment target isolation, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, hand-rolled CocoaPods podspec automation), authored in SAD v0.9.7 to close the macOS-runtime anchor gap that this SDD unit had previously bridged by parallelism from SAD-061 / SAD-062. SDD-117 remains reserved-but-unauthored for the deferred `ios_voice_unit` trait back-fill noted by SDD-111. |
| SAD-088, SAD-089, SAD-090, SAD-091 | Covered by `SDD-120` (realtime-audio benchmark harness and advisory CI infrastructure — criterion-based bench harness, dhat-backed heap-allocation-count metric, SAD-089 baseline JSON post-processor, SAD-090 advisory PR-comment workflow with merge-base baseline read, SAD-091 `workflow_dispatch`-only baseline-update workflow that opens a PR rather than direct-pushing; simpler-form yellow-marker realization treats the SAD-089 baseline as the comparator for both red/green tolerance evaluation and yellow trending detection). Cross-trace: SDD-094 (the bench seam exercises the same `chanora_audio::engine` capture path whose lifecycle is specified by SDD-094). Suggested SWE.4 forward allocation: SWE4-UV-058 through SWE4-UV-062 (verification-engineer follow-up; not authored by this SDD unit). |
## Baseline Candidate 0.9.1 Update
@@ -1776,6 +1883,12 @@ Notes:
| 0.9.8 | 2026-05-18 | Reconciled Android P0 detailed design after Wave 1.5 SysRS / SRS / SAD propagation. SDD-073 (`AndroidBuildConfig`) updated in place (ID preserved) to record `minSdk = 28` per DEC-004 (superseding the prior API 24 baseline), `targetSdk` policy bound to SRS-188 Google Play floor, NDK ABI set pinned to `arm64-v8a / armeabi-v7a / x86_64`, release-signing CI assertion, R8/ProGuard stance, and AAB output per SRS-119 (delegating packaging pipeline detail to SDD-109). SDD-028 (`BackIntentService`) expanded from a one-line stub to an implementable specification covering API 33+ `OnBackInvokedCallback` registration, pre-33 `OnBackPressedDispatcher` fallback, deterministic route-pop ordering, and edge cases (root route → exit candidate, modal/dialog → close-only, active PTT → ignore). Added six new Android P0 SDD units: SDD-105 (`AndroidJniBootstrap` — back-fill for `JNI_OnLoad` / `initChanoraContext` / library-load location, sourced from SAD-081 / SAD-086); SDD-106 (`AndroidPermissionRequester` — RECORD_AUDIO runtime flow with listen-only fallback, settings deep-link for permanent denial, revocation handling, BridgeEvent surface, TransmitModeSelector clamp, sourced from SAD-085); SDD-107 (`AndroidVoiceForegroundService` — service class, manifest entry, notification channel id `chanora.voice.session`, `foregroundServiceType="microphone"` on API 30+, POST_NOTIFICATIONS request on API 33+, lifecycle bound to voice_join / voice_leave / shutdown_if_idle, START_NOT_STICKY on process death, ongoing-notification re-post on dismissal, sourced from SAD-086); SDD-108 (`AndroidAudioModeController` — `setMode(MODE_IN_COMMUNICATION)` with prior-mode snapshot/restore, ref-count semantics composable for P1, JNI surface from Rust audio engine, error paths including AudioManager-unavailable fallback, sourced from SAD-084 — back-fills the v0.9.8 product-decision-register entry); SDD-109 (`AndroidAaBuildPipeline` — bundletool config, ABI splits matching SDD-073, CI signing assertion, versionCode/versionName derivation, sourced from SAD-037 / SAD-063); SDD-110 (`AndroidPttCapability` — pins Android P0 to FocusedPttBackend only, no global key hook, no accessibility-service PTT, capability badge text under `ptt.capability.android.l0`, sourced from SAD-077). Updated §11 SAD-to-SDD coverage matrix to list the new Android P0 SDD range. Strict layered sourcing preserved (`SDD -> SAD` only). |
## Baseline Candidate 0.9.15 Update
| Version | Date | Description |
| --- | --- | --- |
| 0.9.15 | 2026-05-18 | Added realtime-audio benchmark-infrastructure detailed-design unit SDD-120 (`RealtimeAudioBenchmarkHarnessAndAdvisoryCI`) sourced from SAD-088 / SAD-089 / SAD-090 / SAD-091 (the Wave 1.5 benchmark-infrastructure software-architecture chain authored at SAD v0.9.9 from the Option B scope decision propagated through SysRS-307/308/309 → SysDes-156/157/158 → SRS-216..219). SDD-120 is a single cohesive 11-section unit specifying: §1 crate selection (`criterion = "0.5"`, `dhat = "0.3"` dev-deps, three `[[bench]]` entries and two `[[bin]]` entries in `crates/chanora_audio/Cargo.toml`); §2 bench-file layout (three files `realtime_capture.rs` / `opus_codec.rs` / `resampler.rs` plus a shared `common.rs` synthetic-input module); §3 bench function signatures with explicit warmup (100-call pre-warm into `CaptureState::ingest` at `crates/chanora_audio/src/engine.rs:1207` before 1000-call dhat measurement window; criterion default warmup for the wall-clock / encode / decode / throughput benches), with the canonical `opus_decode_latency` metric pinned to `audiopus::coder::Decoder::decode_float` directly rather than `tsclientlib::AudioHandler::fill_buffer` because the latter conflates Opus decode with jitter-buffer dequeue and PCM mix; §4 deterministic synthetic-input generation (440 Hz sine at amplitude 0.5, no RNG, byte-stable across runs); §5 JSON post-processing via a Rust binary `crates/chanora_audio/benches/emit_baseline.rs` that parses criterion's `target/criterion/.../estimates.json` files and projects them into the SRS-217 schema; §6 `.github/workflows/bench-advisory.yml` (PR + push triggers, `dtolnay/rust-toolchain@stable` + `Swatinem/rust-cache@v2` matching the existing `ci.yml` pattern, audio system deps `libasound2-dev libpulse-dev pkg-config libopus-dev`, `fetch-depth: 0` for merge-base resolution, `git show $MERGE_BASE:…/x86_64-unknown-linux-gnu.json` baseline read with a missing-baseline short-circuit, `actions/github-script@v7` PR comment with sentinel-marker de-duplication and `pull_request`-only gating, `actions/upload-artifact@v4` for push-trigger reports, always exit 0 per SRS-218 clause 4); §7 `.github/workflows/bench-baseline-update.yml` (`workflow_dispatch` only — sole writer of SAD-089 per SAD-091, `peter-evans/create-pull-request@v6` opens a PR rather than direct-pushing so baseline updates are review-gated against accidental triggering or runner pollution); §8 yellow-marker realization selecting the SAD-090-permitted simpler form where the SAD-089 baseline JSON is the comparator for both red/green tolerance evaluation and yellow trending detection (no separate run-history artifact), with the comparator algorithm (`compare_baseline.rs`) pinned: 🔴 if `delta_pct > t`, 🟡 if `0.5*t < delta_pct ≤ t`, 🟢 otherwise; zero-tolerance metrics (heap allocation, SRS-219 clause a) collapse to 🔴/🟢 only; §9 cross-traces (SDD-094 audio-engine lifecycle as the bench-seam container; explicit NOT-cross-traces to SDD-118 Android bridge build automation and SDD-076..080 PTT capability surface, both correctly out of scope); §10 deferrals preserved (no telemetry export, no hard CI gate, no multi-host bench, no IDE integration, no Dart-side `flutter_rust_bridge` bench — researcher Tier B4); §11 suggested-forward SWE.4 allocation `SWE4-UV-058` through `SWE4-UV-062` (NOT authored at this layer — verification-engineer follow-up). Updated §11 SAD-to-SDD coverage matrix with a new SAD-088..SAD-091 row mapping to SDD-120. Strict layered sourcing preserved (`SDD -> SAD` only); no SRS, SysDes, SysRS, or SAD edits performed. Recommended next agent: `builder` (Step 6 of the benchmark chain — implements the three bench files, the `common.rs` helper, the two binary post-processors, the `Cargo.toml` dev-dependency + `[[bench]]` + `[[bin]]` additions, and the two workflow YAML files; strictly per this SDD unit). |
## Baseline Candidate 0.9.14 Update
| Version | Date | Description |
+53 -2
View File
@@ -3,7 +3,7 @@
**Product name:** Chanora
**Document type:** SysDes / System Architectural Design
**Process alignment:** Automotive SPICE SYS.3 System Architectural Design
**Version:** 0.9.7
**Version:** 0.9.8
**Status:** Baseline Candidate
**Input baseline:** Chanora SysRS v0.6
**Target application type:** Cross-platform voice client application system
@@ -2940,9 +2940,58 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through
- Allocated SysRS: SysRS-002, SysRS-050, SysRS-193
- Notes: Parallel in role to SysDes-133 (iOS runtime baseline) and SysDes-135 (Android runtime baseline); neither SysDes-133 nor SysDes-135 is modified by this item. Source SysRS selection: SysRS-002 anchors the multi-platform obligation that explicitly enumerates macOS as a target client platform, SysRS-050 anchors the macOS runtime environment obligation for native desktop integration, and SysRS-193 anchors the signed/notarized macOS build obligation that the `.framework` `Versions/A` layout and universal-binary policy must satisfy in the release pipeline. No macOS-specific deployment-target or universal-binary SysRS item currently exists; if a finer-grained macOS minimum-runtime SysRS item is required (analogous to SysRS-286 for iOS and SysRS-288 for Android), the systems-requirements owner should consider authoring it in a follow-up — this SysDes item is structured so that such a future SysRS item can be added to `Allocated SysRS` without restructuring the element. SysDes-155 cites the podspec file path only and does not embed its current deployment-target value; the value itself remains owned by the Build Configuration subsystem.
**SysDes-156**: The system architecture shall allocate to SE-13 (Audio Subsystem) the obligation to expose its realtime capture and playback paths to benchmark instrumentation, such that the maintained numeric performance baselines authorized by SysRS-307 are measurable as a deterministic, automated comparison surface. The metric set authored at this layer is: (i) **heap allocation count per realtime callback after warmup** — zero allocations are expected on the steady-state realtime audio thread, where warmup is defined as the first N callbacks with N implementation-specific (recommended N=100 callbacks; refined at SDD); (ii) **per-callback wall-clock time expressed as a fraction of the cpal stream's reported audio frame period**, captured and reported as p50, p95, and p99 of the callback wall-clock as a fraction of that period; (iii) **Opus encode latency** — end-to-end wall-clock time for `encoder.encode_float()` on a 960-sample (20 ms) frame, captured per call; (iv) **Opus decode latency** — same shape, decoder side; (v) **resampler throughput** in samples-per-second produced at the canonical rate pairs 44.1 kHz → 48 kHz, 16 kHz → 48 kHz, and 48 kHz → 48 kHz passthrough (the passthrough pair serves as a control point). The baseline storage format is declared at this layer as structured JSON committed to a deterministic location in the repository; the exact path is delegated to SDD. Each JSON record shall include the metric value, the host architecture, the toolchain version, the git SHA of the measurement, and a timestamp, so that a baseline snapshot is reproducible and traceable to the commit that produced it. This SysDes item does not authorize off-device transmission of any measured baseline data and is consistent with SysRS-295.
- Status: Baseline Candidate
- Type: System Element Allocation / Performance Verification Surface
- Stage: P0 / MVP
- Allocated to: SE-13 (Audio Subsystem) — owns the realtime path being measured; the realtime callback, the Opus encode/decode paths, and the resampler are all SE-13 surfaces
- Downstream artifact: SRS, SAD, SDD, Verification
- Verification method: Automated Benchmark Execution (criterion crate per SDD); Inspection of the JSON baseline records committed to the default branch
- Verification owner: Audio / Platform QA
- ASPICE SYS.3 alignment: Element allocation, dynamic architecture, resource and performance design, verification handoff
- Allocated SysRS: SysRS-307 (primary derivation); refines SysRS-180 / SysRS-181 / SysRS-186 from prescriptive intent into a measurable contract surface
- Notes: The numeric warmup constant (recommended N=100), the exact JSON repository path, the benchmark crate selection (recommendation: criterion), and the harness wiring against the cpal stream are SDD concerns and are not authored here. The cpal stream's reported period referenced in metric (ii) is the stream period reported by the existing audio capture/playback path owned by SE-13; this SysDes item does not authorize a new platform integration. The control-point passthrough at 48 kHz → 48 kHz in metric (v) provides a zero-arithmetic baseline that isolates harness overhead from resampling cost. Metrics (i) and (v) are the structural-property metrics (no acceptable variance for allocations; throughput is a steady-state measurement); metrics (ii)(iv) are latency-distribution metrics whose tolerance characterisation is the subject of SysDes-158.
**SysDes-157**: The system architecture shall allocate to SE-18 (Deployment and Release Environment) the integration of the realtime-audio benchmark suite (SysDes-156) into the existing CI workflow as an **advisory** regression-reporting surface, satisfying SysRS-308. The design constraints authored at this layer are: (1) **Trigger** — the benchmark workflow shall execute on every pull request opened against the default branch and on every merge to the default branch; it shall not be triggered on tags and shall not be triggered on non-default-branch pushes. (2) **Host scope** — the benchmark workflow shall execute on the Linux x86_64 GitHub Actions runner (`ubuntu-latest`); other host architectures are explicitly out of scope for this revision and shall not be added by SDD or by implementation without a future SysRS clause authorising multi-host benchmarking. (3) **Surface** — the result shall be rendered on the PR status-check view as a named check (suggested name "Benchmark / advisory" or equivalent; the exact check name is delegated to SDD) and shall additionally be rendered as a markdown table posted to the PR conversation, comparing each metric against the most recent baseline on the default branch with the percentage delta and a clear visual marker (e.g., 🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance). (4) **Non-blocking semantics** — the status check shall **never** report a `failure` status; on regression the check shall report `success` (so it does not block merge) and the regression shall be surfaced exclusively in the PR comment for human reviewer attention. This clause is the SysDes-layer enforcement of SysRS-308's advisory-only contract; any future escalation to a build-failing hard gate requires a separate SysRS authorisation and is out of scope of SysDes-157. (5) **Baseline source** — the comparison input shall be read from the deterministic JSON location on the default branch committed under SysDes-156; on the first run after a new benchmark is added, the baseline file may not yet contain that metric, in which case the report shall state "no baseline; this run establishes the baseline candidate" and shall not produce a comparison delta for that metric. (6) **Baseline update isolation** — a separate, manually-invoked CI workflow (GitHub Actions `workflow_dispatch`) shall be the sole mechanism that writes the baseline JSON on the default branch; the PR-triggered benchmark workflow shall never write the baseline file. This isolation guarantees that baseline updates are intentional, reviewable acts and prevents silent baseline drift from PR runs.
- Status: Baseline Candidate
- Type: System Element Allocation / CI Integration Architecture
- Stage: P0 / MVP
- Allocated to: SE-18 (Deployment and Release Environment) — owns CI/CD per SysDes-036; the CI workflow definitions, the GitHub Actions runner selection, the PR status-check surface, the PR comment rendering, and the baseline-update workflow all live here
- Downstream artifact: SRS, SAD, SDD, Verification
- Verification method: Demo (a CI run on a synthetic PR demonstrates the markdown table report, the visual markers, the non-blocking `success` status on a simulated regression, and the "no baseline" fallback on a metric with no prior baseline)
- Verification owner: Release / Operations QA, Audio / Platform QA (co-verification of the report content)
- ASPICE SYS.3 alignment: Element allocation, deployment architecture, verification handoff
- Allocated SysRS: SysRS-308 (primary derivation); composes with SysRS-307 (baselines being compared against) and SysRS-309 (tolerance window applied by the comparison, refined by SysDes-158); consistent with SysRS-295 (no off-device transmission beyond the existing CI provider surface visible to repository collaborators) and with the SysRS-234..239 verification-family pattern
- Notes: The exact YAML workflow filenames, the exact named status-check string, the exact markdown layout of the comparison table, the exact emoji set, and the exact triggering criteria for the 🟡 "trending" state are SDD concerns and are not authored here. The merge-base baseline-snapshot semantics of the comparison are authored separately in SysDes-158. The Linux x86_64 host-scope clause is the active rule that explicitly defers ARM Android, macOS Apple Silicon, Windows x86_64, and any other host runner; that deferral is preserved here intentionally. Composes with SysDes-036 (CI/CD as a system element) as the SE-18 integration anchor.
**SysDes-158**: The system architecture shall allocate to SE-18 (Deployment and Release Environment) the per-metric tolerance window and the baseline-comparison methodology used by the advisory CI surface (SysDes-157), satisfying SysRS-309. The tolerance window is authored at this layer as a per-metric set rather than as a single global value, because the metrics defined in SysDes-156 have different statistical character and require different comparison rules:
| Metric (per SysDes-156) | Tolerance window | Comparison rule | Rationale |
|---|---|---|---|
| Heap allocation count per realtime callback after warmup | **Zero** | Any non-zero allocation count is reported as a regression (🔴) | Structural property of the code, not a statistical measurement; no acceptable variance |
| Per-callback wall-clock as a fraction of the audio frame period | **+20% on p95** | Compare current-run p95 against baseline p95; report regression if delta exceeds +20%. p50 and p99 are reported but not compared against tolerance (advisory data only) | p95 is the operational tail; p50 is too lenient and p99 is too noisy to gate on at MVP |
| Opus encode latency (960-sample / 20 ms frame) | **+15% on mean** | Compare current-run mean against baseline mean; report regression if delta exceeds +15% | Codec encode latency is a tight loop with low variance; mean is a stable comparator |
| Opus decode latency (same frame shape) | **+15% on mean** | Same shape as encode | Symmetric with encode |
| Resampler throughput (samples/second) at 44.1→48, 16→48, 48→48 passthrough | **10% on samples/second** (lower is worse) | Compare current-run samples/sec against baseline samples/sec; report regression if delta is below 10% | Throughput regression is a *drop* in samples/sec, not an *increase*; sign convention is inverted relative to the latency metrics |
The **comparison methodology** authored at this layer is: each PR-triggered benchmark run shall be compared against the most recent baseline committed to the default branch *at the time of the PR's merge-base commit*, not against the current tip of the default branch. This ensures PR comparisons are stable as the default branch advances during the PR's lifetime, and it also means that rebasing a PR onto a newer default-branch base can change which baseline snapshot is used as the comparator. The +20% starting value recorded as non-binding guidance in SysRS-309 is honoured here only for the per-callback wall-clock metric (and is bound to p95); the other metric tolerances are authored independently of the SysRS-309 suggestion, as authorised by SysRS-309's "single declared value (or one declared value per metric)" allowance.
- Status: Baseline Candidate
- Type: System Element Allocation / Performance Comparison Methodology
- Stage: P0 / MVP
- Allocated to: SE-18 (Deployment and Release Environment) — the comparison logic lives in CI tooling co-located with the workflow authored in SysDes-157
- Downstream artifact: SRS, SAD, SDD, Verification
- Verification method: Review (the per-metric tolerance values are subject to periodic review; SDD ratifies the values as authored here and may refine the harness implementation; future SAD/SDD revisions may re-tune the values as baseline noise characteristics are observed empirically)
- Verification owner: Release / Operations QA, Audio / Platform QA
- ASPICE SYS.3 alignment: Element allocation, architecture decision / rationale, resource and performance design, verification handoff
- Allocated SysRS: SysRS-309 (primary derivation); composes with SysRS-307 (the baselines being compared) and SysRS-308 (the advisory surface that consumes this comparison)
- Notes: The selection of p95 (rather than p99) as the gated tail for the wall-clock metric is an explicit architectural decision recorded here: p99 is reported for diagnostic value but is intentionally not tolerance-gated at MVP because the sample population of a single CI run is too small for p99 to be a stable comparator. The merge-base baseline-snapshot rule is the active comparison methodology; an alternative methodology comparing against the latest default-branch baseline (rather than the merge-base baseline) was considered and intentionally rejected because it produces unstable PR comparisons when the default branch advances during the PR's lifetime. Re-tuning of the per-metric tolerance values may be performed by future SAD or SDD revisions without requiring a new SysRS clause, provided the values remain numeric and per-metric (the structural constraints authored here); a change from the per-metric model to a global single value, or vice versa, would require a SysRS-309 revision.
## 17. Updated SysRS Coverage Statement
This SysDes version covers all known SysRS requirements from `SysRS-001` through `SysRS-306`.
This SysDes version covers all known SysRS requirements from `SysRS-001` through `SysRS-309`.
| SysRS Range | SysDes Coverage |
|---|---|
@@ -2952,6 +3001,7 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through
| SysRS-303 through SysRS-304 | Covered by `SysDes-149` through `SysDes-151` |
| SysRS-305 through SysRS-306 | Covered by `SysDes-152` through `SysDes-154` (with SysDes-154 additionally refining SysRS-055 for the Android voice audio backend) |
| SysRS-002 / SysRS-050 / SysRS-193 (macOS platform-baseline slice) | Additionally refined by `SysDes-155` (macOS runtime baseline, parallel to SysDes-133 / SysDes-135) |
| SysRS-307 through SysRS-309 (realtime-audio benchmark infrastructure) | Covered by `SysDes-156` (audio benchmark surface, SE-13), `SysDes-157` (advisory CI integration, SE-18), and `SysDes-158` (per-metric tolerance window and comparison methodology, SE-18) |
## Baseline Candidate 0.9.1 Update
@@ -2993,4 +3043,5 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through
| Version | Date | Description |
|---|---|---|
| 0.9.8 | 2026-05-18 | Added benchmark-infrastructure SysDes derivations SysDes-156, SysDes-157, and SysDes-158 sourced from SysRS-307 / SysRS-308 / SysRS-309 (Wave 1.5 benchmark chain, Step 2). SysDes-156 allocates the realtime-audio benchmark instrumentation surface (heap allocation count per realtime callback after warmup, per-callback wall-clock as a fraction of the cpal stream period at p50/p95/p99, Opus encode latency on a 960-sample / 20 ms frame, Opus decode latency on the same shape, and resampler throughput at 44.1→48 kHz, 16→48 kHz, and 48→48 kHz passthrough as a control) to SE-13 (Audio Subsystem); declares baseline storage as structured JSON committed to a deterministic repo location (exact path delegated to SDD) capturing metric value, host architecture, toolchain version, git SHA, and timestamp. SysDes-157 allocates the advisory CI integration to SE-18 (Deployment and Release Environment): trigger on every PR against the default branch + every merge to default (not on tags, not on non-default-branch pushes), host = Linux x86_64 (`ubuntu-latest`), surface = PR status-check view rendered as a markdown table with per-metric delta and a visual marker (🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance), strict non-blocking semantics (the check shall never report `failure` status — even on regression it reports `success` and surfaces the regression in the PR comment for human reviewer attention), baseline source = JSON on the default branch with first-run-establishes-candidate fallback, and baseline-update isolation via a separate `workflow_dispatch` workflow (PR-triggered workflow shall never write the baseline file). SysDes-158 authors the per-metric tolerance window and the comparison methodology: heap allocations tolerance = zero (any non-zero count is 🔴), per-callback wall-clock tolerance = +20% on p95 (p50 and p99 are advisory data only), Opus encode/decode latency tolerance = +15% on mean, resampler throughput tolerance = 10% on samples/second; comparison is against the most recent baseline committed to the default branch at the PR's merge-base. Allocated to SE-18 (comparison logic lives in CI tooling). Strict layered sourcing preserved (`SysDes -> SysRS` only). Explicitly **NOT** authored in this update, consistent with the SysRS-307/308/309 deferral: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-307/308/309 do not authorize off-device transmission and SysRS-295 still applies); (b) build-failing hard CI gate (SysRS-308's advisory-only clause is the active rule — SysDes-157's non-blocking semantics enforces it at this layer); (c) multi-host benchmarking (ARM Android, macOS Apple Silicon, etc. — SysDes-157's Linux x86_64 host-scope clause is the active rule). |
| 0.9.7 | 2026-05-18 | Added dedicated macOS runtime baseline element allocation SysDes-155, parallel in role to SysDes-133 (iOS runtime baseline) and SysDes-135 (Android runtime baseline). SysDes-155 anchors the macOS deployment-target policy as a single-source-of-truth concern (citing the podspec file path `apps/chanora_flutter/macos/chanora_bridge.podspec` without embedding its value), the universal-binary (`lipo`'d `arm64` + `x86_64`) packaging policy, the `.framework` `Versions/A` bundle layout convention, and CocoaPods podspec integration as the auto-build mechanism for the Rust cdylib. Allocated to macOS Build Configuration, Platform Adapter Layer (macOS Platform Services), Release Pipeline, and Verification. Source SysRS: SysRS-002 (multi-platform target client platforms including macOS), SysRS-050 (macOS runtime environment / native desktop window integration), SysRS-193 (signed/notarized macOS builds). SysDes-155 is the proper SysDes anchor for SAD-087 and for downstream SRS items that allocate macOS-runtime responsibility, and is the recommended retarget for SAD-087's previously-open Source SysDes follow-up. SysDes-133 and SysDes-135 are unchanged. Strict layered sourcing preserved (`SysDes -> SysRS` only). Follow-up recommendation: systems-requirements owner may consider authoring a finer-grained macOS minimum-runtime SysRS item parallel to SysRS-286 (iOS) and SysRS-288 (Android); SysDes-155 is structured so that such a future SysRS item can be added to its `Allocated SysRS` list without restructuring the element. |