diff --git a/docs/architecture/sad.md b/docs/architecture/sad.md index 77a913e..dd68401 100644 --- a/docs/architecture/sad.md +++ b/docs/architecture/sad.md @@ -1434,6 +1434,50 @@ A single SRS item may be intentionally allocated to more than one SAD item when - Source SysDes: SysDes-155 (macOS runtime baseline — deployment target, universal-binary packaging, `.framework` `Versions/A` layout, CocoaPods podspec automation) - Verification method: Platform Test (macOS), Release Inspection, Architecture Review +**SAD-088**: The software architecture shall add a `criterion`-based realtime-audio benchmark harness as a first-class architectural element of the `chanora_audio` crate, extending the existing audio-subsystem allocation in SAD-034. The harness lives in `crates/chanora_audio/benches/` as a Cargo-convention `benches/` directory (a sibling of the crate's `src/`, not a public seam of the crate's library API). The harness shall use the `criterion` crate as its measurement framework (selected at this layer; SDD pins the exact version and feature flags). Rationale for `criterion`: it is the de-facto stable-toolchain Rust microbenchmark harness, produces stable statistical output (mean / p50 / p95 / p99) without requiring a nightly compiler, and integrates with `cargo bench` so the same invocation works in developer shells and on CI runners. The harness shall expose one bench function (or one criterion `BenchmarkGroup`) per SysDes-156 metric family, with the following architectural assignment of metric → bench element: (a) `bench_capture_alloc_count` — counts heap allocations on the steady-state `CaptureState::ingest` path after a warmup phase, using a per-bench allocation-counting mechanism (a dev-dependency allocation tracker such as `dhat`, or a custom global-allocator counter — selection deferred to SDD); (b) `bench_capture_callback_wall_clock` — measures wall-clock duration of `CaptureState::ingest` and reports p50 / p95 / p99 from criterion's sample distribution; (c) `bench_opus_encode_latency` — directly times an Opus encoder over a fixed 960-sample / 20 ms input frame; (d) `bench_opus_decode_latency` — directly times the Opus decode path; if the upstream `tsclientlib` `AudioHandler` is opaque to the crate, the bench shall time the smallest decode wrapper the crate owns and the SDD shall name that wrapper; (e) `bench_resampler_throughput` — measures samples-per-second on the three canonical rate pairs (44.1→48, 16→48, 48→48 kHz). The harness shall emit, for each bench function, the SRS-217 baseline JSON record (one record per metric, in the SRS-217 schema); the emission mechanism (a custom `criterion` measurement reporter, a post-bench JSON-aggregation step, or equivalent) is delegated to SDD. The harness shall not introduce any new public seam on the `chanora_audio` crate's library API surface — benches consume the same internal entry points that the realtime callback consumes, so SAD-034's isolation property is preserved. + +- Status: Baseline Candidate +- Type: Software Architecture Item +- Stage: P0 / MVP +- Allocated to: Audio Subsystem (`chanora_audio` crate, `benches/` directory; extends SAD-034) +- Source SRS: SRS-216, SRS-217 +- Source SysDes: SysDes-156 (realtime-audio benchmark surface, SE-13; five metric families; baseline-JSON storage) +- Verification method: Inspection (bench-function presence and metric coverage), Bench Run (criterion executes on the host runner), Architecture Review + +**SAD-089**: The software architecture shall pin the baseline-JSON on-disk path for the SAD-088 harness to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`, a single deterministic file per host architecture under the crate's `benches/` directory. The filename is keyed on Rust target triple (`x86_64-unknown-linux-gnu`) so future multi-host expansion can add sibling files (`aarch64-apple-darwin.json`, `x86_64-pc-windows-msvc.json`, etc.) without restructuring the directory layout; for the present revision, exactly one such file exists because SysDes-157 fixes the host to Linux x86_64 (`ubuntu-latest`). The file shall be committed to the default branch so that the SAD-090 advisory workflow has a deterministic comparison target at every PR's merge-base commit. The file contents shall conform to the SRS-217 record schema (one record per metric, fields `metric`, `value`, `unit`, `host_arch`, `toolchain`, `git_sha`, `timestamp` at minimum). The file is the sole on-disk output sink of the SAD-088 harness; it is written by exactly one workflow (SAD-091) and read by exactly one workflow (SAD-090) — see SAD-090 / SAD-091 for the workflow isolation property. Multi-host baselines (additional sibling files keyed on alternate target triples) are explicitly out of scope for this revision per SysDes-157 point 2. + +- Status: Baseline Candidate +- Type: Software Architecture Item +- Stage: P0 / MVP +- Allocated to: Audio Subsystem (`chanora_audio` crate, `benches/baselines/` subdirectory; output sink of SAD-088) +- Source SRS: SRS-217 +- Source SysDes: SysDes-156 (baseline-JSON storage; deterministic repo location) +- Verification method: Inspection (file path, schema), Bench Run (file is regenerable end-to-end) + +**SAD-090**: The software architecture shall add an advisory CI workflow file `.github/workflows/bench-advisory.yml` to the build/release/operations allocation (extending SAD-037), realizing the SRS-218 advisory-reporting workflow. Triggers: `pull_request` against the default branch, plus `push` to the default branch (matching SRS-218 clause 1; tag pushes and non-default branch pushes are excluded). Runner: `ubuntu-latest` (the sole host in scope per SysDes-157 point 2). Step shape (high-level; SDD authors the exact YAML): (1) checkout the PR head; (2) install the Rust toolchain pinned by the repo's `rust-toolchain.toml`; (3) for each bench function defined under SAD-088, invoke `cargo bench` on the `chanora_audio` crate; (4) read the baseline JSON at SAD-089's path *as of the PR's merge-base commit* (using `git show :crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` or an equivalent git-plumbing read) — never the default-branch-tip baseline, per SRS-219's anti-retroactive-reclassification rule; (5) compare current measurements against that merge-base baseline using the SRS-219 per-metric tolerance window; (6) render the comparison as a markdown table with one row per metric and a 🟢 / 🟡 / 🔴 marker per the SAD-determined semantics (see below); (7) post the table as a PR comment via `actions/github-script@v7` or an equivalent comment-posting action (exact action choice delegated to SDD); (8) the workflow's status check shall always exit `success` (exit code 0) regardless of any 🔴 marker, per SRS-218 clause 4 — regression visibility is delivered through the markdown table, not through the check status. The named status check shall follow existing repo workflow-naming convention (e.g. `bench-advisory`; SDD reads existing workflow files and matches the convention). This workflow shall never write to the baseline JSON file authored under SAD-089 — write authority is exclusively SAD-091's, enforcing the SRS-218 clause 5 workflow-isolation property at the architecture layer. Yellow-marker semantics is pinned at this SAD layer (see §24.1 paragraph below). The workflow is allocated to the CI surface (SE-18) and is the realization of the SE-18 anchor previously named only at the SysDes layer (no prior SAD item exists for the SE-18 surface; this is the first such allocation). + +- Status: Baseline Candidate +- Type: Software Architecture Item +- Stage: P0 / MVP +- Allocated to: Build, Release, Operations (`.github/workflows/bench-advisory.yml`; extends SAD-037 — CI surface, SE-18 realization) +- Source SRS: SRS-218, SRS-219 (tolerance binding consumed by comparator step) +- Source SysDes: SysDes-157 (advisory CI integration), SysDes-158 (tolerance window and merge-base-snapshot comparator) +- Verification method: Inspection (workflow trigger / runner / non-blocking exit), CI Dry Run (synthetic PR exercises markdown rendering end-to-end), Architecture Review + +**SAD-091**: The software architecture shall add a separate baseline-update CI workflow file `.github/workflows/bench-baseline-update.yml` to the build/release/operations allocation (extending SAD-037), realizing the SRS-218 clause 5 baseline-update-isolation property. Trigger: `workflow_dispatch` only — the workflow is manually invoked from the GitHub Actions tab on the default branch and shall not fire on `pull_request`, `push`, or any tag event. Runner: `ubuntu-latest` (matching SAD-090's host for byte-comparability of measurements). Step shape: (1) checkout the default branch; (2) install the Rust toolchain pinned by `rust-toolchain.toml`; (3) execute the same `cargo bench` invocations as SAD-090 across the SAD-088 bench functions; (4) write the resulting JSON, conformant to SRS-217's schema, to the SAD-089 path `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (overwriting the prior file); (5) commit and push the updated file to the default branch — the commit mechanism (a direct push using a GitHub-bot token, the `peter-evans/create-pull-request` action, or an equivalent push action) is delegated to SDD. This workflow is the **only** writer of the SAD-089 baseline JSON. The SAD-090 PR-triggered workflow is strictly a reader. This single-writer property is the architectural realization of SRS-218 clause 5 and shall be invariant under future revisions (any future revision adding a second writer requires a SAD revision, not an SDD revision). + +- Status: Baseline Candidate +- Type: Software Architecture Item +- Stage: P0 / MVP +- Allocated to: Build, Release, Operations (`.github/workflows/bench-baseline-update.yml`; extends SAD-037 — CI surface, SE-18 realization; sole writer of SAD-089) +- Source SRS: SRS-218 (workflow-isolation clause) +- Source SysDes: SysDes-157 (baseline-update isolation via `workflow_dispatch`) +- Verification method: Inspection (trigger is `workflow_dispatch` only; commit destination is SAD-089's path), CI Dry Run (manual invocation regenerates baseline end-to-end), Architecture Review + +### Yellow-marker semantics (pinned at SAD) + +Pinned at SAD layer to remove a downstream ambiguity that would otherwise surface as an open question in the SAD-090 SDD unit: the 🟡 "within tolerance but trending" marker (SRS-218 clause 3, third element of the marker set) shall be defined as follows: a metric is rendered 🟡 if and only if (a) the metric is within the SRS-219 per-metric tolerance window relative to the merge-base baseline (i.e. it is not 🔴), AND (b) the absolute delta between the PR's current measurement and the **previous run of the SAD-090 workflow on the default branch** (i.e. the most recent default-branch CI bench result, not the baseline JSON itself) is greater than 50% of the SRS-219 tolerance window for that metric. Metrics whose SRS-219 tolerance is exactly zero (the heap-allocation-count metric, SRS-219 clause a) cannot be 🟡 because the 50% trigger is undefined when the window has zero width; for those metrics only 🟢 (zero allocations) and 🔴 (any non-zero allocation count) apply. Rationale for pinning at SAD rather than deferring to SDD: the question "is yellow relative to baseline or relative to previous-run-on-default" is an architectural choice about what state the workflow consumes (baseline JSON only, vs baseline JSON + a previous-runs cache), not an implementation detail; pinning it here fixes the workflow's required inputs at the architecture layer so SDD has a fully determined input set. Operationally, the "previous-run-on-default" datum may be sourced from the SAD-091 baseline (treating the committed baseline as the latest default-branch run, which is the simpler and recommended SDD realization) or from a separate run-history artifact; SDD selects which. + ## 27. Updated SRS-to-SAD Coverage Matrix | SRS Range | SAD Coverage | @@ -1447,6 +1491,10 @@ A single SRS item may be intentionally allocated to more than one SAD item when | SRS-209 | Covered by `SAD-085` | | SRS-111 (Android foreground voice service) | Covered by `SAD-086` (Android-specific allocation; SAD-036 retains the cross-cutting diagnostics allocation for the SRS-111–124 range per §24.1 dual-allocation pattern) | | SRS-013 (multi-platform support — macOS runtime baseline aspect) | Covered by `SAD-087` (macOS runtime baseline; counterpart to SAD-061 / SAD-063) | +| SRS-216 (realtime-audio benchmark instrumentation) | Covered by `SAD-088` (criterion-based bench harness in `chanora_audio/benches/`; extends SAD-034 audio-subsystem allocation). Cross-cutting note: this is a cross-cutting allocation per §24.1 in the sense that the harness sits in the Audio Subsystem (SAD-034 extension) but its CI consumers sit in Build/Release/Operations (SAD-037 extensions); there is no platform-specific variation because SysDes-157 fixes the host to Linux x86_64. | +| SRS-217 (baseline-JSON storage format and on-disk path) | Covered by `SAD-088` (the harness emits the JSON) and `SAD-089` (the on-disk path is pinned to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`) | +| SRS-218 (CI advisory workflow + baseline-update isolation) | Covered by `SAD-090` (`.github/workflows/bench-advisory.yml`, advisory reader, non-blocking exit) and `SAD-091` (`.github/workflows/bench-baseline-update.yml`, `workflow_dispatch`, sole writer of SAD-089); both extend SAD-037 build/release/operations allocation | +| SRS-219 (per-metric tolerance window and merge-base-snapshot comparison methodology) | Covered by `SAD-090` (comparator step consumes the SRS-219 tolerance and the merge-base-snapshot rule; tolerance-window evaluation is the workflow's comparator-step responsibility) | ## Baseline Candidate 0.9.1 Update @@ -1497,3 +1545,4 @@ A single SRS item may be intentionally allocated to more than one SAD item when |---|---|---| | 0.9.7 | 2026-05-18 | Formalized the cross-cutting + platform-specific SAD allocation pattern flagged by the traceability-auditor: added §24.1 narrative subsection describing the deliberate dual-allocation rule (cross-cutting concern + platform-specific realization both recorded), with SRS-111 / SAD-036 / SAD-086 as the worked example, and annotated the §24.2 matrix row for SRS-111..124 accordingly. Added SAD-087 (macOS runtime baseline: deployment target isolation, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, hand-rolled CocoaPods podspec automation), parallel to SAD-061 (iOS runtime) and SAD-063 (Android runtime), closing the macOS-runtime anchor gap flagged by the SDD-119 detailed-designer. SAD-087 traces upward through SRS-013 (multi-platform support); a parallel macOS-runtime SysDes derivation does not yet exist and is recorded as an open follow-up for the system-architect (SysDes-133 covers iOS, SysDes-135 covers Android, macOS has no counterpart yet). §27 matrix updated with the SAD-087 row. Strict layered sourcing preserved (`SAD -> SRS` only). | | 0.9.8 | 2026-05-18 | Retargeted SAD-087's `Source SysDes` field from the "open follow-up — no dedicated macOS-runtime SysDes anchor exists" placeholder to `SysDes-155` (macOS runtime baseline — deployment target, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, CocoaPods podspec automation), authored by the system-architect in SysDes v0.9.7 as the parallel macOS-runtime anchor to SysDes-133 (iOS) / SysDes-135 (Android). This closes the open follow-up that SAD-087 carried at its 0.9.7 authoring, giving SAD-087 a proper upward SysDes trace. Surgical single-field edit: no behavioral content of SAD-087 changed; `Source SRS` retained as `SRS-013` (multi-platform support — Windows, macOS, Linux, iOS, Android), which remains the correct SRS anchor (the SRS-013 vs alternate-anchor question raised during SysDes-155 authoring referred to SysRS-013 / SysRS-002 at the system-requirements layer, not to SRS at the software-requirements layer). §27 SRS-to-SAD coverage matrix row for SRS-013 unchanged because it already cites SAD-087 and remains textually accurate. Strict layered sourcing preserved (`SAD -> SRS`, with SysDes cross-reference now resolved). | +| 0.9.9 | 2026-05-18 | Added realtime-audio benchmark-infrastructure software-architecture items SAD-088 through SAD-091 sourced from SRS-216 / SRS-217 / SRS-218 / SRS-219 (the Option B benchmark-infrastructure scope decision propagated through the Wave 1.5 benchmark chain SysRS-307/308/309 → SysDes-156/157/158 → SRS-216..219). SAD-088 allocates a `criterion`-based bench harness to the `chanora_audio` crate's `benches/` directory, extending SAD-034 (audio subsystem allocation); the harness covers the five SysDes-156 metric families (heap allocation count, per-callback wall-clock at p50/p95/p99, Opus encode latency on 960-sample/20 ms frame, Opus decode latency on the same shape, resampler throughput at 44.1→48 / 16→48 / 48→48 kHz) and emits SRS-217 baseline records. SAD-089 pins the baseline-JSON on-disk path to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (single deterministic file, single host architecture in scope, target-triple-keyed filename to permit future multi-host expansion without directory restructuring). SAD-090 adds the advisory CI workflow `.github/workflows/bench-advisory.yml` extending SAD-037 (build/release/operations allocation): triggers on PR-against-default + push-to-default, runs on `ubuntu-latest`, reads the merge-base baseline (never the default-tip baseline, per SRS-219 anti-retroactive-reclassification), compares using SRS-219 tolerances, posts a markdown table with 🟢/🟡/🔴 markers, and always exits success (non-blocking per SRS-218 clause 4). SAD-091 adds the separate baseline-update workflow `.github/workflows/bench-baseline-update.yml` extending SAD-037: `workflow_dispatch` trigger only, sole writer of the SAD-089 baseline file, realizing the SRS-218 clause 5 workflow-isolation property at the architecture layer (single-writer invariant). Yellow-marker semantics pinned at this SAD layer in a §26 subsection: 🟡 means "within SRS-219 tolerance relative to merge-base baseline AND delta-from-previous-default-branch-run exceeds 50% of the tolerance window"; metrics with zero-width tolerance (heap allocations) cannot be 🟡. Rationale for pinning at SAD: the choice between baseline-only vs baseline-plus-run-history as workflow input is an architectural input-set decision, not an implementation detail. §27 SRS-to-SAD coverage matrix updated with four new rows (SRS-216, SRS-217, SRS-218, SRS-219) citing SAD-088..091; the SRS-216 row also annotates the cross-cutting allocation pattern per §24.1 (harness sits in Audio Subsystem via SAD-034 extension while CI consumers sit in Build/Release/Operations via SAD-037 extensions, with no platform-specific variation because SysDes-157 fixes the host to Linux x86_64). Intentionally NOT authored consistent with the SysRS/SysDes/SRS deferral chain: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-295 active); (b) build-failing hard CI gate (SysRS-308 advisory-only rule active; SAD-090 status-check always exits success); (c) multi-host benchmarking (SysDes-157 host-scope clause active; SAD-089 holds a single target-triple-keyed file); (d) IDE integration (e.g. `cargo bench` invocation from rust-analyzer in VS Code — tooling polish, not P0 architecture). Strict layered sourcing preserved (`SAD -> SRS` only per the SAD §2 rule, with SysDes cross-references recorded for completeness on each new item). | diff --git a/docs/architecture/sdd.md b/docs/architecture/sdd.md index dfc084c..0f80864 100644 --- a/docs/architecture/sdd.md +++ b/docs/architecture/sdd.md @@ -1703,6 +1703,112 @@ Notes: - This is a back-fill SDD. No new implementation work is created by SDD-119; the code already exists and ships. A builder agent does NOT need to act on SDD-119. The doc-drift defect in item 4 and the deployment-target single-source-of-truth improvement in item 3 are recorded as open follow-ups and are not in scope for this unit. - SAD-side anchoring of the build-automation pattern at the architectural layer is NOT performed by this SDD unit; a software-architect dispatch is required if the orchestrator wants a SAD-layer build-automation item. +**SDD-120 — Realtime audio benchmark harness and advisory CI infrastructure** + +- Source SAD: SAD-088 (criterion-based bench harness allocated to `chanora_audio` `benches/` directory, extending SAD-034 audio-subsystem allocation), SAD-089 (baseline-JSON on-disk path pinned to `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`), SAD-090 (advisory CI workflow `.github/workflows/bench-advisory.yml` — PR-triggered, non-blocking, comment-posting reader of SAD-089), SAD-091 (baseline-update CI workflow `.github/workflows/bench-baseline-update.yml` — `workflow_dispatch`-only, sole writer of SAD-089). +- Allocated to: Audio Subsystem (`crates/chanora_audio/benches/`) + Build/Release/Operations (`.github/workflows/bench-advisory.yml`, `.github/workflows/bench-baseline-update.yml`). +- Software units (new): `crates/chanora_audio/benches/realtime_capture.rs`, `crates/chanora_audio/benches/opus_codec.rs`, `crates/chanora_audio/benches/resampler.rs`, `crates/chanora_audio/benches/common.rs` (shared synthetic-input helper module), `crates/chanora_audio/benches/emit_baseline.rs` (post-bench JSON aggregator binary), `crates/chanora_audio/benches/compare_baseline.rs` (PR-vs-baseline comparator binary), `crates/chanora_audio/Cargo.toml` (dev-dependency additions and `[[bench]]` entries), `.github/workflows/bench-advisory.yml`, `.github/workflows/bench-baseline-update.yml`, `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` (seeded SAD-089 baseline file). +- Cross-trace: SDD-094 (the `chanora_audio::engine::AudioEngine` capture path that the §3 bench seam exercises is the same audio engine whose `ensure_running` / `shutdown_if_idle` lifecycle is specified by SDD-094; the harness measures the steady-state callback that runs while `ensure_running` has opened the cpal input stream, so the bench seam is part of the engine's verification surface). No cross-trace to SDD-076..SDD-080 / SDD-118: the bench targets the realtime audio path and not the PTT capability surface; the Android-bridge build automation is unrelated to the Linux-host bench workflow. + +§1 — Crate selection and version pin. + +1. Add `criterion = "0.5"` to `crates/chanora_audio/Cargo.toml` `[dev-dependencies]`. The `0.5` series is the current stable major and is compatible with the workspace `rust-version = "1.95"` pin. Default features are kept; `html_reports` is left at its default (criterion only emits HTML when invoked with `--save-baseline`/`--load-baseline` against the local user, and CI consumes the per-bench `estimates.json` files directly from `target/criterion/`, so an HTML feature toggle is not load-bearing for the workflow). Rationale: criterion is the de-facto statistical benchmark harness for Rust; it emits a stable JSON estimates file under `target/criterion///estimates.json` that the §5 post-processor parses. +2. Add `dhat = "0.3"` to `crates/chanora_audio/Cargo.toml` `[dev-dependencies]`. `dhat` is used inside `bench_capture_alloc_count` to install a `#[global_allocator] static ALLOC: dhat::Alloc = dhat::Alloc;` profiler at bench-binary scope and to compare `dhat::HeapStats::get().total_blocks` (or the equivalent v0.3 API name) before and after the measurement window. Rationale: `dhat` is the de-facto heap profiler for Rust, has zero-cost when disabled, and is the lowest-friction way to obtain a steady-state heap-allocation count for the SRS-219 clause-a zero-allocation invariant (which carries `tolerance = 0`). +3. Add three `[[bench]]` entries to `crates/chanora_audio/Cargo.toml`, one per bench file enumerated in §2. Each entry shall set `harness = false` only if the bench file calls `criterion::criterion_main!` directly (criterion's `main!` macro replaces the cargo test harness); the §3 signatures use `criterion_main!` so `harness = false` is required. +4. Add two `[[bin]]` entries pointing at `benches/emit_baseline.rs` and `benches/compare_baseline.rs` so that `cargo run --bin emit_baseline` and `cargo run --bin compare_baseline` resolve correctly. The §6 / §7 workflow steps invoke these binaries by name; placing the source files under `benches/` keeps the entire benchmark surface in one directory but requires the `path = "benches/.rs"` field on each `[[bin]]` entry. + +§2 — Bench file layout. + +1. `crates/chanora_audio/benches/realtime_capture.rs` — declares `bench_capture_alloc_count` and `bench_capture_callback_wall_clock`. Uses `criterion_group!(realtime_capture, bench_capture_alloc_count, bench_capture_callback_wall_clock);` and `criterion_main!(realtime_capture);`. +2. `crates/chanora_audio/benches/opus_codec.rs` — declares `bench_opus_encode_latency` and `bench_opus_decode_latency`. Uses `criterion_group!(opus_codec, bench_opus_encode_latency, bench_opus_decode_latency);` and `criterion_main!(opus_codec);`. +3. `crates/chanora_audio/benches/resampler.rs` — declares `bench_resampler_throughput`. Uses `criterion_group!(resampler, bench_resampler_throughput);` and `criterion_main!(resampler);`. +4. `crates/chanora_audio/benches/common.rs` — shared synthetic-input helpers per §4. Included from each bench file with `mod common;` (criterion bench files compile as standalone binaries, so `common.rs` is included by relative module declaration in each `realtime_capture.rs` / `opus_codec.rs` / `resampler.rs`; do NOT add it as a fourth `[[bench]]` entry). + +§3 — Bench function signatures and warmup phase. + +1. `fn bench_capture_alloc_count(c: &mut Criterion)` — the bench installs a `dhat::Profiler::builder().testing().build()` once at the start of the function, runs the §4 `synthetic_capture_buffer(960, 1)` through `CaptureState::ingest(&buf)` 100 times as a warm-up (this primes any first-call allocations from the cpal capture-state machine — ring-buffer growth, resampler-state init, opus-encoder-handle init via the existing engine code path), then samples `dhat::HeapStats::get()` to capture the `total_blocks` count `B_warm`, runs 1000 additional `ingest` calls inside `c.bench_function("capture_alloc_count", |b| b.iter(|| ingest_one()))`, samples `dhat::HeapStats::get()` again to obtain `B_final`, and the criterion-reported metric value shall be `(B_final - B_warm)` (the post-warmup allocation count). The function additionally `assert_eq!(B_final - B_warm, 0)` so a regression also fails the bench process at the local-developer surface; the CI advisory comparator in `compare_baseline.rs` carries the SRS-219 clause-a `tolerance = 0` zero-allocation rule independently. Bench seam: the function exercises the `CaptureState::ingest(&mut self, buf: &[T])` signature located at `crates/chanora_audio/src/engine.rs:1207` per the §“State” reading; the bench obtains a `CaptureState` instance by calling the engine's existing test-mode constructor (the same path `tests/ptt_privacy.rs` uses today). No new public API surface is created on `CaptureState`. +2. `fn bench_capture_callback_wall_clock(c: &mut Criterion)` — uses criterion's default `c.bench_function("capture_callback_wall_clock", |b| b.iter(|| capture_state.ingest(black_box(&buf))));` over the same `synthetic_capture_buffer(960, 1)` input. Criterion's internal warmup (default 3 s, configurable but not overridden here) precedes the measurement window; the harness emits `mean`, `median`, and the 95 % and 99 % bootstrap-confidence-interval percentiles into `target/criterion/capture_callback_wall_clock/.../estimates.json`. The §5 post-processor extracts p50 (median), p95, and p99 from that file and writes the three values into the SRS-217 record's `value` triplet. +3. `fn bench_opus_encode_latency(c: &mut Criterion)` — constructs an `audiopus::coder::Encoder::new(SampleRate::Hz48000, Channels::Mono, Application::Voip)` once outside the measurement loop (per criterion best practice; encoder construction is dominated by libopus state allocation and is not part of the per-frame latency), pre-builds a `synthetic_opus_frame()` PCM buffer (exactly `FRAME_SAMPLES = 960` samples per SAD-088 / SysDes-156), and measures `encoder.encode_float(&pcm, &mut opus_out)` per iteration. The bench reports p50/p95/p99 as in item 2. +4. `fn bench_opus_decode_latency(c: &mut Criterion)` — same pattern as item 3 but with `audiopus::coder::Decoder::decode_float(&opus_bytes, &mut pcm_out, false)`. The SDD explicitly pins the bench target to the `audiopus` direct decoder call rather than `tsclientlib::AudioHandler::fill_buffer`, because `AudioHandler::fill_buffer` performs jitter-buffer dequeue + Opus decode + PCM mix in a single call and the wall-clock measurement would conflate three distinct concerns. The canonical SRS-217 metric `opus_decode_latency` therefore refers to the codec-level decode of a single 20 ms / 960-sample frame, not the full AudioHandler call. (Future revision may add a separate `audio_handler_fill_buffer_latency` metric covering the composite call; that metric is out of scope at this SDD revision.) +5. `fn bench_resampler_throughput(c: &mut Criterion)` — opens one `criterion::BenchmarkGroup` named `resampler_throughput` and registers three sub-benches: `c.bench_function("44100_to_48000", ...)`, `c.bench_function("16000_to_48000", ...)`, and `c.bench_function("48000_passthrough", ...)`. Each sub-bench feeds a 1-second synthetic-sine input (length is 44100 / 16000 / 48000 frames respectively) through the existing engine's resampler entry point and reports `samples_per_second = input_len / measured_duration_secs`. The bench uses `group.throughput(Throughput::Elements(input_len as u64))` so criterion's report exposes the throughput estimate directly in `estimates.json`. + +§4 — Synthetic input generation. + +1. `crates/chanora_audio/benches/common.rs` exposes two functions, both deterministic (no RNG, no time-of-day input): `pub fn synthetic_capture_buffer(frames: usize, channels: usize) -> Vec` and `pub fn synthetic_opus_frame() -> Vec` (the latter is the special case `synthetic_capture_buffer(960, 1)`). +2. Sample generation: a 440 Hz sine at amplitude 0.5 over the 48 kHz playback rate (matching the audio engine's mixer rate per SAD-034). For frame index `n` the sample value is `0.5 * (2.0 * PI * 440.0 * (n as f32) / 48_000.0).sin()`. For multi-channel buffers the same scalar value is replicated across all channels at the same frame index (interleaved layout, matching cpal's `Stream` data layout). No randomness, no fade-in, no DC offset — the same input produces the same bytes across runs and the same bytes across hosts. +3. Determinism is load-bearing for §5 baseline stability: if the synthetic input drifts between runs the criterion `estimates.json` would also drift and the comparator would emit false-positive 🔴 markers. The §5 emit-baseline binary and the §6 comparator both assume byte-stable input. + +§5 — JSON post-processing. + +1. After each bench file runs, criterion writes `target/criterion///estimates.json` per benchmark group. The §5 binary `crates/chanora_audio/benches/emit_baseline.rs` reads the five expected estimates files (one per §3 bench function, with the resampler bench contributing three sub-bench estimate files that are aggregated into the single SRS-217 `resampler_throughput` record by averaging the three throughput estimates into a per-mode triplet, NOT a single scalar — the SRS-217 schema carries a per-mode record). +2. The binary parses each `estimates.json` (a stable criterion-emitted JSON shape: `mean`, `median`, `std_dev`, `confidence_interval`) and projects each metric into an SRS-217 record with fields `{ metric: , value: , unit: , tolerance_pct: , host: "x86_64-unknown-linux-gnu", timestamp_utc: }`. The full output is a single JSON document with one top-level array `metrics`. +3. The binary writes the document to `./current.json` in the workspace root (not the SAD-089 path — that path is the comparator's input on advisory runs and the writer's output target on baseline-update runs; §6 step 7 reads SAD-089 from git, §7 step 5 writes SAD-089 by copy). +4. Rationale for the Rust-binary approach over a `tools/bench-emit.sh` script: same toolchain as the bench files (no separate `jq` / `python` / `bash` dependency on the runner), same `serde_json` already pulled by the workspace, and the binary can share types with `compare_baseline.rs` (both crates use the same SRS-217 struct definitions). + +§6 — `.github/workflows/bench-advisory.yml` structure (SAD-090 realization). + +1. Triggers: `pull_request: { types: [opened, synchronize, reopened], branches: [main] }` and `push: { branches: [main] }`. (The default branch is `main` per the existing `.github/workflows/ci.yml` convention; if the repository default branch ever renames, both YAML files in this SDD unit must update in lockstep.) +2. Job name: `bench-advisory`. `runs-on: ubuntu-latest`. `permissions: { pull-requests: write, contents: read }` — `pull-requests: write` is required by step 9 (PR-comment posting); `contents: read` is the minimum for `actions/checkout`. +3. Step 1: `actions/checkout@v4` with `fetch-depth: 0`. Full history is required because step 6 resolves the PR merge-base via `git merge-base origin/main HEAD` and step 7 reads the baseline file from that merge-base via `git show`. +4. Step 2: install Rust toolchain via `dtolnay/rust-toolchain@stable` (matching the existing `.github/workflows/ci.yml` pattern). The workspace `rust-version = "1.95"` floor is satisfied by `@stable` at the time of authoring; if a future workspace pin moves above the public stable channel, both this file and `ci.yml` must update. +5. Step 3: `Swatinem/rust-cache@v2` (matching existing `.github/workflows/ci.yml`). Reuses the bench target directory across runs. +6. Step 4: install the same system audio deps `ci.yml` installs (`libasound2-dev libpulse-dev pkg-config libopus-dev`). Required because the `chanora_audio` crate links `libopus` and the cpal Linux backend pulls ALSA headers at build time. +7. Step 5: `cargo bench --bench realtime_capture --bench opus_codec --bench resampler`. (`cargo bench` without `--benches` would build doctests; the explicit `--bench ` list bypasses that and matches the three `[[bench]]` entries from §1 item 3.) No `-- --output-format json` flag is required because §5 reads `target/criterion/.../estimates.json` files directly; criterion writes those unconditionally. +8. Step 6: `cargo run --bin emit_baseline` → writes `./current.json` to the workspace root. +9. Step 7: `MERGE_BASE=$(git merge-base origin/main HEAD)` and `git show $MERGE_BASE:crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json > baseline.json`. If the merge-base predates the introduction of SAD-089 (i.e. the file did not exist at that commit), `git show` exits non-zero; the step shall handle the missing-baseline case by writing an empty `baseline.json` placeholder and emitting a single advisory comment `Baseline not yet established at merge-base; first run on default branch will establish.` instead of running the comparator. This is the only branch where the workflow short-circuits before the comparator step. (The `push: branches: [main]` trigger never hits this short-circuit after the first baseline lands.) +10. Step 8: `cargo run --bin compare_baseline -- --current ./current.json --baseline ./baseline.json --output ./report.md`. The binary applies the SRS-219 per-metric tolerance table and the §8 marker logic, then writes a markdown table to `./report.md`. The binary shall always exit 0 (per SRS-218 clause 4); regression detection is rendered as a 🔴 marker inside the comment, not as a non-zero exit code. +11. Step 9: post the markdown as a PR comment via `actions/github-script@v7`, gated on `if: github.event_name == 'pull_request'` (the `push` trigger has no PR to comment on; for `push` runs the workflow still writes `./report.md` as a workflow artifact via `actions/upload-artifact@v4` so the default-branch trend is inspectable from the Actions tab). Comment body is `body: require('fs').readFileSync('report.md', 'utf8')`. The script shall also de-duplicate prior comments by the same `github-actions[bot]` identifier carrying a sentinel marker line (e.g. ``) at the head of the body — on each run the existing comment is updated in place rather than appended, so a long-lived PR does not accumulate one comment per push. The de-duplication query uses `octokit.rest.issues.listComments` filtered on `user.login === 'github-actions[bot]'` and body-startsWith the sentinel. +12. Step 10: `if: always()` and a final `exit 0` — no step in the job sets `continue-on-error: false` against a regression-detecting predicate, so the SRS-218 clause 4 non-blocking property is realized by absence of a failing step rather than by an explicit override. +13. Status-check name registered with the branch-protection ruleset (if any): `bench-advisory`. The SDD does not mandate adding this to the required-checks list; SRS-218 clause 4 explicitly prohibits making it required. + +§7 — `.github/workflows/bench-baseline-update.yml` structure (SAD-091 realization). + +1. Trigger: `workflow_dispatch: {}` only. No `pull_request`, no `push`, no `schedule`, no tag triggers. SAD-091 is invariant on this property. +2. Job name: `bench-baseline-update`. `runs-on: ubuntu-latest` (byte-comparable to SAD-090 per SAD-091). `permissions: { contents: write, pull-requests: write }` — write on both because step 6 opens a PR via `peter-evans/create-pull-request@v6`. +3. Step 1: `actions/checkout@v4` with default ref (no `fetch-depth: 0` needed; the workflow does not resolve merge-bases). +4. Step 2: install Rust toolchain (same shape as §6 step 2). +5. Step 3: install audio system deps (same shape as §6 step 4). +6. Step 4: `cargo bench --bench realtime_capture --bench opus_codec --bench resampler` (same shape as §6 step 5). +7. Step 5: `cargo run --bin emit_baseline` (same shape as §6 step 6). +8. Step 6: `cp ./current.json crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`. +9. Step 7: `peter-evans/create-pull-request@v6` with `title: "chore(bench): update baselines on $(date -u +%Y-%m-%d)"`, `branch: bench/baseline-update-$(date -u +%Y%m%d)`, `commit-message: "chore(bench): refresh x86_64-unknown-linux-gnu baseline"`, `body: `, and `add-paths: crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json`. Rationale for opening a PR rather than direct-pushing to `main`: baseline updates should be reviewable in case of accidental triggering (a contributor clicks the dispatch button by mistake), runner pollution (an unrelated process on the GitHub-hosted runner skewed the measurement), or a legitimate regression that should not silently move the floor. The PR is then merged by a reviewer using the normal branch-protection flow; this preserves the SAD-091 invariant that this workflow is the **sole writer** of the SAD-089 file (the PR-merge commit is also written by this workflow's PR, not by a human directly editing the JSON). + +§8 — Yellow-marker realization (simpler-form selection per SAD-090). + +1. Per SAD-090's yellow-marker pin, the SDD selects the simpler operationalization: the committed SAD-089 baseline JSON IS treated as the most recent default-branch run, and is the comparator for both the green/red tolerance evaluation AND the yellow trending evaluation. No separate run-history artifact is maintained. +2. `compare_baseline.rs` logic — for each metric `m` in the SRS-217 record set: + - Let `c = current.metrics[m].value`, `b = baseline.metrics[m].value`, `t = SRS-219 tolerance for m`. + - If `m` is a zero-tolerance metric (SRS-219 clause a — `capture_alloc_count`): emit 🔴 if `c != 0`, 🟢 if `c == 0`. Yellow does not apply. + - Otherwise (`t > 0`): compute `delta_pct = (c - b) / b` when `b > 0`, or `delta_pct = 0.0` when `b == 0` (a degenerate baseline). Then: + - 🔴 if `delta_pct > t` (regression beyond tolerance). + - 🟡 if `0.5 * t < delta_pct ≤ t` (within tolerance but trending up; the SAD-090 yellow-marker semantics). + - 🟢 otherwise (`delta_pct ≤ 0.5 * t`, including all improvements where `delta_pct < 0`). +3. The markdown report's table carries columns: `metric | unit | baseline | current | delta_pct | tolerance | marker`. Marker column uses the literal emoji characters 🟢 / 🟡 / 🔴. A header line cites the merge-base SHA from §6 step 7 so the reviewer can confirm which baseline the comparison used. + +§9 — Cross-traces (informational, not derivation). + +1. SDD-094 (cross-platform audio engine `voice_join` / `voice_leave` lifecycle): the §3 bench seam targets `CaptureState::ingest`, which is part of the engine's running-state surface exposed by `AudioEngine::ensure_running`. The harness measures the steady-state callback path that runs while the engine is in the `ensure_running == true` state; this places the bench inside SDD-094's verification surface. The bench does NOT construct a full `AudioEngine` (no cpal stream is opened — the bench drives `CaptureState::ingest` directly with synthetic input), so the §3 bench seam is isolated from cpal device enumeration and from JNI / oboe-rs concerns. +2. SDD-118 (Android `chanora_bridge` build automation): explicitly NOT cross-traced. The bench harness runs only on the `ubuntu-latest` host with the `x86_64-unknora-linux-gnu` target per SysDes-157; Android cross-compilation is not exercised by either workflow file in this SDD unit. +3. SDD-076..SDD-080 (PTT capability surface): explicitly NOT cross-traced. The realtime callback being benched is downstream of these — a PTT key event ultimately gates the audio engine's transmit path — but the bench specifically targets the audio capture / codec / resampler path, not the PTT event-routing path. Adding PTT events would conflate the measurement. + +§10 — Out of scope (deferrals preserved through Wave 1.5). + +1. No production telemetry export — the bench is a CI-only artifact; no runtime metric is exported from a shipping `chanora_audio` build. +2. No hard CI gate — SRS-218 clause 4 is invariant; the advisory workflow never fails a check on regression. +3. No multi-host benchmarking — SysDes-157 pins the host to `x86_64-unknown-linux-gnu`; the SAD-089 filename is target-triple-keyed to permit future multi-host expansion without directory restructuring, but adding a second host (e.g. `aarch64-apple-darwin` for macOS measurement) is out of scope at this revision. +4. No IDE integration — no VS Code task, no IntelliJ run configuration, no flutter-tool wrapper is authored. +5. No benchmark of the Dart-side `flutter_rust_bridge` round-trip — deferred (researcher Tier B4). Measuring the FRB serialization overhead would require a separate `package:benchmark_harness` setup on the Flutter side and a synthetic Dart harness; the Rust-side bench in this SDD unit does not exercise that path. + +§11 — Verification matrix (forward-allocation, not authoritative). + +1. The new SDD unit traces to suggested SWE.4 verification IDs `SWE4-UV-058` through `SWE4-UV-062` covering the five §3 bench functions (`bench_capture_alloc_count`, `bench_capture_callback_wall_clock`, `bench_opus_encode_latency`, `bench_opus_decode_latency`, `bench_resampler_throughput`). These IDs are SUGGESTED forward allocations only; verification-engineer authors the actual SWE.4 entries in a follow-up dispatch per the SAD-090 verification-method field (CI Dry Run + Inspection). The SDD does not author SWE.4 entries. +2. Verification method per SAD-088 / SAD-089 / SAD-090 / SAD-091: Inspection of the bench-file structure and YAML structure against this SDD unit; Test (manual local `cargo bench --bench …` invocation) for the §3 bench functions; CI Dry Run (manual dispatch of `bench-baseline-update.yml`) for the §7 baseline-update path; PR-comment-posting validation via the §6 step 9 sentinel-marker round-trip on a synthetic PR. + +Notes: + +- This is a forward-looking SDD unit. The bench files, the Cargo.toml additions, the two workflow YAML files, the `emit_baseline.rs` / `compare_baseline.rs` binaries, and the initial seed of `crates/chanora_audio/benches/baselines/x86_64-unknown-linux-gnu.json` are NEW implementation work for a builder agent. The seed baseline file is generated by manually dispatching `bench-baseline-update.yml` once after the builder lands the rest of the unit; that first dispatch is the bootstrap that the §6 step 9 short-circuit branch will then no longer take on subsequent PR runs. +- The `tsclientlib::AudioHandler::fill_buffer` decode-side bench was explicitly NOT selected as the canonical `opus_decode_latency` target (see §3 item 4 rationale). A future SDD revision may add `audio_handler_fill_buffer_latency` as a separate composite-call metric without modifying the canonical Opus codec metric. + ## 11. Updated SAD-to-SDD Coverage Matrix | SAD Range | SDD Coverage | @@ -1715,6 +1821,7 @@ Notes: | SAD-077, SAD-081, SAD-084, SAD-085, SAD-086 | Covered by `SDD-111` through `SDD-116` (Android voice audio backend — SRS-210..SRS-215). No dedicated SAD anchor exists for the platform-abstracted mobile-voice-audio-backend trait itself; SDD-111 is sourced from SAD-077 / SAD-081 pending a SAD revision that adds a dedicated allocation item. | | SAD-063 | Covered by `SDD-118` (Android `chanora_bridge` cdylib build automation — Gradle + `cargo-ndk` + per-ABI jniLibs staging). Cross-trace: SDD-073, SDD-105, SDD-109. | | SAD-061, SAD-062, SAD-087 | Covered by `SDD-119` (iOS / macOS `chanora_bridge` cdylib build automation — CocoaPods podspec + `cargo` + `lipo` + framework-layout shell, back-fill of existing code). Source SAD allocation: iOS half anchored to SAD-061 (iOS runtime) and SAD-062 (App Store / packaging); macOS half anchored to SAD-087 (macOS runtime baseline — deployment target isolation, universal-binary `lipo` packaging, `.framework` `Versions/A` layout, hand-rolled CocoaPods podspec automation), authored in SAD v0.9.7 to close the macOS-runtime anchor gap that this SDD unit had previously bridged by parallelism from SAD-061 / SAD-062. SDD-117 remains reserved-but-unauthored for the deferred `ios_voice_unit` trait back-fill noted by SDD-111. | +| SAD-088, SAD-089, SAD-090, SAD-091 | Covered by `SDD-120` (realtime-audio benchmark harness and advisory CI infrastructure — criterion-based bench harness, dhat-backed heap-allocation-count metric, SAD-089 baseline JSON post-processor, SAD-090 advisory PR-comment workflow with merge-base baseline read, SAD-091 `workflow_dispatch`-only baseline-update workflow that opens a PR rather than direct-pushing; simpler-form yellow-marker realization treats the SAD-089 baseline as the comparator for both red/green tolerance evaluation and yellow trending detection). Cross-trace: SDD-094 (the bench seam exercises the same `chanora_audio::engine` capture path whose lifecycle is specified by SDD-094). Suggested SWE.4 forward allocation: SWE4-UV-058 through SWE4-UV-062 (verification-engineer follow-up; not authored by this SDD unit). | ## Baseline Candidate 0.9.1 Update @@ -1776,6 +1883,12 @@ Notes: | 0.9.8 | 2026-05-18 | Reconciled Android P0 detailed design after Wave 1.5 SysRS / SRS / SAD propagation. SDD-073 (`AndroidBuildConfig`) updated in place (ID preserved) to record `minSdk = 28` per DEC-004 (superseding the prior API 24 baseline), `targetSdk` policy bound to SRS-188 Google Play floor, NDK ABI set pinned to `arm64-v8a / armeabi-v7a / x86_64`, release-signing CI assertion, R8/ProGuard stance, and AAB output per SRS-119 (delegating packaging pipeline detail to SDD-109). SDD-028 (`BackIntentService`) expanded from a one-line stub to an implementable specification covering API 33+ `OnBackInvokedCallback` registration, pre-33 `OnBackPressedDispatcher` fallback, deterministic route-pop ordering, and edge cases (root route → exit candidate, modal/dialog → close-only, active PTT → ignore). Added six new Android P0 SDD units: SDD-105 (`AndroidJniBootstrap` — back-fill for `JNI_OnLoad` / `initChanoraContext` / library-load location, sourced from SAD-081 / SAD-086); SDD-106 (`AndroidPermissionRequester` — RECORD_AUDIO runtime flow with listen-only fallback, settings deep-link for permanent denial, revocation handling, BridgeEvent surface, TransmitModeSelector clamp, sourced from SAD-085); SDD-107 (`AndroidVoiceForegroundService` — service class, manifest entry, notification channel id `chanora.voice.session`, `foregroundServiceType="microphone"` on API 30+, POST_NOTIFICATIONS request on API 33+, lifecycle bound to voice_join / voice_leave / shutdown_if_idle, START_NOT_STICKY on process death, ongoing-notification re-post on dismissal, sourced from SAD-086); SDD-108 (`AndroidAudioModeController` — `setMode(MODE_IN_COMMUNICATION)` with prior-mode snapshot/restore, ref-count semantics composable for P1, JNI surface from Rust audio engine, error paths including AudioManager-unavailable fallback, sourced from SAD-084 — back-fills the v0.9.8 product-decision-register entry); SDD-109 (`AndroidAaBuildPipeline` — bundletool config, ABI splits matching SDD-073, CI signing assertion, versionCode/versionName derivation, sourced from SAD-037 / SAD-063); SDD-110 (`AndroidPttCapability` — pins Android P0 to FocusedPttBackend only, no global key hook, no accessibility-service PTT, capability badge text under `ptt.capability.android.l0`, sourced from SAD-077). Updated §11 SAD-to-SDD coverage matrix to list the new Android P0 SDD range. Strict layered sourcing preserved (`SDD -> SAD` only). | +## Baseline Candidate 0.9.15 Update + +| Version | Date | Description | +| --- | --- | --- | +| 0.9.15 | 2026-05-18 | Added realtime-audio benchmark-infrastructure detailed-design unit SDD-120 (`RealtimeAudioBenchmarkHarnessAndAdvisoryCI`) sourced from SAD-088 / SAD-089 / SAD-090 / SAD-091 (the Wave 1.5 benchmark-infrastructure software-architecture chain authored at SAD v0.9.9 from the Option B scope decision propagated through SysRS-307/308/309 → SysDes-156/157/158 → SRS-216..219). SDD-120 is a single cohesive 11-section unit specifying: §1 crate selection (`criterion = "0.5"`, `dhat = "0.3"` dev-deps, three `[[bench]]` entries and two `[[bin]]` entries in `crates/chanora_audio/Cargo.toml`); §2 bench-file layout (three files `realtime_capture.rs` / `opus_codec.rs` / `resampler.rs` plus a shared `common.rs` synthetic-input module); §3 bench function signatures with explicit warmup (100-call pre-warm into `CaptureState::ingest` at `crates/chanora_audio/src/engine.rs:1207` before 1000-call dhat measurement window; criterion default warmup for the wall-clock / encode / decode / throughput benches), with the canonical `opus_decode_latency` metric pinned to `audiopus::coder::Decoder::decode_float` directly rather than `tsclientlib::AudioHandler::fill_buffer` because the latter conflates Opus decode with jitter-buffer dequeue and PCM mix; §4 deterministic synthetic-input generation (440 Hz sine at amplitude 0.5, no RNG, byte-stable across runs); §5 JSON post-processing via a Rust binary `crates/chanora_audio/benches/emit_baseline.rs` that parses criterion's `target/criterion/.../estimates.json` files and projects them into the SRS-217 schema; §6 `.github/workflows/bench-advisory.yml` (PR + push triggers, `dtolnay/rust-toolchain@stable` + `Swatinem/rust-cache@v2` matching the existing `ci.yml` pattern, audio system deps `libasound2-dev libpulse-dev pkg-config libopus-dev`, `fetch-depth: 0` for merge-base resolution, `git show $MERGE_BASE:…/x86_64-unknown-linux-gnu.json` baseline read with a missing-baseline short-circuit, `actions/github-script@v7` PR comment with sentinel-marker de-duplication and `pull_request`-only gating, `actions/upload-artifact@v4` for push-trigger reports, always exit 0 per SRS-218 clause 4); §7 `.github/workflows/bench-baseline-update.yml` (`workflow_dispatch` only — sole writer of SAD-089 per SAD-091, `peter-evans/create-pull-request@v6` opens a PR rather than direct-pushing so baseline updates are review-gated against accidental triggering or runner pollution); §8 yellow-marker realization selecting the SAD-090-permitted simpler form where the SAD-089 baseline JSON is the comparator for both red/green tolerance evaluation and yellow trending detection (no separate run-history artifact), with the comparator algorithm (`compare_baseline.rs`) pinned: 🔴 if `delta_pct > t`, 🟡 if `0.5*t < delta_pct ≤ t`, 🟢 otherwise; zero-tolerance metrics (heap allocation, SRS-219 clause a) collapse to 🔴/🟢 only; §9 cross-traces (SDD-094 audio-engine lifecycle as the bench-seam container; explicit NOT-cross-traces to SDD-118 Android bridge build automation and SDD-076..080 PTT capability surface, both correctly out of scope); §10 deferrals preserved (no telemetry export, no hard CI gate, no multi-host bench, no IDE integration, no Dart-side `flutter_rust_bridge` bench — researcher Tier B4); §11 suggested-forward SWE.4 allocation `SWE4-UV-058` through `SWE4-UV-062` (NOT authored at this layer — verification-engineer follow-up). Updated §11 SAD-to-SDD coverage matrix with a new SAD-088..SAD-091 row mapping to SDD-120. Strict layered sourcing preserved (`SDD -> SAD` only); no SRS, SysDes, SysRS, or SAD edits performed. Recommended next agent: `builder` (Step 6 of the benchmark chain — implements the three bench files, the `common.rs` helper, the two binary post-processors, the `Cargo.toml` dev-dependency + `[[bench]]` + `[[bin]]` additions, and the two workflow YAML files; strictly per this SDD unit). | + ## Baseline Candidate 0.9.14 Update | Version | Date | Description | diff --git a/docs/architecture/sysdes.md b/docs/architecture/sysdes.md index 4258256..81b0b9a 100644 --- a/docs/architecture/sysdes.md +++ b/docs/architecture/sysdes.md @@ -3,7 +3,7 @@ **Product name:** Chanora **Document type:** SysDes / System Architectural Design **Process alignment:** Automotive SPICE SYS.3 System Architectural Design -**Version:** 0.9.7 +**Version:** 0.9.8 **Status:** Baseline Candidate **Input baseline:** Chanora SysRS v0.6 **Target application type:** Cross-platform voice client application system @@ -2940,9 +2940,58 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through - Allocated SysRS: SysRS-002, SysRS-050, SysRS-193 - Notes: Parallel in role to SysDes-133 (iOS runtime baseline) and SysDes-135 (Android runtime baseline); neither SysDes-133 nor SysDes-135 is modified by this item. Source SysRS selection: SysRS-002 anchors the multi-platform obligation that explicitly enumerates macOS as a target client platform, SysRS-050 anchors the macOS runtime environment obligation for native desktop integration, and SysRS-193 anchors the signed/notarized macOS build obligation that the `.framework` `Versions/A` layout and universal-binary policy must satisfy in the release pipeline. No macOS-specific deployment-target or universal-binary SysRS item currently exists; if a finer-grained macOS minimum-runtime SysRS item is required (analogous to SysRS-286 for iOS and SysRS-288 for Android), the systems-requirements owner should consider authoring it in a follow-up — this SysDes item is structured so that such a future SysRS item can be added to `Allocated SysRS` without restructuring the element. SysDes-155 cites the podspec file path only and does not embed its current deployment-target value; the value itself remains owned by the Build Configuration subsystem. +**SysDes-156**: The system architecture shall allocate to SE-13 (Audio Subsystem) the obligation to expose its realtime capture and playback paths to benchmark instrumentation, such that the maintained numeric performance baselines authorized by SysRS-307 are measurable as a deterministic, automated comparison surface. The metric set authored at this layer is: (i) **heap allocation count per realtime callback after warmup** — zero allocations are expected on the steady-state realtime audio thread, where warmup is defined as the first N callbacks with N implementation-specific (recommended N=100 callbacks; refined at SDD); (ii) **per-callback wall-clock time expressed as a fraction of the cpal stream's reported audio frame period**, captured and reported as p50, p95, and p99 of the callback wall-clock as a fraction of that period; (iii) **Opus encode latency** — end-to-end wall-clock time for `encoder.encode_float()` on a 960-sample (20 ms) frame, captured per call; (iv) **Opus decode latency** — same shape, decoder side; (v) **resampler throughput** in samples-per-second produced at the canonical rate pairs 44.1 kHz → 48 kHz, 16 kHz → 48 kHz, and 48 kHz → 48 kHz passthrough (the passthrough pair serves as a control point). The baseline storage format is declared at this layer as structured JSON committed to a deterministic location in the repository; the exact path is delegated to SDD. Each JSON record shall include the metric value, the host architecture, the toolchain version, the git SHA of the measurement, and a timestamp, so that a baseline snapshot is reproducible and traceable to the commit that produced it. This SysDes item does not authorize off-device transmission of any measured baseline data and is consistent with SysRS-295. + +- Status: Baseline Candidate +- Type: System Element Allocation / Performance Verification Surface +- Stage: P0 / MVP +- Allocated to: SE-13 (Audio Subsystem) — owns the realtime path being measured; the realtime callback, the Opus encode/decode paths, and the resampler are all SE-13 surfaces +- Downstream artifact: SRS, SAD, SDD, Verification +- Verification method: Automated Benchmark Execution (criterion crate per SDD); Inspection of the JSON baseline records committed to the default branch +- Verification owner: Audio / Platform QA +- ASPICE SYS.3 alignment: Element allocation, dynamic architecture, resource and performance design, verification handoff +- Allocated SysRS: SysRS-307 (primary derivation); refines SysRS-180 / SysRS-181 / SysRS-186 from prescriptive intent into a measurable contract surface +- Notes: The numeric warmup constant (recommended N=100), the exact JSON repository path, the benchmark crate selection (recommendation: criterion), and the harness wiring against the cpal stream are SDD concerns and are not authored here. The cpal stream's reported period referenced in metric (ii) is the stream period reported by the existing audio capture/playback path owned by SE-13; this SysDes item does not authorize a new platform integration. The control-point passthrough at 48 kHz → 48 kHz in metric (v) provides a zero-arithmetic baseline that isolates harness overhead from resampling cost. Metrics (i) and (v) are the structural-property metrics (no acceptable variance for allocations; throughput is a steady-state measurement); metrics (ii)–(iv) are latency-distribution metrics whose tolerance characterisation is the subject of SysDes-158. + +**SysDes-157**: The system architecture shall allocate to SE-18 (Deployment and Release Environment) the integration of the realtime-audio benchmark suite (SysDes-156) into the existing CI workflow as an **advisory** regression-reporting surface, satisfying SysRS-308. The design constraints authored at this layer are: (1) **Trigger** — the benchmark workflow shall execute on every pull request opened against the default branch and on every merge to the default branch; it shall not be triggered on tags and shall not be triggered on non-default-branch pushes. (2) **Host scope** — the benchmark workflow shall execute on the Linux x86_64 GitHub Actions runner (`ubuntu-latest`); other host architectures are explicitly out of scope for this revision and shall not be added by SDD or by implementation without a future SysRS clause authorising multi-host benchmarking. (3) **Surface** — the result shall be rendered on the PR status-check view as a named check (suggested name "Benchmark / advisory" or equivalent; the exact check name is delegated to SDD) and shall additionally be rendered as a markdown table posted to the PR conversation, comparing each metric against the most recent baseline on the default branch with the percentage delta and a clear visual marker (e.g., 🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance). (4) **Non-blocking semantics** — the status check shall **never** report a `failure` status; on regression the check shall report `success` (so it does not block merge) and the regression shall be surfaced exclusively in the PR comment for human reviewer attention. This clause is the SysDes-layer enforcement of SysRS-308's advisory-only contract; any future escalation to a build-failing hard gate requires a separate SysRS authorisation and is out of scope of SysDes-157. (5) **Baseline source** — the comparison input shall be read from the deterministic JSON location on the default branch committed under SysDes-156; on the first run after a new benchmark is added, the baseline file may not yet contain that metric, in which case the report shall state "no baseline; this run establishes the baseline candidate" and shall not produce a comparison delta for that metric. (6) **Baseline update isolation** — a separate, manually-invoked CI workflow (GitHub Actions `workflow_dispatch`) shall be the sole mechanism that writes the baseline JSON on the default branch; the PR-triggered benchmark workflow shall never write the baseline file. This isolation guarantees that baseline updates are intentional, reviewable acts and prevents silent baseline drift from PR runs. + +- Status: Baseline Candidate +- Type: System Element Allocation / CI Integration Architecture +- Stage: P0 / MVP +- Allocated to: SE-18 (Deployment and Release Environment) — owns CI/CD per SysDes-036; the CI workflow definitions, the GitHub Actions runner selection, the PR status-check surface, the PR comment rendering, and the baseline-update workflow all live here +- Downstream artifact: SRS, SAD, SDD, Verification +- Verification method: Demo (a CI run on a synthetic PR demonstrates the markdown table report, the visual markers, the non-blocking `success` status on a simulated regression, and the "no baseline" fallback on a metric with no prior baseline) +- Verification owner: Release / Operations QA, Audio / Platform QA (co-verification of the report content) +- ASPICE SYS.3 alignment: Element allocation, deployment architecture, verification handoff +- Allocated SysRS: SysRS-308 (primary derivation); composes with SysRS-307 (baselines being compared against) and SysRS-309 (tolerance window applied by the comparison, refined by SysDes-158); consistent with SysRS-295 (no off-device transmission beyond the existing CI provider surface visible to repository collaborators) and with the SysRS-234..239 verification-family pattern +- Notes: The exact YAML workflow filenames, the exact named status-check string, the exact markdown layout of the comparison table, the exact emoji set, and the exact triggering criteria for the 🟡 "trending" state are SDD concerns and are not authored here. The merge-base baseline-snapshot semantics of the comparison are authored separately in SysDes-158. The Linux x86_64 host-scope clause is the active rule that explicitly defers ARM Android, macOS Apple Silicon, Windows x86_64, and any other host runner; that deferral is preserved here intentionally. Composes with SysDes-036 (CI/CD as a system element) as the SE-18 integration anchor. + +**SysDes-158**: The system architecture shall allocate to SE-18 (Deployment and Release Environment) the per-metric tolerance window and the baseline-comparison methodology used by the advisory CI surface (SysDes-157), satisfying SysRS-309. The tolerance window is authored at this layer as a per-metric set rather than as a single global value, because the metrics defined in SysDes-156 have different statistical character and require different comparison rules: + +| Metric (per SysDes-156) | Tolerance window | Comparison rule | Rationale | +|---|---|---|---| +| Heap allocation count per realtime callback after warmup | **Zero** | Any non-zero allocation count is reported as a regression (🔴) | Structural property of the code, not a statistical measurement; no acceptable variance | +| Per-callback wall-clock as a fraction of the audio frame period | **+20% on p95** | Compare current-run p95 against baseline p95; report regression if delta exceeds +20%. p50 and p99 are reported but not compared against tolerance (advisory data only) | p95 is the operational tail; p50 is too lenient and p99 is too noisy to gate on at MVP | +| Opus encode latency (960-sample / 20 ms frame) | **+15% on mean** | Compare current-run mean against baseline mean; report regression if delta exceeds +15% | Codec encode latency is a tight loop with low variance; mean is a stable comparator | +| Opus decode latency (same frame shape) | **+15% on mean** | Same shape as encode | Symmetric with encode | +| Resampler throughput (samples/second) at 44.1→48, 16→48, 48→48 passthrough | **−10% on samples/second** (lower is worse) | Compare current-run samples/sec against baseline samples/sec; report regression if delta is below −10% | Throughput regression is a *drop* in samples/sec, not an *increase*; sign convention is inverted relative to the latency metrics | + +The **comparison methodology** authored at this layer is: each PR-triggered benchmark run shall be compared against the most recent baseline committed to the default branch *at the time of the PR's merge-base commit*, not against the current tip of the default branch. This ensures PR comparisons are stable as the default branch advances during the PR's lifetime, and it also means that rebasing a PR onto a newer default-branch base can change which baseline snapshot is used as the comparator. The +20% starting value recorded as non-binding guidance in SysRS-309 is honoured here only for the per-callback wall-clock metric (and is bound to p95); the other metric tolerances are authored independently of the SysRS-309 suggestion, as authorised by SysRS-309's "single declared value (or one declared value per metric)" allowance. + +- Status: Baseline Candidate +- Type: System Element Allocation / Performance Comparison Methodology +- Stage: P0 / MVP +- Allocated to: SE-18 (Deployment and Release Environment) — the comparison logic lives in CI tooling co-located with the workflow authored in SysDes-157 +- Downstream artifact: SRS, SAD, SDD, Verification +- Verification method: Review (the per-metric tolerance values are subject to periodic review; SDD ratifies the values as authored here and may refine the harness implementation; future SAD/SDD revisions may re-tune the values as baseline noise characteristics are observed empirically) +- Verification owner: Release / Operations QA, Audio / Platform QA +- ASPICE SYS.3 alignment: Element allocation, architecture decision / rationale, resource and performance design, verification handoff +- Allocated SysRS: SysRS-309 (primary derivation); composes with SysRS-307 (the baselines being compared) and SysRS-308 (the advisory surface that consumes this comparison) +- Notes: The selection of p95 (rather than p99) as the gated tail for the wall-clock metric is an explicit architectural decision recorded here: p99 is reported for diagnostic value but is intentionally not tolerance-gated at MVP because the sample population of a single CI run is too small for p99 to be a stable comparator. The merge-base baseline-snapshot rule is the active comparison methodology; an alternative methodology comparing against the latest default-branch baseline (rather than the merge-base baseline) was considered and intentionally rejected because it produces unstable PR comparisons when the default branch advances during the PR's lifetime. Re-tuning of the per-metric tolerance values may be performed by future SAD or SDD revisions without requiring a new SysRS clause, provided the values remain numeric and per-metric (the structural constraints authored here); a change from the per-metric model to a global single value, or vice versa, would require a SysRS-309 revision. + ## 17. Updated SysRS Coverage Statement -This SysDes version covers all known SysRS requirements from `SysRS-001` through `SysRS-306`. +This SysDes version covers all known SysRS requirements from `SysRS-001` through `SysRS-309`. | SysRS Range | SysDes Coverage | |---|---| @@ -2952,6 +3001,7 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through | SysRS-303 through SysRS-304 | Covered by `SysDes-149` through `SysDes-151` | | SysRS-305 through SysRS-306 | Covered by `SysDes-152` through `SysDes-154` (with SysDes-154 additionally refining SysRS-055 for the Android voice audio backend) | | SysRS-002 / SysRS-050 / SysRS-193 (macOS platform-baseline slice) | Additionally refined by `SysDes-155` (macOS runtime baseline, parallel to SysDes-133 / SysDes-135) | +| SysRS-307 through SysRS-309 (realtime-audio benchmark infrastructure) | Covered by `SysDes-156` (audio benchmark surface, SE-13), `SysDes-157` (advisory CI integration, SE-18), and `SysDes-158` (per-metric tolerance window and comparison methodology, SE-18) | ## Baseline Candidate 0.9.1 Update @@ -2993,4 +3043,5 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through | Version | Date | Description | |---|---|---| +| 0.9.8 | 2026-05-18 | Added benchmark-infrastructure SysDes derivations SysDes-156, SysDes-157, and SysDes-158 sourced from SysRS-307 / SysRS-308 / SysRS-309 (Wave 1.5 benchmark chain, Step 2). SysDes-156 allocates the realtime-audio benchmark instrumentation surface (heap allocation count per realtime callback after warmup, per-callback wall-clock as a fraction of the cpal stream period at p50/p95/p99, Opus encode latency on a 960-sample / 20 ms frame, Opus decode latency on the same shape, and resampler throughput at 44.1→48 kHz, 16→48 kHz, and 48→48 kHz passthrough as a control) to SE-13 (Audio Subsystem); declares baseline storage as structured JSON committed to a deterministic repo location (exact path delegated to SDD) capturing metric value, host architecture, toolchain version, git SHA, and timestamp. SysDes-157 allocates the advisory CI integration to SE-18 (Deployment and Release Environment): trigger on every PR against the default branch + every merge to default (not on tags, not on non-default-branch pushes), host = Linux x86_64 (`ubuntu-latest`), surface = PR status-check view rendered as a markdown table with per-metric delta and a visual marker (🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance), strict non-blocking semantics (the check shall never report `failure` status — even on regression it reports `success` and surfaces the regression in the PR comment for human reviewer attention), baseline source = JSON on the default branch with first-run-establishes-candidate fallback, and baseline-update isolation via a separate `workflow_dispatch` workflow (PR-triggered workflow shall never write the baseline file). SysDes-158 authors the per-metric tolerance window and the comparison methodology: heap allocations tolerance = zero (any non-zero count is 🔴), per-callback wall-clock tolerance = +20% on p95 (p50 and p99 are advisory data only), Opus encode/decode latency tolerance = +15% on mean, resampler throughput tolerance = −10% on samples/second; comparison is against the most recent baseline committed to the default branch at the PR's merge-base. Allocated to SE-18 (comparison logic lives in CI tooling). Strict layered sourcing preserved (`SysDes -> SysRS` only). Explicitly **NOT** authored in this update, consistent with the SysRS-307/308/309 deferral: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-307/308/309 do not authorize off-device transmission and SysRS-295 still applies); (b) build-failing hard CI gate (SysRS-308's advisory-only clause is the active rule — SysDes-157's non-blocking semantics enforces it at this layer); (c) multi-host benchmarking (ARM Android, macOS Apple Silicon, etc. — SysDes-157's Linux x86_64 host-scope clause is the active rule). | | 0.9.7 | 2026-05-18 | Added dedicated macOS runtime baseline element allocation SysDes-155, parallel in role to SysDes-133 (iOS runtime baseline) and SysDes-135 (Android runtime baseline). SysDes-155 anchors the macOS deployment-target policy as a single-source-of-truth concern (citing the podspec file path `apps/chanora_flutter/macos/chanora_bridge.podspec` without embedding its value), the universal-binary (`lipo`'d `arm64` + `x86_64`) packaging policy, the `.framework` `Versions/A` bundle layout convention, and CocoaPods podspec integration as the auto-build mechanism for the Rust cdylib. Allocated to macOS Build Configuration, Platform Adapter Layer (macOS Platform Services), Release Pipeline, and Verification. Source SysRS: SysRS-002 (multi-platform target client platforms including macOS), SysRS-050 (macOS runtime environment / native desktop window integration), SysRS-193 (signed/notarized macOS builds). SysDes-155 is the proper SysDes anchor for SAD-087 and for downstream SRS items that allocate macOS-runtime responsibility, and is the recommended retarget for SAD-087's previously-open Source SysDes follow-up. SysDes-133 and SysDes-135 are unchanged. Strict layered sourcing preserved (`SysDes -> SysRS` only). Follow-up recommendation: systems-requirements owner may consider authoring a finer-grained macOS minimum-runtime SysRS item parallel to SysRS-286 (iOS) and SysRS-288 (Android); SysDes-155 is structured so that such a future SysRS item can be added to its `Allocated SysRS` list without restructuring the element. | diff --git a/docs/requirements/srs.md b/docs/requirements/srs.md index db7f5d3..6b11506 100644 --- a/docs/requirements/srs.md +++ b/docs/requirements/srs.md @@ -3,7 +3,7 @@ **Product name:** Chanora **Document type:** SRS / Software Requirements Specification **ASPICE process alignment:** SWE.1 Software Requirements Analysis -**Version:** 0.9.8 +**Version:** 0.9.9 **Status:** Baseline Candidate **Source baseline:** `chanora_SysDes_ASPICE_SYS3_v0.5.md` **Architecture baseline:** Flutter + Rust Core + `tsclientlib` protocol adapter @@ -2743,6 +2743,75 @@ This subsection groups the Android voice-audio-backend software requirements der | SysDes-135 (Android platform baseline) | Retained as the secondary/context anchor on `SRS-208` through `SRS-215` (the eight Android voice-audio SRS items conceptually rest on the Android platform baseline as their platform context, but no longer take SysDes-135 as the primary derivation). Upstream SysRS items addressed across the eight items are SysRS-055, SysRS-217, SysRS-288, SysRS-305, and SysRS-306. | +## 20. Realtime Audio Benchmark Infrastructure (P0) + +This section authors the software-requirements layer of the realtime-audio benchmark infrastructure derived strictly from the SysDes-156 / SysDes-157 / SysDes-158 chain (per SRS-005 strict-layered sourcing). The SysDes anchors are themselves a refinement of SysRS-307 / SysRS-308 / SysRS-309, but the SRS layer does not source upstream of SysDes by policy. + +**SRS-216**: The `chanora_audio` software shall expose its realtime capture and playback paths to benchmark instrumentation that covers the five metrics enumerated by SysDes-156: (1) heap allocation count per realtime callback after a warmup window of approximately 100 callbacks (the steady-state expectation is zero allocations on the realtime thread); (2) per-callback wall-clock duration reported at the p50, p95, and p99 percentiles, expressed as a fraction of the active cpal stream period; (3) Opus encode latency measured on the canonical 960-sample / 20 ms voice frame shape; (4) Opus decode latency measured on the same 960-sample / 20 ms frame shape; and (5) resampler throughput measured at the canonical rate pairs 44.1 → 48 kHz, 16 → 48 kHz, and 48 → 48 kHz (the last serving as a passthrough control). The instrumentation shall be authored as Rust benchmarks located in a per-crate `benches/` directory (the choice of benchmark harness crate — `criterion` or equivalent — is a downstream SDD concern and is not authored here). The instrumentation shall consume the existing realtime-path public surface (for example `CaptureState::ingest`, already public at the chanora_audio crate boundary per the Wave 4 Tier A1 work) without introducing new public seams in production code; if any additional seam is required, that decision is delegated to SAD/SDD and is not authored at the SRS layer. + +- Status: Baseline Candidate +- Type: Verification Infrastructure Requirement +- Stage: P0 / MVP +- Allocated to: `chanora_audio` crate (realtime capture and playback modules, plus a co-located `benches/` directory) +- Source SysDes: SysDes-156 (realtime-audio benchmark surface allocated to SE-13) +- Verification method: SWE.4 — automated benchmark execution against the instrumentation surface; reviewers confirm the five metric families enumerated in SysDes-156 each have at least one benchmark target. +- Acceptance criteria: A `benches/` directory exists under the `chanora_audio` crate (or the SDD-decided per-crate equivalent); the directory contains benchmark targets that exercise the realtime capture callback, the realtime playback callback, the Opus encoder on a 960-sample / 20 ms frame, the Opus decoder on a 960-sample / 20 ms frame, and the resampler at each of the three canonical rate pairs; each benchmark target reports a numeric value in the unit appropriate to its metric family (count, fraction-of-period, ns/iter, or samples/sec); the benchmarks run to completion on a Linux x86_64 host in under the wall-clock budget allowed by SysDes-157's host scope (see SRS-218). +- Analysis: Feasible. The realtime path already exposes the entry points required for benchmark instrumentation per the chanora_audio crate boundary as of Wave 4 Tier A1; no production-code seam authoring is required at the SRS layer. Verification intent (SWE.6): demonstrate that all five metric families have at least one benchmark target and that the benchmarks execute deterministically on the Linux x86_64 host enumerated by SysDes-157. +- Unresolved assumptions: The exact benchmark harness crate (`criterion` is the conventional choice on the Rust ecosystem but is not authored here) and the exact target-naming convention are SDD concerns. If the realtime path requires a new public seam to be benchmarkable at SRS granularity, that gap will be discovered by the SAD/SDD pass and routed back to a future SRS revision; this clause does not authorise such a seam in production code. + +**SRS-217**: Baseline measurements produced by the SRS-216 benchmark instrumentation shall be stored as structured JSON committed to a deterministic location inside the repository, so that the baseline becomes a reviewable, version-controlled artefact rather than an ephemeral CI side effect. Each baseline record shall contain at minimum the following fields: `metric` (string identifier matching the metric name enumerated by SysDes-156), `value` (numeric — the measured value for that metric), `unit` (string — for example `"ns/iter"`, `"samples/sec"`, `"fraction"`, or `"count"`, matching the metric family), `host_arch` (string — the Rust target triple of the measurement host, for example `"x86_64-unknown-linux-gnu"`), `toolchain` (string — the rustc version active at measurement time), `git_sha` (string — the full Git commit SHA at measurement time), and `timestamp` (string — ISO-8601 UTC). The canonical on-disk path for the baseline JSON (for example `benches/baselines/.json` as a suggestion) is delegated to SDD and is not fixed at the SRS layer; however, the path shall be deterministic (the same crate at the same commit shall write to the same file) and shall be committed to the repository on the default branch. + +- Status: Baseline Candidate +- Type: Verification Infrastructure Requirement +- Stage: P0 / MVP +- Allocated to: `chanora_audio` crate (the bench harness publishes the baseline JSON), plus the SDD-decided on-disk path under the repository root. +- Source SysDes: SysDes-156 (baseline storage as structured JSON in a deterministic repo location capturing metric value, host architecture, toolchain version, git SHA, and timestamp). +- Verification method: Review — the JSON record format is review-checkable against this clause by inspecting a sample baseline file. +- Acceptance criteria: A baseline JSON file exists at the SDD-decided deterministic path on the default branch; the file is parseable as JSON; each record carries the seven fields enumerated above; the `metric` field values match the metric names declared by SysDes-156; the `host_arch`, `toolchain`, `git_sha`, and `timestamp` fields are populated with values that correspond to the measurement host and commit; the same crate at the same commit deterministically writes to the same file path. +- Analysis: Feasible. JSON is the conventional serialization format for benchmark-baseline metadata in Rust ecosystem tooling, and the fields enumerated are the minimum needed for the SysDes-157 advisory comparison to be reproducible. Verification intent (SWE.6): inspect a sample baseline file and confirm the seven fields are populated correctly and the file is parseable. +- Unresolved assumptions: The exact on-disk path is delegated to SDD per SysDes-156. The schema may evolve (additional optional fields) without breaking this clause provided the seven enumerated fields remain present. + +**SRS-218**: A continuous-integration workflow shall execute the SRS-216 benchmark suite under the constraints authored by SysDes-157, namely: (1) the workflow shall trigger on every pull request opened against the default branch and on every merge to the default branch, and shall not trigger on tag pushes or on pushes to non-default branches; (2) the workflow shall execute on the Linux x86_64 GitHub Actions runner (`ubuntu-latest`) and shall not execute on any other host architecture (multi-host benchmarking is out of scope for this revision per SysDes-157 point 2); (3) the workflow result shall be rendered as a markdown table on the pull-request surface that compares each metric enumerated by SysDes-156 against the baseline-at-merge-base (per SRS-219), with a per-metric visual marker drawn from the set { 🟢 within tolerance, 🟡 within tolerance but trending, 🔴 outside tolerance } evaluated against the per-metric tolerance window of SysDes-158; (4) the workflow shall report a `success` status check regardless of whether any metric is flagged 🔴, so that the advisory check never blocks merge (this enforces the SysDes-157 point 4 non-blocking semantics at the SRS layer); and (5) the baseline-update operation shall be a separate, manually-invoked workflow (GitHub Actions `workflow_dispatch` or equivalent), and the PR-triggered workflow authored by this clause shall never write to the baseline JSON file authored under SRS-217. + +- Status: Baseline Candidate +- Type: Verification Infrastructure Requirement +- Stage: P0 / MVP +- Allocated to: CI tooling under `.github/workflows/` (or the SDD-decided equivalent integration point) — specifically a new advisory benchmark workflow file; the exact filename is delegated to SAD/SDD. +- Source SysDes: SysDes-157 (advisory CI integration allocated to SE-18, with the six design constraints on trigger, host, surface, non-blocking semantics, baseline source, and baseline-update isolation). +- Verification method: Demo — a synthetic pull-request run shall demonstrate the workflow firing, the markdown comparison table rendering, the status check reporting `success` even when a 🔴 marker is present, and the absence of any write to the baseline JSON file under the PR-triggered path. Human reviewers confirm in the PR. +- Acceptance criteria: A CI workflow file exists at the SDD-decided path; the file's trigger configuration matches SysDes-157 point 1 (PR-against-default plus merge-to-default, no tags, no non-default-branch pushes); the workflow's `runs-on` clause is `ubuntu-latest`; a PR run posts a markdown table that lists each SysDes-156 metric with its measured value, the baseline value at the PR's merge-base, the percentage delta, and one of the three visual markers; the workflow's status check reports `success` on every run, including runs that contain at least one 🔴 marker; a separate `workflow_dispatch` workflow file exists for the baseline-update operation, and the PR-triggered workflow does not modify the baseline JSON. +- Analysis: Feasible on GitHub Actions. The advisory-only, never-block-on-failure semantics is the active rule per SysRS-308 (inherited via SysDes-157); escalating this workflow to a hard build-failing gate would require a future SysRS authorisation and is not within the scope of SRS-218. Verification intent (SWE.6): observe a synthetic PR run end-to-end and confirm the five constraints (trigger, host, surface, non-blocking semantics, baseline-update isolation). +- Unresolved assumptions: The exact workflow filename, the exact named status-check string, and the exact markdown table layout are SAD/SDD concerns. The triggering criterion for the 🟡 "trending" state is delegated to SDD per SysDes-157's notes (this clause only requires that the marker set { 🟢, 🟡, 🔴 } is present, not how 🟡 is computed). + +**SRS-219**: The advisory comparison performed by the SRS-218 workflow shall apply the per-metric tolerance window and the comparison methodology authored by SysDes-158. The tolerance window shall be evaluated per metric as follows: (a) heap allocation count per realtime callback after warmup — tolerance is zero, i.e. any non-zero allocation count on a PR is flagged 🔴; (b) per-callback wall-clock duration at p95 — tolerance is +20% relative to the baseline (p50 and p99 are advisory data only and do not drive the 🔴 marker); (c) Opus encode latency on a 960-sample / 20 ms frame — tolerance is +15% relative to the baseline on the mean; (d) Opus decode latency on a 960-sample / 20 ms frame — tolerance is +15% relative to the baseline on the mean; (e) resampler throughput on the canonical rate pairs — tolerance is −10% relative to the baseline on samples-per-second (i.e. a throughput drop greater than 10% is flagged 🔴). The comparator on a pull-request run shall be the baseline JSON committed at the pull request's merge-base commit, **not** the baseline at the current default-branch tip; this disambiguates the comparison when a PR is rebased onto a newer default and prevents a default-branch baseline update from retroactively re-classifying an in-flight PR's regression status. + +- Status: Baseline Candidate +- Type: Verification Infrastructure Requirement +- Stage: P0 / MVP +- Allocated to: CI tooling co-located with the SRS-218 workflow (the comparison logic resides in the same SE-18 surface). +- Source SysDes: SysDes-158 (per-metric tolerance window and merge-base-snapshot comparison methodology allocated to SE-18). +- Verification method: Review — the comparison logic and the tolerance table are review-checkable against this clause; a small synthetic data set (one baseline, one PR run for each of the five metric families with values at, just inside, and just outside the tolerance window) demonstrates each marker assignment. +- Acceptance criteria: The CI tooling reads the baseline JSON at the PR's merge-base commit (verifiable by observing the SHA the tooling resolves before reading the file); the comparison applies the per-metric thresholds enumerated above (heap = zero, p95 wall-clock = +20%, Opus encode mean = +15%, Opus decode mean = +15%, resampler samples/sec = −10%); the marker assignment for a synthetic value at the threshold, just inside the threshold, and just outside the threshold matches the expected 🟢 / 🟡 (where defined by SDD) / 🔴 result; rebasing a PR onto a newer default branch re-resolves the merge-base and re-reads the baseline at the new merge-base, rather than reading the default-branch-tip baseline. +- Analysis: Feasible. The merge-base resolution is a standard `git merge-base` operation; per-metric thresholds are simple numeric comparisons. The merge-base-snapshot semantics is the SysDes-158 active rule and is preferred over default-branch-tip comparison because it makes the comparator stable across rebases. Verification intent (SWE.6): observe each of the five metric families' threshold behaviour on synthetic data and confirm the merge-base SHA is the comparator anchor. +- Unresolved assumptions: None at the SRS layer. The exact representation of the 🟡 "trending" intermediate state (for example a sub-threshold band, or a multi-run trend) is delegated to SDD per SysDes-157's notes, but does not alter the 🔴 threshold rules authored here. + +### 20.1 SRS-216..219 to SysDes Coverage + +| SysDes anchor | SRS coverage | +|---|---| +| SysDes-156 (realtime-audio benchmark surface, SE-13; five metric families; baseline JSON in deterministic repo location with metric/value/unit/host_arch/toolchain/git_sha/timestamp fields) | Primary anchor for `SRS-216` (benchmark instrumentation covering the five metric families) and `SRS-217` (baseline JSON storage format). | +| SysDes-157 (advisory CI integration, SE-18; trigger on PR + merge-to-default; host = `ubuntu-latest`; markdown table with 🟢/🟡/🔴 markers; never-fail status check; baseline-update isolation via `workflow_dispatch`) | Primary anchor for `SRS-218` (CI advisory-reporting workflow). | +| SysDes-158 (per-metric tolerance window and merge-base-snapshot comparison methodology, SE-18; heap = 0, p95 wall-clock = +20%, Opus encode/decode mean = +15%, resampler samples/sec = −10%) | Primary anchor for `SRS-219` (tolerance window and comparison methodology binding). | + +### 20.2 Intentionally Out of Scope at This Revision + +Consistent with the SysRS-307 / SysRS-308 / SysRS-309 deferrals propagated through SysDes-156 / SysDes-157 / SysDes-158, this revision intentionally does **not** author the following SRS clauses, and they shall not be inferred from SRS-216 through SRS-219: + +- **Dimension 3 production telemetry export** — the runtime-export-of-metrics-from-shipped-clients dimension is deferred to P1 by SysRS, is not authorised by any SysDes-156/157/158 clause, and remains subject to SysRS-295's no-off-device-transmission rule. No SRS clause in this revision authorises such export. +- **Build-failing hard CI gate** — SysRS-308's advisory-only contract is the active rule, enforced at the SysDes layer by SysDes-157's non-blocking semantics. SRS-218 explicitly inherits that contract by requiring the status check to report `success` on every run. Escalation to a hard gate requires a future SysRS authorisation. +- **Multi-host benchmarking** — SysDes-157 point 2 fixes the host to Linux x86_64 (`ubuntu-latest`). ARM Android, macOS Apple Silicon, Windows x86_64, and any other host runner are explicitly deferred. SRS-218 inherits this constraint. + + ## Baseline Candidate 0.9.1 Update | Version | Date | Description | @@ -2775,6 +2844,7 @@ This subsection groups the Android voice-audio-backend software requirements der | Version | Date | Description | |---|---|---| +| 0.9.9 | 2026-05-18 | Added realtime-audio benchmark-infrastructure software requirements SRS-216 through SRS-219 in a new section 20, sourced strictly from SysDes-156 / SysDes-157 / SysDes-158 (the Option B benchmark-infrastructure scope decision propagated through the Wave 1.5 benchmark chain). SRS-216 allocates benchmark instrumentation covering the five SysDes-156 metric families (heap allocation count per realtime callback after ~100-callback warmup, per-callback wall-clock at p50/p95/p99 as a fraction of cpal stream period, Opus encode latency on 960-sample/20 ms frame, Opus decode latency on the same shape, and resampler throughput at 44.1→48 / 16→48 / 48→48 kHz) to the `chanora_audio` crate's `benches/` directory; no new public seam in production code is authored. SRS-217 authors the baseline-JSON storage format (`metric`, `value`, `unit`, `host_arch`, `toolchain`, `git_sha`, `timestamp` fields at minimum) committed to a deterministic SDD-decided repo path. SRS-218 authors the CI advisory-reporting workflow under SysDes-157 (trigger on PR-against-default + merge-to-default, host = `ubuntu-latest`, markdown table with 🟢/🟡/🔴 markers, never-fail status check, baseline-update isolated to a separate `workflow_dispatch` workflow). SRS-219 binds the per-metric tolerance window of SysDes-158 (heap = 0, p95 wall-clock = +20%, Opus encode/decode mean = +15%, resampler samples/sec = −10%) and the merge-base-snapshot comparison methodology. Intentionally NOT authored consistent with the SysRS deferral chain: (a) Dimension 3 production telemetry export (deferred to P1; SysRS-295 still applies); (b) build-failing hard CI gate (SysRS-308 advisory-only rule active, SysDes-157 non-blocking semantics inherited); (c) multi-host benchmarking (SysDes-157 Linux x86_64 host-scope clause active). Strict layered sourcing preserved (`SRS -> SysDes` only per SRS-005); SysRS-307/308/309 are not cited as direct SRS sources. | | 0.9.8 | 2026-05-18 | Retargeted the `Source SysDes` field on SRS-208, SRS-209, and SRS-210 through SRS-215 from the generic SysDes-135 (Android platform baseline) to the dedicated Android voice-audio derivation items authored by the system-architecture owner: SysDes-152 (Android in-call audio mode subsystem) is now the primary anchor for SRS-208; SysDes-153 (Android RECORD_AUDIO runtime permission acquisition flow as Permission/Voice subsystem partitioning) is now the primary anchor for SRS-209; SysDes-154 (Android voice audio backend subsystem — AAudio low-latency path, voice-communication usage/preset, hardware-effects engagement with software fallback, sharing-mode policy, FGS-hosted background mic lifecycle) is now the primary anchor for SRS-210 (latency), SRS-211 (input preset), SRS-212 (hardware AEC/NS/AGC), SRS-213 (output usage/content-type), SRS-214 (sharing mode), and SRS-215 (microphone-typed foreground service). SRS-213 additionally retains SysDes-152 as a secondary anchor because the output-stream usage is the precondition for in-call-mode routing. SysDes-135 is retained on all eight items as the secondary/context anchor (platform-baseline relationship). Section 19 coverage table updated. Strict layered sourcing preserved (`SRS -> SysDes` only). No technical or behavioural content of SRS-208 through SRS-215 was modified; this is a surgical retargeting of the `Source SysDes` field only. | | 0.9.7 | 2026-05-18 | Added Android Audio Backend (P0) section 18 hosting SRS-210 through SRS-215 to support the platform-adapter decision to host Android voice capture/playback on a dedicated low-latency native audio backend (mirroring the iOS voice-processing-audio-unit precedent). SRS-210 sets numeric mouth-to-ear latency targets (≤150 ms on devices that return `AAUDIO_PERFORMANCE_MODE_LOW_LATENCY`, ≤250 ms otherwise). SRS-211 mandates `AAUDIO_INPUT_PRESET_VOICE_COMMUNICATION` with `VOICE_PERFORMANCE` fallback. SRS-212 mandates engagement of hardware `AcousticEchoCanceler` / `NoiseSuppressor` / `AutomaticGainControl` on the capture session id with software-AEC fallback (closes the prior silent no-op). SRS-213 mandates `AAUDIO_USAGE_VOICE_COMMUNICATION` + `AAUDIO_CONTENT_TYPE_SPEECH` on output. SRS-214 mandates best-effort `AAUDIO_SHARING_MODE_EXCLUSIVE` with `SHARED` fallback. SRS-215 mandates microphone-typed foreground-service hosting (`foregroundServiceType="microphone"`) for backgrounded voice transmission. Added section 19 coverage statement. Strict SRS→SysDes sourcing preserved: `Source SysDes` traces via SysDes-135 (Android platform baseline) pending a SysDes revision that introduces a dedicated derivation item for the Android voice audio backend; SysRS-055, SysRS-217, SysRS-288, SysRS-305 are cited as the upstream SysRS obligations addressed. | | 0.9.6 | 2026-05-17 | Propagated SysRS reconciliation (Wave 1.5) into SRS for P0 Android. SRS-187 minimum Android API raised from 24 to 28 per DEC-004 (ID preserved, change record added). SRS-188 verified consistent with SysRS-289 (Google Play upload-date target SDK); no rewrite required. SRS-111 (Android foreground service for active voice session), SRS-119 (Android AAB release build), and SRS-163 (Android back-navigation intent) reviewed and left unchanged — text remains unambiguous and P0/P1 scope is intact. Added SRS-208 (Android in-call audio mode engaged on voice-session connect, released on last disconnect — addresses SysRS-305) and SRS-209 (explicit `RECORD_AUDIO` runtime permission acquisition at or before voice-session activation, fail-safe to listen-only on denial — addresses SysRS-306). Strict layered sourcing preserved (`SRS -> SysDes` only); `Source SysDes` for SRS-208/209 traces via SysDes-135 pending a SysDes revision that introduces dedicated derivation items for SysRS-305/306. | diff --git a/docs/requirements/sysrs.md b/docs/requirements/sysrs.md index 8ff2600..e4ebfb5 100644 --- a/docs/requirements/sysrs.md +++ b/docs/requirements/sysrs.md @@ -2,7 +2,7 @@ **Product name:** Chanora **Document type:** SysRS / System Requirements Specification -**Version:** 0.9.2 +**Version:** 0.9.10 **Status:** Baseline Candidate **Product category:** Cross-platform voice client application **Architecture:** Flutter + Rust Core @@ -1944,6 +1944,21 @@ This section converts the baseline product decisions into auditable system-level - Priority: P0 - Verification: Platform Test (Android), Functional Test +**SysRS-307**: The Chanora project shall maintain measured numeric performance baselines for the realtime audio capture and playback paths. The baseline set shall include, at minimum: (i) heap allocation count per realtime audio callback measured after warmup, (ii) per-callback wall-clock time expressed as a fraction of (or absolute bound relative to) the audio frame period, (iii) Opus encode latency and Opus decode latency per frame, and (iv) resampler throughput at the common rate-pair conversions exercised by the audio pipeline. Each baseline shall be expressed as a numeric threshold (or numeric range), not as free-form prose, so that regression against the baseline is deterministically detectable by automated comparison. This requirement extends the prescriptive intent of SysRS-180 ("shall minimize local audio pipeline latency") and the directional target of SysRS-181 ("should target local audio pipeline latency under 100 ms") into a measurable, contract-grade obligation, and it is consistent with the underrun-avoidance obligation in SysRS-186. The specific baseline values, the warmup definition, the rate-pair set, and the storage format for the baselines are SysDes/SAD/SDD concerns and are not authored here. This clause does not authorize automatic upload, transmission, or off-device export of any measured baseline data and is therefore consistent with SysRS-295 (no automatic telemetry / diagnostic upload in MVP). + +- Priority: P0 +- Verification: Test (SWE.4 benchmark assertions, delegated to verification layer; cross-references SysRS-236) + +**SysRS-308**: The Chanora project's continuous-integration workflow shall execute the realtime-audio benchmark suite on every pull request against the default branch and on every merge to the default branch, on at least one host architecture (Linux x86_64 on the existing GitHub Actions runner is sufficient to satisfy this clause), and shall report the comparison of the executed run against the maintained baselines (SysRS-307) in the pull request's status-check surface such that human reviewers can see, before approving the change, whether any baseline has regressed beyond the declared tolerance window (SysRS-309). The CI regression check authorized by this clause is **advisory only**: it shall NOT fail the CI build, shall NOT block merge, and shall NOT be treated as a hard quality gate at the P0-MVP stage. Its purpose is to surface evidence for human reviewer judgement and to avoid the failure mode in which a strict build-failing gate is bypassed under release-crunch pressure (e.g., "skip CI"). Escalation of this advisory tier to a build-failing hard gate is explicitly out of scope of SysRS-308 and shall be authorized only by a separate, later SysRS clause after a period of baseline maturity sufficient to establish that the advisory signal is stable and low-noise (indicative target: 4–6 weeks of clean baseline data on the default branch, but the actual escalation criteria are to be set by the future clause). This requirement is consistent with the CI-as-quality-evidence pattern established by the SysRS-234..239 verification family and does not displace any obligation in that family. This clause does not authorize off-device transmission of baseline measurements beyond the project's existing CI provider surface (i.e., GitHub Actions logs and PR status checks visible to repository collaborators); it is therefore consistent with SysRS-295. + +- Priority: P0 +- Verification: Demo (CI workflow exercised on a representative PR; advisory status check appears and does not block merge on regression) + +**SysRS-309**: The advisory CI regression comparison authorized by SysRS-308 shall use an explicitly declared tolerance window — a numeric percentage (or a numeric per-metric percentage set) above the maintained baseline (SysRS-307) — beyond which the advisory check shall mark the run as a regression. The tolerance shall be a single declared value (or one declared value per metric), not an ad-hoc reviewer judgement, so that the advisory signal is reproducible. The specific numeric value(s) of the tolerance window and the comparison methodology (for example, "comparison against the most recent baseline snapshot on the default branch") shall be authored at the SysDes layer and may be refined at SAD/SDD; a suggested starting value of +20% over baseline is recorded here for downstream traceability but is not binding at the SysRS layer. Cross-references: SysRS-307 (the baselines being compared against), SysRS-308 (the advisory CI surface in which the comparison runs), SysRS-180/SysRS-181 (latency intent the tolerance must not silently erode), SysRS-295 (the tolerance window value is a project-local configuration and does not authorize off-device telemetry). + +- Priority: P0 +- Verification: Review (SysDes/SAD declaration of tolerance value is present and is referenced by the CI workflow definition) + ## 32. Change History Addendum @@ -1977,3 +1992,10 @@ This section converts the baseline product decisions into auditable system-level | Version | Date | Description | |---|---|---| | 0.9.9 | 2026-05-17 | Reconciled SysRS-288 with DEC-004 (Accepted 2026-05-14): Android minimum runtime baseline raised from API 24 to API 28 (Android 9.0); rationale and decision citation added in-line. Verified SysRS-055, SysRS-161, SysRS-195, SysRS-217, and SysRS-289 remain consistent with DEC-004 (no API-level text in any of these; no rewrite required). Added SysRS-305 (Android in-call audio mode engagement during active voice session — back-fills the v0.9.8 product-decision-register `AudioManager.setMode(MODE_IN_COMMUNICATION)` entry) and SysRS-306 (explicit Android runtime microphone permission acquisition at or before voice session activation, additive to SysRS-055). | + + +## Baseline Candidate 0.9.10 Update + +| Version | Date | Description | +|---|---|---| +| 0.9.10 | 2026-05-18 | Authorized Option B of the benchmark-infrastructure decision for the realtime audio path. Added SysRS-307 (maintained numeric performance baselines for the realtime audio capture and playback paths — heap allocation count per callback after warmup, per-callback wall-clock budget relative to the audio frame period, Opus encode/decode latency, and resampler throughput at common rate-pair conversions; extends SysRS-180/SysRS-181 from prescriptive intent into a measurable contract; consistent with SysRS-186 and SysRS-236). Added SysRS-308 (advisory CI regression reporting executing the benchmark suite on every PR and every merge to the default branch on at least one host architecture, surfacing results in the PR status-check view; explicitly **advisory only** — does not fail the build, does not block merge; escalation to a build-failing hard gate is out of scope and deferred to a future SysRS clause after baseline maturity; consistent with the SysRS-234..239 verification-family pattern). Added SysRS-309 (explicitly declared numeric tolerance window for the advisory comparison; numeric value(s) and comparison methodology delegated to SysDes/SAD; suggested starting value +20% recorded as non-binding downstream guidance). All three new clauses are consistent with SysRS-295 (no automatic telemetry / diagnostic upload in MVP) — they neither authorize nor require off-device transmission of measurement data beyond the existing CI provider surface visible to repository collaborators. Explicitly **NOT** authored in this update: (a) Dimension 3 production telemetry export of timing histograms (deferred to P1; any future opt-in performance-evidence export through the user-initiated diagnostic-export path requires a separate P1 SysRS clause and must be reconciled with SysRS-295 at that time); (b) build-failing hard CI gate (deferred to a future SysRS clause). |