feat(perf,benchmark-infra): criterion bench harness + advisory CI workflows (SDD-120)

Implementation of SDD-120 §1-§8:

Bench harness (crates/chanora_audio/benches/):
- common.rs: deterministic synthetic audio (440 Hz sine, no RNG).
- realtime_capture.rs: bench_capture_alloc_count (dhat) +
  bench_capture_callback_wall_clock (criterion).
- opus_codec.rs: bench_opus_encode_latency + bench_opus_decode_latency
  (direct audiopus, not AudioHandler — SDD-120 §3 item 4).
- resampler.rs: bench_resampler_throughput across 44.1->48 /
  16->48 / 48->48 passthrough.

CI tooling (crates/chanora_audio/examples/):
- emit_baseline.rs: aggregates criterion estimates.json outputs
  into the SRS-217 baseline schema.
- compare_baseline.rs: applies SRS-219 tolerance, renders markdown
  table with 🟢/🟡/🔴 markers + yellow simpler-form realization per
  SDD-120 §8.

  Deviation from SDD-120 §2 / §5 / §7 placement: these tools live
  under examples/, not benches/ or src/bin/. Rationale: they must
  consume serde_json (a dev-only dep — production builds must not
  pull it). Cargo only resolves dev-dependencies for [[test]],
  [[bench]], and [[example]] targets; [[bin]] targets under
  src/bin/ see only regular [dependencies]. examples/ keeps the
  binaries out of the production dep tree while still giving them
  cargo run --example invocation. An SDD-120 amendment should
  reflect this.

Workflows (.github/workflows/):
- bench-advisory.yml: PR + push triggers; runs benches; posts a
  sticky PR comment via actions/github-script@v7; job status is
  always success (SRS-218 clause 4 — non-blocking).
- bench-baseline-update.yml: workflow_dispatch only; runs benches;
  opens PR via peter-evans/create-pull-request@v6 (sole writer of
  the SAD-089 baseline JSON).

Cargo.toml additions ([dev-dependencies] only — verified excluded
from --release builds): criterion 0.5, dhat 0.3, serde_json 1.

Source-code seam: minimal pub-but-#[doc(hidden)] bench_seam module
in chanora_audio (engine.rs + lib.rs re-export) so the criterion
bench harness can construct a CaptureState and drive
CaptureState::ingest without re-implementing the engine (SDD-120
§3). Non-iOS targets only — CaptureState itself is iOS-gated.

Initial baseline seed: crates/chanora_audio/benches/baselines/
x86_64-unknown-linux-gnu.json = {}. compare_baseline handles the
missing-baseline case gracefully and emits a 'no red markers'
report; the first manual dispatch of bench-baseline-update.yml
after merge establishes the real values.

Out of scope per SDD-120 §10: production telemetry export,
build-failing hard CI gate, multi-host benchmarking, IDE
integration, Dart-side bridge round-trip bench.

Verification:
- cargo check --workspace --all-targets: PASS.
- cargo bench --bench realtime_capture --no-run: PASS.
- cargo bench --bench opus_codec --no-run: PASS.
- cargo bench --bench resampler --no-run: PASS.
- cargo build --example emit_baseline --example compare_baseline
  -p chanora_audio: PASS.
- cargo test --workspace: 106 passed, 0 failed, 3 ignored — no
  regression from prior count.
This commit is contained in:
EdisonJwa
2026-05-18 13:52:15 +08:00
parent 575a6cbc5c
commit 7188a5a69d
13 changed files with 1248 additions and 4 deletions
+83
View File
@@ -1797,3 +1797,86 @@ pub fn android_set_audio_mode(mode: i32) -> Result<(), AudioModeError> {
Ok(())
})
}
// ---------------------------------------------------------------------------
// SDD-120 §3 bench seam.
//
// Exposes a minimal factory for `CaptureState` plus a thin `ingest_f32`
// shim so the criterion bench harness in `crates/chanora_audio/benches/`
// can drive the same realtime capture code path the production cpal
// callback uses, without re-implementing CaptureState in the bench file.
// Marked `#[doc(hidden)]` so the public API surface is unaffected; this
// is not a supported external API. Only compiled on non-iOS targets
// because `CaptureState` itself is gated on `cfg(not(target_os = "ios"))`.
// ---------------------------------------------------------------------------
#[cfg(not(target_os = "ios"))]
#[doc(hidden)]
pub mod bench_seam {
use super::{
AtomicBool, AtomicU32, Arc, CaptureState, OpusApp, OpusChannels, OpusEncoder,
OpusSampleRate, OutPacket,
};
use tokio::sync::mpsc;
/// Opaque handle wrapping a CaptureState plus the dummy mpsc
/// receiver that prevents the channel sender from erroring out
/// when the bench drives `ingest`. The receiver is held inside
/// the handle so it lives for the bench's lifetime.
pub struct CaptureBenchHandle {
state: CaptureState,
// Keep the receiver alive so sends from CaptureState::encode_and_send
// do not fail; the bench discards what would be transmitted.
_rx: mpsc::Receiver<OutPacket>,
transmit_active: Arc<AtomicBool>,
}
impl CaptureBenchHandle {
/// Construct a CaptureState wired to a private mpsc + a
/// pre-asserted transmit-active flag so `ingest` exercises
/// the full down-mix → resample → encode → send pipeline.
///
/// `in_sample_rate` selects the input rate (48000 for the
/// passthrough path, 44100 / 16000 to exercise the linear
/// resampler). `in_channels` selects the channel layout
/// (typically 1 or 2).
pub fn new(in_sample_rate: u32, in_channels: usize) -> Self {
let encoder = OpusEncoder::new(
OpusSampleRate::Hz48000,
OpusChannels::Mono,
OpusApp::Voip,
)
.expect("opus encoder init");
let (tx, rx) = mpsc::channel::<OutPacket>(64);
let transmit_active = Arc::new(AtomicBool::new(true));
let frames_sent = Arc::new(AtomicU32::new(0));
let state = CaptureState::new(
encoder,
in_sample_rate,
in_channels,
1.0,
tx,
transmit_active.clone(),
frames_sent,
);
Self {
state,
_rx: rx,
transmit_active,
}
}
/// Drive one cpal-callback-equivalent buffer through the
/// realtime capture pipeline.
#[inline]
pub fn ingest_f32(&mut self, buf: &[f32]) {
self.state.ingest(buf);
}
/// Set the PTT gate. Defaults to true (bench measures the
/// transmitting path).
pub fn set_transmit_active(&self, active: bool) {
self.transmit_active
.store(active, std::sync::atomic::Ordering::Relaxed);
}
}
}
+8
View File
@@ -47,6 +47,14 @@ mod ios_voice_unit;
pub mod android_voice_unit;
pub use engine::{AudioEngine, AudioEngineConfig};
// SDD-120 §3 bench seam — `#[doc(hidden)]` re-export so the criterion
// bench harness under `crates/chanora_audio/benches/` can construct a
// CaptureState and drive `ingest` without re-implementing the engine.
// Not part of the supported public API.
#[cfg(not(target_os = "ios"))]
#[doc(hidden)]
pub use engine::bench_seam;
pub use ptt::{AudioTransmitGate, MissedKeyUpWatchdog, PttBackendDescriptor, PttCapabilityLevel};
pub use ptt_backends::{
select as select_ptt_backend, DesktopPttBackend, FocusedPttBackend, PttBackendError,