feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
This commit is contained in:
@@ -1,30 +1,37 @@
|
||||
//! # `chanora_audio`
|
||||
//!
|
||||
//! Audio subsystem. Per SAD §7.2:
|
||||
//! Audio subsystem promoted from `poc/audio-capture-playback-spike`
|
||||
//! and wired against `chanora_protocol`'s voice channels.
|
||||
//!
|
||||
//! * capture from the platform default input device
|
||||
//! * DSP chain: high-pass filter → noise suppression → echo
|
||||
//! cancellation → automatic gain control → gate
|
||||
//! (per-effect defaults: DEC-007..010)
|
||||
//! * Opus encode/decode
|
||||
//! * jitter buffer, mixer
|
||||
//! * playback to the platform default output device
|
||||
//! ## What's wired in this Beta
|
||||
//!
|
||||
//! Crate selection per DEC-011.1: `cpal` on desktop and Android.
|
||||
//! iOS audio crate remains deferred.
|
||||
//! * Default-input capture via `cpal` (DEC-011.1)
|
||||
//! * Frame-aligned 20 ms / 48 kHz mono Opus encoding via `audiopus`
|
||||
//! * Forward encoded frames to the protocol crate as `OutPacket`s
|
||||
//! * Inbound voice packets fed to `tsclientlib::audio::AudioHandler`
|
||||
//! which owns Opus decode + per-client jitter buffer + mix
|
||||
//! * Mixed f32 PCM pulled by the cpal output callback at 48 kHz stereo
|
||||
//! * Push-to-talk: capture stream is permanently open; encoding is
|
||||
//! gated by an atomic `ptt_active` flag
|
||||
//!
|
||||
//! Empirical evidence from PoC:
|
||||
//! `poc/audio-capture-playback-spike` (Linux/PipeWire) +
|
||||
//! `poc/audio-capture-playback-android-spike` (Android 14 arm64-v8a,
|
||||
//! physical device).
|
||||
//! ## What's NOT wired in this Beta
|
||||
//!
|
||||
//! ## Status
|
||||
//!
|
||||
//! Scaffold only. PoC code is not promoted here yet.
|
||||
//! * AEC / AGC / NS / HPF DSP chain (DEC-007/008/009/010 — Beta+
|
||||
//! work; the toggles in `AudioEffects` are honoured by *naming*
|
||||
//! but the filters are no-ops)
|
||||
//! * Mobile audio paths (DEC-011.1 desktop + Android proven; this
|
||||
//! integration is desktop-only for v0.2.0-beta.1)
|
||||
//! * Hot-plug device-change handling
|
||||
//! * Sample-rate adaptation if the device cannot do 48 kHz / mono in
|
||||
//! the format we request (returns `AudioError::StreamConfig`)
|
||||
//! * Multi-channel speaker layouts beyond stereo
|
||||
|
||||
#![forbid(unsafe_code)]
|
||||
#![warn(missing_docs)]
|
||||
|
||||
mod engine;
|
||||
|
||||
pub use engine::{AudioEngine, AudioEngineConfig};
|
||||
|
||||
use thiserror::Error;
|
||||
|
||||
/// Errors raised by the audio subsystem.
|
||||
@@ -39,15 +46,20 @@ pub enum AudioError {
|
||||
/// The audio backend rejected a stream configuration.
|
||||
#[error("stream config rejected: {0}")]
|
||||
StreamConfig(String),
|
||||
/// Opus codec init/encode/decode failure.
|
||||
#[error("opus: {0}")]
|
||||
Opus(String),
|
||||
/// A backend-specific failure surfaced without a typed mapping.
|
||||
/// Production code must narrow this further as failure modes
|
||||
/// are catalogued.
|
||||
#[error("audio backend: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
/// Audio-effect toggles. Defaults match DEC-007 (AEC),
|
||||
/// DEC-008 (AGC), DEC-009 (NS), DEC-010 (HPF) — all enabled.
|
||||
///
|
||||
/// Note: in Beta v0.2.0-beta.1 the actual DSP filters are not yet
|
||||
/// implemented; the struct is kept here as the public API surface so
|
||||
/// later work can flip an internal flag without breaking callers.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct AudioEffects {
|
||||
/// Acoustic echo cancellation (DEC-007).
|
||||
|
||||
Reference in New Issue
Block a user