feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
This commit is contained in:
@@ -17,4 +17,4 @@ chanora_storage = { path = "../../crates/chanora_storage" }
|
||||
chanora_diagnostics = { path = "../../crates/chanora_diagnostics" }
|
||||
thiserror.workspace = true
|
||||
tracing.workspace = true
|
||||
tokio = { version = "1", features = ["sync", "rt"] }
|
||||
tokio = { version = "1", features = ["sync", "rt", "macros"] }
|
||||
|
||||
@@ -32,6 +32,7 @@ use std::sync::Arc;
|
||||
use thiserror::Error;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
pub use chanora_audio::{AudioEngine, AudioEngineConfig};
|
||||
pub use chanora_protocol::{
|
||||
ChannelInfo, ClientInfo, ConnectConfig, ProtocolError, ServerSnapshot,
|
||||
};
|
||||
@@ -65,13 +66,21 @@ pub enum CoreError {
|
||||
/// was already active (forbidden by DEC-006).
|
||||
#[error("already connected")]
|
||||
AlreadyConnected,
|
||||
/// Audio engine is not running.
|
||||
#[error("audio not started")]
|
||||
AudioNotStarted,
|
||||
}
|
||||
|
||||
struct ConnectedState {
|
||||
protocol: chanora_protocol::ProtocolClient,
|
||||
audio: Option<chanora_audio::AudioEngine>,
|
||||
}
|
||||
|
||||
/// The top-level Chanora session. Owns at most one active server
|
||||
/// connection (DEC-006).
|
||||
#[derive(Clone)]
|
||||
pub struct ChanoraSession {
|
||||
inner: Arc<Mutex<Option<chanora_protocol::ProtocolClient>>>,
|
||||
inner: Arc<Mutex<Option<ConnectedState>>>,
|
||||
}
|
||||
|
||||
impl ChanoraSession {
|
||||
@@ -83,7 +92,8 @@ impl ChanoraSession {
|
||||
}
|
||||
|
||||
/// Connect to a server. Fails with [`CoreError::AlreadyConnected`]
|
||||
/// if a connection is already active (DEC-006).
|
||||
/// if a connection is already active (DEC-006). Audio is not
|
||||
/// started automatically; call [`Self::start_audio`] after.
|
||||
pub async fn connect(&self, cfg: ConnectConfig) -> Result<ServerSnapshot, CoreError> {
|
||||
let mut guard = self.inner.lock().await;
|
||||
if guard.is_some() {
|
||||
@@ -91,15 +101,18 @@ impl ChanoraSession {
|
||||
}
|
||||
let client = chanora_protocol::ProtocolClient::connect(cfg).await?;
|
||||
let snap = client.snapshot().await?;
|
||||
*guard = Some(client);
|
||||
*guard = Some(ConnectedState {
|
||||
protocol: client,
|
||||
audio: None,
|
||||
});
|
||||
Ok(snap)
|
||||
}
|
||||
|
||||
/// Return a fresh snapshot of the current server state.
|
||||
pub async fn snapshot(&self) -> Result<ServerSnapshot, CoreError> {
|
||||
let guard = self.inner.lock().await;
|
||||
let client = guard.as_ref().ok_or(CoreError::NotConnected)?;
|
||||
Ok(client.snapshot().await?)
|
||||
let state = guard.as_ref().ok_or(CoreError::NotConnected)?;
|
||||
Ok(state.protocol.snapshot().await?)
|
||||
}
|
||||
|
||||
/// True if a connection is currently active.
|
||||
@@ -107,11 +120,53 @@ impl ChanoraSession {
|
||||
self.inner.lock().await.is_some()
|
||||
}
|
||||
|
||||
/// Start the audio engine attached to the current connection.
|
||||
/// Fails if not connected. Idempotent — calling twice replaces
|
||||
/// the engine.
|
||||
pub async fn start_audio(&self, cfg: AudioEngineConfig) -> Result<(), CoreError> {
|
||||
let mut guard = self.inner.lock().await;
|
||||
let state = guard.as_mut().ok_or(CoreError::NotConnected)?;
|
||||
|
||||
// Tear down any prior engine.
|
||||
if let Some(mut prev) = state.audio.take() {
|
||||
prev.stop();
|
||||
}
|
||||
|
||||
let voice_out = state.protocol.voice_out();
|
||||
let voice_in = state
|
||||
.protocol
|
||||
.take_voice_in()
|
||||
.ok_or(CoreError::Invariant("voice_in already taken"))?;
|
||||
let engine = chanora_audio::AudioEngine::start(cfg, voice_out, voice_in)?;
|
||||
state.audio = Some(engine);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Set push-to-talk state. No-op if audio not started.
|
||||
pub async fn set_ptt(&self, active: bool) -> Result<(), CoreError> {
|
||||
let guard = self.inner.lock().await;
|
||||
let state = guard.as_ref().ok_or(CoreError::NotConnected)?;
|
||||
let audio = state.audio.as_ref().ok_or(CoreError::AudioNotStarted)?;
|
||||
audio.set_ptt(active);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Read audio engine statistics: (frames_sent, frames_received, ptt_active).
|
||||
pub async fn audio_stats(&self) -> Result<(u32, u32, bool), CoreError> {
|
||||
let guard = self.inner.lock().await;
|
||||
let state = guard.as_ref().ok_or(CoreError::NotConnected)?;
|
||||
let audio = state.audio.as_ref().ok_or(CoreError::AudioNotStarted)?;
|
||||
Ok((audio.frames_sent(), audio.frames_received(), audio.ptt()))
|
||||
}
|
||||
|
||||
/// Disconnect from the server. No-op if not connected.
|
||||
pub async fn disconnect(&self) -> Result<(), CoreError> {
|
||||
let mut guard = self.inner.lock().await;
|
||||
if let Some(client) = guard.take() {
|
||||
client.disconnect().await;
|
||||
if let Some(mut state) = guard.take() {
|
||||
if let Some(mut audio) = state.audio.take() {
|
||||
audio.stop();
|
||||
}
|
||||
state.protocol.disconnect().await;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -145,4 +200,11 @@ mod tests {
|
||||
let r = s.connect(ConnectConfig::default()).await;
|
||||
assert!(matches!(r, Err(CoreError::Protocol(ProtocolError::Invalid(_)))));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ptt_without_audio_errors() {
|
||||
let s = ChanoraSession::new();
|
||||
let r = s.set_ptt(true).await;
|
||||
assert!(matches!(r, Err(CoreError::NotConnected)));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user