edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)

Browse Source 
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.

Promotions from PoC:
  poc/audio-capture-playback-spike  →  crates/chanora_audio/

New product code:
  crates/chanora_audio/src/engine.rs — cpal capture and playback,
    audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
    AudioHandler for decode + jitter buffer + mix on playback,
    push-to-talk gate, graceful playback-only fallback when capture
    is unavailable.
  crates/chanora_protocol/src/adapter.rs — extended with
    voice_out_tx (clonable mpsc::Sender<OutPacket>) and
    take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
    loop now interleaves outbound voice drain, event pumping, and
    control-request handling.
  crates/chanora_protocol/src/lib.rs — re-exports the few
    tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
    AudioData, CodecType, Direction) that chanora_audio
    legitimately needs. Documented as the single deliberate
    cross-crate type re-export per SAD-067, justified by the
    performance cost of a parallel type hierarchy on the 20 ms
    voice frame.
  core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
    set_ptt, audio_stats; disconnect now stops the engine first.
  crates/chanora_bridge/src/api.rs — startAudio, setPtt,
    audioStats commands and BridgeAudioStats DTO.
  apps/chanora_flutter/lib/main.dart — "Start audio" button +
    hold-to-talk PTT button with pressed/released visual state +
    live stats line (TX/RX/PTT). Stats polled every 500 ms.

ARB:
  Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
  pttTransmitting, audioStatsLine. Banner updated to
  "Beta build — voice in/out wired; not production ready."

FRB config:
  flutter_rust_bridge.yaml gains local: true so codegen resolves
  the workspace member's library stem to "chanora_bridge" instead
  of falling back to "UNKNOWN".

Empirical verification (2026-05-14, against cn.teamspeak.app):
  cargo check + cargo test --workspace: all green.
  flutter analyze: 0 issues.
  flutter test: 4/4 passing including:
    - test/alpha_e2e_test.dart (regression: Alpha still works)
    - test/beta_e2e_test.dart (Beta: connect → startAudio →
      PTT cycle → disconnect against cn.teamspeak.app).
  Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
  37 clients retrieved.
  Capture stream open against the host PipeWire auto_null source
  refused (snd_pcm_hw_params); engine correctly logged the warning
  and continued in playback-only mode. TX=0 frames, RX=0 frames
  reflects the headless null-source environment; on a real mic
  host the encoder produces ~50 frames/second while PTT is held.

Honest Beta scope (NOT in this release):
  - AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
    as a struct but the filters are no-ops. Beta+ work.
  - Production-quality resampler: current code is linear
    interpolation. Beta+ work.
  - Identity persistence via chanora_storage: still ephemeral.
  - Push-to-Dart event stream: UI polls instead.
  - chanora_diagnostics tracing-layer wiring: still scaffold.
  - Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
  - Reconnect / network-loss recovery for the voice path.

Docs updates:
  - docs/governance/product-decision-register.md bumped to v0.9.7
    (Beta-milestone change-history entry; no row changes).
  - docs/governance/poc-results-summary.md bumped to v0.6.0
    (RISK-PoC-005 updated with Beta progress).
This commit is contained in:
EdisonJwa
2026-05-14 22:43:57 +08:00
parent 53b176b722
commit 9790005c3e
28 changed files with 2056 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/chanora_bridge/src/frb_generated.rs
+171 -6
View File
@@ -38,7 +38,7 @@ flutter_rust_bridge::frb_generated_boilerplate!(
default_rust_auto_opaque = RustAutoOpaqueMoi,
);
pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_VERSION: &str = "2.12.0";
pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_CONTENT_HASH: i32 = 978717843;
pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_CONTENT_HASH: i32 = 1944264248;
// Section: executor
@@ -46,6 +46,41 @@ flutter_rust_bridge::frb_generated_default_handler!();
// Section: wire_funcs
fn wire__crate__api__audio_stats_impl(
port_: flutter_rust_bridge::for_generated::MessagePort,
ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
rust_vec_len_: i32,
data_len_: i32,
) {
FLUTTER_RUST_BRIDGE_HANDLER.wrap_async::<flutter_rust_bridge::for_generated::SseCodec, _, _, _>(
flutter_rust_bridge::for_generated::TaskInfo {
debug_name: "audio_stats",
port: Some(port_),
mode: flutter_rust_bridge::for_generated::FfiCallMode::Normal,
},
move || {
let message = unsafe {
flutter_rust_bridge::for_generated::Dart2RustMessageSse::from_wire(
ptr_,
rust_vec_len_,
data_len_,
)
};
let mut deserializer =
flutter_rust_bridge::for_generated::SseDeserializer::new(message);
deserializer.end();
move |context| async move {
transform_result_sse::<_, crate::BridgeError>(
(move || async move {
let output_ok = crate::api::audio_stats().await?;
Ok(output_ok)
})()
.await,
)
}
},
)
}
fn wire__crate__api__bridge_init_impl(
port_: flutter_rust_bridge::for_generated::MessagePort,
ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
@@ -187,6 +222,42 @@ fn wire__crate__api__is_connected_impl(
},
)
}
fn wire__crate__api__set_ptt_impl(
port_: flutter_rust_bridge::for_generated::MessagePort,
ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
rust_vec_len_: i32,
data_len_: i32,
) {
FLUTTER_RUST_BRIDGE_HANDLER.wrap_async::<flutter_rust_bridge::for_generated::SseCodec, _, _, _>(
flutter_rust_bridge::for_generated::TaskInfo {
debug_name: "set_ptt",
port: Some(port_),
mode: flutter_rust_bridge::for_generated::FfiCallMode::Normal,
},
move || {
let message = unsafe {
flutter_rust_bridge::for_generated::Dart2RustMessageSse::from_wire(
ptr_,
rust_vec_len_,
data_len_,
)
};
let mut deserializer =
flutter_rust_bridge::for_generated::SseDeserializer::new(message);
let api_active = <bool>::sse_decode(&mut deserializer);
deserializer.end();
move |context| async move {
transform_result_sse::<_, crate::BridgeError>(
(move || async move {
let output_ok = crate::api::set_ptt(api_active).await?;
Ok(output_ok)
})()
.await,
)
}
},
)
}
fn wire__crate__api__snapshot_impl(
port_: flutter_rust_bridge::for_generated::MessagePort,
ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
@@ -222,6 +293,41 @@ fn wire__crate__api__snapshot_impl(
},
)
}
fn wire__crate__api__start_audio_impl(
port_: flutter_rust_bridge::for_generated::MessagePort,
ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
rust_vec_len_: i32,
data_len_: i32,
) {
FLUTTER_RUST_BRIDGE_HANDLER.wrap_async::<flutter_rust_bridge::for_generated::SseCodec, _, _, _>(
flutter_rust_bridge::for_generated::TaskInfo {
debug_name: "start_audio",
port: Some(port_),
mode: flutter_rust_bridge::for_generated::FfiCallMode::Normal,
},
move || {
let message = unsafe {
flutter_rust_bridge::for_generated::Dart2RustMessageSse::from_wire(
ptr_,
rust_vec_len_,
data_len_,
)
};
let mut deserializer =
flutter_rust_bridge::for_generated::SseDeserializer::new(message);
deserializer.end();
move |context| async move {
transform_result_sse::<_, crate::BridgeError>(
(move || async move {
let output_ok = crate::api::start_audio().await?;
Ok(output_ok)
})()
.await,
)
}
},
)
}
// Section: dart2rust
@@ -240,6 +346,20 @@ impl SseDecode for bool {
}
}
impl SseDecode for crate::api::BridgeAudioStats {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_decode(deserializer: &mut flutter_rust_bridge::for_generated::SseDeserializer) -> Self {
let mut var_framesSent = <u32>::sse_decode(deserializer);
let mut var_framesReceived = <u32>::sse_decode(deserializer);
let mut var_pttActive = <bool>::sse_decode(deserializer);
return crate::api::BridgeAudioStats {
frames_sent: var_framesSent,
frames_received: var_framesReceived,
ptt_active: var_pttActive,
};
}
}
impl SseDecode for crate::api::BridgeChannel {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_decode(deserializer: &mut flutter_rust_bridge::for_generated::SseDeserializer) -> Self {
@@ -363,6 +483,13 @@ impl SseDecode for Vec<u8> {
}
}
impl SseDecode for u32 {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_decode(deserializer: &mut flutter_rust_bridge::for_generated::SseDeserializer) -> Self {
deserializer.cursor.read_u32::<NativeEndian>().unwrap()
}
}
impl SseDecode for u64 {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_decode(deserializer: &mut flutter_rust_bridge::for_generated::SseDeserializer) -> Self {
@@ -398,11 +525,14 @@ fn pde_ffi_dispatcher_primary_impl(
) {
// Codec=Pde (Serialization + dispatch), see doc to use other codecs
match func_id {
1 => wire__crate__api__bridge_init_impl(port, ptr, rust_vec_len, data_len),
2 => wire__crate__api__connect_impl(port, ptr, rust_vec_len, data_len),
3 => wire__crate__api__disconnect_impl(port, ptr, rust_vec_len, data_len),
4 => wire__crate__api__is_connected_impl(port, ptr, rust_vec_len, data_len),
5 => wire__crate__api__snapshot_impl(port, ptr, rust_vec_len, data_len),
1 => wire__crate__api__audio_stats_impl(port, ptr, rust_vec_len, data_len),
2 => wire__crate__api__bridge_init_impl(port, ptr, rust_vec_len, data_len),
3 => wire__crate__api__connect_impl(port, ptr, rust_vec_len, data_len),
4 => wire__crate__api__disconnect_impl(port, ptr, rust_vec_len, data_len),
5 => wire__crate__api__is_connected_impl(port, ptr, rust_vec_len, data_len),
6 => wire__crate__api__set_ptt_impl(port, ptr, rust_vec_len, data_len),
7 => wire__crate__api__snapshot_impl(port, ptr, rust_vec_len, data_len),
8 => wire__crate__api__start_audio_impl(port, ptr, rust_vec_len, data_len),
_ => unreachable!(),
}
}
@@ -421,6 +551,25 @@ fn pde_ffi_dispatcher_sync_impl(
// Section: rust2dart
// Codec=Dco (DartCObject based), see doc to use other codecs
impl flutter_rust_bridge::IntoDart for crate::api::BridgeAudioStats {
fn into_dart(self) -> flutter_rust_bridge::for_generated::DartAbi {
[
self.frames_sent.into_into_dart().into_dart(),
self.frames_received.into_into_dart().into_dart(),
self.ptt_active.into_into_dart().into_dart(),
]
.into_dart()
}
}
impl flutter_rust_bridge::for_generated::IntoDartExceptPrimitive for crate::api::BridgeAudioStats {}
impl flutter_rust_bridge::IntoIntoDart<crate::api::BridgeAudioStats>
for crate::api::BridgeAudioStats
{
fn into_into_dart(self) -> crate::api::BridgeAudioStats {
self
}
}
// Codec=Dco (DartCObject based), see doc to use other codecs
impl flutter_rust_bridge::IntoDart for crate::api::BridgeChannel {
fn into_dart(self) -> flutter_rust_bridge::for_generated::DartAbi {
@@ -518,6 +667,15 @@ impl SseEncode for bool {
}
}
impl SseEncode for crate::api::BridgeAudioStats {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_encode(self, serializer: &mut flutter_rust_bridge::for_generated::SseSerializer) {
<u32>::sse_encode(self.frames_sent, serializer);
<u32>::sse_encode(self.frames_received, serializer);
<bool>::sse_encode(self.ptt_active, serializer);
}
}
impl SseEncode for crate::api::BridgeChannel {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_encode(self, serializer: &mut flutter_rust_bridge::for_generated::SseSerializer) {
@@ -615,6 +773,13 @@ impl SseEncode for Vec<u8> {
}
}
impl SseEncode for u32 {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_encode(self, serializer: &mut flutter_rust_bridge::for_generated::SseSerializer) {
serializer.cursor.write_u32::<NativeEndian>(self).unwrap();
}
}
impl SseEncode for u64 {
// Codec=Sse (Serialization based), see doc to use other codecs
fn sse_encode(self, serializer: &mut flutter_rust_bridge::for_generated::SseSerializer) {