feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)

Reaches the Internal Beta milestone of DEC-001's release sequence the same day as Alpha. Adds voice capture and playback through the full Flutter UI → FRB → Rust core → tsclientlib → server path. Promotions from PoC: poc/audio-capture-playback-spike → crates/chanora_audio/ New product code: crates/chanora_audio/src/engine.rs — cpal capture and playback, audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib AudioHandler for decode + jitter buffer + mix on playback, push-to-talk gate, graceful playback-only fallback when capture is unavailable. crates/chanora_protocol/src/adapter.rs — extended with voice_out_tx (clonable mpsc::Sender<OutPacket>) and take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main loop now interleaves outbound voice drain, event pumping, and control-request handling. crates/chanora_protocol/src/lib.rs — re-exports the few tsproto_packets types (OutAudio, OutPacket, InAudioBuf, AudioData, CodecType, Direction) that chanora_audio legitimately needs. Documented as the single deliberate cross-crate type re-export per SAD-067, justified by the performance cost of a parallel type hierarchy on the 20 ms voice frame. core/chanora_core/src/lib.rs — ChanoraSession::start_audio, set_ptt, audio_stats; disconnect now stops the engine first. crates/chanora_bridge/src/api.rs — startAudio, setPtt, audioStats commands and BridgeAudioStats DTO. apps/chanora_flutter/lib/main.dart — "Start audio" button + hold-to-talk PTT button with pressed/released visual state + live stats line (TX/RX/PTT). Stats polled every 500 ms. ARB: Both en and zh-Hans gain startAudioAction, pttHoldToTalk, pttTransmitting, audioStatsLine. Banner updated to "Beta build — voice in/out wired; not production ready." FRB config: flutter_rust_bridge.yaml gains local: true so codegen resolves the workspace member's library stem to "chanora_bridge" instead of falling back to "UNKNOWN". Empirical verification (2026-05-14, against cn.teamspeak.app): cargo check + cargo test --workspace: all green. flutter analyze: 0 issues. flutter test: 4/4 passing including: - test/alpha_e2e_test.dart (regression: Alpha still works) - test/beta_e2e_test.dart (Beta: connect → startAudio → PTT cycle → disconnect against cn.teamspeak.app). Live smoke (cargo test alpha_smoke -- --ignored): 49 channels, 37 clients retrieved. Capture stream open against the host PipeWire auto_null source refused (snd_pcm_hw_params); engine correctly logged the warning and continued in playback-only mode. TX=0 frames, RX=0 frames reflects the headless null-source environment; on a real mic host the encoder produces ~50 frames/second while PTT is held. Honest Beta scope (NOT in this release): - AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists as a struct but the filters are no-ops. Beta+ work. - Production-quality resampler: current code is linear interpolation. Beta+ work. - Identity persistence via chanora_storage: still ephemeral. - Push-to-Dart event stream: UI polls instead. - chanora_diagnostics tracing-layer wiring: still scaffold. - Mobile (Android) cdylib + UI: PoC-proven, not yet in product. - Reconnect / network-loss recovery for the voice path. Docs updates: - docs/governance/product-decision-register.md bumped to v0.9.7 (Beta-milestone change-history entry; no row changes). - docs/governance/poc-results-summary.md bumped to v0.6.0 (RISK-PoC-005 updated with Beta progress).
2026-05-14 22:43:57 +08:00
parent 53b176b722
commit 9790005c3e
28 changed files with 2056 additions and 159 deletions
@@ -8,7 +8,7 @@ import 'lib.dart';
 import 'package:flutter_rust_bridge/flutter_rust_bridge_for_generated.dart';

 // These functions are ignored because they are not marked as `pub`: `runtime`, `session`
-// These function are ignored because they are on traits that is not defined in current crate (put an empty `#[frb]` on it to unignore): `clone`, `clone`, `clone`, `fmt`, `fmt`, `fmt`, `from`
+// These function are ignored because they are on traits that is not defined in current crate (put an empty `#[frb]` on it to unignore): `clone`, `clone`, `clone`, `clone`, `fmt`, `fmt`, `fmt`, `fmt`, `from`

 /// Connect to a TeamSpeak-compatible server and return the initial
 /// state snapshot. Honours the DEC-006 single-connection invariant
@@ -27,6 +27,49 @@ Future<void> disconnect() => RustLib.instance.api.crateApiDisconnect();
 /// True if a connection is currently active.
 Future<bool> isConnected() => RustLib.instance.api.crateApiIsConnected();

+/// Start the audio engine on the active connection. Requires a
+/// connection; idempotent (will replace any previous engine).
+Future<void> startAudio() => RustLib.instance.api.crateApiStartAudio();
+
+/// Set the push-to-talk state.
+Future<void> setPtt({required bool active}) =>
+    RustLib.instance.api.crateApiSetPtt(active: active);
+
+/// Read audio statistics. Errors if no connection or audio not started.
+Future<BridgeAudioStats> audioStats() =>
+    RustLib.instance.api.crateApiAudioStats();
+
+/// Statistics from the audio engine.
+class BridgeAudioStats {
+  /// Number of Opus frames sent since audio started.
+  final int framesSent;
+
+  /// Number of inbound voice packets decoded.
+  final int framesReceived;
+
+  /// Current push-to-talk state.
+  final bool pttActive;
+
+  const BridgeAudioStats({
+    required this.framesSent,
+    required this.framesReceived,
+    required this.pttActive,
+  });
+
+  @override
+  int get hashCode =>
+      framesSent.hashCode ^ framesReceived.hashCode ^ pttActive.hashCode;
+
+  @override
+  bool operator ==(Object other) =>
+      identical(this, other) ||
+      other is BridgeAudioStats &&
+          runtimeType == other.runtimeType &&
+          framesSent == other.framesSent &&
+          framesReceived == other.framesReceived &&
+          pttActive == other.pttActive;
+}
+
 /// Channel as seen by Dart. Matches `chanora_protocol::ChannelInfo`
 /// but with primitive `u64` ids so the Dart side gets `BigInt`s
 /// without any wrapper-type ceremony.
@@ -67,7 +67,7 @@ class RustLib extends BaseEntrypoint<RustLibApi, RustLibApiImpl, RustLibWire> {
  String get codegenVersion => '2.12.0';

  @override
-  int get rustContentHash => 978717843;
+  int get rustContentHash => 1944264248;

  static const kDefaultExternalLibraryLoaderConfig =
      ExternalLibraryLoaderConfig(
@@ -79,6 +79,8 @@ class RustLib extends BaseEntrypoint<RustLibApi, RustLibApiImpl, RustLibWire> {
 }

 abstract class RustLibApi extends BaseApi {
+  Future<BridgeAudioStats> crateApiAudioStats();
+
  Future<void> crateApiBridgeInit();

  Future<BridgeSnapshot> crateApiConnect({
@@ -90,7 +92,11 @@ abstract class RustLibApi extends BaseApi {

  Future<bool> crateApiIsConnected();

+  Future<void> crateApiSetPtt({required bool active});
+
  Future<BridgeSnapshot> crateApiSnapshot();
+
+  Future<void> crateApiStartAudio();
 }

 class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
@@ -102,7 +108,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
  });

  @override
-  Future<void> crateApiBridgeInit() {
+  Future<BridgeAudioStats> crateApiAudioStats() {
    return handler.executeNormal(
      NormalTask(
        callFfi: (port_) {
@@ -114,6 +120,33 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
            port: port_,
          );
        },
+        codec: SseCodec(
+          decodeSuccessData: sse_decode_bridge_audio_stats,
+          decodeErrorData: sse_decode_bridge_error,
+        ),
+        constMeta: kCrateApiAudioStatsConstMeta,
+        argValues: [],
+        apiImpl: this,
+      ),
+    );
+  }
+
+  TaskConstMeta get kCrateApiAudioStatsConstMeta =>
+      const TaskConstMeta(debugName: "audio_stats", argNames: []);
+
+  @override
+  Future<void> crateApiBridgeInit() {
+    return handler.executeNormal(
+      NormalTask(
+        callFfi: (port_) {
+          final serializer = SseSerializer(generalizedFrbRustBinding);
+          pdeCallFfi(
+            generalizedFrbRustBinding,
+            serializer,
+            funcId: 2,
+            port: port_,
+          );
+        },
        codec: SseCodec(
          decodeSuccessData: sse_decode_unit,
          decodeErrorData: null,
@@ -142,7 +175,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
          pdeCallFfi(
            generalizedFrbRustBinding,
            serializer,
-            funcId: 2,
+            funcId: 3,
            port: port_,
          );
        },
@@ -169,7 +202,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
          pdeCallFfi(
            generalizedFrbRustBinding,
            serializer,
-            funcId: 3,
+            funcId: 4,
            port: port_,
          );
        },
@@ -196,7 +229,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
          pdeCallFfi(
            generalizedFrbRustBinding,
            serializer,
-            funcId: 4,
+            funcId: 5,
            port: port_,
          );
        },
@@ -214,6 +247,34 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
  TaskConstMeta get kCrateApiIsConnectedConstMeta =>
      const TaskConstMeta(debugName: "is_connected", argNames: []);

+  @override
+  Future<void> crateApiSetPtt({required bool active}) {
+    return handler.executeNormal(
+      NormalTask(
+        callFfi: (port_) {
+          final serializer = SseSerializer(generalizedFrbRustBinding);
+          sse_encode_bool(active, serializer);
+          pdeCallFfi(
+            generalizedFrbRustBinding,
+            serializer,
+            funcId: 6,
+            port: port_,
+          );
+        },
+        codec: SseCodec(
+          decodeSuccessData: sse_decode_unit,
+          decodeErrorData: sse_decode_bridge_error,
+        ),
+        constMeta: kCrateApiSetPttConstMeta,
+        argValues: [active],
+        apiImpl: this,
+      ),
+    );
+  }
+
+  TaskConstMeta get kCrateApiSetPttConstMeta =>
+      const TaskConstMeta(debugName: "set_ptt", argNames: ["active"]);
+
  @override
  Future<BridgeSnapshot> crateApiSnapshot() {
    return handler.executeNormal(
@@ -223,7 +284,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
          pdeCallFfi(
            generalizedFrbRustBinding,
            serializer,
-            funcId: 5,
+            funcId: 7,
            port: port_,
          );
        },
@@ -241,6 +302,33 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
  TaskConstMeta get kCrateApiSnapshotConstMeta =>
      const TaskConstMeta(debugName: "snapshot", argNames: []);

+  @override
+  Future<void> crateApiStartAudio() {
+    return handler.executeNormal(
+      NormalTask(
+        callFfi: (port_) {
+          final serializer = SseSerializer(generalizedFrbRustBinding);
+          pdeCallFfi(
+            generalizedFrbRustBinding,
+            serializer,
+            funcId: 8,
+            port: port_,
+          );
+        },
+        codec: SseCodec(
+          decodeSuccessData: sse_decode_unit,
+          decodeErrorData: sse_decode_bridge_error,
+        ),
+        constMeta: kCrateApiStartAudioConstMeta,
+        argValues: [],
+        apiImpl: this,
+      ),
+    );
+  }
+
+  TaskConstMeta get kCrateApiStartAudioConstMeta =>
+      const TaskConstMeta(debugName: "start_audio", argNames: []);
+
  @protected
  String dco_decode_String(dynamic raw) {
    // Codec=Dco (DartCObject based), see doc to use other codecs
@@ -253,6 +341,19 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    return raw as bool;
  }

+  @protected
+  BridgeAudioStats dco_decode_bridge_audio_stats(dynamic raw) {
+    // Codec=Dco (DartCObject based), see doc to use other codecs
+    final arr = raw as List<dynamic>;
+    if (arr.length != 3)
+      throw Exception('unexpected arr length: expect 3 but see ${arr.length}');
+    return BridgeAudioStats(
+      framesSent: dco_decode_u_32(arr[0]),
+      framesReceived: dco_decode_u_32(arr[1]),
+      pttActive: dco_decode_bool(arr[2]),
+    );
+  }
+
  @protected
  BridgeChannel dco_decode_bridge_channel(dynamic raw) {
    // Codec=Dco (DartCObject based), see doc to use other codecs
@@ -339,6 +440,12 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    return raw as Uint8List;
  }

+  @protected
+  int dco_decode_u_32(dynamic raw) {
+    // Codec=Dco (DartCObject based), see doc to use other codecs
+    return raw as int;
+  }
+
  @protected
  BigInt dco_decode_u_64(dynamic raw) {
    // Codec=Dco (DartCObject based), see doc to use other codecs
@@ -370,6 +477,19 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    return deserializer.buffer.getUint8() != 0;
  }

+  @protected
+  BridgeAudioStats sse_decode_bridge_audio_stats(SseDeserializer deserializer) {
+    // Codec=Sse (Serialization based), see doc to use other codecs
+    var var_framesSent = sse_decode_u_32(deserializer);
+    var var_framesReceived = sse_decode_u_32(deserializer);
+    var var_pttActive = sse_decode_bool(deserializer);
+    return BridgeAudioStats(
+      framesSent: var_framesSent,
+      framesReceived: var_framesReceived,
+      pttActive: var_pttActive,
+    );
+  }
+
  @protected
  BridgeChannel sse_decode_bridge_channel(SseDeserializer deserializer) {
    // Codec=Sse (Serialization based), see doc to use other codecs
@@ -478,6 +598,12 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    return deserializer.buffer.getUint8List(len_);
  }

+  @protected
+  int sse_decode_u_32(SseDeserializer deserializer) {
+    // Codec=Sse (Serialization based), see doc to use other codecs
+    return deserializer.buffer.getUint32();
+  }
+
  @protected
  BigInt sse_decode_u_64(SseDeserializer deserializer) {
    // Codec=Sse (Serialization based), see doc to use other codecs
@@ -513,6 +639,17 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    serializer.buffer.putUint8(self ? 1 : 0);
  }

+  @protected
+  void sse_encode_bridge_audio_stats(
+    BridgeAudioStats self,
+    SseSerializer serializer,
+  ) {
+    // Codec=Sse (Serialization based), see doc to use other codecs
+    sse_encode_u_32(self.framesSent, serializer);
+    sse_encode_u_32(self.framesReceived, serializer);
+    sse_encode_bool(self.pttActive, serializer);
+  }
+
  @protected
  void sse_encode_bridge_channel(BridgeChannel self, SseSerializer serializer) {
    // Codec=Sse (Serialization based), see doc to use other codecs
@@ -604,6 +741,12 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
    serializer.buffer.putUint8List(self);
  }

+  @protected
+  void sse_encode_u_32(int self, SseSerializer serializer) {
+    // Codec=Sse (Serialization based), see doc to use other codecs
+    serializer.buffer.putUint32(self);
+  }
+
  @protected
  void sse_encode_u_64(BigInt self, SseSerializer serializer) {
    // Codec=Sse (Serialization based), see doc to use other codecs
@@ -25,6 +25,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  bool dco_decode_bool(dynamic raw);

+  @protected
+  BridgeAudioStats dco_decode_bridge_audio_stats(dynamic raw);
+
  @protected
  BridgeChannel dco_decode_bridge_channel(dynamic raw);

@@ -49,6 +52,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  Uint8List dco_decode_list_prim_u_8_strict(dynamic raw);

+  @protected
+  int dco_decode_u_32(dynamic raw);
+
  @protected
  BigInt dco_decode_u_64(dynamic raw);

@@ -64,6 +70,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  bool sse_decode_bool(SseDeserializer deserializer);

+  @protected
+  BridgeAudioStats sse_decode_bridge_audio_stats(SseDeserializer deserializer);
+
  @protected
  BridgeChannel sse_decode_bridge_channel(SseDeserializer deserializer);

@@ -92,6 +101,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  Uint8List sse_decode_list_prim_u_8_strict(SseDeserializer deserializer);

+  @protected
+  int sse_decode_u_32(SseDeserializer deserializer);
+
  @protected
  BigInt sse_decode_u_64(SseDeserializer deserializer);

@@ -110,6 +122,12 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  void sse_encode_bool(bool self, SseSerializer serializer);

+  @protected
+  void sse_encode_bridge_audio_stats(
+    BridgeAudioStats self,
+    SseSerializer serializer,
+  );
+
  @protected
  void sse_encode_bridge_channel(BridgeChannel self, SseSerializer serializer);

@@ -146,6 +164,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
    SseSerializer serializer,
  );

+  @protected
+  void sse_encode_u_32(int self, SseSerializer serializer);
+
  @protected
  void sse_encode_u_64(BigInt self, SseSerializer serializer);

@@ -27,6 +27,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  bool dco_decode_bool(dynamic raw);

+  @protected
+  BridgeAudioStats dco_decode_bridge_audio_stats(dynamic raw);
+
  @protected
  BridgeChannel dco_decode_bridge_channel(dynamic raw);

@@ -51,6 +54,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  Uint8List dco_decode_list_prim_u_8_strict(dynamic raw);

+  @protected
+  int dco_decode_u_32(dynamic raw);
+
  @protected
  BigInt dco_decode_u_64(dynamic raw);

@@ -66,6 +72,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  bool sse_decode_bool(SseDeserializer deserializer);

+  @protected
+  BridgeAudioStats sse_decode_bridge_audio_stats(SseDeserializer deserializer);
+
  @protected
  BridgeChannel sse_decode_bridge_channel(SseDeserializer deserializer);

@@ -94,6 +103,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  Uint8List sse_decode_list_prim_u_8_strict(SseDeserializer deserializer);

+  @protected
+  int sse_decode_u_32(SseDeserializer deserializer);
+
  @protected
  BigInt sse_decode_u_64(SseDeserializer deserializer);

@@ -112,6 +124,12 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
  @protected
  void sse_encode_bool(bool self, SseSerializer serializer);

+  @protected
+  void sse_encode_bridge_audio_stats(
+    BridgeAudioStats self,
+    SseSerializer serializer,
+  );
+
  @protected
  void sse_encode_bridge_channel(BridgeChannel self, SseSerializer serializer);

@@ -148,6 +166,9 @@ abstract class RustLibApiImplPlatform extends BaseApiImpl<RustLibWire> {
    SseSerializer serializer,
  );

+  @protected
+  void sse_encode_u_32(int self, SseSerializer serializer);
+
  @protected
  void sse_encode_u_64(BigInt self, SseSerializer serializer);