feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
This commit is contained in:
+97
-1
@@ -6,7 +6,103 @@ This project is expected to follow a Conventional Commits style workflow.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added — Alpha build (v0.1.0-alpha.1)
|
||||
### Added — Beta build (v0.2.0-beta.1)
|
||||
|
||||
- **Voice in/out wired end-to-end through the Flutter UI.** Per DEC-001
|
||||
this reaches the Internal Beta milestone. Build hash: see the
|
||||
`v0.2.0-beta.1` git tag.
|
||||
- `crates/chanora_audio/` promoted from scaffold to a working engine:
|
||||
- cpal-based capture (mic gain, linear resampling to 48 kHz, mono
|
||||
down-mix) and playback (48 kHz stereo, requested config).
|
||||
- `audiopus::Encoder` for Opus VoIP encoding (20 ms / 960-sample
|
||||
mono frames).
|
||||
- `tsclientlib::audio::AudioHandler` for the decode + per-client
|
||||
jitter buffer + mix on the playback side.
|
||||
- Push-to-talk gate: encoder is bypassed entirely when PTT is off,
|
||||
so no spurious silence frames leak out.
|
||||
- Graceful playback-only fallback: if the host has no usable mic
|
||||
(typical for headless CI / users who deny the mic permission),
|
||||
capture logs a warning and the engine continues with output only.
|
||||
`AudioEngine::capture_active()` exposes this for the UI.
|
||||
- Live counters: `frames_sent` / `frames_received` / `ptt()`.
|
||||
- `crates/chanora_protocol/` extended with voice channels:
|
||||
- `ProtocolClient::voice_out()` returns a clonable
|
||||
`mpsc::Sender<OutPacket>` for outbound frames.
|
||||
- `ProtocolClient::take_voice_in()` returns a one-shot
|
||||
`mpsc::Receiver<InboundVoice>` of decoded `S2C` / `S2CWhisper`
|
||||
packets, with the originating `from_client` ID extracted.
|
||||
- Re-exports the few `tsproto_packets::packets` types (`OutAudio`,
|
||||
`OutPacket`, `InAudioBuf`, `AudioData`, `CodecType`, `Direction`)
|
||||
that `chanora_audio` legitimately needs. This is the **only**
|
||||
deliberate cross-crate type re-export; per SAD-067 the audio
|
||||
path is performance-sensitive and a parallel type hierarchy
|
||||
would force a copy per 20 ms frame.
|
||||
- Connection task interleaves outbound voice (drained first per
|
||||
loop iteration), event pumping, and control-request handling.
|
||||
- `core/chanora_core::ChanoraSession` audio API:
|
||||
- `start_audio(AudioEngineConfig)` — starts the engine attached to
|
||||
the active connection. Idempotent.
|
||||
- `set_ptt(bool)` — toggles transmission. No-op without an engine.
|
||||
- `audio_stats()` → `(frames_sent, frames_received, ptt_active)`.
|
||||
- `disconnect()` now stops the engine before disconnecting the
|
||||
protocol task.
|
||||
- `crates/chanora_bridge/` audio surface:
|
||||
- `start_audio()`, `set_ptt(active)`, `audio_stats()` Dart-callable
|
||||
commands.
|
||||
- `BridgeAudioStats { frames_sent, frames_received, ptt_active }`
|
||||
DTO.
|
||||
- Mapped `CoreError::AudioNotStarted` and `CoreError::Audio(_)`
|
||||
arms in `BridgeError::From<CoreError>`.
|
||||
- `apps/chanora_flutter/`:
|
||||
- Beta UI rewrite of `main.dart`: "Start audio" button after
|
||||
connect; hold-to-talk button with pressed/released visual state;
|
||||
live audio-stats line below the PTT (`TX … frames • RX … frames
|
||||
• PTT on/off`).
|
||||
- ARB key set expanded with `startAudioAction`, `pttHoldToTalk`,
|
||||
`pttTransmitting`, `audioStatsLine` in both `en` and `zh-Hans`.
|
||||
- `test/beta_e2e_test.dart` exercises the full
|
||||
Dart → FRB → chanora_bridge → chanora_core → chanora_audio
|
||||
path against `cn.teamspeak.app`. Verifies connect, audio start,
|
||||
PTT toggle, disconnect.
|
||||
- `flutter_rust_bridge.yaml` now sets `local: true` so the codegen
|
||||
resolves the workspace member's library name correctly. Without
|
||||
this, the generated Dart side fell back to `libUNKNOWN.so` and
|
||||
failed to load the cdylib.
|
||||
|
||||
### Changed
|
||||
|
||||
- `flutter_rust_bridge.yaml`: added `local: true`.
|
||||
- `chanora_bridge::api`: `BridgeError::From<CoreError>` now maps
|
||||
`CoreError::AudioNotStarted` to `BridgeError::InvalidCommand` and
|
||||
`CoreError::Audio(_)` to `BridgeError::Connection`.
|
||||
- `apps/chanora_flutter/test/widget_test.dart`: banner-string
|
||||
expectations updated from "Alpha build" to "Beta build" and from
|
||||
"Alpha 版本" to "Beta 版本".
|
||||
- `docs/governance/product-decision-register.md` bumped to v0.9.7
|
||||
with a Beta-milestone change-history entry. No decision rows
|
||||
change.
|
||||
- `docs/governance/poc-results-summary.md` bumped to v0.6.0 with a
|
||||
Beta-milestone change-history entry; RISK-PoC-005 updated to
|
||||
reflect Beta progress.
|
||||
|
||||
### Notes (Beta scope honesty)
|
||||
|
||||
- DSP chain (AEC / AGC / NS / HPF per DEC-007..010) is **not yet
|
||||
implemented**. `AudioEffects` exists as a struct but its filters
|
||||
are no-ops in v0.2.0-beta.1. Real DSP is queued for Beta+ work.
|
||||
- The capture resampler is a simple linear interpolator. Production
|
||||
quality requires a proper resampler in Beta+.
|
||||
- Identity is still ephemeral per connect; persistence via
|
||||
`chanora_storage` is queued.
|
||||
- No live event stream into Dart yet — the UI fetches snapshots and
|
||||
audio stats on a timer instead of subscribing to push events.
|
||||
- `chanora_diagnostics` is still a scaffold; no redaction wired into
|
||||
`tracing` yet.
|
||||
- Audio engine is desktop-only in this Beta. Mobile bundle of the
|
||||
bridge cdylib + UI verification was proven by the PoC but is not
|
||||
re-built into product code in this milestone.
|
||||
|
||||
### Carry-over from Alpha (v0.1.0-alpha.1)
|
||||
|
||||
- **First Alpha build wires the connect → snapshot → disconnect cycle
|
||||
end-to-end from the Flutter UI to a live TeamSpeak-compatible
|
||||
|
||||
Reference in New Issue
Block a user