Edison Jwa
01a4a9ed28
Add purpose, architecture, and public API summary to each crate README following chanora_resolver pattern. Update verification master plan with new evidence sources and entry/exit criteria.
67 lines
3.5 KiB
Markdown
67 lines
3.5 KiB
Markdown
# chanora_audio
|
|
|
|
Real-time audio subsystem: capture, Opus encoding/decoding, voice rendering, PTT gating, and audio processing. Promoted from `poc/audio-capture-playback-spike`.
|
|
|
|
## Architecture
|
|
|
|
### Engine
|
|
|
|
- **`AudioEngine`** — the primary type. Starts a platform audio backend (capture + playback), wires an `AudioTransmitGate` for PTT gating, and feeds encoded Opus frames to the protocol layer via `voice_out`. Inbound voice packets are decoded and mixed by `tsclientlib::audio::AudioHandler` and pulled by the platform output callback at 48 kHz stereo.
|
|
|
|
### Platform backends (cfg-gated)
|
|
|
|
| Target | Backend | Notes |
|
|
|---|---|---|
|
|
| Android | Oboe (via `android_voice_unit`) | Requires `ndk_context` before start |
|
|
| iOS/macOS | Apple VoiceProcessingIO (`ios_voice_unit`) | Platform AEC/AGC/NS, route-change handling |
|
|
| Linux | SDL (`sdl_output`) | PulseAudio/ALSA via SDL |
|
|
| Other desktop | cpal | Fallback |
|
|
|
|
### Key modules
|
|
|
|
- **`audio_processing`** — P1 audio processing config, stats, route policy, effect ownership (Platform/Sonora/WebRTC APM)
|
|
- **`opus_voice`** — 20 ms / 48 kHz mono Opus encode/decode via `audiopus`
|
|
- **`transmit_mode`** — `TransmitMode` enum: Ptt, Continuous, VoiceActivity
|
|
- **`transmit_selector`** — `TransmitModeSelector` combining mode, hard-mute, PTT gate, permission gate, and in-channel state
|
|
- **`ptt`** — `AudioTransmitGate` (atomic bool), `PttCapabilityLevel`, `PttBackendDescriptor`
|
|
- **`ptt_backends`** — platform PTT backends: `DesktopPttBackend` (Linux portal), `FocusedPttBackend` (in-app fallback)
|
|
- **`release_tail`** — `ReleaseTailTimer` for configurable PTT release delay (default 200 ms, max 500 ms)
|
|
- **`vad`** — Voice-activity detection: Silero ONNX (desktop), WebRTC fallback, energy debug
|
|
- **`voice_render`** — mixes per-client decoded f32 PCM into the output buffer
|
|
- **`debug_wav`** — optional WAV file dump for diagnostics (DIAG_002/003)
|
|
- **`mobile_voice_backend`** — shared mobile voice-unit lifecycle abstraction
|
|
- **`frame`** — frame-aligned buffer utilities
|
|
|
|
## Public API Summary
|
|
|
|
### Types
|
|
|
|
| Type | Role |
|
|
|---|---|
|
|
| `AudioEngine` | Start/stop audio, set gain/mute/volume, read stats |
|
|
| `AudioEngineConfig` | Capture/playback device selection, PTT initial state, processing config |
|
|
| `AudioDeviceInfo` / `AudioDeviceList` | Device enumeration |
|
|
| `AudioTransmitGate` | Atomic PTT gate |
|
|
| `TransmitMode` / `TransmitModeSelector` | Mode selection with hard-mute clamp |
|
|
| `ReleaseTailTimer` | Configurable release delay (SDD-096) |
|
|
| `PttBinding` / `PttInputClass` | PTT key binding types |
|
|
| `PttBackendDescriptor` / `PttCapabilityLevel` | Capability query |
|
|
| `AudioProcessingConfig` / `AudioProcessingStats` | P1 processing control and telemetry |
|
|
| `AudioRoute` | Speaker/Earpiece/Wired/Bluetooth enum |
|
|
| `AudioEffects` | Effect toggles (AEC/AGC/NS/HPF), all enabled by default (DEC-007..010) |
|
|
| `AudioError` | Typed error catalogue |
|
|
|
|
### Key functions
|
|
|
|
- `AudioEngine::start_with_gate(cfg, voice_out, voice_in, gate)` — construct and start
|
|
- `AudioEngine::stop()` — tear down
|
|
- `list_audio_devices()` — enumerate available input/output devices
|
|
- `select_ptt_backend()` — choose the best PTT backend for the current platform
|
|
|
|
## Platform notes
|
|
|
|
- Android requires `initChanoraContext` (NDK context) before engine start.
|
|
- iOS/macOS uses VoiceProcessingIO for platform AEC/AGC/NS in the default route.
|
|
- Desktop can use Silero ONNX VAD when the model file is available.
|
|
- `bench_seam` is exposed (`#[doc(hidden)]`) for criterion benchmarks on non-mobile targets.
|