feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task

Audio engine (Rust):
- Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD
- Hardware effects (JNI) with software fallback per-effect
- Render reference buffer for AEC between output/capture callbacks
- Voice activity gate: suppress transmission when speaker muted (all platforms)
- Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI
- ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17)
- VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix)
- TEN VAD default backend (was Silero)
- Platform→WebrtcApm resolution after hardware binding
- oboe-rs edisonjwa fork with get_raw_session_id()

Android Kotlin:
- AndroidAudioFocusController + AndroidBluetoothScoController
- AndroidAudioLifecycleController (route changes to Flutter)
- ProGuard rules for new controllers

Flutter UI:
- VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM)
- VoiceStatusChip: mute warning border + Speaker muted label
- BBCode welcome message parser (BbCodeText, case-insensitive)
- Welcome message foldable (expanded by default)
- Link trust dialog (domain wildcards, SharedPreferences)
- HapticFeedback on voice sheet opener
- Server name in AppBar, version v0.1.0
- Default channel (id=1) visible, serverquery clients hidden
- flutter_foreground_task integration

Config:
- ort load-dynamic on all non-iOS (Android/Linux/Windows)
- ONNX Runtime AAR 1.26.0
- ndarray moved to common deps (was Apple-only)
This commit is contained in:
Edison Jwa
2026-05-22 09:29:57 +09:00
parent 6af4ecab0f
commit bf284018e6
37 changed files with 3676 additions and 703 deletions
+24 -20
View File
@@ -58,7 +58,7 @@ impl AudioRoute {
pub enum IosVoiceProcessingMode {
/// Shipping default: Apple VoiceProcessingIO owns AEC/NS/AGC.
PlatformVoiceProcessing,
/// Experimental Sonora capture-processing path.
/// Experimental raw capture-processing path.
SonoraExperimental,
}
@@ -69,7 +69,7 @@ pub enum AudioBackend {
PlatformVoiceProcessing,
/// Rust-native Sonora backend.
Sonora,
/// Future WebRTC APM backend.
/// WebRTC Audio Processing Module backend.
WebrtcApm,
/// No processing.
Noop,
@@ -168,9 +168,9 @@ impl Default for AudioProcessingConfig {
route: AudioRoute::Speaker,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::PlatformVoiceProcessing,
vad_backend: VadBackend::SileroOnnx,
vad_backend: VadBackend::TenVad,
aec: EffectOwner::Platform,
// iOS VPIO owns NS/AGC on the default shipping path. Rust/Sonora
// iOS VPIO owns NS/AGC on the default shipping path. Software
// effects are opt-in through the experimental raw route only.
ns: EffectOwner::Platform,
agc: EffectOwner::Platform,
@@ -194,19 +194,23 @@ impl AudioProcessingConfig {
}
if self.ios_mode == IosVoiceProcessingMode::PlatformVoiceProcessing
&& (self.processing_backend == AudioBackend::Sonora
|| self.processing_backend == AudioBackend::WebrtcApm
|| self.aec == EffectOwner::Sonora
|| self.aec == EffectOwner::WebrtcApm
|| self.ns == EffectOwner::Sonora
|| self.agc == EffectOwner::Sonora)
|| self.ns == EffectOwner::WebrtcApm
|| self.agc == EffectOwner::Sonora
|| self.agc == EffectOwner::WebrtcApm)
{
return Err(AudioError::InvalidAudioProcessingConfig(
"Sonora cannot be enabled with iOS VoiceProcessingIO".to_string(),
"software audio processing cannot be enabled with iOS VoiceProcessingIO"
.to_string(),
));
}
if self.ios_mode == IosVoiceProcessingMode::SonoraExperimental {
if self.processing_backend != AudioBackend::Sonora {
if self.processing_backend != AudioBackend::WebrtcApm {
return Err(AudioError::InvalidAudioProcessingConfig(
"ios Sonora experimental mode requires the Sonora processing backend"
.to_string(),
"ios raw processing mode requires the WebRTC APM backend".to_string(),
));
}
}
@@ -246,9 +250,9 @@ mod tests {
}
#[test]
fn platform_voice_processing_rejects_sonora_effects() {
fn platform_voice_processing_rejects_software_effects() {
let config = AudioProcessingConfig {
ns: EffectOwner::Sonora,
ns: EffectOwner::WebrtcApm,
..AudioProcessingConfig::default()
};
@@ -261,13 +265,13 @@ mod tests {
}
#[test]
fn sonora_experimental_allows_full_sonora_chain() {
fn raw_processing_allows_full_webrtc_apm_chain() {
let config = AudioProcessingConfig {
ios_mode: IosVoiceProcessingMode::SonoraExperimental,
processing_backend: AudioBackend::Sonora,
aec: EffectOwner::Sonora,
ns: EffectOwner::Sonora,
agc: EffectOwner::Sonora,
processing_backend: AudioBackend::WebrtcApm,
aec: EffectOwner::WebrtcApm,
ns: EffectOwner::WebrtcApm,
agc: EffectOwner::WebrtcApm,
..AudioProcessingConfig::default()
};
@@ -275,13 +279,13 @@ mod tests {
}
#[test]
fn sonora_experimental_rejects_non_sonora_backend() {
fn raw_processing_rejects_non_webrtc_apm_backend() {
let config = AudioProcessingConfig {
ios_mode: IosVoiceProcessingMode::SonoraExperimental,
processing_backend: AudioBackend::PlatformVoiceProcessing,
aec: EffectOwner::Sonora,
ns: EffectOwner::Sonora,
agc: EffectOwner::Sonora,
aec: EffectOwner::WebrtcApm,
ns: EffectOwner::WebrtcApm,
agc: EffectOwner::WebrtcApm,
..AudioProcessingConfig::default()
};