edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task

Browse Source 
Audio engine (Rust):
- Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD
- Hardware effects (JNI) with software fallback per-effect
- Render reference buffer for AEC between output/capture callbacks
- Voice activity gate: suppress transmission when speaker muted (all platforms)
- Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI
- ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17)
- VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix)
- TEN VAD default backend (was Silero)
- Platform→WebrtcApm resolution after hardware binding
- oboe-rs edisonjwa fork with get_raw_session_id()

Android Kotlin:
- AndroidAudioFocusController + AndroidBluetoothScoController
- AndroidAudioLifecycleController (route changes to Flutter)
- ProGuard rules for new controllers

Flutter UI:
- VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM)
- VoiceStatusChip: mute warning border + Speaker muted label
- BBCode welcome message parser (BbCodeText, case-insensitive)
- Welcome message foldable (expanded by default)
- Link trust dialog (domain wildcards, SharedPreferences)
- HapticFeedback on voice sheet opener
- Server name in AppBar, version v0.1.0
- Default channel (id=1) visible, serverquery clients hidden
- flutter_foreground_task integration

Config:
- ort load-dynamic on all non-iOS (Android/Linux/Windows)
- ONNX Runtime AAR 1.26.0
- ndarray moved to common deps (was Apple-only)
This commit is contained in:
Edison Jwa
2026-05-22 09:29:57 +09:00
parent 6af4ecab0f
commit bf284018e6
37 changed files with 3676 additions and 703 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/chanora_audio/src/vad/silero_onnx.rs
+13 -36
View File
@@ -75,13 +75,9 @@ pub struct SileroOnnxVad {
}
enum SileroInner {
#[cfg(any(target_os = "ios", target_os = "macos"))]
Onnx(OnnxSession),
#[allow(dead_code)]
Stub,
}
#[cfg(any(target_os = "ios", target_os = "macos"))]
struct OnnxSession {
session: ort::session::Session,
}
@@ -90,20 +86,11 @@ impl SileroOnnxVad {
/// Attempt to load the Silero v6 ONNX model from `model_path`.
///
/// Returns `None` when the model file is missing, the ONNX Runtime
/// is unavailable, or the platform is not iOS/macOS.
/// is unavailable, or the platform does not support ONNX.
pub fn try_new(model_path: &str) -> Option<Self> {
#[cfg(any(target_os = "ios", target_os = "macos"))]
{
Self::try_new_onnx(model_path)
}
#[cfg(not(any(target_os = "ios", target_os = "macos")))]
{
let _ = model_path;
None
}
Self::try_new_onnx(model_path)
}
#[cfg(any(target_os = "ios", target_os = "macos"))]
fn try_new_onnx(model_path: &str) -> Option<Self> {
use tracing::{error, info};
@@ -116,13 +103,8 @@ impl SileroOnnxVad {
return None;
}
#[cfg(target_os = "macos")]
if let Some(path) = bundled_onnxruntime_path() {
let _ = ort::init_from(path.to_string_lossy()).commit();
}
let session_result = std::panic::catch_unwind(|| {
ort::session::Session::builder().and_then(|b| b.commit_from_file(model_path))
ort::session::Session::builder().and_then(|mut b| b.commit_from_file(model_path))
});
match session_result {
@@ -193,7 +175,6 @@ impl SileroOnnxVad {
/// 16 kHz frame: concatenate prior context, pass `input/state/sr` to
/// ONNX, persist `stateN`, then refresh context from the current frame.
/// Updates `last_probability` and returns the new value.
#[cfg(any(target_os = "ios", target_os = "macos"))]
fn calc_level(&mut self, audio_frame: &[f32]) -> f32 {
use ort::value::Value;
use tracing::error;
@@ -279,7 +260,7 @@ impl SileroOnnxVad {
}
#[cfg(target_os = "macos")]
fn bundled_onnxruntime_path() -> Option<std::path::PathBuf> {
pub(crate) fn bundled_onnxruntime_path_for_vad() -> Option<std::path::PathBuf> {
let exe = std::env::current_exe().ok()?;
let app_dir = exe.parent()?;
let framework = app_dir
@@ -307,16 +288,9 @@ impl VoiceActivityDetector for SileroOnnxVad {
if self.accum.len() >= SILERO_FRAME_16K {
let audio_frame: Vec<f32> = self.accum[..SILERO_FRAME_16K].to_vec();
#[cfg(any(target_os = "ios", target_os = "macos"))]
{
if matches!(self.inner, SileroInner::Onnx(_)) {
self.calc_level(&audio_frame);
} else {
self.update_context_from_frame(&audio_frame);
}
}
#[cfg(not(any(target_os = "ios", target_os = "macos")))]
{
if matches!(self.inner, SileroInner::Onnx(_)) {
self.calc_level(&audio_frame);
} else {
self.update_context_from_frame(&audio_frame);
}
// Drain the accumulator (keep any overflow for next frame).
@@ -357,9 +331,9 @@ impl SileroOnnxVadWorker {
pub fn try_new(model_path: &str) -> Option<Self> {
let vad = SileroOnnxVad::try_new(model_path)?;
let latest_probability = Arc::new(AtomicU32::new(0.0_f32.to_bits()));
let latest_processed_seq = Arc::new(AtomicU64::new(0));
let latest_processed_seq = Arc::new(AtomicU64::new(u64::MAX));
let alive = Arc::new(AtomicBool::new(true));
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(8);
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(32);
let latest_probability_for_thread = latest_probability.clone();
let latest_processed_seq_for_thread = latest_processed_seq.clone();
let alive_for_thread = alive.clone();
@@ -414,7 +388,10 @@ impl SileroOnnxVadWorker {
/// True when the worker is too far behind to trust its latest
/// probability for the current frame.
pub fn is_stale(&self, capture_seq: u64) -> bool {
self.lag_frames(capture_seq) > SILERO_MAX_STALE_FRAMES
let latest = self
.latest_processed_seq
.load(std::sync::atomic::Ordering::Relaxed);
latest == u64::MAX || capture_seq.saturating_sub(latest) > SILERO_MAX_STALE_FRAMES
}
}