edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audio): desktop Silero ONNX VAD + Windows PTT modernization + MSVC CRT build fix (#37)

Browse Source 
* feat(audio): add Silero ONNX VAD with WebRTC fallback

Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active.

* feat(audio): integrate desktop VAD worker into capture engine

Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding.

Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default.

Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self).

* fix(audio): modernize Windows PTT to current windows-rs API

Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate.

* build(windows): force MSVC release CRT for audiopus cmake builds

audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link.

Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK.

* build(flutter): update pubspec.lock after plugin additions

Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature.

* fix(audio): address PR #37 review findings

Six fixes from independent PR review:

1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform
   CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd
   globally broke non-Windows hosts because cmake-rs would try to
   execute a .cmd file on macOS/Linux. Instead, set
   CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY=
   MultiThreadedDLL as env vars that CMake reads natively. MSVC-
   specific vars are safely ignored by GCC/Clang toolchains. Delete
   the now-unnecessary wrapper script.

2. IMPORTANT: Join the Silero worker thread in Drop instead of
   detaching it. The old code dropped the JoinHandle which detaches
   the thread; the new code calls handle.join() after closing the
   channel, ensuring the ONNX session is cleaned up before the
   worker is replaced during config changes.

3. IMPORTANT: Single-try_lock refactor of the capture VAD callback.
   The double try_lock (policy probe + send) is replaced by a single
   scoped try_lock that both probes availability and sends the frame.
   The guard is dropped before the fallback path, which needs &mut
   self for mark_vad_fallback_active. This also eliminates the
   VadWorkerPolicy enum and callback_vad_worker_policy function,
   whose behavior is now inlined into the callback.

4. IMPORTANT: Remove tracing from the realtime capture callback.
   mark_vad_fallback_active and sync_vad_backend emitted info!/warn!
   from the audio thread. Replace with silent atomic state
   publishing via SharedAudioProcessingStats; the bridge stats
   stream already exposes vad_fallback_active for diagnostics.

5. IMPORTANT: Defer ONNX model load outside the worker mutex.
   apply_desktop_vad_backend_to_worker now constructs the new worker
   before taking the lock, then swaps it in under a short hold.
   This prevents the realtime callback from being blocked during
   model I/O + thread spawn.

6. MINOR: Remove unused VadBackend import from vad/mod.rs after
   deleting the policy code.

* fix(audio): address PR #37 second-pass review findings

5-agent review found 5 blocking issues. All addressed:

1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper
   but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc]
   and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows
   hosts are unaffected.

2. BLOCKER: processing_backend normalized in set_audio_processing_config
   on desktop (cfg-gated override to Noop), mirroring startup default.

3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config.
   Fixed misleading doc comment in core/lib.rs.

4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement.
   Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated.

5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android).
   Cargo.toml ort dependency target cfg narrowed similarly.

6. Realtime callback debt documented as TODO at CaptureState::ingest.

* fix(audio): exclude ort dep on Android target

ort does not provide first-class Android prebuilts in our pin, mirror the
iOS/macOS exclusion so cargo metadata succeeds for android targets.

* test(audio): fix stale select_ptt_backend import in ptt_privacy

The helper moved out of the ptt_backends submodule onto the crate root;
update the integration test imports so the test compiles again.

* build(windows): scope MSVC release CRT cmake wrapper via Cargo [env]

Cargo's [target.<triple>] table only forwards a fixed allowlist
(linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there
was silently dropped and audiopus_sys kept linking the debug CRT,
producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors
on x86_64-pc-windows-msvc test builds.

Move the override to Cargo's [env] table using cc/cmake-rs's
target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it
applies to MSVC targets only and not to host tooling. Add stdout
markers to the wrapper so its invocation is provable in cargo -vv logs.

Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc
--lib --no-run now links cleanly; CMakeCache.txt records
CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW.

* fix(flutter): gate VoiceActivity transmit mode by platform support

VoiceActivity relies on the native VAD worker, which is only wired up
on Windows, Linux, and Android. Showing the option on iOS, macOS, or
web let users select a mode that silently never transmitted.

Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers
in voice_settings_controls.dart, hide the VAD row in voice_compact.dart
and drop the VAD segment from the settings dialog when unsupported.
Keep the legacy const transmitModeSegments for the existing widget test
and add two new tests covering the gated helper.
This commit is contained in:
Edison Jwa
2026-06-09 20:47:16 +09:00
committed by GitHub
parent 5c3dd70bba
commit 2f6d45fb04
25 changed files with 967 additions and 210 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/chanora_audio/Cargo.toml
+1 -1
View File
@@ -51,7 +51,7 @@ coreaudio-rs = "0.14"
# on the main queue to avoid the VPIO RPC timeout on iOS simulator.
dispatch2 = "0.3"
[target.'cfg(not(target_os = "ios"))'.dependencies]
[target.'cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))'.dependencies]
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "api-24"] }
[target.'cfg(target_os = "android")'.dependencies]
crates/chanora_audio/src/audio_processing.rs
+2 -4
View File
@@ -404,10 +404,8 @@ impl Default for SharedAudioProcessingStats {
}
impl SharedAudioProcessingStats {
/// Store the raw input dBFS level (desktop capture path).
/// Mobile platforms use [`Self::update_capture`] instead, which
/// also records VAD state; this lighter method is for the cpal
/// capture path that has no VAD pipeline.
/// Store the raw input dBFS level for capture paths that do not
/// update the full processing/VAD snapshot on this callback.
pub fn set_input_dbfs(&self, dbfs: f32) {
self.input_dbfs.store(dbfs.to_bits(), Ordering::Relaxed);
}
crates/chanora_audio/src/engine.rs
+634 -15
View File
@@ -307,6 +307,8 @@ pub struct AudioEngine {
output_muted: Arc<AtomicBool>,
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
#[cfg(not(target_os = "android"))]
audio_handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
#[cfg(target_os = "android")]
@@ -741,7 +743,7 @@ impl AudioEngine {
) -> Option<cpal::Device>
where
DefaultFn: Fn(&cpal::Host) -> Option<cpal::Device>,
AllFn: Fn(&cpal::Host) -> Result<Devices, cpal::DevicesError>,
AllFn: Fn(&cpal::Host) -> Result<Devices, cpal::Error>,
Devices: IntoIterator<Item = cpal::Device>,
{
if let Some(id) = prefer {
@@ -823,8 +825,8 @@ impl AudioEngine {
let frames_received = Arc::new(AtomicU32::new(0));
let output_gain = Arc::new(AtomicU32::new(1.0_f32.to_bits()));
let output_muted = Arc::new(AtomicBool::new(false));
let audio_processing_config = Arc::new(Mutex::new(crate::AudioProcessingConfig::default()));
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
let (audio_processing_config, audio_processing_stats, silero_vad_worker) =
new_desktop_audio_processing_state();
// ---------- Capture ----------
// Capture is best-effort. If the platform default input
@@ -838,6 +840,9 @@ impl AudioEngine {
transmit_flag_for_capture,
frames_sent.clone(),
cfg.mic_gain,
cfg.voice_activity_selector.clone(),
audio_processing_config.clone(),
silero_vad_worker.clone(),
audio_processing_stats.clone(),
);
let (input_stream, capture_active) = match capture_result {
@@ -981,11 +986,13 @@ impl AudioEngine {
Ok(Self {
transmit_gate,
frames_sent,
frames_received,
output_gain,
output_muted,
audio_processing_config,
audio_processing_stats,
silero_vad_worker,
audio_handler,
_input_stream: Mutex::new(input_stream),
_output_stream: Mutex::new(Some(output_stream)),
@@ -1667,15 +1674,40 @@ impl AudioEngine {
/// Apply a voice-processing config after validating iOS invariants.
pub fn set_audio_processing_config(
&self,
config: crate::AudioProcessingConfig,
mut config: crate::AudioProcessingConfig,
) -> Result<(), AudioError> {
#[cfg(target_os = "ios")]
config.validate_for_ios()?;
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
{
config.processing_backend = crate::AudioBackend::Noop;
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
self.apply_desktop_vad_backend(&config);
let mut guard = self.audio_processing_config.lock().unwrap();
*guard = config;
Ok(())
}
/// Re-apply the current audio processing config.
///
/// Used by the core layer to trigger VAD worker reload after a
/// model-path change (the epoch increments but the worker is only
/// reconstructed when `set_audio_processing_config` is called).
pub fn reload_audio_processing_config(&self) -> Result<(), AudioError> {
let config = self.audio_processing_config_snapshot();
self.set_audio_processing_config(config)
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
fn apply_desktop_vad_backend(&self, config: &crate::AudioProcessingConfig) {
apply_desktop_vad_backend_to_worker(
config,
&self.silero_vad_worker,
self.audio_processing_stats.as_ref(),
);
}
/// Current voice-processing stats snapshot.
pub fn audio_processing_stats(&self) -> crate::AudioProcessingStats {
let config = self.audio_processing_config.lock().unwrap().clone();
@@ -1777,10 +1809,369 @@ fn release_android_audio_mode_for_startup_rollback(
audio_mode_stack.release()
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
fn new_desktop_audio_processing_state() -> (
Arc<Mutex<crate::AudioProcessingConfig>>,
Arc<crate::SharedAudioProcessingStats>,
Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
) {
let mut config = crate::AudioProcessingConfig::default();
// Desktop cpal capture does not use a platform voice-processing API.
// The global default (PlatformVoiceProcessing) is correct for iOS/macOS
// VPIO but would mislabel the desktop path in bridge diagnostics and
// set `platform_voice_processing_enabled = true` when no such
// processing exists. Override to Noop; the bridge/UI stats layer
// will then report the accurate backend.
config.processing_backend = crate::AudioBackend::Noop;
let audio_processing_config = Arc::new(Mutex::new(config.clone()));
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
let silero_vad_worker = Arc::new(Mutex::new(None));
apply_desktop_vad_backend_to_worker(
&config,
&silero_vad_worker,
audio_processing_stats.as_ref(),
);
(
audio_processing_config,
audio_processing_stats,
silero_vad_worker,
)
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
fn apply_desktop_vad_backend_to_worker(
config: &crate::AudioProcessingConfig,
silero_vad_worker: &Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
audio_processing_stats: &crate::SharedAudioProcessingStats,
) {
if config.vad_backend != crate::VadBackend::SileroOnnx {
let mut worker_guard = silero_vad_worker.lock().unwrap();
if worker_guard.is_some() {
info!(
target: "chanora_audio",
backend = config.vad_backend.as_str(),
"desktop: Silero ONNX VAD worker cleared because another VAD backend is selected"
);
}
*worker_guard = None;
audio_processing_stats.set_vad_fallback_active(false);
return;
}
// Load model and spawn worker BEFORE taking the lock so the
// realtime capture callback is not blocked on try_lock() during
// model I/O + thread spawn. The old worker (if any) is dropped
// after the new one is installed under the short lock hold.
let model_path = crate::vad::silero_model_bundle_path();
info!(
target: "chanora_audio",
path = %model_path,
"desktop: Silero ONNX VAD selected; loading model worker"
);
let new_worker = crate::vad::silero_onnx::SileroOnnxVadWorker::try_new(&model_path);
{
let mut worker_guard = silero_vad_worker.lock().unwrap();
*worker_guard = new_worker;
}
// Check result after releasing the lock. Re-acquire is cheap and
// ensures we log the correct state without holding the mutex.
let worker_installed = silero_vad_worker.lock().unwrap().is_some();
if worker_installed {
info!(
target: "chanora_audio",
path = %model_path,
"desktop: Silero ONNX VAD worker loaded"
);
audio_processing_stats.set_vad_fallback_active(false);
} else {
warn!(
target: "chanora_audio",
path = %model_path,
"desktop: Silero ONNX VAD worker unavailable; WebRTC fallback will be used"
);
audio_processing_stats.set_vad_fallback_active(true);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_capture_voice_activity_opens_selector_from_speech() {
let gate = crate::ptt::AudioTransmitGate::new(false);
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
selector.set_mode(crate::TransmitMode::VoiceActivity);
selector.set_in_channel(true);
let encoder = crate::opus_voice::new_voip_encoder("desktop VAD test").unwrap();
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
let frames_sent = Arc::new(AtomicU32::new(0));
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
voice_out_tx,
frames_sent,
"desktop-vad-test",
)
.unwrap();
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
let mut capture = CaptureState::new(
encoder,
SAMPLE_RATE,
1,
1.0,
voice_out_tx,
gate.flag_arc(),
Some(selector.clone()),
Arc::new(Mutex::new(crate::AudioProcessingConfig {
vad_backend: crate::VadBackend::WebrtcVad,
..crate::AudioProcessingConfig::default()
})),
Arc::new(Mutex::new(None)),
stats.clone(),
);
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
for (idx, sample) in voiced.iter_mut().enumerate() {
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
*sample = phase.sin() * 0.4;
}
for _ in 0..6 {
capture.ingest(&voiced);
}
assert!(
selector.voice_activity_open(),
"desktop capture must feed VAD and open VoiceActivity selector before transmit is already active"
);
assert!(
gate.load(),
"VoiceActivity selector should publish transmit gate"
);
let snapshot = stats.snapshot(&crate::AudioProcessingConfig::default());
assert!(snapshot.vad_active, "stats should expose active VAD");
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_startup_audio_processing_state_applies_default_silero_fallback() {
let _guard = crate::vad::SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
crate::vad::clear_silero_model_path_for_test();
let (config, stats, worker) = new_desktop_audio_processing_state();
assert_eq!(
config.lock().unwrap().vad_backend,
crate::VadBackend::SileroOnnx,
"desktop startup config should keep the default Silero backend selected"
);
assert!(
worker.lock().unwrap().is_none(),
"missing startup model should not create an ONNX worker"
);
let snapshot = stats.snapshot(&config.lock().unwrap());
assert!(
snapshot.vad_fallback_active,
"desktop startup should mark WebRTC fallback active when the default Silero worker cannot load"
);
crate::vad::clear_silero_model_path_for_test();
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_set_audio_processing_config_normalizes_default_backend_to_noop() {
let audio_processing_config = Arc::new(Mutex::new(crate::AudioProcessingConfig::default()));
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
let engine = AudioEngine {
transmit_gate: crate::ptt::AudioTransmitGate::new(false),
frames_sent: Arc::new(AtomicU32::new(0)),
frames_received: Arc::new(AtomicU32::new(0)),
output_gain: Arc::new(AtomicU32::new(1.0_f32.to_bits())),
output_muted: Arc::new(AtomicBool::new(false)),
audio_processing_config: audio_processing_config.clone(),
audio_processing_stats,
silero_vad_worker: Arc::new(Mutex::new(None)),
audio_handler: Arc::new(Mutex::new(AudioHandler::new())),
_input_stream: Mutex::new(None),
_output_stream: Mutex::new(None),
shutdown_tx: None,
capture_active: false,
};
engine
.set_audio_processing_config(crate::AudioProcessingConfig::default())
.unwrap();
assert_eq!(
engine.audio_processing_config_snapshot().processing_backend,
crate::AudioBackend::Noop,
"desktop setter should report the actual no-op processing backend for default configs"
);
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_capture_silero_selection_reports_fallback_when_worker_unavailable() {
let _guard = crate::vad::SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
crate::vad::clear_silero_model_path_for_test();
let gate = crate::ptt::AudioTransmitGate::new(false);
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
selector.set_mode(crate::TransmitMode::VoiceActivity);
selector.set_in_channel(true);
let encoder = crate::opus_voice::new_voip_encoder("desktop Silero fallback test").unwrap();
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
let frames_sent = Arc::new(AtomicU32::new(0));
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
voice_out_tx,
frames_sent,
"desktop-silero-fallback-test",
)
.unwrap();
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
let config = Arc::new(Mutex::new(crate::AudioProcessingConfig {
vad_backend: crate::VadBackend::SileroOnnx,
..crate::AudioProcessingConfig::default()
}));
let mut capture = CaptureState::new(
encoder,
SAMPLE_RATE,
1,
1.0,
voice_out_tx,
gate.flag_arc(),
Some(selector),
config.clone(),
Arc::new(Mutex::new(None)),
stats.clone(),
);
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
for (idx, sample) in voiced.iter_mut().enumerate() {
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
*sample = phase.sin() * 0.4;
}
capture.ingest(&voiced);
let snapshot = stats.snapshot(&config.lock().unwrap());
assert_eq!(snapshot.vad_backend, crate::VadBackend::SileroOnnx);
assert!(
snapshot.vad_fallback_active,
"desktop Silero selection should make WebRTC fallback visible when ONNX worker cannot load"
);
crate::vad::clear_silero_model_path_for_test();
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_capture_vad_consumes_only_new_pcm_while_transmitting() {
let gate = crate::ptt::AudioTransmitGate::new(true);
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
selector.set_mode(crate::TransmitMode::VoiceActivity);
selector.set_in_channel(true);
let encoder = crate::opus_voice::new_voip_encoder("desktop VAD duplicate test").unwrap();
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
let frames_sent = Arc::new(AtomicU32::new(0));
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
voice_out_tx,
frames_sent,
"desktop-vad-duplicate-test",
)
.unwrap();
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
let mut capture = CaptureState::new(
encoder,
SAMPLE_RATE,
1,
1.0,
voice_out_tx,
gate.flag_arc(),
Some(selector),
Arc::new(Mutex::new(crate::AudioProcessingConfig {
vad_backend: crate::VadBackend::Disabled,
..crate::AudioProcessingConfig::default()
})),
Arc::new(Mutex::new(None)),
stats,
);
capture.pcm_accum.extend(std::iter::repeat_n(
0.1_f32,
crate::frame::FRAME_10MS_SAMPLES + 240,
));
capture.pending_10ms[..240].fill(0.1);
capture.pending_10ms_len = 240;
capture.capture_frame_seq = 1;
capture.pcm_accum.extend(std::iter::repeat_n(0.2_f32, 240));
capture.process_pending_vad_frames(crate::frame::FRAME_10MS_SAMPLES + 240);
assert_eq!(
capture.capture_frame_seq, 2,
"VAD should consume only the 240 samples appended by the current ingest and complete one pending 10 ms frame"
);
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
#[test]
fn desktop_capture_silero_stale_enqueued_worker_uses_fallback() {
let gate = crate::ptt::AudioTransmitGate::new(false);
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
selector.set_mode(crate::TransmitMode::VoiceActivity);
selector.set_in_channel(true);
let encoder =
crate::opus_voice::new_voip_encoder("desktop stale Silero fallback test").unwrap();
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
let frames_sent = Arc::new(AtomicU32::new(0));
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
voice_out_tx,
frames_sent,
"desktop-stale-silero-fallback-test",
)
.unwrap();
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
let config = Arc::new(Mutex::new(crate::AudioProcessingConfig {
vad_backend: crate::VadBackend::SileroOnnx,
..crate::AudioProcessingConfig::default()
}));
let worker = Arc::new(Mutex::new(Some(
crate::vad::silero_onnx::SileroOnnxVadWorker::stale_test_worker(),
)));
let mut capture = CaptureState::new(
encoder,
SAMPLE_RATE,
1,
1.0,
voice_out_tx,
gate.flag_arc(),
Some(selector),
config.clone(),
worker,
stats.clone(),
);
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
for (idx, sample) in voiced.iter_mut().enumerate() {
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
*sample = phase.sin() * 0.4;
}
capture.process_10ms_capture_frame(&voiced);
let snapshot = stats.snapshot(&config.lock().unwrap());
assert!(
snapshot.vad_fallback_active,
"stale Silero worker output should report active WebRTC fallback"
);
assert!(
snapshot.vad_probability > 0.5,
"stale Silero worker output should use WebRTC fallback probability instead of forced silence"
);
}
#[test]
fn android_startup_rollback_releases_acquired_mode_snapshot() {
let mut stack = crate::mode_stack::ModeStack::new();
@@ -1806,6 +2197,9 @@ fn try_open_capture(
transmit_active: Arc<AtomicBool>,
frames_sent: Arc<AtomicU32>,
mic_gain: f32,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
) -> Result<cpal::Stream, AudioError> {
let in_cfg = in_dev
@@ -1845,6 +2239,9 @@ fn try_open_capture(
"cpal-capture",
)?,
transmit_active,
voice_activity_selector,
audio_processing_config,
silero_vad_worker,
audio_processing_stats,
)));
@@ -1883,6 +2280,17 @@ struct CaptureState {
/// The PTT transmission gate. Read once per outbound frame; the
/// CaptureState never mutates this flag.
transmit_active: Arc<AtomicBool>,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
vad_detector: crate::vad::WebRtcFallbackVad,
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
silero_model_epoch: u64,
current_vad_backend: crate::VadBackend,
fallback_warned_backend: Option<crate::VadBackend>,
capture_frame_seq: u64,
vad_state: crate::voice_activity::VoiceActivityStateMachine,
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
pending_10ms: [f32; crate::frame::FRAME_10MS_SAMPLES],
pending_10ms_len: usize,
/// Pre-allocated mono downmix buffer. Resized in-place each
/// callback; `clear()` retains capacity. SDD-094 realtime-thread
/// invariant: this avoids the heap allocation that the prior fix
@@ -1926,6 +2334,9 @@ impl CaptureState {
mic_gain: f32,
voice_out_tx: crate::opus_voice::EncodedVoiceFrameSender,
transmit_active: Arc<AtomicBool>,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
) -> Self {
Self {
@@ -1939,6 +2350,17 @@ impl CaptureState {
opus_out: [0u8; crate::opus_voice::MAX_OPUS_FRAME],
voice_out_tx,
transmit_active,
voice_activity_selector,
vad_detector: crate::vad::WebRtcFallbackVad::default(),
silero_vad_worker,
silero_model_epoch: crate::vad::silero_model_epoch(),
current_vad_backend: crate::VadBackend::Disabled,
fallback_warned_backend: None,
capture_frame_seq: 0,
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
audio_processing_config,
pending_10ms: [0.0; crate::frame::FRAME_10MS_SAMPLES],
pending_10ms_len: 0,
mono_scratch: Vec::with_capacity(4096),
frame_scratch: Vec::with_capacity(FRAME_SAMPLES),
audio_processing_stats,
@@ -1952,6 +2374,16 @@ impl CaptureState {
/// Consume an arbitrary-rate, multichannel cpal buffer; produce
/// 48 kHz mono frames; encode and send when `transmit_active`
/// is true (PTT engaged).
// TODO(realtime-audio): This method runs on the cpal audio callback
// thread with a ~10 ms deadline. Known pre-existing violations of the
// realtime safety constraint that should be addressed in a future
// iteration:
// 1. Blocking Mutex::lock().unwrap() on self (via cpal callback)
// 2. Potential allocation in mono_scratch.reserve() and
// pcm_accum.extend_from_slice() when buffer capacity is exceeded
// 3. Encode-path warn!/error! logging via send_voip_frame closures
// These were present before the VAD integration and are not
// introduced by this changeset.
fn ingest<T: ToF32 + Copy>(&mut self, buf: &[T]) {
// 1. Down-mix to mono (pre-gain). Always performed so the level
// meter reflects real mic input even when PTT is released.
@@ -1981,15 +2413,11 @@ impl CaptureState {
}
}
if !self.transmit_active.load(Ordering::Relaxed) {
self.pcm_accum.clear();
return;
}
// 2. Resample to 48 kHz if needed. We re-borrow
// `mono_scratch` as a shared slice per branch to satisfy
// the borrow checker against `&mut self` on the
// resample path.
let vad_start_offset = self.pcm_accum.len();
if self.in_sample_rate == SAMPLE_RATE {
// Disjoint-borrow: copy the slice into pcm_accum without
// aliasing &mut self.
@@ -2008,6 +2436,13 @@ impl CaptureState {
self.mono_scratch = mono;
}
self.process_pending_vad_frames(vad_start_offset);
if !self.transmit_active.load(Ordering::Relaxed) {
self.pcm_accum.clear();
return;
}
// 3. Encode any complete frames. Clamp each sample to
// [-1.0, 1.0] before handing to libopus's float encoder —
// out-of-range samples are hard-clipped inside libopus,
@@ -2055,6 +2490,179 @@ impl CaptureState {
}
}
fn process_pending_vad_frames(&mut self, start_offset: usize) {
let mut offset = start_offset.min(self.pcm_accum.len());
while offset < self.pcm_accum.len() {
let remaining = crate::frame::FRAME_10MS_SAMPLES - self.pending_10ms_len;
let take = remaining.min(self.pcm_accum.len() - offset);
self.pending_10ms[self.pending_10ms_len..self.pending_10ms_len + take]
.copy_from_slice(&self.pcm_accum[offset..offset + take]);
self.pending_10ms_len += take;
offset += take;
if self.pending_10ms_len == crate::frame::FRAME_10MS_SAMPLES {
let frame = self.pending_10ms;
self.process_10ms_capture_frame(&frame);
self.pending_10ms_len = 0;
}
}
}
fn mark_vad_fallback_active(&mut self, failed_backend: crate::VadBackend) {
// Realtime callback: do not log here. Publish state via atomics
// and let a non-realtime consumer translate transitions into
// info/warn events. The transmit/diagnostic stats stream
// already exposes vad_fallback_active for this purpose.
self.fallback_warned_backend = Some(failed_backend);
}
fn sync_vad_backend(&mut self, voice_activity_mode: bool, vad_backend: crate::VadBackend) {
if !voice_activity_mode {
self.current_vad_backend = crate::VadBackend::Disabled;
self.fallback_warned_backend = None;
self.audio_processing_stats.set_vad_fallback_active(false);
return;
}
let silero_epoch = crate::vad::silero_model_epoch();
let silero_changed =
vad_backend == crate::VadBackend::SileroOnnx && silero_epoch != self.silero_model_epoch;
if vad_backend == self.current_vad_backend && !silero_changed {
return;
}
self.current_vad_backend = vad_backend;
self.silero_model_epoch = silero_epoch;
self.fallback_warned_backend = None;
self.vad_state.reset();
match vad_backend {
crate::VadBackend::SileroOnnx => {
let worker_available = self
.silero_vad_worker
.try_lock()
.map(|worker| worker.is_some())
.unwrap_or(false);
if worker_available {
self.audio_processing_stats.set_vad_fallback_active(false);
} else {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
}
}
crate::VadBackend::WebrtcVad => {
self.audio_processing_stats.set_vad_fallback_active(false);
}
crate::VadBackend::EnergyDebug => {
self.audio_processing_stats.set_vad_fallback_active(true);
}
crate::VadBackend::Disabled => {
self.audio_processing_stats.set_vad_fallback_active(false);
}
}
}
fn process_10ms_capture_frame(&mut self, frame: &[f32; crate::frame::FRAME_10MS_SAMPLES]) {
let input_dbfs = crate::frame::dbfs(frame);
let (vad_backend, vad_hangover) = self
.audio_processing_config
.try_lock()
.map(|cfg| (cfg.vad_backend, cfg.vad_hangover_ms))
.unwrap_or((
crate::VadBackend::WebrtcVad,
crate::voice_activity::VAD_HANGOVER_MS,
));
let voice_activity_mode = self
.voice_activity_selector
.as_ref()
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
.unwrap_or(false);
if voice_activity_mode {
self.sync_vad_backend(true, vad_backend);
self.vad_state.configure(
crate::voice_activity::VAD_OPEN_AFTER_MS,
vad_hangover,
crate::voice_activity::VAD_MIN_TX_MS,
);
} else {
self.sync_vad_backend(false, vad_backend);
}
let (vad_probability, gate_open, used_fallback_vad) = if voice_activity_mode {
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
let capture_seq = self.capture_frame_seq;
let mut used_fallback_vad = false;
let vad = match vad_backend {
crate::VadBackend::Disabled => crate::vad::VadOutput {
probability: 1.0,
speech: true,
},
crate::VadBackend::SileroOnnx => {
// Single `try_lock` that both probes availability and
// sends the frame. The guard is scoped to the block
// so it drops before the fallback path (which needs
// `&mut self` for `mark_vad_fallback_active`).
// Returns Some(VadOutput) on a successful, non-stale
// send; None means "fall back to WebRTC VAD".
let worker_output = {
let guard = self.silero_vad_worker.try_lock().ok();
guard.and_then(|guard| {
let worker = guard.as_ref()?;
if worker.try_send(capture_seq, frame) && !worker.is_stale(capture_seq)
{
let p = worker.latest_probability();
Some(crate::vad::VadOutput {
probability: p,
speech: p >= 0.5,
})
} else {
None
}
})
};
if let Some(output) = worker_output {
output
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
frame,
)
}
}
crate::VadBackend::WebrtcVad | crate::VadBackend::EnergyDebug => {
used_fallback_vad = vad_backend == crate::VadBackend::EnergyDebug;
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, frame)
}
};
(
vad.probability,
self.vad_state.update(vad.speech),
used_fallback_vad,
)
} else {
(0.0, false, false)
};
self.audio_processing_stats
.set_vad_fallback_active(used_fallback_vad);
let vad_active = voice_activity_mode && gate_open;
if let Some(selector) = &self.voice_activity_selector {
selector.set_voice_activity_open(vad_active);
}
self.audio_processing_stats.update_capture(
input_dbfs,
input_dbfs,
vad_probability,
vad_active,
self.transmit_active.load(Ordering::Relaxed),
);
self.audio_processing_stats
.record_capture_frame(frame.iter().all(|sample| sample.abs() <= 0.000_001));
}
/// Simple linear resampler for `in_sample_rate → 48000`.
///
/// The resampler maintains continuity across cpal buffer
@@ -2144,8 +2752,8 @@ where
{
let stream = device
.build_input_stream(
config,
move |data: &[T], _| {
*config,
move |data: &[T], _: &cpal::InputCallbackInfo| {
let mut s = state.lock().unwrap();
s.ingest(data);
},
@@ -2211,8 +2819,8 @@ where
.unwrap_or_else(std::time::Instant::now);
let stream = device
.build_output_stream(
config,
move |out: &mut [T], _| {
*config,
move |out: &mut [T], _: &cpal::OutputCallbackInfo| {
let cb_start = std::time::Instant::now();
let muted = output_muted.load(Ordering::Relaxed);
let dev_frames = out.len() / dev_channels.max(1);
@@ -2620,14 +3228,25 @@ pub mod bench_seam {
let (tx, rx) = mpsc::channel::<OutPacket>(64);
let transmit_active = Arc::new(AtomicBool::new(true));
let frames_sent = Arc::new(AtomicU32::new(0));
// Bridge the bench's private mpsc<OutPacket> to the
// realtime-thread-safe EncodedVoiceFrameSender that
// CaptureState expects. The worker task forwards
// encoded Opus frames to `tx` via `frames_sent`.
let voice_out_tx =
crate::opus_voice::start_out_packet_worker(tx, frames_sent, "cpal-bench")
.expect("start_out_packet_worker");
let state = CaptureState::new(
encoder,
in_sample_rate,
in_channels,
1.0,
tx,
voice_out_tx,
transmit_active.clone(),
frames_sent,
None,
Arc::new(std::sync::Mutex::new(
crate::AudioProcessingConfig::default(),
)),
Arc::new(std::sync::Mutex::new(None)),
Arc::new(crate::SharedAudioProcessingStats::default()),
);
Self {
crates/chanora_audio/src/ptt_backends/windows.rs
+48 -36
View File
@@ -26,7 +26,7 @@ use std::thread;
use tracing::{info, warn};
use windows::core::{w, PCWSTR};
use windows::Win32::Foundation::{HMODULE, HWND, LPARAM, LRESULT, WPARAM};
use windows::Win32::Foundation::{HINSTANCE, HMODULE, HWND, LPARAM, LRESULT, WPARAM};
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::UI::Input::{
GetRawInputData, RegisterRawInputDevices, HRAWINPUT, RAWINPUT, RAWINPUTDEVICE, RAWINPUTHEADER,
@@ -35,7 +35,7 @@ use windows::Win32::UI::Input::{
use windows::Win32::UI::WindowsAndMessaging::{
CallNextHookEx, CreateWindowExW, DefWindowProcW, DispatchMessageW, GetMessageW,
PostThreadMessageW, RegisterClassExW, SetWindowsHookExW, TranslateMessage, UnhookWindowsHookEx,
HC_ACTION, HHOOK, HOOKPROC, KBDLLHOOKSTRUCT, MSG, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
HC_ACTION, HOOKPROC, KBDLLHOOKSTRUCT, MSG, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
WINDOW_EX_STYLE, WINDOW_STYLE, WM_INPUT, WM_KEYDOWN, WM_KEYUP, WM_QUIT, WM_SYSKEYDOWN,
WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP, WNDCLASSEXW, XBUTTON1, XBUTTON2,
};
@@ -423,7 +423,7 @@ unsafe fn run_raw_input_loop(
// class.
let _atom = RegisterClassExW(&wc);
let hwnd = unsafe {
let hwnd = match unsafe {
CreateWindowExW(
WINDOW_EX_STYLE(0),
class_name,
@@ -433,13 +433,23 @@ unsafe fn run_raw_input_loop(
0,
0,
0,
HWND(HWND_MESSAGE_PTR),
Some(HWND(HWND_MESSAGE_PTR as *mut core::ffi::c_void)),
None,
h_instance,
Some(HINSTANCE(h_instance.0)),
None,
)
} {
Ok(h) => h,
Err(_) => {
warn!(
target: "chanora_audio",
"windows ptt: CreateWindowExW(HWND_MESSAGE) returned null"
);
report!(false);
return false;
}
};
if hwnd.0 == 0 {
if hwnd.0.is_null() {
warn!(
target: "chanora_audio",
"windows ptt: CreateWindowExW(HWND_MESSAGE) returned null"
@@ -517,13 +527,13 @@ unsafe fn run_raw_input_loop(
usUsagePage: 0x01,
usUsage: 0x06,
dwFlags: RIDEV_REMOVE,
hwndTarget: HWND(0),
hwndTarget: HWND(std::ptr::null_mut()),
},
RAWINPUTDEVICE {
usUsagePage: 0x01,
usUsage: 0x02,
dwFlags: RIDEV_REMOVE,
hwndTarget: HWND(0),
hwndTarget: HWND(std::ptr::null_mut()),
},
];
let _ = RegisterRawInputDevices(&undo, std::mem::size_of::<RAWINPUTDEVICE>() as u32);
@@ -544,7 +554,7 @@ unsafe extern "system" fn raw_input_wnd_proc(
}
unsafe fn handle_wm_input(lparam: LPARAM) {
let h_raw = HRAWINPUT(lparam.0);
let h_raw = HRAWINPUT(lparam.0 as *mut core::ffi::c_void);
let mut size: u32 = 0;
let header_sz = std::mem::size_of::<RAWINPUTHEADER>() as u32;
// First call: query buffer size.
@@ -841,31 +851,33 @@ unsafe fn run_hook_loop(
let kbd_proc: HOOKPROC = Some(kbd_hook_proc);
let mouse_proc: HOOKPROC = Some(mouse_hook_proc);
let kbd_hook = match SetWindowsHookExW(WH_KEYBOARD_LL, kbd_proc, h_instance, 0) {
Ok(h) => h,
Err(e) => {
warn!(
target: "chanora_audio",
error = %e,
"windows ptt: SetWindowsHookExW(WH_KEYBOARD_LL) failed"
);
report!(false);
return false;
}
};
let mouse_hook = match SetWindowsHookExW(WH_MOUSE_LL, mouse_proc, h_instance, 0) {
Ok(h) => h,
Err(e) => {
warn!(
target: "chanora_audio",
error = %e,
"windows ptt: SetWindowsHookExW(WH_MOUSE_LL) failed"
);
let _ = UnhookWindowsHookEx(kbd_hook);
report!(false);
return false;
}
};
let kbd_hook =
match SetWindowsHookExW(WH_KEYBOARD_LL, kbd_proc, Some(HINSTANCE(h_instance.0)), 0) {
Ok(h) => h,
Err(e) => {
warn!(
target: "chanora_audio",
error = %e,
"windows ptt: SetWindowsHookExW(WH_KEYBOARD_LL) failed"
);
report!(false);
return false;
}
};
let mouse_hook =
match SetWindowsHookExW(WH_MOUSE_LL, mouse_proc, Some(HINSTANCE(h_instance.0)), 0) {
Ok(h) => h,
Err(e) => {
warn!(
target: "chanora_audio",
error = %e,
"windows ptt: SetWindowsHookExW(WH_MOUSE_LL) failed"
);
let _ = UnhookWindowsHookEx(kbd_hook);
report!(false);
return false;
}
};
info!(
target: "chanora_audio",
@@ -908,7 +920,7 @@ unsafe extern "system" fn kbd_hook_proc(code: i32, wparam: WPARAM, lparam: LPARA
}
});
}
CallNextHookEx(HHOOK(0), code, wparam, lparam)
CallNextHookEx(None, code, wparam, lparam)
}
/// Pure-logic dispatcher for a low-level keyboard hook event (L0
@@ -943,7 +955,7 @@ unsafe extern "system" fn mouse_hook_proc(code: i32, wparam: WPARAM, lparam: LPA
}
});
}
CallNextHookEx(HHOOK(0), code, wparam, lparam)
CallNextHookEx(None, code, wparam, lparam)
}
/// Pure-logic dispatcher for a low-level mouse hook event (L0
crates/chanora_audio/src/vad/mod.rs
+27 -61
View File
@@ -8,17 +8,17 @@
#[cfg(any(target_os = "ios", target_os = "macos"))]
pub mod apple_coreml;
pub mod resampler;
#[cfg(not(target_os = "ios"))]
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
pub mod silero_onnx;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{OnceLock, RwLock};
use crate::frame::{f32_to_i16, i16_to_f32};
use crate::{AudioError, VadBackend};
use crate::AudioError;
use resampler::{Downsampler48to16, INPUT_FRAME_10MS};
#[cfg(not(target_os = "ios"))]
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
pub use silero_onnx::SileroOnnxVad;
/// Voice activity detector output for one 10 ms frame.
@@ -108,39 +108,12 @@ pub fn process_i16_10ms(detector: &mut dyn VoiceActivityDetector, samples: &[i16
detector.process_10ms(&frame)
}
/// Callback-side policy for optional model-backed VAD workers.
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
pub(crate) enum VadWorkerPolicy {
/// Keep using the already-available model worker.
UseWorker,
/// No worker may be constructed on the callback thread; use WebRTC fallback.
UseFallback,
/// This backend does not need a model worker.
NotModelBacked,
}
/// Decide whether a realtime callback may use a model-backed VAD worker.
///
/// Model/worker construction is intentionally absent from this policy: if a
/// worker is not already present, callbacks must stay nonblocking and fall back.
pub(crate) fn callback_vad_worker_policy(
voice_activity_mode: bool,
backend: VadBackend,
worker_available: bool,
) -> VadWorkerPolicy {
if !voice_activity_mode || backend != VadBackend::SileroOnnx {
return VadWorkerPolicy::NotModelBacked;
}
if worker_available {
VadWorkerPolicy::UseWorker
} else {
VadWorkerPolicy::UseFallback
}
}
static SILERO_MODEL_PATH_OVERRIDE: OnceLock<RwLock<Option<String>>> = OnceLock::new();
static SILERO_MODEL_EPOCH: AtomicU64 = AtomicU64::new(0);
#[cfg(test)]
pub(crate) static SILERO_MODEL_PATH_TEST_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
fn silero_model_path_override() -> &'static RwLock<Option<String>> {
SILERO_MODEL_PATH_OVERRIDE.get_or_init(|| RwLock::new(None))
}
@@ -175,6 +148,19 @@ pub fn silero_model_epoch() -> u64 {
SILERO_MODEL_EPOCH.load(Ordering::Relaxed)
}
#[cfg(test)]
pub(crate) fn clear_silero_model_path_for_test() {
set_silero_model_path_for_test(None);
}
#[cfg(test)]
pub(crate) fn set_silero_model_path_for_test(path: Option<String>) {
if let Ok(mut guard) = silero_model_path_override().write() {
*guard = path;
SILERO_MODEL_EPOCH.fetch_add(1, Ordering::Relaxed);
}
}
/// Return the expected path of the Silero VAD v6 ONNX model on
/// supported platforms.
/// The model is shipped as a Flutter asset and copied to the app's
@@ -264,13 +250,20 @@ mod tests {
#[test]
fn set_silero_model_path_rejects_missing_file() {
let _guard = SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
clear_silero_model_path_for_test();
let result = set_silero_model_path("/definitely/not/a/silero_vad.onnx");
assert!(result.is_err());
clear_silero_model_path_for_test();
}
#[test]
fn set_silero_model_path_updates_override_and_epoch() {
let _guard = SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
clear_silero_model_path_for_test();
let path =
std::env::temp_dir().join(format!("chanora_test_silero_{}.onnx", std::process::id()));
std::fs::write(&path, b"test").unwrap();
@@ -280,34 +273,7 @@ mod tests {
assert!(silero_model_epoch() > before);
assert_eq!(silero_model_bundle_path(), path.to_string_lossy());
clear_silero_model_path_for_test();
let _ = std::fs::remove_file(path);
}
#[test]
fn callback_policy_uses_existing_model_worker_only() {
assert_eq!(
callback_vad_worker_policy(true, VadBackend::SileroOnnx, true),
VadWorkerPolicy::UseWorker
);
assert_eq!(
callback_vad_worker_policy(true, VadBackend::SileroOnnx, false),
VadWorkerPolicy::UseFallback
);
}
#[test]
fn callback_policy_keeps_disabled_and_webrtc_paths_worker_free() {
assert_eq!(
callback_vad_worker_policy(false, VadBackend::SileroOnnx, false),
VadWorkerPolicy::NotModelBacked
);
assert_eq!(
callback_vad_worker_policy(true, VadBackend::Disabled, false),
VadWorkerPolicy::NotModelBacked
);
assert_eq!(
callback_vad_worker_policy(true, VadBackend::WebrtcVad, false),
VadWorkerPolicy::NotModelBacked
);
}
}
crates/chanora_audio/src/vad/silero_onnx.rs
+33 -1
View File
@@ -361,6 +361,31 @@ impl SileroOnnxVadWorker {
})
}
#[cfg(test)]
pub(crate) fn stale_test_worker() -> Self {
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(64);
let alive = Arc::new(AtomicBool::new(true));
let alive_for_thread = alive.clone();
let handle = std::thread::Builder::new()
.name("chanora-silero-vad-stale-test".to_string())
.spawn(move || {
while alive_for_thread.load(Ordering::Relaxed) {
match rx.recv_timeout(std::time::Duration::from_millis(10)) {
Ok(_) | Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,
}
}
})
.ok();
Self {
tx: Some(tx),
latest_probability: Arc::new(AtomicU32::new(0.0_f32.to_bits())),
latest_processed_seq: Arc::new(AtomicU64::new(u64::MAX)),
alive,
handle,
}
}
/// Best-effort enqueue of a 10 ms frame for background inference.
pub fn try_send(&self, seq: u64, frame: &[f32; super::resampler::INPUT_FRAME_10MS]) -> bool {
let Some(tx) = &self.tx else {
@@ -393,8 +418,15 @@ impl SileroOnnxVadWorker {
impl Drop for SileroOnnxVadWorker {
fn drop(&mut self) {
self.alive.store(false, Ordering::Relaxed);
// Drop the sender first so the worker thread's rx.recv() returns
// Err and the loop exits promptly.
let _ = self.tx.take();
let _ = self.handle.take();
// Join the thread instead of detaching. The channel close
// unblocks rx.recv() so the join is bounded; it waits at most
// until the current in-flight inference completes.
if let Some(handle) = self.handle.take() {
let _ = handle.join();
}
}
}
crates/chanora_audio/tests/ptt_privacy.rs
+2 -2
View File
@@ -196,8 +196,8 @@ fn windows_exercise() {
// start/stop lifecycle through the public trait surface so
// any info!/warn! the factory or the backend's `start` path
// emits is captured by the layer.
use chanora_audio::ptt_backends::{select_ptt_backend, PttBinding, PttInputClass};
use chanora_audio::AudioTransmitGate;
use chanora_audio::ptt_backends::{PttBinding, PttInputClass};
use chanora_audio::{select_ptt_backend, AudioTransmitGate};
let mut backend = select_ptt_backend();
let gate = AudioTransmitGate::new(false);