edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(voice): real-time mic input level metering at 30 Hz (#25)

Browse Source 
* feat(voice): add real-time mic input level metering at 30 Hz

Expose input RMS from the audio engine through the bridge as a
dedicated Rust→Dart Stream<double>, replacing the binary on/off
indicator with a proportional dBFS level meter.

Rust side:
- chanora_audio: add set_input_dbfs/input_dbfs accessors to
  SharedAudioProcessingStats; restructure CaptureState::ingest()
  to compute dBFS from mono buffer before the PTT guard so the
  meter shows mic activity even when not transmitting.
- chanora_core: widen audio_stats() return to include f32 input
  level.
- chanora_bridge: add input_level: f32 to BridgeAudioStats and
  new input_level_stream(sink: StreamSink<f32>) that pushes at
  ~30 Hz via tokio interval task.
- Update frb_generated.rs serialization for the new field.

Flutter side:
- VoiceLevelMeter: accept optional double level (dBFS), map
  -60..0 dBFS to 0..1 fill fraction, animate with
  TweenAnimationBuilder for smooth transitions.
- voice_compact.dart: subscribe to inputLevelStream in the voice
  details sheet for 30 Hz meter updates, keeping 250 ms poll for
  TX/RX counters.
- voice_bar.dart: accept optional inputLevel from the stream.
- main.dart: subscribe to inputLevelStream, pass to VoiceBar.

* chore: sync Flutter build config and dependency updates

- Add Flutter migrator flags to gradle.properties (builtInKotlin, newDsl)
- Add FlutterGeneratedPluginSwiftPackage to iOS/macOS Xcode projects
- Update meta 1.17→1.18, test_api 0.7.10→0.7.11
- Rebuild chanora_bridge framework for macOS
- Update Podfile.lock for iOS and macOS

* fix(voice): correct meter animation, pre-gain dBFS, stream lifecycle, and protocol warnings

B1: Convert VoiceLevelMeter to StatefulWidget tracking previous fill
     as Tween begin so the meter animates smoothly instead of resetting
     to zero on every frame.

B2: Compute dBFS from pre-gain mono samples in CaptureState::ingest()
     so the level meter reflects raw mic input, matching mobile paths.

B4: End input_level_stream after 10 consecutive session errors instead
     of emitting -120 dBFS forever when the session is gone.

Also fixes all 13 clippy warnings in chanora_protocol: collapsed
nested if-let patterns, replaced .ok() + Some matching with Ok, used
? operator, and introduced EventChannels struct to reduce the four
helper functions below the 7-argument threshold.

* fix(voice): use MissedTickBehavior::Skip for level meter stream and align dBFS doc

Set MissedTickBehavior::Skip on the input_level_stream tokio interval
so slow audio_stats() calls skip missed ticks instead of bursting,
preventing CPU spikes on the UI meter thread.

Align VoiceLevelMeter class doc: the mapping floors at -60 dBFS
(via dbfsToFraction), not the full -120 range.
This commit is contained in:
Edison Jwa
2026-06-05 20:57:16 +09:00
committed by GitHub
parent 2c7b68e21e
commit 82441f3d97
25 changed files with 519 additions and 258 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/chanora_audio/src/audio_processing.rs
+13
View File
@@ -404,6 +404,19 @@ impl Default for SharedAudioProcessingStats {
}
impl SharedAudioProcessingStats {
/// Store the raw input dBFS level (desktop capture path).
/// Mobile platforms use [`Self::update_capture`] instead, which
/// also records VAD state; this lighter method is for the cpal
/// capture path that has no VAD pipeline.
pub fn set_input_dbfs(&self, dbfs: f32) {
self.input_dbfs.store(dbfs.to_bits(), Ordering::Relaxed);
}
/// Read the current input dBFS level.
pub fn input_dbfs(&self) -> f32 {
f32::from_bits(self.input_dbfs.load(Ordering::Relaxed))
}
/// Store capture levels and VAD state.
pub fn update_capture(
&self,
crates/chanora_audio/src/engine.rs
+31 -17
View File
@@ -838,6 +838,7 @@ impl AudioEngine {
transmit_flag_for_capture,
frames_sent.clone(),
cfg.mic_gain,
audio_processing_stats.clone(),
);
let (input_stream, capture_active) = match capture_result {
Ok(s) => (Some(s), true),
@@ -1597,6 +1598,11 @@ impl AudioEngine {
self.frames_received.load(Ordering::Relaxed)
}
/// Current microphone input level in dBFS (-120.0 = silence, 0.0 = clipping).
pub fn input_level(&self) -> f32 {
self.audio_processing_stats.input_dbfs()
}
/// Current audio-processing config snapshot.
pub fn audio_processing_config_snapshot(&self) -> crate::AudioProcessingConfig {
self.audio_processing_config.lock().unwrap().clone()
@@ -1717,6 +1723,7 @@ fn try_open_capture(
transmit_active: Arc<AtomicBool>,
frames_sent: Arc<AtomicU32>,
mic_gain: f32,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
) -> Result<cpal::Stream, AudioError> {
let in_cfg = in_dev
.default_input_config()
@@ -1752,6 +1759,7 @@ fn try_open_capture(
voice_out_tx,
transmit_active,
frames_sent,
audio_processing_stats,
)));
let stream = match in_format {
@@ -1803,6 +1811,7 @@ struct CaptureState {
/// capacity so the drain-into-frame path skips the allocator
/// after warmup. Same precedent as `mono_scratch` above.
frame_scratch: Vec<f32>,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
@@ -1815,6 +1824,7 @@ impl CaptureState {
voice_out_tx: mpsc::Sender<OutPacket>,
transmit_active: Arc<AtomicBool>,
frames_sent: Arc<AtomicU32>,
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
) -> Self {
Self {
encoder,
@@ -1828,12 +1838,9 @@ impl CaptureState {
voice_out_tx,
transmit_active,
frames_sent,
// Generous upper bound for typical cpal periods
// (commonly 256..1024 frames); `clear()` retains the
// backing allocation across callbacks. See struct doc.
mono_scratch: Vec::with_capacity(4096),
// Exact upper bound: drain pulls FRAME_SAMPLES at a time.
frame_scratch: Vec::with_capacity(FRAME_SAMPLES),
audio_processing_stats,
}
}
@@ -1841,17 +1848,8 @@ impl CaptureState {
/// 48 kHz mono frames; encode and send when `transmit_active`
/// is true (PTT engaged).
fn ingest<T: ToF32 + Copy>(&mut self, buf: &[T]) {
if !self.transmit_active.load(Ordering::Relaxed) {
// Drain accumulator while muted so we don't pop on PTT release.
self.pcm_accum.clear();
return;
}
// 1. Down-mix to mono + gain.
// Reuse `self.mono_scratch` to avoid a per-callback Vec
// allocation on the realtime audio thread; see struct
// doc and the engine.rs:1389-1397 precedent for why this
// matters for user-perceptible audio popping.
// 1. Down-mix to mono (pre-gain). Always performed so the level
// meter reflects real mic input even when PTT is released.
let in_channels = self.in_channels;
let mic_gain = self.mic_gain;
self.mono_scratch.clear();
@@ -1859,8 +1857,23 @@ impl CaptureState {
self.mono_scratch.reserve(frame_count);
for frame in buf.chunks(in_channels) {
let sum: f32 = frame.iter().map(|s| s.to_f32_sample()).sum();
self.mono_scratch
.push((sum / frame.len() as f32) * mic_gain);
self.mono_scratch.push(sum / frame.len() as f32);
}
// Measure dBFS from pre-gain samples so the level meter
// reflects the raw mic input, not the amplified signal.
self.audio_processing_stats
.set_input_dbfs(crate::frame::dbfs(&self.mono_scratch));
if mic_gain != 1.0 {
for s in &mut self.mono_scratch {
*s *= mic_gain;
}
}
if !self.transmit_active.load(Ordering::Relaxed) {
self.pcm_accum.clear();
return;
}
// 2. Resample to 48 kHz if needed. We re-borrow
@@ -2488,6 +2501,7 @@ pub mod bench_seam {
tx,
transmit_active.clone(),
frames_sent,
Arc::new(crate::SharedAudioProcessingStats::default()),
);
Self {
state,