edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audio,ios): wire VPIO render callback to AudioHandler (commit 4/5, rc.8+64)

Browse Source 
Replace the silence-emitting render callback from commit 1 with
real playback that drives AudioHandler::fill_buffer and downmixes
its 48 kHz stereo f32 output to the i16 mono buffer VPIO expects.

Pipeline per render callback (mirrors the cpal-output + sdl_output
contracts so the platform-neutral playback path is preserved):

1. Lock the shared Arc<Mutex<AudioHandler>>, ask fill_buffer to
   populate a stereo-f32 scratch slice of length 2*num_frames.
   AudioHandler runs Opus decode + per-client jitter buffer + mix
   internally. Same primitive every other platform calls.

2. If output_muted is true, zero the i16 output buffer and return.
   We still ran fill_buffer in step 1 so the jitter buffer drains
   while muted — preventing unbounded growth — which matches the
   cpal/SDL backend contract.

3. Downmix stereo -> mono with master gain:
       mono_f32 = (l + r) * 0.5 * gain
       i16_out  = (mono_f32.clamp(-1.0, 1.0) * i16::MAX) as i16
   The 0.5 average preserves total signal energy with 3 dB
   headroom against sum-of-correlated-peaks clipping. Multiply by
   gain after the downmix saves one mul per sample. Hard-clip on
   the i16 cast is acceptable because the upstream stereo signal
   is already in [-1.0, 1.0] from the f32 mix; only gain >1.0
   creates clipping pressure and that path is identical to every
   other backend's i16 conversion.

Closure ownership:
* scratch_stereo: Vec<f32> moved into the FnMut closure. First
  callback grows it to 2*num_frames; subsequent callbacks reuse
  the backing allocation. The audio thread never hits the
  allocator on steady-state callbacks.
* handler_for_render / output_gain_for_render / output_muted_for_render
  are Arc clones taken before the closure literal.

Public API change: AudioEngine -> IosVoiceUnit::start parameters
that were previously underscored (commit 1 placeholder) are now
all consumed by the wiring. Signature is unchanged, just the
binder names lose the leading underscore. engine.rs call-site
is unaffected.

Build verify on Mac (target aarch64-apple-ios): cargo check
clean in 0.33s, no errors, no warnings.

Build counter 63 -> 64 — About dialog shows v1.0.0-rc.8+64.
This commit is contained in:
EdisonJwa
2026-05-17 01:12:57 +08:00
parent 1aa514df75
commit e7c3ffa6d2
2 changed files with 97 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/chanora_flutter/pubspec.yaml
+1 -1
View File
@@ -16,7 +16,7 @@ publish_to: 'none' # Remove this line if you wish to publish to pub.dev
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
# In Windows, build-name is used as the major, minor, and patch parts
# of the product and file versions while build-number is used as the build suffix.
version: 1.0.0-rc.8+63
version: 1.0.0-rc.8+64
environment:
sdk: ^3.11.5
crates/chanora_audio/src/ios_voice_unit.rs
+96 -17
View File
@@ -320,10 +320,11 @@ impl IosVoiceUnit {
/// and SDL backends accept so engine.rs can swap backends with
/// a `cfg`.
///
/// * `_handler` — shared AudioHandler the inbound forwarder
/// * `handler` — shared AudioHandler the inbound forwarder
/// feeds Opus packets into. The output render callback pulls
/// decoded f32 frames from it.
/// * `_output_gain` / `_output_muted` — same atomics the cpal
/// decoded f32 frames from it (48 kHz stereo) and downmixes
/// to the i16 mono buffer VPIO expects.
/// * `output_gain` / `output_muted` — same atomics the cpal
/// and SDL output paths read on every callback so the master
/// volume + local-mute UI works identically across backends.
/// * `voice_out_tx` — channel the capture pipeline sends
@@ -333,14 +334,13 @@ impl IosVoiceUnit {
/// * `frames_sent` — counter the bridge stats surface reads.
/// * `mic_gain` — pre-encode amplitude scale.
///
/// Capture wiring landed in commit 3; playback wiring lands
/// in commit 4 (the render callback still emits silence
/// until then).
/// Capture wiring landed in commit 3; playback wiring landed
/// in commit 4. Route-change observation is commit 5.
#[allow(clippy::too_many_arguments)]
pub fn start(
_handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
_output_gain: Arc<AtomicU32>,
_output_muted: Arc<AtomicBool>,
handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
output_gain: Arc<AtomicU32>,
output_muted: Arc<AtomicBool>,
voice_out_tx: mpsc::Sender<OutPacket>,
transmit_active: Arc<AtomicBool>,
frames_sent: Arc<AtomicU32>,
@@ -445,14 +445,93 @@ impl IosVoiceUnit {
})
.map_err(|e| AudioError::Backend(format!("vpio set input callback: {e}")))?;
// Install the render callback that emits playback samples
// to the hardware. The buffer iOS hands us is uninitialised
// — we MUST fill it (writing silence if we have nothing,
// never leaving stale frames). Commit 4 replaces the silence
// loop with an AudioHandler::fill_buffer + downmix path.
unit.set_render_callback(|args: render_callback::Args<data::Interleaved<i16>>| {
for s in args.data.buffer.iter_mut() {
*s = 0;
// Install the render callback that drives playback. The
// buffer iOS hands us is uninitialised — we MUST fill it
// (writing silence if we have nothing, never leaving stale
// frames).
//
// Pipeline per callback:
// 1. Lock the AudioHandler, ask it to fill a scratch
// f32 stereo buffer (length = 2 * num_frames). The
// handler runs Opus decode + per-client jitter
// buffer + mix. Same primitive cpal + SDL output
// paths use; this is the platform-neutral playback
// contract from `tsclientlib::audio::AudioHandler`.
// 2. Downmix to i16 mono with master gain. VPIO expects
// mono int16 (the stream format we pinned above);
// the handler produces stereo f32. We average L+R
// to a single mono channel rather than dropping R —
// the cpal-side mono-output path made the same
// mistake briefly (commit 6a4dbad / fix) and lost
// half the spatial mix.
// 3. Local-mute zeroes the output but STILL drains
// AudioHandler in step 1 so its jitter buffer
// doesn't grow unbounded while muted. This is the
// contract every other backend follows (matches
// SdlOutput::callback and the cpal output stream).
//
// The closure owns the f32 scratch Vec so we re-use the
// backing allocation across callbacks. The first few
// callbacks may grow it; after that the audio thread
// never touches the allocator.
let mut scratch_stereo: Vec<f32> = Vec::with_capacity(FRAME_SAMPLES_MONO * 2);
let handler_for_render = handler.clone();
let output_gain_for_render = output_gain.clone();
let output_muted_for_render = output_muted.clone();
unit.set_render_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
let out: &mut [i16] = args.data.buffer;
// VPIO with our pinned stream format gives us a mono
// buffer; `num_frames` == `out.len()`. Stereo f32
// scratch is twice that.
let num_frames = out.len();
let needed = num_frames * 2;
if scratch_stereo.len() < needed {
scratch_stereo.resize(needed, 0.0);
}
// Zero only the live slice. AudioHandler::fill_buffer
// writes silence into untouched samples but residual
// values from earlier callbacks (when the buffer was
// bigger) would leak through otherwise. `fill`
// optimises to memset.
scratch_stereo[..needed].fill(0.0);
{
let mut h = handler_for_render.lock().unwrap();
let _removed_ids = h.fill_buffer(&mut scratch_stereo[..needed]);
// `_removed_ids` lists clients whose stream just
// finished draining; the bridge layer surfaces
// "stopped speaking" via BridgeEvent::VoiceState
// already, so we ignore it here (matches SDL +
// cpal output behaviour).
}
if output_muted_for_render.load(Ordering::Relaxed) {
for s in out.iter_mut() {
*s = 0;
}
return Ok(());
}
let gain = f32::from_bits(output_gain_for_render.load(Ordering::Relaxed));
// Downmix stereo -> mono with gain. The `(l + r) * 0.5`
// averaging preserves total signal energy with a 3 dB
// headroom (sum-of-correlated peaks at full scale
// would otherwise clip). Multiplying by gain after
// the downmix saves one multiplication per sample.
//
// Cast to i16 with saturate-on-overflow. Hard-clip is
// acceptable here because the upstream signal is
// already in [-1.0, 1.0] from the f32 stereo mix;
// gain values >1.0 are the only path to clipping and
// hard-clip at the engine boundary is what every other
// backend's i16 path does (see the cpal-side
// FromF32 for i16 impl in engine.rs).
for (i, dst) in out.iter_mut().enumerate() {
let l = scratch_stereo[i * 2];
let r = scratch_stereo[i * 2 + 1];
let mono_f32 = (l + r) * 0.5 * gain;
let clamped = mono_f32.clamp(-1.0, 1.0);
*dst = (clamped * i16::MAX as f32) as i16;
}
Ok(())
})