edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio,ios): route playback via media channel (.default + .defaultToSpeaker) (rc.8+70)

Browse Source 
User report after the 8x boost commit (e85a6d3): playback STILL
broken, but now the diagnostic clearly shows the actual problem.
Render-callback peak_out_i16 SATURATES at 32767 on speech peaks
(cb=600, 1100, 1200, 2400) because the 8x boost amplifies an
already-loud signal into hard clipping. Quiet content reaches
audible level but loud peaks are catastrophically distorted.

The 8x boost was treating the wrong cause.

Real root cause (researched online after user prompted: 'this is
iOS a popular platform, there must be solutions'): iOS has TWO
independent audio channels:

  In-call channel  (.voiceChat / .videoChat modes)
    * Routes through the phone-call audio path.
    * Aggressively ducks non-voice content to the earpiece.
    * Volume controlled by a separate in-call hardware
      register, not the side buttons when not actively on a
      phone call.

  Media channel  (.default mode)
    * Routes through the standard media playback path.
    * No automatic ducking.
    * Volume controlled by the side volume buttons normally.

With AVAudioSession mode .voiceChat, iOS sends our output
through the in-call channel which plays at 'earpiece-level'
loudness on the speaker too. Signal is technically present but
buried under the speaker's noise floor. With mode .default +
.defaultToSpeaker option, output routes via media channel and
plays at normal loudness.

Both Twilio (video-quickstart-ios) and Daily.co (patched WebRTC
module) document the same workaround and use VPIO for AEC while
keeping the session mode at .default for loud playback:

  github.com/twilio/video-quickstart-ios/issues/522
  stackoverflow.com/questions/79834998 (Daily.co)

The user also noticed 'tx/rx almost no changes even receiving
packages' \u2014 likely a misinterpretation of the frames counter
not advancing as fast as expected during quiet voice; AudioHandler
returns silence when its jitter buffer is in buffering_samples
state which doesn't fire 'decode failed' but also doesn't
increment frames_received. The real issue is still the playback
ducking; the counter behaviour is a downstream symptom.

Changes:

1. AppDelegate.swift: AVAudioSession mode .voiceChat -> .default
   with options [.defaultToSpeaker, .allowBluetoothHFP,
   .allowBluetoothA2DP]. VPIO continues to do its job (AEC, NS,
   AGC on the mic side); only the playback routing changes.
   The earlier 'speaker selector silent under .default' bug
   does NOT apply because we no longer use cpal RemoteIO \u2014
   VPIO honours overrideOutputAudioPort under any mode.

2. ios_voice_unit.rs: revert the 8x output boost from e85a6d3.
   With media-channel routing, signal levels are correct and
   no software amplification is needed. Render callback restored
   to plain (l+r)*0.5*gain downmix.

3. ios_voice_unit.rs: revert the BypassVoiceProcessing toggle
   from c16318c. The VPIO chain stays enabled so we keep
   capture-side AEC/AGC/NS for free \u2014 the playback breakage
   it was trying to fix was the wrong layer all along.

4. ios_voice_unit.rs: drop the diagnostic render-callback log
   line. Production-clean code; can be re-enabled by reverting
   the diff in the closure if future debugging needs it.

Build counter 69 -> 70.
This commit is contained in:
EdisonJwa
2026-05-17 02:11:07 +08:00
parent e85a6d36d7
commit 6e0bf21295
3 changed files with 96 additions and 190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/chanora_flutter/ios/Runner/AppDelegate.swift
+62 -31
View File
@@ -31,43 +31,74 @@ import AVFoundation
let session = AVAudioSession.sharedInstance()
try session.setCategory(
.playAndRecord,
mode: .voiceChat,
// Mode rationale (revisited after the iOS VPIO migration):
mode: .default,
// Mode rationale (re-revisited after the "low playback
// volume" investigation, May 2026):
//
// We previously tried .default mode to fix the
// "speaker/receiver toggle is silent" bug \u2014 that bug was
// ultimately caused by cpal's iOS RemoteIO unit binding to
// a stale physical transducer, not by .voiceChat itself.
// After migrating to coreaudio-rs + kAudioUnitSubType_VoiceProcessingIO
// We've cycled through .voiceChat -> .default -> .voiceChat
// -> .default. Final answer is .default with
// .defaultToSpeaker, driven by these findings:
//
// The earlier "speaker selector silent" bug under
// .voiceChat was caused by cpal's RemoteIO unit binding
// to a stale physical transducer. After migrating to
// coreaudio-rs + kAudioUnitSubType_VoiceProcessingIO
// (see crates/chanora_audio/src/ios_voice_unit.rs) the
// route binding is correct under either mode because VPIO
// is the canonical voice unit and natively re-binds on
// overrideOutputAudioPort. So .default lost its only
// benefit.
// route binding is correct under either mode because
// VPIO is the canonical voice unit and re-binds on
// overrideOutputAudioPort. So route switching is no
// longer a deciding factor.
//
// Under .default mode, VPIO's output-side voice processing
// chain (echo subtraction, noise gating) interprets low-
// amplitude playback signal as "no farend audio" and
// aggressively gates inter-phoneme content. Symptom in
// testing: musicbot (loud, continuous signal) plays fine,
// human voice (peaks ~-6 dB, average ~-40 dB, classic
// 20 dB peak-to-average ratio) sounds broken and
// unintelligible \u2014 the quiet samples between phonemes
// get gated out, destroying intelligibility.
// The "broken playback quality" bug under either
// .voiceChat or .default (with VPIO) was actually NOT
// a VPIO problem at all. iOS has TWO independent audio
// channels: the in-call channel (used by .voiceChat /
// .videoChat modes) and the media channel (used by
// .default). The in-call channel:
// * Routes through the phone-call audio path
// * Aggressively ducks non-voice content to the
// earpiece (Apple's "speakerphone vs ear" UX)
// * Volume controlled by separate in-call volume
// hardware, not the side buttons (when not in a
// phone call)
// The media channel:
// * Routes through the standard media playback path
// * No automatic ducking
// * Volume controlled by the side volume buttons
//
// Apple's documentation explicitly pairs VPIO with
// AVAudioSessionModeVoiceChat. WebRTC's reference iOS
// ADM implementation uses the same pair. Under
// .voiceChat mode VPIO's internal AGC/AEC/NS thresholds
// are tuned for telephony-style speech and pass quiet
// inter-phoneme content through cleanly.
// Even with VPIO + .voiceChat producing a perfectly
// good signal, iOS's in-call channel routing made it
// play at "earpiece" loudness on the speaker too \u2014
// user-perceived as "broken and poor" because the
// signal is technically there but barely audible against
// the loud iPhone speaker's noise floor.
//
// Category options unchanged \u2014 .allowBluetoothHFP +
// .allowBluetoothA2DP permit BT headsets in both
// directions regardless of mode.
options: [.allowBluetoothHFP, .allowBluetoothA2DP]
// Twilio's video-quickstart-ios and Daily.co's patched
// WebRTC both document the same workaround: use .default
// mode with .defaultToSpeaker option even when using
// VPIO for AEC. The VPIO unit itself still does its job
// (echo cancellation, noise suppression, AGC on the mic
// path) \u2014 only the playback routing changes.
//
// References:
// * https://github.com/twilio/video-quickstart-ios/issues/522
// * https://stackoverflow.com/questions/79834998 (Daily.co)
//
// Options:
// .defaultToSpeaker : route output to the main speaker
// (not the earpiece) by default
// when no headphones are connected.
// This is what makes the audio
// actually audible at normal
// loudness.
// .allowBluetoothHFP : permit Bluetooth Hands-Free
// Profile headsets as both input
// and output.
// .allowBluetoothA2DP : permit higher-quality A2DP
// output-only Bluetooth devices.
options: [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP]
)
NSLog("chanora_flutter: AVAudioSession category set (playAndRecord/voiceChat)")
NSLog("chanora_flutter: AVAudioSession category set (playAndRecord/default + defaultToSpeaker)")
} catch {
NSLog("chanora_flutter: AVAudioSession setCategory failed: \(error)")
}
apps/chanora_flutter/pubspec.yaml
+1 -1
View File
@@ -16,7 +16,7 @@ publish_to: 'none' # Remove this line if you wish to publish to pub.dev
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
# In Windows, build-name is used as the major, minor, and patch parts
# of the product and file versions while build-number is used as the build suffix.
version: 1.0.0-rc.8+69
version: 1.0.0-rc.8+70
environment:
sdk: ^3.11.5
crates/chanora_audio/src/ios_voice_unit.rs
+33 -158
View File
@@ -387,64 +387,15 @@ impl IosVoiceUnit {
)
.map_err(|e| AudioError::Backend(format!("vpio enable input I/O: {e}")))?;
// Bypass VPIO's built-in voice processing (AEC + AGC + NS).
//
// Rationale: with the AVAudioSession in mode .voiceChat
// (the documented VPIO pair), VPIO's output-side gating
// chain still aggressively gates quiet inter-phoneme
// content as "noise". User-reported symptom: musicbot
// audio plays cleanly (loud, near-continuous signal),
// but human voice with normal 20 dB peak-to-average
// ratio sounds broken and unintelligible because the
// average-level content gets chopped out between
// phonemes.
//
// Bypassing voice processing turns VPIO into effectively
// a vanilla I/O unit \u2014 raw mic samples in, raw
// playback out. The trade-offs:
//
// * Capture lost: Apple's AEC + AGC + NS on the mic
// input. The user's current setup reports clean
// capture without these (likely using a headset
// where AEC is unnecessary, or not generating
// echo loud enough to matter). If echo loops back
// when on speakerphone, we'll need to either
// re-enable VPIO selectively or ship software AEC
// (DEC-007).
// * Playback gained: signal passes through verbatim.
// Quiet inter-phoneme samples are no longer gated.
//
// Property constants from Apple's AudioUnitProperties.h
// (also exposed via objc2-audio-toolbox):
// kAUVoiceIOProperty_BypassVoiceProcessing = 2100
//
// Set on (Scope::Global, Element::Input = 1) per WebRTC's
// reference iOS audio device manager
// (voice_processing_audio_unit.mm). The bypass flag is a
// UInt32 with 1 = bypass.
const K_AU_VOICE_IO_PROPERTY_BYPASS_VOICE_PROCESSING: u32 = 2100;
let bypass: u32 = 1;
if let Err(e) = unit.set_property(
K_AU_VOICE_IO_PROPERTY_BYPASS_VOICE_PROCESSING,
Scope::Global,
Element::Input,
Some(&bypass),
) {
// Soft-fail: if the bypass property is rejected on
// an exotic iOS version, log and continue. The unit
// is still usable with default voice processing
// (with whatever quality issues that brings).
warn!(
target: "chanora_audio",
error = %e,
"vpio: BypassVoiceProcessing set failed; continuing with VPIO defaults"
);
} else {
info!(
target: "chanora_audio",
"vpio: voice processing bypassed (no AEC/AGC/NS) for cleaner playback"
);
}
// Note: we keep VPIO's voice processing chain ENABLED
// (AEC + AGC + NS on the mic path) because it gives us
// clean capture for free. The historical playback
// breakage we attributed to this chain (commit c16318c
// tried to bypass it) was actually caused by the
// AVAudioSession mode .voiceChat ducking output to the
// earpiece \u2014 fixed in AppDelegate.swift by switching
// to .default + .defaultToSpeaker. With the session mode
// correct, voice processing can stay on.
// Stream format. Apple's iOS canonical format for
// AudioUnits is Linear PCM, 16-bit signed integer samples
@@ -537,15 +488,6 @@ impl IosVoiceUnit {
let handler_for_render = handler.clone();
let output_gain_for_render = output_gain.clone();
let output_muted_for_render = output_muted.clone();
// Diagnostic: count render-callback invocations and log
// a sampled line every ~100 callbacks (= ~2 seconds @ a
// typical iOS 20-50 Hz callback rate). Lets us correlate
// the user-perceived audio quality with what's actually
// flowing through the callback in realtime: how big the
// VPIO buffer is, whether AudioHandler is returning
// actual content or silence, what the peak / RMS of the
// signal looks like at each pipeline stage.
let mut cb_count: u64 = 0;
unit.set_render_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
let out: &mut [i16] = args.data.buffer;
// VPIO with our pinned stream format gives us a mono
@@ -573,17 +515,6 @@ impl IosVoiceUnit {
// cpal output behaviour).
}
// Compute scratch-stereo peak BEFORE applying gain or
// downmix so the diagnostic line reflects what
// AudioHandler actually produced.
let mut peak_stereo_f32: f32 = 0.0;
for &s in scratch_stereo[..needed].iter() {
let a = s.abs();
if a > peak_stereo_f32 {
peak_stereo_f32 = a;
}
}
if output_muted_for_render.load(Ordering::Relaxed) {
for s in out.iter_mut() {
*s = 0;
@@ -592,93 +523,37 @@ impl IosVoiceUnit {
}
let gain = f32::from_bits(output_gain_for_render.load(Ordering::Relaxed));
// Downmix stereo -> mono with gain. The `(l + r) * 0.5`
// Downmix stereo -> mono with master gain. `(l + r) * 0.5`
// averaging preserves total signal energy with a 3 dB
// headroom (sum-of-correlated peaks at full scale
// would otherwise clip). Multiplying by gain after
// the downmix saves one multiplication per sample.
// headroom against sum-of-correlated-peaks clipping.
// Multiplying by gain after the downmix saves one
// multiplication per sample.
//
// iOS-specific fixed output boost: AudioHandler delivers
// f32 stereo content matching what the remote client
// encoded. Most desktop TS3 clients ship audio at -30
// to -45 dB FS (peak ~0.005-0.03), well below speaker-
// ready levels. On Linux/macOS/Windows our cpal+SDL
// paths play that level through OS audio mixers that
// apply additional system-volume amplification, so it
// reaches the user's ears at sensible loudness. iOS's
// VPIO output is NOT amplified by the system mixer —
// it goes nearly raw to the speaker, so the same -40
// dB signal is barely audible.
//
// We compensate with a fixed 8x boost (= +18 dB) on
// top of the user-controllable output_gain. Brings a
// -40 dB signal up to -22 dB (normal speakerphone
// level) while keeping the user's UI gain slider
// functional in a useful range. Hard-clip at 1.0
// prevents the boost from clipping legitimate loud
// signals (musicbot at peak 0.5 -> 4.0 -> clamped to
// 1.0, audible distortion only on extremely loud
// sustained content).
//
// This is consistent with how Discord / Zoom / FaceTime
// iOS clients apply an internal output normalization
// on top of the user-facing volume slider.
const IOS_OUTPUT_BOOST: f32 = 8.0;
let effective_gain = gain * IOS_OUTPUT_BOOST;
// Cast to i16 with saturate-on-overflow. Hard-clip is
// acceptable here because the upstream signal is
// already in [-1.0, 1.0] from the f32 stereo mix;
// the IOS_OUTPUT_BOOST multiplier above is the
// primary path to clipping pressure and saturating
// at ±1.0 is the standard answer (matches the
// cpal-side FromF32 for i16 impl in engine.rs).
let mut peak_out_i16: i16 = 0;
// acceptable here because the upstream f32 stereo signal
// is already in [-1.0, 1.0] from the AudioHandler mix;
// gain values >1.0 are the only path to clipping and
// hard-clip at the engine boundary matches what every
// other backend's i16 path does (see the cpal-side
// FromF32 for i16 impl in engine.rs).
//
// Note: no iOS-specific output boost. The historical
// "low playback volume" reports on VPIO output stemmed
// from the AVAudioSession mode .voiceChat aggressively
// ducking output to the earpiece. Once the session mode
// is set to .default with .defaultToSpeaker (see
// AppDelegate.swift), the speaker drives at normal
// media-channel loudness and no software boost is
// needed. Pattern documented by Twilio's WebRTC iOS
// SDK + Daily.co; reverts the 8x boost from c16318c +
// e85a6d3 which were treating the symptom not the
// cause.
for (i, dst) in out.iter_mut().enumerate() {
let l = scratch_stereo[i * 2];
let r = scratch_stereo[i * 2 + 1];
let mono_f32 = (l + r) * 0.5 * effective_gain;
let mono_f32 = (l + r) * 0.5 * gain;
let clamped = mono_f32.clamp(-1.0, 1.0);
let sample = (clamped * i16::MAX as f32) as i16;
*dst = sample;
let a = sample.unsigned_abs() as i16;
if a > peak_out_i16 {
peak_out_i16 = a;
}
}
// Diagnostic sample: log once every 100 callbacks
// (~2 s) so we can see what the pipeline is producing
// when the user reports "broken" audio. The fields
// tell us:
// * num_frames — how big each VPIO callback is.
// Should be ~960 (20 ms) or ~1104
// (23 ms) at 48 kHz; wildly off
// values point at format mismatch.
// * peak_stereo — peak f32 absolute value of the
// AudioHandler output. Zero =
// handler is producing silence
// (jitter-buffer underrun or no
// remote audio). Non-zero =
// decoded content is reaching the
// callback.
// * peak_out — peak i16 absolute value of the
// downmixed mono signal we hand
// VPIO. Zero with non-zero
// peak_stereo would indicate the
// downmix is broken.
// * gain — current master output gain.
cb_count = cb_count.wrapping_add(1);
if cb_count.is_multiple_of(100) {
info!(
target: "chanora_audio",
cb = cb_count,
num_frames,
peak_stereo_f32 = peak_stereo_f32,
peak_out_i16,
gain,
"ios VPIO render callback diagnostic sample"
);
*dst = (clamped * i16::MAX as f32) as i16;
}
Ok(())
})