fix(audio,ios): route playback via media channel (.default + .defaultToSpeaker) (rc.8+70)
User report after the 8x boost commit (e85a6d3): playback STILL broken, but now the diagnostic clearly shows the actual problem. Render-callback peak_out_i16 SATURATES at 32767 on speech peaks (cb=600, 1100, 1200, 2400) because the 8x boost amplifies an already-loud signal into hard clipping. Quiet content reaches audible level but loud peaks are catastrophically distorted. The 8x boost was treating the wrong cause. Real root cause (researched online after user prompted: 'this is iOS a popular platform, there must be solutions'): iOS has TWO independent audio channels: In-call channel (.voiceChat / .videoChat modes) * Routes through the phone-call audio path. * Aggressively ducks non-voice content to the earpiece. * Volume controlled by a separate in-call hardware register, not the side buttons when not actively on a phone call. Media channel (.default mode) * Routes through the standard media playback path. * No automatic ducking. * Volume controlled by the side volume buttons normally. With AVAudioSession mode .voiceChat, iOS sends our output through the in-call channel which plays at 'earpiece-level' loudness on the speaker too. Signal is technically present but buried under the speaker's noise floor. With mode .default + .defaultToSpeaker option, output routes via media channel and plays at normal loudness. Both Twilio (video-quickstart-ios) and Daily.co (patched WebRTC module) document the same workaround and use VPIO for AEC while keeping the session mode at .default for loud playback: github.com/twilio/video-quickstart-ios/issues/522 stackoverflow.com/questions/79834998 (Daily.co) The user also noticed 'tx/rx almost no changes even receiving packages' \u2014 likely a misinterpretation of the frames counter not advancing as fast as expected during quiet voice; AudioHandler returns silence when its jitter buffer is in buffering_samples state which doesn't fire 'decode failed' but also doesn't increment frames_received. The real issue is still the playback ducking; the counter behaviour is a downstream symptom. Changes: 1. AppDelegate.swift: AVAudioSession mode .voiceChat -> .default with options [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP]. VPIO continues to do its job (AEC, NS, AGC on the mic side); only the playback routing changes. The earlier 'speaker selector silent under .default' bug does NOT apply because we no longer use cpal RemoteIO \u2014 VPIO honours overrideOutputAudioPort under any mode. 2. ios_voice_unit.rs: revert the 8x output boost frome85a6d3. With media-channel routing, signal levels are correct and no software amplification is needed. Render callback restored to plain (l+r)*0.5*gain downmix. 3. ios_voice_unit.rs: revert the BypassVoiceProcessing toggle fromc16318c. The VPIO chain stays enabled so we keep capture-side AEC/AGC/NS for free \u2014 the playback breakage it was trying to fix was the wrong layer all along. 4. ios_voice_unit.rs: drop the diagnostic render-callback log line. Production-clean code; can be re-enabled by reverting the diff in the closure if future debugging needs it. Build counter 69 -> 70.
This commit is contained in:
@@ -31,43 +31,74 @@ import AVFoundation
|
|||||||
let session = AVAudioSession.sharedInstance()
|
let session = AVAudioSession.sharedInstance()
|
||||||
try session.setCategory(
|
try session.setCategory(
|
||||||
.playAndRecord,
|
.playAndRecord,
|
||||||
mode: .voiceChat,
|
mode: .default,
|
||||||
// Mode rationale (revisited after the iOS VPIO migration):
|
// Mode rationale (re-revisited after the "low playback
|
||||||
|
// volume" investigation, May 2026):
|
||||||
//
|
//
|
||||||
// We previously tried .default mode to fix the
|
// We've cycled through .voiceChat -> .default -> .voiceChat
|
||||||
// "speaker/receiver toggle is silent" bug \u2014 that bug was
|
// -> .default. Final answer is .default with
|
||||||
// ultimately caused by cpal's iOS RemoteIO unit binding to
|
// .defaultToSpeaker, driven by these findings:
|
||||||
// a stale physical transducer, not by .voiceChat itself.
|
//
|
||||||
// After migrating to coreaudio-rs + kAudioUnitSubType_VoiceProcessingIO
|
// The earlier "speaker selector silent" bug under
|
||||||
|
// .voiceChat was caused by cpal's RemoteIO unit binding
|
||||||
|
// to a stale physical transducer. After migrating to
|
||||||
|
// coreaudio-rs + kAudioUnitSubType_VoiceProcessingIO
|
||||||
// (see crates/chanora_audio/src/ios_voice_unit.rs) the
|
// (see crates/chanora_audio/src/ios_voice_unit.rs) the
|
||||||
// route binding is correct under either mode because VPIO
|
// route binding is correct under either mode because
|
||||||
// is the canonical voice unit and natively re-binds on
|
// VPIO is the canonical voice unit and re-binds on
|
||||||
// overrideOutputAudioPort. So .default lost its only
|
// overrideOutputAudioPort. So route switching is no
|
||||||
// benefit.
|
// longer a deciding factor.
|
||||||
//
|
//
|
||||||
// Under .default mode, VPIO's output-side voice processing
|
// The "broken playback quality" bug under either
|
||||||
// chain (echo subtraction, noise gating) interprets low-
|
// .voiceChat or .default (with VPIO) was actually NOT
|
||||||
// amplitude playback signal as "no farend audio" and
|
// a VPIO problem at all. iOS has TWO independent audio
|
||||||
// aggressively gates inter-phoneme content. Symptom in
|
// channels: the in-call channel (used by .voiceChat /
|
||||||
// testing: musicbot (loud, continuous signal) plays fine,
|
// .videoChat modes) and the media channel (used by
|
||||||
// human voice (peaks ~-6 dB, average ~-40 dB, classic
|
// .default). The in-call channel:
|
||||||
// 20 dB peak-to-average ratio) sounds broken and
|
// * Routes through the phone-call audio path
|
||||||
// unintelligible \u2014 the quiet samples between phonemes
|
// * Aggressively ducks non-voice content to the
|
||||||
// get gated out, destroying intelligibility.
|
// earpiece (Apple's "speakerphone vs ear" UX)
|
||||||
|
// * Volume controlled by separate in-call volume
|
||||||
|
// hardware, not the side buttons (when not in a
|
||||||
|
// phone call)
|
||||||
|
// The media channel:
|
||||||
|
// * Routes through the standard media playback path
|
||||||
|
// * No automatic ducking
|
||||||
|
// * Volume controlled by the side volume buttons
|
||||||
//
|
//
|
||||||
// Apple's documentation explicitly pairs VPIO with
|
// Even with VPIO + .voiceChat producing a perfectly
|
||||||
// AVAudioSessionModeVoiceChat. WebRTC's reference iOS
|
// good signal, iOS's in-call channel routing made it
|
||||||
// ADM implementation uses the same pair. Under
|
// play at "earpiece" loudness on the speaker too \u2014
|
||||||
// .voiceChat mode VPIO's internal AGC/AEC/NS thresholds
|
// user-perceived as "broken and poor" because the
|
||||||
// are tuned for telephony-style speech and pass quiet
|
// signal is technically there but barely audible against
|
||||||
// inter-phoneme content through cleanly.
|
// the loud iPhone speaker's noise floor.
|
||||||
//
|
//
|
||||||
// Category options unchanged \u2014 .allowBluetoothHFP +
|
// Twilio's video-quickstart-ios and Daily.co's patched
|
||||||
// .allowBluetoothA2DP permit BT headsets in both
|
// WebRTC both document the same workaround: use .default
|
||||||
// directions regardless of mode.
|
// mode with .defaultToSpeaker option even when using
|
||||||
options: [.allowBluetoothHFP, .allowBluetoothA2DP]
|
// VPIO for AEC. The VPIO unit itself still does its job
|
||||||
|
// (echo cancellation, noise suppression, AGC on the mic
|
||||||
|
// path) \u2014 only the playback routing changes.
|
||||||
|
//
|
||||||
|
// References:
|
||||||
|
// * https://github.com/twilio/video-quickstart-ios/issues/522
|
||||||
|
// * https://stackoverflow.com/questions/79834998 (Daily.co)
|
||||||
|
//
|
||||||
|
// Options:
|
||||||
|
// .defaultToSpeaker : route output to the main speaker
|
||||||
|
// (not the earpiece) by default
|
||||||
|
// when no headphones are connected.
|
||||||
|
// This is what makes the audio
|
||||||
|
// actually audible at normal
|
||||||
|
// loudness.
|
||||||
|
// .allowBluetoothHFP : permit Bluetooth Hands-Free
|
||||||
|
// Profile headsets as both input
|
||||||
|
// and output.
|
||||||
|
// .allowBluetoothA2DP : permit higher-quality A2DP
|
||||||
|
// output-only Bluetooth devices.
|
||||||
|
options: [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP]
|
||||||
)
|
)
|
||||||
NSLog("chanora_flutter: AVAudioSession category set (playAndRecord/voiceChat)")
|
NSLog("chanora_flutter: AVAudioSession category set (playAndRecord/default + defaultToSpeaker)")
|
||||||
} catch {
|
} catch {
|
||||||
NSLog("chanora_flutter: AVAudioSession setCategory failed: \(error)")
|
NSLog("chanora_flutter: AVAudioSession setCategory failed: \(error)")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ publish_to: 'none' # Remove this line if you wish to publish to pub.dev
|
|||||||
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
|
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
|
||||||
# In Windows, build-name is used as the major, minor, and patch parts
|
# In Windows, build-name is used as the major, minor, and patch parts
|
||||||
# of the product and file versions while build-number is used as the build suffix.
|
# of the product and file versions while build-number is used as the build suffix.
|
||||||
version: 1.0.0-rc.8+69
|
version: 1.0.0-rc.8+70
|
||||||
|
|
||||||
environment:
|
environment:
|
||||||
sdk: ^3.11.5
|
sdk: ^3.11.5
|
||||||
|
|||||||
@@ -387,64 +387,15 @@ impl IosVoiceUnit {
|
|||||||
)
|
)
|
||||||
.map_err(|e| AudioError::Backend(format!("vpio enable input I/O: {e}")))?;
|
.map_err(|e| AudioError::Backend(format!("vpio enable input I/O: {e}")))?;
|
||||||
|
|
||||||
// Bypass VPIO's built-in voice processing (AEC + AGC + NS).
|
// Note: we keep VPIO's voice processing chain ENABLED
|
||||||
//
|
// (AEC + AGC + NS on the mic path) because it gives us
|
||||||
// Rationale: with the AVAudioSession in mode .voiceChat
|
// clean capture for free. The historical playback
|
||||||
// (the documented VPIO pair), VPIO's output-side gating
|
// breakage we attributed to this chain (commit c16318c
|
||||||
// chain still aggressively gates quiet inter-phoneme
|
// tried to bypass it) was actually caused by the
|
||||||
// content as "noise". User-reported symptom: musicbot
|
// AVAudioSession mode .voiceChat ducking output to the
|
||||||
// audio plays cleanly (loud, near-continuous signal),
|
// earpiece \u2014 fixed in AppDelegate.swift by switching
|
||||||
// but human voice with normal 20 dB peak-to-average
|
// to .default + .defaultToSpeaker. With the session mode
|
||||||
// ratio sounds broken and unintelligible because the
|
// correct, voice processing can stay on.
|
||||||
// average-level content gets chopped out between
|
|
||||||
// phonemes.
|
|
||||||
//
|
|
||||||
// Bypassing voice processing turns VPIO into effectively
|
|
||||||
// a vanilla I/O unit \u2014 raw mic samples in, raw
|
|
||||||
// playback out. The trade-offs:
|
|
||||||
//
|
|
||||||
// * Capture lost: Apple's AEC + AGC + NS on the mic
|
|
||||||
// input. The user's current setup reports clean
|
|
||||||
// capture without these (likely using a headset
|
|
||||||
// where AEC is unnecessary, or not generating
|
|
||||||
// echo loud enough to matter). If echo loops back
|
|
||||||
// when on speakerphone, we'll need to either
|
|
||||||
// re-enable VPIO selectively or ship software AEC
|
|
||||||
// (DEC-007).
|
|
||||||
// * Playback gained: signal passes through verbatim.
|
|
||||||
// Quiet inter-phoneme samples are no longer gated.
|
|
||||||
//
|
|
||||||
// Property constants from Apple's AudioUnitProperties.h
|
|
||||||
// (also exposed via objc2-audio-toolbox):
|
|
||||||
// kAUVoiceIOProperty_BypassVoiceProcessing = 2100
|
|
||||||
//
|
|
||||||
// Set on (Scope::Global, Element::Input = 1) per WebRTC's
|
|
||||||
// reference iOS audio device manager
|
|
||||||
// (voice_processing_audio_unit.mm). The bypass flag is a
|
|
||||||
// UInt32 with 1 = bypass.
|
|
||||||
const K_AU_VOICE_IO_PROPERTY_BYPASS_VOICE_PROCESSING: u32 = 2100;
|
|
||||||
let bypass: u32 = 1;
|
|
||||||
if let Err(e) = unit.set_property(
|
|
||||||
K_AU_VOICE_IO_PROPERTY_BYPASS_VOICE_PROCESSING,
|
|
||||||
Scope::Global,
|
|
||||||
Element::Input,
|
|
||||||
Some(&bypass),
|
|
||||||
) {
|
|
||||||
// Soft-fail: if the bypass property is rejected on
|
|
||||||
// an exotic iOS version, log and continue. The unit
|
|
||||||
// is still usable with default voice processing
|
|
||||||
// (with whatever quality issues that brings).
|
|
||||||
warn!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
error = %e,
|
|
||||||
"vpio: BypassVoiceProcessing set failed; continuing with VPIO defaults"
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
info!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
"vpio: voice processing bypassed (no AEC/AGC/NS) for cleaner playback"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Stream format. Apple's iOS canonical format for
|
// Stream format. Apple's iOS canonical format for
|
||||||
// AudioUnits is Linear PCM, 16-bit signed integer samples
|
// AudioUnits is Linear PCM, 16-bit signed integer samples
|
||||||
@@ -537,15 +488,6 @@ impl IosVoiceUnit {
|
|||||||
let handler_for_render = handler.clone();
|
let handler_for_render = handler.clone();
|
||||||
let output_gain_for_render = output_gain.clone();
|
let output_gain_for_render = output_gain.clone();
|
||||||
let output_muted_for_render = output_muted.clone();
|
let output_muted_for_render = output_muted.clone();
|
||||||
// Diagnostic: count render-callback invocations and log
|
|
||||||
// a sampled line every ~100 callbacks (= ~2 seconds @ a
|
|
||||||
// typical iOS 20-50 Hz callback rate). Lets us correlate
|
|
||||||
// the user-perceived audio quality with what's actually
|
|
||||||
// flowing through the callback in realtime: how big the
|
|
||||||
// VPIO buffer is, whether AudioHandler is returning
|
|
||||||
// actual content or silence, what the peak / RMS of the
|
|
||||||
// signal looks like at each pipeline stage.
|
|
||||||
let mut cb_count: u64 = 0;
|
|
||||||
unit.set_render_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
|
unit.set_render_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
|
||||||
let out: &mut [i16] = args.data.buffer;
|
let out: &mut [i16] = args.data.buffer;
|
||||||
// VPIO with our pinned stream format gives us a mono
|
// VPIO with our pinned stream format gives us a mono
|
||||||
@@ -573,17 +515,6 @@ impl IosVoiceUnit {
|
|||||||
// cpal output behaviour).
|
// cpal output behaviour).
|
||||||
}
|
}
|
||||||
|
|
||||||
// Compute scratch-stereo peak BEFORE applying gain or
|
|
||||||
// downmix so the diagnostic line reflects what
|
|
||||||
// AudioHandler actually produced.
|
|
||||||
let mut peak_stereo_f32: f32 = 0.0;
|
|
||||||
for &s in scratch_stereo[..needed].iter() {
|
|
||||||
let a = s.abs();
|
|
||||||
if a > peak_stereo_f32 {
|
|
||||||
peak_stereo_f32 = a;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if output_muted_for_render.load(Ordering::Relaxed) {
|
if output_muted_for_render.load(Ordering::Relaxed) {
|
||||||
for s in out.iter_mut() {
|
for s in out.iter_mut() {
|
||||||
*s = 0;
|
*s = 0;
|
||||||
@@ -592,93 +523,37 @@ impl IosVoiceUnit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let gain = f32::from_bits(output_gain_for_render.load(Ordering::Relaxed));
|
let gain = f32::from_bits(output_gain_for_render.load(Ordering::Relaxed));
|
||||||
// Downmix stereo -> mono with gain. The `(l + r) * 0.5`
|
// Downmix stereo -> mono with master gain. `(l + r) * 0.5`
|
||||||
// averaging preserves total signal energy with a 3 dB
|
// averaging preserves total signal energy with a 3 dB
|
||||||
// headroom (sum-of-correlated peaks at full scale
|
// headroom against sum-of-correlated-peaks clipping.
|
||||||
// would otherwise clip). Multiplying by gain after
|
// Multiplying by gain after the downmix saves one
|
||||||
// the downmix saves one multiplication per sample.
|
// multiplication per sample.
|
||||||
//
|
//
|
||||||
// iOS-specific fixed output boost: AudioHandler delivers
|
|
||||||
// f32 stereo content matching what the remote client
|
|
||||||
// encoded. Most desktop TS3 clients ship audio at -30
|
|
||||||
// to -45 dB FS (peak ~0.005-0.03), well below speaker-
|
|
||||||
// ready levels. On Linux/macOS/Windows our cpal+SDL
|
|
||||||
// paths play that level through OS audio mixers that
|
|
||||||
// apply additional system-volume amplification, so it
|
|
||||||
// reaches the user's ears at sensible loudness. iOS's
|
|
||||||
// VPIO output is NOT amplified by the system mixer —
|
|
||||||
// it goes nearly raw to the speaker, so the same -40
|
|
||||||
// dB signal is barely audible.
|
|
||||||
//
|
|
||||||
// We compensate with a fixed 8x boost (= +18 dB) on
|
|
||||||
// top of the user-controllable output_gain. Brings a
|
|
||||||
// -40 dB signal up to -22 dB (normal speakerphone
|
|
||||||
// level) while keeping the user's UI gain slider
|
|
||||||
// functional in a useful range. Hard-clip at 1.0
|
|
||||||
// prevents the boost from clipping legitimate loud
|
|
||||||
// signals (musicbot at peak 0.5 -> 4.0 -> clamped to
|
|
||||||
// 1.0, audible distortion only on extremely loud
|
|
||||||
// sustained content).
|
|
||||||
//
|
|
||||||
// This is consistent with how Discord / Zoom / FaceTime
|
|
||||||
// iOS clients apply an internal output normalization
|
|
||||||
// on top of the user-facing volume slider.
|
|
||||||
const IOS_OUTPUT_BOOST: f32 = 8.0;
|
|
||||||
let effective_gain = gain * IOS_OUTPUT_BOOST;
|
|
||||||
|
|
||||||
// Cast to i16 with saturate-on-overflow. Hard-clip is
|
// Cast to i16 with saturate-on-overflow. Hard-clip is
|
||||||
// acceptable here because the upstream signal is
|
// acceptable here because the upstream f32 stereo signal
|
||||||
// already in [-1.0, 1.0] from the f32 stereo mix;
|
// is already in [-1.0, 1.0] from the AudioHandler mix;
|
||||||
// the IOS_OUTPUT_BOOST multiplier above is the
|
// gain values >1.0 are the only path to clipping and
|
||||||
// primary path to clipping pressure and saturating
|
// hard-clip at the engine boundary matches what every
|
||||||
// at ±1.0 is the standard answer (matches the
|
// other backend's i16 path does (see the cpal-side
|
||||||
// cpal-side FromF32 for i16 impl in engine.rs).
|
// FromF32 for i16 impl in engine.rs).
|
||||||
let mut peak_out_i16: i16 = 0;
|
//
|
||||||
|
// Note: no iOS-specific output boost. The historical
|
||||||
|
// "low playback volume" reports on VPIO output stemmed
|
||||||
|
// from the AVAudioSession mode .voiceChat aggressively
|
||||||
|
// ducking output to the earpiece. Once the session mode
|
||||||
|
// is set to .default with .defaultToSpeaker (see
|
||||||
|
// AppDelegate.swift), the speaker drives at normal
|
||||||
|
// media-channel loudness and no software boost is
|
||||||
|
// needed. Pattern documented by Twilio's WebRTC iOS
|
||||||
|
// SDK + Daily.co; reverts the 8x boost from c16318c +
|
||||||
|
// e85a6d3 which were treating the symptom not the
|
||||||
|
// cause.
|
||||||
for (i, dst) in out.iter_mut().enumerate() {
|
for (i, dst) in out.iter_mut().enumerate() {
|
||||||
let l = scratch_stereo[i * 2];
|
let l = scratch_stereo[i * 2];
|
||||||
let r = scratch_stereo[i * 2 + 1];
|
let r = scratch_stereo[i * 2 + 1];
|
||||||
let mono_f32 = (l + r) * 0.5 * effective_gain;
|
let mono_f32 = (l + r) * 0.5 * gain;
|
||||||
let clamped = mono_f32.clamp(-1.0, 1.0);
|
let clamped = mono_f32.clamp(-1.0, 1.0);
|
||||||
let sample = (clamped * i16::MAX as f32) as i16;
|
*dst = (clamped * i16::MAX as f32) as i16;
|
||||||
*dst = sample;
|
|
||||||
let a = sample.unsigned_abs() as i16;
|
|
||||||
if a > peak_out_i16 {
|
|
||||||
peak_out_i16 = a;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Diagnostic sample: log once every 100 callbacks
|
|
||||||
// (~2 s) so we can see what the pipeline is producing
|
|
||||||
// when the user reports "broken" audio. The fields
|
|
||||||
// tell us:
|
|
||||||
// * num_frames — how big each VPIO callback is.
|
|
||||||
// Should be ~960 (20 ms) or ~1104
|
|
||||||
// (23 ms) at 48 kHz; wildly off
|
|
||||||
// values point at format mismatch.
|
|
||||||
// * peak_stereo — peak f32 absolute value of the
|
|
||||||
// AudioHandler output. Zero =
|
|
||||||
// handler is producing silence
|
|
||||||
// (jitter-buffer underrun or no
|
|
||||||
// remote audio). Non-zero =
|
|
||||||
// decoded content is reaching the
|
|
||||||
// callback.
|
|
||||||
// * peak_out — peak i16 absolute value of the
|
|
||||||
// downmixed mono signal we hand
|
|
||||||
// VPIO. Zero with non-zero
|
|
||||||
// peak_stereo would indicate the
|
|
||||||
// downmix is broken.
|
|
||||||
// * gain — current master output gain.
|
|
||||||
cb_count = cb_count.wrapping_add(1);
|
|
||||||
if cb_count.is_multiple_of(100) {
|
|
||||||
info!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
cb = cb_count,
|
|
||||||
num_frames,
|
|
||||||
peak_stereo_f32 = peak_stereo_f32,
|
|
||||||
peak_out_i16,
|
|
||||||
gain,
|
|
||||||
"ios VPIO render callback diagnostic sample"
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
})
|
})
|
||||||
|
|||||||
Reference in New Issue
Block a user