204 lines
8.1 KiB
Rust
204 lines
8.1 KiB
Rust
//! Linux output stream via SDL2 (Qint / upstream `tsclientlib`
|
|
//! `ts_to_audio` pattern).
|
|
//!
|
|
//! ## Why SDL2 and not cpal on Linux
|
|
//!
|
|
//! cpal's Linux backend opens raw ALSA's `default` PCM. On most
|
|
//! modern distributions (Arch with `pipewire-alsa`, Fedora 38+,
|
|
//! Debian/Ubuntu with PipeWire) that virtual device still goes
|
|
//! through ALSA's `dmix` + `plug` layers when bypassing the
|
|
//! PulseAudio/PipeWire client. The `plug` layer's default
|
|
//! resampler is **nearest-neighbour**, which causes pronounced
|
|
//! aliasing on the 48 kHz → 44.1 kHz step that the
|
|
//! `tsclientlib::AudioHandler` output requires. Users perceive
|
|
//! this as constant crackling and popping. cpal also picks a
|
|
//! small default period size (≈256 frames), which leaves no
|
|
//! headroom for kernel scheduler jitter and causes additional
|
|
//! xruns.
|
|
//!
|
|
//! SDL2 on the same systems opens the audio device through the
|
|
//! SDL audio driver — which prefers PulseAudio when available
|
|
//! and falls back to ALSA otherwise. On a PipeWire box the SDL
|
|
//! PulseAudio driver lands inside PipeWire's PulseAudio
|
|
//! compatibility layer, whose resampler is high quality. SDL
|
|
//! also defaults to a buffer ≈ samples-requested, so we get
|
|
//! exactly one Opus frame (20 ms) per callback.
|
|
//!
|
|
//! This file replaces the cpal output path on Linux only. The
|
|
//! capture path stays on cpal until we have a reason to swap it
|
|
//! (the user reports outbound is currently fine). Windows continues
|
|
//! to use cpal's WASAPI backend; Apple platforms use direct
|
|
//! VoiceProcessingIO AudioUnits for the voice path.
|
|
//!
|
|
//! ## Threading & lifecycle
|
|
//!
|
|
//! `sdl2::AudioDevice<CB>` is `!Send + !Sync` — the SDL audio
|
|
//! lock is associated with the calling thread. We open the device
|
|
//! on the same thread that calls `Self::start_with_gate` (the
|
|
//! tokio worker that runs `chanora_core::ChanoraSession::start_audio`)
|
|
//! and never move it. The outer `AudioEngine` already carries an
|
|
//! `unsafe impl Send` to bypass cpal's identical constraint; we
|
|
//! reuse that and stash the SDL device behind a `Mutex<Option<…>>`
|
|
//! the same way cpal does.
|
|
//!
|
|
//! Dropping the `SdlOutput` closes the SDL device cleanly and
|
|
//! drops the playback callback — that releases the
|
|
//! `Arc<Mutex<AudioHandler>>` clone the callback held.
|
|
|
|
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
|
use std::sync::{Arc, Mutex};
|
|
|
|
use sdl2::audio::{AudioCallback, AudioDevice, AudioSpecDesired};
|
|
use sdl2::AudioSubsystem;
|
|
use tracing::{info, warn};
|
|
use tsclientlib::audio::AudioHandler;
|
|
|
|
use crate::engine::SessionAudioId;
|
|
use crate::AudioError;
|
|
|
|
/// 20 ms at 48 kHz mono == one Opus frame's worth of samples.
|
|
/// Stereo doubles the byte count but the frame-count stays the
|
|
/// same. Aligning the callback size to the Opus frame keeps the
|
|
/// jitter-buffer / playback handshake tight (no fractional frame
|
|
/// reads inside fill_buffer).
|
|
const FRAME_SAMPLES: u16 = 960;
|
|
|
|
/// SDL output device wrapper. Holds the live `AudioDevice` so its
|
|
/// callback keeps firing for the engine's lifetime, plus a
|
|
/// reference to the same `AudioHandler` the inbound forwarder
|
|
/// pushes into.
|
|
pub struct SdlOutput {
|
|
// Drop order: device first (stops the callback), then any
|
|
// remaining references release naturally. We keep `_subsystem`
|
|
// alive because dropping the AudioSubsystem before the device
|
|
// would invalidate SDL's internal state.
|
|
device: AudioDevice<TsPlaybackCallback>,
|
|
_subsystem: AudioSubsystem,
|
|
}
|
|
|
|
impl SdlOutput {
|
|
/// Open the default SDL playback device at the AudioHandler's
|
|
/// native format (48 kHz stereo f32) and start the device
|
|
/// playing immediately. The callback drains the AudioHandler
|
|
/// directly — no user-side resampling.
|
|
///
|
|
/// `output_gain` is read on every callback to apply the
|
|
/// master-volume slider; `output_muted` zeroes the output (but
|
|
/// still drains AudioHandler so its jitter buffer doesn't grow
|
|
/// unbounded while muted). These two atomics share the same
|
|
/// definitions the cpal path uses, so the same FFI surface
|
|
/// (`set_output_gain` / `set_output_muted`) drives both.
|
|
pub fn start(
|
|
handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
|
|
output_gain: Arc<AtomicU32>,
|
|
output_muted: Arc<AtomicBool>,
|
|
) -> Result<Self, AudioError> {
|
|
let sdl = sdl2::init().map_err(|e| AudioError::Backend(format!("sdl init: {e}")))?;
|
|
let subsystem = sdl
|
|
.audio()
|
|
.map_err(|e| AudioError::Backend(format!("sdl audio subsystem: {e}")))?;
|
|
|
|
info!(
|
|
target: "chanora_audio",
|
|
driver = subsystem.current_audio_driver(),
|
|
"sdl audio subsystem initialised"
|
|
);
|
|
|
|
let desired = AudioSpecDesired {
|
|
freq: Some(48_000),
|
|
channels: Some(2),
|
|
samples: Some(FRAME_SAMPLES),
|
|
};
|
|
|
|
let device = AudioDevice::open_playback(&subsystem, None, &desired, |spec| {
|
|
info!(
|
|
target: "chanora_audio",
|
|
freq = spec.freq,
|
|
channels = spec.channels,
|
|
samples = spec.samples,
|
|
"sdl playback spec accepted"
|
|
);
|
|
TsPlaybackCallback {
|
|
handler,
|
|
output_gain,
|
|
output_muted,
|
|
}
|
|
})
|
|
.map_err(|e| AudioError::Backend(format!("sdl open_playback: {e}")))?;
|
|
|
|
// Begin pumping audio frames out. SDL's device starts paused;
|
|
// resume() flips it into the playing state. The callback
|
|
// will fire repeatedly at ~50 Hz (every 20 ms) thereafter.
|
|
device.resume();
|
|
|
|
Ok(Self {
|
|
device,
|
|
_subsystem: subsystem,
|
|
})
|
|
}
|
|
}
|
|
|
|
impl Drop for SdlOutput {
|
|
fn drop(&mut self) {
|
|
// AudioDevice::drop closes the device which stops the
|
|
// callback. We log so the chanora.log timeline matches
|
|
// engine shutdown.
|
|
warn!(target: "chanora_audio", "sdl playback device closing");
|
|
}
|
|
}
|
|
|
|
/// Playback callback invoked by SDL's audio thread. The shape
|
|
/// mirrors the upstream `tsclientlib` example's `SdlCallback`:
|
|
/// zero the output buffer (so silent regions emit silence rather
|
|
/// than stale memory), then ask the `AudioHandler` to fill in
|
|
/// whatever decoded frames it has buffered.
|
|
struct TsPlaybackCallback {
|
|
handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
|
|
output_gain: Arc<AtomicU32>,
|
|
output_muted: Arc<AtomicBool>,
|
|
}
|
|
|
|
impl AudioCallback for TsPlaybackCallback {
|
|
type Channel = f32;
|
|
|
|
fn callback(&mut self, buffer: &mut [f32]) {
|
|
// The buffer is interleaved stereo at 48 kHz from SDL.
|
|
// Length is FRAME_SAMPLES * 2 (= 1920 f32) per the spec
|
|
// we requested.
|
|
for sample in buffer.iter_mut() {
|
|
*sample = 0.0;
|
|
}
|
|
// Lock window kept as tight as the upstream example —
|
|
// fill_buffer does the actual Opus decode + jitter logic.
|
|
// Contention with the inbound forwarder is the same as
|
|
// in the cpal path, but the upstream design has shipped
|
|
// this way for years.
|
|
{
|
|
let mut data = self.handler.lock().unwrap();
|
|
let _removed_ids = data.fill_buffer(buffer);
|
|
// `_removed_ids` is the list of clients whose stream the
|
|
// handler just finished draining. We could publish that
|
|
// upward as a "stopped speaking" hint, but the existing
|
|
// BridgeEvent::VoiceState already covers that case via
|
|
// the bridge layer; ignoring matches upstream behaviour.
|
|
}
|
|
|
|
// Apply local mute + master gain after fill so the jitter
|
|
// buffer still drains while muted (matching the cpal path's
|
|
// contract). `output_gain` is encoded as f32 bits inside an
|
|
// AtomicU32 — same encoding the cpal path uses.
|
|
if self.output_muted.load(Ordering::Relaxed) {
|
|
for sample in buffer.iter_mut() {
|
|
*sample = 0.0;
|
|
}
|
|
return;
|
|
}
|
|
let gain = f32::from_bits(self.output_gain.load(Ordering::Relaxed));
|
|
if (gain - 1.0).abs() > f32::EPSILON {
|
|
for sample in buffer.iter_mut() {
|
|
*sample *= gain;
|
|
}
|
|
}
|
|
}
|
|
}
|