feat(poc/audio): add audio capture/playback spike (partial — desktop only)

Proof-of-concept addressing the audio exit criterion from
docs/architecture/proof-of-concept-plan.md §2:
  "Capture/playback works on at least one desktop and one mobile
   target."

PARTIAL PASS. The desktop half is verified on Linux; the mobile
half is NOT verified by this PoC and remains a documented open gap.

Implements via cpal (matching DEC-011 'platform-native first'):
  - AudioCapture::record_to_wav opens the default input device,
    handles f32/i16/u16 sample formats, down-mixes to mono, writes
    16-bit PCM WAV via hound.
  - AudioPlayback::play_wav opens the default output device, picks
    a stream config matching the WAV, blocks until drained.
  - synth_sine_wav produces a deterministic 440 Hz test signal for
    headless verification of the playback path when no microphone
    is available.
  - Typed AudioError DTO with NoInputDevice, NoOutputDevice,
    DefaultConfig, BuildStream, PlayStream, Wav, Io,
    UnsupportedFormat arms.

Verified on 2026-05-13 (Linux + cpal + PipeWire). Capture stream
opened against the system default input; build failed against the
auto_null source (typed AudioError::BuildStream returned cleanly,
demonstrating the production error path); fallback to synth fired;
playback drove 24,000 frames to completion through
Rust → cpal → ALSA → pcm_pipewire → PipeWire → auto_null.
Both audio.rs tests pass.

Mobile gap (explicit, NOT closed):
  - Android Oboe path not built or run.
  - iOS AVAudioEngine path not built or run.

Surfaced finding for the decision register: DEC-011 does not pin an
audio crate. The PoC uses cpal; production code needs an owner
ruling, ideally after the mobile spike closes the gap.

Out of scope: DSP (HPF/NS/AEC/AGC), Opus encode/decode, jitter
buffer, mixer, latency measurement, bit-exact loopback, device
permission flows. These belong to chanora_audio.

Authority: PoC plan §2, DEC-011, SysDes audio subsystem.
Not product code; not promoted into chanora_audio.
This commit is contained in:
EdisonJwa
2026-05-14 12:27:04 +08:00
parent 06ec6f2965
commit d5b53996bc
10 changed files with 2125 additions and 0 deletions
@@ -0,0 +1,179 @@
//! Audio capture: open the default input device, record N seconds of
//! 16-bit mono PCM, write to a WAV file.
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use cpal::{Sample, SampleFormat};
use hound::{SampleFormat as HoundSampleFormat, WavSpec, WavWriter};
use thiserror::Error;
use tracing::{info, warn};
#[derive(Debug, Error)]
pub enum AudioError {
#[error("no input device available")]
NoInputDevice,
#[error("no output device available")]
NoOutputDevice,
#[error("device default config error: {0}")]
DefaultConfig(#[from] cpal::DefaultStreamConfigError),
#[error("device supported config error: {0}")]
SupportedConfig(#[from] cpal::SupportedStreamConfigsError),
#[error("stream build error: {0}")]
BuildStream(#[from] cpal::BuildStreamError),
#[error("stream play error: {0}")]
PlayStream(#[from] cpal::PlayStreamError),
#[error("wav io: {0}")]
Wav(#[from] hound::Error),
#[error("io: {0}")]
Io(#[from] std::io::Error),
#[error("unsupported sample format: {0:?}")]
UnsupportedFormat(SampleFormat),
}
pub struct AudioCapture;
impl AudioCapture {
/// Capture `duration` from the default input device into `path` as
/// 16-bit mono PCM WAV. Returns the number of frames written.
pub fn record_to_wav(path: &Path, duration: Duration) -> Result<u64, AudioError> {
let host = cpal::default_host();
let device = host.default_input_device().ok_or(AudioError::NoInputDevice)?;
let device_name = device.name().unwrap_or_else(|_| "<unknown>".to_string());
let config = device.default_input_config()?;
let sample_rate = config.sample_rate().0;
let channels = config.channels();
let sample_format = config.sample_format();
info!(
target: "spike",
device = %device_name,
sample_rate, channels, ?sample_format,
"opening capture stream"
);
let spec = WavSpec {
channels: 1, // we down-mix to mono
sample_rate,
bits_per_sample: 16,
sample_format: HoundSampleFormat::Int,
};
let writer = Arc::new(Mutex::new(Some(WavWriter::create(path, spec)?)));
let frames = Arc::new(Mutex::new(0u64));
let err_fn = |e| warn!(target: "spike", error = %e, "capture stream error");
let stream = match sample_format {
SampleFormat::F32 => build_input_stream::<f32>(
&device,
&config.into(),
writer.clone(),
frames.clone(),
channels,
err_fn,
)?,
SampleFormat::I16 => build_input_stream::<i16>(
&device,
&config.into(),
writer.clone(),
frames.clone(),
channels,
err_fn,
)?,
SampleFormat::U16 => build_input_stream::<u16>(
&device,
&config.into(),
writer.clone(),
frames.clone(),
channels,
err_fn,
)?,
other => return Err(AudioError::UnsupportedFormat(other)),
};
stream.play()?;
std::thread::sleep(duration);
drop(stream); // stops capture
// Finalize the WAV.
if let Some(w) = writer.lock().unwrap().take() {
w.finalize()?;
}
let n = *frames.lock().unwrap();
info!(target: "spike", frames = n, "capture finished");
Ok(n)
}
}
fn build_input_stream<T>(
device: &cpal::Device,
config: &cpal::StreamConfig,
writer: Arc<Mutex<Option<WavWriter<std::io::BufWriter<std::fs::File>>>>>,
frames: Arc<Mutex<u64>>,
channels: u16,
err_fn: fn(cpal::StreamError),
) -> Result<cpal::Stream, AudioError>
where
T: SizedSample + ToI16,
{
let stream = device.build_input_stream(
config,
move |data: &[T], _| {
let mut w_guard = writer.lock().unwrap();
if let Some(w) = w_guard.as_mut() {
let mut n = 0u64;
for frame in data.chunks(channels as usize) {
// Down-mix to mono by averaging channels.
let mut acc: i32 = 0;
for s in frame {
acc += s.to_i16() as i32;
}
let mono = (acc / frame.len() as i32) as i16;
let _ = w.write_sample(mono);
n += 1;
}
*frames.lock().unwrap() += n;
}
},
err_fn,
None,
)?;
Ok(stream)
}
/// `cpal::SizedSample` is the public marker. We expose it so the
/// generic stream builder can constrain `T` without leaking cpal types
/// out of this module.
pub trait SizedSample: cpal::SizedSample + Send + 'static {}
impl<T: cpal::SizedSample + Send + 'static> SizedSample for T {}
/// Adapter so we can convert any supported sample format down to i16
/// for WAV writing.
pub trait ToI16 {
fn to_i16(&self) -> i16;
}
impl ToI16 for i16 {
fn to_i16(&self) -> i16 {
*self
}
}
impl ToI16 for u16 {
fn to_i16(&self) -> i16 {
(i32::from(*self) - i32::from(i16::MAX) - 1) as i16
}
}
impl ToI16 for f32 {
fn to_i16(&self) -> i16 {
let v = (*self * f32::from(i16::MAX)).clamp(f32::from(i16::MIN), f32::from(i16::MAX));
v as i16
}
}
// `Sample` trait import is required for cpal's older method shapes; we
// keep an explicit no-op use so this stays consistent across versions.
#[allow(dead_code)]
fn _assert_sample<T: Sample>() {}
@@ -0,0 +1,36 @@
//! Chanora PoC — audio capture / playback spike.
//!
//! Authority:
//! * `docs/architecture/proof-of-concept-plan.md` §2 — Audio
//! capture/playback spike. Exit criterion: "Capture/playback works
//! on at least one desktop and one mobile target."
//! * DEC-011 — platform-native first; Rust/WebRTC-style as fallback.
//! * SysDes audio subsystem.
//!
//! Honest scope:
//! * Linux desktop is verified here via `cpal` against the system
//! audio stack (PipeWire / Pulse / ALSA, whichever is active).
//! * **Mobile targets are NOT verified by this PoC.** That is a
//! documented gap, not a passed criterion.
//!
//! What the PoC shows:
//! * `AudioCapture` — opens the default input device and writes
//! 16-bit PCM samples to a WAV file via `hound`.
//! * `AudioPlayback` — opens the default output device and plays
//! back a WAV file end-to-end.
//! * `synth_sine_wav` — when no input device is available
//! (genuinely headless box), the PoC can still exercise the
//! playback path against a synthesised waveform. This is what
//! keeps the spike useful in CI.
//!
//! Production code (`chanora_audio`) will own the real DSP chain
//! (HPF, NS, AEC, AGC, Opus encode/decode, jitter buffer, mixer).
//! None of that lives here.
pub mod capture;
pub mod playback;
pub mod synth;
pub use capture::{AudioCapture, AudioError};
pub use playback::AudioPlayback;
pub use synth::synth_sine_wav;
@@ -0,0 +1,89 @@
//! CLI driver — capture or playback or both.
use std::path::PathBuf;
use std::process::ExitCode;
use std::time::Duration;
use audio_capture_playback_spike::{synth_sine_wav, AudioCapture, AudioPlayback};
use tracing::{info, warn};
fn main() -> ExitCode {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
)
.init();
let mode = std::env::args().nth(1).unwrap_or_else(|| "roundtrip".into());
let tmp = std::env::temp_dir();
let cap_path = tmp.join("chanora_audio_spike_capture.wav");
let play_path = tmp.join("chanora_audio_spike_synth.wav");
let result: anyhow::Result<()> = match mode.as_str() {
"capture" => do_capture(&cap_path),
"synth" => do_synth(&play_path),
"playback" => do_playback(&play_path),
"roundtrip" => do_roundtrip(&cap_path, &play_path),
other => {
eprintln!("unknown mode: {other}. Use one of: capture, synth, playback, roundtrip.");
return ExitCode::from(2);
}
};
if let Err(e) = result {
eprintln!("error: {e:#}");
return ExitCode::from(1);
}
ExitCode::SUCCESS
}
fn do_capture(path: &PathBuf) -> anyhow::Result<()> {
info!(target: "spike", "capture → {}", path.display());
let frames = AudioCapture::record_to_wav(path, Duration::from_secs(1))?;
println!("captured {} frames to {}", frames, path.display());
Ok(())
}
fn do_synth(path: &PathBuf) -> anyhow::Result<()> {
info!(target: "spike", "synth → {}", path.display());
let (frames, sr) = synth_sine_wav(path, 440.0, Duration::from_millis(500), 48_000)?;
println!("synthesised {} frames @ {} Hz to {}", frames, sr, path.display());
Ok(())
}
fn do_playback(path: &PathBuf) -> anyhow::Result<()> {
info!(target: "spike", "playback ← {}", path.display());
AudioPlayback::play_wav(path)?;
println!("playback finished");
Ok(())
}
fn do_roundtrip(cap_path: &PathBuf, play_path: &PathBuf) -> anyhow::Result<()> {
// Try real capture first; if no input device or the capture failed,
// fall back to synthesised audio so the playback path is still
// exercised. This mirrors what production code would do during
// a "device test" UI on headless or permission-denied hosts.
let to_play = match AudioCapture::record_to_wav(cap_path, Duration::from_millis(750)) {
Ok(n) if n > 0 => {
info!(target: "spike", "captured {} frames; playing back", n);
cap_path.clone()
}
Ok(_) => {
warn!(target: "spike", "capture wrote 0 frames; falling back to synth");
let (n, sr) = synth_sine_wav(play_path, 440.0, Duration::from_millis(500), 48_000)?;
info!(target: "spike", frames = n, sample_rate = sr, "synthesised fallback");
play_path.clone()
}
Err(e) => {
warn!(target: "spike", error = %e, "capture unavailable; falling back to synth");
let (n, sr) = synth_sine_wav(play_path, 440.0, Duration::from_millis(500), 48_000)?;
info!(target: "spike", frames = n, sample_rate = sr, "synthesised fallback");
play_path.clone()
}
};
AudioPlayback::play_wav(&to_play)?;
println!("round-trip complete (played from {})", to_play.display());
Ok(())
}
@@ -0,0 +1,153 @@
//! Audio playback: open the default output device, stream a WAV file
//! to it until the file is consumed.
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use cpal::{SampleFormat, SizedSample};
use hound::WavReader;
use tracing::{info, warn};
use crate::capture::AudioError;
pub struct AudioPlayback;
impl AudioPlayback {
/// Play a 16-bit-PCM WAV file through the default output device.
/// Blocks until the file has been fully streamed.
pub fn play_wav(path: &Path) -> Result<(), AudioError> {
let host = cpal::default_host();
let device = host.default_output_device().ok_or(AudioError::NoOutputDevice)?;
let device_name = device.name().unwrap_or_else(|_| "<unknown>".to_string());
// Read all WAV samples up-front (PoC; production code would
// stream in chunks). Resample is not done here — we pick a
// device config that matches the file's sample rate when
// possible.
let mut reader = WavReader::open(path)?;
let wav_spec = reader.spec();
let samples: Vec<i16> = reader.samples::<i16>().collect::<Result<_, _>>()?;
info!(
target: "spike",
device = %device_name,
file_sr = wav_spec.sample_rate,
file_channels = wav_spec.channels,
samples = samples.len(),
"preparing playback"
);
// Pick a supported config that matches the file's channel count
// and sample rate where possible.
let default_cfg = device.default_output_config()?;
let config = cpal::StreamConfig {
channels: default_cfg.channels(),
sample_rate: cpal::SampleRate(wav_spec.sample_rate),
buffer_size: cpal::BufferSize::Default,
};
let cursor = Arc::new(Mutex::new(0usize));
let total = samples.len();
let samples_arc = Arc::new(samples);
let out_channels = config.channels as usize;
let done = Arc::new(Mutex::new(false));
let err_fn = |e| warn!(target: "spike", error = %e, "playback stream error");
let stream = match default_cfg.sample_format() {
SampleFormat::F32 => build_output_stream::<f32>(
&device, &config, samples_arc.clone(), cursor.clone(), out_channels, done.clone(), err_fn,
)?,
SampleFormat::I16 => build_output_stream::<i16>(
&device, &config, samples_arc.clone(), cursor.clone(), out_channels, done.clone(), err_fn,
)?,
SampleFormat::U16 => build_output_stream::<u16>(
&device, &config, samples_arc.clone(), cursor.clone(), out_channels, done.clone(), err_fn,
)?,
other => return Err(AudioError::UnsupportedFormat(other)),
};
stream.play()?;
// Wait for the callback to drain the buffer.
let timeout = Duration::from_secs_f64(
(total as f64 / wav_spec.sample_rate as f64 / wav_spec.channels as f64) + 2.0,
);
let start = std::time::Instant::now();
loop {
if *done.lock().unwrap() {
break;
}
if start.elapsed() > timeout {
warn!(target: "spike", "playback timeout reached");
break;
}
std::thread::sleep(Duration::from_millis(50));
}
drop(stream);
let consumed = *cursor.lock().unwrap();
info!(target: "spike", consumed, total, "playback finished");
Ok(())
}
}
fn build_output_stream<T>(
device: &cpal::Device,
config: &cpal::StreamConfig,
samples: Arc<Vec<i16>>,
cursor: Arc<Mutex<usize>>,
out_channels: usize,
done: Arc<Mutex<bool>>,
err_fn: fn(cpal::StreamError),
) -> Result<cpal::Stream, AudioError>
where
T: SizedSample + FromI16 + Send + 'static,
{
let stream = device.build_output_stream(
config,
move |out: &mut [T], _| {
let mut idx = cursor.lock().unwrap();
for frame in out.chunks_mut(out_channels) {
if *idx >= samples.len() {
// End of file — write silence; signal done.
for s in frame.iter_mut() {
*s = T::from_i16(0);
}
*done.lock().unwrap() = true;
} else {
let sample = samples[*idx];
*idx += 1;
for s in frame.iter_mut() {
*s = T::from_i16(sample);
}
}
}
},
err_fn,
None,
)?;
Ok(stream)
}
pub trait FromI16 {
fn from_i16(v: i16) -> Self;
}
impl FromI16 for i16 {
fn from_i16(v: i16) -> Self {
v
}
}
impl FromI16 for u16 {
fn from_i16(v: i16) -> Self {
(i32::from(v) + i32::from(i16::MAX) + 1) as u16
}
}
impl FromI16 for f32 {
fn from_i16(v: i16) -> Self {
f32::from(v) / f32::from(i16::MAX)
}
}
@@ -0,0 +1,38 @@
//! Synthesised waveform for environments with no real input device.
//!
//! Generates a 16-bit-PCM mono WAV containing a sine wave at the
//! requested frequency and duration. This lets the playback path be
//! exercised against the platform output stack even when capture is
//! unavailable (typical headless CI host).
use std::path::Path;
use std::time::Duration;
use hound::{SampleFormat, WavSpec, WavWriter};
use crate::capture::AudioError;
/// Write a sine-wave WAV file. Returns (frames_written, sample_rate).
pub fn synth_sine_wav(
path: &Path,
freq_hz: f32,
duration: Duration,
sample_rate: u32,
) -> Result<(u64, u32), AudioError> {
let spec = WavSpec {
channels: 1,
sample_rate,
bits_per_sample: 16,
sample_format: SampleFormat::Int,
};
let mut w = WavWriter::create(path, spec)?;
let total_frames = ((duration.as_secs_f64() * sample_rate as f64) as u64).max(1);
let amp = f32::from(i16::MAX) * 0.5;
for n in 0..total_frames {
let t = n as f32 / sample_rate as f32;
let s = (2.0 * std::f32::consts::PI * freq_hz * t).sin() * amp;
w.write_sample(s as i16)?;
}
w.finalize()?;
Ok((total_frames, sample_rate))
}