Commit Graph
121 Commits
Author SHA1 Message Date
Edison Jwa 7c9660572e chore: refresh iOS project metadata 2026-05-25 01:20:36 +09:00
Edison Jwa 1326b03301 build: add Android release APK packaging script 2026-05-25 01:20:16 +09:00
Edison Jwa 5515ff6643 feat: stabilize voice activity and audio routing 2026-05-25 01:19:09 +09:00
Edison Jwa d7556cd39f feat: request Android startup permissions 2026-05-25 01:10:10 +09:00
Edison Jwa 11d4e7b7da fix: stabilize Android permission state names 2026-05-25 01:08:02 +09:00
Edison Jwa b8195acc9b perf: avoid duplicate bootstrap work on join 2026-05-24 21:02:16 +09:00
Edison Jwa b494f8902d perf: short-circuit granted mic permission checks 2026-05-24 20:54:49 +09:00
Edison Jwa 7d5d8c2c90 feat: integrate chat voice and diagnostics client 2026-05-23 06:51:55 +09:00
Edison Jwa 7e28791ec2 Handle pokes outside chat tabs 2026-05-23 05:25:36 +09:00
Edison Jwa 738b274748 chore: bump build number to +76 2026-05-22 10:03:17 +09:00
Edison Jwa bf284018e6 feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task
Audio engine (Rust):
- Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD
- Hardware effects (JNI) with software fallback per-effect
- Render reference buffer for AEC between output/capture callbacks
- Voice activity gate: suppress transmission when speaker muted (all platforms)
- Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI
- ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17)
- VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix)
- TEN VAD default backend (was Silero)
- Platform→WebrtcApm resolution after hardware binding
- oboe-rs edisonjwa fork with get_raw_session_id()

Android Kotlin:
- AndroidAudioFocusController + AndroidBluetoothScoController
- AndroidAudioLifecycleController (route changes to Flutter)
- ProGuard rules for new controllers

Flutter UI:
- VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM)
- VoiceStatusChip: mute warning border + Speaker muted label
- BBCode welcome message parser (BbCodeText, case-insensitive)
- Welcome message foldable (expanded by default)
- Link trust dialog (domain wildcards, SharedPreferences)
- HapticFeedback on voice sheet opener
- Server name in AppBar, version v0.1.0
- Default channel (id=1) visible, serverquery clients hidden
- flutter_foreground_task integration

Config:
- ort load-dynamic on all non-iOS (Android/Linux/Windows)
- ONNX Runtime AAR 1.26.0
- ndarray moved to common deps (was Apple-only)
2026-05-22 09:29:57 +09:00
Edison Jwa 6af4ecab0f feat(voice): add iOS VAD runtime support 2026-05-21 20:51:45 +09:00
Edison Jwa 171baf6e41 fix(voice): clamp transmit when speakers muted 2026-05-20 16:37:48 +09:00
Edison Jwa f8a9c7a422 fix(android): add output device picker 2026-05-20 16:26:13 +09:00
Edison Jwa f4db27b565 fix(android): expose audio output devices 2026-05-20 16:26:07 +09:00
Edison Jwa 256874997b fix(android): request microphone permission on startup 2026-05-20 14:52:34 +09:00
Edison Jwa bc1bd89424 fix(android): strengthen user voice indicators 2026-05-20 14:52:34 +09:00
Edison Jwa b21fc16ee6 fix(android): clarify channel and voice status icons 2026-05-20 14:52:34 +09:00
Edison Jwa 34121b2eb7 fix(android): use row-only channel joins 2026-05-20 14:52:34 +09:00
Edison Jwa 48a38dba46 fix(android): prompt from locked channel rows 2026-05-20 14:52:34 +09:00
Edison Jwa f8b6485390 fix(android): show password action only for locked channels 2026-05-20 14:52:34 +09:00
Edison Jwa 273eaf21c1 fix(android): add channel password join action 2026-05-20 14:52:34 +09:00
Edison Jwa 42d0ca0953 fix(android): polish app chrome and join guard 2026-05-20 14:52:34 +09:00
Edison Jwa bc70da51df fix(android): derive current channel from snapshot 2026-05-20 14:52:34 +09:00
Edison Jwa 3ae8e1ab77 fix(android): adapt fold layout and diagnostics 2026-05-20 14:52:34 +09:00
Edison Jwa 30ff955439 fix(android): clear permission-owned hard mute 2026-05-20 14:52:34 +09:00
Edison Jwa eb1f4b895e fix(android): remove cpal product wording 2026-05-20 14:52:34 +09:00
Edison Jwa 210a6a0e11 fix(android): unblock mic permission startup 2026-05-20 14:52:34 +09:00
Edison Jwa d6763787aa feat(android): initialise Oboe runtime context 2026-05-20 14:52:33 +09:00
Edison Jwa 8c253f1d4d feat(voice): harden Android audio and channel joins 2026-05-19 01:58:07 +09:00
EdisonJwa 477394d83e fix(android,build): restore multi-ABI build via cmake-rs patch (DEC-032 exit)
Closes DEC-032. Restores the canonical Android ABI set
{arm64-v8a, armeabi-v7a, x86_64} per SDD-073 item 4 / SDD-118 item 3.

Root cause was the audiopus_sys + cmake-rs + NDK toolchain-file gap:
cargo-ndk 4.x sets ANDROID_ABI / ANDROID_PLATFORM as env vars per
invocation, but upstream cmake-rs 0.x does not forward them to the
child cmake invocation as -D variables, so armeabi-v7a and x86_64
configure steps fell through to the toolchain-file default and
failed to build.

Fix:
- Cargo.toml: add a workspace [patch.crates-io] stanza pinning the
  cmake crate to fork pr2502/cmake-rs @ commit
  bdad5edc569d82151922c5c6c4685b1563f12aa1 (branch android-build),
  which carries cmake-rs PR #257
  (https://github.com/rust-lang/cmake-rs/pull/257). The patch is a
  9-line addition that forwards ANDROID_ABI and ANDROID_PLATFORM
  from the env to the child cmake as -D variables.
- Cargo.lock: regenerated by 'cargo update -p cmake'; the lone
  cmake entry now points at the fork rev.
- apps/chanora_flutter/android/app/build.gradle.kts: restore
  abiFilters to {arm64-v8a, armeabi-v7a, x86_64}; remove the
  TODO(x86_64/armv7 follow-up) comment.
- docs/governance/product-decision-register.md: mark DEC-032 as
  Resolved (2026-05-18) with the resolution mechanism, update the
  §3 / §7 rows, and append a 0.9.8.1 change-history entry.

Verification (host: Linux):
  cargo update -p cmake                          -> pulled fork rev
  cargo check --workspace --all-targets          -> PASS
  cargo test --workspace                         -> PASS (no regressions)
  cargo ndk --platform 28 -t arm64-v8a build -p chanora_bridge   -> PASS
  cargo ndk --platform 28 -t armeabi-v7a build -p chanora_bridge -> PASS
  cargo ndk --platform 28 -t x86_64 build -p chanora_bridge      -> PASS

Upstream tracking: re-evaluate the [patch.crates-io] override once
cmake-rs PR #257 merges and a fresh cmake release lands on
crates.io; at that point switch to a plain dep bump and remove the
override.
2026-05-18 14:48:16 +08:00
EdisonJwa a77a9b2ef2 docs(sdd-119)+feat(macos): consolidate deployment-target SoT (SDD-119 amendment v0.9.17)
Resolve the macOS half of the SDD-119 item 3 single-source-of-truth follow-up. The chanora_bridge cdylib macOS deployment-target floor ('10.15') was previously hard-coded at 7 sites across Podfile and chanora_bridge.podspec; this commit collapses them to a single Ruby constant declaration in a new SoT file.

Selected Option B (Ruby constant) over Option A (.xcconfig — rejected because the podspec prepare_command runs before any xcconfig is applied) and Option C (versioned text file — rejected as overkill given both consumers are already Ruby). Realizes SAD-087(a)'s 'single macOS build-configuration location' mandate.

Out of scope: the pbxproj 'MACOSX_DEPLOYMENT_TARGET = 10.15' lines (565/666/717) belong to the PBXProject default config and are independently overridden to '11.0' at the Runner PBXNativeTarget level; they are the Runner app's floor, not the bridge cdylib's floor. The iOS half (IPHONEOS_DEPLOYMENT_TARGET=13.0) remains an open follow-up.

Files: new apps/chanora_flutter/macos/macos_deployment_target.rb (MACOS_BRIDGE_DEPLOYMENT_TARGET = '10.15'.freeze); Podfile + chanora_bridge.podspec require_relative the constant and consume it at 7 sites; docs/architecture/sdd.md SDD-119 item 3 rewritten + Notes bullet updated + v0.9.17 changelog entry.
2026-05-18 14:39:14 +08:00
EdisonJwa 4c19410556 test+diag: SWE.4 unit tests for Rust paths, Dart service tests, diagnostics audio.android section
Verification + diagnostics:

- apps/chanora_flutter/test/services/back_intent_policy_test.dart:
  8-case truth table for the BackIntentPolicy pure function
  (SWE4-UV-042).
- apps/chanora_flutter/test/services/back_intent_service_test.dart:
  5 channel-routing tests (SWE4-UV-042, SWE5-IV-019 unit slice).
- apps/chanora_flutter/test/services/android_permissions_service_test.dart:
  12 tests covering inbound channel events, outbound requests,
  state-machine transitions, and non-Android short-circuit
  (SWE4-UV-041).
- SDD/SRS trace headers added to alpha_e2e_test.dart,
  beta_e2e_test.dart, widget_test.dart so the existing test ↔ ID
  mapping is discoverable by grep.
- chanora_diagnostics: extends DiagnosticExport with android_audio:
  Option<String> for the SDD-116 evidence schema (requested /
  achieved performance mode + sharing mode + input preset + sample
  rate + frames per burst, per-effect engagement, latency tier).
  The bridge's export_diagnostics() now embeds the Android section
  when current_android_audio_diagnostics() returns Some.

All 93 workspace Rust tests and 26 Dart test/services tests pass.

Trace: SDD-090, SDD-112, SDD-113, SDD-116, SWE4-UV-041, SWE4-UV-042,
SWE4-UV-045, SWE4-UV-047, SWE4-UV-048, SWE4-UV-049, SWE4-UV-051,
SWE4-UV-052, SWE5-IV-019.
2026-05-18 12:48:28 +08:00
EdisonJwa c4145a8727 feat(flutter,android): permission state banner + AndroidPermissionsService + voice-join gate
Dart consumer for the Android permission state pipeline. New
AndroidPermissionsService listens on the app.chanora/android_permissions
MethodChannel and exposes a ValueListenable for the UI. The voice-join
flow in main.dart calls ensureRecordAudio() before rust.voiceJoin and
clamps to listen-only via setHardMute on denial. A non-modal banner
above the VoiceBar surfaces the Grant / Open Settings action depending
on whether the state is Denied or PermanentlyDenied. On non-Android
hosts the service short-circuits to granted; the banner is never built.

Also adds the BackIntentService Dart consumer (back_intent_policy +
back_intent_service) which the Kotlin BackIntentBridge invokes via
MethodChannel for deterministic route-pop ordering.

Trace: SDD-028, SDD-106, SRS-163, SRS-209.
2026-05-18 12:48:28 +08:00
EdisonJwa 0dc8297568 feat(android,p0): foreground service, permission requester, application class, build automation
Android P0 platform shell:

- ChanoraApplication: early System.loadLibrary("c++_shared") +
  System.loadLibrary("chanora_bridge") so JNI is hot before
  MainActivity.onCreate.
- MainActivity: configureFlutterEngine + onResume/onDestroy wiring
  for BackIntentBridge and AndroidPermissionRequester; publishes
  permission-state changes through both the MethodChannel (Dart UI)
  and the JNI hook (Rust audio engine).
- AndroidVoiceForegroundService: microphone-type foreground service
  with notification channel chanora.voice.session per SDD-107.
- AndroidPermissionRequester: RECORD_AUDIO state machine with
  persisted "has-ever-requested" flag so PermanentlyDenied is
  correctly distinguished from never-asked across cold launches.
- BackIntentBridge: API 33+ OnBackInvokedCallback + pre-33
  OnBackPressedDispatcher with deterministic Dart-side policy.
- MethodChannels: centralized constants for app.chanora/*.
- build.gradle.kts: SDD-118 Gradle automation that auto-builds the
  Rust cdylib via cargo-ndk with per-ABI Exec tasks, minimal-env
  isolation, CMAKE_TOOLCHAIN_FILE pinning, libc++_shared.so staging,
  release-inspection assertion. abiFilters temporarily reduced to
  arm64-v8a only per DEC-032 (multi-ABI restoration pending).
- AndroidManifest.xml: INTERNET, RECORD_AUDIO, FOREGROUND_SERVICE,
  FOREGROUND_SERVICE_MICROPHONE, POST_NOTIFICATIONS,
  MODIFY_AUDIO_SETTINGS, BLUETOOTH_CONNECT permissions; service
  declaration with foregroundServiceType=microphone.
- proguard-rules.pro: keep rules for JNI native methods + Flutter
  plugin entry points + FRB bindings.

Trace: SDD-073, SDD-105, SDD-106, SDD-107, SDD-108, SDD-110, SDD-118,
SRS-111, SRS-119, SRS-163, SRS-187, SRS-209, SRS-215.
2026-05-18 12:35:19 +08:00
EdisonJwa 7966a7c8c6 feat(bridge,android): BridgeEvent::PermissionState + JNI publish hook + c++_shared link
Per SDD-106 §5 add BridgeEvent::PermissionState{permission, state}
with the PermissionStateKind enum (Granted, Denied, PermanentlyDenied,
Unknown). The Kotlin side publishes mid-session permission changes
through a new JNI entry point Java_app_chanora_chanora_1flutter
_MainActivity_publishPermissionState routed by the new
permission_jni.rs module; the Rust audio engine subscribes and
authoritatively clamps the transmit gate (see SDD-106 §6).

Adds crates/chanora_bridge/build.rs to emit
cargo:rustc-link-lib=dylib=c++_shared on Android so libchanora_bridge
.so carries DT_NEEDED libc++_shared.so; this is required by Android
API 24+ per-library linker namespaces to resolve __cxa_pure_virtual
and friends at System.loadLibrary time.

Includes the FRB-regenerated Dart counterparts so each commit is
independently buildable.

Trace: SDD-105, SDD-106 §5, SDD-118 item 6 (extended).
2026-05-18 12:32:01 +08:00
Edison Jwa dc9c5c0a4e feat: multi-platform bug fixes, Android audio path, and build tooling
Flutter UI fixes:
- Fix stale channel badge/speaker when moved by others (derive current
  channel from ownClientId instead of optimistic local state)
- Fix Linux PTT via focused fallback key handler
- Distinguish ServerQuery clients with terminal icon in client list
- Reduce duplicate current-channel badge display
- Prevent PTT key-bind save from permanently closing voice settings
- Fix Linux GTK reopen-after-close (quit app on window destroy)
- Fix focused PTT: consume key events, release held keys on
  disconnect/leave-channel/mode/backend changes, suppress stale errors

Flutter Rust bridge:
- Thread is_server_query flag through protocol→bridge→Dart
- Add own_client_id to BridgeSnapshot DTO
- Add log_file_path_str() for platform log path queries

Rust protocol:
- Add ServerQuery test coverage (query_client_type_maps_to_server_query_flag)
- Split reqwest TLS: native-tls for desktop/iOS, rustls for Android

Rust audio:
- Upgrade cpal 0.16→0.17.3 with API adjustments (SampleRate, description())
- Suppress Android-only dead-code warnings (open_log_file, keyring_account)

Android build tooling:
- tools/build-opus-android.sh: NDK auto-discovery, correct CMake
  Android variables (ANDROID_ABI, ANDROID_PLATFORM), portable baseline
- tools/build-android-rust.sh: build+copy Rust cdylib for arm64-v8a,
  armeabi-v7a, x86_64 into android/app/src/main/jniLibs/
- Add jniLibs/ to .gitignore

Rust bridge:
- Guard open_log_file() on non-Android (Android uses logcat)
2026-05-18 00:13:21 +09:00
Edison Jwa 00692b76a2 fix(macOS): enable Hardened Runtime + outgoing network for all configs
Debug and Profile had ENABLE_OUTGOING_NETWORK_CONNECTIONS = NO which
blocked outbound TCP. All three configs now have:
- ENABLE_HARDENED_RUNTIME = YES
- ENABLE_OUTGOING_NETWORK_CONNECTIONS = YES

This was the root cause of the 'Operation not permitted' connection
failure on macOS Sequoia.
2026-05-17 22:32:30 +09:00
Edison Jwa 72c6e14797 feat: modern macOS window chrome + Linux build script
macOS:
- Transparent title bar with hidden title, full-size content view
- macOS: inline Row header (no AppBar) with 56px traffic-light pad
- Other platforms: standard Material AppBar unchanged
- App name 'Chanora' in CFBundleName/CFBundleDisplayName (iOS + macOS)
- NSLocalNetworkUsageDescription added to both platforms

Linux:
- tools/build-linux.sh: builds Rust .so + Flutter bundle + tarball
- Verifies GTK3, libopus dev headers, Rust target
- Copies libchanora_bridge.so into bundle/lib/
2026-05-17 22:29:42 +09:00
Edison Jwa 63b102ed27 feat: add NSLocalNetworkUsageDescription to iOS and macOS Info.plist
Required for local network privacy prompt on macOS 15+ and iOS 14+.
App appears in System Settings → Local Network after connecting to a
LAN server. Internet-hosted servers only need network.client entitlement.
2026-05-17 22:10:39 +09:00
Edison Jwa 4ecea09bbc feat: add permission-denied dialog for macOS network/mic access
When macOS denies network access (PermissionDenied), show a localized
dialog explaining how to grant permission in System Settings, with
an 'Open System Settings' button that opens directly to the Local
Network privacy pane.

Also adds author info (Edison Jwa) to About dialog and moves
diagnostics button from AppBar into About dialog.
2026-05-17 21:57:56 +09:00
Edison Jwa b97cc9589b chore: upgrade dependencies to fix low-severity vulnerability
- connectivity_plus 6.1.5 → 7.1.1
- package_info_plus 8.3.1 → 10.1.0
- package_info_plus_platform_interface 3.2.1 → 4.1.0
- win32 5.15.0 → 6.2.0
- json_annotation 4.11.0 → 4.12.0
- ffi_leak_tracker added (0.1.2)
2026-05-17 21:48:24 +09:00
Edison Jwa 618b6930fe feat: add macOS bridge podspec, Windows project, sync versions to 0.2.0-beta.1 2026-05-17 21:42:54 +09:00
Edison Jwa 7a59f5b9a1 feat(ios,p0): iOS P0 platform, audio fixes, channel UX 2026-05-17 22:00:00 +09:00
EdisonJwa a1fefc8ab6 fix(audio,ios): revert ring buffer back to direct fill_buffer call (rc.8+75)
The ring-buffer architecture (rc.8+73..+74) was making playback
strictly worse. Diagnostic data at +74 conclusively showed:

  * Producer task ran perfectly at 50 Hz (250 ticks per 5 s).
  * AudioHandler returned silence on 65-84% of fill_buffer calls
    even when window_peak_f32 reached 0.98 (full-scale audio).
  * Ring buffer never accumulated beyond 30 ms because consumer
    (VPIO render callback at 43.5 Hz, ~1440 samples per call)
    drained samples faster than the 50 Hz producer could push
    them, in net effect.

The producer drained AudioHandler at 50 Hz \u2014 slightly faster
than iOS VPIO actually consumes audio. Each fill_buffer call
asked for 20 ms but adjacent Opus packets hadn't arrived yet, so
fill_buffer returned mostly silence. Linux/SDL's same pattern
works because SDL calls fill_buffer at EXACTLY the device
callback rate (50 Hz = 20 ms per buffer); the rates match.

Fix: revert to direct fill_buffer call from the render callback
(the SDL pattern in tsclientlib's own reference example at
tsclientlib/examples/audio_utils/ts_to_audio.rs). The render
callback now:

  1. Resizes scratch_stereo Vec to 2 * num_frames f32 if needed
  2. Zeros the live slice (fill_buffer is additive, not clearing)
  3. Locks AudioHandler, calls fill_buffer(scratch_stereo)
  4. Downmixes L+R -> mono i16 with master gain into out[]
  5. Applies output_muted bypass
  6. Tracks peak_out + audio/silence ratios for diagnostic

The closure owns scratch_stereo across callbacks for stable
allocation. Same memory model as Linux/SDL.

Removed:
  * tokio::spawn producer task
  * rtrb dep + RingBuffer<i16> + Producer/Consumer split
  * tokio::sync::oneshot shutdown channel
  * producer_shutdown_tx field on IosVoiceUnit struct
  * RING_BUFFER_SAMPLES / PRODUCER_TICK_MS constants
  * Producer-side diagnostic counters

Diagnostic kept: cb / num_frames / frames_changes /
callbacks_with_audio / callbacks_with_silence / peak_out_i16 /
gain. Logged every 100 callbacks.

The choppy / clicks symptom is independent of the buffer
architecture \u2014 it's whatever AudioHandler is doing on iOS
that's different from Linux. Next investigation step is to
either (a) switch from VPIO to RemoteIO unit (lose Apple's
voice processing entirely), or (b) understand why AudioHandler
returns silence so often on iOS-arrival packet timing patterns.

Build counter 74 -> 75.
2026-05-17 13:05:09 +08:00
EdisonJwa a9aa19ecdd diag(audio,ios): comprehensive producer + consumer ring-buffer metrics (rc.8+74)
External reviewer correctly identified that the +73 ring-buffer
commit didn't fix the symptom but the architecture is still
right. We need to distinguish two possible causes:

  (a) Producer task isn't running (or running too rarely) so
      ring stays underfilled.
  (b) Producer IS running but fill_buffer returns zeros most of
      the time (AudioHandler stuck in buffering_samples state
      or no packets reaching it).

The +73 render-side diagnostic was insufficient: we logged
underruns + peak_out_i16 but not what the producer was
actually pushing. This commit adds producer-side metrics
rolled up every 5 s (250 ticks at 20 ms):

Producer task:
  producer_ticks           : timer firings (= ~250 per 5 s window;
                             fewer = tokio scheduler stalled)
  produced_chunks          : pushes into ring (= ticks - drops)
  fill_buffer_calls        : AudioHandler queries
  fill_buffer_zero_returns : ticks where scratch came back all
                             zeros (no decoded content to play)
  ring_full_drops          : ticks where ring was full and we
                             skipped the push
  ring_min/max_samples     : depth envelope across window
  ring_min/max_ms          : same in milliseconds
  window_peak_f32          : max scratch sample across window
  window_rms_f32           : RMS of all scratch samples across
                             window

Render callback (per-100-callback as before, plus new fields):
  ring_avail_before        : Consumer::slots() before this read
                             (= how many samples were sitting in
                              the ring at callback entry)
  read_frames              : samples successfully popped
  zero_filled              : samples zero-filled because ring
                             was empty (= num_frames - read_frames)
  underruns / underrun_samples / peak_out_i16 / clip_count_i16
                           : as before

Reading the next iteration's log:

  If producer_ticks << 250  per 5 s window:
      tokio scheduler isn't running the task fast enough.
      Move producer to its own dedicated runtime, or use
      std::thread + std::sync::mpsc + std::thread::sleep
      instead of tokio.

  If producer_ticks ~= 250 AND fill_buffer_zero_returns is
  high (most ticks return silence):
      AudioHandler isn't decoding packets fast enough OR is
      stuck buffering. Bug is upstream in protocol layer
      packet delivery or AudioHandler's jitter state machine.
      The ring buffer architecture cannot fix this.

  If producer_ticks ~= 250 AND fill_buffer_zero_returns is
  low AND ring_min_ms stays >100ms AND underruns are low BUT
  consumer's peak_out_i16 is still 0:
      Something is wrong between push and pop. Lock-free
      ring corruption, or wrong stride.

Pure diagnostic. No behavioural change beyond the logging.
Producer scratch envelope scan is O(scratch.len()) = 1920
samples per 20 ms tick = ~96k iterations/sec on the audio
producer thread \u2014 negligible CPU.

Build counter 73 -> 74.
2026-05-17 02:47:35 +08:00
EdisonJwa 99584fbc1a fix(audio,ios): decouple AudioHandler from VPIO render callback via ring buffer (rc.8+73)
User confirmed at +72 the symptom is 'voice + constant clicks +
choppy fragments'. The diagnostic data conclusively pointed to
iOS VPIO render-callback timing as the cause:

  * frames_changes=60+ per 100 callbacks at cb>=1800
    iOS keeps switching num_frames between 960 and 1104
    on roughly 60% of callbacks
  * peak_out_i16 is sensible (2500-16870, never clipping)
    when AudioHandler returns content
  * input_was_zero=true on most callbacks during active speech
    AudioHandler keeps entering buffering_samples state

The cause: previous render callback called fill_buffer
synchronously every iOS audio thread invocation. With iOS
calling at irregular rates with irregular sizes, AudioHandler's
jitter buffer (sized around 20 ms Opus frames) cannot satisfy
arbitrary-sized requests and falls back to returning silence
(&[] empty slice) on misaligned reads. The silent gaps in the
middle of the output buffer create discontinuities = audible
clicks; the missing-tail content produces choppy fragments.

Fix (architectural): decouple the AudioHandler decoder from the
VPIO render callback via a lock-free SPSC ring buffer.

  Producer (tokio task, 50 Hz):
    every 20 ms:
      fill_buffer(scratch_stereo_f32, 1920 = 20 ms stereo)
      downmix L+R -> mono i16 (960 samples)
      ring_buffer.push_slice(mono_i16)

  Consumer (VPIO render callback, iOS audio thread):
    every callback:
      pop num_frames samples from ring buffer into out
      zero-fill tail on underrun

Why it works:
  * Producer always asks AudioHandler for a stable 20 ms chunk
    (perfectly aligned with internal Opus frame size). No more
    buffering_samples false-triggers.
  * Consumer pulls whatever iOS asks for whenever iOS schedules
    it; ring buffer's 200 ms depth absorbs the callback jitter.
  * This is the standard pattern every production VoIP audio
    engine uses (WebRTC, Discord, FaceTime) to bridge bursty
    Opus decoders to bursty platform audio callbacks.

Implementation:
  * New dep: rtrb 0.3.4 (RustAudio realtime-safe SPSC ring
    buffer, 6.8M downloads, lock-free push/pop with no
    allocation on the audio thread).
  * RING_BUFFER_SAMPLES = 9600 (200 ms mono i16 at 48 kHz).
    Sized for 10x producer ticks of headroom.
  * PRODUCER_TICK_MS = 20 (matches Opus 50 Hz packet rate).
    set_missed_tick_behavior(Skip) to avoid burst catch-up on
    runtime stalls.
  * Producer task spawned in IosVoiceUnit::start, shutdown
    via tokio::oneshot when IosVoiceUnit drops.
  * Render callback is now just: pop into out, zero-fill tail,
    apply mute then gain.
  * Gain applied CONSUMER-side so user volume changes take
    effect within one callback (<= 200 ms latency).
  * Underrun diagnostics: count underrun callbacks + total
    zero-filled samples, log every 100 callbacks.

Threading + safety:
  * rtrb is lock-free SPSC. Audio thread never blocks.
  * Producer can block briefly on Arc<Mutex<AudioHandler>>
    contention with the inbound forwarder (handle_packet), but
    not with the audio thread.
  * Producer task is owned by tokio runtime; explicit shutdown
    channel ensures it exits when the engine stops.

Build verify:
  * Linux host: cargo check clean in 4.06s (downloads rtrb 0.3.4).
  * iOS Mac:   cargo check clean in 2.14s.

Build counter 72 -> 73.
2026-05-17 02:39:42 +08:00
EdisonJwa 53e09ea091 diag(audio,ios): comprehensive render-callback metrics per external review (rc.8+72)
External code review pushed back on the 'iPhone speaker hardware
distortion' hypothesis and pointed out we need more than just
peak measurements. The reviewer's checklist:

  * peak_i16
  * rms_i16
  * num_clipped_samples (abs >= 32767)
  * zero_fill_count / underrun_count
  * callback_frame_count variability
  * decoded_packet_duration_ms
  * input_was_zero
  * actual ASBD / actual sample rate

The format diagnostics at +71 already showed iOS honoured
48 kHz Int16 mono on both buses and that .default mode +
.defaultToSpeaker routed to the speaker correctly with
outputVolume=0.45. So format + route are confirmed correct.
The remaining mystery is WHY 'loud but distorted' \u2014 we need
sample-level metrics to isolate where in the pipeline the
breakage occurs.

This commit instruments the VPIO render callback with:

  * num_frames + frames_changes : detects iOS re-negotiating
                                 buffer size between callbacks
                                 (which would imply jitter the
                                 fixed scratch_stereo Vec can't
                                 absorb cleanly).
  * peak_stereo + rms_stereo  : characterises AudioHandler's
                                output BEFORE our downmix.
                                Distinguishes 'real audio
                                arriving' from 'silence'.
  * peak_out_i16 + clip_count : measures what we hand VPIO.
                                clip_count > 0 means we're
                                clipping at our boundary even
                                with gain=1.0 \u2014 indicates
                                upstream is over-driven.
  * input_was_zero            : explicit silence/no-talker
                                indicator separate from peak=0
                                which could mean tiny content
                                rounded to 0.

Reviewer's preferred diagnostic path is to dump PCM to file
and play with ffplay externally; that's iOS-impractical
without a shared filesystem path the user can extract via
Files.app. Instead we sample the same metrics in-callback at
~2 Hz which gives us the same information at run time.

Pure diagnostic. No behavioural change. Counters live in the
FnMut closure so the audio thread cost is one branch +
counter increment per callback, plus a one-pass RMS sum +
peak scan every 100 callbacks.

Build counter 71 -> 72.

Reviewer also recommended a headphone test in parallel \u2014
that will be done by the user (out-of-band) at the next
test cycle to determine whether the symptom changes when
audio leaves the speaker path.
2026-05-17 02:25:31 +08:00
EdisonJwa 2735c55c97 diag(audio,ios): log actual VPIO + AVAudioSession state post-init (rc.8+71)
Per external review (helpful checklist from ChatGPT-style analysis
pointing out we never verified that iOS actually accepted our
preferred sample rate / channels / format): preferredSampleRate
and preferredIOBufferDuration are HINTS, not guarantees. iOS may
substitute its own values if the hardware can't satisfy our
preference. If VPIO is running at 44.1 kHz Float32 stereo while
our render callback writes 48 kHz Int16 mono into the buffer,
the symptoms would match what user reports (broken playback,
pitch shifted, severe distortion) and our previous diagnostics
wouldn't catch it because they only sampled signal-level metrics.

This commit adds two diagnostic emissions to verify:

1. AppDelegate.swift::activateAudioSession: after setActive
   succeeds, log the ACTUAL session state \u2014 category, mode,
   sampleRate, ioBufferDuration, current route (inputs +
   outputs), outputVolume. Lets us see whether iOS honoured our
   .default + .defaultToSpeaker setup and which physical route
   it picked at launch.

2. ios_voice_unit.rs::IosVoiceUnit::start: after unit.start()
   succeeds, log the actual OUTPUT and INPUT stream formats
   VPIO accepted (sample_rate, channels, sample_format, flags).
   If these differ from our requested 48 kHz Int16 mono, we
   have a format-substitution problem.

Three possible outcomes from the next test:

* Both diagnostics confirm 48 kHz Int16 mono on both buses and
  the session sampleRate=48000 -> format is correct; the
  playback breakage is somewhere else (e.g. AudioHandler
  jitter buffer behaviour, route binding, or hardware mixer).

* Session sampleRate != 48000 -> we need to insert a sample
  rate converter or pin AVAudioSession's
  setPreferredSampleRate(48000) explicitly in Swift before
  setActive.

* VPIO substituted Float32 for our Int16 request -> our render
  callback is writing i16 magnitudes into a Float32 buffer
  which would explain the distortion. Fix: write Float32
  directly using data::Interleaved<f32> instead of i16.

Build counter 70 -> 71. Pure diagnostic; no behavioural
change.
2026-05-17 02:17:37 +08:00
EdisonJwa 6e0bf21295 fix(audio,ios): route playback via media channel (.default + .defaultToSpeaker) (rc.8+70)
User report after the 8x boost commit (e85a6d3): playback STILL
broken, but now the diagnostic clearly shows the actual problem.
Render-callback peak_out_i16 SATURATES at 32767 on speech peaks
(cb=600, 1100, 1200, 2400) because the 8x boost amplifies an
already-loud signal into hard clipping. Quiet content reaches
audible level but loud peaks are catastrophically distorted.

The 8x boost was treating the wrong cause.

Real root cause (researched online after user prompted: 'this is
iOS a popular platform, there must be solutions'): iOS has TWO
independent audio channels:

  In-call channel  (.voiceChat / .videoChat modes)
    * Routes through the phone-call audio path.
    * Aggressively ducks non-voice content to the earpiece.
    * Volume controlled by a separate in-call hardware
      register, not the side buttons when not actively on a
      phone call.

  Media channel  (.default mode)
    * Routes through the standard media playback path.
    * No automatic ducking.
    * Volume controlled by the side volume buttons normally.

With AVAudioSession mode .voiceChat, iOS sends our output
through the in-call channel which plays at 'earpiece-level'
loudness on the speaker too. Signal is technically present but
buried under the speaker's noise floor. With mode .default +
.defaultToSpeaker option, output routes via media channel and
plays at normal loudness.

Both Twilio (video-quickstart-ios) and Daily.co (patched WebRTC
module) document the same workaround and use VPIO for AEC while
keeping the session mode at .default for loud playback:

  github.com/twilio/video-quickstart-ios/issues/522
  stackoverflow.com/questions/79834998 (Daily.co)

The user also noticed 'tx/rx almost no changes even receiving
packages' \u2014 likely a misinterpretation of the frames counter
not advancing as fast as expected during quiet voice; AudioHandler
returns silence when its jitter buffer is in buffering_samples
state which doesn't fire 'decode failed' but also doesn't
increment frames_received. The real issue is still the playback
ducking; the counter behaviour is a downstream symptom.

Changes:

1. AppDelegate.swift: AVAudioSession mode .voiceChat -> .default
   with options [.defaultToSpeaker, .allowBluetoothHFP,
   .allowBluetoothA2DP]. VPIO continues to do its job (AEC, NS,
   AGC on the mic side); only the playback routing changes.
   The earlier 'speaker selector silent under .default' bug
   does NOT apply because we no longer use cpal RemoteIO \u2014
   VPIO honours overrideOutputAudioPort under any mode.

2. ios_voice_unit.rs: revert the 8x output boost from e85a6d3.
   With media-channel routing, signal levels are correct and
   no software amplification is needed. Render callback restored
   to plain (l+r)*0.5*gain downmix.

3. ios_voice_unit.rs: revert the BypassVoiceProcessing toggle
   from c16318c. The VPIO chain stays enabled so we keep
   capture-side AEC/AGC/NS for free \u2014 the playback breakage
   it was trying to fix was the wrong layer all along.

4. ios_voice_unit.rs: drop the diagnostic render-callback log
   line. Production-clean code; can be re-enabled by reverting
   the diff in the closure if future debugging needs it.

Build counter 69 -> 70.
2026-05-17 02:11:07 +08:00