Commit Graph
105 Commits
Author SHA1 Message Date
Edison Jwa 484dad1072 fix(audio): eliminate Android output stutter via Oboe config + lock-free callback
Phase 1 — Oboe configuration:
- Change output stream from Usage::VoiceCommunication to Usage::Game with
  ContentType::Sonification to avoid forcing the Legacy (OpenSL ES) data
  path on most devices (Oboe issue #2075)
- Switch output format from i16 Mono to f32 Stereo, matching Qint's proven
  configuration and eliminating per-callback downmix conversion
- Set buffer size to 2x burst after stream open, reducing default buffer
  from 8-20x burst to 2x burst for lower latency
- Remove scratch Mutex<Vec<f32>>; callback writes directly to Oboe buffer

Phase 2 — Lock-free output callback:
- Add audio_event_queue.rs: lock-free SPSC bridge using crossbeam ArrayQueue
  with separate packet (lossy) and control (reliable) channels
- OutputCallback now owns AudioHandler directly (no Arc<Mutex<>> on Android)
- Inbound forwarder pushes packets via AudioEventProducer (no mutex)
- set_client_volume pushes control commands via event queue on Android
- iOS/desktop Arc<Mutex<AudioHandler>> path unchanged
2026-06-04 00:51:25 +09:00
Edison Jwa 29afbb5e97 fix: address post-event-driven issues and client info parity (#16)
* fix(ui): restore speaking status indicators

Speaking state (isSpeaking) is computed from voice activity timestamps
in the protocol layer and cannot be represented as a discrete delta.
The event-driven refactor removed periodic snapshot refreshes, causing
speaking indicators to go stale.

Adds a 750ms periodic snapshot refresh (matching SPEAKING_ACTIVITY_WINDOW)
while the audio stats timer is active (in-channel only). Structural
changes (moves, joins, leaves) are still handled by instant deltas.

* fix(ui): hide server query clients from delta joins

When a ServerQuery client sends a message, a ClientJoined delta fires.
Before PR#15 the periodic snapshot rebuild would include the SQ client
but the snapshot_view filter hid it. With deltas, the client persisted
in the local snapshot. Now ClientJoined deltas skip SQ clients entirely.

* fix(proto): log getconnectioninfo errors instead of silently discarding

Ping and packet loss showing 'Unknown' in the client info sheet is
caused by getconnectioninfo failures being silently swallowed. Now
logs the error with the client_id so the root cause can be diagnosed
(e.g. missing b_client_connectioninfo_view permission on the server).

Also logs clientgetvariables failures.

* fix(proto): refresh non-self client profiles before mapping

* feat(protocol): add ping deviation to client profiles

* chore(ui): regenerate Flutter bridge bindings for ping deviation

* fix(l10n): add ping deviation labels to client info

* feat(ui): show ping deviation in client info sheet
2026-06-03 18:26:29 +09:00
Edison Jwa 808324f374 feat: event-driven UI updates for instant channel switching (#15)
* chore: regenerate Cargo.lock after rebase

* fix(ui): add 1s cool-down to prevent double-tap channel join

voiceJoin returns instantly (fire-and-forget protocol), so the
pending-join guard clears before a second tap lands. The cool-down
prevents the rapid channel oscillation and ClientIsFlooding (524)
that results from double-tapping.

* fix(ui): handle ChannelAlreadyIn as success, ClientIsFlooding with backoff

- ChannelAlreadyIn (0x0302): treat as silent success, update UI state
- ClientIsFlooding (0x020c): show localized snackbar, extend cooldown 5s
- Add l10n strings for flooding error (en + zh)

* fix(proto): use Windows TS3 client version for broadest compatibility

Matches Qint's default (Windows_3_X_X__1). Avoids server-side
behavioral differences with TS5 version strings.

* fix(proto): patch tsproto-types to handle short P-256 coordinates

BigInt::to_bytes_be() strips leading zeros, causing WrongPublicKeyLength
when a server's ephemeral key coordinate starts with 0x00. Patch from
EdisonJwa/tsclientlib fix/p256-short-coordinate-pad branch left-pads
coordinates to the P-256 field size instead of rejecting them.

* refactor(core): stop watchdog from emitting SnapshotChanged

The watchdog now serves only as a liveness probe (miss counting for
reconnection). UI updates are handled entirely by the event-driven
delta path (ProtocolDelta → SessionEvent → BridgeEvent → Flutter).

Removes signature tracking and SnapshotChanged emission from the
supervisor loop. The initial snapshot is still fetched via the
Connected event handler in Flutter.

* refactor(ui): remove channel-join cooldown guard

With event-driven deltas the UI updates instantly on channel moves,
so the 1-second cooldown is no longer needed. Double-taps are handled
by the server (ChannelAlreadyIn → success) and the pending-channel-id
guard prevents overlapping requests.

Also removes the _lastJoinCompletedAt field entirely.

* fix(core): reattach event forwarders after reconnect

The reconnect path swapped in a new ProtocolClient but never took
chat_rx, activity_rx, or delta_rx from it. After the first reconnect,
the event-driven UI pipeline was dead.

Fix by extracting spawn_event_forwarders() helper called on both
initial connect and reconnect. Also replaces lossy try_recv+sleep
polling with proper recv().await for push-based delivery.

* feat(protocol): enrich delta schema with all snapshot-visible fields

ClientJoined now carries input_muted, output_muted, is_server_query,
talk_power, talk_power_granted. ChannelAdded/ChannelUpdated now carry
has_password and needed_talk_power. ClientUpdated also carries
is_server_query, talk_power, talk_power_granted.

This prevents local snapshot drift where fabricated defaults could
hide password requirements, talk-power restrictions, or client type.

* refactor: remove dead SnapshotChanged variant end-to-end

SnapshotChanged is no longer emitted since the watchdog was refactored
to liveness-only. Removes the variant from SessionEvent, BridgeEvent,
and the Flutter switch statement. FRB bindings regenerated.

* fix(ci): regenerate license inventory and fix iOS submodule fetch

- Regenerate docs/security/license-inventory.md to match current lockfile
- Remove submodules: true from checkout (causes hard fail on private submodule)
- Add explicit git submodule update --init --depth=1 with || true fallback
- Check silero-coreml/Package.swift instead of directory existence
2026-06-03 18:20:33 +09:00
Edison Jwa 2c3c3873dc fix(diag): record bridge and audio interruption diagnostics (#18)
* fix(diag): record bridge events in diagnostics page

Connection lost/reconnecting/disconnected and iOS audio interruption
events now appear in the diagnostics dialog alongside existing error
snackbar entries.

* fix(diag): reduce audio callback sample verbosity to debug

The render callback diagnostic sample logged every 100 callbacks
(~2s) at INFO level, flooding the 256-entry release log buffer and
pushing out useful events. Changed to DEBUG so it only appears in
debug builds with the larger 4096-entry buffer.

* fix(diag): add timestamps to Rust diagnostic log entries
2026-06-03 18:09:25 +09:00
Edison Jwa dab90852a3 Merge pull request #13 from EdisonJwa/feat/apple-coreml-vad
Add Apple CoreML Silero VAD
2026-06-02 20:14:17 +09:00
Edison Jwa 1813bbaa0c fix(ci): keep benchmark advisory non-blocking 2026-06-02 20:01:58 +09:00
Edison Jwa ddf858cc6c fix(audio): address CoreML VAD review feedback 2026-06-02 19:39:11 +09:00
Edison Jwa 7d47a1e14b chore(prefetch): rename workspace crate 2026-06-02 13:33:41 +09:00
Edison Jwa 813a38e92b feat(audio): add Apple CoreML Silero VAD 2026-06-02 01:52:08 +09:00
Edison Jwa 25e6a16eb1 Merge pull request #10 from EdisonJwa/fix/ios-audio-lifecycle-lag
fix(ios): stabilize audio lifecycle startup
2026-05-31 23:43:32 +09:00
Edison Jwa ecb9ae9636 fix(audio): restart iOS voice unit in place 2026-05-31 22:30:56 +09:00
Edison Jwa 6f7063971d fix(audio): use WebRTC VAD on iOS 2026-05-31 22:30:28 +09:00
Edison Jwa c02d4e6df3 fix(bridge): serialize iOS audio lifecycle events 2026-05-31 22:29:38 +09:00
Edison Jwa 681f3b636f test(storage): isolate temp directories in tests 2026-05-29 19:19:20 +09:00
Edison Jwa fe6e07353e chore: restore product scaffold to rollback baseline 2026-05-29 14:02:04 +09:00
Edison Jwa 2896f14ec9 fix: show one linux audio backend 2026-05-25 18:31:20 +09:00
Edison Jwa d03ea937e6 fix: refine linux voice runtime behavior 2026-05-25 18:26:49 +09:00
Edison Jwa a2d686d9d0 feat: promote linux native audio path 2026-05-25 17:42:06 +09:00
Edison Jwa b8df25a195 Reduce Linux setup ambiguity and surface desktop input/message failures honestly
Clarify ONNX Runtime guidance with direct-open install hints, restore desktop WebRTC VAD visibility, map mouse side buttons through focused PTT capture/runtime paths, and wait for server acks before showing chat sends as successful.

Constraint: Linux release UX must stay functional when ONNX Runtime is optional and GNOME portal availability varies
Rejected: Keep desktop VAD locked to Silero only | misleads users when ONNX Runtime is skipped
Confidence: medium
Scope-risk: moderate
Directive: Preserve the protocol send-ack wait path for chat so UI success always tracks real server acceptance
Tested: flutter analyze lib/main.dart lib/widgets/chat_views.dart lib/widgets/input_dialogs.dart lib/widgets/startup_dependency_screen.dart; flutter test test/widgets/input_dialogs_test.dart test/widgets/chat_views_test.dart test/services/startup_dependency_check_test.dart test/widgets/startup_dependency_screen_test.dart test/widgets/voice_settings_controls_test.dart test/widgets/audio_processing_config_state_test.dart; cargo test -p chanora_protocol --lib; cargo test -p chanora_audio ptt_backends --lib
Not-tested: Live manual GNOME portal rebind/global PTT on a real desktop session; observer-bot chat against a live server after the sender-name fallback change
2026-05-25 11:55:10 +09:00
Edison Jwa 0648203100 build: lock resolver dependencies 2026-05-25 02:58:53 +09:00
Edison Jwa 6f64062fd8 chore: clean Android voice build warnings 2026-05-25 01:27:28 +09:00
Edison Jwa 5515ff6643 feat: stabilize voice activity and audio routing 2026-05-25 01:19:09 +09:00
Edison Jwa eb9014cd81 feat: add TeamSpeak address resolver 2026-05-25 01:12:50 +09:00
Edison Jwa 7d5d8c2c90 feat: integrate chat voice and diagnostics client 2026-05-23 06:51:55 +09:00
Edison Jwa bf284018e6 feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task
Audio engine (Rust):
- Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD
- Hardware effects (JNI) with software fallback per-effect
- Render reference buffer for AEC between output/capture callbacks
- Voice activity gate: suppress transmission when speaker muted (all platforms)
- Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI
- ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17)
- VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix)
- TEN VAD default backend (was Silero)
- Platform→WebrtcApm resolution after hardware binding
- oboe-rs edisonjwa fork with get_raw_session_id()

Android Kotlin:
- AndroidAudioFocusController + AndroidBluetoothScoController
- AndroidAudioLifecycleController (route changes to Flutter)
- ProGuard rules for new controllers

Flutter UI:
- VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM)
- VoiceStatusChip: mute warning border + Speaker muted label
- BBCode welcome message parser (BbCodeText, case-insensitive)
- Welcome message foldable (expanded by default)
- Link trust dialog (domain wildcards, SharedPreferences)
- HapticFeedback on voice sheet opener
- Server name in AppBar, version v0.1.0
- Default channel (id=1) visible, serverquery clients hidden
- flutter_foreground_task integration

Config:
- ort load-dynamic on all non-iOS (Android/Linux/Windows)
- ONNX Runtime AAR 1.26.0
- ndarray moved to common deps (was Apple-only)
2026-05-22 09:29:57 +09:00
Edison Jwa 6af4ecab0f feat(voice): add iOS VAD runtime support 2026-05-21 20:51:45 +09:00
Edison Jwa b65cedfa95 fix(android): recover from input stream failures 2026-05-20 14:52:34 +09:00
Edison Jwa b21fc16ee6 fix(android): clarify channel and voice status icons 2026-05-20 14:52:34 +09:00
Edison Jwa f8b6485390 fix(android): show password action only for locked channels 2026-05-20 14:52:34 +09:00
Edison Jwa 3ae8e1ab77 fix(android): adapt fold layout and diagnostics 2026-05-20 14:52:34 +09:00
Edison Jwa eb1f4b895e fix(android): remove cpal product wording 2026-05-20 14:52:34 +09:00
Edison Jwa 210a6a0e11 fix(android): unblock mic permission startup 2026-05-20 14:52:34 +09:00
Edison Jwa 26b06ce786 fix(android): expose voice diagnostics safely 2026-05-20 14:52:33 +09:00
Edison Jwa d6763787aa feat(android): initialise Oboe runtime context 2026-05-20 14:52:33 +09:00
Edison Jwa c87b47f064 feat(audio): prefer native voice backends 2026-05-20 14:52:33 +09:00
Edison Jwa 8c253f1d4d feat(voice): harden Android audio and channel joins 2026-05-19 01:58:07 +09:00
EdisonJwa 7188a5a69d feat(perf,benchmark-infra): criterion bench harness + advisory CI workflows (SDD-120)
Implementation of SDD-120 §1-§8:

Bench harness (crates/chanora_audio/benches/):
- common.rs: deterministic synthetic audio (440 Hz sine, no RNG).
- realtime_capture.rs: bench_capture_alloc_count (dhat) +
  bench_capture_callback_wall_clock (criterion).
- opus_codec.rs: bench_opus_encode_latency + bench_opus_decode_latency
  (direct audiopus, not AudioHandler — SDD-120 §3 item 4).
- resampler.rs: bench_resampler_throughput across 44.1->48 /
  16->48 / 48->48 passthrough.

CI tooling (crates/chanora_audio/examples/):
- emit_baseline.rs: aggregates criterion estimates.json outputs
  into the SRS-217 baseline schema.
- compare_baseline.rs: applies SRS-219 tolerance, renders markdown
  table with 🟢/🟡/🔴 markers + yellow simpler-form realization per
  SDD-120 §8.

  Deviation from SDD-120 §2 / §5 / §7 placement: these tools live
  under examples/, not benches/ or src/bin/. Rationale: they must
  consume serde_json (a dev-only dep — production builds must not
  pull it). Cargo only resolves dev-dependencies for [[test]],
  [[bench]], and [[example]] targets; [[bin]] targets under
  src/bin/ see only regular [dependencies]. examples/ keeps the
  binaries out of the production dep tree while still giving them
  cargo run --example invocation. An SDD-120 amendment should
  reflect this.

Workflows (.github/workflows/):
- bench-advisory.yml: PR + push triggers; runs benches; posts a
  sticky PR comment via actions/github-script@v7; job status is
  always success (SRS-218 clause 4 — non-blocking).
- bench-baseline-update.yml: workflow_dispatch only; runs benches;
  opens PR via peter-evans/create-pull-request@v6 (sole writer of
  the SAD-089 baseline JSON).

Cargo.toml additions ([dev-dependencies] only — verified excluded
from --release builds): criterion 0.5, dhat 0.3, serde_json 1.

Source-code seam: minimal pub-but-#[doc(hidden)] bench_seam module
in chanora_audio (engine.rs + lib.rs re-export) so the criterion
bench harness can construct a CaptureState and drive
CaptureState::ingest without re-implementing the engine (SDD-120
§3). Non-iOS targets only — CaptureState itself is iOS-gated.

Initial baseline seed: crates/chanora_audio/benches/baselines/
x86_64-unknown-linux-gnu.json = {}. compare_baseline handles the
missing-baseline case gracefully and emits a 'no red markers'
report; the first manual dispatch of bench-baseline-update.yml
after merge establishes the real values.

Out of scope per SDD-120 §10: production telemetry export,
build-failing hard CI gate, multi-host benchmarking, IDE
integration, Dart-side bridge round-trip bench.

Verification:
- cargo check --workspace --all-targets: PASS.
- cargo bench --bench realtime_capture --no-run: PASS.
- cargo bench --bench opus_codec --no-run: PASS.
- cargo bench --bench resampler --no-run: PASS.
- cargo build --example emit_baseline --example compare_baseline
  -p chanora_audio: PASS.
- cargo test --workspace: 106 passed, 0 failed, 3 ignored — no
  regression from prior count.
2026-05-18 13:52:15 +08:00
EdisonJwa d13b56d379 perf(audio): pre-allocate capture scratch buffers to avoid realtime-thread Vec allocs (SDD-094)
The capture cpal callback (CaptureState::ingest) ran two heap
allocations per callback on the realtime audio thread:

  1. engine.rs:1196-1202 — fresh `mono: Vec<f32>` for the downmix
     output, once per cpal callback (50–100 Hz).
  2. engine.rs:1217-1218 — `pcm_accum.drain(..FRAME_SAMPLES).collect()`
     building a fresh Vec<f32> of 960 samples per Opus frame.

Both sites mirror the pattern already fixed for the output side at
engine.rs:1389-1397, where allocating per callback on glibc malloc
was correlated with user-perceptible audio popping. The output-side
fix replaced the per-callback allocation with a pre-allocated
`scratch` Vec that is cleared and resized in place; this commit
applies the same template to the capture side.

Changes:
- Add `mono_scratch: Vec<f32>` and `frame_scratch: Vec<f32>` to
  CaptureState. Initialised with Vec::with_capacity(4096) and
  Vec::with_capacity(FRAME_SAMPLES=960) respectively in
  CaptureState::new.
- Replace the downmix Vec construction with in-place push into
  `self.mono_scratch`; `clear()` retains capacity across callbacks.
- Replace the drain().collect() with `self.frame_scratch.extend(
  self.pcm_accum.drain(..FRAME_SAMPLES))`; same capacity-retention.
- The resampler call uses std::mem::take to swap the scratch buffer
  out for the duration of the &mut self call, then moves it back —
  the backing allocation is preserved across callbacks.

Algorithm semantics are unchanged: same downmix arithmetic, same
clamp loop, same Opus encode call sequence. Only the storage
strategy differs.

Out of scope (intentionally not touched):
- Android audio path (android_voice_unit.rs, mobile_voice_backend.rs):
  researcher constraint C-4 — the Android cpal data path is mid-
  migration and being replaced.
- Output callback at engine.rs:1409+: the only obvious per-callback
  allocation there (`scratch`) was already fixed; a fuller audit
  is a separate scope decision.
- The `scratch` buffer at engine.rs:1389-1397 — already correct.

Verification:
- cargo check --workspace --all-targets: passes.
- cargo test --workspace: 106 passed / 0 failed / 3 ignored.
- cargo clippy --workspace --all-targets: no new lints introduced;
  the one warning inside the edited region (clamp-like pattern at
  line 1266) was pre-existing on the copied clamp loop.
2026-05-18 13:01:52 +08:00
EdisonJwa 4c19410556 test+diag: SWE.4 unit tests for Rust paths, Dart service tests, diagnostics audio.android section
Verification + diagnostics:

- apps/chanora_flutter/test/services/back_intent_policy_test.dart:
  8-case truth table for the BackIntentPolicy pure function
  (SWE4-UV-042).
- apps/chanora_flutter/test/services/back_intent_service_test.dart:
  5 channel-routing tests (SWE4-UV-042, SWE5-IV-019 unit slice).
- apps/chanora_flutter/test/services/android_permissions_service_test.dart:
  12 tests covering inbound channel events, outbound requests,
  state-machine transitions, and non-Android short-circuit
  (SWE4-UV-041).
- SDD/SRS trace headers added to alpha_e2e_test.dart,
  beta_e2e_test.dart, widget_test.dart so the existing test ↔ ID
  mapping is discoverable by grep.
- chanora_diagnostics: extends DiagnosticExport with android_audio:
  Option<String> for the SDD-116 evidence schema (requested /
  achieved performance mode + sharing mode + input preset + sample
  rate + frames per burst, per-effect engagement, latency tier).
  The bridge's export_diagnostics() now embeds the Android section
  when current_android_audio_diagnostics() returns Some.

All 93 workspace Rust tests and 26 Dart test/services tests pass.

Trace: SDD-090, SDD-112, SDD-113, SDD-116, SWE4-UV-041, SWE4-UV-042,
SWE4-UV-045, SWE4-UV-047, SWE4-UV-048, SWE4-UV-049, SWE4-UV-051,
SWE4-UV-052, SWE5-IV-019.
2026-05-18 12:48:28 +08:00
EdisonJwa 7966a7c8c6 feat(bridge,android): BridgeEvent::PermissionState + JNI publish hook + c++_shared link
Per SDD-106 §5 add BridgeEvent::PermissionState{permission, state}
with the PermissionStateKind enum (Granted, Denied, PermanentlyDenied,
Unknown). The Kotlin side publishes mid-session permission changes
through a new JNI entry point Java_app_chanora_chanora_1flutter
_MainActivity_publishPermissionState routed by the new
permission_jni.rs module; the Rust audio engine subscribes and
authoritatively clamps the transmit gate (see SDD-106 §6).

Adds crates/chanora_bridge/build.rs to emit
cargo:rustc-link-lib=dylib=c++_shared on Android so libchanora_bridge
.so carries DT_NEEDED libc++_shared.so; this is required by Android
API 24+ per-library linker namespaces to resolve __cxa_pure_virtual
and friends at System.loadLibrary time.

Includes the FRB-regenerated Dart counterparts so each commit is
independently buildable.

Trace: SDD-105, SDD-106 §5, SDD-118 item 6 (extended).
2026-05-18 12:32:01 +08:00
EdisonJwa 56222d190e feat(audio): clamp transmit selector on RECORD_AUDIO permission state (SDD-106 §6)
Per SDD-106 §6 add a permission-state clamp to TransmitModeSelector.
When RECORD_AUDIO is Denied or PermanentlyDenied the transmit gate
is forced false regardless of PTT or voice-activity state; on
Granted the clamp releases and normal transmit decisions resume.
The clamp takes precedence over PTT and hard_mute in the decision
ordering documented inline.

Three new tests cover the clamp behavior, the release-on-grant
transition, and the non-RECORD_AUDIO ignore path.

Trace: SDD-106 §6, SRS-209.
2026-05-18 10:56:17 +08:00
EdisonJwa 78190c0694 feat(audio,android): wire engine + AudioManager JNI through ModeStack (SDD-108 §2)
Replace the prior one-shot android_engage_voice_communication call
with a ModeStack-mediated acquire/release pair. AudioEngine snapshots
the system audio mode on first acquire via android_get_audio_mode()
and restores it on last release via android_set_audio_mode(prior).
MODE_IN_COMMUNICATION (3) is engaged across the voice-session lifetime
per SDD-108.

Includes the Android AudioManager getMode/setMode JNI helpers
(placed in chanora_audio::engine alongside the existing JNI surface)
and the small ptt.rs touch needed for the SDD-108 ID-tag on the
existing tests.

Trace: SDD-108, SDD-115.
2026-05-18 10:53:18 +08:00
EdisonJwa 76c6d1d40c feat(audio,android): add MobileVoiceAudioBackend + AndroidVoiceUnit (oboe-rs)
Add the cross-platform MobileVoiceAudioBackend trait, plus the
Android implementation AndroidVoiceUnit backed by oboe-rs 0.6.x.
AndroidVoiceUnit owns AAudio stream setup with VoiceCommunication
usage/preset, performance-mode LowLatency request, sharing-mode
Exclusive best-effort, hardware AEC/NS/AGC engagement via JNI, and
the diagnostics snapshot publish path used by SDD-116 evidence
collection.

Cargo.toml: adds oboe = "0.6" under the Android target.

Trace: SDD-111, SDD-112, SDD-113, SRS-210, SRS-211, SRS-212, SRS-213,
SRS-214.
2026-05-18 10:38:19 +08:00
EdisonJwa da0b208075 feat(audio): add ModeStack pure refcount helper (SDD-108)
Introduce ModeStack, a pure-Rust refcount-composable wrapper for
Android audio-mode acquire/release with prior-mode snapshot. Per
SDD-108 §1/§2 the engine snapshots the system audio mode on first
acquire and restores it on last release; composed acquires are
no-ops while the mode is held.

ModeStack is panic-free; release-on-zero returns AlreadyReleased
rather than panicking. Six SWE4-UV-045-tagged unit tests cover the
acquire/release semantics on the host target.

Trace: SDD-108, SWE4-UV-045.
2026-05-18 10:30:25 +08:00
Edison Jwa dc9c5c0a4e feat: multi-platform bug fixes, Android audio path, and build tooling
Flutter UI fixes:
- Fix stale channel badge/speaker when moved by others (derive current
  channel from ownClientId instead of optimistic local state)
- Fix Linux PTT via focused fallback key handler
- Distinguish ServerQuery clients with terminal icon in client list
- Reduce duplicate current-channel badge display
- Prevent PTT key-bind save from permanently closing voice settings
- Fix Linux GTK reopen-after-close (quit app on window destroy)
- Fix focused PTT: consume key events, release held keys on
  disconnect/leave-channel/mode/backend changes, suppress stale errors

Flutter Rust bridge:
- Thread is_server_query flag through protocol→bridge→Dart
- Add own_client_id to BridgeSnapshot DTO
- Add log_file_path_str() for platform log path queries

Rust protocol:
- Add ServerQuery test coverage (query_client_type_maps_to_server_query_flag)
- Split reqwest TLS: native-tls for desktop/iOS, rustls for Android

Rust audio:
- Upgrade cpal 0.16→0.17.3 with API adjustments (SampleRate, description())
- Suppress Android-only dead-code warnings (open_log_file, keyring_account)

Android build tooling:
- tools/build-opus-android.sh: NDK auto-discovery, correct CMake
  Android variables (ANDROID_ABI, ANDROID_PLATFORM), portable baseline
- tools/build-android-rust.sh: build+copy Rust cdylib for arm64-v8a,
  armeabi-v7a, x86_64 into android/app/src/main/jniLibs/
- Add jniLibs/ to .gitignore

Rust bridge:
- Guard open_log_file() on non-Android (Android uses logcat)
2026-05-18 00:13:21 +09:00
Edison Jwa 7a59f5b9a1 feat(ios,p0): iOS P0 platform, audio fixes, channel UX 2026-05-17 22:00:00 +09:00
EdisonJwa a1fefc8ab6 fix(audio,ios): revert ring buffer back to direct fill_buffer call (rc.8+75)
The ring-buffer architecture (rc.8+73..+74) was making playback
strictly worse. Diagnostic data at +74 conclusively showed:

  * Producer task ran perfectly at 50 Hz (250 ticks per 5 s).
  * AudioHandler returned silence on 65-84% of fill_buffer calls
    even when window_peak_f32 reached 0.98 (full-scale audio).
  * Ring buffer never accumulated beyond 30 ms because consumer
    (VPIO render callback at 43.5 Hz, ~1440 samples per call)
    drained samples faster than the 50 Hz producer could push
    them, in net effect.

The producer drained AudioHandler at 50 Hz \u2014 slightly faster
than iOS VPIO actually consumes audio. Each fill_buffer call
asked for 20 ms but adjacent Opus packets hadn't arrived yet, so
fill_buffer returned mostly silence. Linux/SDL's same pattern
works because SDL calls fill_buffer at EXACTLY the device
callback rate (50 Hz = 20 ms per buffer); the rates match.

Fix: revert to direct fill_buffer call from the render callback
(the SDL pattern in tsclientlib's own reference example at
tsclientlib/examples/audio_utils/ts_to_audio.rs). The render
callback now:

  1. Resizes scratch_stereo Vec to 2 * num_frames f32 if needed
  2. Zeros the live slice (fill_buffer is additive, not clearing)
  3. Locks AudioHandler, calls fill_buffer(scratch_stereo)
  4. Downmixes L+R -> mono i16 with master gain into out[]
  5. Applies output_muted bypass
  6. Tracks peak_out + audio/silence ratios for diagnostic

The closure owns scratch_stereo across callbacks for stable
allocation. Same memory model as Linux/SDL.

Removed:
  * tokio::spawn producer task
  * rtrb dep + RingBuffer<i16> + Producer/Consumer split
  * tokio::sync::oneshot shutdown channel
  * producer_shutdown_tx field on IosVoiceUnit struct
  * RING_BUFFER_SAMPLES / PRODUCER_TICK_MS constants
  * Producer-side diagnostic counters

Diagnostic kept: cb / num_frames / frames_changes /
callbacks_with_audio / callbacks_with_silence / peak_out_i16 /
gain. Logged every 100 callbacks.

The choppy / clicks symptom is independent of the buffer
architecture \u2014 it's whatever AudioHandler is doing on iOS
that's different from Linux. Next investigation step is to
either (a) switch from VPIO to RemoteIO unit (lose Apple's
voice processing entirely), or (b) understand why AudioHandler
returns silence so often on iOS-arrival packet timing patterns.

Build counter 74 -> 75.
2026-05-17 13:05:09 +08:00
EdisonJwa a9aa19ecdd diag(audio,ios): comprehensive producer + consumer ring-buffer metrics (rc.8+74)
External reviewer correctly identified that the +73 ring-buffer
commit didn't fix the symptom but the architecture is still
right. We need to distinguish two possible causes:

  (a) Producer task isn't running (or running too rarely) so
      ring stays underfilled.
  (b) Producer IS running but fill_buffer returns zeros most of
      the time (AudioHandler stuck in buffering_samples state
      or no packets reaching it).

The +73 render-side diagnostic was insufficient: we logged
underruns + peak_out_i16 but not what the producer was
actually pushing. This commit adds producer-side metrics
rolled up every 5 s (250 ticks at 20 ms):

Producer task:
  producer_ticks           : timer firings (= ~250 per 5 s window;
                             fewer = tokio scheduler stalled)
  produced_chunks          : pushes into ring (= ticks - drops)
  fill_buffer_calls        : AudioHandler queries
  fill_buffer_zero_returns : ticks where scratch came back all
                             zeros (no decoded content to play)
  ring_full_drops          : ticks where ring was full and we
                             skipped the push
  ring_min/max_samples     : depth envelope across window
  ring_min/max_ms          : same in milliseconds
  window_peak_f32          : max scratch sample across window
  window_rms_f32           : RMS of all scratch samples across
                             window

Render callback (per-100-callback as before, plus new fields):
  ring_avail_before        : Consumer::slots() before this read
                             (= how many samples were sitting in
                              the ring at callback entry)
  read_frames              : samples successfully popped
  zero_filled              : samples zero-filled because ring
                             was empty (= num_frames - read_frames)
  underruns / underrun_samples / peak_out_i16 / clip_count_i16
                           : as before

Reading the next iteration's log:

  If producer_ticks << 250  per 5 s window:
      tokio scheduler isn't running the task fast enough.
      Move producer to its own dedicated runtime, or use
      std::thread + std::sync::mpsc + std::thread::sleep
      instead of tokio.

  If producer_ticks ~= 250 AND fill_buffer_zero_returns is
  high (most ticks return silence):
      AudioHandler isn't decoding packets fast enough OR is
      stuck buffering. Bug is upstream in protocol layer
      packet delivery or AudioHandler's jitter state machine.
      The ring buffer architecture cannot fix this.

  If producer_ticks ~= 250 AND fill_buffer_zero_returns is
  low AND ring_min_ms stays >100ms AND underruns are low BUT
  consumer's peak_out_i16 is still 0:
      Something is wrong between push and pop. Lock-free
      ring corruption, or wrong stride.

Pure diagnostic. No behavioural change beyond the logging.
Producer scratch envelope scan is O(scratch.len()) = 1920
samples per 20 ms tick = ~96k iterations/sec on the audio
producer thread \u2014 negligible CPU.

Build counter 73 -> 74.
2026-05-17 02:47:35 +08:00
EdisonJwa 99584fbc1a fix(audio,ios): decouple AudioHandler from VPIO render callback via ring buffer (rc.8+73)
User confirmed at +72 the symptom is 'voice + constant clicks +
choppy fragments'. The diagnostic data conclusively pointed to
iOS VPIO render-callback timing as the cause:

  * frames_changes=60+ per 100 callbacks at cb>=1800
    iOS keeps switching num_frames between 960 and 1104
    on roughly 60% of callbacks
  * peak_out_i16 is sensible (2500-16870, never clipping)
    when AudioHandler returns content
  * input_was_zero=true on most callbacks during active speech
    AudioHandler keeps entering buffering_samples state

The cause: previous render callback called fill_buffer
synchronously every iOS audio thread invocation. With iOS
calling at irregular rates with irregular sizes, AudioHandler's
jitter buffer (sized around 20 ms Opus frames) cannot satisfy
arbitrary-sized requests and falls back to returning silence
(&[] empty slice) on misaligned reads. The silent gaps in the
middle of the output buffer create discontinuities = audible
clicks; the missing-tail content produces choppy fragments.

Fix (architectural): decouple the AudioHandler decoder from the
VPIO render callback via a lock-free SPSC ring buffer.

  Producer (tokio task, 50 Hz):
    every 20 ms:
      fill_buffer(scratch_stereo_f32, 1920 = 20 ms stereo)
      downmix L+R -> mono i16 (960 samples)
      ring_buffer.push_slice(mono_i16)

  Consumer (VPIO render callback, iOS audio thread):
    every callback:
      pop num_frames samples from ring buffer into out
      zero-fill tail on underrun

Why it works:
  * Producer always asks AudioHandler for a stable 20 ms chunk
    (perfectly aligned with internal Opus frame size). No more
    buffering_samples false-triggers.
  * Consumer pulls whatever iOS asks for whenever iOS schedules
    it; ring buffer's 200 ms depth absorbs the callback jitter.
  * This is the standard pattern every production VoIP audio
    engine uses (WebRTC, Discord, FaceTime) to bridge bursty
    Opus decoders to bursty platform audio callbacks.

Implementation:
  * New dep: rtrb 0.3.4 (RustAudio realtime-safe SPSC ring
    buffer, 6.8M downloads, lock-free push/pop with no
    allocation on the audio thread).
  * RING_BUFFER_SAMPLES = 9600 (200 ms mono i16 at 48 kHz).
    Sized for 10x producer ticks of headroom.
  * PRODUCER_TICK_MS = 20 (matches Opus 50 Hz packet rate).
    set_missed_tick_behavior(Skip) to avoid burst catch-up on
    runtime stalls.
  * Producer task spawned in IosVoiceUnit::start, shutdown
    via tokio::oneshot when IosVoiceUnit drops.
  * Render callback is now just: pop into out, zero-fill tail,
    apply mute then gain.
  * Gain applied CONSUMER-side so user volume changes take
    effect within one callback (<= 200 ms latency).
  * Underrun diagnostics: count underrun callbacks + total
    zero-filled samples, log every 100 callbacks.

Threading + safety:
  * rtrb is lock-free SPSC. Audio thread never blocks.
  * Producer can block briefly on Arc<Mutex<AudioHandler>>
    contention with the inbound forwarder (handle_packet), but
    not with the audio thread.
  * Producer task is owned by tokio runtime; explicit shutdown
    channel ensures it exits when the engine stops.

Build verify:
  * Linux host: cargo check clean in 4.06s (downloads rtrb 0.3.4).
  * iOS Mac:   cargo check clean in 2.14s.

Build counter 72 -> 73.
2026-05-17 02:39:42 +08:00
EdisonJwa 53e09ea091 diag(audio,ios): comprehensive render-callback metrics per external review (rc.8+72)
External code review pushed back on the 'iPhone speaker hardware
distortion' hypothesis and pointed out we need more than just
peak measurements. The reviewer's checklist:

  * peak_i16
  * rms_i16
  * num_clipped_samples (abs >= 32767)
  * zero_fill_count / underrun_count
  * callback_frame_count variability
  * decoded_packet_duration_ms
  * input_was_zero
  * actual ASBD / actual sample rate

The format diagnostics at +71 already showed iOS honoured
48 kHz Int16 mono on both buses and that .default mode +
.defaultToSpeaker routed to the speaker correctly with
outputVolume=0.45. So format + route are confirmed correct.
The remaining mystery is WHY 'loud but distorted' \u2014 we need
sample-level metrics to isolate where in the pipeline the
breakage occurs.

This commit instruments the VPIO render callback with:

  * num_frames + frames_changes : detects iOS re-negotiating
                                 buffer size between callbacks
                                 (which would imply jitter the
                                 fixed scratch_stereo Vec can't
                                 absorb cleanly).
  * peak_stereo + rms_stereo  : characterises AudioHandler's
                                output BEFORE our downmix.
                                Distinguishes 'real audio
                                arriving' from 'silence'.
  * peak_out_i16 + clip_count : measures what we hand VPIO.
                                clip_count > 0 means we're
                                clipping at our boundary even
                                with gain=1.0 \u2014 indicates
                                upstream is over-driven.
  * input_was_zero            : explicit silence/no-talker
                                indicator separate from peak=0
                                which could mean tiny content
                                rounded to 0.

Reviewer's preferred diagnostic path is to dump PCM to file
and play with ffplay externally; that's iOS-impractical
without a shared filesystem path the user can extract via
Files.app. Instead we sample the same metrics in-callback at
~2 Hz which gives us the same information at run time.

Pure diagnostic. No behavioural change. Counters live in the
FnMut closure so the audio thread cost is one branch +
counter increment per callback, plus a one-pass RMS sum +
peak scan every 100 callbacks.

Build counter 71 -> 72.

Reviewer also recommended a headphone test in parallel \u2014
that will be done by the user (out-of-band) at the next
test cycle to determine whether the symptom changes when
audio leaves the speaker path.
2026-05-17 02:25:31 +08:00