docs/codebase-analysis-v2
28
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edison Jwa
|
2f6d45fb04 |
feat(audio): desktop Silero ONNX VAD + Windows PTT modernization + MSVC CRT build fix (#37)
* feat(audio): add Silero ONNX VAD with WebRTC fallback Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active. * feat(audio): integrate desktop VAD worker into capture engine Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding. Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default. Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self). * fix(audio): modernize Windows PTT to current windows-rs API Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate. * build(windows): force MSVC release CRT for audiopus cmake builds audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link. Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK. * build(flutter): update pubspec.lock after plugin additions Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature. * fix(audio): address PR #37 review findings Six fixes from independent PR review: 1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd globally broke non-Windows hosts because cmake-rs would try to execute a .cmd file on macOS/Linux. Instead, set CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY= MultiThreadedDLL as env vars that CMake reads natively. MSVC- specific vars are safely ignored by GCC/Clang toolchains. Delete the now-unnecessary wrapper script. 2. IMPORTANT: Join the Silero worker thread in Drop instead of detaching it. The old code dropped the JoinHandle which detaches the thread; the new code calls handle.join() after closing the channel, ensuring the ONNX session is cleaned up before the worker is replaced during config changes. 3. IMPORTANT: Single-try_lock refactor of the capture VAD callback. The double try_lock (policy probe + send) is replaced by a single scoped try_lock that both probes availability and sends the frame. The guard is dropped before the fallback path, which needs &mut self for mark_vad_fallback_active. This also eliminates the VadWorkerPolicy enum and callback_vad_worker_policy function, whose behavior is now inlined into the callback. 4. IMPORTANT: Remove tracing from the realtime capture callback. mark_vad_fallback_active and sync_vad_backend emitted info!/warn! from the audio thread. Replace with silent atomic state publishing via SharedAudioProcessingStats; the bridge stats stream already exposes vad_fallback_active for diagnostics. 5. IMPORTANT: Defer ONNX model load outside the worker mutex. apply_desktop_vad_backend_to_worker now constructs the new worker before taking the lock, then swaps it in under a short hold. This prevents the realtime callback from being blocked during model I/O + thread spawn. 6. MINOR: Remove unused VadBackend import from vad/mod.rs after deleting the policy code. * fix(audio): address PR #37 second-pass review findings 5-agent review found 5 blocking issues. All addressed: 1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc] and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows hosts are unaffected. 2. BLOCKER: processing_backend normalized in set_audio_processing_config on desktop (cfg-gated override to Noop), mirroring startup default. 3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config. Fixed misleading doc comment in core/lib.rs. 4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement. Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated. 5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android). Cargo.toml ort dependency target cfg narrowed similarly. 6. Realtime callback debt documented as TODO at CaptureState::ingest. * fix(audio): exclude ort dep on Android target ort does not provide first-class Android prebuilts in our pin, mirror the iOS/macOS exclusion so cargo metadata succeeds for android targets. * test(audio): fix stale select_ptt_backend import in ptt_privacy The helper moved out of the ptt_backends submodule onto the crate root; update the integration test imports so the test compiles again. * build(windows): scope MSVC release CRT cmake wrapper via Cargo [env] Cargo's [target.<triple>] table only forwards a fixed allowlist (linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there was silently dropped and audiopus_sys kept linking the debug CRT, producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors on x86_64-pc-windows-msvc test builds. Move the override to Cargo's [env] table using cc/cmake-rs's target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it applies to MSVC targets only and not to host tooling. Add stdout markers to the wrapper so its invocation is provable in cargo -vv logs. Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc --lib --no-run now links cleanly; CMakeCache.txt records CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW. * fix(flutter): gate VoiceActivity transmit mode by platform support VoiceActivity relies on the native VAD worker, which is only wired up on Windows, Linux, and Android. Showing the option on iOS, macOS, or web let users select a mode that silently never transmitted. Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers in voice_settings_controls.dart, hide the VAD row in voice_compact.dart and drop the VAD segment from the settings dialog when unsupported. Keep the legacy const transmitModeSegments for the existing widget test and add two new tests covering the gated helper. |
||
|
Edison Jwa
|
d59da05f93 |
refactor(audio): share AudioHandler between iOS and macOS, bump deps
crates/chanora_audio/src/engine.rs: drop the macOS-specific event-queue producer/consumer path; macOS now uses the iOS-style direct AudioHandler::fill_buffer in the VPIO render callback. The shared AudioHandler is an Arc<Mutex<...>>; the realtime callback uses try_lock so it never blocks on the tokio decode task (see ios_voice_unit.rs render callback). crates/chanora_audio/src/mobile_voice_backend.rs: update VoiceAudioParams cfg gates — handler is now the iOS/macOS/desktop shape (Arc<Mutex<AudioHandler<SessionAudioId>>>), event_producer is Android-only. crates/chanora_audio/src/lib.rs: widen the audio_event_queue module visibility to test so the macOS-specific path can be exercised by the unit test suite. Cargo.toml: bump cpal 0.17.3 -> 0.18.0, jni 0.21 -> 0.22.4, windows 0.54 -> 0.62, criterion 0.5 -> 0.8. Cargo.lock follows. |
||
|
Edison Jwa
|
5f1423c349 |
feat(voice): unified mobile voice bar with gesture-isolated PTT row (#22)
* feat(voice): unified mobile voice bar with gesture-isolated PTT row Replace separate VoiceStatusChip + VoicePttButton with a single CompactVoiceBar widget that combines both into a two-row layout: - Control row (tap): status text, mute, deafen, settings chevron - PTT row (hold): full-width hold-to-talk, shown only in PTT mode Gesture isolation prevents mis-touch between rows: the control row uses tap-only InkWell/IconButton while the PTT row uses a raw Listener for pointer-down/up events. Key changes: - Add CompactVoiceBar widget with state-colored container (normal, muted, talk-power-blocked) - Remove mute/deafen IconButtons from AppBar headerActions - Restructure voice details sheet into primary section + collapsible ExpansionTiles (audio processing, PTT capability, debug) - Optimistic state updates for mute/deafen to eliminate tap delay - Instant PTT visual feedback (no AnimatedContainer fade) - Constant geometry across all states (no layout shift on toggle) * fix(voice): preserve current PTT button format * feat(voice): move mute/deafen controls into VoiceStatusChip * fix(voice): ensure consistent chip height across mute states Remove isSelected/selectedIcon from IconButtons inside VoiceStatusChip. Material 3 toggle IconButtons (_SelectableIconButton) can vary in height when the selected state changes due to tap target sizing. Use simple conditional icons instead and set shrinkWrap tap target size with tight constraints for stable 40x40 buttons regardless of state. * fix(voice): remove leftover duplicate mute/deafen buttons in VoiceStatusChip * fix(voice): replace unsafe stereo cast with bytemuck and localise talk-power tooltip Replace the raw-pointer `&mut [(f32, f32)]` to `&mut [f32]` cast in the oboe output callback with `bytemuck::cast_slice_mut`, eliminating the unsafe block and relying on bytemuck compile-time NoUninit verification instead. Add voiceTalkPowerBlocked l10n key (en + zh) and replace the only remaining hard-coded English tooltip in VoiceStatusChip with it. |
||
|
Edison Jwa
|
2902a8bcd5 |
fix(audio): eliminate Android output stutter via Oboe config + lock-free callback (#20)
* fix(audio): eliminate Android output stutter via Oboe config + lock-free callback Phase 1 — Oboe configuration: - Change output stream from Usage::VoiceCommunication to Usage::Game with ContentType::Sonification to avoid forcing the Legacy (OpenSL ES) data path on most devices (Oboe issue #2075) - Switch output format from i16 Mono to f32 Stereo, matching Qint's proven configuration and eliminating per-callback downmix conversion - Set buffer size to 2x burst after stream open, reducing default buffer from 8-20x burst to 2x burst for lower latency - Remove scratch Mutex<Vec<f32>>; callback writes directly to Oboe buffer Phase 2 — Lock-free output callback: - Add audio_event_queue.rs: lock-free SPSC bridge using crossbeam ArrayQueue with separate packet (lossy) and control (reliable) channels - OutputCallback now owns AudioHandler directly (no Arc<Mutex<>> on Android) - Inbound forwarder pushes packets via AudioEventProducer (no mutex) - set_client_volume pushes control commands via event queue on Android - iOS/desktop Arc<Mutex<AudioHandler>> path unchanged * fix(audio): address PR #20 review findings - Store AudioEventConsumer directly in OutputCallback to eliminate per-callback Arc clone on the real-time audio thread - Add SAFETY comment for the unsafe from_raw_parts_mut transmute - Bound set_client_volume spin-loop to 64 retries with warn log - Remove redundant crossbeam-utils direct dependency - Regenerate license inventory for new crossbeam deps (CI fix) * fix(audio): use ASCII TODO punctuation |
||
|
Edison Jwa
|
6f7063971d | fix(audio): use WebRTC VAD on iOS | ||
|
Edison Jwa
|
fe6e07353e | chore: restore product scaffold to rollback baseline | ||
|
Edison Jwa
|
a2d686d9d0 | feat: promote linux native audio path | ||
|
Edison Jwa
|
5515ff6643 | feat: stabilize voice activity and audio routing | ||
|
Edison Jwa
|
7d5d8c2c90 | feat: integrate chat voice and diagnostics client | ||
|
Edison Jwa
|
bf284018e6 |
feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task
Audio engine (Rust): - Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD - Hardware effects (JNI) with software fallback per-effect - Render reference buffer for AEC between output/capture callbacks - Voice activity gate: suppress transmission when speaker muted (all platforms) - Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI - ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17) - VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix) - TEN VAD default backend (was Silero) - Platform→WebrtcApm resolution after hardware binding - oboe-rs edisonjwa fork with get_raw_session_id() Android Kotlin: - AndroidAudioFocusController + AndroidBluetoothScoController - AndroidAudioLifecycleController (route changes to Flutter) - ProGuard rules for new controllers Flutter UI: - VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM) - VoiceStatusChip: mute warning border + Speaker muted label - BBCode welcome message parser (BbCodeText, case-insensitive) - Welcome message foldable (expanded by default) - Link trust dialog (domain wildcards, SharedPreferences) - HapticFeedback on voice sheet opener - Server name in AppBar, version v0.1.0 - Default channel (id=1) visible, serverquery clients hidden - flutter_foreground_task integration Config: - ort load-dynamic on all non-iOS (Android/Linux/Windows) - ONNX Runtime AAR 1.26.0 - ndarray moved to common deps (was Apple-only) |
||
|
Edison Jwa
|
6af4ecab0f | feat(voice): add iOS VAD runtime support | ||
|
Edison Jwa
|
c87b47f064 | feat(audio): prefer native voice backends | ||
|
EdisonJwa
|
7188a5a69d |
feat(perf,benchmark-infra): criterion bench harness + advisory CI workflows (SDD-120)
Implementation of SDD-120 §1-§8: Bench harness (crates/chanora_audio/benches/): - common.rs: deterministic synthetic audio (440 Hz sine, no RNG). - realtime_capture.rs: bench_capture_alloc_count (dhat) + bench_capture_callback_wall_clock (criterion). - opus_codec.rs: bench_opus_encode_latency + bench_opus_decode_latency (direct audiopus, not AudioHandler — SDD-120 §3 item 4). - resampler.rs: bench_resampler_throughput across 44.1->48 / 16->48 / 48->48 passthrough. CI tooling (crates/chanora_audio/examples/): - emit_baseline.rs: aggregates criterion estimates.json outputs into the SRS-217 baseline schema. - compare_baseline.rs: applies SRS-219 tolerance, renders markdown table with 🟢/🟡/🔴 markers + yellow simpler-form realization per SDD-120 §8. Deviation from SDD-120 §2 / §5 / §7 placement: these tools live under examples/, not benches/ or src/bin/. Rationale: they must consume serde_json (a dev-only dep — production builds must not pull it). Cargo only resolves dev-dependencies for [[test]], [[bench]], and [[example]] targets; [[bin]] targets under src/bin/ see only regular [dependencies]. examples/ keeps the binaries out of the production dep tree while still giving them cargo run --example invocation. An SDD-120 amendment should reflect this. Workflows (.github/workflows/): - bench-advisory.yml: PR + push triggers; runs benches; posts a sticky PR comment via actions/github-script@v7; job status is always success (SRS-218 clause 4 — non-blocking). - bench-baseline-update.yml: workflow_dispatch only; runs benches; opens PR via peter-evans/create-pull-request@v6 (sole writer of the SAD-089 baseline JSON). Cargo.toml additions ([dev-dependencies] only — verified excluded from --release builds): criterion 0.5, dhat 0.3, serde_json 1. Source-code seam: minimal pub-but-#[doc(hidden)] bench_seam module in chanora_audio (engine.rs + lib.rs re-export) so the criterion bench harness can construct a CaptureState and drive CaptureState::ingest without re-implementing the engine (SDD-120 §3). Non-iOS targets only — CaptureState itself is iOS-gated. Initial baseline seed: crates/chanora_audio/benches/baselines/ x86_64-unknown-linux-gnu.json = {}. compare_baseline handles the missing-baseline case gracefully and emits a 'no red markers' report; the first manual dispatch of bench-baseline-update.yml after merge establishes the real values. Out of scope per SDD-120 §10: production telemetry export, build-failing hard CI gate, multi-host benchmarking, IDE integration, Dart-side bridge round-trip bench. Verification: - cargo check --workspace --all-targets: PASS. - cargo bench --bench realtime_capture --no-run: PASS. - cargo bench --bench opus_codec --no-run: PASS. - cargo bench --bench resampler --no-run: PASS. - cargo build --example emit_baseline --example compare_baseline -p chanora_audio: PASS. - cargo test --workspace: 106 passed, 0 failed, 3 ignored — no regression from prior count. |
||
|
EdisonJwa
|
76c6d1d40c |
feat(audio,android): add MobileVoiceAudioBackend + AndroidVoiceUnit (oboe-rs)
Add the cross-platform MobileVoiceAudioBackend trait, plus the Android implementation AndroidVoiceUnit backed by oboe-rs 0.6.x. AndroidVoiceUnit owns AAudio stream setup with VoiceCommunication usage/preset, performance-mode LowLatency request, sharing-mode Exclusive best-effort, hardware AEC/NS/AGC engagement via JNI, and the diagnostics snapshot publish path used by SDD-116 evidence collection. Cargo.toml: adds oboe = "0.6" under the Android target. Trace: SDD-111, SDD-112, SDD-113, SRS-210, SRS-211, SRS-212, SRS-213, SRS-214. |
||
|
Edison Jwa
|
dc9c5c0a4e |
feat: multi-platform bug fixes, Android audio path, and build tooling
Flutter UI fixes: - Fix stale channel badge/speaker when moved by others (derive current channel from ownClientId instead of optimistic local state) - Fix Linux PTT via focused fallback key handler - Distinguish ServerQuery clients with terminal icon in client list - Reduce duplicate current-channel badge display - Prevent PTT key-bind save from permanently closing voice settings - Fix Linux GTK reopen-after-close (quit app on window destroy) - Fix focused PTT: consume key events, release held keys on disconnect/leave-channel/mode/backend changes, suppress stale errors Flutter Rust bridge: - Thread is_server_query flag through protocol→bridge→Dart - Add own_client_id to BridgeSnapshot DTO - Add log_file_path_str() for platform log path queries Rust protocol: - Add ServerQuery test coverage (query_client_type_maps_to_server_query_flag) - Split reqwest TLS: native-tls for desktop/iOS, rustls for Android Rust audio: - Upgrade cpal 0.16→0.17.3 with API adjustments (SampleRate, description()) - Suppress Android-only dead-code warnings (open_log_file, keyring_account) Android build tooling: - tools/build-opus-android.sh: NDK auto-discovery, correct CMake Android variables (ANDROID_ABI, ANDROID_PLATFORM), portable baseline - tools/build-android-rust.sh: build+copy Rust cdylib for arm64-v8a, armeabi-v7a, x86_64 into android/app/src/main/jniLibs/ - Add jniLibs/ to .gitignore Rust bridge: - Guard open_log_file() on non-Android (Android uses logcat) |
||
|
EdisonJwa
|
a1fefc8ab6 |
fix(audio,ios): revert ring buffer back to direct fill_buffer call (rc.8+75)
The ring-buffer architecture (rc.8+73..+74) was making playback
strictly worse. Diagnostic data at +74 conclusively showed:
* Producer task ran perfectly at 50 Hz (250 ticks per 5 s).
* AudioHandler returned silence on 65-84% of fill_buffer calls
even when window_peak_f32 reached 0.98 (full-scale audio).
* Ring buffer never accumulated beyond 30 ms because consumer
(VPIO render callback at 43.5 Hz, ~1440 samples per call)
drained samples faster than the 50 Hz producer could push
them, in net effect.
The producer drained AudioHandler at 50 Hz \u2014 slightly faster
than iOS VPIO actually consumes audio. Each fill_buffer call
asked for 20 ms but adjacent Opus packets hadn't arrived yet, so
fill_buffer returned mostly silence. Linux/SDL's same pattern
works because SDL calls fill_buffer at EXACTLY the device
callback rate (50 Hz = 20 ms per buffer); the rates match.
Fix: revert to direct fill_buffer call from the render callback
(the SDL pattern in tsclientlib's own reference example at
tsclientlib/examples/audio_utils/ts_to_audio.rs). The render
callback now:
1. Resizes scratch_stereo Vec to 2 * num_frames f32 if needed
2. Zeros the live slice (fill_buffer is additive, not clearing)
3. Locks AudioHandler, calls fill_buffer(scratch_stereo)
4. Downmixes L+R -> mono i16 with master gain into out[]
5. Applies output_muted bypass
6. Tracks peak_out + audio/silence ratios for diagnostic
The closure owns scratch_stereo across callbacks for stable
allocation. Same memory model as Linux/SDL.
Removed:
* tokio::spawn producer task
* rtrb dep + RingBuffer<i16> + Producer/Consumer split
* tokio::sync::oneshot shutdown channel
* producer_shutdown_tx field on IosVoiceUnit struct
* RING_BUFFER_SAMPLES / PRODUCER_TICK_MS constants
* Producer-side diagnostic counters
Diagnostic kept: cb / num_frames / frames_changes /
callbacks_with_audio / callbacks_with_silence / peak_out_i16 /
gain. Logged every 100 callbacks.
The choppy / clicks symptom is independent of the buffer
architecture \u2014 it's whatever AudioHandler is doing on iOS
that's different from Linux. Next investigation step is to
either (a) switch from VPIO to RemoteIO unit (lose Apple's
voice processing entirely), or (b) understand why AudioHandler
returns silence so often on iOS-arrival packet timing patterns.
Build counter 74 -> 75.
|
||
|
EdisonJwa
|
99584fbc1a |
fix(audio,ios): decouple AudioHandler from VPIO render callback via ring buffer (rc.8+73)
User confirmed at +72 the symptom is 'voice + constant clicks +
choppy fragments'. The diagnostic data conclusively pointed to
iOS VPIO render-callback timing as the cause:
* frames_changes=60+ per 100 callbacks at cb>=1800
iOS keeps switching num_frames between 960 and 1104
on roughly 60% of callbacks
* peak_out_i16 is sensible (2500-16870, never clipping)
when AudioHandler returns content
* input_was_zero=true on most callbacks during active speech
AudioHandler keeps entering buffering_samples state
The cause: previous render callback called fill_buffer
synchronously every iOS audio thread invocation. With iOS
calling at irregular rates with irregular sizes, AudioHandler's
jitter buffer (sized around 20 ms Opus frames) cannot satisfy
arbitrary-sized requests and falls back to returning silence
(&[] empty slice) on misaligned reads. The silent gaps in the
middle of the output buffer create discontinuities = audible
clicks; the missing-tail content produces choppy fragments.
Fix (architectural): decouple the AudioHandler decoder from the
VPIO render callback via a lock-free SPSC ring buffer.
Producer (tokio task, 50 Hz):
every 20 ms:
fill_buffer(scratch_stereo_f32, 1920 = 20 ms stereo)
downmix L+R -> mono i16 (960 samples)
ring_buffer.push_slice(mono_i16)
Consumer (VPIO render callback, iOS audio thread):
every callback:
pop num_frames samples from ring buffer into out
zero-fill tail on underrun
Why it works:
* Producer always asks AudioHandler for a stable 20 ms chunk
(perfectly aligned with internal Opus frame size). No more
buffering_samples false-triggers.
* Consumer pulls whatever iOS asks for whenever iOS schedules
it; ring buffer's 200 ms depth absorbs the callback jitter.
* This is the standard pattern every production VoIP audio
engine uses (WebRTC, Discord, FaceTime) to bridge bursty
Opus decoders to bursty platform audio callbacks.
Implementation:
* New dep: rtrb 0.3.4 (RustAudio realtime-safe SPSC ring
buffer, 6.8M downloads, lock-free push/pop with no
allocation on the audio thread).
* RING_BUFFER_SAMPLES = 9600 (200 ms mono i16 at 48 kHz).
Sized for 10x producer ticks of headroom.
* PRODUCER_TICK_MS = 20 (matches Opus 50 Hz packet rate).
set_missed_tick_behavior(Skip) to avoid burst catch-up on
runtime stalls.
* Producer task spawned in IosVoiceUnit::start, shutdown
via tokio::oneshot when IosVoiceUnit drops.
* Render callback is now just: pop into out, zero-fill tail,
apply mute then gain.
* Gain applied CONSUMER-side so user volume changes take
effect within one callback (<= 200 ms latency).
* Underrun diagnostics: count underrun callbacks + total
zero-filled samples, log every 100 callbacks.
Threading + safety:
* rtrb is lock-free SPSC. Audio thread never blocks.
* Producer can block briefly on Arc<Mutex<AudioHandler>>
contention with the inbound forwarder (handle_packet), but
not with the audio thread.
* Producer task is owned by tokio runtime; explicit shutdown
channel ensures it exits when the engine stops.
Build verify:
* Linux host: cargo check clean in 4.06s (downloads rtrb 0.3.4).
* iOS Mac: cargo check clean in 2.14s.
Build counter 72 -> 73.
|
||
|
EdisonJwa
|
af686ca7a6 |
feat(audio,ios): add coreaudio-rs dep + IosVoiceUnit skeleton (commit 1/5)
Skeleton scaffolding for the iOS VoiceProcessingIO backend that
will replace cpal on iOS. This commit lands the dependency + the
module + a constructable AudioUnit that emits silence and drops
input; nothing in engine.rs is wired up yet (that is commit 2).
Compilation contract for this commit:
* Linux / Windows / macOS / Android builds unaffected (the new
module is target_os='ios' gated, the new dep is in
'[target."cfg(target_os = \"ios\")"]').
* iOS build pulls in coreaudio-rs 0.14, constructs a VPIO unit,
pins stream format to 48 kHz Int16 mono on both buses, installs
no-op input + silence-emitting render callbacks, initializes,
and starts. No audio is actually moved until commits 3/4.
Why VPIO and not RemoteIO via cpal: cpal's iOS backend opens
RemoteIO with no control over stream format / buffer size /
channels and produces a mono-only output element that stays bound
to the route present at construction time. End-user symptom on
iPhone 16 Pro iOS 18.7.8: tapping Speaker in the picker flips
AVAudioSession.currentRoute.outputs to Speaker (confirmed in our
diagnostic logs from commit
|
||
|
EdisonJwa
|
f0ddb160a0 |
fix(protocol,audio,ios): native-tls instead of rustls+aws-lc-rs
Building the bridge for `aarch64-apple-ios` failed in two ways with
the previous TLS stack:
1. `aws-lc-sys` (transitive: rustls -> aws-lc-rs -> aws-lc-sys)
does not cross-compile cleanly to iOS — the build produced
undefined symbols for architecture arm64 (mldsa44, ec_GFp_mont,
etc).
2. `audiopus_sys` linked against the wrong iOS runtime version,
missing `___chkstk_darwin`.
Following the rustls-platform-verifier docs and the standard Rust+
iOS+TLS pattern used by 1Password / Signal / rustup / Bitwarden,
this commit swaps the TLS provider to **native-tls** so each
platform picks its own:
* macOS + iOS -> Security.framework (no external C deps)
* Windows -> SChannel
* Linux/BSD -> system OpenSSL
Changes:
crates/chanora_protocol/Cargo.toml
crates/chanora_audio/Cargo.toml
* Drop `default-tls` from tsclientlib's features. The remaining
`audio` feature is what we actually use; default-tls was a
reqwest convenience that picked rustls+aws-lc-rs.
* Add a direct `reqwest` dep with `default-features = false,
features = ["charset", "http2", "native-tls"]`. Cargo's
workspace feature unification carries this through the
transitive `tsclientlib -> reqwest` chain.
apps/chanora_flutter/ios/Podfile
* Uncomment `platform :ios, '13.0'` so CocoaPods stops emitting
the implicit-platform warning and Xcode's iOS deployment-
target check is honored.
apps/chanora_flutter/ios/Podfile.lock
* Generated by `pod install` after the platform pin. Committed so
iOS builds on other developer machines pull the exact same Pod
versions.
apps/chanora_flutter/ios/Runner.xcodeproj/project.pbxproj
apps/chanora_flutter/ios/Runner.xcworkspace/contents.xcworkspacedata
* CocoaPods auto-integration: adds Pods_Runner.framework +
Pods_RunnerTests.framework references and the Pods xcconfig
file references. Standard `pod install` output; reviewing the
diff shows only Pod-bookkeeping additions, no signing or
target-config drift.
Verified end-to-end on the M1 Mac (coder@100.118.130.73):
cargo build --release -p chanora_bridge 29.09 s
cargo build --release --target aarch64-apple-ios 24.32 s
(with IPHONEOS_DEPLOYMENT_TARGET=13.0 and
CMAKE_POLICY_VERSION_MINIMUM=3.5 in the env to satisfy the
audiopus_sys cmake invocation; documented as a P1 build-glue
follow-up.)
flutter build ios --release --no-codesign ok
Built build/ios/iphoneos/Runner.app (16.9 MB)
Xcode GUI build of Runner.xcworkspace ok
(after the user opened Runner.xcworkspace, NOT
Runner.xcodeproj, and Clean Build Folder.)
Tests on macOS unchanged: chanora_audio 34 / 0 / 0.
DEC-025: iOS + macOS officially in scope for P0.
|
||
|
EdisonJwa
|
d9c330c23b |
feat(audio,linux): output via SDL2; cpal stays on Windows/macOS
User reported persistent crackling/popping from peer audio on Linux even
after fixing the 48k->device-rate resampler boundary discontinuities,
clamping pre-Opus-encode peaks, and pre-allocating the playback scratch
buffer. Logs confirmed cpal opened raw ALSA at 44.1k native, no callback
budget violations, no underrun warnings -- yet the audio was still poor.
Root cause: cpal on Linux opens raw ALSA's 'default' PCM. On modern
PipeWire / pipewire-alsa boxes that virtual device routes through ALSA's
dmix + plug layers, whose default resampler is nearest-neighbour. cpal
also picks a small default period size (~256 frames / 5.8 ms) leaving no
headroom for kernel scheduler jitter. Both effects compound into the
crackling the user heard.
Upstream tsclientlib's own audio example
(tsclientlib/examples/audio_utils/ts_to_audio.rs) and the official Qint
client both use SDL2 with AudioSpecDesired { freq: 48000, channels: 2,
samples: 960 }. SDL2 on the same systems routes through PipeWire's PA
bridge (or PulseAudio directly), both carrying high-quality resamplers.
Fix:
* Add sdl2 = '0.37' as a target_os=linux dependency. Links libSDL2-2.0
.so (Arch sdl2-compat over SDL3, Debian libsdl2-2.0-0, Fedora SDL2).
* New module crates/chanora_audio/src/sdl_output.rs implementing
SdlOutput: opens a 48 kHz stereo 960-frame callback that zeroes the
buffer and calls AudioHandler::fill_buffer directly (no user-side
resampler). Master gain + hard-mute atomics wired in identically to
the cpal callback so set_output_gain / set_output_muted keep working.
* engine.rs cfg-gated: target_os='linux' builds SdlOutput; everywhere
else continues with the cpal output path (including the device-native-
rate negotiation and resampler-continuity fixes shipped earlier --
those remain correct on Windows/macOS where cpal targets WASAPI /
CoreAudio cleanly).
* The cpal output helpers (build_output_stream, PlaybackResampleState,
FromF32) are now cfg(not(target_os='linux'))-gated so the Linux
build doesn't emit dead-code warnings.
Capture path still cpal on every platform -- outbound audio was not
reported as bad. Resampler-continuity fix on the capture side stays:
microphone -> Opus encoder still goes through the linear interpolator
with the last-sample anchor.
Tests: 32 / 0 / 0 (chanora_audio), workspace 78 / 0 / 1 unchanged.
|
||
|
EdisonJwa
|
21945979a3 |
test(audio,ptt): comprehensive Windows P0 unit-test suite (L0-L11)
Layered test coverage for the Windows PTT subsystem ahead of the
v1.0.0-rc.8 official release sign-off.
L0 (refactor)
- Extract three pure-logic dispatchers from the existing WndProc /
LowLevelKeyboardProc / LowLevelMouseProc bodies in
crates/chanora_audio/src/ptt_backends/windows.rs:
dispatch_raw_input(ctx, &RAWINPUT)
dispatch_hook_keyboard(ctx, wparam, &KBDLLHOOKSTRUCT)
dispatch_hook_mouse(ctx, wparam, &MSLLHOOKSTRUCT)
Each takes a small Context (AtomicBinding + AudioTransmitGate +
flags) and is callable without spinning up any Win32 plumbing.
The real Win32 procs unchanged structurally; they unpack lparam
and forward to the dispatchers. AtomicBinding / RawInputContext
/ HookContext / resolve_binding are now pub(crate) so the
in-file test module can drive them.
L1 — windows_keymap full-table sweep (+13 tests)
Every key_label_to_vk arm, all A-Z + a-z, all 0-9, F1-F20,
navigation, modifiers, OEM punctuation, numpad. Exhaustive
mouse_label_to_button cases including the 0x08 / 0x10 /
unknown-bitmask fallbacks.
L2 — AtomicBinding lock-free correctness
store/read round-trip, clear(), Default = zeros, single-writer
/ single-reader concurrency, many-readers / single-writer.
L3 — resolve_binding dispatcher tests
All PttInputClass variants, well-known labels, unknown-label
fallback, mismatched class+label rejection, mouse bitmask
resolution.
L4 — Backend state-machine
Both WindowsRawInputBackend and WindowsHookBackend:
descriptor() pre-arm vs post-arm (L0Focused -> L2/L3), start()
with None binding rejection, rebind() in-place, stop()
clears + idempotent, stop() after stop() no-op.
L5 — dispatch_raw_input table
Keyboard match/non-match, key-down/key-up via Flags & 0x01,
no-binding short-circuit, mouse XBUTTON1/XBUTTON2 down/up
matching the bound button, unhandled HID type. RAWINPUT structs
built via mem::zeroed plus field-fill, owning the unsafe in
the test layer where it belongs.
L6 — dispatch_hook_keyboard + dispatch_hook_mouse
WM_KEYDOWN / WM_KEYUP / WM_SYSKEYDOWN / WM_SYSKEYUP for the
keyboard path, WM_XBUTTONDOWN / WM_XBUTTONUP for the mouse
path. Same shape as L5.
L7 — Privacy invariant (crates/chanora_audio/tests/ptt_privacy.rs)
New cross-platform integration test installs a custom
tracing_subscriber Layer that records every emitted event's
target + field names. Exercises the public PTT API plus (on
Windows) the backend factory. Asserts no field name in the
banned list (vk, scan_code, keysym, key_label, bound_key,
binding, platform_key, VKey, wVk, wScan, kbflags, mouseflags)
is ever emitted and every field belongs to the DEC-027
allow-list. Adds tracing-subscriber as a dev-dependency on
chanora_audio.
L8 — Full-chain integration in core/chanora_core/src/ptt.rs
Windows-only mod windows_full_chain_tests:
zero-tail full chain (synchronous)
default-tail full chain (200 ms wait then off)
mid-press rebind abandons in-flight press
L9/L10/L11 — tools/windows-smoke.cmd + tools/windows-smoke.md
Batch smoke script + operator doc. cargo build, flutter build,
artifact existence + size checks, headless launch with stderr
capture, bridge-initialised log assertion. Distinct exit codes
per failure step. Doc explains invocation + common failure
modes.
Verification (Linux)
- cargo check --workspace: clean.
- cargo test --workspace: 78 passed / 0 failed / 3 ignored.
76 cross-platform unit tests (unchanged) plus the new
ptt_privacy integration test plus one new ignored portal smoke
test.
The Windows-gated tests (~49 new) compile and run on the Korean
Windows 11 host where they belong; cross-compile from Linux is
not configured locally. The smoke script is the production
acceptance gate for rc.8 on Windows.
Deviations from the original plan are minor (single ignored
real-runtime test rather than per-platform attribute, L7 uses
public API rather than pub(crate) dispatchers, dispatchers live
inside windows.rs rather than a sibling module) and documented
in the subagent report.
|
||
|
EdisonJwa
|
f9d20d8585 |
fix(audio,windows): adapt Raw Input + hook backends to windows-rs 0.54 API shape
Initial Task C commit (
|
||
|
EdisonJwa
|
77c2a1def4 |
feat(audio,windows): real Raw Input + low-level hook global PTT (SDD-083 / SDD-084)
The v1.0.0-rc.7 Windows backends were thread::sleep stubs that
reported optimistic L2GlobalHoldToTalk / L3GlobalWithMouseButtons
descriptors without actually registering for any global key events.
Surfaced on Windows verification as:
* 'even press to talk key was set, still only the hold to talk
button is work for talk'
* 'and displayed as L2GlobalHoldToTalk(raw-input)'
* 'cannot continuous transmission'
This commit implements the real backends:
WindowsRawInputBackend (preferred Windows rung, SDD-083):
* Hidden message-only window via
CreateWindowExW(..., HWND_MESSAGE, ...).
* RegisterRawInputDevices with RIDEV_INPUTSINK on
Usage Page 0x01 / Usage 0x06 (keyboard) and 0x02 (mouse) so
events fire globally — including when Chanora is unfocused.
* WndProc handling WM_INPUT: GetRawInputData ->
keyboard.VKey vs bound vk, or mouse.usButtonFlags vs bound
side-button index. Down -> gate.set(true); up -> gate.set(false).
* Dedicated chanora-rawinput thread runs GetMessageW /
TranslateMessage / DispatchMessageW until stop() posts
WM_QUIT via PostThreadMessageW.
WindowsHookBackend (fallback rung, SDD-084):
* SetWindowsHookExW(WH_KEYBOARD_LL) + WH_MOUSE_LL on a
dedicated chanora-llhook thread.
* Hook procs translate KBDLLHOOKSTRUCT.vkCode and
MSLLHOOKSTRUCT.mouseData against the same shared
AtomicBinding.
* UnhookWindowsHookEx on teardown.
Both backends:
* Honest descriptor() reporting: backends start reporting
L0Focused; level upgrades to L2 / L3 only after a real
arming success (RegisterRawInputDevices or SetWindowsHookEx
returning Ok). This fixes the 'L2 reported but doesn't fire'
complaint by making the badge tell the truth — if Raw Input
registration fails at runtime the user sees the L0Focused
info-icon explanation sheet instead of being told L2 works.
* AtomicBinding (class / vk / mouse_btn) for lock-free hot
path. Translation lives in
crates/chanora_audio/src/ptt_backends/windows_keymap.rs
which maps Flutter LogicalKeyboardKey.keyLabel strings
(e.g. 'Space', 'F10', 'A') to Win32 VK_* codes; mouse
side-button bitmask strings ('mouse-side-button:8' /
':16') to RawInput button indices (4 / 5).
* Per-thread context (thread_local RefCell) carries the
gate + binding to the WndProc / hook proc without needing
raw-pointer user-data plumbing.
Diagnostic logging:
* AudioEngine::start now logs default_input_config and
default_output_config explicitly with the channels /
sample_rate / sample_format that cpal reports, so a
build_*_stream failure on locale-specific Windows hosts
(reported on ko-KR Windows 11 as 'Start Audio Button not
work') becomes diagnosable from the stderr log alone.
* build_output_stream surfaces the requested config in the
tracing::error! record on failure.
Privacy (DEC-027 / SDD-090): the windows.rs and
windows_keymap.rs hot paths NEVER log raw VKs, scan codes,
keysyms, key labels, or button identifiers. Only the
platform-neutral input class ('keyboard' /
'mouse-side-button') and the backend id appear in the tracing
stream. The SDD-090 PttSanitizer Layer is the defence-in-depth
net but this code does not rely on it.
Tests: 4 new windows-only unit tests in windows_keymap (ASCII
letters / digits / Space + Fn / unknown / mouse button index).
They compile only under cfg(target_os = "windows") so the
Linux workspace test count is unchanged at 59/0/3.
Cargo deps: adds windows = '0.54' (target_os = windows) with
the feature set needed for RawInput + hooks. 0.54 matches
the version already transitive through the workspace.
Verified on Linux: cargo check --workspace clean, cargo test
--workspace 59/0/3 (windows-gated tests skip on Linux). The
real exercise of this commit will happen on the Korean Windows
11 host (100.84.219.45) at the next build.
|
||
|
EdisonJwa
|
82d012a46b |
feat(ptt): live Linux GNOME-Wayland portal session flow (DEC-025)
Promotes the Linux backend from probe-only to a live
`org.freedesktop.portal.GlobalShortcuts` session, closing the
gen2 v0.9.3 baseline's last Linux-side code item. Both gaps I
flagged on the review pass are addressed:
* Stop now closes the portal session through the dedicated
`org.freedesktop.portal.Session` interface (not the
request-cancel `Request` interface — that would only abort a
pending Request, not release the bound shortcuts).
* Ten new unit tests cover `classify_shortcuts_value`,
`publish_bound`, `publish_l0`, and the `SHORTCUT_ID` stability
contract using synthesised `OwnedValue` payloads. Live D-Bus
coverage stays in the `linux_portal_smoke` ignored
integration test (RR-PTT-004).
Live session lifecycle (gen2 Q5b — lazy, single backend instance):
1. `start(gate, binding)` spawns one `tokio::spawn` worker that
owns an async `zbus::Connection` (sharing the bridge's
tokio runtime per Q4a).
2. `CreateSession` with fresh random `handle_token` /
`session_handle_token` tokens. The worker awaits the portal
`Response` signal via a `RequestProxy` subscription and
extracts `session_handle` from the results dict.
3. `BindShortcuts(session_handle, [("chanora-ptt", { description
= "Chanora push-to-talk" })], "", {})`. The portal opens its
own system-managed dialog asking the user to choose a key
— Chanora itself never reads raw key events. The audio
engine continues at `L0Focused` while the dialog is open;
the descriptor watch publishes the transition once the
portal returns.
4. On `response_code == 0`: classify the `trigger_description`
substring (heuristic: contains "mouse" -> MouseSideButton,
else Keyboard), publish `L2GlobalHoldToTalk` (or `L3` for
mouse) through the watch sender. The raw trigger_description
string is never logged (DEC-027 / SRS-202).
5. On `response_code == 1` (cancelled) or `>= 2` (failure):
publish `L0Focused` through the watch sender. The user can
retry via the UI "Configure" button (gen2 Q6a).
6. The worker enters a `tokio::select!` loop multiplexing the
`cmd_rx` channel (Rebind / Stop) and the `Activated` /
`Deactivated` signals. Matching signals scoped to this
session handle and `chanora-ptt` shortcut id drive
`gate.set(true/false)`.
7. `Rebind` re-runs `BindShortcuts` on the same session.
8. `Stop` calls `org.freedesktop.portal.Session.Close()` on
the session-handle object path, clears the gate, exits.
UX (gen2 Q3a): when `_pttBackendId == 'gnome-wayland-portal'`,
the Flutter "Configure" button skips the in-app
`_PttBindingCaptureDialog` and shows a SnackBar telling the user
their desktop environment will open its own shortcut dialog.
The button delegates to `setPttBinding(keyboard, "portal")`
which nudges the backend; the portal handles the rest. New ARB
key `pttConfigurePortalRedirect` in en + zh-Hans.
Trait surface (cross-cutting):
* `DesktopPttBackend::descriptor_watch()` is a new trait method
with a default impl returning a never-firing receiver.
Backends with async capability transitions (only the Linux
portal backend today) override it to return the live watch
sender's receiver.
* `chanora_core::ChanoraSession::start_audio` subscribes to the
active backend's `descriptor_watch()` and spawns a forwarder
task that re-emits `SessionEvent::PttCapability` on every
transition. The initial value is emitted synchronously.
`Cargo.toml` (Linux-only):
* `futures-util` (std features, no executor) for stream
consumption on the portal signal subscriptions.
* `rand 0.8` for fresh per-process portal tokens.
* `zbus` continues at v5 with the `tokio` + `blocking-api`
features.
Tests
-----
* `chanora_audio` rises from 8 to 18 unit tests. New
coverage on the Linux module:
- `classify_returns_none_when_shortcut_id_missing`
- `classify_returns_keyboard_for_typical_trigger_description`
- `classify_returns_keyboard_when_trigger_description_missing`
- `classify_detects_mouse_substring`
- `classify_is_case_insensitive_on_mouse_substring`
- `publish_bound_keyboard_publishes_L2_with_keyboard_class`
- `publish_bound_mouse_publishes_L3`
- `publish_bound_none_publishes_L2_keyboard_default`
- `publish_l0_clears_descriptor`
- `shortcut_id_is_stable`
* Workspace total: 67 unit + integration tests, all green with
`CHANORA_DISABLE_KEYRING=1` (was 57 at v1.0.0-rc.4).
* New `crates/chanora_audio/tests/linux_portal_smoke.rs`
ignored integration test (RR-PTT-004 evidence path). Run on
a GNOME-on-Wayland host with
`cargo test -p chanora_audio --test linux_portal_smoke -- --ignored --nocapture`.
Documentation
-------------
* `docs/architecture/desktop-ptt-architecture.md` §5.3 rewritten
to describe the realised lifecycle; v0.9.4 change-history
entry added.
* `docs/governance/product-decision-register.md` v0.9.10
change-history entry recording the code-side promotion. No
decision rows mutate.
* `docs/release/release-readiness-go-nogo-record.md` RR-PTT-004
flipped from `Open` to `Implemented (live trace pending)`;
v0.9.5 change-history entry.
Verification
------------
* `cargo test --workspace`: 67/67 green.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: zero new warnings.
* `tools/dump_flutter_licenses.sh`: 94 packages, 0 without
LICENSE.
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` +
`flutter build linux --release`: clean Linux x86_64 bundle.
* Live portal trace (RR-PTT-004) — **not run**. The dev shell
is a TTY without a Wayland session. The user will run the
ignored smoke test from inside a GNOME-on-Wayland session
when available.
No Windows / macOS / iOS live verification in this commit (hosts
unavailable). The Windows + macOS backend scaffolds remain in
place reporting their target capability honestly; live OS-call
wiring is queued for their respective platform owners'
reference hosts per `docs/governance/staged-release-plan.md`.
|
||
|
EdisonJwa
|
5199e3d005 |
feat(ptt): full desktop backend ladder + missed-key-up watchdog (gen2 v0.9.3 follow-up)
Lands SDD-081..088 + SDD-092 implementations on top of v1.0.0-rc.3.
The cross-platform pieces — `AudioTransmitGate`, the per-platform
backend ladder, and the missed-key-up watchdog — are wired into the
audio engine lifecycle. Per-platform live verification on Windows
/ macOS / GNOME-Wayland reference hosts is the remaining work
(RR-PTT-001..006/008 in `release-readiness-go-nogo-record.md`).
`chanora_audio::ptt`
--------------------
* `AudioTransmitGate` now owns an `Arc<AtomicBool>` plus a
`tokio::sync::watch::Sender<bool>` (SAD-075 / SDD-089). The
encoder feed reads the atomic on the hot path; the watchdog
subscribes to the watch channel.
* `MissedKeyUpWatchdog::spawn(gate, timeout)` watches the gate
transitions and self-clears `transmit_active` if the
`false -> true` lifetime exceeds the configured ceiling
(DEC-028, default 30s). Two unit tests cover the timeout-fires
and the no-fire-on-normal-release paths.
`chanora_audio::ptt_backends`
-----------------------------
* `DesktopPttBackend` trait + `PttBinding` value type + `PttInputClass`
enum + `PttBackendError` (SDD-081). `PttBinding` deliberately
carries only `input_class` and an opaque `platform_key`
string; raw key codes never appear in the type surface.
* `select()` factory (SAD-071): runtime ladder evaluation per
OS. Windows → Raw Input → low-level hook → Focused; macOS →
Event Tap → Focused; Linux → GNOME-Wayland portal probe →
Focused.
* `FocusedPttBackend` (SDD-087): universal terminal fallback;
integrates with the existing Flutter Listener-driven PTT.
* `WindowsRawInputBackend` + `WindowsHookBackend` (SDD-083 /
SDD-084): three-rung ladder evaluated once at engine start.
Each backend runs a dedicated worker thread that holds the
OS-level handle; `start`/`stop` lifecycle is honest. Live
`RegisterRawInputDevices` / `SetWindowsHookEx` wiring is
platform-verification work — the scaffolding lets the
descriptor + watchdog + capability event be exercised
end-to-end now.
* `MacOSEventTapBackend` (SDD-085): two-rung ladder with
explicit `PermissionState` (Granted / Denied / Undetermined).
`Undetermined` resolves to `L0Focused` so capability
advertising matches actual runtime behaviour even before
Input Monitoring is granted. Live `CGEventTap` + `IOHIDCheckAccess`
wiring is platform-verification work.
* `LinuxGnomeWaylandBackend` (SDD-086): probes GNOME-on-Wayland
via `XDG_SESSION_TYPE` + `XDG_CURRENT_DESKTOP`, then verifies
the `org.freedesktop.portal.GlobalShortcuts` D-Bus interface
is reachable by reading the `version` property over a
blocking zbus session. Reports `gnome-wayland-portal` /
`L2GlobalHoldToTalk`. Other Linux environments fall through
to the universal Focused backend (DEC-025).
`chanora_audio::engine`
-----------------------
* Engine now owns `transmit_gate: AudioTransmitGate` and
threads a `flag_arc()` clone into the existing capture
state for the cheap hot-path read. `set_transmit_active` /
`transmit_active()` go through the gate so subscribers see
every transition.
* `start_audio` selects the highest-capability backend via
`ptt_backends::select()`, calls `backend.start(gate, none())`,
and spawns the watchdog. Both are released in `stop()` and
on Drop.
* New `engine.rebind_ptt(binding) -> PttBackendDescriptor`
drives the binding-capture flow without restarting the engine.
* New `engine.ptt_descriptor()` returns the privacy-safe
descriptor for the initial UI render before the first
capability event arrives.
`chanora_core`
--------------
* Re-exports `PttBinding` + `PttInputClass`.
* New `ChanoraSession::set_ptt_binding(binding)` — calls
`audio.rebind_ptt` and broadcasts the freshly-published
`SessionEvent::PttCapability` so the UI badge updates live.
* New `ChanoraSession::ptt_descriptor()` for the initial render.
`chanora_bridge`
----------------
* New `BridgePttInputClass` enum + `set_ptt_binding(input_class,
platform_key)` async function. The `platform_key` string is
opaque to the bridge and never logged.
* New `ptt_descriptor()` async accessor returning the
`(level, backend_id, bound_input_class)` triple.
Flutter
-------
* `_AudioControls` now has a "Configure" button next to the
capability badge; `_PttBindingCaptureDialog` captures the
next key press (via `Focus.onKeyEvent`) or mouse side button
(via `Listener.onPointerDown` filtered to button bitmasks
`0x08` / `0x10`). The captured value is the platform-neutral
`LogicalKeyboardKey.keyLabel` or `mouse-side-button:{button}`.
* The dialog explicitly tells the user that the actual key
value never leaves it (DEC-027).
* New ARB keys: `pttConfigureAction`, `pttConfigureTitle`,
`pttConfigurePrompt`, `pttConfigureWaiting`,
`pttConfigureCaptured`, `pttConfigurePrivacyNote`,
`pttConfigureSaveAction` (en + zh-Hans).
Dependencies
------------
* `chanora_audio` adds (Linux only) `zbus = "5"` with the
`tokio` runtime selector + `blocking-api` feature for the
GlobalShortcuts portal probe.
* `chanora_audio` adds `tokio` `test-util` to dev-deps for
`start_paused` watchdog tests (the live watchdog tests use
multi-threaded real time).
Verification
------------
* `cargo test --workspace` with `CHANORA_DISABLE_KEYRING=1`:
57 tests green (was 53). chanora_audio rises from 4 to 8.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: regenerates
`docs/security/license-inventory.{md,html}`. The crate count
rises from 364 to 383 with the addition of the zbus tree.
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE (unchanged).
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` + `flutter build
linux --release`: clean Linux x86_64 bundle.
Documentation
-------------
* `docs/release/release-readiness-go-nogo-record.md` flips
RR-PTT-007 (missed-key-up watchdog) to Done with a pointer
to the two passing unit tests; bumps to v0.9.4. Live
per-platform traces (RR-PTT-001..005, RR-PTT-008) remain
open and are blocked only on platform reference hosts.
Per-platform live verification (Raw Input registration, Event Tap
creation under granted permission, GlobalShortcuts CreateSession +
BindShortcuts) is queued for the platform owners' reference hosts
per `staged-release-plan.md`.
|
||
|
EdisonJwa
|
780fd7eca2 |
feat(beta): External Beta — passwords, channel join, mute, bookmarks, encrypted identity
The v0.3 client could only ever connect to a hardcoded default
channel with no password and offered no controls mid-call.
External Beta closes those gaps and tightens identity-at-rest.
User-facing additions
---------------------
* **Server password** on the connect form. Plumbed through
`BridgeError`-aware `connect(host, nickname, password)`. Empty
string means "no password" — no behaviour change for open
servers.
* **Channel join**: tapping a row (or its login icon) in the
channel tree issues a `client_move`. Names containing "🔒" or
"password" prompt for a channel password first.
* **Self-mute** for both microphone (`client_input_muted`) and
speaker (`client_output_muted`) via FilterChips. Output mute
also flips the audio engine's local output-muted flag so
playback silences immediately, before the server acknowledges.
* **Master output gain** slider (0–200%). Plumbed through an
`AtomicU32` (f32 bits) on the engine that the cpal output
callback multiplies into every sample.
* **Bookmarks**: SQLite-backed list with Save / Connect / Delete
actions. Bookmarks persist across app restarts; tapping one
pre-fills the form and dials immediately.
Hardening
---------
* **Encrypted identity at rest** (RISK-PoC-002 closure for the
file-only threat model). ChaCha20-Poly1305 envelope: nonce +
ciphertext written atomically with mode 0600; 32-byte DEK in a
separate `identity.dek` file. Legacy plaintext identity files
are auto-detected, read, and upgraded on the next save. Full OS-
keyring integration is still v0.4 work — documented in the
store's doc comment.
* **Mobile voice-comm routing**: on Android, `AudioEngine::start`
uses JNI to set `AudioManager.setMode(MODE_IN_COMMUNICATION)`
when `cfg.mobile_voice_preset` is true (default). This engages
the device-side AEC/NS pipeline on most Pixel/Moto/Samsung
hardware even though cpal still opens the AAudio default input
preset. Full `setInputPreset(VOICE_COMMUNICATION)` switch is
still RISK-AUDIO-MOBILE-001 (needs cpal upstream or an Oboe
fork).
* **Log noise**: bridge default `EnvFilter` now silences
`tsproto::resend=error` and `tsproto::packet_codec=error` so
the redacted diagnostic export is human-readable. Still
overridable via `RUST_LOG=...`.
Engineering
-----------
* **`chanora_storage`** gains `BookmarkRepository` (rusqlite
bundled) with `add` / `update` / `delete` / `list`. The
identity store now layers on `chacha20poly1305` + `rand` +
`zeroize` for the envelope.
* **`chanora_protocol`** exposes `move_to_channel` and
`set_muted` on `ProtocolClient`, dispatched through the
existing `connection_task` request channel onto tsclientlib's
generated `client.client_move(...)` and
`state.client_update().set_input_muted/set_output_muted(...)`
paths.
* **`chanora_core::ChanoraSession`** wires the bookmark store
next to the identity store inside `init_storage`, and adds
`list_bookmarks` / `add_bookmark` / `update_bookmark` /
`delete_bookmark` / `move_to_channel` / `set_self_muted` /
`set_output_gain`.
* **`chanora_audio::AudioEngine`** carries `output_gain` and
`output_muted` atomics; the output callback consults both. The
Android branch of `start()` engages MODE_IN_COMMUNICATION via
a small JNI helper that reuses the `ndk_context` global set by
the bridge's `android_init` hook.
* **`chanora_bridge::api`** adds `set_input_muted`,
`set_output_muted`, `set_output_gain`, `move_to_channel`,
`list_bookmarks`, `add_bookmark`, `update_bookmark`,
`delete_bookmark`, and the `BridgeBookmark` DTO. FRB v2.12
codegen regenerated.
Tests + CI
----------
* `chanora_storage` test count rises from 3 to 8 — bookmark CRUD
round-trip, missing-row → `NotFound`, encrypted round-trip
(verifies ciphertext is not the plaintext on disk), and the
legacy plaintext upgrade path.
* New `.github/workflows/ci.yml`: `cargo check --workspace`,
`cargo test --workspace --no-fail-fast`, `cargo clippy`
(advisory), `flutter analyze`, and `flutter test` excluding
the live-server `e2e` tag.
Live-verified on Moto G Stylus 5G against cn.teamspeak.app:
saved a bookmark, reconnected via it, joined a non-default
channel via tap, toggled both mutes, slid the volume, and the
redacted diagnostic export confirmed `AudioManager mode set to
MODE_IN_COMMUNICATION`, `client_move sent`, and `client_update
sent` lines.
|
||
|
EdisonJwa
|
9790005c3e |
feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
|
||
|
EdisonJwa
|
974dda9601 |
feat(scaffold): create product workspace + Rust crates + Flutter app
Implements the canonical implementation directory layout adopted by
DEC-022 (register v0.9.5). Closes the scaffolding phase; no PoC code
has been promoted in yet (per proof-of-concept-plan.md §4 a PoC is
not product code unless explicitly promoted).
Rust workspace
==============
Top-level Cargo.toml declares seven workspace members:
core/chanora_core top-level Rust API + orchestration
crates/chanora_protocol tsclientlib isolation (SAD-067, SysDes-011/029)
crates/chanora_state state sync, reducers, deltas
crates/chanora_audio capture, DSP, Opus, jitter, mixer, playback
crates/chanora_storage non-secret DB + platform secure store
crates/chanora_diagnostics logs, redaction, export
crates/chanora_bridge typed Flutter/Rust DTOs
Workspace-wide pins:
license = "MIT OR Apache-2.0" (DEC-020)
rust-version = "1.95"
edition = "2021"
The Flutter app (apps/chanora_flutter) is NOT a Cargo workspace
member; it is owned by the Flutter / Gradle toolchain and is in the
workspace exclude array along with every poc/* spike.
Each crate ships:
* a Cargo.toml referring to workspace.dependencies pins;
* a lib.rs with #![forbid(unsafe_code)] + #![warn(missing_docs)],
a typed Error enum, and the public types relevant to the
subsystem's role per SAD §7.2;
* minimal unit tests so
running 1 test
test tests::defaults_match_decisions ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::session_can_be_constructed ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::marker_matches_poc ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::state_transitions_compile ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s is non-empty.
chanora_core's CoreError type-wraps every subsystem error via
#[from] so callers can match on origin without parsing strings.
chanora_audio's AudioEffects struct defaults all four effects to
true, matching DEC-007 (AEC), DEC-008 (AGC), DEC-009 (NS),
DEC-010 (HPF). A unit test pins this so a future regression that
flips a default fails immediately.
chanora_diagnostics exports REDACTION_MARKER = "[REDACTED]",
identical to the PoC's marker so audit grep patterns survive the
promotion.
chanora_bridge's BridgeError is Serialize + Deserialize so it can
flow across the FRB 2.x boundary (DEC-014).
Empirical verification: cargo check --workspace clean, cargo test
--workspace clean (7 unit tests + 7 doc-test runners, all passing)
against Rust 1.95.0 stable.
Flutter app
===========
apps/chanora_flutter created with .
DEC-004 applied: minSdk overridden to 28 in
android/app/build.gradle.kts with a comment that points back at the
decision register and forbids lowering it without re-opening DEC-004.
DEC-015 applied: shipped English + Simplified Chinese at MVP.
- pubspec.yaml gains flutter_localizations + intl + generate:true.
- l10n.yaml emits lib/l10n/generated/AppL10n (no synthetic package
— that was removed in Flutter 3.41+).
- lib/l10n/app_en.arb is the source of truth; lib/l10n/app_zh.arb
mirrors the key set in zh-Hans. ARB metadata explicitly
reaffirms ADR-008: server-provided content (channel names,
nicknames, welcome banners) is preserved verbatim and never
translated.
lib/main.dart and test/widget_test.dart were rewritten from the
template counter into a minimal localized scaffold that proves
both locales render correctly.
Empirical verification: reports no issues;
runs the two locale smoke tests and both pass.
|