d03ea937e6040c849f3204fa0fb051804fe560a1
32
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edison Jwa
|
d03ea937e6 | fix: refine linux voice runtime behavior | ||
|
Edison Jwa
|
a2d686d9d0 | feat: promote linux native audio path | ||
|
Edison Jwa
|
5515ff6643 | feat: stabilize voice activity and audio routing | ||
|
Edison Jwa
|
7d5d8c2c90 | feat: integrate chat voice and diagnostics client | ||
|
Edison Jwa
|
bf284018e6 |
feat: Android Oboe voice backend — WebRTC APM, VAD, HW/SW toggle, BBCode welcome, link trust, foreground task
Audio engine (Rust): - Android Oboe: WebRTC APM (AEC/NS/AGC/HPF) + TEN/Silero ONNX VAD - Hardware effects (JNI) with software fallback per-effect - Render reference buffer for AEC between output/capture callbacks - Voice activity gate: suppress transmission when speaker muted (all platforms) - Audio focus (SDD-109) + Bluetooth SCO (SDD-110) via JNI - ONNX Runtime 1.26 via ort 2.0.0-rc.12 (down from rc.10, ndarray 0.17) - VAD worker channel capacity 8→32, initial seq u64::MAX (warm-up fix) - TEN VAD default backend (was Silero) - Platform→WebrtcApm resolution after hardware binding - oboe-rs edisonjwa fork with get_raw_session_id() Android Kotlin: - AndroidAudioFocusController + AndroidBluetoothScoController - AndroidAudioLifecycleController (route changes to Flutter) - ProGuard rules for new controllers Flutter UI: - VoiceSettings: Android HW/SW toggle (Platform auto / WebRTC APM) - VoiceStatusChip: mute warning border + Speaker muted label - BBCode welcome message parser (BbCodeText, case-insensitive) - Welcome message foldable (expanded by default) - Link trust dialog (domain wildcards, SharedPreferences) - HapticFeedback on voice sheet opener - Server name in AppBar, version v0.1.0 - Default channel (id=1) visible, serverquery clients hidden - flutter_foreground_task integration Config: - ort load-dynamic on all non-iOS (Android/Linux/Windows) - ONNX Runtime AAR 1.26.0 - ndarray moved to common deps (was Apple-only) |
||
|
Edison Jwa
|
6af4ecab0f | feat(voice): add iOS VAD runtime support | ||
|
Edison Jwa
|
eb1f4b895e | fix(android): remove cpal product wording | ||
|
Edison Jwa
|
c87b47f064 | feat(audio): prefer native voice backends | ||
|
Edison Jwa
|
8c253f1d4d | feat(voice): harden Android audio and channel joins | ||
|
EdisonJwa
|
7188a5a69d |
feat(perf,benchmark-infra): criterion bench harness + advisory CI workflows (SDD-120)
Implementation of SDD-120 §1-§8: Bench harness (crates/chanora_audio/benches/): - common.rs: deterministic synthetic audio (440 Hz sine, no RNG). - realtime_capture.rs: bench_capture_alloc_count (dhat) + bench_capture_callback_wall_clock (criterion). - opus_codec.rs: bench_opus_encode_latency + bench_opus_decode_latency (direct audiopus, not AudioHandler — SDD-120 §3 item 4). - resampler.rs: bench_resampler_throughput across 44.1->48 / 16->48 / 48->48 passthrough. CI tooling (crates/chanora_audio/examples/): - emit_baseline.rs: aggregates criterion estimates.json outputs into the SRS-217 baseline schema. - compare_baseline.rs: applies SRS-219 tolerance, renders markdown table with 🟢/🟡/🔴 markers + yellow simpler-form realization per SDD-120 §8. Deviation from SDD-120 §2 / §5 / §7 placement: these tools live under examples/, not benches/ or src/bin/. Rationale: they must consume serde_json (a dev-only dep — production builds must not pull it). Cargo only resolves dev-dependencies for [[test]], [[bench]], and [[example]] targets; [[bin]] targets under src/bin/ see only regular [dependencies]. examples/ keeps the binaries out of the production dep tree while still giving them cargo run --example invocation. An SDD-120 amendment should reflect this. Workflows (.github/workflows/): - bench-advisory.yml: PR + push triggers; runs benches; posts a sticky PR comment via actions/github-script@v7; job status is always success (SRS-218 clause 4 — non-blocking). - bench-baseline-update.yml: workflow_dispatch only; runs benches; opens PR via peter-evans/create-pull-request@v6 (sole writer of the SAD-089 baseline JSON). Cargo.toml additions ([dev-dependencies] only — verified excluded from --release builds): criterion 0.5, dhat 0.3, serde_json 1. Source-code seam: minimal pub-but-#[doc(hidden)] bench_seam module in chanora_audio (engine.rs + lib.rs re-export) so the criterion bench harness can construct a CaptureState and drive CaptureState::ingest without re-implementing the engine (SDD-120 §3). Non-iOS targets only — CaptureState itself is iOS-gated. Initial baseline seed: crates/chanora_audio/benches/baselines/ x86_64-unknown-linux-gnu.json = {}. compare_baseline handles the missing-baseline case gracefully and emits a 'no red markers' report; the first manual dispatch of bench-baseline-update.yml after merge establishes the real values. Out of scope per SDD-120 §10: production telemetry export, build-failing hard CI gate, multi-host benchmarking, IDE integration, Dart-side bridge round-trip bench. Verification: - cargo check --workspace --all-targets: PASS. - cargo bench --bench realtime_capture --no-run: PASS. - cargo bench --bench opus_codec --no-run: PASS. - cargo bench --bench resampler --no-run: PASS. - cargo build --example emit_baseline --example compare_baseline -p chanora_audio: PASS. - cargo test --workspace: 106 passed, 0 failed, 3 ignored — no regression from prior count. |
||
|
EdisonJwa
|
d13b56d379 |
perf(audio): pre-allocate capture scratch buffers to avoid realtime-thread Vec allocs (SDD-094)
The capture cpal callback (CaptureState::ingest) ran two heap
allocations per callback on the realtime audio thread:
1. engine.rs:1196-1202 — fresh `mono: Vec<f32>` for the downmix
output, once per cpal callback (50–100 Hz).
2. engine.rs:1217-1218 — `pcm_accum.drain(..FRAME_SAMPLES).collect()`
building a fresh Vec<f32> of 960 samples per Opus frame.
Both sites mirror the pattern already fixed for the output side at
engine.rs:1389-1397, where allocating per callback on glibc malloc
was correlated with user-perceptible audio popping. The output-side
fix replaced the per-callback allocation with a pre-allocated
`scratch` Vec that is cleared and resized in place; this commit
applies the same template to the capture side.
Changes:
- Add `mono_scratch: Vec<f32>` and `frame_scratch: Vec<f32>` to
CaptureState. Initialised with Vec::with_capacity(4096) and
Vec::with_capacity(FRAME_SAMPLES=960) respectively in
CaptureState::new.
- Replace the downmix Vec construction with in-place push into
`self.mono_scratch`; `clear()` retains capacity across callbacks.
- Replace the drain().collect() with `self.frame_scratch.extend(
self.pcm_accum.drain(..FRAME_SAMPLES))`; same capacity-retention.
- The resampler call uses std::mem::take to swap the scratch buffer
out for the duration of the &mut self call, then moves it back —
the backing allocation is preserved across callbacks.
Algorithm semantics are unchanged: same downmix arithmetic, same
clamp loop, same Opus encode call sequence. Only the storage
strategy differs.
Out of scope (intentionally not touched):
- Android audio path (android_voice_unit.rs, mobile_voice_backend.rs):
researcher constraint C-4 — the Android cpal data path is mid-
migration and being replaced.
- Output callback at engine.rs:1409+: the only obvious per-callback
allocation there (`scratch`) was already fixed; a fuller audit
is a separate scope decision.
- The `scratch` buffer at engine.rs:1389-1397 — already correct.
Verification:
- cargo check --workspace --all-targets: passes.
- cargo test --workspace: 106 passed / 0 failed / 3 ignored.
- cargo clippy --workspace --all-targets: no new lints introduced;
the one warning inside the edited region (clamp-like pattern at
line 1266) was pre-existing on the copied clamp loop.
|
||
|
EdisonJwa
|
78190c0694 |
feat(audio,android): wire engine + AudioManager JNI through ModeStack (SDD-108 §2)
Replace the prior one-shot android_engage_voice_communication call with a ModeStack-mediated acquire/release pair. AudioEngine snapshots the system audio mode on first acquire via android_get_audio_mode() and restores it on last release via android_set_audio_mode(prior). MODE_IN_COMMUNICATION (3) is engaged across the voice-session lifetime per SDD-108. Includes the Android AudioManager getMode/setMode JNI helpers (placed in chanora_audio::engine alongside the existing JNI surface) and the small ptt.rs touch needed for the SDD-108 ID-tag on the existing tests. Trace: SDD-108, SDD-115. |
||
|
Edison Jwa
|
dc9c5c0a4e |
feat: multi-platform bug fixes, Android audio path, and build tooling
Flutter UI fixes: - Fix stale channel badge/speaker when moved by others (derive current channel from ownClientId instead of optimistic local state) - Fix Linux PTT via focused fallback key handler - Distinguish ServerQuery clients with terminal icon in client list - Reduce duplicate current-channel badge display - Prevent PTT key-bind save from permanently closing voice settings - Fix Linux GTK reopen-after-close (quit app on window destroy) - Fix focused PTT: consume key events, release held keys on disconnect/leave-channel/mode/backend changes, suppress stale errors Flutter Rust bridge: - Thread is_server_query flag through protocol→bridge→Dart - Add own_client_id to BridgeSnapshot DTO - Add log_file_path_str() for platform log path queries Rust protocol: - Add ServerQuery test coverage (query_client_type_maps_to_server_query_flag) - Split reqwest TLS: native-tls for desktop/iOS, rustls for Android Rust audio: - Upgrade cpal 0.16→0.17.3 with API adjustments (SampleRate, description()) - Suppress Android-only dead-code warnings (open_log_file, keyring_account) Android build tooling: - tools/build-opus-android.sh: NDK auto-discovery, correct CMake Android variables (ANDROID_ABI, ANDROID_PLATFORM), portable baseline - tools/build-android-rust.sh: build+copy Rust cdylib for arm64-v8a, armeabi-v7a, x86_64 into android/app/src/main/jniLibs/ - Add jniLibs/ to .gitignore Rust bridge: - Guard open_log_file() on non-Android (Android uses logcat) |
||
|
Edison Jwa
|
7a59f5b9a1 | feat(ios,p0): iOS P0 platform, audio fixes, channel UX | ||
|
EdisonJwa
|
9502580b5a |
chore(audio,ios): silence cpal-side dead_code on iOS + regen Podfile.lock (rc.8+62)
Two follow-ups after the iOS Rust build went green at |
||
|
EdisonJwa
|
5de6eccc0c |
fix(audio,ios): correct ios_voice_unit imports + iOS stop() field access (rc.8+61)
iOS build of chanora_audio (target aarch64-apple-ios) failed with
four compilation errors after commit 2 landed. Root causes were
all simple symbol-path / cfg-gating mistakes from the skeleton
commit; the underlying design is unchanged.
1. ios_voice_unit.rs: wrong import path for OutPacket. The
chanora_protocol crate re-exports it at the crate root
(`pub use ...::OutPacket` in lib.rs line 52), not from a
`voice` submodule (which doesn't exist).
- use chanora_protocol::voice::OutPacket;
+ use chanora_protocol::OutPacket;
2. ios_voice_unit.rs: LinearPcmFlags lives in
`coreaudio::audio_unit::audio_format`, not in
`stream_format` (the doc page lists it under StreamFormat but
the actual module path is the upstream Apple naming).
- use coreaudio::audio_unit::stream_format::LinearPcmFlags;
+ use coreaudio::audio_unit::audio_format::LinearPcmFlags;
3. ios_voice_unit.rs: `Ordering` import unused (commit 1
skeleton callbacks don't load atomics yet — that comes in
commits 3 + 4). Remove from the std::sync::atomic import to
silence the unused_imports warning.
4. engine.rs::AudioEngine::stop(): the existing body unconditionally
touched self._input_stream and self._output_stream, but commit
2 cfg-gated those fields away on iOS (and added an iOS-only
_ios_voice_unit field in their place). Split the field drop
logic with the same target_os = "ios" cfg so each platform
only touches the fields it actually has.
Also clean up two pre-existing warnings exposed by the iOS cfg
gating:
5. engine.rs: `tracing::{error, warn}` were imported
unconditionally but are only used inside cpal log lines.
Cfg-gate the import to not(target_os = "ios").
6. engine.rs: `AudioData`, `CodecType`, `OutAudio` from
chanora_protocol are only referenced in the Opus encoder feed
inside CaptureState — cpal-side only. Cfg-gate to
not(target_os = "ios"); keep `InboundVoice` + `OutPacket`
on the unconditional path because the inbound forwarder + (in
commit 3) the iOS capture pipeline both reference them.
Also fix a stray duplicate `#[cfg(not(target_os = "ios"))]`
attribute that landed on line 18 in commit 2.
Build verify (Linux host): cargo check -p chanora_audio clean
in 0.53s. iOS-side check pending on Mac.
Build counter bumped 60 -> 61 — the About dialog will display
v1.0.0-rc.8+61 so the user can confirm the build under test
matches this commit.
|
||
|
EdisonJwa
|
3b1724e970 |
feat(audio,ios): wire IosVoiceUnit into AudioEngine, cfg-gate cpal away on iOS (commit 2/5)
Split AudioEngine::start_with_gate into two backends:
* start_with_gate_cpal — non-iOS path, the existing cpal + (SDL on
Linux) flow, renamed verbatim, no
behavioural change.
* start_with_gate_ios — iOS path, constructs a single
IosVoiceUnit (VoiceProcessingIO via
coreaudio-rs) for combined mic + speaker.
Spawns the same inbound forwarder task
that pumps Opus packets into AudioHandler.
The public entry point start_with_gate dispatches at the top via
cfg(target_os = "ios") so callers stay backend-agnostic.
Struct field changes:
* _input_stream : cfg-gated to not(ios)
* _output_stream : cfg-gated to not(ios), keeps the
Linux=SdlOutput / else=cpal::Stream split
* _ios_voice_unit: new field, cfg-gated to ios, owns the VPIO
AudioUnit for the engine's lifetime.
Module-level cfg-gating:
* All cpal-only helpers (try_open_capture, build_input_stream,
build_output_stream, CaptureState + impl, ToF32 / FromF32 traits
and impls, PlaybackResampleState) are now wrapped with
#[cfg(not(target_os = "ios"))]. Same for the audiopus
encoder + cpal trait imports — iOS doesn't pull libopus into the
engine yet (commit 3 will, once the VPIO input callback wires
into CaptureState).
Behaviour on iOS for THIS commit:
* AudioEngine starts cleanly, IosVoiceUnit::start succeeds (VPIO
unit allocates + initialises + starts).
* Mic capture is dropped (the input callback is a no-op stub).
* Output emits silence (the render callback fills the buffer with
zeros).
* Inbound forwarder still runs and pushes Opus packets into
AudioHandler — they accumulate in the jitter buffer but no
fill_buffer drain happens (commit 4 fixes that), so the buffer
will grow up to MAX_BUFFER_TIME (~0.5 s) and then tsclientlib
starts dropping the oldest frames. This is fine for now — the
point of this commit is verifying the AudioUnit constructs +
starts cleanly on the device. Audible silence is the expected
state until commits 3/4 land.
Build verify (Linux host): cargo check -p chanora_audio clean in
0.53s. iOS-side compile happens on the Mac via the Xcode build
the user will trigger next.
|
||
|
EdisonJwa
|
6a4dbad60e |
fix(ios,audio): use AVAudioSession mode .default + correct mono downmix
Two changes that together address the user-reported 'speaker selector not working' AND 'audio quality bad' symptoms on iPhone: 1. AppDelegate.swift: AVAudioSession mode .voiceChat -> .default .voiceChat binds the underlying AudioUnit's output element to a SINGLE physical transducer (the receiver/earpiece) at session- configure time. overrideOutputAudioPort updates AVAudioSession's route metadata so currentRoute.outputs reports Speaker, but the AudioUnit's output binding is stale and audio keeps routing to the original transducer. Net: tapping Speaker in the picker flipped the route in our log but produced no audible change. .voiceChat also enables iOS's telephony processing chain (forced mono output, aggressive AGC, heavy noise gating) which explains the 'garbled / watery / metallic' quality complaints. .default mode uses iOS's standard audio graph: stereo output, no AGC, no telephony post-processing, AudioUnit re-binds live when the route changes. Same mode Music.app and most non-telephony apps use. Trade-off: we lose iOS hardware AEC. If users report speakerphone echo we'll add software AEC (DEC-030). Category options unchanged \u2014 .allowBluetoothHFP + .allowBluetoothA2DP still permit BT headsets in both directions. 2. engine.rs::build_output_stream: mono device downmix fix The mixing path at dev_channels==1 previously wrote only the L channel of AudioHandler's stereo output into the single mono device channel and discarded R entirely. Anything panned right in the stereo voice mix was silently lost \u2014 on .voiceChat speakerphone (forced mono device) this manifested as quiet remote speakers being inaudible. Fix: when dev_channels==1, output = (L + R) * 0.5 instead of just L. The dev_channels>=2 branch is unchanged. With change #1 iOS will typically expose stereo so this branch is rarely hit, but the fix is correct for any genuinely-mono sink (some BT car-audio profiles, USB mono headsets). |
||
|
EdisonJwa
|
54ec8dd5a8 |
feat(audio): tune Opus encoder for VoIP (bitrate 32k, complexity 10, inband FEC, 5% PLC)
User report: 'sound heard are too poor' on iPhone iOS \u2014 garbled/
robotic + stutters/dropouts.
The Opus encoder ran with audiopus defaults: 'auto' bitrate that
drops to ~6 kbps during silence (sounds watery on speech resume),
inband FEC disabled (single packet loss = silent gap), no packet-
loss percentage hint (encoder can't budget bits for redundancy).
On lossy mobile networks (cell, WiFi roaming) this combination
sounds noticeably worse than the same Opus stream from a desktop
client. Garbled = silence-bitrate transitions; stutters = packet
loss without FEC.
Fix: tune the encoder once at construction with values derived
from RFC 6716 \u00a77.1 (Opus VoIP recommendations), Discord's voice
client tuning, and the Mumble defaults.
* set_bitrate(32_000) \u2014 sweet spot for mono speech. Below
24 kbps starts to sound watery;
above 64 kbps wastes bandwidth.
Discord uses 64 kbps; Mumble
defaults to 40 kbps; we pick
32 kbps as a conservative VoIP
value that survives ~100 kbps
uplinks comfortably.
* set_complexity(10) \u2014 max quality. The CPU cost on a
modern iPhone (A14+) or any
desktop is negligible (~0.5 % of
a single core for 48 kHz mono).
* set_inband_fec(true) \u2014 Opus inserts a low-bitrate copy
of the previous frame inside the
current packet so single-packet
loss can be reconstructed from
the next packet. Essential on
lossy mobile. The decoder side
(tsclientlib's AudioHandler)
auto-handles FEC frames; no
receiver-side change needed.
* set_packet_loss_perc(5) \u2014 tells the encoder to budget
bits for 5 % expected loss.
Higher values trade audio
quality for resilience.
Each setter is wrapped in a soft-fail: if an exotic libopus build
rejects one of these, we log + continue with the still-functional
encoder rather than aborting the audio engine. info!-log a one-
liner per-engine-start summarising the tuned values so a future
diagnostic export can correlate audio reports with the active
configuration.
Application::Voip mode was already set (engine.rs:630, unchanged);
the new calls layer on top of that mode's defaults.
cargo test -p chanora_audio --release --lib: 32 passed.
flutter build ios --release --no-codesign: 17.2 s, Runner.app
30.4 MB.
|
||
|
EdisonJwa
|
d4c04b6a72 |
fix(ui,audio,ios,macos): eight P0 mobile fixes
User report from sideloaded iPhone build, in order of priority: #4 'Could not join channel: audio: audio backend: build_output_stream: The requested stream configuration is not supported by the device.' Cause: we forced cpal::BufferSize::Fixed(2048) on the output and input streams unconditionally on non-Linux. iOS CoreAudio RemoteIO units reject arbitrary buffer-size requests with that exact error. Windows WASAPI needs the pinning for shared-mode jitter, but macOS / iOS do not. Fix: cfg-gate Fixed(2048) to target_os = 'windows'; everywhere else use BufferSize::Default and let the platform HAL pick. crates/chanora_audio/src/engine.rs. #5 'Could not join channel: invariant violated: voice_in already taken' Cause: start_audio tore down the old engine BEFORE attempting to construct the new one, and consumed voice_in (an mpsc Receiver that can only be taken once) early. When the new engine failed mid-construction (e.g. because of #4 above) the session was left with: no audio engine, voice_in consumed, no way to retry without reconnect. The second voice_join attempt surfaced the invariant message. Fix: build the new engine BEFORE tearing down the old. Only swap state.audio if construction succeeded. crates/chanora_ core/src/lib.rs::ChanoraSession::start_audio. Additionally added a put_voice_in helper to the protocol adapter ( crates/chanora_protocol/src/adapter.rs) for a future broadcast-channel migration; the helper is unused on the immediate fix path but documents the intent. #3 'permission request would better on first open' Cause: AVAudioSession only triggers the mic-permission prompt the first time it tries to record. We never recorded until voice_join, so the prompt fired then. Fix iOS: AVAudioSession.sharedInstance().requestRecordPermission in AppDelegate.swift::application(_:didFinishLaunchingWithOptions:). Fix macOS: AVCaptureDevice.requestAccess(for: .audio) in macos/Runner/AppDelegate.swift::applicationDidFinishLaunching. Both run non-blocking; user can deny without crashing app launch, and voice_join then surfaces a clearer downstream error when the engine fails to open the input device. #1 + #2 'one-column upper takes too much space; Push to Talk button at bottom would be better' Layout rework for narrow-mode (single column, mobile shape): - Flipped the stacking order in main.dart so Voice Bar moves to the BOTTOM of the body and the channel tree (Expanded) fills above. Wide-mode (Row, >= 840 dp) layout unchanged. - Inside the Voice Bar on touch-only hosts, moved the on-screen Push to Talk button to be the LAST element of the Voice Bar (was Row 3). Order now: pill + mutes, mode badge + settings, level meter, stats line, release-tail caption, PTT button. The button is closest to the user's thumb when the Voice Bar is pinned to the bottom of a narrow-layout screen. #6 'remove right top debug badge' debugShowCheckedModeBanner: false on the MaterialApp. Release builds never showed it anyway; this only affects local dev / debug builds. #7 'what does the refresh button use for? nothing happened' Removed. The snapshot updates via BridgeEvent::SnapshotChanged are pushed from the bridge — a manual rust.snapshot() call was redundant. Now only the Diagnostics + Disconnect actions remain in the AppBar trailing row when connected. #8 'Bind Key related function should not be added to a mobile platform' widgets/voice_settings.dart: bind-key OutlinedButton is now #cfg'd out when Platform.isIOS || Platform.isAndroid. The release-tail slider stays because it still applies to the on-screen PTT button. Capability badge in voice_bar.dart also hidden on mobile (it would always show L0Focused which is redundant with the visible on-screen button). Tests + analyze: chanora_audio 34/0/0 on macOS, workspace 78/0/1 on Linux; flutter analyze clean (6 pre-existing Radio.groupValue infos). flutter build ios --release --no-codesign: 28.8 s clean (Runner.app 29.9 MB). |
||
|
EdisonJwa
|
f3320715ea |
feat(audio,ios): AVAudioSession PlayAndRecord+voiceChat in AppDelegate
iOS AVAudioSession must be configured BEFORE Flutter starts its
audio pipeline; the canonical place is application(_:didFinishLaunching\
WithOptions:) in AppDelegate.swift. This commit:
apps/chanora_flutter/ios/Runner/AppDelegate.swift:
* import AVFoundation
* In application(_:didFinishLaunchingWithOptions:), call
AVAudioSession.sharedInstance().setCategory(.playAndRecord,
mode: .voiceChat,
options: [.defaultToSpeaker, .allowBluetooth, .allowBluetoothA2DP])
followed by setActive(true). Failures are NSLogged but do not
block app launch — cpal's CoreAudio backend will still come up
against the default iOS routing.
This shape:
* routes the receiver/speaker like a phone call (.playAndRecord +
.voiceChat),
* engages on-device AEC / NS where supported,
* defaults to speaker so users don't have to hold the phone to
their ear,
* permits Bluetooth headsets (AirPods et al. just work).
crates/chanora_audio/src/engine.rs:
* Replace the iOS engine-start placeholder log line ('binding
pending — Chanora iOS audio is documented-only for Beta') with
an honest acknowledgment that the AVAudioSession configuration
lives Swift-side. The Rust engine acknowledges the request, then
cpal opens its CoreAudio streams against the session.
iOS-only Rust code is #[cfg(target_os = "ios")]-gated so this commit
is no-op on every other platform.
SRS-197: iOS/macOS audio routing contract. DEC-025: iOS officially
in scope for P0 (Focused PTT only — Apple's sandbox model has no
global PTT analogue).
|
||
|
EdisonJwa
|
d9c330c23b |
feat(audio,linux): output via SDL2; cpal stays on Windows/macOS
User reported persistent crackling/popping from peer audio on Linux even
after fixing the 48k->device-rate resampler boundary discontinuities,
clamping pre-Opus-encode peaks, and pre-allocating the playback scratch
buffer. Logs confirmed cpal opened raw ALSA at 44.1k native, no callback
budget violations, no underrun warnings -- yet the audio was still poor.
Root cause: cpal on Linux opens raw ALSA's 'default' PCM. On modern
PipeWire / pipewire-alsa boxes that virtual device routes through ALSA's
dmix + plug layers, whose default resampler is nearest-neighbour. cpal
also picks a small default period size (~256 frames / 5.8 ms) leaving no
headroom for kernel scheduler jitter. Both effects compound into the
crackling the user heard.
Upstream tsclientlib's own audio example
(tsclientlib/examples/audio_utils/ts_to_audio.rs) and the official Qint
client both use SDL2 with AudioSpecDesired { freq: 48000, channels: 2,
samples: 960 }. SDL2 on the same systems routes through PipeWire's PA
bridge (or PulseAudio directly), both carrying high-quality resamplers.
Fix:
* Add sdl2 = '0.37' as a target_os=linux dependency. Links libSDL2-2.0
.so (Arch sdl2-compat over SDL3, Debian libsdl2-2.0-0, Fedora SDL2).
* New module crates/chanora_audio/src/sdl_output.rs implementing
SdlOutput: opens a 48 kHz stereo 960-frame callback that zeroes the
buffer and calls AudioHandler::fill_buffer directly (no user-side
resampler). Master gain + hard-mute atomics wired in identically to
the cpal callback so set_output_gain / set_output_muted keep working.
* engine.rs cfg-gated: target_os='linux' builds SdlOutput; everywhere
else continues with the cpal output path (including the device-native-
rate negotiation and resampler-continuity fixes shipped earlier --
those remain correct on Windows/macOS where cpal targets WASAPI /
CoreAudio cleanly).
* The cpal output helpers (build_output_stream, PlaybackResampleState,
FromF32) are now cfg(not(target_os='linux'))-gated so the Linux
build doesn't emit dead-code warnings.
Capture path still cpal on every platform -- outbound audio was not
reported as bad. Resampler-continuity fix on the capture side stays:
microphone -> Opus encoder still goes through the linear interpolator
with the last-sample anchor.
Tests: 32 / 0 / 0 (chanora_audio), workspace 78 / 0 / 1 unchanged.
|
||
|
EdisonJwa
|
6d4975bd6e |
fix(ptt,ui): Continuous mode no longer self-disables after 30 s; rename PTT label to Mic
Issue 1: in Continuous transmit mode the talk indicator turned
gray-out / mic disabled after ~30 s and could only be revived by
toggling mic mute. Root cause: SAD-079 MissedKeyUpWatchdog
subscribed to AudioTransmitGate.transmit_active and force-cleared
it after 30 s of true. In PTT mode this is correct (stuck key =
bug). In Continuous mode transmit_active is *supposed* to stay
true indefinitely; the watchdog assumption doesn't hold.
Fix: the watchdog now subscribes to a new ptt_held watch on the
TransmitModeSelector (the raw key-state input, not the resolved
gate). In Continuous mode ptt_held is never set true, so the
watchdog never fires. In PTT mode it still fires on a stuck
key-down as before. The session owns the watchdog (was on the
engine) so it survives engine restarts; it's spawned lazily on the
first start_audio.
MissedKeyUpWatchdog gains spawn_on_signal(rx, on_timeout, timeout)
alongside the existing spawn(gate, timeout) — old shape preserved
for backwards compat. run_watchdog generalised to take any
watch::Receiver<bool> + Box<dyn Fn() + Send + Sync>.
Two new tests:
- watchdog_on_signal_does_not_fire_when_ptt_held_stays_false
(the Continuous-mode regression test)
- watchdog_on_signal_fires_when_signal_stays_true
(the stuck-key case still fires)
Issue 2: the Voice Bar stats line said 'PTT on/off' even when the
user was in Continuous mode where no PTT key is involved. Renamed
to 'Mic on/off' (mode-neutral) and l10n-ised the on/off literal:
- en: 'Mic on' / 'Mic off'
- zh: '麦克风 开启' / '麦克风 关闭'
cargo test --workspace --lib: 80 passed / 0 failed / 1 ignored
(was 78, +2 watchdog tests).
flutter analyze: clean (6 pre-existing Radio.groupValue infos).
|
||
|
EdisonJwa
|
ba444d94bd |
feat(audio,bridge,flutter): v1 audio + PTT lifecycle implementation (SDD-094..097)
Implement the SDD-094 / SDD-095 / SDD-096 / SDD-097 detailed designs
committed in
|
||
|
EdisonJwa
|
77c2a1def4 |
feat(audio,windows): real Raw Input + low-level hook global PTT (SDD-083 / SDD-084)
The v1.0.0-rc.7 Windows backends were thread::sleep stubs that
reported optimistic L2GlobalHoldToTalk / L3GlobalWithMouseButtons
descriptors without actually registering for any global key events.
Surfaced on Windows verification as:
* 'even press to talk key was set, still only the hold to talk
button is work for talk'
* 'and displayed as L2GlobalHoldToTalk(raw-input)'
* 'cannot continuous transmission'
This commit implements the real backends:
WindowsRawInputBackend (preferred Windows rung, SDD-083):
* Hidden message-only window via
CreateWindowExW(..., HWND_MESSAGE, ...).
* RegisterRawInputDevices with RIDEV_INPUTSINK on
Usage Page 0x01 / Usage 0x06 (keyboard) and 0x02 (mouse) so
events fire globally — including when Chanora is unfocused.
* WndProc handling WM_INPUT: GetRawInputData ->
keyboard.VKey vs bound vk, or mouse.usButtonFlags vs bound
side-button index. Down -> gate.set(true); up -> gate.set(false).
* Dedicated chanora-rawinput thread runs GetMessageW /
TranslateMessage / DispatchMessageW until stop() posts
WM_QUIT via PostThreadMessageW.
WindowsHookBackend (fallback rung, SDD-084):
* SetWindowsHookExW(WH_KEYBOARD_LL) + WH_MOUSE_LL on a
dedicated chanora-llhook thread.
* Hook procs translate KBDLLHOOKSTRUCT.vkCode and
MSLLHOOKSTRUCT.mouseData against the same shared
AtomicBinding.
* UnhookWindowsHookEx on teardown.
Both backends:
* Honest descriptor() reporting: backends start reporting
L0Focused; level upgrades to L2 / L3 only after a real
arming success (RegisterRawInputDevices or SetWindowsHookEx
returning Ok). This fixes the 'L2 reported but doesn't fire'
complaint by making the badge tell the truth — if Raw Input
registration fails at runtime the user sees the L0Focused
info-icon explanation sheet instead of being told L2 works.
* AtomicBinding (class / vk / mouse_btn) for lock-free hot
path. Translation lives in
crates/chanora_audio/src/ptt_backends/windows_keymap.rs
which maps Flutter LogicalKeyboardKey.keyLabel strings
(e.g. 'Space', 'F10', 'A') to Win32 VK_* codes; mouse
side-button bitmask strings ('mouse-side-button:8' /
':16') to RawInput button indices (4 / 5).
* Per-thread context (thread_local RefCell) carries the
gate + binding to the WndProc / hook proc without needing
raw-pointer user-data plumbing.
Diagnostic logging:
* AudioEngine::start now logs default_input_config and
default_output_config explicitly with the channels /
sample_rate / sample_format that cpal reports, so a
build_*_stream failure on locale-specific Windows hosts
(reported on ko-KR Windows 11 as 'Start Audio Button not
work') becomes diagnosable from the stderr log alone.
* build_output_stream surfaces the requested config in the
tracing::error! record on failure.
Privacy (DEC-027 / SDD-090): the windows.rs and
windows_keymap.rs hot paths NEVER log raw VKs, scan codes,
keysyms, key labels, or button identifiers. Only the
platform-neutral input class ('keyboard' /
'mouse-side-button') and the backend id appear in the tracing
stream. The SDD-090 PttSanitizer Layer is the defence-in-depth
net but this code does not rely on it.
Tests: 4 new windows-only unit tests in windows_keymap (ASCII
letters / digits / Space + Fn / unknown / mouse button index).
They compile only under cfg(target_os = "windows") so the
Linux workspace test count is unchanged at 59/0/3.
Cargo deps: adds windows = '0.54' (target_os = windows) with
the feature set needed for RawInput + hooks. 0.54 matches
the version already transitive through the workspace.
Verified on Linux: cargo check --workspace clean, cargo test
--workspace 59/0/3 (windows-gated tests skip on Linux). The
real exercise of this commit will happen on the Korean Windows
11 host (100.84.219.45) at the next build.
|
||
|
EdisonJwa
|
9831624079 |
feat(ptt): close P0 unit-boundary gaps (SDD-088 / SDD-090 / SDD-091)
The P0 audit on v0.9.4-docs found three SDD items whose specified
software units were inlined into other types rather than packaged as
named units at the SDD-defined boundary:
* SDD-088 PttController — backend ownership + binding mutex +
capability watch lived split between AudioEngine and
ChanoraSession. Extracted into chanora_core::ptt::PttController.
AudioEngine now owns only the cpal streams and the missed-key-up
watchdog (SDD-092); the platform input backend, the active
PttBinding, and the capability watch::Sender live in the
controller. ChanoraSession::start_audio constructs the controller
against the engine's gate; disconnect/reconnect/restart paths
tear it down through stop().await before the engine.
* SDD-090 PttSanitizer — banned-field check was inlined as
PttBanCheckVisitor inside RedactingLogLayer::on_event. Extracted
into a generic PttSanitizer<L> tracing_subscriber::Layer that
decorates an inner Layer (canonical pairing:
RedactingLogLayer::with_sanitizer). The inner layer keeps its
own structural ban check as defence-in-depth for bare-install
callers.
* SDD-091 PttCapabilityBadge — Voice Bar badge was anonymous
Padding/Tooltip/Row inside _AudioControlsState.build. Extracted
into a public PttCapabilityBadge widget and added the
SDD-091-specified per-platform explanation sheet that opens on
the info-icon tap when the resolved capability is L0Focused.
New l10n strings (en + zh) cover the sheet copy.
Tests:
* 2 new unit tests for PttController (arm + descriptor watch)
* 1 new unit test for PttSanitizer (end-to-end through a real
tracing subscriber proving banned drop + safe forward)
cargo test --workspace: 55 passed / 0 failed / 3 ignored
cargo deny check: advisories ok, bans ok, licenses ok, sources ok
flutter analyze: no issues
tools/validate_docs.py: zero undefined refs, zero direct-layer
violations (pre-existing 35 old-package-name warning unchanged)
No SDD/SAD/SRS doc changes — the contracts already named these
units; this commit aligns code unit boundaries with those contracts.
|
||
|
EdisonJwa
|
82d012a46b |
feat(ptt): live Linux GNOME-Wayland portal session flow (DEC-025)
Promotes the Linux backend from probe-only to a live
`org.freedesktop.portal.GlobalShortcuts` session, closing the
gen2 v0.9.3 baseline's last Linux-side code item. Both gaps I
flagged on the review pass are addressed:
* Stop now closes the portal session through the dedicated
`org.freedesktop.portal.Session` interface (not the
request-cancel `Request` interface — that would only abort a
pending Request, not release the bound shortcuts).
* Ten new unit tests cover `classify_shortcuts_value`,
`publish_bound`, `publish_l0`, and the `SHORTCUT_ID` stability
contract using synthesised `OwnedValue` payloads. Live D-Bus
coverage stays in the `linux_portal_smoke` ignored
integration test (RR-PTT-004).
Live session lifecycle (gen2 Q5b — lazy, single backend instance):
1. `start(gate, binding)` spawns one `tokio::spawn` worker that
owns an async `zbus::Connection` (sharing the bridge's
tokio runtime per Q4a).
2. `CreateSession` with fresh random `handle_token` /
`session_handle_token` tokens. The worker awaits the portal
`Response` signal via a `RequestProxy` subscription and
extracts `session_handle` from the results dict.
3. `BindShortcuts(session_handle, [("chanora-ptt", { description
= "Chanora push-to-talk" })], "", {})`. The portal opens its
own system-managed dialog asking the user to choose a key
— Chanora itself never reads raw key events. The audio
engine continues at `L0Focused` while the dialog is open;
the descriptor watch publishes the transition once the
portal returns.
4. On `response_code == 0`: classify the `trigger_description`
substring (heuristic: contains "mouse" -> MouseSideButton,
else Keyboard), publish `L2GlobalHoldToTalk` (or `L3` for
mouse) through the watch sender. The raw trigger_description
string is never logged (DEC-027 / SRS-202).
5. On `response_code == 1` (cancelled) or `>= 2` (failure):
publish `L0Focused` through the watch sender. The user can
retry via the UI "Configure" button (gen2 Q6a).
6. The worker enters a `tokio::select!` loop multiplexing the
`cmd_rx` channel (Rebind / Stop) and the `Activated` /
`Deactivated` signals. Matching signals scoped to this
session handle and `chanora-ptt` shortcut id drive
`gate.set(true/false)`.
7. `Rebind` re-runs `BindShortcuts` on the same session.
8. `Stop` calls `org.freedesktop.portal.Session.Close()` on
the session-handle object path, clears the gate, exits.
UX (gen2 Q3a): when `_pttBackendId == 'gnome-wayland-portal'`,
the Flutter "Configure" button skips the in-app
`_PttBindingCaptureDialog` and shows a SnackBar telling the user
their desktop environment will open its own shortcut dialog.
The button delegates to `setPttBinding(keyboard, "portal")`
which nudges the backend; the portal handles the rest. New ARB
key `pttConfigurePortalRedirect` in en + zh-Hans.
Trait surface (cross-cutting):
* `DesktopPttBackend::descriptor_watch()` is a new trait method
with a default impl returning a never-firing receiver.
Backends with async capability transitions (only the Linux
portal backend today) override it to return the live watch
sender's receiver.
* `chanora_core::ChanoraSession::start_audio` subscribes to the
active backend's `descriptor_watch()` and spawns a forwarder
task that re-emits `SessionEvent::PttCapability` on every
transition. The initial value is emitted synchronously.
`Cargo.toml` (Linux-only):
* `futures-util` (std features, no executor) for stream
consumption on the portal signal subscriptions.
* `rand 0.8` for fresh per-process portal tokens.
* `zbus` continues at v5 with the `tokio` + `blocking-api`
features.
Tests
-----
* `chanora_audio` rises from 8 to 18 unit tests. New
coverage on the Linux module:
- `classify_returns_none_when_shortcut_id_missing`
- `classify_returns_keyboard_for_typical_trigger_description`
- `classify_returns_keyboard_when_trigger_description_missing`
- `classify_detects_mouse_substring`
- `classify_is_case_insensitive_on_mouse_substring`
- `publish_bound_keyboard_publishes_L2_with_keyboard_class`
- `publish_bound_mouse_publishes_L3`
- `publish_bound_none_publishes_L2_keyboard_default`
- `publish_l0_clears_descriptor`
- `shortcut_id_is_stable`
* Workspace total: 67 unit + integration tests, all green with
`CHANORA_DISABLE_KEYRING=1` (was 57 at v1.0.0-rc.4).
* New `crates/chanora_audio/tests/linux_portal_smoke.rs`
ignored integration test (RR-PTT-004 evidence path). Run on
a GNOME-on-Wayland host with
`cargo test -p chanora_audio --test linux_portal_smoke -- --ignored --nocapture`.
Documentation
-------------
* `docs/architecture/desktop-ptt-architecture.md` §5.3 rewritten
to describe the realised lifecycle; v0.9.4 change-history
entry added.
* `docs/governance/product-decision-register.md` v0.9.10
change-history entry recording the code-side promotion. No
decision rows mutate.
* `docs/release/release-readiness-go-nogo-record.md` RR-PTT-004
flipped from `Open` to `Implemented (live trace pending)`;
v0.9.5 change-history entry.
Verification
------------
* `cargo test --workspace`: 67/67 green.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: zero new warnings.
* `tools/dump_flutter_licenses.sh`: 94 packages, 0 without
LICENSE.
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` +
`flutter build linux --release`: clean Linux x86_64 bundle.
* Live portal trace (RR-PTT-004) — **not run**. The dev shell
is a TTY without a Wayland session. The user will run the
ignored smoke test from inside a GNOME-on-Wayland session
when available.
No Windows / macOS / iOS live verification in this commit (hosts
unavailable). The Windows + macOS backend scaffolds remain in
place reporting their target capability honestly; live OS-call
wiring is queued for their respective platform owners'
reference hosts per `docs/governance/staged-release-plan.md`.
|
||
|
EdisonJwa
|
5199e3d005 |
feat(ptt): full desktop backend ladder + missed-key-up watchdog (gen2 v0.9.3 follow-up)
Lands SDD-081..088 + SDD-092 implementations on top of v1.0.0-rc.3.
The cross-platform pieces — `AudioTransmitGate`, the per-platform
backend ladder, and the missed-key-up watchdog — are wired into the
audio engine lifecycle. Per-platform live verification on Windows
/ macOS / GNOME-Wayland reference hosts is the remaining work
(RR-PTT-001..006/008 in `release-readiness-go-nogo-record.md`).
`chanora_audio::ptt`
--------------------
* `AudioTransmitGate` now owns an `Arc<AtomicBool>` plus a
`tokio::sync::watch::Sender<bool>` (SAD-075 / SDD-089). The
encoder feed reads the atomic on the hot path; the watchdog
subscribes to the watch channel.
* `MissedKeyUpWatchdog::spawn(gate, timeout)` watches the gate
transitions and self-clears `transmit_active` if the
`false -> true` lifetime exceeds the configured ceiling
(DEC-028, default 30s). Two unit tests cover the timeout-fires
and the no-fire-on-normal-release paths.
`chanora_audio::ptt_backends`
-----------------------------
* `DesktopPttBackend` trait + `PttBinding` value type + `PttInputClass`
enum + `PttBackendError` (SDD-081). `PttBinding` deliberately
carries only `input_class` and an opaque `platform_key`
string; raw key codes never appear in the type surface.
* `select()` factory (SAD-071): runtime ladder evaluation per
OS. Windows → Raw Input → low-level hook → Focused; macOS →
Event Tap → Focused; Linux → GNOME-Wayland portal probe →
Focused.
* `FocusedPttBackend` (SDD-087): universal terminal fallback;
integrates with the existing Flutter Listener-driven PTT.
* `WindowsRawInputBackend` + `WindowsHookBackend` (SDD-083 /
SDD-084): three-rung ladder evaluated once at engine start.
Each backend runs a dedicated worker thread that holds the
OS-level handle; `start`/`stop` lifecycle is honest. Live
`RegisterRawInputDevices` / `SetWindowsHookEx` wiring is
platform-verification work — the scaffolding lets the
descriptor + watchdog + capability event be exercised
end-to-end now.
* `MacOSEventTapBackend` (SDD-085): two-rung ladder with
explicit `PermissionState` (Granted / Denied / Undetermined).
`Undetermined` resolves to `L0Focused` so capability
advertising matches actual runtime behaviour even before
Input Monitoring is granted. Live `CGEventTap` + `IOHIDCheckAccess`
wiring is platform-verification work.
* `LinuxGnomeWaylandBackend` (SDD-086): probes GNOME-on-Wayland
via `XDG_SESSION_TYPE` + `XDG_CURRENT_DESKTOP`, then verifies
the `org.freedesktop.portal.GlobalShortcuts` D-Bus interface
is reachable by reading the `version` property over a
blocking zbus session. Reports `gnome-wayland-portal` /
`L2GlobalHoldToTalk`. Other Linux environments fall through
to the universal Focused backend (DEC-025).
`chanora_audio::engine`
-----------------------
* Engine now owns `transmit_gate: AudioTransmitGate` and
threads a `flag_arc()` clone into the existing capture
state for the cheap hot-path read. `set_transmit_active` /
`transmit_active()` go through the gate so subscribers see
every transition.
* `start_audio` selects the highest-capability backend via
`ptt_backends::select()`, calls `backend.start(gate, none())`,
and spawns the watchdog. Both are released in `stop()` and
on Drop.
* New `engine.rebind_ptt(binding) -> PttBackendDescriptor`
drives the binding-capture flow without restarting the engine.
* New `engine.ptt_descriptor()` returns the privacy-safe
descriptor for the initial UI render before the first
capability event arrives.
`chanora_core`
--------------
* Re-exports `PttBinding` + `PttInputClass`.
* New `ChanoraSession::set_ptt_binding(binding)` — calls
`audio.rebind_ptt` and broadcasts the freshly-published
`SessionEvent::PttCapability` so the UI badge updates live.
* New `ChanoraSession::ptt_descriptor()` for the initial render.
`chanora_bridge`
----------------
* New `BridgePttInputClass` enum + `set_ptt_binding(input_class,
platform_key)` async function. The `platform_key` string is
opaque to the bridge and never logged.
* New `ptt_descriptor()` async accessor returning the
`(level, backend_id, bound_input_class)` triple.
Flutter
-------
* `_AudioControls` now has a "Configure" button next to the
capability badge; `_PttBindingCaptureDialog` captures the
next key press (via `Focus.onKeyEvent`) or mouse side button
(via `Listener.onPointerDown` filtered to button bitmasks
`0x08` / `0x10`). The captured value is the platform-neutral
`LogicalKeyboardKey.keyLabel` or `mouse-side-button:{button}`.
* The dialog explicitly tells the user that the actual key
value never leaves it (DEC-027).
* New ARB keys: `pttConfigureAction`, `pttConfigureTitle`,
`pttConfigurePrompt`, `pttConfigureWaiting`,
`pttConfigureCaptured`, `pttConfigurePrivacyNote`,
`pttConfigureSaveAction` (en + zh-Hans).
Dependencies
------------
* `chanora_audio` adds (Linux only) `zbus = "5"` with the
`tokio` runtime selector + `blocking-api` feature for the
GlobalShortcuts portal probe.
* `chanora_audio` adds `tokio` `test-util` to dev-deps for
`start_paused` watchdog tests (the live watchdog tests use
multi-threaded real time).
Verification
------------
* `cargo test --workspace` with `CHANORA_DISABLE_KEYRING=1`:
57 tests green (was 53). chanora_audio rises from 4 to 8.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: regenerates
`docs/security/license-inventory.{md,html}`. The crate count
rises from 364 to 383 with the addition of the zbus tree.
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE (unchanged).
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` + `flutter build
linux --release`: clean Linux x86_64 bundle.
Documentation
-------------
* `docs/release/release-readiness-go-nogo-record.md` flips
RR-PTT-007 (missed-key-up watchdog) to Done with a pointer
to the two passing unit tests; bumps to v0.9.4. Live
per-platform traces (RR-PTT-001..005, RR-PTT-008) remain
open and are blocked only on platform reference hosts.
Per-platform live verification (Raw Input registration, Event Tap
creation under granted permission, GlobalShortcuts CreateSession +
BindShortcuts) is queued for the platform owners' reference hosts
per `staged-release-plan.md`.
|
||
|
EdisonJwa
|
7b21916049 |
feat(ptt): code-side initial split — transmit_active / capability badge / sanitizer
Implements the gen2 v0.9.3 doc baseline's first slice of code work:
* SRS-201: split the audio engine's `ptt` AtomicBool into the
authoritative `transmit_active` flag. The legacy `set_ptt` /
`ptt` accessors are retained as `#[doc(hidden)]` thin wrappers
so the existing bridge command and the existing Flutter
hold-to-talk UI keep compiling.
* SAD-075 / SDD-089 acknowledged at the type level: only
`AudioEngine::set_transmit_active` (or its legacy alias)
mutates the flag; the encoder feed reads it once per outbound
frame and never writes.
* SDD-082: new `chanora_audio::ptt` module ships the
`PttCapabilityLevel` enum (`L0Focused`, `L1GlobalShortcut`,
`L2GlobalHoldToTalk`, `L3GlobalWithMouseButtons`,
`L4DeviceAware` reserved) with a stable `as_str` mapping and
an `is_global` classifier.
* SDD-087: `PttBackendDescriptor::focused()` constant value for
the universal Focused-PTT fallback. The struct shape carries
only privacy-safe fields (`level`, `backend_id`,
`bound_input_class`) — a key code cannot fit through this
surface by construction (DEC-027).
* SAD-077 / SDD-090: `RedactingLogLayer` now hosts the
`PttBanCheckVisitor` and the `PTT_BANNED_FIELDS` constant
(`key_code`, `scan_code`, `virtual_key`, `vk`, `keysym`,
`keysym_string`, `key_sequence`, `key_press_history`,
`key_timing`). Any record whose field set names a banned key
is dropped before reaching the in-memory log sink or the
user-initiated diagnostic export. The check is structural and
runs ahead of formatting / redaction.
* `SessionEvent::PttCapability` carries the diagnostics-safe
descriptor through the broadcast event stream;
`chanora_core::ChanoraSession::start_audio` publishes the
Focused-PTT descriptor when the audio engine starts (SRS-196
/ SDD-091).
* `BridgeEvent::PttCapability` mirrors the event across the
FFI boundary. flutter_rust_bridge codegen regenerated.
* Flutter `_AudioControls` renders a capability badge above the
PTT button: a globe icon for Global levels, a focus-frame
icon for `L0Focused`, plus a Tooltip exposing the bound input
class. New ARB key `pttCapabilityBadge(level, backend)` in
`app_en.arb` and `app_zh.arb`.
Per-platform global PTT backends (`WindowsRawInputBackend`,
`MacOSEventTapBackend`, `LinuxGnomeWaylandBackend`) and the
`MissedKeyUpWatchdog` task land in a separate follow-up commit;
this milestone ships only PTT-L0 universally so the application's
runtime capability reporting is honest from day one.
Tests
-----
* `chanora_audio` rises from 1 to 4 unit tests covering
`PttCapabilityLevel::as_str`, `is_global`, and the
`PttBackendDescriptor::focused()` shape contract.
* `chanora_diagnostics` rises from 9 to 11 unit tests covering
the new `PttBanCheckVisitor` over every banned field name and
the `PTT_BANNED_FIELDS` stability assertion.
* Workspace total: 53 unit + integration tests, all green with
`CHANORA_DISABLE_KEYRING=1` (was 49 at v1.0.0-rc.2).
* `flutter analyze`: clean.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate`: zero warnings (license inventory
regenerated).
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE.
* Linux x86_64 release bundle builds clean.
No Android live verification in this commit per the user's note
that the test device was removed. Android arm64-v8a continues to
build via the same `cargo ndk` path; runtime reporting on Android
is `L0Focused` for the foreseeable future.
|
||
|
EdisonJwa
|
780fd7eca2 |
feat(beta): External Beta — passwords, channel join, mute, bookmarks, encrypted identity
The v0.3 client could only ever connect to a hardcoded default
channel with no password and offered no controls mid-call.
External Beta closes those gaps and tightens identity-at-rest.
User-facing additions
---------------------
* **Server password** on the connect form. Plumbed through
`BridgeError`-aware `connect(host, nickname, password)`. Empty
string means "no password" — no behaviour change for open
servers.
* **Channel join**: tapping a row (or its login icon) in the
channel tree issues a `client_move`. Names containing "🔒" or
"password" prompt for a channel password first.
* **Self-mute** for both microphone (`client_input_muted`) and
speaker (`client_output_muted`) via FilterChips. Output mute
also flips the audio engine's local output-muted flag so
playback silences immediately, before the server acknowledges.
* **Master output gain** slider (0–200%). Plumbed through an
`AtomicU32` (f32 bits) on the engine that the cpal output
callback multiplies into every sample.
* **Bookmarks**: SQLite-backed list with Save / Connect / Delete
actions. Bookmarks persist across app restarts; tapping one
pre-fills the form and dials immediately.
Hardening
---------
* **Encrypted identity at rest** (RISK-PoC-002 closure for the
file-only threat model). ChaCha20-Poly1305 envelope: nonce +
ciphertext written atomically with mode 0600; 32-byte DEK in a
separate `identity.dek` file. Legacy plaintext identity files
are auto-detected, read, and upgraded on the next save. Full OS-
keyring integration is still v0.4 work — documented in the
store's doc comment.
* **Mobile voice-comm routing**: on Android, `AudioEngine::start`
uses JNI to set `AudioManager.setMode(MODE_IN_COMMUNICATION)`
when `cfg.mobile_voice_preset` is true (default). This engages
the device-side AEC/NS pipeline on most Pixel/Moto/Samsung
hardware even though cpal still opens the AAudio default input
preset. Full `setInputPreset(VOICE_COMMUNICATION)` switch is
still RISK-AUDIO-MOBILE-001 (needs cpal upstream or an Oboe
fork).
* **Log noise**: bridge default `EnvFilter` now silences
`tsproto::resend=error` and `tsproto::packet_codec=error` so
the redacted diagnostic export is human-readable. Still
overridable via `RUST_LOG=...`.
Engineering
-----------
* **`chanora_storage`** gains `BookmarkRepository` (rusqlite
bundled) with `add` / `update` / `delete` / `list`. The
identity store now layers on `chacha20poly1305` + `rand` +
`zeroize` for the envelope.
* **`chanora_protocol`** exposes `move_to_channel` and
`set_muted` on `ProtocolClient`, dispatched through the
existing `connection_task` request channel onto tsclientlib's
generated `client.client_move(...)` and
`state.client_update().set_input_muted/set_output_muted(...)`
paths.
* **`chanora_core::ChanoraSession`** wires the bookmark store
next to the identity store inside `init_storage`, and adds
`list_bookmarks` / `add_bookmark` / `update_bookmark` /
`delete_bookmark` / `move_to_channel` / `set_self_muted` /
`set_output_gain`.
* **`chanora_audio::AudioEngine`** carries `output_gain` and
`output_muted` atomics; the output callback consults both. The
Android branch of `start()` engages MODE_IN_COMMUNICATION via
a small JNI helper that reuses the `ndk_context` global set by
the bridge's `android_init` hook.
* **`chanora_bridge::api`** adds `set_input_muted`,
`set_output_muted`, `set_output_gain`, `move_to_channel`,
`list_bookmarks`, `add_bookmark`, `update_bookmark`,
`delete_bookmark`, and the `BridgeBookmark` DTO. FRB v2.12
codegen regenerated.
Tests + CI
----------
* `chanora_storage` test count rises from 3 to 8 — bookmark CRUD
round-trip, missing-row → `NotFound`, encrypted round-trip
(verifies ciphertext is not the plaintext on disk), and the
legacy plaintext upgrade path.
* New `.github/workflows/ci.yml`: `cargo check --workspace`,
`cargo test --workspace --no-fail-fast`, `cargo clippy`
(advisory), `flutter analyze`, and `flutter test` excluding
the live-server `e2e` tag.
Live-verified on Moto G Stylus 5G against cn.teamspeak.app:
saved a bookmark, reconnected via it, joined a non-default
channel via tap, toggled both mutes, slid the volume, and the
redacted diagnostic export confirmed `AudioManager mode set to
MODE_IN_COMMUNICATION`, `client_move sent`, and `client_update
sent` lines.
|
||
|
EdisonJwa
|
fd181c014c |
feat(audio): A.5 — surface mobile voice-preset + effects toggles in AudioEngineConfig
The Beta scope for mobile DSP is OS-source-driven (Android `MediaRecorder.AudioSource.VOICE_COMMUNICATION`, iOS `AVAudioSession.Mode.voiceChat`) — letting the platform's built-in AEC / NS engage instead of shipping our own DSP chain on constrained devices. Linux desktop stays a deliberate no-op: PipeWire / ALSA's default source is correct for desktop voice and adding a software AEC there would regress against an already-good baseline. This commit lands the *config surface* through every layer: * `AudioEngineConfig` gains `effects: AudioEffects` (mirrors the DEC-007/008/009/010 toggles) and `mobile_voice_preset: bool` (default `true`). * On Android, `AudioEngine::start` logs the preset + effects requests so a future cpal / Oboe upstream switch can be observed via the redacted diagnostic export. * On iOS, the same log line documents the binding gap — Chanora iOS audio is documented-only for Beta per the release notes. * On Linux desktop, the flags are honoured by name but the engine continues to use the default ALSA / PipeWire source. No behaviour change. RISK-AUDIO-MOBILE-001 (new) tracks the actual preset switch. The follow-up work either pulls in an Oboe-based input host or waits for cpal upstream to expose `set_input_preset`. Either way the config flag is forward-compatible — callers do not need to change when the binding lands. |
||
|
EdisonJwa
|
9790005c3e |
feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
|