89bbfa1e18545013506c34cfcce51645ba374597
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edison Jwa
|
82441f3d97 |
feat(voice): real-time mic input level metering at 30 Hz (#25)
* feat(voice): add real-time mic input level metering at 30 Hz
Expose input RMS from the audio engine through the bridge as a
dedicated Rust→Dart Stream<double>, replacing the binary on/off
indicator with a proportional dBFS level meter.
Rust side:
- chanora_audio: add set_input_dbfs/input_dbfs accessors to
SharedAudioProcessingStats; restructure CaptureState::ingest()
to compute dBFS from mono buffer before the PTT guard so the
meter shows mic activity even when not transmitting.
- chanora_core: widen audio_stats() return to include f32 input
level.
- chanora_bridge: add input_level: f32 to BridgeAudioStats and
new input_level_stream(sink: StreamSink<f32>) that pushes at
~30 Hz via tokio interval task.
- Update frb_generated.rs serialization for the new field.
Flutter side:
- VoiceLevelMeter: accept optional double level (dBFS), map
-60..0 dBFS to 0..1 fill fraction, animate with
TweenAnimationBuilder for smooth transitions.
- voice_compact.dart: subscribe to inputLevelStream in the voice
details sheet for 30 Hz meter updates, keeping 250 ms poll for
TX/RX counters.
- voice_bar.dart: accept optional inputLevel from the stream.
- main.dart: subscribe to inputLevelStream, pass to VoiceBar.
* chore: sync Flutter build config and dependency updates
- Add Flutter migrator flags to gradle.properties (builtInKotlin, newDsl)
- Add FlutterGeneratedPluginSwiftPackage to iOS/macOS Xcode projects
- Update meta 1.17→1.18, test_api 0.7.10→0.7.11
- Rebuild chanora_bridge framework for macOS
- Update Podfile.lock for iOS and macOS
* fix(voice): correct meter animation, pre-gain dBFS, stream lifecycle, and protocol warnings
B1: Convert VoiceLevelMeter to StatefulWidget tracking previous fill
as Tween begin so the meter animates smoothly instead of resetting
to zero on every frame.
B2: Compute dBFS from pre-gain mono samples in CaptureState::ingest()
so the level meter reflects raw mic input, matching mobile paths.
B4: End input_level_stream after 10 consecutive session errors instead
of emitting -120 dBFS forever when the session is gone.
Also fixes all 13 clippy warnings in chanora_protocol: collapsed
nested if-let patterns, replaced .ok() + Some matching with Ok, used
? operator, and introduced EventChannels struct to reduce the four
helper functions below the 7-argument threshold.
* fix(voice): use MissedTickBehavior::Skip for level meter stream and align dBFS doc
Set MissedTickBehavior::Skip on the input_level_stream tokio interval
so slow audio_stats() calls skip missed ticks instead of bursting,
preventing CPU spikes on the UI meter thread.
Align VoiceLevelMeter class doc: the mapping floors at -60 dBFS
(via dbfsToFraction), not the full -120 range.
|
||
|
Edison Jwa
|
7d5d8c2c90 | feat: integrate chat voice and diagnostics client | ||
|
Edison Jwa
|
6af4ecab0f | feat(voice): add iOS VAD runtime support | ||
|
Edison Jwa
|
7a59f5b9a1 | feat(ios,p0): iOS P0 platform, audio fixes, channel UX | ||
|
EdisonJwa
|
d4c04b6a72 |
fix(ui,audio,ios,macos): eight P0 mobile fixes
User report from sideloaded iPhone build, in order of priority: #4 'Could not join channel: audio: audio backend: build_output_stream: The requested stream configuration is not supported by the device.' Cause: we forced cpal::BufferSize::Fixed(2048) on the output and input streams unconditionally on non-Linux. iOS CoreAudio RemoteIO units reject arbitrary buffer-size requests with that exact error. Windows WASAPI needs the pinning for shared-mode jitter, but macOS / iOS do not. Fix: cfg-gate Fixed(2048) to target_os = 'windows'; everywhere else use BufferSize::Default and let the platform HAL pick. crates/chanora_audio/src/engine.rs. #5 'Could not join channel: invariant violated: voice_in already taken' Cause: start_audio tore down the old engine BEFORE attempting to construct the new one, and consumed voice_in (an mpsc Receiver that can only be taken once) early. When the new engine failed mid-construction (e.g. because of #4 above) the session was left with: no audio engine, voice_in consumed, no way to retry without reconnect. The second voice_join attempt surfaced the invariant message. Fix: build the new engine BEFORE tearing down the old. Only swap state.audio if construction succeeded. crates/chanora_ core/src/lib.rs::ChanoraSession::start_audio. Additionally added a put_voice_in helper to the protocol adapter ( crates/chanora_protocol/src/adapter.rs) for a future broadcast-channel migration; the helper is unused on the immediate fix path but documents the intent. #3 'permission request would better on first open' Cause: AVAudioSession only triggers the mic-permission prompt the first time it tries to record. We never recorded until voice_join, so the prompt fired then. Fix iOS: AVAudioSession.sharedInstance().requestRecordPermission in AppDelegate.swift::application(_:didFinishLaunchingWithOptions:). Fix macOS: AVCaptureDevice.requestAccess(for: .audio) in macos/Runner/AppDelegate.swift::applicationDidFinishLaunching. Both run non-blocking; user can deny without crashing app launch, and voice_join then surfaces a clearer downstream error when the engine fails to open the input device. #1 + #2 'one-column upper takes too much space; Push to Talk button at bottom would be better' Layout rework for narrow-mode (single column, mobile shape): - Flipped the stacking order in main.dart so Voice Bar moves to the BOTTOM of the body and the channel tree (Expanded) fills above. Wide-mode (Row, >= 840 dp) layout unchanged. - Inside the Voice Bar on touch-only hosts, moved the on-screen Push to Talk button to be the LAST element of the Voice Bar (was Row 3). Order now: pill + mutes, mode badge + settings, level meter, stats line, release-tail caption, PTT button. The button is closest to the user's thumb when the Voice Bar is pinned to the bottom of a narrow-layout screen. #6 'remove right top debug badge' debugShowCheckedModeBanner: false on the MaterialApp. Release builds never showed it anyway; this only affects local dev / debug builds. #7 'what does the refresh button use for? nothing happened' Removed. The snapshot updates via BridgeEvent::SnapshotChanged are pushed from the bridge — a manual rust.snapshot() call was redundant. Now only the Diagnostics + Disconnect actions remain in the AppBar trailing row when connected. #8 'Bind Key related function should not be added to a mobile platform' widgets/voice_settings.dart: bind-key OutlinedButton is now #cfg'd out when Platform.isIOS || Platform.isAndroid. The release-tail slider stays because it still applies to the on-screen PTT button. Capability badge in voice_bar.dart also hidden on mobile (it would always show L0Focused which is redundant with the visible on-screen button). Tests + analyze: chanora_audio 34/0/0 on macOS, workspace 78/0/1 on Linux; flutter analyze clean (6 pre-existing Radio.groupValue infos). flutter build ios --release --no-codesign: 28.8 s clean (Runner.app 29.9 MB). |
||
|
EdisonJwa
|
278df25fd7 |
feat(ui,ptt): on-screen touch-and-hold PTT button for iOS / iPadOS / Android
iOS / iPadOS / Android have no hardware keyboard for the user to
bind a PTT key on. Up to now the VoiceBar showed only a
'Push to talk: bound key —' hint that didn't lead anywhere usable.
Add a touch-and-hold on-screen PTT button rendered only on
touch-only platforms (Platform.isIOS || Platform.isAndroid; web
hosts and desktop continue to use the hardware-key path
unchanged).
apps/chanora_flutter/lib/widgets/voice_bar.dart:
* New module-private `_isTouchOnlyPttHost` predicate.
* VoiceBar gains an `onPttHeldChanged: ValueChanged<bool>`
constructor param. Desktop callers wire it but never invoke it
because the button is not rendered there.
* Row 3 (the PTT-only secondary content) now branches:
- on touch-only hosts -> renders the new `_PttHoldButton` plus
a small release-tail hint underneath
- on hardware-keyboard hosts -> renders the same bound-key +
release-tail one-liner as before, unchanged.
* New `_PttHoldButton` StatefulWidget. Uses a single
GestureDetector covering onTapDown / onTapUp / onTapCancel /
onPanDown / onPanEnd / onPanCancel so the held edges fire on
finger-down and the released edge fires when the user lifts
OR drags off OR another gesture in the arena wins. Visual
feedback mirrors the level-meter active flag.
apps/chanora_flutter/lib/main.dart:
* New `_onOnscreenPttHeldChanged(bool held)` method that calls
`rust.setPtt(active: held)`. The bridge's set_ptt routes the
edge through the same release-tail timer + transmit-mode
selector that desktop hardware keys use (SDD-096 / SAD-083),
so behaviour parity is preserved.
flutter analyze: clean (6 pre-existing Radio.groupValue infos).
flutter build ios --release --no-codesign: clean (Runner.app 29.9 MB).
DEC-025: iPhone + iPad + Android in scope; this commit makes PTT
mode actually usable on those platforms. The 'Focused' capability
badge wording in ios-p0-acceptance.md / ipad-p0-acceptance.md
already documents the on-screen button as the only PTT input;
this commit makes that documentation true.
|
||
|
EdisonJwa
|
25b5bb3a6d |
fix(ui): wide-mode banner placement + channel-pill overflow
Two related VoiceBar / scaffold issues on wide windows: 1. Banner placement The 'not production ready' tertiaryContainer banner sat full-width above the body Column. In wide layouts (>=840 dp) where the connected view splits into Voice Bar (320 dp) + channel tree (Expanded), the banner spanned both columns and dwarfed the channel-tree pane. Rework: wrap the body in an outer LayoutBuilder so the placement decision can read bodyConstraints.maxWidth. When wide AND connected AND snapshot != null, render the banner inside the left 320 dp SizedBox above the VoiceBar. In every other state (narrow, idle, connecting) the banner stays pinned full-width at the top. 2. Channel-name pill overflow The Container holding the channel pill had no width constraint and Text(channelName) had no overflow handling. Long channel names made the pill extend past the column's 320 dp; mute icons slid under the adjacent channel tree. Rework: pill wrapped in Flexible(flex: 100, fit: FlexFit.loose); inner Text gets maxLines: 1, overflow: TextOverflow.ellipsis, softWrap: false. Spacer keeps default flex 1; the 100:1 ratio means short names hug their intrinsic width and long names take ~99% of the remaining space then ellipsize. Mute icons stay pinned right. flutter analyze: clean (6 pre-existing Radio.groupValue infos only). |
||
|
EdisonJwa
|
181b3368d4 |
fix(audio,voice,log): six P0 issues from Korean Windows test
1. Voice Bar 'Leave voice' button removed entirely. TeamSpeak users are always in some channel; Discord/Mumble-style leave is the wrong model. To stop being heard / hearing, mute mic / speaker. To physically move, tap a different channel. The voiceLeave bridge call + _onLeaveVoice stay as dead code for now (marked unused) so existing tests/integrations don't break. 2. voice_join now confirms the move actually applied server-side by polling the snapshot for up to 1.5 s and matching our own client's channel against the requested one. If the server rejected the move (no permission, wrong password, channel full), voice_join rolls the selector back to in_channel=false and returns Err so the UI surfaces the failure instead of showing a fake 'joined' state. 3. ServerSnapshot + BridgeSnapshot gain own_client_id so the UI can identify our row without name-matching. find_own_in reads it directly. 4. set_self_muted now also clamps the TransmitModeSelector's hard_mute when input is muted server-side. Without this, the Opus encoder kept producing frames after setInputMuted(true), tsclientlib refused each one with 'Sending audio while muted', and the log grew to 200 MB on the Korean host. 5. tsclientlib WARN spam suppressed via tracing filter (tsclientlib=error). Belt-and-braces on top of fix 4. 6. Log file is now rotated at every launch (not just when >4 MiB). Two generations kept: chanora.log.1 (previous) and chanora.log.2 (the one before). The bug that produced 200 MB files was a chatty subsystem flooding a single session; the per-launch rotate keeps disk use bounded by what one session can produce in its lifetime. Bonus Windows fix (separate from the six but found in the same log): the Raw Input + Hook backends now signal readiness BEFORE blocking on GetMessageW. Previously init_tx.send was called after the loop returned (i.e. on WM_QUIT, which never happens during arming), so the main thread's 2 s readiness probe always timed out and the backend reported L0Focused even when registration succeeded. Both run_raw_input_loop and run_hook_loop now take an init_tx parameter and call report!(true) right after a successful registration, and report!(false) on every early-fail return. cargo check --workspace: clean. cargo test --workspace --lib: 80 passed / 0 failed / 1 ignored. flutter analyze: clean (6 pre-existing Radio.groupValue infos). |
||
|
EdisonJwa
|
6d4975bd6e |
fix(ptt,ui): Continuous mode no longer self-disables after 30 s; rename PTT label to Mic
Issue 1: in Continuous transmit mode the talk indicator turned
gray-out / mic disabled after ~30 s and could only be revived by
toggling mic mute. Root cause: SAD-079 MissedKeyUpWatchdog
subscribed to AudioTransmitGate.transmit_active and force-cleared
it after 30 s of true. In PTT mode this is correct (stuck key =
bug). In Continuous mode transmit_active is *supposed* to stay
true indefinitely; the watchdog assumption doesn't hold.
Fix: the watchdog now subscribes to a new ptt_held watch on the
TransmitModeSelector (the raw key-state input, not the resolved
gate). In Continuous mode ptt_held is never set true, so the
watchdog never fires. In PTT mode it still fires on a stuck
key-down as before. The session owns the watchdog (was on the
engine) so it survives engine restarts; it's spawned lazily on the
first start_audio.
MissedKeyUpWatchdog gains spawn_on_signal(rx, on_timeout, timeout)
alongside the existing spawn(gate, timeout) — old shape preserved
for backwards compat. run_watchdog generalised to take any
watch::Receiver<bool> + Box<dyn Fn() + Send + Sync>.
Two new tests:
- watchdog_on_signal_does_not_fire_when_ptt_held_stays_false
(the Continuous-mode regression test)
- watchdog_on_signal_fires_when_signal_stays_true
(the stuck-key case still fires)
Issue 2: the Voice Bar stats line said 'PTT on/off' even when the
user was in Continuous mode where no PTT key is involved. Renamed
to 'Mic on/off' (mode-neutral) and l10n-ised the on/off literal:
- en: 'Mic on' / 'Mic off'
- zh: '麦克风 开启' / '麦克风 关闭'
cargo test --workspace --lib: 80 passed / 0 failed / 1 ignored
(was 78, +2 watchdog tests).
flutter analyze: clean (6 pre-existing Radio.groupValue infos).
|
||
|
EdisonJwa
|
82b99ebcff |
feat(ui,voice): add speaker mute button to VoiceBar
Speaker (output) mute existed in the legacy _AudioControls widget and the rust.setOutputMuted bridge call but was lost when SDD-097 replaced _AudioControls with VoiceBar. Mic mute carried over; speaker mute did not. Wire it back: VoiceBar gains an outputMuted prop + onToggleOutputMute callback and renders a headset/headset_off icon next to the existing mic mute. main.dart wires the existing _toggleOutputMute handler (previously dead-code with // ignore: unused_element). The bridge call setOutputMuted already does both effects together: local engine silencer + server-broadcast ClientOutputMuted flag. l10n: rename voiceHardMuteLabel to 'Mute microphone'/'麦克风静音' to distinguish from the new voiceOutputMuteLabel 'Mute speakers'/ '扬声器静音'. flutter analyze: clean (6 pre-existing Radio.groupValue infos). |
||
|
EdisonJwa
|
64878e3a7d |
fix(ui,voice): consolidate to single Voice settings entry point
The capability badge had its own 'Configure' TextButton that opened the bind-key flow, while the Voice Bar's settings gear also reached bind-key through the settings dialog. Two paths, same destination — confusing and pointless duplication that the user flagged. Resolution: the gear is the only configuration entry point. The capability badge becomes information-only — it still shows the detected PTT level + backend and (for L0Focused) the info-icon explanation sheet, but no Configure button. The badge no longer takes or props. The Voice Bar drops the callback added in |
||
|
EdisonJwa
|
6a41a0b4db |
fix(ui,voice): five Voice Bar / settings UX bugs
1. Hard-mute now informs the server (setInputMuted) in addition to clamping the local TransmitGate. Without the server-side flag, other clients keep seeing us un-muted; without the local clamp a beat of in-flight audio leaks through. Drive both together so the mic icon and the actual silence land at the same time. 2. Split the badge's Configure affordance from the Voice Bar's 'Voice settings' gear. The gear opens the mode + release-tail dialog (onConfigure); the badge's configure opens the bind-key capture flow directly (new onBindKey). Previously both routed to the settings dialog, so 'Voice settings' and the badge's 'Configure' were the same screen — useless duplication. 3. Bind-key label is now PTT-only. The mode-badge row no longer prints 'PTT: Space' when Continuous / Voice Activity is selected. A new PTT-only secondary line carries the bound key plus the release-tail value together, hidden entirely for non-PTT modes. 4. Release-tail row is now PTT-only in BOTH the Voice Bar and the Voice settings dialog. The dialog previously kept the slider visible across all modes; switching to Continuous left the user staring at a control that did nothing. 5. PTT capability badge is now PTT-only. In Continuous and Voice Activity modes there is no key binding to surface a capability for, so the 'L0Focused (focused)' line + its info sheet and the Configure button disappear from the Voice Bar when the user isn't in PTT mode. All five fixes are pure UI; no Rust changes needed. flutter analyze remains clean (6 pre-existing Radio.groupValue deprecation infos). |
||
|
EdisonJwa
|
ba444d94bd |
feat(audio,bridge,flutter): v1 audio + PTT lifecycle implementation (SDD-094..097)
Implement the SDD-094 / SDD-095 / SDD-096 / SDD-097 detailed designs
committed in
|