Clarify ONNX Runtime guidance with direct-open install hints, restore desktop WebRTC VAD visibility, map mouse side buttons through focused PTT capture/runtime paths, and wait for server acks before showing chat sends as successful.
Constraint: Linux release UX must stay functional when ONNX Runtime is optional and GNOME portal availability varies
Rejected: Keep desktop VAD locked to Silero only | misleads users when ONNX Runtime is skipped
Confidence: medium
Scope-risk: moderate
Directive: Preserve the protocol send-ack wait path for chat so UI success always tracks real server acceptance
Tested: flutter analyze lib/main.dart lib/widgets/chat_views.dart lib/widgets/input_dialogs.dart lib/widgets/startup_dependency_screen.dart; flutter test test/widgets/input_dialogs_test.dart test/widgets/chat_views_test.dart test/services/startup_dependency_check_test.dart test/widgets/startup_dependency_screen_test.dart test/widgets/voice_settings_controls_test.dart test/widgets/audio_processing_config_state_test.dart; cargo test -p chanora_protocol --lib; cargo test -p chanora_audio ptt_backends --lib
Not-tested: Live manual GNOME portal rebind/global PTT on a real desktop session; observer-bot chat against a live server after the sender-name fallback change
Add structured NSLog instrumentation to AppDelegate.swift and debugPrint
chains in voice_compact.dart::_AudioOutputPickerSheetState so we can
correlate user picker taps with what iOS actually does to the route.
Three diagnostic streams:
* 'chanora.session[<tag>]' from Swift — full session snapshot (category,
mode, sampleRate, ioBufferDuration, current route inputs+outputs,
preferredInput) emitted on every setActive and every
AVAudioSession.routeChangeNotification with the reason decoded
(override / routeConfigurationChange / newDeviceAvailable / etc).
* 'chanora.route[<tag>]' from Dart — current route's inputs+outputs
emitted before/after every overrideOutputAudioPort or
setPreferredInput call, plus a delayed re-check at +250 ms to detect
silent reverts.
* Existing 'chanora: ...' debugPrint lines from the picker now include
the OK case (override returned, setPreferredInput returned) so we see
a positive signal in the log when the API didn't throw.
Used to root-cause the 'speaker selector not working' issue: the
hypothesis is that cpal's RemoteIO AudioUnit reacts to its own format
configuration notifications by triggering routeConfigurationChange
that reverts our Dart-side override. The logs will confirm or deny
this — if we see 'chanora.session[routeChange.override] out=Speaker'
followed by 'chanora.session[routeChange.routeConfigurationChange]
out=Receiver' within a few hundred ms, that's the smoking gun.
Pure diagnostic commit. No behavioural change. Logs are NSLog +
debugPrint so they appear in Xcode console / 'flutter logs' / the
device log via Console.app or 'devicectl device log'.
User report: 'speakerphone (built-in mic input)' logs printed
successfully but audio output didn't actually switch to speaker.
No exception thrown by either AVAudioSession call.
Root cause:
Previous _selectSpeaker called:
1. await overrideOutputAudioPort(.speaker)
2. await setPreferredInput(builtInMic)
Apple-documented behavior in .voiceChat mode: when
setPreferredInput is called, iOS recalculates the entire route
based on the natural input/output pairing for the selected port.
Built-in mic's natural output pairing is the receiver/earpiece
(matches the 'I'm talking on a phone' UX of .voiceChat). So step 2
caused iOS to SILENTLY REVERT the speaker override from step 1
and route output back through the earpiece.
Net: await chain returned without exception (both calls 'succeeded'
in API terms), debugPrint logged the success message, but the user
heard the call audio coming out of the earpiece, not the speaker.
Same issue affected _selectReceiver (after the speaker bug fix
was reverted): redundant setPreferredInput(builtInMic) call risked
the same recalc race.
Fix:
* _selectSpeaker: only call overrideOutputAudioPort(.speaker).
No setPreferredInput. The override alone is sufficient \u2014 the
input stays on whatever the system was already using (built-in
mic by default, or BT/wired if connected).
* _selectReceiver: only call overrideOutputAudioPort(.none).
No setPreferredInput. Removing the speaker override naturally
returns to .voiceChat's default route (receiver).
* _selectInput (BT / wired / USB): unchanged. These inputs PAIR
their own output device, so .none + setPreferredInput is the
correct combo (user hears audio through the same device they
speak into).
flutter build ios --release --no-codesign: 21.2 s, Runner.app
30.4 MB.
User report: 'speaker change not work' \u2014 selecting Speaker or
iPhone receiver in the audio output picker had no audible effect.
Root cause:
AVAudioSession was configured with category options
[.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP] +
mode .voiceChat. The .defaultToSpeaker flag tells iOS 'this app's
baseline output route is the speakerphone, even though .voiceChat
mode would normally route to the receiver.'
When the user picked Speaker:
overrideOutputAudioPort(.speaker) <- already at speaker baseline; no-op
When the user picked iPhone receiver:
overrideOutputAudioPort(.none) <- removes speaker OVERRIDE,
restores baseline = .defaultToSpeaker
= speakerphone. Receiver
row silently mapped to speaker.
So both rows produced the same audible state. The picker UI changed
the selected radio but the route didn't actually move.
Fix:
1. AppDelegate.swift: drop .defaultToSpeaker from options. With
pure .voiceChat mode (no .defaultToSpeaker), the baseline is
the receiver/earpiece. overrideOutputAudioPort then works as
documented:
Default = receiver
overrideOutputAudioPort(.speaker) -> speakerphone
overrideOutputAudioPort(.none) -> back to receiver
BT/AirPods connected -> automatic
Wired headphones plugged in -> automatic
2. voice_compact.dart: replace 'catch (_) {/* ignore */}' silent
swallow with debugPrint logging of (a) the actual exception
and (b) which route was selected. So if iOS rejects an
override (e.g. wired headphones plugged in), we can see WHY
in the device log instead of a silent picker no-op.
3. _selectSpeaker also now sets preferredInput to the built-in
mic so input + output stay consistent. Previously the
speakerphone override could leave the mic still routed to a
previously-selected BT input \u2014 user hears self through
speaker but server hears nothing.
flutter build ios --release --no-codesign: 21.9 s, Runner.app
30.4 MB.
User reported: the 'output device' picker only listed AirPlay
destinations (other iPhones / AirPlay speakers / AppleTV) and not
the speaker / iPhone receiver / AirPods / wired headset choices.
Root cause: audio_router 1.1.1's iOS path uses AVRoutePickerView,
which is Apple's **AirPlay** picker UI \u2014 by design it only lists
AirPlay-eligible output destinations, NOT the input/output route
choices we need (speaker vs receiver vs Bluetooth HFP vs wired).
AVRoutePickerView is the right UI for 'cast audio elsewhere'; for
'pick how I hear / talk' (VoIP) the right primitive is direct
AVAudioSession calls.
Fix: replace audio_router with audio_session 0.2.3 (Ryan Heise,
verified publisher, 865k downloads, MIT). audio_session exposes:
* AVAudioSession.availableInputs \u2014 enumerate every real input
port: builtInMic, bluetoothHfp, bluetoothA2dp, headsetMic
(wired), usbAudio, carAudio, airPlay.
* AVAudioSession.currentRoute \u2014 .inputs + .outputs of the
active route.
* AVAudioSession.setPreferredInput(port) \u2014 switch the input
(HFP / wired / USB / car audio also move output to themselves).
* AVAudioSession.overrideOutputAudioPort(.speaker | .none) \u2014
toggle built-in speakerphone vs receiver/earpiece.
* AVAudioSession.routeChangeStream \u2014 live notifications when
the user plugs / unplugs / connects a device while the picker
is open.
This is exactly the same primitive Discord, WhatsApp, FaceTime
use for their VoIP audio chooser. No native UI plugin needed.
New widgets in voice_compact.dart:
* _AudioOutputTile: shows the active output port name (Speaker /
iPhone / AirPods / 'Phil's Wired Headset' / etc.) with the
matching icon. Subscribes to routeChangeStream for live
updates. Tap opens _AudioOutputPickerSheet.
* _AudioOutputPickerSheet: bottom sheet with 'Choose audio' title
and a Discord-style list:
- Speaker (volume_up)
- iPhone (phone_in_talk; the receiver/earpiece)
- <BT name> (bluetooth_audio)
- <Wired headset> (headset)
- <USB / Car> (usb / directions_car)
Selected row is highlighted + has a check mark. Tap routes:
- Speaker -> overrideOutputAudioPort(.speaker)
- iPhone -> overrideOutputAudioPort(.none) + setPreferredInput(builtInMic)
- External -> overrideOutputAudioPort(.none) + setPreferredInput(port)
* _PickerRow: shared row widget with selected/check styling.
AppDelegate.swift is unchanged: the manual AVAudioSession
.setCategory(playAndRecord / .voiceChat) we already do at launch
(0466000 / 4ee2b38) is fully compatible with audio_session \u2014 the
plugin only adds Dart-side accessors over the same underlying
AVAudioSession singleton.
Removed l10n keys not used anymore (audioRouteUsb was already gone).
Kept audioRouteSpeaker / Receiver / Bluetooth / WiredHeadset /
CarAudio / Airplay / Unknown \u2014 all still used by the new picker.
flutter analyze: 6 pre-existing Radio deprecation infos (unchanged).
flutter build ios --release --no-codesign: 54.9 s, Runner.app
30.4 MB (+200 KB vs audio_router build).
Three user-reported issues addressed at once.
1. Audio output displaying as Unknown on iOS
The route tile only set _device from currentDeviceStream events,
which fire on route *changes*. On first sheet open with no route
change yet, _device was null \u2192 _deviceLabel fell through to
audioRouteUnknown.
Fix: query AudioRouterPlatform.instance.getCurrentDevice() in
initState before attaching the stream listener. Plugin returns the
current AVAudioSession route synchronously (well, via Future) so
the tile renders Speaker / iPhone receiver / AirPods / etc.
immediately on first open. Errors swallowed \u2014 the stream remains
authoritative for subsequent updates.
2. 'Adjust mode & release tail' too deep (chip \u2192 modal \u2192 button \u2192 dialog)
Inlined the mode radio buttons and release-tail slider directly
into the voice modal sheet. Dropped the OutlinedButton 'Adjust'
trigger and the nested VoiceSettingsDialog dispatch entirely on
mobile.
Modal sheet is now a single-screen control panel:
Title 'Voice'
--------
Audio output: <current route> > (iOS/Android only)
--------
Transmit mode
\u25c9 PTT
\u25cb Continuous
\u25cb Voice activity (Coming soon) (disabled)
--------
Release tail 200 ms
[\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u25cf\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501] (0\u20131000 ms, step 50)
Bound key: F (desktop only)
--------
Level meter
TX/RX frame counts
PTT capability badge (desktop only)
VoiceSettingsDialog is retained for the wide-mode VoiceBar
'configure' button (desktop entrypoint) and the PTT-bind flow, so
desktop UX is unaffected.
New widgets: _VoiceSheetBody (StatefulWidget with local _mode +
_tail), _ModeRow (RadioListTile-shaped row with optional disabled
state for VoiceActivity). New API on showVoiceDetailsSheet:
onModeChanged + onReleaseTailChanged callbacks (replace
onAdjustVoiceSettings). Wiring in main.dart writes through to
rust.setTransmitMode / rust.setReleaseTailMs and mirrors _state.
l10n: dropped voiceAdjustSettings (en + zh). Added voiceBoundKeyLabel
(en + zh) for the desktop-only bound-key row.
3. iOS first-tap-keyboard regression (flutter/flutter#181474)
The 79f8360 _kickFocus workaround (unfocus + microtask refocus on
every TextField.onTap) was kept, but extended with a tap-outside-
to-unfocus GestureDetector wrapping the connect form Column. This
guarantees the FocusNode is in the unfocused state when the next
field tap arrives, so the focus transition is always false\u2192true
on first tap.
GestureDetector(HitTestBehavior.translucent, onTap: unfocus) is the
canonical pattern recommended in the flutter/flutter#181474 thread
+ several older iOS keyboard issues. Translucent behaviour means it
catches taps on the column padding / empty regions without
swallowing taps on the TextFields themselves (those have
onTap: _kickFocus already).
flutter analyze: 6 pre-existing Radio.groupValue deprecation infos
in voice_settings.dart (unchanged). flutter build ios --release
--no-codesign: 27.9 s, Runner.app 30.2 MB (unchanged).
Awaiting iPhone retest to confirm all three fixes.
Two user-reported issues addressed:
1. 'on supported devices such as iphone or android user should be
able to select audio device such as speaker or airpods or phone'
2. 'the bottom folder is duplicated with the app bar settings'
Issue 1 \u2014 audio output route picker:
Added the audio_router 1.1.1 plugin (MIT, supports iOS + Android)
which renders the platform-native picker:
* iOS: Apple AVRoutePickerView system sheet \u2014 the same UI as
Control Center's audio chooser. Lists Speaker / iPhone receiver /
AirPods / connected Bluetooth devices / AirPlay / CarPlay.
System manages the device list; we don't have to track route
changes manually.
* Android (post-rc.8 when we wire the platform): Material Design 3
dialog backed by AudioManager.setCommunicationDevice() with
SCO Bluetooth + USB headsets filtered for VoIP.
The picker prerequisite documented by the plugin (audio session
must be playAndRecord/voiceChat before the picker fires) is already
satisfied by our AppDelegate.swift configuration from commit 0466000.
The new _AudioOutputTile widget in voice_compact.dart subscribes to
AudioRouter.currentDeviceStream so the row label + icon auto-update
when the user plugs in headphones, connects AirPods, etc. \u2014 no
manual KVO observation needed.
Tile is mobile-only (Platform.isIOS || Platform.isAndroid). Desktop
hosts continue to use the system mixer; the tile is hidden.
Plugin caveat: the published enum AudioSourceType has no .usb
variant (despite README mentioning USB support). We map only the
seven real enum cases: builtinSpeaker / builtinReceiver / bluetooth /
wiredHeadset / carAudio / airplay / unknown.
Issue 2 \u2014 collapse voice controls into the single modal sheet:
The AppBar gear icon (Icons.tune) that opened VoiceSettingsDialog
was removed. It duplicated the configuration entry point that the
status chip \u2192 modal-sheet path already provides, and the user found
that duplication confusing on a phone-narrow screen where AppBar
real estate is precious.
The voice modal sheet (showVoiceDetailsSheet) is now the **single**
voice-controls surface on mobile, with layout (top to bottom):
1. Audio output route picker tile (iOS / Android only) \u2014 new.
2. Mode + bind / release-tail recap (display only).
3. 'Adjust mode & release tail' OutlinedButton that closes the
sheet and opens the same VoiceSettingsDialog the gear icon
used to open. One config form, not two.
4. Mic level meter.
5. TX / RX frame counts + mic state.
6. PTT capability badge (desktop only).
Sheet title renamed from 'Voice settings' (which collided with the
gear-icon tooltip) to 'Voice'. New l10n keys: voiceSheetTitle,
voiceAdjustSettings, audioOutputLabel, audioRoute{Speaker,Receiver,
Bluetooth,WiredHeadset,CarAudio,Airplay,Unknown}. en + zh translated.
Build: flutter build ios --release --no-codesign clean, 50.5 s,
Runner.app 30.2 MB (+200 KB from audio_router). flutter analyze
clean (6 pre-existing Radio.groupValue deprecation infos in
voice_settings.dart, unchanged).
Restructure the narrow / mobile body layout around three principles
(Hoober thumb-zone research validated, Material 3 components):
1. Channel tree gets ~85% of the screen height.
2. Live voice state is always visible in a 2-line status chip
above the PTT button.
3. PTT button is wide, bottom-anchored (thumb-natural lower zone).
4. Frequent toggles (mic mute, headset mute, voice settings) live
in the AppBar so they don't compete with the channel tree.
5. Non-essential live readouts (level meter, TX/RX, capability
badge) live in a modal sheet opened by tapping the status chip
-- progressive disclosure.
apps/chanora_flutter/lib/widgets/voice_compact.dart (new file)
* VoiceStatusChip: 2-line live readout. Line 1 = mode + bind hint;
Line 2 = release-tail + 'Mic on/off'. Tap opens
showVoiceDetailsSheet.
* VoicePttButton: 56 dp wide bottom-anchored Push to Talk button.
Same touch-and-hold gestures as the previous _PttHoldButton.
* showVoiceDetailsSheet: modal bottom sheet with mode recap +
bind/tail hint + level meter + TX/RX counts + PTT capability
badge (desktop-only).
apps/chanora_flutter/lib/main.dart
* AppBar gains mic-mute, headset-mute, voice-settings icons when
in voice channel AND MediaQuery width < 840 dp (mobile only).
Wide mode keeps these controls inside the existing VoiceBar
widget unchanged.
* AppBar title becomes a Row of 'app name + channel chip' when
in voice channel.
* Narrow-mode body restructured: Expanded(channelTree) +
VoiceStatusChip + VoicePttButton (latter only when in voice
channel AND PTT mode). The old narrow-mode VoiceBar is gone;
wide-mode VoiceBar is unchanged.
* New _onOpenVoiceDetailsSheet handler bridges the chip-tap to
showVoiceDetailsSheet.
* New top-level helper _isTouchOnlyPttHost mirrors the helpers
in widgets/voice_bar.dart and widgets/voice_settings.dart so
the AppBar + narrow-mode chip can branch consistently.
apps/chanora_flutter/lib/l10n/app_en.arb
apps/chanora_flutter/lib/l10n/app_zh.arb
apps/chanora_flutter/lib/l10n/generated/* (regenerated)
* New string voicePttHoldHint = 'Hold the button' / '按住按钮'.
Surfaced in line 1 of VoiceStatusChip on touch-only hosts and
in the modal sheet's PTT line where the desktop equivalent
would name a bound key.
Wide-mode (>= 840 dp) layout intentionally unchanged so the
signed-off rc.8 desktop verification still applies.
flutter analyze: clean (6 pre-existing Radio.groupValue infos).
Local cargo + flutter analyze pass; Mac was offline at commit
time so iOS device build verification is pending the next sync.
Edison Jwa