b20e6b663a8f2d7e84bc261f9aad0de213d94ad8
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edison Jwa
|
b20e6b663a |
fix(audio): replace Mutex unwrap with poisoned-mutex recovery (TODO-006)
Replace 42 .lock().unwrap() calls with .unwrap_or_else(|e| e.into_inner()) across 7 files. Poisoned mutex recovery prevents panics in realtime audio callbacks. Add SAFETY comment to WebRtcFallbackVad Send impl (TODO-007). |
||
|
Edison Jwa
|
2f6d45fb04 |
feat(audio): desktop Silero ONNX VAD + Windows PTT modernization + MSVC CRT build fix (#37)
* feat(audio): add Silero ONNX VAD with WebRTC fallback Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active. * feat(audio): integrate desktop VAD worker into capture engine Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding. Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default. Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self). * fix(audio): modernize Windows PTT to current windows-rs API Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate. * build(windows): force MSVC release CRT for audiopus cmake builds audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link. Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK. * build(flutter): update pubspec.lock after plugin additions Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature. * fix(audio): address PR #37 review findings Six fixes from independent PR review: 1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd globally broke non-Windows hosts because cmake-rs would try to execute a .cmd file on macOS/Linux. Instead, set CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY= MultiThreadedDLL as env vars that CMake reads natively. MSVC- specific vars are safely ignored by GCC/Clang toolchains. Delete the now-unnecessary wrapper script. 2. IMPORTANT: Join the Silero worker thread in Drop instead of detaching it. The old code dropped the JoinHandle which detaches the thread; the new code calls handle.join() after closing the channel, ensuring the ONNX session is cleaned up before the worker is replaced during config changes. 3. IMPORTANT: Single-try_lock refactor of the capture VAD callback. The double try_lock (policy probe + send) is replaced by a single scoped try_lock that both probes availability and sends the frame. The guard is dropped before the fallback path, which needs &mut self for mark_vad_fallback_active. This also eliminates the VadWorkerPolicy enum and callback_vad_worker_policy function, whose behavior is now inlined into the callback. 4. IMPORTANT: Remove tracing from the realtime capture callback. mark_vad_fallback_active and sync_vad_backend emitted info!/warn! from the audio thread. Replace with silent atomic state publishing via SharedAudioProcessingStats; the bridge stats stream already exposes vad_fallback_active for diagnostics. 5. IMPORTANT: Defer ONNX model load outside the worker mutex. apply_desktop_vad_backend_to_worker now constructs the new worker before taking the lock, then swaps it in under a short hold. This prevents the realtime callback from being blocked during model I/O + thread spawn. 6. MINOR: Remove unused VadBackend import from vad/mod.rs after deleting the policy code. * fix(audio): address PR #37 second-pass review findings 5-agent review found 5 blocking issues. All addressed: 1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc] and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows hosts are unaffected. 2. BLOCKER: processing_backend normalized in set_audio_processing_config on desktop (cfg-gated override to Noop), mirroring startup default. 3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config. Fixed misleading doc comment in core/lib.rs. 4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement. Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated. 5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android). Cargo.toml ort dependency target cfg narrowed similarly. 6. Realtime callback debt documented as TODO at CaptureState::ingest. * fix(audio): exclude ort dep on Android target ort does not provide first-class Android prebuilts in our pin, mirror the iOS/macOS exclusion so cargo metadata succeeds for android targets. * test(audio): fix stale select_ptt_backend import in ptt_privacy The helper moved out of the ptt_backends submodule onto the crate root; update the integration test imports so the test compiles again. * build(windows): scope MSVC release CRT cmake wrapper via Cargo [env] Cargo's [target.<triple>] table only forwards a fixed allowlist (linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there was silently dropped and audiopus_sys kept linking the debug CRT, producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors on x86_64-pc-windows-msvc test builds. Move the override to Cargo's [env] table using cc/cmake-rs's target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it applies to MSVC targets only and not to host tooling. Add stdout markers to the wrapper so its invocation is provable in cargo -vv logs. Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc --lib --no-run now links cleanly; CMakeCache.txt records CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW. * fix(flutter): gate VoiceActivity transmit mode by platform support VoiceActivity relies on the native VAD worker, which is only wired up on Windows, Linux, and Android. Showing the option on iOS, macOS, or web let users select a mode that silently never transmitted. Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers in voice_settings_controls.dart, hide the VAD row in voice_compact.dart and drop the VAD segment from the settings dialog when unsupported. Keep the legacy const transmitModeSegments for the existing widget test and add two new tests covering the gated helper. |
||
|
Edison Jwa
|
8c4f85ee70 | refactor: reuse built-ins and shared helpers | ||
|
Edison Jwa
|
fe6e07353e | chore: restore product scaffold to rollback baseline | ||
|
Edison Jwa
|
d03ea937e6 | fix: refine linux voice runtime behavior | ||
|
Edison Jwa
|
b8df25a195 |
Reduce Linux setup ambiguity and surface desktop input/message failures honestly
Clarify ONNX Runtime guidance with direct-open install hints, restore desktop WebRTC VAD visibility, map mouse side buttons through focused PTT capture/runtime paths, and wait for server acks before showing chat sends as successful. Constraint: Linux release UX must stay functional when ONNX Runtime is optional and GNOME portal availability varies Rejected: Keep desktop VAD locked to Silero only | misleads users when ONNX Runtime is skipped Confidence: medium Scope-risk: moderate Directive: Preserve the protocol send-ack wait path for chat so UI success always tracks real server acceptance Tested: flutter analyze lib/main.dart lib/widgets/chat_views.dart lib/widgets/input_dialogs.dart lib/widgets/startup_dependency_screen.dart; flutter test test/widgets/input_dialogs_test.dart test/widgets/chat_views_test.dart test/services/startup_dependency_check_test.dart test/widgets/startup_dependency_screen_test.dart test/widgets/voice_settings_controls_test.dart test/widgets/audio_processing_config_state_test.dart; cargo test -p chanora_protocol --lib; cargo test -p chanora_audio ptt_backends --lib Not-tested: Live manual GNOME portal rebind/global PTT on a real desktop session; observer-bot chat against a live server after the sender-name fallback change |
||
|
Edison Jwa
|
6af4ecab0f | feat(voice): add iOS VAD runtime support | ||
|
Edison Jwa
|
7a59f5b9a1 | feat(ios,p0): iOS P0 platform, audio fixes, channel UX | ||
|
EdisonJwa
|
383b707e7c |
feat(audio,macos): live CGEventTap PTT capture (SDD-085)
Replace the macOS PTT backend's worker-thread stub (which had just
slept) with a full CGEventTap implementation:
* extern "C" bindings to CGEventTapCreate, CGEventGetIntegerValueField,
CGEventTapEnable, CFMachPortCreateRunLoopSource, CFRunLoopGetCurrent,
CFRunLoopAddSource/RemoveSource, CFRunLoopRun/Stop, CFRelease, plus
the kCFRunLoopCommonModes static.
* tap_callback: C-ABI extern fn that reads bound keycode /
mouse-button from atomics, matches the incoming event, and toggles
the AudioTransmitGate. Returns the event unchanged (listen-only
tap, no event modification). Privacy-safe: never logs raw key
codes or button numbers (DEC-027).
* Event mask covers kCGEventKeyDown, kCGEventKeyUp,
kCGEventOtherMouseDown, kCGEventOtherMouseUp; also handles the
kCGEventTapDisabledBy{Timeout,UserInput} notifications by logging
a degraded-mode warning.
* Worker thread captures the CFRunLoopRef via a Send-marked
RunLoopHandle newtype so stop() can call CFRunLoopStop from the
audio engine thread.
* Box<TapState> is leaked into the worker via a Send-marked
TapStatePtr newtype; reclaimed on worker exit so the gate's Arc
refcount stays correct.
* Flutter logical-key labels are mapped to Carbon virtual keycodes
via label_to_macos_keycode (covers letters, digits, function keys,
navigation, common punctuation). Mouse-side-button labels resolve
via label_to_macos_mouse_button (3 = Mouse4, 4 = Mouse5).
* refresh_bound_atomics() rebuilds bound_keycode + bound_mouse_button
on start() and rebind() so the tap callback sees the new binding
without re-arming the tap.
Tests: chanora_audio 34 / 0 / 0 on macOS (was 28 before this commit).
Added: keymap_letters, keymap_function_keys, keymap_navigation,
keymap_unknown_returns_none, mouse_button_map, runloop_handle_is_send.
Verified the live IOHIDCheckAccess returns Undetermined (Unknown=2) on
a fresh M1 box where Input Monitoring has never been requested; the
1.5 s permission-watcher re-publishes the descriptor on user grant
or revoke without restart.
DEC-025: macOS desktop officially in scope. SAD-073: two-level PTT
ladder. SRS-198: honest capability advertising. DEC-027: privacy.
|
||
|
EdisonJwa
|
ade50488d9 |
feat(audio,macos): live IOHIDCheckAccess for Input Monitoring permission
Replaces the macOS PTT backend's query_permission() stub (which had
returned Undetermined unconditionally) with a real IOKit call:
extern "C" { fn IOHIDCheckAccess(request_type: u32) -> u32; }
IOHIDCheckAccess(kIOHIDRequestTypeListenEvent = 1)
Returns Granted (0), Denied (1), or Unknown (2). The existing 1.5 s
re-query worker now drives real descriptor transitions when the user
grants or revokes Input Monitoring in System Settings: the watch
sender republishes the descriptor, ChanoraSession forwards
BridgeEvent::PttCapability, and the Flutter capability badge updates
within ~1.5 s without an app restart.
Verified live on the M1 Mac:
rustc /tmp/check_perm.rs && ./check_perm
IOHIDCheckAccess(ListenEvent) = 2 (Unknown)
This is the expected initial state on a fresh box where Chanora has
not yet attempted CGEventTapCreate; once the next commit lands the
event-tap worker, the macOS Input Monitoring prompt will fire on
first audio start and the value transitions to Granted/Denied.
Tests: chanora_audio 28 / 0 / 0 on macOS (Linux had 32; the 4-test
delta is the Linux-only portal probe tests). The existing 7 macOS
backend unit tests still cover the descriptor builder + state
machine purely; they don't exercise the live IOKit call (which
would need a TCC-aware test harness).
SDD-085 reference: macOS Event Tap backend / L2 / L3 capability;
SRS-198 honest capability advertising.
|
||
|
EdisonJwa
|
73066749e3 |
fix(audio,linux): isolate zbus blocking probe on a fresh OS thread
LinuxGnomeWaylandBackend::probe() called zbus::blocking::Connection::session() directly. The blocking facade internally constructs a current-thread tokio runtime and block_on()s its async D-Bus client. probe() runs from PttController::new (sync) which is called from start_audio (async on the bridge tokio runtime). Nested runtimes panic with 'Cannot start a runtime from within a runtime'. Symptom on Linux: the first voice-channel join surfaced a SnackBar 'Could not join channel: join: task N panicked ...' while the channel-move command had already succeeded server-side. User saw 'channel joined but voice not enabled'. Fix: run the cheap blocking probe on a dedicated std::thread (no ambient runtime), join it synchronously, propagate the version / error. Probe is microseconds; the join cost is negligible. |
||
|
EdisonJwa
|
2511b24982 |
fix(audio,protocol,ui): TC-2.3 + TC-10 + TC-13 + channel tree hierarchy
Five user-reported defects + one auto-test regression-catcher.
== TC-2.3: capability badge stuck at L0Focused on Korean Win 11 ==
Root cause: in WindowsRawInputBackend::start() (and the parallel
WindowsHookBackend), the worker thread's armed.store(ok, ...) only
ran AFTER GetMessageW returned (i.e. on WM_QUIT). During normal
arming the message pump runs forever, so armed stayed at its
initial false value, and descriptor() reported L0Focused even
though RegisterRawInputDevices had succeeded.
Fix: run_raw_input_loop and run_hook_loop now take armed as a
parameter and flip it to true inside the loop right after the
successful registration, before blocking on GetMessageW. The
outer setter is kept as a belt-and-braces clear-on-failure path.
Also drops the redundant outer 'Raw Input armed' / 'low-level
hook armed' log lines — the in-loop 'Raw Input devices
registered' / 'low-level hooks installed' messages already
convey arming success with full context.
New tests raw_input_backend_start_flips_armed_to_l2 and
hook_backend_start_flips_armed_to_l2 call real start(), sleep
80 ms, assert descriptor().level == L2GlobalHoldToTalk. Replaces
the previous #[ignore]'d real-start smoke test which never
asserted on the descriptor.
== TC-10: no-permission channel rejection invisible ==
Root cause 1: chanora_protocol::adapter::move_self_to used the
fire-and-forget send() on the client_move command. TS3 server
replies with a typed error event the adapter discarded, and
move_to_channel returned Ok regardless.
Root cause 2: even when chanora_core::voice_join detected the
non-confirmation via snapshot polling, the rolled-back error
flowed into the connect-form-area _error string which is hidden
post-connect. The user saw no feedback.
Fix:
* New ProtocolError::ServerRejected { code: u32, message: String }
carries the canonical TS3 error code per the official catalogue
at https://github.com/ReSpeak/tsdeclarations (Errors.csv).
* move_self_to now uses send_with_result, returns a MessageHandle.
The connection-task loop holds a pending_moves HashMap keyed by
MessageHandle, services StreamItem::MessageResult by looking up
and resolving the reply with either Ok or the typed
ServerRejected.
* Pending entries have a 3 s deadline so a server that never
replies doesn't leak the reply channel — expired entries fall
back to Ok and let the snapshot poll handle confirmation.
* voice_join short-circuits on ServerRejected (no need for the
full snapshot poll), still polls for confirmation as a
belt-and-braces fallback for legacy servers; on poll failure
emits ServerRejected with sentinel code 0x0001 (undefined).
* New BridgeError::ServerRejected mirror with the same fields;
CoreError → BridgeError mapping preserves the typed variant.
* Flutter _onJoinChannel shows a floating SnackBar with a
localised message selected by error code (channelJoinFailed*
l10n entries). 6 known codes mapped to specific messages
(insufficient permission, wrong password, channel full,
family limit, private channel, timeout); everything else
falls back to the server-supplied generic message.
== TC-13: mouse side-button capture only works on text field ==
The _PttBindingCaptureDialog wrapped its Column with a Listener
using the default HitTestBehavior.deferToChild. Pointer events
landing on the dialog's empty padding regions weren't claimed by
any child and so were never delivered to the Listener.
Fix: explicit HitTestBehavior.opaque so the entire dialog area
catches PointerDown events regardless of where the cursor sits.
== Channel tree hierarchy ==
Reported issue: tree rendered as flat list, no indication of
parent-child nesting. The bridge already carries the
field; the renderer just ignored it.
Fix in _SnapshotView: walk the (already DFS-sorted) channel list
and compute each row's depth from its parent's depth. Render
left-padding of depth * 18 dp. Cap depth at 6 to keep deep
hierarchies visually bounded; the cap plateaus silently (no
glyph, channel still tappable, data carries the real depth).
== Responsive layout ==
Connected layout is now LayoutBuilder-driven. Below 840 dp wide
(Material's tablet/desktop breakpoint) the original stacked
column layout is used (Voice Bar on top, channel tree below).
At 840 dp and above the layout becomes a side-by-side Row with
the Voice Bar pinned at 320 dp on the left and the channel tree
Expanded on the right.
Verification
- cargo check --workspace: clean.
- cargo test --workspace --lib: 80 / 0 / 1 (unchanged Linux total;
+2 new Windows-only tests not counted here).
- flutter analyze: clean (6 pre-existing Radio.groupValue infos).
- FRB bindings regenerated to expose BridgeError_ServerRejected.
|
||
|
EdisonJwa
|
606d594dc9 |
fix(audio,windows): init_tx is SyncSender not Sender (windows-only build fix)
run_raw_input_loop / run_hook_loop signatures used Sender<bool> but the callers create the channel via sync_channel which returns SyncSender. Linux cross-check missed this because windows.rs is behind #[cfg(target_os = "windows")]. |
||
|
EdisonJwa
|
181b3368d4 |
fix(audio,voice,log): six P0 issues from Korean Windows test
1. Voice Bar 'Leave voice' button removed entirely. TeamSpeak users are always in some channel; Discord/Mumble-style leave is the wrong model. To stop being heard / hearing, mute mic / speaker. To physically move, tap a different channel. The voiceLeave bridge call + _onLeaveVoice stay as dead code for now (marked unused) so existing tests/integrations don't break. 2. voice_join now confirms the move actually applied server-side by polling the snapshot for up to 1.5 s and matching our own client's channel against the requested one. If the server rejected the move (no permission, wrong password, channel full), voice_join rolls the selector back to in_channel=false and returns Err so the UI surfaces the failure instead of showing a fake 'joined' state. 3. ServerSnapshot + BridgeSnapshot gain own_client_id so the UI can identify our row without name-matching. find_own_in reads it directly. 4. set_self_muted now also clamps the TransmitModeSelector's hard_mute when input is muted server-side. Without this, the Opus encoder kept producing frames after setInputMuted(true), tsclientlib refused each one with 'Sending audio while muted', and the log grew to 200 MB on the Korean host. 5. tsclientlib WARN spam suppressed via tracing filter (tsclientlib=error). Belt-and-braces on top of fix 4. 6. Log file is now rotated at every launch (not just when >4 MiB). Two generations kept: chanora.log.1 (previous) and chanora.log.2 (the one before). The bug that produced 200 MB files was a chatty subsystem flooding a single session; the per-launch rotate keeps disk use bounded by what one session can produce in its lifetime. Bonus Windows fix (separate from the six but found in the same log): the Raw Input + Hook backends now signal readiness BEFORE blocking on GetMessageW. Previously init_tx.send was called after the loop returned (i.e. on WM_QUIT, which never happens during arming), so the main thread's 2 s readiness probe always timed out and the backend reported L0Focused even when registration succeeded. Both run_raw_input_loop and run_hook_loop now take an init_tx parameter and call report!(true) right after a successful registration, and report!(false) on every early-fail return. cargo check --workspace: clean. cargo test --workspace --lib: 80 passed / 0 failed / 1 ignored. flutter analyze: clean (6 pre-existing Radio.groupValue infos). |
||
|
EdisonJwa
|
21945979a3 |
test(audio,ptt): comprehensive Windows P0 unit-test suite (L0-L11)
Layered test coverage for the Windows PTT subsystem ahead of the
v1.0.0-rc.8 official release sign-off.
L0 (refactor)
- Extract three pure-logic dispatchers from the existing WndProc /
LowLevelKeyboardProc / LowLevelMouseProc bodies in
crates/chanora_audio/src/ptt_backends/windows.rs:
dispatch_raw_input(ctx, &RAWINPUT)
dispatch_hook_keyboard(ctx, wparam, &KBDLLHOOKSTRUCT)
dispatch_hook_mouse(ctx, wparam, &MSLLHOOKSTRUCT)
Each takes a small Context (AtomicBinding + AudioTransmitGate +
flags) and is callable without spinning up any Win32 plumbing.
The real Win32 procs unchanged structurally; they unpack lparam
and forward to the dispatchers. AtomicBinding / RawInputContext
/ HookContext / resolve_binding are now pub(crate) so the
in-file test module can drive them.
L1 — windows_keymap full-table sweep (+13 tests)
Every key_label_to_vk arm, all A-Z + a-z, all 0-9, F1-F20,
navigation, modifiers, OEM punctuation, numpad. Exhaustive
mouse_label_to_button cases including the 0x08 / 0x10 /
unknown-bitmask fallbacks.
L2 — AtomicBinding lock-free correctness
store/read round-trip, clear(), Default = zeros, single-writer
/ single-reader concurrency, many-readers / single-writer.
L3 — resolve_binding dispatcher tests
All PttInputClass variants, well-known labels, unknown-label
fallback, mismatched class+label rejection, mouse bitmask
resolution.
L4 — Backend state-machine
Both WindowsRawInputBackend and WindowsHookBackend:
descriptor() pre-arm vs post-arm (L0Focused -> L2/L3), start()
with None binding rejection, rebind() in-place, stop()
clears + idempotent, stop() after stop() no-op.
L5 — dispatch_raw_input table
Keyboard match/non-match, key-down/key-up via Flags & 0x01,
no-binding short-circuit, mouse XBUTTON1/XBUTTON2 down/up
matching the bound button, unhandled HID type. RAWINPUT structs
built via mem::zeroed plus field-fill, owning the unsafe in
the test layer where it belongs.
L6 — dispatch_hook_keyboard + dispatch_hook_mouse
WM_KEYDOWN / WM_KEYUP / WM_SYSKEYDOWN / WM_SYSKEYUP for the
keyboard path, WM_XBUTTONDOWN / WM_XBUTTONUP for the mouse
path. Same shape as L5.
L7 — Privacy invariant (crates/chanora_audio/tests/ptt_privacy.rs)
New cross-platform integration test installs a custom
tracing_subscriber Layer that records every emitted event's
target + field names. Exercises the public PTT API plus (on
Windows) the backend factory. Asserts no field name in the
banned list (vk, scan_code, keysym, key_label, bound_key,
binding, platform_key, VKey, wVk, wScan, kbflags, mouseflags)
is ever emitted and every field belongs to the DEC-027
allow-list. Adds tracing-subscriber as a dev-dependency on
chanora_audio.
L8 — Full-chain integration in core/chanora_core/src/ptt.rs
Windows-only mod windows_full_chain_tests:
zero-tail full chain (synchronous)
default-tail full chain (200 ms wait then off)
mid-press rebind abandons in-flight press
L9/L10/L11 — tools/windows-smoke.cmd + tools/windows-smoke.md
Batch smoke script + operator doc. cargo build, flutter build,
artifact existence + size checks, headless launch with stderr
capture, bridge-initialised log assertion. Distinct exit codes
per failure step. Doc explains invocation + common failure
modes.
Verification (Linux)
- cargo check --workspace: clean.
- cargo test --workspace: 78 passed / 0 failed / 3 ignored.
76 cross-platform unit tests (unchanged) plus the new
ptt_privacy integration test plus one new ignored portal smoke
test.
The Windows-gated tests (~49 new) compile and run on the Korean
Windows 11 host where they belong; cross-compile from Linux is
not configured locally. The smoke script is the production
acceptance gate for rc.8 on Windows.
Deviations from the original plan are minor (single ignored
real-runtime test rather than per-platform attribute, L7 uses
public API rather than pub(crate) dispatchers, dispatchers live
inside windows.rs rather than a sibling module) and documented
in the subagent report.
|
||
|
EdisonJwa
|
f9d20d8585 |
fix(audio,windows): adapt Raw Input + hook backends to windows-rs 0.54 API shape
Initial Task C commit (
|
||
|
EdisonJwa
|
77c2a1def4 |
feat(audio,windows): real Raw Input + low-level hook global PTT (SDD-083 / SDD-084)
The v1.0.0-rc.7 Windows backends were thread::sleep stubs that
reported optimistic L2GlobalHoldToTalk / L3GlobalWithMouseButtons
descriptors without actually registering for any global key events.
Surfaced on Windows verification as:
* 'even press to talk key was set, still only the hold to talk
button is work for talk'
* 'and displayed as L2GlobalHoldToTalk(raw-input)'
* 'cannot continuous transmission'
This commit implements the real backends:
WindowsRawInputBackend (preferred Windows rung, SDD-083):
* Hidden message-only window via
CreateWindowExW(..., HWND_MESSAGE, ...).
* RegisterRawInputDevices with RIDEV_INPUTSINK on
Usage Page 0x01 / Usage 0x06 (keyboard) and 0x02 (mouse) so
events fire globally — including when Chanora is unfocused.
* WndProc handling WM_INPUT: GetRawInputData ->
keyboard.VKey vs bound vk, or mouse.usButtonFlags vs bound
side-button index. Down -> gate.set(true); up -> gate.set(false).
* Dedicated chanora-rawinput thread runs GetMessageW /
TranslateMessage / DispatchMessageW until stop() posts
WM_QUIT via PostThreadMessageW.
WindowsHookBackend (fallback rung, SDD-084):
* SetWindowsHookExW(WH_KEYBOARD_LL) + WH_MOUSE_LL on a
dedicated chanora-llhook thread.
* Hook procs translate KBDLLHOOKSTRUCT.vkCode and
MSLLHOOKSTRUCT.mouseData against the same shared
AtomicBinding.
* UnhookWindowsHookEx on teardown.
Both backends:
* Honest descriptor() reporting: backends start reporting
L0Focused; level upgrades to L2 / L3 only after a real
arming success (RegisterRawInputDevices or SetWindowsHookEx
returning Ok). This fixes the 'L2 reported but doesn't fire'
complaint by making the badge tell the truth — if Raw Input
registration fails at runtime the user sees the L0Focused
info-icon explanation sheet instead of being told L2 works.
* AtomicBinding (class / vk / mouse_btn) for lock-free hot
path. Translation lives in
crates/chanora_audio/src/ptt_backends/windows_keymap.rs
which maps Flutter LogicalKeyboardKey.keyLabel strings
(e.g. 'Space', 'F10', 'A') to Win32 VK_* codes; mouse
side-button bitmask strings ('mouse-side-button:8' /
':16') to RawInput button indices (4 / 5).
* Per-thread context (thread_local RefCell) carries the
gate + binding to the WndProc / hook proc without needing
raw-pointer user-data plumbing.
Diagnostic logging:
* AudioEngine::start now logs default_input_config and
default_output_config explicitly with the channels /
sample_rate / sample_format that cpal reports, so a
build_*_stream failure on locale-specific Windows hosts
(reported on ko-KR Windows 11 as 'Start Audio Button not
work') becomes diagnosable from the stderr log alone.
* build_output_stream surfaces the requested config in the
tracing::error! record on failure.
Privacy (DEC-027 / SDD-090): the windows.rs and
windows_keymap.rs hot paths NEVER log raw VKs, scan codes,
keysyms, key labels, or button identifiers. Only the
platform-neutral input class ('keyboard' /
'mouse-side-button') and the backend id appear in the tracing
stream. The SDD-090 PttSanitizer Layer is the defence-in-depth
net but this code does not rely on it.
Tests: 4 new windows-only unit tests in windows_keymap (ASCII
letters / digits / Space + Fn / unknown / mouse button index).
They compile only under cfg(target_os = "windows") so the
Linux workspace test count is unchanged at 59/0/3.
Cargo deps: adds windows = '0.54' (target_os = windows) with
the feature set needed for RawInput + hooks. 0.54 matches
the version already transitive through the workspace.
Verified on Linux: cargo check --workspace clean, cargo test
--workspace 59/0/3 (windows-gated tests skip on Linux). The
real exercise of this commit will happen on the Korean Windows
11 host (100.84.219.45) at the next build.
|
||
|
EdisonJwa
|
03f5d6bca3 |
fix(p0): close three P0 coverage gaps after rc.5 audit
Audited every `Priority: P0` row in `docs/requirements/{sysrs,srs}.md`
against the live code. Three items needed work; this commit closes
all three.
Gap A — SysRS-262 + SysRS-282 (screen-reader semantics + accessible
labels for the PTT control)
-------------------------------------------------------------------
The Flutter PTT control is a custom `Listener` over a `Container`
— not a built-in `Button`, so the platform accessibility tree had
no idea it was an interactive control. Screen readers
(VoiceOver, TalkBack, NVDA, Orca) would have read the visible text
without announcing the control role or its toggled state.
Wrap the Listener in a `Semantics(button: true, toggled: _pressed,
label: …, hint: …, excludeSemantics: true)` so the platform
accessibility tree carries the right role, the current state
("Hold to talk" / "Transmitting"), and a usage hint. The
`excludeSemantics: true` argument suppresses the duplicate child
nodes the Container + Row + Icon + Text would otherwise generate
on top of our explicit label.
SysRS-263 (no colour-only state) is preserved: the visible label
and the mic icon already differentiate the two states without
relying on the colour transition.
New ARB key `pttHoldToTalkSemanticsHint` in `app_en.arb` and
`app_zh.arb`.
Gap B — SRS-198 (macOS async permission re-check)
-------------------------------------------------
The macOS backend queried `query_permission()` once at
construction and never re-checked. That violates SRS-198's
"upgrade to the appropriate Global level only after the user
grants the required permission" — once Chanora is running, a
runtime grant must lift the descriptor from `L0Focused` to a
Global level without an app restart.
Substantive rewrite of `crates/chanora_audio/src/ptt_backends/macos.rs`:
* `permission: PermissionState` becomes `permission: Arc<AtomicU8>`,
enabling cross-thread updates without a Mutex.
`PermissionState::{to_u8, from_u8}` carry the encoding.
* The backend owns a `tokio::sync::watch::Sender<PttBackendDescriptor>`
and overrides `DesktopPttBackend::descriptor_watch()` to hand
out subscribers; `chanora_core::ChanoraSession::start_audio`
already forwards transitions to `SessionEvent::PttCapability`.
* `start()` spawns a `chanora-perm-watch` OS thread that polls
`query_permission()` every 1.5 s and republishes the
descriptor on every transition. Polling rather than KVO /
notifications because Input-Monitoring has no public
change-notification API on macOS; 1.5 s is sufficient for a
user grant + return-to-Chanora cycle.
* `rebind()` also republishes the descriptor so a
`keyboard → mouse-side-button` change updates the badge.
* Six new unit tests on the platform-independent
`build_descriptor` and the atomic encoding contract. They
only compile under `target_os = "macos"` (consistent with
the rest of the module), so the Linux dev-host workspace
test count is unchanged.
`query_permission()` itself still returns `Undetermined` until
the IOKit live link lands in the macOS platform-verification
commit; the re-query loop will engage the upgrade path
automatically the moment that function returns real values.
Gap C — SRS-200 (Linux mouse-side-button portal-dependence)
-----------------------------------------------------------
`desktop-ptt-architecture.md` §5.3 already described the
heuristic classifier. Added one explicit sentence stating that
Linux mouse-side-button support is *portal-dependent*: Chanora
never claims a fixed Mouse4/Mouse5 binding on Linux; the portal
decides what inputs it accepts in the current session, and the
classifier degrades to `keyboard` whenever the portal's
description does not contain "mouse". This matches the SRS-200
text verbatim and removes the ambiguity over what "Linux
support follows the portal" means in practice.
Verification
------------
* `cargo test --workspace` (with `CHANORA_DISABLE_KEYRING=1`):
all 67 Linux-side tests green (unchanged). The new macOS
unit tests count under `target_os = "macos"` only — they
will report once the macOS reference host runs `cargo test`.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `flutter analyze`: clean (no new accessibility warnings).
* Linux release bundle builds clean.
P0 audit summary
----------------
After this commit every Priority: P0 row in `sysrs.md` and
`srs.md` has a concrete implementation. The remaining open items
are all live verification, not code:
* Per-platform live PTT traces on Windows / macOS reference
hosts (RR-PTT-001..003, RR-PTT-008) — hosts unavailable
locally; queued for platform owners.
* Linux GNOME-Wayland live trace (RR-PTT-004) — implemented
in rc.5; awaiting live host trace.
* Linux non-tested compositor fallback trace (RR-PTT-005) —
Open.
* Diagnostic-export key-leak inspection (RR-PTT-006) — Open
but trivially testable on any host with PTT bound.
* DEC-012 legal review — engineering hand-off complete since
rc.2.
|
||
|
EdisonJwa
|
82d012a46b |
feat(ptt): live Linux GNOME-Wayland portal session flow (DEC-025)
Promotes the Linux backend from probe-only to a live
`org.freedesktop.portal.GlobalShortcuts` session, closing the
gen2 v0.9.3 baseline's last Linux-side code item. Both gaps I
flagged on the review pass are addressed:
* Stop now closes the portal session through the dedicated
`org.freedesktop.portal.Session` interface (not the
request-cancel `Request` interface — that would only abort a
pending Request, not release the bound shortcuts).
* Ten new unit tests cover `classify_shortcuts_value`,
`publish_bound`, `publish_l0`, and the `SHORTCUT_ID` stability
contract using synthesised `OwnedValue` payloads. Live D-Bus
coverage stays in the `linux_portal_smoke` ignored
integration test (RR-PTT-004).
Live session lifecycle (gen2 Q5b — lazy, single backend instance):
1. `start(gate, binding)` spawns one `tokio::spawn` worker that
owns an async `zbus::Connection` (sharing the bridge's
tokio runtime per Q4a).
2. `CreateSession` with fresh random `handle_token` /
`session_handle_token` tokens. The worker awaits the portal
`Response` signal via a `RequestProxy` subscription and
extracts `session_handle` from the results dict.
3. `BindShortcuts(session_handle, [("chanora-ptt", { description
= "Chanora push-to-talk" })], "", {})`. The portal opens its
own system-managed dialog asking the user to choose a key
— Chanora itself never reads raw key events. The audio
engine continues at `L0Focused` while the dialog is open;
the descriptor watch publishes the transition once the
portal returns.
4. On `response_code == 0`: classify the `trigger_description`
substring (heuristic: contains "mouse" -> MouseSideButton,
else Keyboard), publish `L2GlobalHoldToTalk` (or `L3` for
mouse) through the watch sender. The raw trigger_description
string is never logged (DEC-027 / SRS-202).
5. On `response_code == 1` (cancelled) or `>= 2` (failure):
publish `L0Focused` through the watch sender. The user can
retry via the UI "Configure" button (gen2 Q6a).
6. The worker enters a `tokio::select!` loop multiplexing the
`cmd_rx` channel (Rebind / Stop) and the `Activated` /
`Deactivated` signals. Matching signals scoped to this
session handle and `chanora-ptt` shortcut id drive
`gate.set(true/false)`.
7. `Rebind` re-runs `BindShortcuts` on the same session.
8. `Stop` calls `org.freedesktop.portal.Session.Close()` on
the session-handle object path, clears the gate, exits.
UX (gen2 Q3a): when `_pttBackendId == 'gnome-wayland-portal'`,
the Flutter "Configure" button skips the in-app
`_PttBindingCaptureDialog` and shows a SnackBar telling the user
their desktop environment will open its own shortcut dialog.
The button delegates to `setPttBinding(keyboard, "portal")`
which nudges the backend; the portal handles the rest. New ARB
key `pttConfigurePortalRedirect` in en + zh-Hans.
Trait surface (cross-cutting):
* `DesktopPttBackend::descriptor_watch()` is a new trait method
with a default impl returning a never-firing receiver.
Backends with async capability transitions (only the Linux
portal backend today) override it to return the live watch
sender's receiver.
* `chanora_core::ChanoraSession::start_audio` subscribes to the
active backend's `descriptor_watch()` and spawns a forwarder
task that re-emits `SessionEvent::PttCapability` on every
transition. The initial value is emitted synchronously.
`Cargo.toml` (Linux-only):
* `futures-util` (std features, no executor) for stream
consumption on the portal signal subscriptions.
* `rand 0.8` for fresh per-process portal tokens.
* `zbus` continues at v5 with the `tokio` + `blocking-api`
features.
Tests
-----
* `chanora_audio` rises from 8 to 18 unit tests. New
coverage on the Linux module:
- `classify_returns_none_when_shortcut_id_missing`
- `classify_returns_keyboard_for_typical_trigger_description`
- `classify_returns_keyboard_when_trigger_description_missing`
- `classify_detects_mouse_substring`
- `classify_is_case_insensitive_on_mouse_substring`
- `publish_bound_keyboard_publishes_L2_with_keyboard_class`
- `publish_bound_mouse_publishes_L3`
- `publish_bound_none_publishes_L2_keyboard_default`
- `publish_l0_clears_descriptor`
- `shortcut_id_is_stable`
* Workspace total: 67 unit + integration tests, all green with
`CHANORA_DISABLE_KEYRING=1` (was 57 at v1.0.0-rc.4).
* New `crates/chanora_audio/tests/linux_portal_smoke.rs`
ignored integration test (RR-PTT-004 evidence path). Run on
a GNOME-on-Wayland host with
`cargo test -p chanora_audio --test linux_portal_smoke -- --ignored --nocapture`.
Documentation
-------------
* `docs/architecture/desktop-ptt-architecture.md` §5.3 rewritten
to describe the realised lifecycle; v0.9.4 change-history
entry added.
* `docs/governance/product-decision-register.md` v0.9.10
change-history entry recording the code-side promotion. No
decision rows mutate.
* `docs/release/release-readiness-go-nogo-record.md` RR-PTT-004
flipped from `Open` to `Implemented (live trace pending)`;
v0.9.5 change-history entry.
Verification
------------
* `cargo test --workspace`: 67/67 green.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: zero new warnings.
* `tools/dump_flutter_licenses.sh`: 94 packages, 0 without
LICENSE.
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` +
`flutter build linux --release`: clean Linux x86_64 bundle.
* Live portal trace (RR-PTT-004) — **not run**. The dev shell
is a TTY without a Wayland session. The user will run the
ignored smoke test from inside a GNOME-on-Wayland session
when available.
No Windows / macOS / iOS live verification in this commit (hosts
unavailable). The Windows + macOS backend scaffolds remain in
place reporting their target capability honestly; live OS-call
wiring is queued for their respective platform owners'
reference hosts per `docs/governance/staged-release-plan.md`.
|
||
|
EdisonJwa
|
5199e3d005 |
feat(ptt): full desktop backend ladder + missed-key-up watchdog (gen2 v0.9.3 follow-up)
Lands SDD-081..088 + SDD-092 implementations on top of v1.0.0-rc.3.
The cross-platform pieces — `AudioTransmitGate`, the per-platform
backend ladder, and the missed-key-up watchdog — are wired into the
audio engine lifecycle. Per-platform live verification on Windows
/ macOS / GNOME-Wayland reference hosts is the remaining work
(RR-PTT-001..006/008 in `release-readiness-go-nogo-record.md`).
`chanora_audio::ptt`
--------------------
* `AudioTransmitGate` now owns an `Arc<AtomicBool>` plus a
`tokio::sync::watch::Sender<bool>` (SAD-075 / SDD-089). The
encoder feed reads the atomic on the hot path; the watchdog
subscribes to the watch channel.
* `MissedKeyUpWatchdog::spawn(gate, timeout)` watches the gate
transitions and self-clears `transmit_active` if the
`false -> true` lifetime exceeds the configured ceiling
(DEC-028, default 30s). Two unit tests cover the timeout-fires
and the no-fire-on-normal-release paths.
`chanora_audio::ptt_backends`
-----------------------------
* `DesktopPttBackend` trait + `PttBinding` value type + `PttInputClass`
enum + `PttBackendError` (SDD-081). `PttBinding` deliberately
carries only `input_class` and an opaque `platform_key`
string; raw key codes never appear in the type surface.
* `select()` factory (SAD-071): runtime ladder evaluation per
OS. Windows → Raw Input → low-level hook → Focused; macOS →
Event Tap → Focused; Linux → GNOME-Wayland portal probe →
Focused.
* `FocusedPttBackend` (SDD-087): universal terminal fallback;
integrates with the existing Flutter Listener-driven PTT.
* `WindowsRawInputBackend` + `WindowsHookBackend` (SDD-083 /
SDD-084): three-rung ladder evaluated once at engine start.
Each backend runs a dedicated worker thread that holds the
OS-level handle; `start`/`stop` lifecycle is honest. Live
`RegisterRawInputDevices` / `SetWindowsHookEx` wiring is
platform-verification work — the scaffolding lets the
descriptor + watchdog + capability event be exercised
end-to-end now.
* `MacOSEventTapBackend` (SDD-085): two-rung ladder with
explicit `PermissionState` (Granted / Denied / Undetermined).
`Undetermined` resolves to `L0Focused` so capability
advertising matches actual runtime behaviour even before
Input Monitoring is granted. Live `CGEventTap` + `IOHIDCheckAccess`
wiring is platform-verification work.
* `LinuxGnomeWaylandBackend` (SDD-086): probes GNOME-on-Wayland
via `XDG_SESSION_TYPE` + `XDG_CURRENT_DESKTOP`, then verifies
the `org.freedesktop.portal.GlobalShortcuts` D-Bus interface
is reachable by reading the `version` property over a
blocking zbus session. Reports `gnome-wayland-portal` /
`L2GlobalHoldToTalk`. Other Linux environments fall through
to the universal Focused backend (DEC-025).
`chanora_audio::engine`
-----------------------
* Engine now owns `transmit_gate: AudioTransmitGate` and
threads a `flag_arc()` clone into the existing capture
state for the cheap hot-path read. `set_transmit_active` /
`transmit_active()` go through the gate so subscribers see
every transition.
* `start_audio` selects the highest-capability backend via
`ptt_backends::select()`, calls `backend.start(gate, none())`,
and spawns the watchdog. Both are released in `stop()` and
on Drop.
* New `engine.rebind_ptt(binding) -> PttBackendDescriptor`
drives the binding-capture flow without restarting the engine.
* New `engine.ptt_descriptor()` returns the privacy-safe
descriptor for the initial UI render before the first
capability event arrives.
`chanora_core`
--------------
* Re-exports `PttBinding` + `PttInputClass`.
* New `ChanoraSession::set_ptt_binding(binding)` — calls
`audio.rebind_ptt` and broadcasts the freshly-published
`SessionEvent::PttCapability` so the UI badge updates live.
* New `ChanoraSession::ptt_descriptor()` for the initial render.
`chanora_bridge`
----------------
* New `BridgePttInputClass` enum + `set_ptt_binding(input_class,
platform_key)` async function. The `platform_key` string is
opaque to the bridge and never logged.
* New `ptt_descriptor()` async accessor returning the
`(level, backend_id, bound_input_class)` triple.
Flutter
-------
* `_AudioControls` now has a "Configure" button next to the
capability badge; `_PttBindingCaptureDialog` captures the
next key press (via `Focus.onKeyEvent`) or mouse side button
(via `Listener.onPointerDown` filtered to button bitmasks
`0x08` / `0x10`). The captured value is the platform-neutral
`LogicalKeyboardKey.keyLabel` or `mouse-side-button:{button}`.
* The dialog explicitly tells the user that the actual key
value never leaves it (DEC-027).
* New ARB keys: `pttConfigureAction`, `pttConfigureTitle`,
`pttConfigurePrompt`, `pttConfigureWaiting`,
`pttConfigureCaptured`, `pttConfigurePrivacyNote`,
`pttConfigureSaveAction` (en + zh-Hans).
Dependencies
------------
* `chanora_audio` adds (Linux only) `zbus = "5"` with the
`tokio` runtime selector + `blocking-api` feature for the
GlobalShortcuts portal probe.
* `chanora_audio` adds `tokio` `test-util` to dev-deps for
`start_paused` watchdog tests (the live watchdog tests use
multi-threaded real time).
Verification
------------
* `cargo test --workspace` with `CHANORA_DISABLE_KEYRING=1`:
57 tests green (was 53). chanora_audio rises from 4 to 8.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: regenerates
`docs/security/license-inventory.{md,html}`. The crate count
rises from 364 to 383 with the addition of the zbus tree.
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE (unchanged).
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` + `flutter build
linux --release`: clean Linux x86_64 bundle.
Documentation
-------------
* `docs/release/release-readiness-go-nogo-record.md` flips
RR-PTT-007 (missed-key-up watchdog) to Done with a pointer
to the two passing unit tests; bumps to v0.9.4. Live
per-platform traces (RR-PTT-001..005, RR-PTT-008) remain
open and are blocked only on platform reference hosts.
Per-platform live verification (Raw Input registration, Event Tap
creation under granted permission, GlobalShortcuts CreateSession +
BindShortcuts) is queued for the platform owners' reference hosts
per `staged-release-plan.md`.
|