Edison Jwa
2f6d45fb04
* feat(audio): add Silero ONNX VAD with WebRTC fallback Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active. * feat(audio): integrate desktop VAD worker into capture engine Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding. Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default. Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self). * fix(audio): modernize Windows PTT to current windows-rs API Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate. * build(windows): force MSVC release CRT for audiopus cmake builds audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link. Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK. * build(flutter): update pubspec.lock after plugin additions Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature. * fix(audio): address PR #37 review findings Six fixes from independent PR review: 1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd globally broke non-Windows hosts because cmake-rs would try to execute a .cmd file on macOS/Linux. Instead, set CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY= MultiThreadedDLL as env vars that CMake reads natively. MSVC- specific vars are safely ignored by GCC/Clang toolchains. Delete the now-unnecessary wrapper script. 2. IMPORTANT: Join the Silero worker thread in Drop instead of detaching it. The old code dropped the JoinHandle which detaches the thread; the new code calls handle.join() after closing the channel, ensuring the ONNX session is cleaned up before the worker is replaced during config changes. 3. IMPORTANT: Single-try_lock refactor of the capture VAD callback. The double try_lock (policy probe + send) is replaced by a single scoped try_lock that both probes availability and sends the frame. The guard is dropped before the fallback path, which needs &mut self for mark_vad_fallback_active. This also eliminates the VadWorkerPolicy enum and callback_vad_worker_policy function, whose behavior is now inlined into the callback. 4. IMPORTANT: Remove tracing from the realtime capture callback. mark_vad_fallback_active and sync_vad_backend emitted info!/warn! from the audio thread. Replace with silent atomic state publishing via SharedAudioProcessingStats; the bridge stats stream already exposes vad_fallback_active for diagnostics. 5. IMPORTANT: Defer ONNX model load outside the worker mutex. apply_desktop_vad_backend_to_worker now constructs the new worker before taking the lock, then swaps it in under a short hold. This prevents the realtime callback from being blocked during model I/O + thread spawn. 6. MINOR: Remove unused VadBackend import from vad/mod.rs after deleting the policy code. * fix(audio): address PR #37 second-pass review findings 5-agent review found 5 blocking issues. All addressed: 1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc] and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows hosts are unaffected. 2. BLOCKER: processing_backend normalized in set_audio_processing_config on desktop (cfg-gated override to Noop), mirroring startup default. 3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config. Fixed misleading doc comment in core/lib.rs. 4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement. Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated. 5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android). Cargo.toml ort dependency target cfg narrowed similarly. 6. Realtime callback debt documented as TODO at CaptureState::ingest. * fix(audio): exclude ort dep on Android target ort does not provide first-class Android prebuilts in our pin, mirror the iOS/macOS exclusion so cargo metadata succeeds for android targets. * test(audio): fix stale select_ptt_backend import in ptt_privacy The helper moved out of the ptt_backends submodule onto the crate root; update the integration test imports so the test compiles again. * build(windows): scope MSVC release CRT cmake wrapper via Cargo [env] Cargo's [target.<triple>] table only forwards a fixed allowlist (linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there was silently dropped and audiopus_sys kept linking the debug CRT, producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors on x86_64-pc-windows-msvc test builds. Move the override to Cargo's [env] table using cc/cmake-rs's target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it applies to MSVC targets only and not to host tooling. Add stdout markers to the wrapper so its invocation is provable in cargo -vv logs. Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc --lib --no-run now links cleanly; CMakeCache.txt records CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW. * fix(flutter): gate VoiceActivity transmit mode by platform support VoiceActivity relies on the native VAD worker, which is only wired up on Windows, Linux, and Android. Showing the option on iOS, macOS, or web let users select a mode that silently never transmitted. Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers in voice_settings_controls.dart, hide the VAD row in voice_compact.dart and drop the VAD segment from the settings dialog when unsupported. Keep the legacy const transmitModeSegments for the existing widget test and add two new tests covering the gated helper.
12 KiB
12 KiB
Chanora Software Architecture Description
Lifecycle: SWE.2 Software Architectural Design
Document status: DV meeting baseline candidate
Date: 2026-05-29
Direct upstream source: docs/srs.md
Related system allocation: docs/sysdes.md
1. Purpose
This Software Architecture Description defines Chanora's software architecture for DV review. It bridges SRS software requirements to SWE.3 detailed design and to SWE.5/SWE.6 verification planning.
This baseline captures the architecture visible in the current repository. It is sufficient for DV traceability review, while deeper per-module algorithms remain in docs/architecture/sdd.md and source-level design.
2. Architectural Scope
Chanora is a Flutter application with a Rust core. Flutter owns the user-facing shell, Material 3 widgets, localization, permission UX, and platform service presentation. Rust owns connection orchestration, protocol isolation, audio processing, storage coordination, diagnostics, server resolution, prefetch policy, and bridge DTOs.
3. Upstream SRS Allocation
| SRS group | Architectural allocation |
|---|---|
| SRS-003, SRS-008 through SRS-016 | Cross-platform app shell, Flutter UI, Rust core, platform adapters |
| SRS-017 through SRS-030 | Flutter UI, state presentation, connection and voice controls |
| SRS-031 through SRS-035 | Bridge layer and typed DTO boundary |
| SRS-036 through SRS-043 | Rust core connection lifecycle and state behavior |
| SRS-044 through SRS-053 | Protocol adapter and TeamSpeak-compatible server boundary |
| SRS-054 through SRS-061 | State synchronization and replay/reducer verification hooks |
| SRS-062 through SRS-083 | Audio subsystem, DSP, codec, PTT, mute/deaf, metering |
| SRS-084 through SRS-095 | Storage, secure storage, identity, diagnostics-sensitive data |
| SRS-096 through SRS-102 | Diagnostics, export, redaction, troubleshooting hooks |
| SRS-103 through SRS-123 | Platform adapters, packaging, release behavior |
| SRS-124 through SRS-143 | Verification support, analysis requirements, traceability rules |
| SRS-144 through SRS-184 | Material 3, adaptive UI, accessibility, localization, Unicode, app initialization |
| SRS-185 through SRS-218 | Platform baselines, PTT capability, transmit mode, no automatic telemetry, benchmark advisory |
4. Component Architecture
| Component | Repository location | Responsibility | Direct architectural dependencies |
|---|---|---|---|
| Flutter app shell | apps/chanora_flutter/lib/main.dart, services, widgets |
App startup, screen composition, user actions, localization, Material 3 UI | Generated Rust bridge, platform plugins, Flutter services |
| Flutter service layer | apps/chanora_flutter/lib/services/ |
Permission flows, lifecycle policy, host prefetch debounce, link trust, state mapping, platform back intent | Flutter app shell, generated bridge APIs, platform plugins |
| Flutter widget layer | apps/chanora_flutter/lib/widgets/ |
Connect UI, channel tree, chat, voice controls, settings, diagnostics surfaces | Flutter services, generated DTOs, design tokens |
| Bridge layer | crates/chanora_bridge, apps/chanora_flutter/lib/src/rust/ |
Typed Flutter/Rust boundary and generated bindings | Rust core, Flutter generated code |
| Rust core | core/chanora_core |
Connection lifecycle, orchestration, reconnect behavior, storage coordination, voice state, bridge-facing event DTOs | Protocol, audio, storage, diagnostics, state, resolver/prefetch |
| Protocol adapter | crates/chanora_protocol |
Isolate tsclientlib, expose typed protocol DTOs/errors |
Rust core, external compatible server |
| State sync | crates/chanora_state |
Snapshot/delta model, channel join helpers, reducer behavior | Rust core, protocol DTOs |
| Audio subsystem | crates/chanora_audio |
Capture/playback, Opus, DSP, PTT, voice activity reservation, platform units | Rust core, platform APIs, protocol audio path |
| Storage | crates/chanora_storage |
Bookmarks, identities, encrypted local data, platform keyring integration | Rust core, platform secure storage |
| Diagnostics | crates/chanora_diagnostics |
Redaction, log sink, export bundle, known-secret registry | Rust core, Flutter diagnostics UI |
| Server resolver | crates/chanora_resolver |
SRV/TSDNS/DNS fallback resolution | Rust core, prefetch crate |
| Server prefetch | crates/chanora_prefetch, Flutter prefetch_debouncer.dart |
Invisible host-field resolution warming, TTL cache, generation safety | Resolver, Flutter connect UI, Rust core |
5. Static Architecture View
Flutter UI/widgets/services
-> generated Dart bridge API
-> chanora_bridge
-> chanora_core
-> chanora_protocol -> tsclientlib -> external compatible server
-> chanora_state
-> chanora_audio -> platform audio APIs / Opus / DSP
-> chanora_storage -> platform secure storage / SQLite
-> chanora_diagnostics
-> chanora_prefetch -> chanora_resolver -> network DNS/TSDNS
The bridge is the trust and type boundary between Flutter and Rust. Flutter must not directly depend on protocol-library internals. Rust core must not expose platform-specific storage or audio details to UI code except through stable DTOs and capability fields.
Current Core locality note: the public Core Interface remains available through chanora_core::* re-exports, while branch simplify-project-review has started moving internal Core responsibilities into focused Modules (events.rs, network_diagnostics.rs). This is an internal maintainability split, not a public Interface change.
6. Runtime Flow Architecture
6.1 Connect Flow
User enters host/bookmark
-> Flutter connect widgets
-> optional prefetch debounce
-> bridge connect command
-> Rust core supervisor
-> resolver / prefetch cache
-> protocol adapter
-> external compatible server
-> state snapshot/events
-> bridge event stream
-> Flutter state mapper and widgets
6.2 Voice Flow
Microphone / platform input
-> audio capture unit
-> DSP chain: HPF, NS, AEC, AGC where active
-> PTT/mute/transmit gate
-> Opus encode
-> protocol adapter
-> external compatible server
External server voice
-> protocol adapter
-> jitter/decode path
-> mixer / per-user controls
-> platform output
6.3 Diagnostics Flow
Runtime event or error
-> diagnostic log sink / known-secret registry
-> redactor
-> user-initiated export bundle
-> Flutter share/export surface
7. Interface Catalogue
| Interface | Producer | Consumer | Architectural rule |
|---|---|---|---|
| Bridge command DTOs | Flutter generated API | chanora_bridge, Rust core |
Stable typed DTOs; no raw protocol-library types cross to Flutter |
| Bridge event DTOs | Rust core / bridge | Flutter services/widgets | User-safe errors and capability fields are explicit |
| Protocol DTOs | chanora_protocol |
Rust core, state sync | Protocol adapter isolates tsclientlib |
| Audio configuration | Flutter settings / Rust core | chanora_audio |
Voice modes and processing flags are explicit; Windows/Linux desktop VAD-backed VoiceActivity is enabled only where runtime evidence exists, with unsupported platforms disabled/deferred |
| Storage records | Storage crate | Rust core / Flutter UI via bridge | Secrets stay behind secure-storage abstraction |
| Diagnostic bundles | Diagnostics crate | Flutter diagnostics UI | Redaction runs before export or display |
| Platform capability records | Platform adapters/audio/PTT backends | UI and release record | UI/release wording must not over-claim capability |
8. Dependency Rules
| Rule | Rationale |
|---|---|
| Flutter UI depends on generated bridge APIs, not Rust internals | Keeps UI stable across Rust implementation changes |
| Rust core orchestrates crates but protocol/audio/storage crates remain separately testable | Supports SWE.4 unit verification and bounded responsibilities |
Protocol adapter is the only component that owns tsclientlib coupling |
Protects the app from protocol-library leakage |
| Diagnostics redaction must be reusable by runtime logging and export | Prevents split redaction behavior |
| Platform-specific behavior stays in platform adapters or audio platform units | Keeps cross-platform logic testable and reduces conditional sprawl |
| Release claims consume capability records and release evidence | Prevents over-claiming PTT, signing, packaging, or secure-storage behavior |
9. Non-Functional Allocation
| Concern | Architectural mechanism | Verification owner |
|---|---|---|
| Real-time audio responsiveness | Rust audio subsystem, benchmark advisory, bounded callback behavior | Audio / Platform QA |
| Privacy and no automatic telemetry | User-initiated diagnostics, no automatic upload policy | Security / Privacy QA |
| Secure secret handling | Platform secure-storage abstraction and encrypted local storage | Security / QA |
| Cross-platform UI | Flutter Material 3, design tokens, responsive widgets | Software QA / UX |
| Protocol compatibility | tsclientlib adapter isolation and compatible-server matrix |
Protocol / Integration QA |
| Release reproducibility | CI, build scripts, artifact hashes, release-readiness record | Release / Operations QA |
10. Architectural Decisions Captured by This Baseline
| Decision | Architectural outcome |
|---|---|
| Flutter + Rust split | Flutter owns presentation; Rust owns protocol/audio/storage/diagnostics core behavior |
tsclientlib isolation |
Protocol compatibility is behind chanora_protocol |
| Secure storage abstraction | Platform storage details do not leak into UI or unrelated crates |
| Advisory audio benchmarks | Performance regressions are surfaced without making CI a hard release gate at this stage |
| PTT capability levels | Platform PTT support is represented as capability data and must match release wording |
| VoiceActivity platform scope | Windows/Linux desktop VoiceActivity is implemented through the capture VAD path; unsupported platforms remain disabled/deferred until backend allocation and runtime verification exist |
| No automatic diagnostic upload in MVP | Diagnostics are local and user-initiated unless future approved requirements change policy |
11. Verification Handoff
| Verification plan | SAD handoff |
|---|---|
| SWE.4 | Component boundaries define unit-test ownership for Flutter services/widgets and Rust crates |
| SWE.5 | Interface catalogue and runtime flows define integration paths |
| SWE.6 | SRS allocation and acceptance flows define software acceptance evidence |
| SYS.4 | Platform capability and external-server boundaries define system integration evidence |
12. Traceability to SRS
This SAD derives only from docs/srs.md. The broad SRS group-to-component allocation in section 3 is the controlling SWE.2 trace for DV. Detailed item-level trace is represented by the SRS coverage matrix and docs/governance/traceability-matrix.md.
13. Open Architecture Risks
| Risk | Impact | Control |
|---|---|---|
| SAD item numbering from historical status references is not reconstructed in this baseline | Existing references such as SAD-043 and SAD-046 are not itemized here |
Treat this as a DV baseline SAD; add itemized SAD IDs in a follow-up if process requires strict ID-level review |
| Some architecture views are textual rather than C4 diagrams | Reviewers may request visual C4 views | Record as documentation hardening, not a blocker for DV baseline if textual views are accepted |
| Release/platform architecture evidence is incomplete | Public release remains blocked | Controlled by release-readiness and waiver records |
| Android runtime verification is not automatic in local reviews | Android permission/audio/lifecycle regressions can pass Rust-only tests | Require adb devices -l with a connected device/emulator and Android smoke evidence before claiming Android runtime success |
| Protocol voice packet re-export is an intentional exception to full protocol isolation | Future changes may accidentally widen the protocol/audio Seam | Document and keep the voice wire exception narrow, or move packet construction fully into chanora_protocol |
14. DV Conclusion
This SWE.2 baseline is sufficient to remove the missing-SAD traceability gap for DV review. It does not replace candidate test evidence or final release approval.