edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
chanora/docs/architecture/sdd.md
T
Edison Jwa 2f6d45fb04 feat(audio): desktop Silero ONNX VAD + Windows PTT modernization + MSVC CRT build fix (#37) 
* feat(audio): add Silero ONNX VAD with WebRTC fallback

Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active.

* feat(audio): integrate desktop VAD worker into capture engine

Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding.

Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default.

Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self).

* fix(audio): modernize Windows PTT to current windows-rs API

Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate.

* build(windows): force MSVC release CRT for audiopus cmake builds

audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link.

Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK.

* build(flutter): update pubspec.lock after plugin additions

Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature.

* fix(audio): address PR #37 review findings

Six fixes from independent PR review:

1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform
   CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd
   globally broke non-Windows hosts because cmake-rs would try to
   execute a .cmd file on macOS/Linux. Instead, set
   CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY=
   MultiThreadedDLL as env vars that CMake reads natively. MSVC-
   specific vars are safely ignored by GCC/Clang toolchains. Delete
   the now-unnecessary wrapper script.

2. IMPORTANT: Join the Silero worker thread in Drop instead of
   detaching it. The old code dropped the JoinHandle which detaches
   the thread; the new code calls handle.join() after closing the
   channel, ensuring the ONNX session is cleaned up before the
   worker is replaced during config changes.

3. IMPORTANT: Single-try_lock refactor of the capture VAD callback.
   The double try_lock (policy probe + send) is replaced by a single
   scoped try_lock that both probes availability and sends the frame.
   The guard is dropped before the fallback path, which needs &mut
   self for mark_vad_fallback_active. This also eliminates the
   VadWorkerPolicy enum and callback_vad_worker_policy function,
   whose behavior is now inlined into the callback.

4. IMPORTANT: Remove tracing from the realtime capture callback.
   mark_vad_fallback_active and sync_vad_backend emitted info!/warn!
   from the audio thread. Replace with silent atomic state
   publishing via SharedAudioProcessingStats; the bridge stats
   stream already exposes vad_fallback_active for diagnostics.

5. IMPORTANT: Defer ONNX model load outside the worker mutex.
   apply_desktop_vad_backend_to_worker now constructs the new worker
   before taking the lock, then swaps it in under a short hold.
   This prevents the realtime callback from being blocked during
   model I/O + thread spawn.

6. MINOR: Remove unused VadBackend import from vad/mod.rs after
   deleting the policy code.

* fix(audio): address PR #37 second-pass review findings

5-agent review found 5 blocking issues. All addressed:

1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper
   but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc]
   and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows
   hosts are unaffected.

2. BLOCKER: processing_backend normalized in set_audio_processing_config
   on desktop (cfg-gated override to Noop), mirroring startup default.

3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config.
   Fixed misleading doc comment in core/lib.rs.

4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement.
   Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated.

5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android).
   Cargo.toml ort dependency target cfg narrowed similarly.

6. Realtime callback debt documented as TODO at CaptureState::ingest.

* fix(audio): exclude ort dep on Android target

ort does not provide first-class Android prebuilts in our pin, mirror the
iOS/macOS exclusion so cargo metadata succeeds for android targets.

* test(audio): fix stale select_ptt_backend import in ptt_privacy

The helper moved out of the ptt_backends submodule onto the crate root;
update the integration test imports so the test compiles again.

* build(windows): scope MSVC release CRT cmake wrapper via Cargo [env]

Cargo's [target.<triple>] table only forwards a fixed allowlist
(linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there
was silently dropped and audiopus_sys kept linking the debug CRT,
producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors
on x86_64-pc-windows-msvc test builds.

Move the override to Cargo's [env] table using cc/cmake-rs's
target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it
applies to MSVC targets only and not to host tooling. Add stdout
markers to the wrapper so its invocation is provable in cargo -vv logs.

Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc
--lib --no-run now links cleanly; CMakeCache.txt records
CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW.

* fix(flutter): gate VoiceActivity transmit mode by platform support

VoiceActivity relies on the native VAD worker, which is only wired up
on Windows, Linux, and Android. Showing the option on iOS, macOS, or
web let users select a mode that silently never transmitted.

Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers
in voice_settings_controls.dart, hide the VAD row in voice_compact.dart
and drop the VAD segment from the settings dialog when unsupported.
Keep the legacy const transmitModeSegments for the existing widget test
and add two new tests covering the gated helper.
2026-06-09 20:47:16 +09:00

162 lines
14 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Chanora Software Detailed Design
**Lifecycle:** SWE.3 Software Detailed Design and Unit Construction Handoff
**Document status:** DV meeting baseline candidate
**Date:** 2026-05-29
**Direct upstream source:** `docs/architecture/sad.md`
**Related software requirements:** `docs/srs.md`
## 1. Purpose
This Software Detailed Design defines the module-level design details needed for SWE.4 unit verification and SWE.5 integration verification. It is based on the current repository layout and the SWE.2 architecture baseline.
## 2. Module Catalogue
| SDD module | Source location | Primary responsibility | Upstream SAD component |
|---|---|---|---|
| SDD-MOD-001 Flutter app bootstrap | `apps/chanora_flutter/lib/services/app_bootstrap.dart`, `main.dart` | Initialize Rust bridge, localization, app services, theme/design baseline | Flutter app shell |
| SDD-MOD-002 Connect UI | `apps/chanora_flutter/lib/widgets/connect_widgets.dart` | Host/bookmark inputs, connect actions, pre-request UX | Flutter widget layer |
| SDD-MOD-003 Snapshot and channel UI | `snapshot_view.dart`, `snapshot_state_mapper.dart`, `channel_spacer.dart` | Present channel tree, clients, and mapped state | Flutter widget/service layer |
| SDD-MOD-004 Chat UI | `chat_views.dart`, `bbcode_text.dart` | Channel text rendering and BBCode-safe display | Flutter widget layer |
| SDD-MOD-005 Voice UI | `voice_bar.dart`, `voice_compact.dart`, `voice_settings*.dart`, `voice_level_meter.dart`, `ptt_capability_badge.dart` | Voice controls, processing settings, metering, PTT capability | Flutter widget layer |
| SDD-MOD-006 Platform services | `android_permissions_service.dart`, `ios_permissions_service.dart`, `audio_lifecycle_service.dart`, `back_intent_*`, `link_trust_service.dart` | Permission, lifecycle, navigation, route/link trust behavior | Flutter service layer |
| SDD-MOD-007 Bridge API | `crates/chanora_bridge/src/api.rs`, generated Dart/Rust bridge files | Typed command/event boundary | Bridge layer |
| SDD-MOD-008 Rust core supervisor | `core/chanora_core/src/lib.rs`, `events.rs`, `network_diagnostics.rs`, `ptt.rs` | Connection orchestration, reconnect, bridge-facing event DTOs, network diagnostics, PTT state, storage coordination | Rust core |
| SDD-MOD-009 Protocol adapter | `crates/chanora_protocol/src/` | `tsclientlib` isolation, DTO/error mapping | Protocol adapter |
| SDD-MOD-010 State sync | `crates/chanora_state/src/lib.rs`, `channel_join.rs` | Snapshot/delta model, reducer, channel join support | State sync |
| SDD-MOD-011 Audio subsystem | `crates/chanora_audio/src/` | Audio capture/playback, DSP, Opus, PTT, mode stack, platform units | Audio subsystem |
| SDD-MOD-012 Storage | `crates/chanora_storage/src/lib.rs` | Bookmarks, identity storage, encrypted local records, keyring abstraction | Storage |
| SDD-MOD-013 Diagnostics | `crates/chanora_diagnostics/src/lib.rs` | Redaction, log sink, known-secret registry, export bundle | Diagnostics |
| SDD-MOD-014 Resolution and prefetch | `crates/chanora_resolver/src/lib.rs`, `crates/chanora_prefetch/src/lib.rs`, `prefetch_debouncer.dart` | SRV/TSDNS/DNS fallback and generation-safe resolution warming | Server resolver / prefetch |
| SDD-MOD-015 Build and release hooks | `.github/workflows/`, `tools/`, platform project files | CI, unsigned iOS build, benchmark advisory, platform smoke procedures | Release / platform architecture |
## 3. Bridge Boundary Design
The bridge boundary is the only supported Flutter-to-Rust command path. Dart code uses generated APIs under `apps/chanora_flutter/lib/src/rust/`; Rust exposes bridge functions through `crates/chanora_bridge/src/api.rs`.
Design rules:
| Rule | Detail |
|---|---|
| DTO stability | DTO fields must be explicit and serializable through Flutter Rust Bridge generation |
| Error safety | Rust errors exposed to Flutter must be user-safe or mapped before display |
| Secret handling | Secrets may cross only as command inputs or protected DTO fields and must be registered for diagnostic redaction where relevant |
| Capability reporting | Platform and PTT capability fields must reflect actual active backend state |
| Regeneration control | Generated bridge files are implementation artifacts and must be regenerated when bridge API signatures change |
## 4. Connection and State Design
| Detail | Design |
|---|---|
| Connection lifecycle | Rust core owns connect/disconnect/reconnect decisions and suppresses reconnect after user disconnect |
| Backoff | Reconnect uses exponential backoff as described in implementation status, capped at 60 seconds |
| Core internal Modules | `lib.rs` remains the public Interface and orchestration entry point; `events.rs` owns public event/bridge-facing DTOs re-exported by `lib.rs`; `network_diagnostics.rs` owns private connect/loss counters and the last-loss ring buffer |
| Server resolution | Resolver performs SRV/TSDNS/DNS fallback; prefetch cache may warm but must not be required for connect success |
| Snapshot mapping | Rust state and bridge DTOs are mapped into Flutter view models by `snapshot_state_mapper.dart` |
| Channel join | Channel join logic and errors are represented through Rust state/protocol handling and Flutter error mapper service |
| Reducers | `chanora_state` owns snapshot/delta reducer design with unit coverage for snapshot, delta, reconnect, duplicate normalization, disconnected/lost suppression, unknown-client voice activity, deterministic ordering, and channel-delete/client cleanup. Current runtime UI refresh still flows through `chanora_core` snapshot/probe paths; full live-event folding through `chanora_state::reduce` is an integration follow-up. |
## 5. Audio Detailed Design
| Audio element | Design detail |
|---|---|
| Capture/playback | Platform-specific units handle Android, iOS, desktop/fallback paths behind Rust audio abstractions |
| Codec | Opus encode/decode lives in `opus_voice.rs` and associated audio modules |
| DSP chain | High-pass filter, noise suppression, echo cancellation, and AGC are represented by audio processing modules/backends |
| Transmit control | `TransmitMode` supports `Ptt`, `Continuous`, and `VoiceActivity`; `VoiceActivity` is active for Windows/Linux desktop capture when VAD is configured, while mobile, macOS, and unverified-platform enablement remain deferred |
| VoiceActivity gate (capture-side) | `voice_activity::VoiceActivityStateMachine` is the 10 ms-cadence gate for `TransmitMode::VoiceActivity`; open-after 40 ms (debounce), hangover 500 ms (anti-chatter), min-tx 200 ms (anti-flicker), weak-hold 30-100 frames (anti-stale-VAD); live `configure()` re-clamps existing timers on settings change without resetting state; 9 unit tests cover the main paths |
| PTT | Desktop/mobile backends expose capability level and active backend; missed-key-up watchdog prevents stuck transmit |
| Release tail | Tail handling prevents abrupt cutoffs after PTT release where configured |
| Render peak limiter | `voice_render::limit_peak_inplace` is a single-pass, allocation-free per-frame peak scaler applied in both the macOS and iOS render callbacks before the i16 downmix; default threshold 0.99 prevents hard clipping on multi-client mixes that sum past 0 dBFS while remaining transparent for normal voice levels (allocation-free, lock-free, safe on the realtime audio thread) |
| macOS render cadence (producer + ring) | `ios_voice_unit.rs:851-961` runs a 20 ms tokio producer task that calls `AudioHandler::fill_buffer(1920)` and `force_push`es each sample into a `crossbeam ArrayQueue<f32>` (SPSC-effective, MPMC-but-wait-free-per-end); ring capacity 12000 samples ≈ 6.25× pull quantum; 100 ms prebuffer (`PREBUFFER_SAMPLES = 9600` stereo f32) before the VPIO render callback starts draining, matching Mumble's playout margin and WebRTC's kStartDelayMs order of magnitude |
| iOS render cadence (direct-fill) | `ios_voice_unit.rs:968-1034` does `AudioHandler::fill_buffer` directly in the VPIO render callback (VPIO on iOS requests 480-frame ≈ 10 ms slices that align with tsclientlib's 20 ms Opus frame); scratch buffer preallocated to 4096×2 f32 at setup time so the realtime callback never `resize()`s; `try_lock` (not `lock`) on the AudioHandler mutex so contention never stalls the realtime IO thread; on `WouldBlock` the callback emits silence and increments `callback_xrun` |
| VPIO ducking config (macOS 14+) | `ios_voice_unit.rs` writes an 8-byte `AuVoiceIoOtherAudioDuckingConfiguration` struct (`m_enable_advanced_ducking = 0` disables dynamic voice-activity-driven ducking; `m_ducking_level = kAUVoiceIOOtherAudioDuckingLevelMin = 10`) to selector `kAUVoiceIOProperty_OtherAudioDuckingConfiguration` (= 2108) on the VoiceProcessingIO AudioUnit at startup, minimising the ducking of other apps' audio during a voice session; on macOS 13 the property is silently ignored (VPIO returns the default ducking behaviour) and the code logs a debug message and continues |
| Benchmarks | Realtime capture, Opus, and resampler benchmarks provide advisory baseline evidence |
## 6. Storage and Secret Design
| Storage item | Design detail |
|---|---|
| Bookmarks | Stored locally through the storage crate and surfaced in Flutter connect UI |
| Identity references | Stored through `IdentityFileStore` and platform secure storage where available |
| Passwords/secrets | Encrypted at rest using the current storage design; Android Keystore-backed DEK is deferred and must be disclosed |
| CI keyring behavior | CI disables real keyring access with `CHANORA_DISABLE_KEYRING=1` to avoid headless blocking |
| Fallback behavior | Platform fallback modes must be represented as limitations in release/security evidence |
## 7. Diagnostics Detailed Design
| Diagnostic element | Design detail |
|---|---|
| Log sink | Runtime logs can be captured by diagnostic sinks for export |
| Known-secret registry | Runtime secrets are registered for redaction where applicable |
| Redactor | Redacts configured sensitive patterns before export |
| Export bundle | Diagnostic export is JSON-based and user-initiated |
| Upload policy | MVP has no automatic diagnostic, telemetry, or crash upload |
## 8. Flutter UI Detailed Design
| UI area | Design detail |
|---|---|
| Design tokens | `chanora_tokens.dart` centralizes product styling over Material 3 |
| Platform capability display | `platform_capabilities.dart` and PTT capability widgets expose platform-specific support honestly |
| Localization | Generated localization files provide English and Simplified Chinese resources |
| Responsive behavior | Current widgets support compact/mobile-oriented layouts; expanded side-pane hardening remains P1/P2 as recorded |
| Accessibility | Critical status should use text/icons/semantics and not color alone; verification remains through UI tests/audit |
| UI settings persistence | `UiPreferencesService` persists host, nickname, permission explanation state, and theme mode through `shared_preferences`; invalid stored theme values fall back to system theme |
## 9. Build and Release Detailed Design
| Build/release item | Design detail |
|---|---|
| Rust CI | `.github/workflows/ci.yml` runs cargo check/test and advisory clippy |
| Flutter CI | `.github/workflows/ci.yml` runs Flutter pub get, analyze, and tests |
| Supply chain | CI runs cargo-deny and license inventory checks |
| iOS unsigned build | CI runs `flutter build ios --release --no-codesign` |
| Audio benchmarks | `bench-advisory.yml` runs audio benchmarks and posts advisory evidence |
| Platform packages | Public binary packaging/signing/notarization remains release-gated |
## 10. Verification Hook Design
| Module | SWE.4 unit hooks | SWE.5/SWE.6 integration hooks |
|---|---|---|
| Flutter services/widgets | Dart unit/widget tests under `apps/chanora_flutter/test/` | Widget/system demos and candidate device smoke |
| Bridge | API compile/generation checks | Flutter-to-Rust command/event smoke |
| Rust core | Cargo tests | Compatible-server lifecycle demo |
| Protocol | DTO/error mapping tests | Protocol compatibility matrix and server demo |
| State sync | Reducer tests | Snapshot/delta/reconnect integration evidence |
| Audio | DSP/codec/PTT tests and benchmarks | Platform audio loopback/device demo |
| Storage | Repository/encryption/keyring-disabled tests | Platform secure-storage audit |
| Diagnostics | Redaction/export tests | User-initiated export inspection |
| Release hooks | CI workflow validation | Release readiness record and artifact evidence |
## 11. Traceability to SAD
| SAD component | SDD modules |
|---|---|
| Flutter app shell | SDD-MOD-001 |
| Flutter service layer | SDD-MOD-006, SDD-MOD-014 |
| Flutter widget layer | SDD-MOD-002 through SDD-MOD-005 |
| Bridge layer | SDD-MOD-007 |
| Rust core | SDD-MOD-008 |
| Protocol adapter | SDD-MOD-009 |
| State sync | SDD-MOD-010 |
| Audio subsystem | SDD-MOD-011 |
| Storage | SDD-MOD-012 |
| Diagnostics | SDD-MOD-013 |
| Server resolver/prefetch | SDD-MOD-014 |
| Release/platform architecture | SDD-MOD-015 |
## 12. Open Detailed-Design Risks
| Risk | Impact | Control |
|---|---|---|
| Detailed item IDs from historical SDD references are not reconstructed | Existing references such as `SDD-109` are not itemized in this baseline | Treat this as a DV baseline SDD and add strict item numbering later if required |
| Some module designs are summarized rather than API-by-API | May be insufficient for final process audit | Use this as DV baseline; deepen high-risk modules before final release gate |
| Android Keystore-backed DEK is not implemented | Limits storage/security design claims | Controlled by waiver and release-readiness records |
| Full event replay tooling and live reducer integration evidence are absent | Limits state verification design beyond reducer unit behavior | Controlled as P1 gap and runtime-integration follow-up |
| Android runtime smoke is blocked when no device/emulator is attached | Android permission/audio/lifecycle paths cannot be claimed from Rust tests alone | Require `adb devices -l` and Android smoke evidence before closing Android verification claims |
## 13. DV Conclusion
This SWE.3 baseline is sufficient to remove the missing-SDD traceability gap for DV review and to feed SWE.4/SWE.5 verification plans. It does not close release evidence gaps or replace source-level tests.
Reference in New Issue View Git Blame Copy Permalink