p256 PR draft ready for ReSpeak/tsclientlib submission.
Performance benchmarks document with tsclientlib baselines and
Chanora current metrics. Web client assessed as infeasible without
UDP proxy infrastructure.
tsdeclarations eval: direct codegen redundant (already via tsproto),
but permission ID generation and error code enum have high value.
Permission mapping: 253 TS3 permissions identified, 7 P1 for chat/moderation.
Encoding verification: tsclientlib uses strict UTF-8, UCS-2 is edge case.
Mark TODO-059 (auto-reconnect) and TODO-062 (DNS chain) as verified
complete with evidence. Revise TODO-063 from implementation to
verification task (tsclientlib handles encoding). Create protocol
implementation traps doc with 13 traps from YaTQA/ReSpeak/TeaSpeak.
Add purpose, architecture, and public API summary to each crate
README following chanora_resolver pattern. Update verification master
plan with new evidence sources and entry/exit criteria.
Add docs/superpowers/README.md as index for plans and specs.
Add docs/references/flutter-upstream-bugs.md tracking 4 known
Flutter framework workarounds.
Add full codebase analysis report, issue history, test environment
requirements, external research (TeaSpeak/ReSpeak/YaTQA), and offline
protocol references (ReSpeak ts3protocol, TeaSpeak, YaTQA). Add master
TODO list with 88 items across 11 categories. Add missing LICENSE files.
Branch: docs/codebase-analysis-v2
* feat(audio): add Silero ONNX VAD with WebRTC fallback
Introduce SileroOnnxVad and SileroOnnxVadWorker for desktop targets. The worker runs Silero v6 ONNX inference on a dedicated thread, accumulating 10 ms frames into the 512-sample 16 kHz input the model expects. Add VadOutput, VoiceActivityDetector trait, and WebRtcFallbackVad to provide a uniform VAD interface with graceful fallback when the ONNX model is unavailable. Wire the new VadBackend variants through AudioProcessingConfig and the snapshot stats so the bridge can report which detector is active.
* feat(audio): integrate desktop VAD worker into capture engine
Wire SileroOnnxVadWorker into the desktop capture path so voice activity can open the transmit gate before encoding. The capture callback now processes all audio through resample, downmix, and VAD unconditionally; transmit_active still gates Opus encoding.
Add new_desktop_audio_processing_state() to construct the config/stats/worker triple, and apply_desktop_vad_backend() to synchronously load or clear the worker on config changes. Override processing_backend to Noop for desktop so bridge diagnostics report the correct backend rather than the iOS-oriented PlatformVoiceProcessing default.
Includes review-driven cleanups: StreamConfig clone to deref per clippy, and a comment explaining why two try_lock calls on silero_vad_worker are structurally necessary (borrow checker requires the policy probe and the fallback path to not share a lock guard because mark_vad_fallback_active takes &mut self).
* fix(audio): modernize Windows PTT to current windows-rs API
Port the Raw Input plus low-level keyboard hook PTT backend to the newer windows-rs patterns: OptionalHandle, Result-returning CreateWindowExW, and None for CallNextHookEx. Replaces the old HHOOK(0) pointer casts. Add deterministic tests for mouse button 4 and 5 press and release driving the gate.
* build(windows): force MSVC release CRT for audiopus cmake builds
audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot use cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults. Point cmake-rs at a small wrapper that injects the policy and cache variables during configure while passing cmake --build, --version, and -E through unchanged. Keeps Opus Debug builds on Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols at test link.
Document that the iOS deployment target is intentionally absent from this file. It is enforced by tools/build-ios.sh and the Xcode project; setting it globally here would make native macOS cargo check runs try to link iPhone objects against the macOS SDK.
* build(flutter): update pubspec.lock after plugin additions
Regenerated lockfile reflecting the local_notifications and connectivity_plus plugin additions from the poke-notifications feature.
* fix(audio): address PR #37 review findings
Six fixes from independent PR review:
1. BLOCKER: Replace Windows-only cmake .cmd wrapper with cross-platform
CMake env vars. Setting CMAKE=tools/cmake-msvc-release-crt.cmd
globally broke non-Windows hosts because cmake-rs would try to
execute a .cmd file on macOS/Linux. Instead, set
CMAKE_POLICY_DEFAULT_CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY=
MultiThreadedDLL as env vars that CMake reads natively. MSVC-
specific vars are safely ignored by GCC/Clang toolchains. Delete
the now-unnecessary wrapper script.
2. IMPORTANT: Join the Silero worker thread in Drop instead of
detaching it. The old code dropped the JoinHandle which detaches
the thread; the new code calls handle.join() after closing the
channel, ensuring the ONNX session is cleaned up before the
worker is replaced during config changes.
3. IMPORTANT: Single-try_lock refactor of the capture VAD callback.
The double try_lock (policy probe + send) is replaced by a single
scoped try_lock that both probes availability and sends the frame.
The guard is dropped before the fallback path, which needs &mut
self for mark_vad_fallback_active. This also eliminates the
VadWorkerPolicy enum and callback_vad_worker_policy function,
whose behavior is now inlined into the callback.
4. IMPORTANT: Remove tracing from the realtime capture callback.
mark_vad_fallback_active and sync_vad_backend emitted info!/warn!
from the audio thread. Replace with silent atomic state
publishing via SharedAudioProcessingStats; the bridge stats
stream already exposes vad_fallback_active for diagnostics.
5. IMPORTANT: Defer ONNX model load outside the worker mutex.
apply_desktop_vad_backend_to_worker now constructs the new worker
before taking the lock, then swaps it in under a short hold.
This prevents the realtime callback from being blocked during
model I/O + thread spawn.
6. MINOR: Remove unused VadBackend import from vad/mod.rs after
deleting the policy code.
* fix(audio): address PR #37 second-pass review findings
5-agent review found 5 blocking issues. All addressed:
1. BLOCKER: CMake env vars don't reach CMake cache. Restored .cmd wrapper
but scoped to Windows MSVC targets only via [target.x86_64-pc-windows-msvc]
and [target.aarch64-pc-windows-msvc] in .cargo/config.toml. Non-Windows
hosts are unaffected.
2. BLOCKER: processing_backend normalized in set_audio_processing_config
on desktop (cfg-gated override to Noop), mirroring startup default.
3. BLOCKER: Model-path reload was already wired via reload_audio_processing_config.
Fixed misleading doc comment in core/lib.rs.
4. BLOCKER: DEC-030 updated to reflect desktop VoiceActivity enablement.
Traceability docs (SRS, SysDes, SAD, SDD, implementation-status) updated.
5. Silero ONNX cfg narrowed to desktop-only (excludes macOS/Android).
Cargo.toml ort dependency target cfg narrowed similarly.
6. Realtime callback debt documented as TODO at CaptureState::ingest.
* fix(audio): exclude ort dep on Android target
ort does not provide first-class Android prebuilts in our pin, mirror the
iOS/macOS exclusion so cargo metadata succeeds for android targets.
* test(audio): fix stale select_ptt_backend import in ptt_privacy
The helper moved out of the ptt_backends submodule onto the crate root;
update the integration test imports so the test compiles again.
* build(windows): scope MSVC release CRT cmake wrapper via Cargo [env]
Cargo's [target.<triple>] table only forwards a fixed allowlist
(linker, runner, rustflags, rustdocflags, ar), so setting CMAKE there
was silently dropped and audiopus_sys kept linking the debug CRT,
producing LNK4098 'MSVCRTD conflicts' and __imp__CrtDbgReportW errors
on x86_64-pc-windows-msvc test builds.
Move the override to Cargo's [env] table using cc/cmake-rs's
target-suffixed CMAKE_<triple> lookup (force=true, relative=true) so it
applies to MSVC targets only and not to host tooling. Add stdout
markers to the wrapper so its invocation is provable in cargo -vv logs.
Verified: cargo test -p chanora_audio --target x86_64-pc-windows-msvc
--lib --no-run now links cleanly; CMakeCache.txt records
CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL and CMP0091=NEW.
* fix(flutter): gate VoiceActivity transmit mode by platform support
VoiceActivity relies on the native VAD worker, which is only wired up
on Windows, Linux, and Android. Showing the option on iOS, macOS, or
web let users select a mode that silently never transmitted.
Add voiceActivityTransmitAvailable + transmitModeSegmentsFor() helpers
in voice_settings_controls.dart, hide the VAD row in voice_compact.dart
and drop the VAD segment from the settings dialog when unsupported.
Keep the legacy const transmitModeSegments for the existing widget test
and add two new tests covering the gated helper.
* fix(ios-audio): add voice join session coordinator
* fix(ios-audio): activate session before voice joins
* docs(ios-audio): align activation lifecycle comments
* fix(ios-audio): keep session active when already-in-channel
The 'already in channel' server response (code 0x0302) is treated as a
successful join by _onJoinChannel: the user stays in the channel and
local state is updated to reflect the joined target. But the underlying
voiceJoin call still raises BridgeError_ServerRejected, which the
joinVoiceChannelWithIosAudioSession helper used to interpret as a join
failure and deactivate the iOS audio session. Result: the UI shows the
user as joined while the audio session is dead and capture/playback
remain silent.
Add an isJoinSuccess predicate to the ordering helper. When the
predicate matches, the helper rethrows (so the caller can still run its
success-on-already-joined branch) without deactivating the session.
Wire _onJoinChannel to pass _isAlreadyInChannel as the predicate so the
0x0302 path keeps the session active.
Adds two regression tests covering the success-on-rethrow and the
predicate-false-still-deactivates paths.
* docs(security): regenerate license inventories
Cargo inventory: pick up chanora_resolver bump from 0.1.0 to
0.2.0-beta.1 so it matches the workspace; also adds a trailing newline
so 'cargo about generate' is idempotent in CI license-drift checks.
Flutter inventory: pick up flutter_local_notifications (+ platform
interfaces) and timezone pulled in by the prior notification
permission work.
Adopt a call-scoped VoIP audio session lifecycle so other apps' audio is
not stopped while Chanora is idle and the in-call session does not get
clobbered by media-server resets unrelated to voice.
AppDelegate.swift
- Set .ambient + .mixWithOthers as the idle baseline so the app does not
hold a VoiceChat session when no call is active.
- Switch to .playAndRecord + .voiceChat + .mixWithOthers + .duckOthers
on demand via the new chanora/ios_audio_session MethodChannel, and
revert to .ambient on deactivate.
- Gate the media-services-reset rebuild on voiceSessionActive so a
stray reset during idle no longer reactivates VoiceChat.
ios_audio_session_controller.dart (new)
- Thin Dart wrapper around chanora/ios_audio_session with activate/
deactivate; no-op on non-iOS; swallows PlatformException to keep
audio start/stop resilient to platform-side races.
main.dart
- Activate the iOS audio session on BridgeEvent_AudioStarted, deactivate
on BridgeEvent_AudioStopped, fire-and-forget via unawaited().
ios_voice_unit.rs
- Add the 10 local bindings required by the render-callback closure
preamble (wav_recorder_for_render, render_recorder_active,
render_ref_len, render_ref_accum, cb_count, last_num_frames,
num_frames_changes, callbacks_with_audio, callbacks_with_silence)
so the iOS target compiles cleanly with the new lifecycle wiring.
Tests
- 5 unit tests in test/services/ios_audio_session_controller_test.dart
cover activate/deactivate on iOS, no-op on non-iOS, and graceful
PlatformException handling.
Docs
- SRS SRS-110 expanded to cover the call-scoped lifecycle invariant.
- SysDes mobile-voice row updated to reflect the MethodChannel and
.ambient idle baseline.
- implementation-status-2026-05-28 voiceChat row flipped to done.
Verification
- flutter analyze: No issues found (2.8s)
- flutter test: 195 passed / 2 skipped / 0 failed
- cargo build -p chanora_audio --target aarch64-apple-ios: clean
- cargo build -p chanora_audio (macOS host): clean
Device QA matrix (Spotify-keeps-playing-while-idle, mix-during-call,
revert-on-call-end, media-services-reset-during-idle) remains pending
on physical hardware.
Inventory regenerated after refactor(audio): share AudioHandler between iOS and macOS, bump deps (6214139), which dropped windows-core 0.54.0 (and its windows-result 0.1.2 transitive dep). Apache-2.0 crate count drops from 333 to 327. No license-class change.
docs/architecture/sdd.md: three new rows in the Audio Detailed Design table. 'VoiceActivity gate (capture-side)' documents voice_activity::VoiceActivityStateMachine — the 10 ms-cadence gate for TransmitMode::VoiceActivity with open-after (40 ms) / hangover (500 ms) / min-tx (200 ms) / weak-hold (30-100 frames) timers, plus live configure() re-clamping behaviour. 'macOS render cadence (producer + ring)' documents the 20 ms tokio producer task + crossbeam ArrayQueue ring with 100 ms prebuffer, the design chosen to decouple ingress quantums (20 ms Opus frames) from egress quantums (whatever VPIO asks for). 'iOS render cadence (direct-fill)' documents the direct-fill callback path with preallocated 4096x2 f32 scratch buffer and try_lock semantics (not blocking lock).
Closes the VoiceActivity / macOS-producer-ring / iOS-direct-fill doc gaps flagged in the PR #27 'Deferred' list.
- docs/sysrs.md: raised macOS minimum runtime in SysRS-310 from 10.15 to 13.0 to match the actual floor in apps/chanora_flutter/macos/chanora_bridge.podspec (MACOSX_DEPLOYMENT_TARGET = 13.0) and macos_deployment_target.rb; added a change-log entry for the raise. SysRS-051 gained a note documenting the iOS/macOS audio-lifecycle asymmetry (iOS has full AVAudioSession lifecycle; macOS is limited to launch-time mic permission + VPIO engine restart on default-device change + VPIO startup readback in the current baseline).
- docs/architecture/sdd.md: added two rows to the Audio Detailed Design table. 'Render peak limiter' documents voice_render::limit_peak_inplace (single-pass, allocation-free, threshold 0.99, applied in both Apple render callbacks before i16 downmix). 'VPIO ducking config (macOS 14+)' documents the 8-byte AuVoiceIoOtherAudioDuckingConfiguration struct write to selector 2108 on the VoiceProcessingIO AudioUnit at startup, with the macOS 13 silent-fallback behaviour.
- docs/governance/product-decision-register.md: added DEC-033 recording the VPIO ducking configuration decision (advanced ducking off, level = Min, macOS 14+ only).
* fix(audio): eliminate Android output stutter via Oboe config + lock-free callback
Phase 1 — Oboe configuration:
- Change output stream from Usage::VoiceCommunication to Usage::Game with
ContentType::Sonification to avoid forcing the Legacy (OpenSL ES) data
path on most devices (Oboe issue #2075)
- Switch output format from i16 Mono to f32 Stereo, matching Qint's proven
configuration and eliminating per-callback downmix conversion
- Set buffer size to 2x burst after stream open, reducing default buffer
from 8-20x burst to 2x burst for lower latency
- Remove scratch Mutex<Vec<f32>>; callback writes directly to Oboe buffer
Phase 2 — Lock-free output callback:
- Add audio_event_queue.rs: lock-free SPSC bridge using crossbeam ArrayQueue
with separate packet (lossy) and control (reliable) channels
- OutputCallback now owns AudioHandler directly (no Arc<Mutex<>> on Android)
- Inbound forwarder pushes packets via AudioEventProducer (no mutex)
- set_client_volume pushes control commands via event queue on Android
- iOS/desktop Arc<Mutex<AudioHandler>> path unchanged
* fix(audio): address PR #20 review findings
- Store AudioEventConsumer directly in OutputCallback to eliminate
per-callback Arc clone on the real-time audio thread
- Add SAFETY comment for the unsafe from_raw_parts_mut transmute
- Bound set_client_volume spin-loop to 64 retries with warn log
- Remove redundant crossbeam-utils direct dependency
- Regenerate license inventory for new crossbeam deps (CI fix)
* fix(audio): use ASCII TODO punctuation
* chore: regenerate Cargo.lock after rebase
* fix(ui): add 1s cool-down to prevent double-tap channel join
voiceJoin returns instantly (fire-and-forget protocol), so the
pending-join guard clears before a second tap lands. The cool-down
prevents the rapid channel oscillation and ClientIsFlooding (524)
that results from double-tapping.
* fix(ui): handle ChannelAlreadyIn as success, ClientIsFlooding with backoff
- ChannelAlreadyIn (0x0302): treat as silent success, update UI state
- ClientIsFlooding (0x020c): show localized snackbar, extend cooldown 5s
- Add l10n strings for flooding error (en + zh)
* fix(proto): use Windows TS3 client version for broadest compatibility
Matches Qint's default (Windows_3_X_X__1). Avoids server-side
behavioral differences with TS5 version strings.
* fix(proto): patch tsproto-types to handle short P-256 coordinates
BigInt::to_bytes_be() strips leading zeros, causing WrongPublicKeyLength
when a server's ephemeral key coordinate starts with 0x00. Patch from
EdisonJwa/tsclientlib fix/p256-short-coordinate-pad branch left-pads
coordinates to the P-256 field size instead of rejecting them.
* refactor(core): stop watchdog from emitting SnapshotChanged
The watchdog now serves only as a liveness probe (miss counting for
reconnection). UI updates are handled entirely by the event-driven
delta path (ProtocolDelta → SessionEvent → BridgeEvent → Flutter).
Removes signature tracking and SnapshotChanged emission from the
supervisor loop. The initial snapshot is still fetched via the
Connected event handler in Flutter.
* refactor(ui): remove channel-join cooldown guard
With event-driven deltas the UI updates instantly on channel moves,
so the 1-second cooldown is no longer needed. Double-taps are handled
by the server (ChannelAlreadyIn → success) and the pending-channel-id
guard prevents overlapping requests.
Also removes the _lastJoinCompletedAt field entirely.
* fix(core): reattach event forwarders after reconnect
The reconnect path swapped in a new ProtocolClient but never took
chat_rx, activity_rx, or delta_rx from it. After the first reconnect,
the event-driven UI pipeline was dead.
Fix by extracting spawn_event_forwarders() helper called on both
initial connect and reconnect. Also replaces lossy try_recv+sleep
polling with proper recv().await for push-based delivery.
* feat(protocol): enrich delta schema with all snapshot-visible fields
ClientJoined now carries input_muted, output_muted, is_server_query,
talk_power, talk_power_granted. ChannelAdded/ChannelUpdated now carry
has_password and needed_talk_power. ClientUpdated also carries
is_server_query, talk_power, talk_power_granted.
This prevents local snapshot drift where fabricated defaults could
hide password requirements, talk-power restrictions, or client type.
* refactor: remove dead SnapshotChanged variant end-to-end
SnapshotChanged is no longer emitted since the watchdog was refactored
to liveness-only. Removes the variant from SessionEvent, BridgeEvent,
and the Flutter switch statement. FRB bindings regenerated.
* fix(ci): regenerate license inventory and fix iOS submodule fetch
- Regenerate docs/security/license-inventory.md to match current lockfile
- Remove submodules: true from checkout (causes hard fail on private submodule)
- Add explicit git submodule update --init --depth=1 with || true fallback
- Check silero-coreml/Package.swift instead of directory existence
Closes DEC-032. Restores the canonical Android ABI set
{arm64-v8a, armeabi-v7a, x86_64} per SDD-073 item 4 / SDD-118 item 3.
Root cause was the audiopus_sys + cmake-rs + NDK toolchain-file gap:
cargo-ndk 4.x sets ANDROID_ABI / ANDROID_PLATFORM as env vars per
invocation, but upstream cmake-rs 0.x does not forward them to the
child cmake invocation as -D variables, so armeabi-v7a and x86_64
configure steps fell through to the toolchain-file default and
failed to build.
Fix:
- Cargo.toml: add a workspace [patch.crates-io] stanza pinning the
cmake crate to fork pr2502/cmake-rs @ commit
bdad5edc569d82151922c5c6c4685b1563f12aa1 (branch android-build),
which carries cmake-rs PR #257
(https://github.com/rust-lang/cmake-rs/pull/257). The patch is a
9-line addition that forwards ANDROID_ABI and ANDROID_PLATFORM
from the env to the child cmake as -D variables.
- Cargo.lock: regenerated by 'cargo update -p cmake'; the lone
cmake entry now points at the fork rev.
- apps/chanora_flutter/android/app/build.gradle.kts: restore
abiFilters to {arm64-v8a, armeabi-v7a, x86_64}; remove the
TODO(x86_64/armv7 follow-up) comment.
- docs/governance/product-decision-register.md: mark DEC-032 as
Resolved (2026-05-18) with the resolution mechanism, update the
§3 / §7 rows, and append a 0.9.8.1 change-history entry.
Verification (host: Linux):
cargo update -p cmake -> pulled fork rev
cargo check --workspace --all-targets -> PASS
cargo test --workspace -> PASS (no regressions)
cargo ndk --platform 28 -t arm64-v8a build -p chanora_bridge -> PASS
cargo ndk --platform 28 -t armeabi-v7a build -p chanora_bridge -> PASS
cargo ndk --platform 28 -t x86_64 build -p chanora_bridge -> PASS
Upstream tracking: re-evaluate the [patch.crates-io] override once
cmake-rs PR #257 merges and a fresh cmake release lands on
crates.io; at that point switch to a plain dep bump and remove the
override.
Author SysRS-310 ratifying the macOS minimum runtime baseline at 10.15
(Catalina) at the SysRS layer, parallel to SysRS-286 (iOS 13.0) and
SysRS-288 (Android API 28).
Closes the Wave 1.5 traceability-audit deferred-but-optional follow-up.
ID allocation: SysRS-310 (not SysRS-290, which is already allocated to
MVP single-active-server-connection scope); monotonic numbering
preserved.
Cross-references SAD-087, SDD-119, SysRS-286, SysRS-288. Verification:
Review + Platform Test on a macOS 10.15 system. Raising the baseline
(e.g., to 11.0 / Big Sur) requires a DEC entry.
Trace: SysRS v0.9.11, SAD-087, SDD-119.
Edison Jwa