a9aa19ecddd4d63878f2c0f77d5019653ec507e2
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 EdisonJwa
|
af686ca7a6 |
feat(audio,ios): add coreaudio-rs dep + IosVoiceUnit skeleton (commit 1/5)
Skeleton scaffolding for the iOS VoiceProcessingIO backend that
will replace cpal on iOS. This commit lands the dependency + the
module + a constructable AudioUnit that emits silence and drops
input; nothing in engine.rs is wired up yet (that is commit 2).
Compilation contract for this commit:
* Linux / Windows / macOS / Android builds unaffected (the new
module is target_os='ios' gated, the new dep is in
'[target."cfg(target_os = \"ios\")"]').
* iOS build pulls in coreaudio-rs 0.14, constructs a VPIO unit,
pins stream format to 48 kHz Int16 mono on both buses, installs
no-op input + silence-emitting render callbacks, initializes,
and starts. No audio is actually moved until commits 3/4.
Why VPIO and not RemoteIO via cpal: cpal's iOS backend opens
RemoteIO with no control over stream format / buffer size /
channels and produces a mono-only output element that stays bound
to the route present at construction time. End-user symptom on
iPhone 16 Pro iOS 18.7.8: tapping Speaker in the picker flips
AVAudioSession.currentRoute.outputs to Speaker (confirmed in our
diagnostic logs from commit
|
||
|
EdisonJwa
|
d9c330c23b |
feat(audio,linux): output via SDL2; cpal stays on Windows/macOS
User reported persistent crackling/popping from peer audio on Linux even
after fixing the 48k->device-rate resampler boundary discontinuities,
clamping pre-Opus-encode peaks, and pre-allocating the playback scratch
buffer. Logs confirmed cpal opened raw ALSA at 44.1k native, no callback
budget violations, no underrun warnings -- yet the audio was still poor.
Root cause: cpal on Linux opens raw ALSA's 'default' PCM. On modern
PipeWire / pipewire-alsa boxes that virtual device routes through ALSA's
dmix + plug layers, whose default resampler is nearest-neighbour. cpal
also picks a small default period size (~256 frames / 5.8 ms) leaving no
headroom for kernel scheduler jitter. Both effects compound into the
crackling the user heard.
Upstream tsclientlib's own audio example
(tsclientlib/examples/audio_utils/ts_to_audio.rs) and the official Qint
client both use SDL2 with AudioSpecDesired { freq: 48000, channels: 2,
samples: 960 }. SDL2 on the same systems routes through PipeWire's PA
bridge (or PulseAudio directly), both carrying high-quality resamplers.
Fix:
* Add sdl2 = '0.37' as a target_os=linux dependency. Links libSDL2-2.0
.so (Arch sdl2-compat over SDL3, Debian libsdl2-2.0-0, Fedora SDL2).
* New module crates/chanora_audio/src/sdl_output.rs implementing
SdlOutput: opens a 48 kHz stereo 960-frame callback that zeroes the
buffer and calls AudioHandler::fill_buffer directly (no user-side
resampler). Master gain + hard-mute atomics wired in identically to
the cpal callback so set_output_gain / set_output_muted keep working.
* engine.rs cfg-gated: target_os='linux' builds SdlOutput; everywhere
else continues with the cpal output path (including the device-native-
rate negotiation and resampler-continuity fixes shipped earlier --
those remain correct on Windows/macOS where cpal targets WASAPI /
CoreAudio cleanly).
* The cpal output helpers (build_output_stream, PlaybackResampleState,
FromF32) are now cfg(not(target_os='linux'))-gated so the Linux
build doesn't emit dead-code warnings.
Capture path still cpal on every platform -- outbound audio was not
reported as bad. Resampler-continuity fix on the capture side stays:
microphone -> Opus encoder still goes through the linear interpolator
with the last-sample anchor.
Tests: 32 / 0 / 0 (chanora_audio), workspace 78 / 0 / 1 unchanged.
|
||
|
EdisonJwa
|
ba444d94bd |
feat(audio,bridge,flutter): v1 audio + PTT lifecycle implementation (SDD-094..097)
Implement the SDD-094 / SDD-095 / SDD-096 / SDD-097 detailed designs
committed in
|
||
|
EdisonJwa
|
5199e3d005 |
feat(ptt): full desktop backend ladder + missed-key-up watchdog (gen2 v0.9.3 follow-up)
Lands SDD-081..088 + SDD-092 implementations on top of v1.0.0-rc.3.
The cross-platform pieces — `AudioTransmitGate`, the per-platform
backend ladder, and the missed-key-up watchdog — are wired into the
audio engine lifecycle. Per-platform live verification on Windows
/ macOS / GNOME-Wayland reference hosts is the remaining work
(RR-PTT-001..006/008 in `release-readiness-go-nogo-record.md`).
`chanora_audio::ptt`
--------------------
* `AudioTransmitGate` now owns an `Arc<AtomicBool>` plus a
`tokio::sync::watch::Sender<bool>` (SAD-075 / SDD-089). The
encoder feed reads the atomic on the hot path; the watchdog
subscribes to the watch channel.
* `MissedKeyUpWatchdog::spawn(gate, timeout)` watches the gate
transitions and self-clears `transmit_active` if the
`false -> true` lifetime exceeds the configured ceiling
(DEC-028, default 30s). Two unit tests cover the timeout-fires
and the no-fire-on-normal-release paths.
`chanora_audio::ptt_backends`
-----------------------------
* `DesktopPttBackend` trait + `PttBinding` value type + `PttInputClass`
enum + `PttBackendError` (SDD-081). `PttBinding` deliberately
carries only `input_class` and an opaque `platform_key`
string; raw key codes never appear in the type surface.
* `select()` factory (SAD-071): runtime ladder evaluation per
OS. Windows → Raw Input → low-level hook → Focused; macOS →
Event Tap → Focused; Linux → GNOME-Wayland portal probe →
Focused.
* `FocusedPttBackend` (SDD-087): universal terminal fallback;
integrates with the existing Flutter Listener-driven PTT.
* `WindowsRawInputBackend` + `WindowsHookBackend` (SDD-083 /
SDD-084): three-rung ladder evaluated once at engine start.
Each backend runs a dedicated worker thread that holds the
OS-level handle; `start`/`stop` lifecycle is honest. Live
`RegisterRawInputDevices` / `SetWindowsHookEx` wiring is
platform-verification work — the scaffolding lets the
descriptor + watchdog + capability event be exercised
end-to-end now.
* `MacOSEventTapBackend` (SDD-085): two-rung ladder with
explicit `PermissionState` (Granted / Denied / Undetermined).
`Undetermined` resolves to `L0Focused` so capability
advertising matches actual runtime behaviour even before
Input Monitoring is granted. Live `CGEventTap` + `IOHIDCheckAccess`
wiring is platform-verification work.
* `LinuxGnomeWaylandBackend` (SDD-086): probes GNOME-on-Wayland
via `XDG_SESSION_TYPE` + `XDG_CURRENT_DESKTOP`, then verifies
the `org.freedesktop.portal.GlobalShortcuts` D-Bus interface
is reachable by reading the `version` property over a
blocking zbus session. Reports `gnome-wayland-portal` /
`L2GlobalHoldToTalk`. Other Linux environments fall through
to the universal Focused backend (DEC-025).
`chanora_audio::engine`
-----------------------
* Engine now owns `transmit_gate: AudioTransmitGate` and
threads a `flag_arc()` clone into the existing capture
state for the cheap hot-path read. `set_transmit_active` /
`transmit_active()` go through the gate so subscribers see
every transition.
* `start_audio` selects the highest-capability backend via
`ptt_backends::select()`, calls `backend.start(gate, none())`,
and spawns the watchdog. Both are released in `stop()` and
on Drop.
* New `engine.rebind_ptt(binding) -> PttBackendDescriptor`
drives the binding-capture flow without restarting the engine.
* New `engine.ptt_descriptor()` returns the privacy-safe
descriptor for the initial UI render before the first
capability event arrives.
`chanora_core`
--------------
* Re-exports `PttBinding` + `PttInputClass`.
* New `ChanoraSession::set_ptt_binding(binding)` — calls
`audio.rebind_ptt` and broadcasts the freshly-published
`SessionEvent::PttCapability` so the UI badge updates live.
* New `ChanoraSession::ptt_descriptor()` for the initial render.
`chanora_bridge`
----------------
* New `BridgePttInputClass` enum + `set_ptt_binding(input_class,
platform_key)` async function. The `platform_key` string is
opaque to the bridge and never logged.
* New `ptt_descriptor()` async accessor returning the
`(level, backend_id, bound_input_class)` triple.
Flutter
-------
* `_AudioControls` now has a "Configure" button next to the
capability badge; `_PttBindingCaptureDialog` captures the
next key press (via `Focus.onKeyEvent`) or mouse side button
(via `Listener.onPointerDown` filtered to button bitmasks
`0x08` / `0x10`). The captured value is the platform-neutral
`LogicalKeyboardKey.keyLabel` or `mouse-side-button:{button}`.
* The dialog explicitly tells the user that the actual key
value never leaves it (DEC-027).
* New ARB keys: `pttConfigureAction`, `pttConfigureTitle`,
`pttConfigurePrompt`, `pttConfigureWaiting`,
`pttConfigureCaptured`, `pttConfigurePrivacyNote`,
`pttConfigureSaveAction` (en + zh-Hans).
Dependencies
------------
* `chanora_audio` adds (Linux only) `zbus = "5"` with the
`tokio` runtime selector + `blocking-api` feature for the
GlobalShortcuts portal probe.
* `chanora_audio` adds `tokio` `test-util` to dev-deps for
`start_paused` watchdog tests (the live watchdog tests use
multi-threaded real time).
Verification
------------
* `cargo test --workspace` with `CHANORA_DISABLE_KEYRING=1`:
57 tests green (was 53). chanora_audio rises from 4 to 8.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate --offline`: regenerates
`docs/security/license-inventory.{md,html}`. The crate count
rises from 364 to 383 with the addition of the zbus tree.
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE (unchanged).
* `flutter analyze`: clean.
* `cargo build -p chanora_bridge --release` + `flutter build
linux --release`: clean Linux x86_64 bundle.
Documentation
-------------
* `docs/release/release-readiness-go-nogo-record.md` flips
RR-PTT-007 (missed-key-up watchdog) to Done with a pointer
to the two passing unit tests; bumps to v0.9.4. Live
per-platform traces (RR-PTT-001..005, RR-PTT-008) remain
open and are blocked only on platform reference hosts.
Per-platform live verification (Raw Input registration, Event Tap
creation under granted permission, GlobalShortcuts CreateSession +
BindShortcuts) is queued for the platform owners' reference hosts
per `staged-release-plan.md`.
|
||
|
EdisonJwa
|
7b21916049 |
feat(ptt): code-side initial split — transmit_active / capability badge / sanitizer
Implements the gen2 v0.9.3 doc baseline's first slice of code work:
* SRS-201: split the audio engine's `ptt` AtomicBool into the
authoritative `transmit_active` flag. The legacy `set_ptt` /
`ptt` accessors are retained as `#[doc(hidden)]` thin wrappers
so the existing bridge command and the existing Flutter
hold-to-talk UI keep compiling.
* SAD-075 / SDD-089 acknowledged at the type level: only
`AudioEngine::set_transmit_active` (or its legacy alias)
mutates the flag; the encoder feed reads it once per outbound
frame and never writes.
* SDD-082: new `chanora_audio::ptt` module ships the
`PttCapabilityLevel` enum (`L0Focused`, `L1GlobalShortcut`,
`L2GlobalHoldToTalk`, `L3GlobalWithMouseButtons`,
`L4DeviceAware` reserved) with a stable `as_str` mapping and
an `is_global` classifier.
* SDD-087: `PttBackendDescriptor::focused()` constant value for
the universal Focused-PTT fallback. The struct shape carries
only privacy-safe fields (`level`, `backend_id`,
`bound_input_class`) — a key code cannot fit through this
surface by construction (DEC-027).
* SAD-077 / SDD-090: `RedactingLogLayer` now hosts the
`PttBanCheckVisitor` and the `PTT_BANNED_FIELDS` constant
(`key_code`, `scan_code`, `virtual_key`, `vk`, `keysym`,
`keysym_string`, `key_sequence`, `key_press_history`,
`key_timing`). Any record whose field set names a banned key
is dropped before reaching the in-memory log sink or the
user-initiated diagnostic export. The check is structural and
runs ahead of formatting / redaction.
* `SessionEvent::PttCapability` carries the diagnostics-safe
descriptor through the broadcast event stream;
`chanora_core::ChanoraSession::start_audio` publishes the
Focused-PTT descriptor when the audio engine starts (SRS-196
/ SDD-091).
* `BridgeEvent::PttCapability` mirrors the event across the
FFI boundary. flutter_rust_bridge codegen regenerated.
* Flutter `_AudioControls` renders a capability badge above the
PTT button: a globe icon for Global levels, a focus-frame
icon for `L0Focused`, plus a Tooltip exposing the bound input
class. New ARB key `pttCapabilityBadge(level, backend)` in
`app_en.arb` and `app_zh.arb`.
Per-platform global PTT backends (`WindowsRawInputBackend`,
`MacOSEventTapBackend`, `LinuxGnomeWaylandBackend`) and the
`MissedKeyUpWatchdog` task land in a separate follow-up commit;
this milestone ships only PTT-L0 universally so the application's
runtime capability reporting is honest from day one.
Tests
-----
* `chanora_audio` rises from 1 to 4 unit tests covering
`PttCapabilityLevel::as_str`, `is_global`, and the
`PttBackendDescriptor::focused()` shape contract.
* `chanora_diagnostics` rises from 9 to 11 unit tests covering
the new `PttBanCheckVisitor` over every banned field name and
the `PTT_BANNED_FIELDS` stability assertion.
* Workspace total: 53 unit + integration tests, all green with
`CHANORA_DISABLE_KEYRING=1` (was 49 at v1.0.0-rc.2).
* `flutter analyze`: clean.
* `cargo deny check`: advisories ok, bans ok, licenses ok,
sources ok.
* `cargo about generate`: zero warnings (license inventory
regenerated).
* `tools/dump_flutter_licenses.sh`: 94 packages, zero without
LICENSE.
* Linux x86_64 release bundle builds clean.
No Android live verification in this commit per the user's note
that the test device was removed. Android arm64-v8a continues to
build via the same `cargo ndk` path; runtime reporting on Android
is `L0Focused` for the foreseeable future.
|
||
|
EdisonJwa
|
9790005c3e |
feat(beta): wire voice in/out end-to-end with push-to-talk (v0.2.0-beta.1)
Reaches the Internal Beta milestone of DEC-001's release sequence the
same day as Alpha. Adds voice capture and playback through the full
Flutter UI → FRB → Rust core → tsclientlib → server path.
Promotions from PoC:
poc/audio-capture-playback-spike → crates/chanora_audio/
New product code:
crates/chanora_audio/src/engine.rs — cpal capture and playback,
audiopus Opus VoIP encoder (48 kHz mono 20 ms frames), tsclientlib
AudioHandler for decode + jitter buffer + mix on playback,
push-to-talk gate, graceful playback-only fallback when capture
is unavailable.
crates/chanora_protocol/src/adapter.rs — extended with
voice_out_tx (clonable mpsc::Sender<OutPacket>) and
take_voice_in() (one-shot mpsc::Receiver<InboundVoice>); main
loop now interleaves outbound voice drain, event pumping, and
control-request handling.
crates/chanora_protocol/src/lib.rs — re-exports the few
tsproto_packets types (OutAudio, OutPacket, InAudioBuf,
AudioData, CodecType, Direction) that chanora_audio
legitimately needs. Documented as the single deliberate
cross-crate type re-export per SAD-067, justified by the
performance cost of a parallel type hierarchy on the 20 ms
voice frame.
core/chanora_core/src/lib.rs — ChanoraSession::start_audio,
set_ptt, audio_stats; disconnect now stops the engine first.
crates/chanora_bridge/src/api.rs — startAudio, setPtt,
audioStats commands and BridgeAudioStats DTO.
apps/chanora_flutter/lib/main.dart — "Start audio" button +
hold-to-talk PTT button with pressed/released visual state +
live stats line (TX/RX/PTT). Stats polled every 500 ms.
ARB:
Both en and zh-Hans gain startAudioAction, pttHoldToTalk,
pttTransmitting, audioStatsLine. Banner updated to
"Beta build — voice in/out wired; not production ready."
FRB config:
flutter_rust_bridge.yaml gains local: true so codegen resolves
the workspace member's library stem to "chanora_bridge" instead
of falling back to "UNKNOWN".
Empirical verification (2026-05-14, against cn.teamspeak.app):
cargo check + cargo test --workspace: all green.
flutter analyze: 0 issues.
flutter test: 4/4 passing including:
- test/alpha_e2e_test.dart (regression: Alpha still works)
- test/beta_e2e_test.dart (Beta: connect → startAudio →
PTT cycle → disconnect against cn.teamspeak.app).
Live smoke (cargo test alpha_smoke -- --ignored): 49 channels,
37 clients retrieved.
Capture stream open against the host PipeWire auto_null source
refused (snd_pcm_hw_params); engine correctly logged the warning
and continued in playback-only mode. TX=0 frames, RX=0 frames
reflects the headless null-source environment; on a real mic
host the encoder produces ~50 frames/second while PTT is held.
Honest Beta scope (NOT in this release):
- AEC / AGC / NS / HPF DSP (DEC-007..010): AudioEffects exists
as a struct but the filters are no-ops. Beta+ work.
- Production-quality resampler: current code is linear
interpolation. Beta+ work.
- Identity persistence via chanora_storage: still ephemeral.
- Push-to-Dart event stream: UI polls instead.
- chanora_diagnostics tracing-layer wiring: still scaffold.
- Mobile (Android) cdylib + UI: PoC-proven, not yet in product.
- Reconnect / network-loss recovery for the voice path.
Docs updates:
- docs/governance/product-decision-register.md bumped to v0.9.7
(Beta-milestone change-history entry; no row changes).
- docs/governance/poc-results-summary.md bumped to v0.6.0
(RISK-PoC-005 updated with Beta progress).
|
||
|
EdisonJwa
|
974dda9601 |
feat(scaffold): create product workspace + Rust crates + Flutter app
Implements the canonical implementation directory layout adopted by
DEC-022 (register v0.9.5). Closes the scaffolding phase; no PoC code
has been promoted in yet (per proof-of-concept-plan.md §4 a PoC is
not product code unless explicitly promoted).
Rust workspace
==============
Top-level Cargo.toml declares seven workspace members:
core/chanora_core top-level Rust API + orchestration
crates/chanora_protocol tsclientlib isolation (SAD-067, SysDes-011/029)
crates/chanora_state state sync, reducers, deltas
crates/chanora_audio capture, DSP, Opus, jitter, mixer, playback
crates/chanora_storage non-secret DB + platform secure store
crates/chanora_diagnostics logs, redaction, export
crates/chanora_bridge typed Flutter/Rust DTOs
Workspace-wide pins:
license = "MIT OR Apache-2.0" (DEC-020)
rust-version = "1.95"
edition = "2021"
The Flutter app (apps/chanora_flutter) is NOT a Cargo workspace
member; it is owned by the Flutter / Gradle toolchain and is in the
workspace exclude array along with every poc/* spike.
Each crate ships:
* a Cargo.toml referring to workspace.dependencies pins;
* a lib.rs with #![forbid(unsafe_code)] + #![warn(missing_docs)],
a typed Error enum, and the public types relevant to the
subsystem's role per SAD §7.2;
* minimal unit tests so
running 1 test
test tests::defaults_match_decisions ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::session_can_be_constructed ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::marker_matches_poc ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::state_transitions_compile ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 1 test
test tests::it_compiles ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s is non-empty.
chanora_core's CoreError type-wraps every subsystem error via
#[from] so callers can match on origin without parsing strings.
chanora_audio's AudioEffects struct defaults all four effects to
true, matching DEC-007 (AEC), DEC-008 (AGC), DEC-009 (NS),
DEC-010 (HPF). A unit test pins this so a future regression that
flips a default fails immediately.
chanora_diagnostics exports REDACTION_MARKER = "[REDACTED]",
identical to the PoC's marker so audit grep patterns survive the
promotion.
chanora_bridge's BridgeError is Serialize + Deserialize so it can
flow across the FRB 2.x boundary (DEC-014).
Empirical verification: cargo check --workspace clean, cargo test
--workspace clean (7 unit tests + 7 doc-test runners, all passing)
against Rust 1.95.0 stable.
Flutter app
===========
apps/chanora_flutter created with .
DEC-004 applied: minSdk overridden to 28 in
android/app/build.gradle.kts with a comment that points back at the
decision register and forbids lowering it without re-opening DEC-004.
DEC-015 applied: shipped English + Simplified Chinese at MVP.
- pubspec.yaml gains flutter_localizations + intl + generate:true.
- l10n.yaml emits lib/l10n/generated/AppL10n (no synthetic package
— that was removed in Flutter 3.41+).
- lib/l10n/app_en.arb is the source of truth; lib/l10n/app_zh.arb
mirrors the key set in zh-Hans. ARB metadata explicitly
reaffirms ADR-008: server-provided content (channel names,
nicknames, welcome banners) is preserved verbatim and never
translated.
lib/main.dart and test/widget_test.dart were rewritten from the
template counter into a minimal localized scaffold that proves
both locales render correctly.
Empirical verification: reports no issues;
runs the two locale smoke tests and both pass.
|