Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2b285491d0 | ||
|
|
413f247378 | ||
|
|
3f9ea4f7b8 | ||
|
|
2f6d45fb04 | ||
|
|
5c3dd70bba | ||
|
|
ef14c22300 | ||
|
|
b841d3f3e4 |
+46
-31
@@ -1,34 +1,49 @@
|
|||||||
# Environment variables set for all cargo invocations in this workspace.
|
# audiopus_sys calls cmake::build(opus_path), so downstream Cargo env cannot
|
||||||
# CMAKE_POLICY_VERSION_MINIMUM is required for audiopus_sys's bundled
|
# call cmake-rs Config::define() to override CMake's MSVC Debug CRT defaults.
|
||||||
# Opus CMake build to succeed on CMake 4.x (which removed compatibility
|
# Instead, point cmake-rs at a small wrapper that injects -D cache/policy
|
||||||
# with cmake_minimum_required < 3.5). audiopus_sys v0.2.2 bundles
|
# variables during configure while passing cmake --build / --version / -E /
|
||||||
# Opus 1.3.1 whose CMakeLists.txt uses a very old minimum version.
|
# --install / --open through unchanged. This keeps Opus Debug builds on
|
||||||
|
# Rust's release dynamic CRT (/MD) instead of CMake's default debug CRT
|
||||||
|
# (/MDd), which otherwise pulls in unresolved __imp__CrtDbgReportW symbols
|
||||||
|
# at test link.
|
||||||
|
#
|
||||||
|
# IMPORTANT: Cargo's `[target.<triple>]` config sections only forward a
|
||||||
|
# fixed allowlist of keys (linker, runner, rustflags, rustdocflags, ar)
|
||||||
|
# to build scripts. Arbitrary keys such as `CMAKE` placed under
|
||||||
|
# `[target.<triple>]` are silently ignored and never reach the
|
||||||
|
# audiopus_sys build script. cmake-rs (via cc-style env resolution)
|
||||||
|
# looks up CMAKE in this order:
|
||||||
|
# 1. CMAKE_<target-triple-with-dashes>
|
||||||
|
# 2. CMAKE_<target_triple_with_underscores>
|
||||||
|
# 3. TARGET_CMAKE (or HOST_CMAKE when host == target)
|
||||||
|
# 4. CMAKE
|
||||||
|
# We therefore scope the wrapper to Windows MSVC targets by setting the
|
||||||
|
# target-suffixed variant in the global [env] section. Non-Windows
|
||||||
|
# hosts (macOS, Linux, iOS, Android) never see CMAKE set and invoke
|
||||||
|
# `cmake` directly.
|
||||||
|
#
|
||||||
|
# NOTE: CMAKE_POLICY_DEFAULT_CMP0091 and CMAKE_MSVC_RUNTIME_LIBRARY cannot
|
||||||
|
# be set via the process environment because CMake does NOT auto-import
|
||||||
|
# them into its cache; they must be passed as `-D` definitions, which the
|
||||||
|
# wrapper does.
|
||||||
|
#
|
||||||
|
# iOS deployment target (DEC-003: iOS 13.0 minimum) is NOT set here. It is
|
||||||
|
# enforced in two places that own the iOS build:
|
||||||
|
# 1. tools/build-ios.sh — sets IPHONEOS_DEPLOYMENT_TARGET for the cargo
|
||||||
|
# invocation and bypasses audiopus_sys's CMake build via
|
||||||
|
# LIBOPUS_STATIC=1 / LIBOPUS_NO_PKG=1 / LIBOPUS_LIB_DIR.
|
||||||
|
# 2. apps/chanora_flutter/ios/Runner.xcodeproj — sets the Xcode
|
||||||
|
# IPHONEOS_DEPLOYMENT_TARGET build setting for the final link.
|
||||||
|
# Setting it globally here would make native macOS `cargo check` runs try
|
||||||
|
# to link iPhone objects against the macOS SDK.
|
||||||
|
|
||||||
[env]
|
[env]
|
||||||
CMAKE_POLICY_VERSION_MINIMUM = "3.5"
|
CMAKE_POLICY_VERSION_MINIMUM = "3.5"
|
||||||
|
|
||||||
# iOS builds must set IPHONEOS_DEPLOYMENT_TARGET in the invoking script
|
# Scope the cmake wrapper to Windows MSVC targets only via the
|
||||||
# or Xcode build phase. Do not set it globally here: native macOS cargo
|
# target-suffixed env var name that cc/cmake-rs already resolve.
|
||||||
# checks also compile bundled C/C++ dependencies, and a global iOS
|
# Force = true so a developer's pre-existing CMAKE_x86_64-pc-windows-msvc
|
||||||
# deployment target makes clang try to link iPhone objects against the
|
# does not silently bypass the wrapper. Relative = true so the path
|
||||||
# macOS SDK.
|
# resolves from the workspace root regardless of where cargo is invoked.
|
||||||
|
CMAKE_x86_64-pc-windows-msvc = { value = "tools/cmake-msvc-release-crt.cmd", force = true, relative = true }
|
||||||
# iOS target linker flags (DEC-003: minimum deployment target iOS 13.0).
|
CMAKE_aarch64-pc-windows-msvc = { value = "tools/cmake-msvc-release-crt.cmd", force = true, relative = true }
|
||||||
#
|
|
||||||
# These rustflags pass -miphoneos-version-min=13.0 to the linker, ensuring
|
|
||||||
# the final binary targets iOS 13.0+. This is defense-in-depth alongside
|
|
||||||
# the IPHONEOS_DEPLOYMENT_TARGET env var above — the env var affects C
|
|
||||||
# compilation (cc crate, CMake), while these rustflags affect the final
|
|
||||||
# link step.
|
|
||||||
#
|
|
||||||
# NOTE: The canonical iOS build is done via tools/build-ios.sh, which
|
|
||||||
# sets LIBOPUS_STATIC=1, LIBOPUS_NO_PKG=1, and LIBOPUS_LIB_DIR to
|
|
||||||
# bypass audiopus_sys's CMake build entirely.
|
|
||||||
|
|
||||||
[target.aarch64-apple-ios]
|
|
||||||
rustflags = ["-C", "link-arg=-miphoneos-version-min=13.0"]
|
|
||||||
|
|
||||||
[target.aarch64-apple-ios-sim]
|
|
||||||
rustflags = ["-C", "link-arg=-miphonesimulator-version-min=13.0"]
|
|
||||||
|
|
||||||
[target.x86_64-apple-ios]
|
|
||||||
rustflags = ["-C", "link-arg=-miphonesimulator-version-min=13.0"]
|
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ EXTERNAL SOURCES:
|
|||||||
:path: ".symlinks/plugins/haptic_kit/ios"
|
:path: ".symlinks/plugins/haptic_kit/ios"
|
||||||
|
|
||||||
SPEC CHECKSUMS:
|
SPEC CHECKSUMS:
|
||||||
chanora_bridge: 26252acdf9ca660ce9c132ad25cd5ad5af467b16
|
chanora_bridge: 27a03592058709f6f38701343eb51c3a55b02da0
|
||||||
Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
|
Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
|
||||||
flutter_foreground_task: a159d2c2173b33699ddb3e6c2a067045d7cebb89
|
flutter_foreground_task: a159d2c2173b33699ddb3e6c2a067045d7cebb89
|
||||||
haptic_kit: b22c4fbb2aa7b0d66f2891f81a9e950ad2de5758
|
haptic_kit: b22c4fbb2aa7b0d66f2891f81a9e950ad2de5758
|
||||||
|
|||||||
@@ -46,7 +46,8 @@ import AVFoundation
|
|||||||
//
|
//
|
||||||
// VoIP configuration is engaged on voice-channel join via the
|
// VoIP configuration is engaged on voice-channel join via the
|
||||||
// `chanora/ios_audio_session` MethodChannel, driven from Dart
|
// `chanora/ios_audio_session` MethodChannel, driven from Dart
|
||||||
// by the BridgeEvent::AudioStarted / AudioStopped lifecycle.
|
// before `voiceJoin` starts VoiceProcessingIO and again as an
|
||||||
|
// idempotent guard on the AudioStarted lifecycle.
|
||||||
do {
|
do {
|
||||||
try AVAudioSession.sharedInstance().setCategory(.ambient, mode: .default)
|
try AVAudioSession.sharedInstance().setCategory(.ambient, mode: .default)
|
||||||
logAudioSessionState(context: "launch-ambient")
|
logAudioSessionState(context: "launch-ambient")
|
||||||
@@ -79,8 +80,8 @@ import AVFoundation
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Activate the VoIP audio session. Called from Dart via the
|
/// Activate the VoIP audio session. Called from Dart via the
|
||||||
/// `chanora/ios_audio_session` channel when a voice channel join
|
/// `chanora/ios_audio_session` channel before a voice channel join
|
||||||
/// reaches the `BridgeEvent::AudioStarted` stage. Configures
|
/// starts VoiceProcessingIO. Configures
|
||||||
/// .playAndRecord + .voiceChat with .mixWithOthers so other apps
|
/// .playAndRecord + .voiceChat with .mixWithOthers so other apps
|
||||||
/// (Spotify, podcasts) can keep playing alongside the voice
|
/// (Spotify, podcasts) can keep playing alongside the voice
|
||||||
/// channel — matching the Telegram group-call UX. Idempotent:
|
/// channel — matching the Telegram group-call UX. Idempotent:
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ import 'services/prefetch_debouncer.dart';
|
|||||||
import 'services/snapshot_state_mapper.dart';
|
import 'services/snapshot_state_mapper.dart';
|
||||||
import 'services/ts3_server_link.dart';
|
import 'services/ts3_server_link.dart';
|
||||||
import 'services/ui_preferences_service.dart';
|
import 'services/ui_preferences_service.dart';
|
||||||
|
import 'services/voice_join_ordering.dart';
|
||||||
import 'src/rust/api.dart' as rust;
|
import 'src/rust/api.dart' as rust;
|
||||||
import 'src/rust/frb_generated.dart';
|
import 'src/rust/frb_generated.dart';
|
||||||
import 'src/rust/lib.dart' as rust_err;
|
import 'src/rust/lib.dart' as rust_err;
|
||||||
@@ -1301,7 +1302,18 @@ class _BetaHomeState extends State<_BetaHome> with WidgetsBindingObserver {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await rust.voiceJoin(channelId: ch.id, password: password ?? '');
|
await joinVoiceChannelWithIosAudioSession(
|
||||||
|
channelId: ch.id,
|
||||||
|
password: password ?? '',
|
||||||
|
voiceJoin: rust.voiceJoin,
|
||||||
|
activateIosAudioSession: iosAudioSessionController.activate,
|
||||||
|
deactivateIosAudioSession: iosAudioSessionController.deactivate,
|
||||||
|
// Server says we are already in the target channel: the user is
|
||||||
|
// still joined to a voice channel, so the iOS audio session must
|
||||||
|
// stay active. The catch below converts this rethrow into the
|
||||||
|
// success-on-already-joined branch.
|
||||||
|
isJoinSuccess: _isAlreadyInChannel,
|
||||||
|
);
|
||||||
if (!mounted) return;
|
if (!mounted) return;
|
||||||
setState(() {
|
setState(() {
|
||||||
_currentVoiceChannelId = ch.id;
|
_currentVoiceChannelId = ch.id;
|
||||||
|
|||||||
@@ -10,9 +10,9 @@ const iosAudioSessionChannelName = 'chanora/ios_audio_session';
|
|||||||
/// launch and leaves it inactive. The session is only switched to
|
/// launch and leaves it inactive. The session is only switched to
|
||||||
/// `.playAndRecord` + `.voiceChat` (with `.mixWithOthers`) while a
|
/// `.playAndRecord` + `.voiceChat` (with `.mixWithOthers`) while a
|
||||||
/// voice channel is actually active. This controller is the Dart
|
/// voice channel is actually active. This controller is the Dart
|
||||||
/// side of that contract — call [activate] when the Rust engine
|
/// side of that contract — call [activate] before the Rust engine
|
||||||
/// emits `BridgeEvent::AudioStarted` and [deactivate] on
|
/// starts VoiceProcessingIO and [deactivate] on
|
||||||
/// `BridgeEvent::AudioStopped`.
|
/// `BridgeEvent::AudioStopped` or failed joins.
|
||||||
///
|
///
|
||||||
/// On non-iOS platforms both methods are no-ops; the platforms
|
/// On non-iOS platforms both methods are no-ops; the platforms
|
||||||
/// handle their own session lifecycle elsewhere (Android via
|
/// handle their own session lifecycle elsewhere (Android via
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
typedef VoiceJoinCallback = Future<void> Function({
|
||||||
|
required BigInt channelId,
|
||||||
|
required String password,
|
||||||
|
});
|
||||||
|
|
||||||
|
typedef IosVoiceSessionActivation = Future<void> Function();
|
||||||
|
typedef IosVoiceSessionDeactivation = Future<void> Function();
|
||||||
|
|
||||||
|
/// Predicate used to recognise `voiceJoin` errors that the caller treats as a
|
||||||
|
/// successful join outcome (e.g. the server replied "already in channel").
|
||||||
|
///
|
||||||
|
/// When this returns `true` for a thrown error, the iOS audio session is kept
|
||||||
|
/// active because the user is still considered joined to the channel. The
|
||||||
|
/// error is still rethrown so the caller can run its success-on-already-joined
|
||||||
|
/// branch and update local state.
|
||||||
|
typedef VoiceJoinSuccessPredicate = bool Function(Object error);
|
||||||
|
|
||||||
|
Future<void> joinVoiceChannelWithIosAudioSession({
|
||||||
|
required BigInt channelId,
|
||||||
|
required String password,
|
||||||
|
required VoiceJoinCallback voiceJoin,
|
||||||
|
required IosVoiceSessionActivation activateIosAudioSession,
|
||||||
|
required IosVoiceSessionDeactivation deactivateIosAudioSession,
|
||||||
|
VoiceJoinSuccessPredicate? isJoinSuccess,
|
||||||
|
}) async {
|
||||||
|
await activateIosAudioSession();
|
||||||
|
try {
|
||||||
|
await voiceJoin(channelId: channelId, password: password);
|
||||||
|
} catch (e) {
|
||||||
|
if (isJoinSuccess != null && isJoinSuccess(e)) {
|
||||||
|
rethrow;
|
||||||
|
}
|
||||||
|
await deactivateIosAudioSession();
|
||||||
|
rethrow;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1339,11 +1339,8 @@ sealed class BridgeEvent with _$BridgeEvent {
|
|||||||
|
|
||||||
/// Bridge iOS voice-processing mode.
|
/// Bridge iOS voice-processing mode.
|
||||||
enum BridgeIosVoiceProcessingMode {
|
enum BridgeIosVoiceProcessingMode {
|
||||||
/// Shipping VPIO path.
|
/// Apple VoiceProcessingIO path.
|
||||||
platformVoiceProcessing,
|
platformVoiceProcessing,
|
||||||
|
|
||||||
/// Experimental Sonora path.
|
|
||||||
sonoraExperimental,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@freezed
|
@freezed
|
||||||
|
|||||||
@@ -638,12 +638,19 @@ class _VoiceSheetBodyState extends State<_VoiceSheetBody> {
|
|||||||
selected: _mode == rust.BridgeTransmitMode.continuous,
|
selected: _mode == rust.BridgeTransmitMode.continuous,
|
||||||
onTap: () => _setMode(rust.BridgeTransmitMode.continuous),
|
onTap: () => _setMode(rust.BridgeTransmitMode.continuous),
|
||||||
),
|
),
|
||||||
_ModeRow(
|
// Voice-activity transmit is only honoured by the engine on
|
||||||
label: l10n.voiceModeVoiceActivity,
|
// hosts that ship a Chanora-owned VAD pipeline (DEC-030:
|
||||||
icon: Icons.graphic_eq,
|
// Windows + Linux desktop and Android). iOS / macOS rely
|
||||||
selected: _mode == rust.BridgeTransmitMode.voiceActivity,
|
// on Apple VoiceProcessingIO and have no VAD bridge, so
|
||||||
onTap: () => _setMode(rust.BridgeTransmitMode.voiceActivity),
|
// hiding the row prevents the UI from advertising a
|
||||||
),
|
// transmit mode the engine cannot honour.
|
||||||
|
if (voiceActivityTransmitAvailable)
|
||||||
|
_ModeRow(
|
||||||
|
label: l10n.voiceModeVoiceActivity,
|
||||||
|
icon: Icons.graphic_eq,
|
||||||
|
selected: _mode == rust.BridgeTransmitMode.voiceActivity,
|
||||||
|
onTap: () => _setMode(rust.BridgeTransmitMode.voiceActivity),
|
||||||
|
),
|
||||||
|
|
||||||
// 3) Release-tail slider (PTT only).
|
// 3) Release-tail slider (PTT only).
|
||||||
if (isPtt) ...[
|
if (isPtt) ...[
|
||||||
|
|||||||
@@ -128,7 +128,12 @@ class _VoiceSettingsDialogState extends State<VoiceSettingsDialog> {
|
|||||||
VoiceSectionHeader(l10n.voiceModeLabel),
|
VoiceSectionHeader(l10n.voiceModeLabel),
|
||||||
SegmentedButton<rust.BridgeTransmitMode>(
|
SegmentedButton<rust.BridgeTransmitMode>(
|
||||||
style: voiceSegmentedButtonStyle(theme),
|
style: voiceSegmentedButtonStyle(theme),
|
||||||
segments: transmitModeSegments,
|
// DEC-030: hide the voice-activity segment on hosts
|
||||||
|
// that ship no Chanora-owned VAD pipeline (iOS,
|
||||||
|
// macOS, web).
|
||||||
|
segments: transmitModeSegmentsFor(
|
||||||
|
voiceActivityAvailable: voiceActivityTransmitAvailable,
|
||||||
|
),
|
||||||
selected: {_mode},
|
selected: {_mode},
|
||||||
onSelectionChanged: (s) => setState(() => _mode = s.first),
|
onSelectionChanged: (s) => setState(() => _mode = s.first),
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -1,3 +1,6 @@
|
|||||||
|
import 'dart:io' show Platform;
|
||||||
|
|
||||||
|
import 'package:flutter/foundation.dart' show kIsWeb;
|
||||||
import 'package:flutter/material.dart';
|
import 'package:flutter/material.dart';
|
||||||
|
|
||||||
import '../src/rust/api.dart' as rust;
|
import '../src/rust/api.dart' as rust;
|
||||||
@@ -10,7 +13,11 @@ ButtonStyle voiceSegmentedButtonStyle(ThemeData theme) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Transmit mode selector segments.
|
/// Transmit mode selector segments — full set, all three modes.
|
||||||
|
///
|
||||||
|
/// This list is kept stable for legacy call sites and tests; UI
|
||||||
|
/// surfaces that must respect DEC-030 platform gating should prefer
|
||||||
|
/// [transmitModeSegmentsFor] with [voiceActivityTransmitAvailable].
|
||||||
const transmitModeSegments = [
|
const transmitModeSegments = [
|
||||||
ButtonSegment(
|
ButtonSegment(
|
||||||
value: rust.BridgeTransmitMode.ptt,
|
value: rust.BridgeTransmitMode.ptt,
|
||||||
@@ -29,6 +36,44 @@ const transmitModeSegments = [
|
|||||||
),
|
),
|
||||||
];
|
];
|
||||||
|
|
||||||
|
/// Transmit mode selector segments, optionally dropping the
|
||||||
|
/// voice-activity entry on hosts that do not ship a VAD pipeline.
|
||||||
|
///
|
||||||
|
/// Voice activity transmit is gated by [voiceActivityTransmitAvailable]
|
||||||
|
/// because the underlying VAD pipeline ships only on Windows, Linux, and
|
||||||
|
/// Android per DEC-030. Builds for unsupported platforms (iOS, macOS,
|
||||||
|
/// web) drop the VAD segment entirely so the UI never advertises a
|
||||||
|
/// transmit mode the engine cannot honour.
|
||||||
|
List<ButtonSegment<rust.BridgeTransmitMode>> transmitModeSegmentsFor({
|
||||||
|
required bool voiceActivityAvailable,
|
||||||
|
}) {
|
||||||
|
if (voiceActivityAvailable) return transmitModeSegments;
|
||||||
|
return const [
|
||||||
|
ButtonSegment(
|
||||||
|
value: rust.BridgeTransmitMode.ptt,
|
||||||
|
label: Text('PTT'),
|
||||||
|
icon: Icon(Icons.radio_button_checked, size: 14),
|
||||||
|
),
|
||||||
|
ButtonSegment(
|
||||||
|
value: rust.BridgeTransmitMode.continuous,
|
||||||
|
label: Text('Always'),
|
||||||
|
icon: Icon(Icons.podcasts, size: 14),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True when this host advertises VAD transmit per DEC-030.
|
||||||
|
///
|
||||||
|
/// The desktop Silero ONNX + WebRTC fallback ships on Windows and
|
||||||
|
/// Linux; the Android Oboe + WebRTC path covers Android. iOS and
|
||||||
|
/// macOS still rely on the Apple VoiceProcessingIO unit and have no
|
||||||
|
/// Chanora-owned VAD pipeline, so the bridge cannot honour the
|
||||||
|
/// voice-activity transmit mode there.
|
||||||
|
bool get voiceActivityTransmitAvailable {
|
||||||
|
if (kIsWeb) return false;
|
||||||
|
return Platform.isWindows || Platform.isLinux || Platform.isAndroid;
|
||||||
|
}
|
||||||
|
|
||||||
/// Android hardware/WebRTC selector segments.
|
/// Android hardware/WebRTC selector segments.
|
||||||
const androidProcessingSegments = [
|
const androidProcessingSegments = [
|
||||||
ButtonSegment(
|
ButtonSegment(
|
||||||
|
|||||||
@@ -5,18 +5,18 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: _fe_analyzer_shared
|
name: _fe_analyzer_shared
|
||||||
sha256: "8d7ff3948166b8ec5da0fbb5962000926b8e02f2ed9b3e51d1738905fbd4c98d"
|
sha256: "3b19a47f6ea7c2632760777c78174f47f6aec1e05f0cd611380d4593b8af1dbc"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "93.0.0"
|
version: "96.0.0"
|
||||||
analyzer:
|
analyzer:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: analyzer
|
name: analyzer
|
||||||
sha256: de7148ed2fcec579b19f122c1800933dfa028f6d9fd38a152b04b1516cec120b
|
sha256: "0c516bc4ad36a1a75759e54d5047cb9d15cded4459df01aa35a0b5ec7db2c2a0"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "10.0.1"
|
version: "10.2.0"
|
||||||
args:
|
args:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -133,10 +133,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: code_assets
|
name: code_assets
|
||||||
sha256: "83ccdaa064c980b5596c35dd64a8d3ecc68620174ab9b90b6343b753aa721687"
|
sha256: bf394f466ba9205f1812a0433b392d6af280f155f56651eda7c18cc32ed493b8
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "1.0.0"
|
version: "1.2.1"
|
||||||
collection:
|
collection:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -197,10 +197,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: dbus
|
name: dbus
|
||||||
sha256: d0c98dcd4f5169878b6cf8f6e0a52403a9dff371a3e2f019697accbf6f44a270
|
sha256: "792974a4007974fbc5c1b5433eb2330a9db3e368c3f906253af4c007d0f49a91"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "0.7.12"
|
version: "0.7.13"
|
||||||
fake_async:
|
fake_async:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -361,18 +361,18 @@ packages:
|
|||||||
dependency: "direct main"
|
dependency: "direct main"
|
||||||
description:
|
description:
|
||||||
name: haptic_kit
|
name: haptic_kit
|
||||||
sha256: "39efffa513c9f8ce3cdded8a4423797f69d71c9281779b83727337f3ee1ed9b8"
|
sha256: "457f825a3413be2651954639bed27bb2987570f75d90c4e8e1cb9be62db2e59d"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "1.0.0"
|
version: "1.0.1"
|
||||||
hooks:
|
hooks:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: hooks
|
name: hooks
|
||||||
sha256: "025f060e86d2d4c3c47b56e33caf7f93bf9283340f26d23424ebcfccf34f621e"
|
sha256: "9a62a50b50b769a737bc0a8ff381f333529df3ab746b2f6b02e83760231455ba"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "1.0.3"
|
version: "2.0.2"
|
||||||
http:
|
http:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -509,14 +509,6 @@ packages:
|
|||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "2.0.0"
|
version: "2.0.0"
|
||||||
native_toolchain_c:
|
|
||||||
dependency: transitive
|
|
||||||
description:
|
|
||||||
name: native_toolchain_c
|
|
||||||
sha256: "6ba77bb18063eebe9de401f5e6437e95e1438af0a87a3a39084fbd37c90df572"
|
|
||||||
url: "https://pub.dev"
|
|
||||||
source: hosted
|
|
||||||
version: "0.17.6"
|
|
||||||
nm:
|
nm:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -529,10 +521,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: objective_c
|
name: objective_c
|
||||||
sha256: "100a1c87616ab6ed41ec263b083c0ef3261ee6cd1dc3b0f35f8ddfa4f996fe52"
|
sha256: "6cb691c686fa2838c6deb34980d426145c2a5d537491cb83d463c33cdbc726ed"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "9.3.0"
|
version: "9.4.1"
|
||||||
package_config:
|
package_config:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -705,10 +697,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: shared_preferences_android
|
name: shared_preferences_android
|
||||||
sha256: e8d4762b1e2e8578fc4d0fd548cebf24afd24f49719c08974df92834565e2c53
|
sha256: a2c49fc1fed7140cadd892d765bd47edbe4ac0b9c7e7e3c493dcb58126f99cf0
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "2.4.23"
|
version: "2.4.25"
|
||||||
shared_preferences_foundation:
|
shared_preferences_foundation:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -862,10 +854,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: url_launcher_android
|
name: url_launcher_android
|
||||||
sha256: "17bc677f0b301615530dd1d67e0a9828cafa2d0b6b6eae4cd3679b7eac4a273c"
|
sha256: b413d49b73867ac08dd2f9890efd3cc11f2a0e577618d50843440a1fb3776c32
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "6.3.30"
|
version: "6.3.32"
|
||||||
url_launcher_ios:
|
url_launcher_ios:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -974,10 +966,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: win32
|
name: win32
|
||||||
sha256: a1fc9eb9248baa05dfc12ed5b66e377b3e23f095eec078e0371622b9033810d9
|
sha256: ba6f4bba816c8d7e3c1580e170f3786d216951cc6b94babc3b814c08d2cb2738
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "6.2.0"
|
version: "6.3.0"
|
||||||
xdg_directories:
|
xdg_directories:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -990,10 +982,10 @@ packages:
|
|||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
name: xml
|
name: xml
|
||||||
sha256: "971043b3a0d3da28727e40ed3e0b5d18b742fa5a68665cca88e74b7876d5e025"
|
sha256: "67f0aff7be013d107995e9b75bf4e7f2c3ef2dfdb2c8e68024bba0a7fd5756a4"
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "6.6.1"
|
version: "7.0.1"
|
||||||
yaml:
|
yaml:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
@@ -1003,5 +995,5 @@ packages:
|
|||||||
source: hosted
|
source: hosted
|
||||||
version: "3.1.3"
|
version: "3.1.3"
|
||||||
sdks:
|
sdks:
|
||||||
dart: ">=3.11.5 <4.0.0"
|
dart: ">=3.12.0 <4.0.0"
|
||||||
flutter: ">=3.38.4"
|
flutter: ">=3.44.0"
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
import 'package:flutter_test/flutter_test.dart';
|
||||||
|
|
||||||
|
import 'package:chanora_flutter/services/voice_join_ordering.dart';
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
group('joinVoiceChannelWithIosAudioSession', () {
|
||||||
|
test('activates the iOS audio session before Rust voiceJoin', () async {
|
||||||
|
final calls = <String>[];
|
||||||
|
|
||||||
|
await joinVoiceChannelWithIosAudioSession(
|
||||||
|
channelId: BigInt.from(42),
|
||||||
|
password: 'secret',
|
||||||
|
activateIosAudioSession: () async {
|
||||||
|
calls.add('activateIosAudioSession');
|
||||||
|
},
|
||||||
|
deactivateIosAudioSession: () async {
|
||||||
|
calls.add('deactivateIosAudioSession');
|
||||||
|
},
|
||||||
|
voiceJoin: ({required channelId, required password}) async {
|
||||||
|
expect(channelId, BigInt.from(42));
|
||||||
|
expect(password, 'secret');
|
||||||
|
calls.add('voiceJoin');
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(calls, ['activateIosAudioSession', 'voiceJoin']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('deactivates the iOS audio session when Rust voiceJoin fails',
|
||||||
|
() async {
|
||||||
|
final calls = <String>[];
|
||||||
|
|
||||||
|
await expectLater(
|
||||||
|
joinVoiceChannelWithIosAudioSession(
|
||||||
|
channelId: BigInt.from(42),
|
||||||
|
password: '',
|
||||||
|
activateIosAudioSession: () async {
|
||||||
|
calls.add('activateIosAudioSession');
|
||||||
|
},
|
||||||
|
deactivateIosAudioSession: () async {
|
||||||
|
calls.add('deactivateIosAudioSession');
|
||||||
|
},
|
||||||
|
voiceJoin: ({required channelId, required password}) async {
|
||||||
|
calls.add('voiceJoin');
|
||||||
|
throw StateError('join rejected');
|
||||||
|
},
|
||||||
|
),
|
||||||
|
throwsStateError,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(calls, [
|
||||||
|
'activateIosAudioSession',
|
||||||
|
'voiceJoin',
|
||||||
|
'deactivateIosAudioSession',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'keeps the iOS audio session active when voiceJoin throws but the '
|
||||||
|
'error is recognised as already-in-channel (treated as success); '
|
||||||
|
'still rethrows so the caller runs its success-on-already-joined branch',
|
||||||
|
() async {
|
||||||
|
final calls = <String>[];
|
||||||
|
|
||||||
|
await expectLater(
|
||||||
|
joinVoiceChannelWithIosAudioSession(
|
||||||
|
channelId: BigInt.from(42),
|
||||||
|
password: '',
|
||||||
|
activateIosAudioSession: () async {
|
||||||
|
calls.add('activateIosAudioSession');
|
||||||
|
},
|
||||||
|
deactivateIosAudioSession: () async {
|
||||||
|
calls.add('deactivateIosAudioSession');
|
||||||
|
},
|
||||||
|
voiceJoin: ({required channelId, required password}) async {
|
||||||
|
calls.add('voiceJoin');
|
||||||
|
throw _FakeAlreadyInChannel();
|
||||||
|
},
|
||||||
|
isJoinSuccess: (error) => error is _FakeAlreadyInChannel,
|
||||||
|
),
|
||||||
|
throwsA(isA<_FakeAlreadyInChannel>()),
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(calls, ['activateIosAudioSession', 'voiceJoin']);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
test(
|
||||||
|
'deactivates the iOS audio session when isJoinSuccess returns false '
|
||||||
|
'for a non-success error',
|
||||||
|
() async {
|
||||||
|
final calls = <String>[];
|
||||||
|
|
||||||
|
await expectLater(
|
||||||
|
joinVoiceChannelWithIosAudioSession(
|
||||||
|
channelId: BigInt.from(42),
|
||||||
|
password: '',
|
||||||
|
activateIosAudioSession: () async {
|
||||||
|
calls.add('activateIosAudioSession');
|
||||||
|
},
|
||||||
|
deactivateIosAudioSession: () async {
|
||||||
|
calls.add('deactivateIosAudioSession');
|
||||||
|
},
|
||||||
|
voiceJoin: ({required channelId, required password}) async {
|
||||||
|
calls.add('voiceJoin');
|
||||||
|
throw StateError('join rejected');
|
||||||
|
},
|
||||||
|
isJoinSuccess: (error) => error is _FakeAlreadyInChannel,
|
||||||
|
),
|
||||||
|
throwsStateError,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(calls, [
|
||||||
|
'activateIosAudioSession',
|
||||||
|
'voiceJoin',
|
||||||
|
'deactivateIosAudioSession',
|
||||||
|
]);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
class _FakeAlreadyInChannel implements Exception {}
|
||||||
@@ -13,6 +13,24 @@ void main() {
|
|||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('gated transmit mode segments drop VAD when unsupported', () {
|
||||||
|
expect(
|
||||||
|
transmitModeSegmentsFor(voiceActivityAvailable: false).map((s) => s.value),
|
||||||
|
[rust.BridgeTransmitMode.ptt, rust.BridgeTransmitMode.continuous],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('gated transmit mode segments include VAD when supported', () {
|
||||||
|
expect(
|
||||||
|
transmitModeSegmentsFor(voiceActivityAvailable: true).map((s) => s.value),
|
||||||
|
[
|
||||||
|
rust.BridgeTransmitMode.ptt,
|
||||||
|
rust.BridgeTransmitMode.continuous,
|
||||||
|
rust.BridgeTransmitMode.voiceActivity,
|
||||||
|
],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
test('shared Android processing segments expose hardware and WebRTC', () {
|
test('shared Android processing segments expose hardware and WebRTC', () {
|
||||||
expect(androidProcessingSegments.map((s) => s.value), [true, false]);
|
expect(androidProcessingSegments.map((s) => s.value), [true, false]);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1116,11 +1116,19 @@ impl ChanoraSession {
|
|||||||
/// Configure the preferred Silero ONNX VAD model path on platforms
|
/// Configure the preferred Silero ONNX VAD model path on platforms
|
||||||
/// that ship the ONNX detector.
|
/// that ship the ONNX detector.
|
||||||
///
|
///
|
||||||
/// This does not require an active connection. Running non-iOS
|
/// Set the Silero VAD model path on supported platforms.
|
||||||
/// audio backends can observe the model-path epoch and reload on
|
///
|
||||||
/// the next capture frame when Silero is selected.
|
/// This does not require an active connection. On desktop, the audio
|
||||||
|
/// engine immediately reloads the Silero ONNX worker if one is active
|
||||||
|
/// or if the model file is now available at the new path.
|
||||||
pub async fn set_vad_model_path(&self, path: String) -> Result<(), CoreError> {
|
pub async fn set_vad_model_path(&self, path: String) -> Result<(), CoreError> {
|
||||||
chanora_audio::vad::set_silero_model_path(&path)?;
|
chanora_audio::vad::set_silero_model_path(&path)?;
|
||||||
|
let guard = self.inner.lock().await;
|
||||||
|
if let Some(state) = guard.as_ref() {
|
||||||
|
if let Some(audio) = state.audio.as_ref() {
|
||||||
|
audio.reload_audio_processing_config()?;
|
||||||
|
}
|
||||||
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ coreaudio-rs = "0.14"
|
|||||||
# on the main queue to avoid the VPIO RPC timeout on iOS simulator.
|
# on the main queue to avoid the VPIO RPC timeout on iOS simulator.
|
||||||
dispatch2 = "0.3"
|
dispatch2 = "0.3"
|
||||||
|
|
||||||
[target.'cfg(not(target_os = "ios"))'.dependencies]
|
[target.'cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))'.dependencies]
|
||||||
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "api-24"] }
|
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "api-24"] }
|
||||||
|
|
||||||
[target.'cfg(target_os = "android")'.dependencies]
|
[target.'cfg(target_os = "android")'.dependencies]
|
||||||
|
|||||||
@@ -56,10 +56,8 @@ impl AudioRoute {
|
|||||||
/// iOS voice-processing mode.
|
/// iOS voice-processing mode.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum IosVoiceProcessingMode {
|
pub enum IosVoiceProcessingMode {
|
||||||
/// Shipping default: Apple VoiceProcessingIO owns AEC/NS/AGC.
|
/// Apple VoiceProcessingIO owns AEC/NS/AGC.
|
||||||
PlatformVoiceProcessing,
|
PlatformVoiceProcessing,
|
||||||
/// Experimental raw capture-processing path.
|
|
||||||
SonoraExperimental,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Processing backend selected by policy/config.
|
/// Processing backend selected by policy/config.
|
||||||
@@ -195,28 +193,20 @@ impl AudioProcessingConfig {
|
|||||||
"bluetooth_a2dp is output-only and cannot transmit duplex voice".to_string(),
|
"bluetooth_a2dp is output-only and cannot transmit duplex voice".to_string(),
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
if self.ios_mode == IosVoiceProcessingMode::PlatformVoiceProcessing
|
if self.processing_backend == AudioBackend::Sonora
|
||||||
&& (self.processing_backend == AudioBackend::Sonora
|
|| self.processing_backend == AudioBackend::WebrtcApm
|
||||||
|| self.processing_backend == AudioBackend::WebrtcApm
|
|| self.aec == EffectOwner::Sonora
|
||||||
|| self.aec == EffectOwner::Sonora
|
|| self.aec == EffectOwner::WebrtcApm
|
||||||
|| self.aec == EffectOwner::WebrtcApm
|
|| self.ns == EffectOwner::Sonora
|
||||||
|| self.ns == EffectOwner::Sonora
|
|| self.ns == EffectOwner::WebrtcApm
|
||||||
|| self.ns == EffectOwner::WebrtcApm
|
|| self.agc == EffectOwner::Sonora
|
||||||
|| self.agc == EffectOwner::Sonora
|
|| self.agc == EffectOwner::WebrtcApm
|
||||||
|| self.agc == EffectOwner::WebrtcApm)
|
|
||||||
{
|
{
|
||||||
return Err(AudioError::InvalidAudioProcessingConfig(
|
return Err(AudioError::InvalidAudioProcessingConfig(
|
||||||
"software audio processing cannot be enabled with iOS VoiceProcessingIO"
|
"software audio processing cannot be enabled with iOS VoiceProcessingIO"
|
||||||
.to_string(),
|
.to_string(),
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
if self.ios_mode == IosVoiceProcessingMode::SonoraExperimental
|
|
||||||
&& self.processing_backend != AudioBackend::WebrtcApm
|
|
||||||
{
|
|
||||||
return Err(AudioError::InvalidAudioProcessingConfig(
|
|
||||||
"ios raw processing mode requires the WebRTC APM backend".to_string(),
|
|
||||||
));
|
|
||||||
}
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -262,34 +252,6 @@ mod tests {
|
|||||||
assert!(config.validate_for_ios().is_err());
|
assert!(config.validate_for_ios().is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn raw_processing_allows_full_webrtc_apm_chain() {
|
|
||||||
let config = AudioProcessingConfig {
|
|
||||||
ios_mode: IosVoiceProcessingMode::SonoraExperimental,
|
|
||||||
processing_backend: AudioBackend::WebrtcApm,
|
|
||||||
aec: EffectOwner::WebrtcApm,
|
|
||||||
ns: EffectOwner::WebrtcApm,
|
|
||||||
agc: EffectOwner::WebrtcApm,
|
|
||||||
..AudioProcessingConfig::default()
|
|
||||||
};
|
|
||||||
|
|
||||||
assert!(config.validate_for_ios().is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn raw_processing_rejects_non_webrtc_apm_backend() {
|
|
||||||
let config = AudioProcessingConfig {
|
|
||||||
ios_mode: IosVoiceProcessingMode::SonoraExperimental,
|
|
||||||
processing_backend: AudioBackend::PlatformVoiceProcessing,
|
|
||||||
aec: EffectOwner::WebrtcApm,
|
|
||||||
ns: EffectOwner::WebrtcApm,
|
|
||||||
agc: EffectOwner::WebrtcApm,
|
|
||||||
..AudioProcessingConfig::default()
|
|
||||||
};
|
|
||||||
|
|
||||||
assert!(config.validate_for_ios().is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn disable_failed_vad_backend_demotes_to_webrtc() {
|
fn disable_failed_vad_backend_demotes_to_webrtc() {
|
||||||
let mut config = AudioProcessingConfig {
|
let mut config = AudioProcessingConfig {
|
||||||
@@ -404,10 +366,8 @@ impl Default for SharedAudioProcessingStats {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl SharedAudioProcessingStats {
|
impl SharedAudioProcessingStats {
|
||||||
/// Store the raw input dBFS level (desktop capture path).
|
/// Store the raw input dBFS level for capture paths that do not
|
||||||
/// Mobile platforms use [`Self::update_capture`] instead, which
|
/// update the full processing/VAD snapshot on this callback.
|
||||||
/// also records VAD state; this lighter method is for the cpal
|
|
||||||
/// capture path that has no VAD pipeline.
|
|
||||||
pub fn set_input_dbfs(&self, dbfs: f32) {
|
pub fn set_input_dbfs(&self, dbfs: f32) {
|
||||||
self.input_dbfs.store(dbfs.to_bits(), Ordering::Relaxed);
|
self.input_dbfs.store(dbfs.to_bits(), Ordering::Relaxed);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -314,15 +314,6 @@ mod tests {
|
|||||||
rec.stop();
|
rec.stop();
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn ios_raw_debug_wav_does_not_push_from_realtime_callback() {
|
|
||||||
let src = include_str!("ios_raw_unit.rs");
|
|
||||||
assert!(
|
|
||||||
!src.contains("push_raw_mic") && !src.contains("push_processed_mic"),
|
|
||||||
"ios raw callbacks must not call WavDebugRecorder push_*_mic until it has a preallocated handoff"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn wav_header_is_44_bytes() {
|
fn wav_header_is_44_bytes() {
|
||||||
// Write to a temp file to test the header.
|
// Write to a temp file to test the header.
|
||||||
|
|||||||
@@ -307,6 +307,8 @@ pub struct AudioEngine {
|
|||||||
output_muted: Arc<AtomicBool>,
|
output_muted: Arc<AtomicBool>,
|
||||||
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
||||||
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
#[cfg(not(target_os = "android"))]
|
#[cfg(not(target_os = "android"))]
|
||||||
audio_handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
|
audio_handler: Arc<Mutex<AudioHandler<SessionAudioId>>>,
|
||||||
#[cfg(target_os = "android")]
|
#[cfg(target_os = "android")]
|
||||||
@@ -393,8 +395,6 @@ unsafe impl Sync for AudioEngine {}
|
|||||||
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
||||||
enum IosVoiceBackend {
|
enum IosVoiceBackend {
|
||||||
Vpio(crate::ios_voice_unit::IosVoiceUnit),
|
Vpio(crate::ios_voice_unit::IosVoiceUnit),
|
||||||
#[cfg(target_os = "ios")]
|
|
||||||
Raw(crate::ios_raw_unit::IosRawUnit),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
||||||
@@ -404,14 +404,12 @@ impl IosVoiceBackend {
|
|||||||
{
|
{
|
||||||
match self {
|
match self {
|
||||||
Self::Vpio(unit) => unit.restart(),
|
Self::Vpio(unit) => unit.restart(),
|
||||||
Self::Raw(unit) => unit.restart(),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
#[cfg(target_os = "macos")]
|
#[cfg(target_os = "macos")]
|
||||||
{
|
{
|
||||||
match self {
|
let _ = self;
|
||||||
Self::Vpio(_unit) => Ok(()),
|
Ok(())
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -420,14 +418,12 @@ impl IosVoiceBackend {
|
|||||||
{
|
{
|
||||||
match self {
|
match self {
|
||||||
Self::Vpio(unit) => unit.pause(),
|
Self::Vpio(unit) => unit.pause(),
|
||||||
Self::Raw(unit) => unit.pause(),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
#[cfg(target_os = "macos")]
|
#[cfg(target_os = "macos")]
|
||||||
{
|
{
|
||||||
match self {
|
let _ = self;
|
||||||
Self::Vpio(_unit) => Ok(()),
|
Ok(())
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -436,14 +432,12 @@ impl IosVoiceBackend {
|
|||||||
{
|
{
|
||||||
match self {
|
match self {
|
||||||
Self::Vpio(unit) => unit.resume(),
|
Self::Vpio(unit) => unit.resume(),
|
||||||
Self::Raw(unit) => unit.resume(),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
#[cfg(target_os = "macos")]
|
#[cfg(target_os = "macos")]
|
||||||
{
|
{
|
||||||
match self {
|
let _ = self;
|
||||||
Self::Vpio(_unit) => Ok(()),
|
Ok(())
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -452,27 +446,6 @@ impl IosVoiceBackend {
|
|||||||
fn open_ios_voice_backend(
|
fn open_ios_voice_backend(
|
||||||
params: crate::mobile_voice_backend::VoiceAudioParams,
|
params: crate::mobile_voice_backend::VoiceAudioParams,
|
||||||
) -> Result<IosVoiceBackend, AudioError> {
|
) -> Result<IosVoiceBackend, AudioError> {
|
||||||
let _cfg = params.audio_processing_config.lock().unwrap().clone();
|
|
||||||
#[cfg(target_os = "ios")]
|
|
||||||
{
|
|
||||||
if _cfg.ios_mode == crate::IosVoiceProcessingMode::SonoraExperimental {
|
|
||||||
let raw_params = params.clone();
|
|
||||||
match crate::ios_raw_unit::IosRawUnit::start(raw_params) {
|
|
||||||
Ok(unit) => {
|
|
||||||
info!(target: "chanora_audio", "ios: RemoteIO/WebRTC APM backend selected");
|
|
||||||
return Ok(IosVoiceBackend::Raw(unit));
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
warn!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
error = %e,
|
|
||||||
"ios: RemoteIO/WebRTC APM backend failed; falling back to VoiceProcessingIO"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let unit = crate::ios_voice_unit::IosVoiceUnit::start(params)?;
|
let unit = crate::ios_voice_unit::IosVoiceUnit::start(params)?;
|
||||||
Ok(IosVoiceBackend::Vpio(unit))
|
Ok(IosVoiceBackend::Vpio(unit))
|
||||||
}
|
}
|
||||||
@@ -741,7 +714,7 @@ impl AudioEngine {
|
|||||||
) -> Option<cpal::Device>
|
) -> Option<cpal::Device>
|
||||||
where
|
where
|
||||||
DefaultFn: Fn(&cpal::Host) -> Option<cpal::Device>,
|
DefaultFn: Fn(&cpal::Host) -> Option<cpal::Device>,
|
||||||
AllFn: Fn(&cpal::Host) -> Result<Devices, cpal::DevicesError>,
|
AllFn: Fn(&cpal::Host) -> Result<Devices, cpal::Error>,
|
||||||
Devices: IntoIterator<Item = cpal::Device>,
|
Devices: IntoIterator<Item = cpal::Device>,
|
||||||
{
|
{
|
||||||
if let Some(id) = prefer {
|
if let Some(id) = prefer {
|
||||||
@@ -823,8 +796,8 @@ impl AudioEngine {
|
|||||||
let frames_received = Arc::new(AtomicU32::new(0));
|
let frames_received = Arc::new(AtomicU32::new(0));
|
||||||
let output_gain = Arc::new(AtomicU32::new(1.0_f32.to_bits()));
|
let output_gain = Arc::new(AtomicU32::new(1.0_f32.to_bits()));
|
||||||
let output_muted = Arc::new(AtomicBool::new(false));
|
let output_muted = Arc::new(AtomicBool::new(false));
|
||||||
let audio_processing_config = Arc::new(Mutex::new(crate::AudioProcessingConfig::default()));
|
let (audio_processing_config, audio_processing_stats, silero_vad_worker) =
|
||||||
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
new_desktop_audio_processing_state();
|
||||||
|
|
||||||
// ---------- Capture ----------
|
// ---------- Capture ----------
|
||||||
// Capture is best-effort. If the platform default input
|
// Capture is best-effort. If the platform default input
|
||||||
@@ -838,6 +811,9 @@ impl AudioEngine {
|
|||||||
transmit_flag_for_capture,
|
transmit_flag_for_capture,
|
||||||
frames_sent.clone(),
|
frames_sent.clone(),
|
||||||
cfg.mic_gain,
|
cfg.mic_gain,
|
||||||
|
cfg.voice_activity_selector.clone(),
|
||||||
|
audio_processing_config.clone(),
|
||||||
|
silero_vad_worker.clone(),
|
||||||
audio_processing_stats.clone(),
|
audio_processing_stats.clone(),
|
||||||
);
|
);
|
||||||
let (input_stream, capture_active) = match capture_result {
|
let (input_stream, capture_active) = match capture_result {
|
||||||
@@ -981,11 +957,13 @@ impl AudioEngine {
|
|||||||
|
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
transmit_gate,
|
transmit_gate,
|
||||||
|
frames_sent,
|
||||||
frames_received,
|
frames_received,
|
||||||
output_gain,
|
output_gain,
|
||||||
output_muted,
|
output_muted,
|
||||||
audio_processing_config,
|
audio_processing_config,
|
||||||
audio_processing_stats,
|
audio_processing_stats,
|
||||||
|
silero_vad_worker,
|
||||||
audio_handler,
|
audio_handler,
|
||||||
_input_stream: Mutex::new(input_stream),
|
_input_stream: Mutex::new(input_stream),
|
||||||
_output_stream: Mutex::new(Some(output_stream)),
|
_output_stream: Mutex::new(Some(output_stream)),
|
||||||
@@ -1667,15 +1645,40 @@ impl AudioEngine {
|
|||||||
/// Apply a voice-processing config after validating iOS invariants.
|
/// Apply a voice-processing config after validating iOS invariants.
|
||||||
pub fn set_audio_processing_config(
|
pub fn set_audio_processing_config(
|
||||||
&self,
|
&self,
|
||||||
config: crate::AudioProcessingConfig,
|
mut config: crate::AudioProcessingConfig,
|
||||||
) -> Result<(), AudioError> {
|
) -> Result<(), AudioError> {
|
||||||
#[cfg(target_os = "ios")]
|
#[cfg(target_os = "ios")]
|
||||||
config.validate_for_ios()?;
|
config.validate_for_ios()?;
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
{
|
||||||
|
config.processing_backend = crate::AudioBackend::Noop;
|
||||||
|
}
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
self.apply_desktop_vad_backend(&config);
|
||||||
let mut guard = self.audio_processing_config.lock().unwrap();
|
let mut guard = self.audio_processing_config.lock().unwrap();
|
||||||
*guard = config;
|
*guard = config;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Re-apply the current audio processing config.
|
||||||
|
///
|
||||||
|
/// Used by the core layer to trigger VAD worker reload after a
|
||||||
|
/// model-path change (the epoch increments but the worker is only
|
||||||
|
/// reconstructed when `set_audio_processing_config` is called).
|
||||||
|
pub fn reload_audio_processing_config(&self) -> Result<(), AudioError> {
|
||||||
|
let config = self.audio_processing_config_snapshot();
|
||||||
|
self.set_audio_processing_config(config)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
fn apply_desktop_vad_backend(&self, config: &crate::AudioProcessingConfig) {
|
||||||
|
apply_desktop_vad_backend_to_worker(
|
||||||
|
config,
|
||||||
|
&self.silero_vad_worker,
|
||||||
|
self.audio_processing_stats.as_ref(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/// Current voice-processing stats snapshot.
|
/// Current voice-processing stats snapshot.
|
||||||
pub fn audio_processing_stats(&self) -> crate::AudioProcessingStats {
|
pub fn audio_processing_stats(&self) -> crate::AudioProcessingStats {
|
||||||
let config = self.audio_processing_config.lock().unwrap().clone();
|
let config = self.audio_processing_config.lock().unwrap().clone();
|
||||||
@@ -1777,10 +1780,369 @@ fn release_android_audio_mode_for_startup_rollback(
|
|||||||
audio_mode_stack.release()
|
audio_mode_stack.release()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
fn new_desktop_audio_processing_state() -> (
|
||||||
|
Arc<Mutex<crate::AudioProcessingConfig>>,
|
||||||
|
Arc<crate::SharedAudioProcessingStats>,
|
||||||
|
Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
|
) {
|
||||||
|
let mut config = crate::AudioProcessingConfig::default();
|
||||||
|
// Desktop cpal capture does not use a platform voice-processing API.
|
||||||
|
// The global default (PlatformVoiceProcessing) is correct for iOS/macOS
|
||||||
|
// VPIO but would mislabel the desktop path in bridge diagnostics and
|
||||||
|
// set `platform_voice_processing_enabled = true` when no such
|
||||||
|
// processing exists. Override to Noop; the bridge/UI stats layer
|
||||||
|
// will then report the accurate backend.
|
||||||
|
config.processing_backend = crate::AudioBackend::Noop;
|
||||||
|
let audio_processing_config = Arc::new(Mutex::new(config.clone()));
|
||||||
|
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let silero_vad_worker = Arc::new(Mutex::new(None));
|
||||||
|
apply_desktop_vad_backend_to_worker(
|
||||||
|
&config,
|
||||||
|
&silero_vad_worker,
|
||||||
|
audio_processing_stats.as_ref(),
|
||||||
|
);
|
||||||
|
(
|
||||||
|
audio_processing_config,
|
||||||
|
audio_processing_stats,
|
||||||
|
silero_vad_worker,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
fn apply_desktop_vad_backend_to_worker(
|
||||||
|
config: &crate::AudioProcessingConfig,
|
||||||
|
silero_vad_worker: &Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
|
audio_processing_stats: &crate::SharedAudioProcessingStats,
|
||||||
|
) {
|
||||||
|
if config.vad_backend != crate::VadBackend::SileroOnnx {
|
||||||
|
let mut worker_guard = silero_vad_worker.lock().unwrap();
|
||||||
|
if worker_guard.is_some() {
|
||||||
|
info!(
|
||||||
|
target: "chanora_audio",
|
||||||
|
backend = config.vad_backend.as_str(),
|
||||||
|
"desktop: Silero ONNX VAD worker cleared because another VAD backend is selected"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
*worker_guard = None;
|
||||||
|
audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load model and spawn worker BEFORE taking the lock so the
|
||||||
|
// realtime capture callback is not blocked on try_lock() during
|
||||||
|
// model I/O + thread spawn. The old worker (if any) is dropped
|
||||||
|
// after the new one is installed under the short lock hold.
|
||||||
|
let model_path = crate::vad::silero_model_bundle_path();
|
||||||
|
info!(
|
||||||
|
target: "chanora_audio",
|
||||||
|
path = %model_path,
|
||||||
|
"desktop: Silero ONNX VAD selected; loading model worker"
|
||||||
|
);
|
||||||
|
let new_worker = crate::vad::silero_onnx::SileroOnnxVadWorker::try_new(&model_path);
|
||||||
|
{
|
||||||
|
let mut worker_guard = silero_vad_worker.lock().unwrap();
|
||||||
|
*worker_guard = new_worker;
|
||||||
|
}
|
||||||
|
// Check result after releasing the lock. Re-acquire is cheap and
|
||||||
|
// ensures we log the correct state without holding the mutex.
|
||||||
|
let worker_installed = silero_vad_worker.lock().unwrap().is_some();
|
||||||
|
if worker_installed {
|
||||||
|
info!(
|
||||||
|
target: "chanora_audio",
|
||||||
|
path = %model_path,
|
||||||
|
"desktop: Silero ONNX VAD worker loaded"
|
||||||
|
);
|
||||||
|
audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
} else {
|
||||||
|
warn!(
|
||||||
|
target: "chanora_audio",
|
||||||
|
path = %model_path,
|
||||||
|
"desktop: Silero ONNX VAD worker unavailable; WebRTC fallback will be used"
|
||||||
|
);
|
||||||
|
audio_processing_stats.set_vad_fallback_active(true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_capture_voice_activity_opens_selector_from_speech() {
|
||||||
|
let gate = crate::ptt::AudioTransmitGate::new(false);
|
||||||
|
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
|
||||||
|
selector.set_mode(crate::TransmitMode::VoiceActivity);
|
||||||
|
selector.set_in_channel(true);
|
||||||
|
|
||||||
|
let encoder = crate::opus_voice::new_voip_encoder("desktop VAD test").unwrap();
|
||||||
|
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
|
||||||
|
let frames_sent = Arc::new(AtomicU32::new(0));
|
||||||
|
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
|
||||||
|
voice_out_tx,
|
||||||
|
frames_sent,
|
||||||
|
"desktop-vad-test",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let mut capture = CaptureState::new(
|
||||||
|
encoder,
|
||||||
|
SAMPLE_RATE,
|
||||||
|
1,
|
||||||
|
1.0,
|
||||||
|
voice_out_tx,
|
||||||
|
gate.flag_arc(),
|
||||||
|
Some(selector.clone()),
|
||||||
|
Arc::new(Mutex::new(crate::AudioProcessingConfig {
|
||||||
|
vad_backend: crate::VadBackend::WebrtcVad,
|
||||||
|
..crate::AudioProcessingConfig::default()
|
||||||
|
})),
|
||||||
|
Arc::new(Mutex::new(None)),
|
||||||
|
stats.clone(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
|
||||||
|
for (idx, sample) in voiced.iter_mut().enumerate() {
|
||||||
|
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
|
||||||
|
*sample = phase.sin() * 0.4;
|
||||||
|
}
|
||||||
|
for _ in 0..6 {
|
||||||
|
capture.ingest(&voiced);
|
||||||
|
}
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
selector.voice_activity_open(),
|
||||||
|
"desktop capture must feed VAD and open VoiceActivity selector before transmit is already active"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
gate.load(),
|
||||||
|
"VoiceActivity selector should publish transmit gate"
|
||||||
|
);
|
||||||
|
let snapshot = stats.snapshot(&crate::AudioProcessingConfig::default());
|
||||||
|
assert!(snapshot.vad_active, "stats should expose active VAD");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_startup_audio_processing_state_applies_default_silero_fallback() {
|
||||||
|
let _guard = crate::vad::SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
|
||||||
|
crate::vad::clear_silero_model_path_for_test();
|
||||||
|
|
||||||
|
let (config, stats, worker) = new_desktop_audio_processing_state();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
config.lock().unwrap().vad_backend,
|
||||||
|
crate::VadBackend::SileroOnnx,
|
||||||
|
"desktop startup config should keep the default Silero backend selected"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
worker.lock().unwrap().is_none(),
|
||||||
|
"missing startup model should not create an ONNX worker"
|
||||||
|
);
|
||||||
|
let snapshot = stats.snapshot(&config.lock().unwrap());
|
||||||
|
assert!(
|
||||||
|
snapshot.vad_fallback_active,
|
||||||
|
"desktop startup should mark WebRTC fallback active when the default Silero worker cannot load"
|
||||||
|
);
|
||||||
|
crate::vad::clear_silero_model_path_for_test();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_set_audio_processing_config_normalizes_default_backend_to_noop() {
|
||||||
|
let audio_processing_config = Arc::new(Mutex::new(crate::AudioProcessingConfig::default()));
|
||||||
|
let audio_processing_stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let engine = AudioEngine {
|
||||||
|
transmit_gate: crate::ptt::AudioTransmitGate::new(false),
|
||||||
|
frames_sent: Arc::new(AtomicU32::new(0)),
|
||||||
|
frames_received: Arc::new(AtomicU32::new(0)),
|
||||||
|
output_gain: Arc::new(AtomicU32::new(1.0_f32.to_bits())),
|
||||||
|
output_muted: Arc::new(AtomicBool::new(false)),
|
||||||
|
audio_processing_config: audio_processing_config.clone(),
|
||||||
|
audio_processing_stats,
|
||||||
|
silero_vad_worker: Arc::new(Mutex::new(None)),
|
||||||
|
audio_handler: Arc::new(Mutex::new(AudioHandler::new())),
|
||||||
|
_input_stream: Mutex::new(None),
|
||||||
|
_output_stream: Mutex::new(None),
|
||||||
|
shutdown_tx: None,
|
||||||
|
capture_active: false,
|
||||||
|
};
|
||||||
|
|
||||||
|
engine
|
||||||
|
.set_audio_processing_config(crate::AudioProcessingConfig::default())
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
engine.audio_processing_config_snapshot().processing_backend,
|
||||||
|
crate::AudioBackend::Noop,
|
||||||
|
"desktop setter should report the actual no-op processing backend for default configs"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_capture_silero_selection_reports_fallback_when_worker_unavailable() {
|
||||||
|
let _guard = crate::vad::SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
|
||||||
|
crate::vad::clear_silero_model_path_for_test();
|
||||||
|
|
||||||
|
let gate = crate::ptt::AudioTransmitGate::new(false);
|
||||||
|
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
|
||||||
|
selector.set_mode(crate::TransmitMode::VoiceActivity);
|
||||||
|
selector.set_in_channel(true);
|
||||||
|
|
||||||
|
let encoder = crate::opus_voice::new_voip_encoder("desktop Silero fallback test").unwrap();
|
||||||
|
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
|
||||||
|
let frames_sent = Arc::new(AtomicU32::new(0));
|
||||||
|
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
|
||||||
|
voice_out_tx,
|
||||||
|
frames_sent,
|
||||||
|
"desktop-silero-fallback-test",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let config = Arc::new(Mutex::new(crate::AudioProcessingConfig {
|
||||||
|
vad_backend: crate::VadBackend::SileroOnnx,
|
||||||
|
..crate::AudioProcessingConfig::default()
|
||||||
|
}));
|
||||||
|
let mut capture = CaptureState::new(
|
||||||
|
encoder,
|
||||||
|
SAMPLE_RATE,
|
||||||
|
1,
|
||||||
|
1.0,
|
||||||
|
voice_out_tx,
|
||||||
|
gate.flag_arc(),
|
||||||
|
Some(selector),
|
||||||
|
config.clone(),
|
||||||
|
Arc::new(Mutex::new(None)),
|
||||||
|
stats.clone(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
|
||||||
|
for (idx, sample) in voiced.iter_mut().enumerate() {
|
||||||
|
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
|
||||||
|
*sample = phase.sin() * 0.4;
|
||||||
|
}
|
||||||
|
capture.ingest(&voiced);
|
||||||
|
|
||||||
|
let snapshot = stats.snapshot(&config.lock().unwrap());
|
||||||
|
assert_eq!(snapshot.vad_backend, crate::VadBackend::SileroOnnx);
|
||||||
|
assert!(
|
||||||
|
snapshot.vad_fallback_active,
|
||||||
|
"desktop Silero selection should make WebRTC fallback visible when ONNX worker cannot load"
|
||||||
|
);
|
||||||
|
crate::vad::clear_silero_model_path_for_test();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_capture_vad_consumes_only_new_pcm_while_transmitting() {
|
||||||
|
let gate = crate::ptt::AudioTransmitGate::new(true);
|
||||||
|
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
|
||||||
|
selector.set_mode(crate::TransmitMode::VoiceActivity);
|
||||||
|
selector.set_in_channel(true);
|
||||||
|
|
||||||
|
let encoder = crate::opus_voice::new_voip_encoder("desktop VAD duplicate test").unwrap();
|
||||||
|
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
|
||||||
|
let frames_sent = Arc::new(AtomicU32::new(0));
|
||||||
|
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
|
||||||
|
voice_out_tx,
|
||||||
|
frames_sent,
|
||||||
|
"desktop-vad-duplicate-test",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let mut capture = CaptureState::new(
|
||||||
|
encoder,
|
||||||
|
SAMPLE_RATE,
|
||||||
|
1,
|
||||||
|
1.0,
|
||||||
|
voice_out_tx,
|
||||||
|
gate.flag_arc(),
|
||||||
|
Some(selector),
|
||||||
|
Arc::new(Mutex::new(crate::AudioProcessingConfig {
|
||||||
|
vad_backend: crate::VadBackend::Disabled,
|
||||||
|
..crate::AudioProcessingConfig::default()
|
||||||
|
})),
|
||||||
|
Arc::new(Mutex::new(None)),
|
||||||
|
stats,
|
||||||
|
);
|
||||||
|
|
||||||
|
capture.pcm_accum.extend(std::iter::repeat_n(
|
||||||
|
0.1_f32,
|
||||||
|
crate::frame::FRAME_10MS_SAMPLES + 240,
|
||||||
|
));
|
||||||
|
capture.pending_10ms[..240].fill(0.1);
|
||||||
|
capture.pending_10ms_len = 240;
|
||||||
|
capture.capture_frame_seq = 1;
|
||||||
|
capture.pcm_accum.extend(std::iter::repeat_n(0.2_f32, 240));
|
||||||
|
|
||||||
|
capture.process_pending_vad_frames(crate::frame::FRAME_10MS_SAMPLES + 240);
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
capture.capture_frame_seq, 2,
|
||||||
|
"VAD should consume only the 240 samples appended by the current ingest and complete one pending 10 ms frame"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
|
#[test]
|
||||||
|
fn desktop_capture_silero_stale_enqueued_worker_uses_fallback() {
|
||||||
|
let gate = crate::ptt::AudioTransmitGate::new(false);
|
||||||
|
let selector = Arc::new(crate::TransmitModeSelector::new(gate.clone()));
|
||||||
|
selector.set_mode(crate::TransmitMode::VoiceActivity);
|
||||||
|
selector.set_in_channel(true);
|
||||||
|
|
||||||
|
let encoder =
|
||||||
|
crate::opus_voice::new_voip_encoder("desktop stale Silero fallback test").unwrap();
|
||||||
|
let (voice_out_tx, _voice_out_rx) = mpsc::channel::<OutPacket>(16);
|
||||||
|
let frames_sent = Arc::new(AtomicU32::new(0));
|
||||||
|
let voice_out_tx = crate::opus_voice::start_out_packet_worker(
|
||||||
|
voice_out_tx,
|
||||||
|
frames_sent,
|
||||||
|
"desktop-stale-silero-fallback-test",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let stats = Arc::new(crate::SharedAudioProcessingStats::default());
|
||||||
|
let config = Arc::new(Mutex::new(crate::AudioProcessingConfig {
|
||||||
|
vad_backend: crate::VadBackend::SileroOnnx,
|
||||||
|
..crate::AudioProcessingConfig::default()
|
||||||
|
}));
|
||||||
|
let worker = Arc::new(Mutex::new(Some(
|
||||||
|
crate::vad::silero_onnx::SileroOnnxVadWorker::stale_test_worker(),
|
||||||
|
)));
|
||||||
|
let mut capture = CaptureState::new(
|
||||||
|
encoder,
|
||||||
|
SAMPLE_RATE,
|
||||||
|
1,
|
||||||
|
1.0,
|
||||||
|
voice_out_tx,
|
||||||
|
gate.flag_arc(),
|
||||||
|
Some(selector),
|
||||||
|
config.clone(),
|
||||||
|
worker,
|
||||||
|
stats.clone(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut voiced = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
|
||||||
|
for (idx, sample) in voiced.iter_mut().enumerate() {
|
||||||
|
let phase = idx as f32 * 2.0 * std::f32::consts::PI * 220.0 / SAMPLE_RATE as f32;
|
||||||
|
*sample = phase.sin() * 0.4;
|
||||||
|
}
|
||||||
|
|
||||||
|
capture.process_10ms_capture_frame(&voiced);
|
||||||
|
|
||||||
|
let snapshot = stats.snapshot(&config.lock().unwrap());
|
||||||
|
assert!(
|
||||||
|
snapshot.vad_fallback_active,
|
||||||
|
"stale Silero worker output should report active WebRTC fallback"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
snapshot.vad_probability > 0.5,
|
||||||
|
"stale Silero worker output should use WebRTC fallback probability instead of forced silence"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn android_startup_rollback_releases_acquired_mode_snapshot() {
|
fn android_startup_rollback_releases_acquired_mode_snapshot() {
|
||||||
let mut stack = crate::mode_stack::ModeStack::new();
|
let mut stack = crate::mode_stack::ModeStack::new();
|
||||||
@@ -1806,6 +2168,9 @@ fn try_open_capture(
|
|||||||
transmit_active: Arc<AtomicBool>,
|
transmit_active: Arc<AtomicBool>,
|
||||||
frames_sent: Arc<AtomicU32>,
|
frames_sent: Arc<AtomicU32>,
|
||||||
mic_gain: f32,
|
mic_gain: f32,
|
||||||
|
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
|
||||||
|
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
||||||
|
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
||||||
) -> Result<cpal::Stream, AudioError> {
|
) -> Result<cpal::Stream, AudioError> {
|
||||||
let in_cfg = in_dev
|
let in_cfg = in_dev
|
||||||
@@ -1845,6 +2210,9 @@ fn try_open_capture(
|
|||||||
"cpal-capture",
|
"cpal-capture",
|
||||||
)?,
|
)?,
|
||||||
transmit_active,
|
transmit_active,
|
||||||
|
voice_activity_selector,
|
||||||
|
audio_processing_config,
|
||||||
|
silero_vad_worker,
|
||||||
audio_processing_stats,
|
audio_processing_stats,
|
||||||
)));
|
)));
|
||||||
|
|
||||||
@@ -1883,6 +2251,17 @@ struct CaptureState {
|
|||||||
/// The PTT transmission gate. Read once per outbound frame; the
|
/// The PTT transmission gate. Read once per outbound frame; the
|
||||||
/// CaptureState never mutates this flag.
|
/// CaptureState never mutates this flag.
|
||||||
transmit_active: Arc<AtomicBool>,
|
transmit_active: Arc<AtomicBool>,
|
||||||
|
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
|
||||||
|
vad_detector: crate::vad::WebRtcFallbackVad,
|
||||||
|
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
|
silero_model_epoch: u64,
|
||||||
|
current_vad_backend: crate::VadBackend,
|
||||||
|
fallback_warned_backend: Option<crate::VadBackend>,
|
||||||
|
capture_frame_seq: u64,
|
||||||
|
vad_state: crate::voice_activity::VoiceActivityStateMachine,
|
||||||
|
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
||||||
|
pending_10ms: [f32; crate::frame::FRAME_10MS_SAMPLES],
|
||||||
|
pending_10ms_len: usize,
|
||||||
/// Pre-allocated mono downmix buffer. Resized in-place each
|
/// Pre-allocated mono downmix buffer. Resized in-place each
|
||||||
/// callback; `clear()` retains capacity. SDD-094 realtime-thread
|
/// callback; `clear()` retains capacity. SDD-094 realtime-thread
|
||||||
/// invariant: this avoids the heap allocation that the prior fix
|
/// invariant: this avoids the heap allocation that the prior fix
|
||||||
@@ -1926,6 +2305,9 @@ impl CaptureState {
|
|||||||
mic_gain: f32,
|
mic_gain: f32,
|
||||||
voice_out_tx: crate::opus_voice::EncodedVoiceFrameSender,
|
voice_out_tx: crate::opus_voice::EncodedVoiceFrameSender,
|
||||||
transmit_active: Arc<AtomicBool>,
|
transmit_active: Arc<AtomicBool>,
|
||||||
|
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
|
||||||
|
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
||||||
|
silero_vad_worker: Arc<Mutex<Option<crate::vad::silero_onnx::SileroOnnxVadWorker>>>,
|
||||||
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
||||||
) -> Self {
|
) -> Self {
|
||||||
Self {
|
Self {
|
||||||
@@ -1939,6 +2321,17 @@ impl CaptureState {
|
|||||||
opus_out: [0u8; crate::opus_voice::MAX_OPUS_FRAME],
|
opus_out: [0u8; crate::opus_voice::MAX_OPUS_FRAME],
|
||||||
voice_out_tx,
|
voice_out_tx,
|
||||||
transmit_active,
|
transmit_active,
|
||||||
|
voice_activity_selector,
|
||||||
|
vad_detector: crate::vad::WebRtcFallbackVad::default(),
|
||||||
|
silero_vad_worker,
|
||||||
|
silero_model_epoch: crate::vad::silero_model_epoch(),
|
||||||
|
current_vad_backend: crate::VadBackend::Disabled,
|
||||||
|
fallback_warned_backend: None,
|
||||||
|
capture_frame_seq: 0,
|
||||||
|
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
|
||||||
|
audio_processing_config,
|
||||||
|
pending_10ms: [0.0; crate::frame::FRAME_10MS_SAMPLES],
|
||||||
|
pending_10ms_len: 0,
|
||||||
mono_scratch: Vec::with_capacity(4096),
|
mono_scratch: Vec::with_capacity(4096),
|
||||||
frame_scratch: Vec::with_capacity(FRAME_SAMPLES),
|
frame_scratch: Vec::with_capacity(FRAME_SAMPLES),
|
||||||
audio_processing_stats,
|
audio_processing_stats,
|
||||||
@@ -1952,6 +2345,16 @@ impl CaptureState {
|
|||||||
/// Consume an arbitrary-rate, multichannel cpal buffer; produce
|
/// Consume an arbitrary-rate, multichannel cpal buffer; produce
|
||||||
/// 48 kHz mono frames; encode and send when `transmit_active`
|
/// 48 kHz mono frames; encode and send when `transmit_active`
|
||||||
/// is true (PTT engaged).
|
/// is true (PTT engaged).
|
||||||
|
// TODO(realtime-audio): This method runs on the cpal audio callback
|
||||||
|
// thread with a ~10 ms deadline. Known pre-existing violations of the
|
||||||
|
// realtime safety constraint that should be addressed in a future
|
||||||
|
// iteration:
|
||||||
|
// 1. Blocking Mutex::lock().unwrap() on self (via cpal callback)
|
||||||
|
// 2. Potential allocation in mono_scratch.reserve() and
|
||||||
|
// pcm_accum.extend_from_slice() when buffer capacity is exceeded
|
||||||
|
// 3. Encode-path warn!/error! logging via send_voip_frame closures
|
||||||
|
// These were present before the VAD integration and are not
|
||||||
|
// introduced by this changeset.
|
||||||
fn ingest<T: ToF32 + Copy>(&mut self, buf: &[T]) {
|
fn ingest<T: ToF32 + Copy>(&mut self, buf: &[T]) {
|
||||||
// 1. Down-mix to mono (pre-gain). Always performed so the level
|
// 1. Down-mix to mono (pre-gain). Always performed so the level
|
||||||
// meter reflects real mic input even when PTT is released.
|
// meter reflects real mic input even when PTT is released.
|
||||||
@@ -1981,15 +2384,11 @@ impl CaptureState {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if !self.transmit_active.load(Ordering::Relaxed) {
|
|
||||||
self.pcm_accum.clear();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// 2. Resample to 48 kHz if needed. We re-borrow
|
// 2. Resample to 48 kHz if needed. We re-borrow
|
||||||
// `mono_scratch` as a shared slice per branch to satisfy
|
// `mono_scratch` as a shared slice per branch to satisfy
|
||||||
// the borrow checker against `&mut self` on the
|
// the borrow checker against `&mut self` on the
|
||||||
// resample path.
|
// resample path.
|
||||||
|
let vad_start_offset = self.pcm_accum.len();
|
||||||
if self.in_sample_rate == SAMPLE_RATE {
|
if self.in_sample_rate == SAMPLE_RATE {
|
||||||
// Disjoint-borrow: copy the slice into pcm_accum without
|
// Disjoint-borrow: copy the slice into pcm_accum without
|
||||||
// aliasing &mut self.
|
// aliasing &mut self.
|
||||||
@@ -2008,6 +2407,13 @@ impl CaptureState {
|
|||||||
self.mono_scratch = mono;
|
self.mono_scratch = mono;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
self.process_pending_vad_frames(vad_start_offset);
|
||||||
|
|
||||||
|
if !self.transmit_active.load(Ordering::Relaxed) {
|
||||||
|
self.pcm_accum.clear();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
// 3. Encode any complete frames. Clamp each sample to
|
// 3. Encode any complete frames. Clamp each sample to
|
||||||
// [-1.0, 1.0] before handing to libopus's float encoder —
|
// [-1.0, 1.0] before handing to libopus's float encoder —
|
||||||
// out-of-range samples are hard-clipped inside libopus,
|
// out-of-range samples are hard-clipped inside libopus,
|
||||||
@@ -2055,6 +2461,179 @@ impl CaptureState {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn process_pending_vad_frames(&mut self, start_offset: usize) {
|
||||||
|
let mut offset = start_offset.min(self.pcm_accum.len());
|
||||||
|
while offset < self.pcm_accum.len() {
|
||||||
|
let remaining = crate::frame::FRAME_10MS_SAMPLES - self.pending_10ms_len;
|
||||||
|
let take = remaining.min(self.pcm_accum.len() - offset);
|
||||||
|
self.pending_10ms[self.pending_10ms_len..self.pending_10ms_len + take]
|
||||||
|
.copy_from_slice(&self.pcm_accum[offset..offset + take]);
|
||||||
|
self.pending_10ms_len += take;
|
||||||
|
offset += take;
|
||||||
|
|
||||||
|
if self.pending_10ms_len == crate::frame::FRAME_10MS_SAMPLES {
|
||||||
|
let frame = self.pending_10ms;
|
||||||
|
self.process_10ms_capture_frame(&frame);
|
||||||
|
self.pending_10ms_len = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mark_vad_fallback_active(&mut self, failed_backend: crate::VadBackend) {
|
||||||
|
// Realtime callback: do not log here. Publish state via atomics
|
||||||
|
// and let a non-realtime consumer translate transitions into
|
||||||
|
// info/warn events. The transmit/diagnostic stats stream
|
||||||
|
// already exposes vad_fallback_active for this purpose.
|
||||||
|
self.fallback_warned_backend = Some(failed_backend);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sync_vad_backend(&mut self, voice_activity_mode: bool, vad_backend: crate::VadBackend) {
|
||||||
|
if !voice_activity_mode {
|
||||||
|
self.current_vad_backend = crate::VadBackend::Disabled;
|
||||||
|
self.fallback_warned_backend = None;
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let silero_epoch = crate::vad::silero_model_epoch();
|
||||||
|
let silero_changed =
|
||||||
|
vad_backend == crate::VadBackend::SileroOnnx && silero_epoch != self.silero_model_epoch;
|
||||||
|
if vad_backend == self.current_vad_backend && !silero_changed {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
self.current_vad_backend = vad_backend;
|
||||||
|
self.silero_model_epoch = silero_epoch;
|
||||||
|
self.fallback_warned_backend = None;
|
||||||
|
self.vad_state.reset();
|
||||||
|
|
||||||
|
match vad_backend {
|
||||||
|
crate::VadBackend::SileroOnnx => {
|
||||||
|
let worker_available = self
|
||||||
|
.silero_vad_worker
|
||||||
|
.try_lock()
|
||||||
|
.map(|worker| worker.is_some())
|
||||||
|
.unwrap_or(false);
|
||||||
|
if worker_available {
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
} else {
|
||||||
|
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
crate::VadBackend::WebrtcVad => {
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
}
|
||||||
|
crate::VadBackend::EnergyDebug => {
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(true);
|
||||||
|
}
|
||||||
|
crate::VadBackend::Disabled => {
|
||||||
|
self.audio_processing_stats.set_vad_fallback_active(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn process_10ms_capture_frame(&mut self, frame: &[f32; crate::frame::FRAME_10MS_SAMPLES]) {
|
||||||
|
let input_dbfs = crate::frame::dbfs(frame);
|
||||||
|
let (vad_backend, vad_hangover) = self
|
||||||
|
.audio_processing_config
|
||||||
|
.try_lock()
|
||||||
|
.map(|cfg| (cfg.vad_backend, cfg.vad_hangover_ms))
|
||||||
|
.unwrap_or((
|
||||||
|
crate::VadBackend::WebrtcVad,
|
||||||
|
crate::voice_activity::VAD_HANGOVER_MS,
|
||||||
|
));
|
||||||
|
let voice_activity_mode = self
|
||||||
|
.voice_activity_selector
|
||||||
|
.as_ref()
|
||||||
|
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
|
||||||
|
.unwrap_or(false);
|
||||||
|
|
||||||
|
if voice_activity_mode {
|
||||||
|
self.sync_vad_backend(true, vad_backend);
|
||||||
|
self.vad_state.configure(
|
||||||
|
crate::voice_activity::VAD_OPEN_AFTER_MS,
|
||||||
|
vad_hangover,
|
||||||
|
crate::voice_activity::VAD_MIN_TX_MS,
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
self.sync_vad_backend(false, vad_backend);
|
||||||
|
}
|
||||||
|
|
||||||
|
let (vad_probability, gate_open, used_fallback_vad) = if voice_activity_mode {
|
||||||
|
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
|
||||||
|
let capture_seq = self.capture_frame_seq;
|
||||||
|
let mut used_fallback_vad = false;
|
||||||
|
let vad = match vad_backend {
|
||||||
|
crate::VadBackend::Disabled => crate::vad::VadOutput {
|
||||||
|
probability: 1.0,
|
||||||
|
speech: true,
|
||||||
|
},
|
||||||
|
crate::VadBackend::SileroOnnx => {
|
||||||
|
// Single `try_lock` that both probes availability and
|
||||||
|
// sends the frame. The guard is scoped to the block
|
||||||
|
// so it drops before the fallback path (which needs
|
||||||
|
// `&mut self` for `mark_vad_fallback_active`).
|
||||||
|
// Returns Some(VadOutput) on a successful, non-stale
|
||||||
|
// send; None means "fall back to WebRTC VAD".
|
||||||
|
let worker_output = {
|
||||||
|
let guard = self.silero_vad_worker.try_lock().ok();
|
||||||
|
guard.and_then(|guard| {
|
||||||
|
let worker = guard.as_ref()?;
|
||||||
|
if worker.try_send(capture_seq, frame) && !worker.is_stale(capture_seq)
|
||||||
|
{
|
||||||
|
let p = worker.latest_probability();
|
||||||
|
Some(crate::vad::VadOutput {
|
||||||
|
probability: p,
|
||||||
|
speech: p >= 0.5,
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
}
|
||||||
|
})
|
||||||
|
};
|
||||||
|
if let Some(output) = worker_output {
|
||||||
|
output
|
||||||
|
} else {
|
||||||
|
used_fallback_vad = true;
|
||||||
|
self.mark_vad_fallback_active(vad_backend);
|
||||||
|
crate::vad::VoiceActivityDetector::process_10ms(
|
||||||
|
&mut self.vad_detector,
|
||||||
|
frame,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
crate::VadBackend::WebrtcVad | crate::VadBackend::EnergyDebug => {
|
||||||
|
used_fallback_vad = vad_backend == crate::VadBackend::EnergyDebug;
|
||||||
|
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, frame)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
(
|
||||||
|
vad.probability,
|
||||||
|
self.vad_state.update(vad.speech),
|
||||||
|
used_fallback_vad,
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
(0.0, false, false)
|
||||||
|
};
|
||||||
|
self.audio_processing_stats
|
||||||
|
.set_vad_fallback_active(used_fallback_vad);
|
||||||
|
|
||||||
|
let vad_active = voice_activity_mode && gate_open;
|
||||||
|
if let Some(selector) = &self.voice_activity_selector {
|
||||||
|
selector.set_voice_activity_open(vad_active);
|
||||||
|
}
|
||||||
|
self.audio_processing_stats.update_capture(
|
||||||
|
input_dbfs,
|
||||||
|
input_dbfs,
|
||||||
|
vad_probability,
|
||||||
|
vad_active,
|
||||||
|
self.transmit_active.load(Ordering::Relaxed),
|
||||||
|
);
|
||||||
|
self.audio_processing_stats
|
||||||
|
.record_capture_frame(frame.iter().all(|sample| sample.abs() <= 0.000_001));
|
||||||
|
}
|
||||||
|
|
||||||
/// Simple linear resampler for `in_sample_rate → 48000`.
|
/// Simple linear resampler for `in_sample_rate → 48000`.
|
||||||
///
|
///
|
||||||
/// The resampler maintains continuity across cpal buffer
|
/// The resampler maintains continuity across cpal buffer
|
||||||
@@ -2144,8 +2723,8 @@ where
|
|||||||
{
|
{
|
||||||
let stream = device
|
let stream = device
|
||||||
.build_input_stream(
|
.build_input_stream(
|
||||||
config,
|
*config,
|
||||||
move |data: &[T], _| {
|
move |data: &[T], _: &cpal::InputCallbackInfo| {
|
||||||
let mut s = state.lock().unwrap();
|
let mut s = state.lock().unwrap();
|
||||||
s.ingest(data);
|
s.ingest(data);
|
||||||
},
|
},
|
||||||
@@ -2211,8 +2790,8 @@ where
|
|||||||
.unwrap_or_else(std::time::Instant::now);
|
.unwrap_or_else(std::time::Instant::now);
|
||||||
let stream = device
|
let stream = device
|
||||||
.build_output_stream(
|
.build_output_stream(
|
||||||
config,
|
*config,
|
||||||
move |out: &mut [T], _| {
|
move |out: &mut [T], _: &cpal::OutputCallbackInfo| {
|
||||||
let cb_start = std::time::Instant::now();
|
let cb_start = std::time::Instant::now();
|
||||||
let muted = output_muted.load(Ordering::Relaxed);
|
let muted = output_muted.load(Ordering::Relaxed);
|
||||||
let dev_frames = out.len() / dev_channels.max(1);
|
let dev_frames = out.len() / dev_channels.max(1);
|
||||||
@@ -2620,14 +3199,25 @@ pub mod bench_seam {
|
|||||||
let (tx, rx) = mpsc::channel::<OutPacket>(64);
|
let (tx, rx) = mpsc::channel::<OutPacket>(64);
|
||||||
let transmit_active = Arc::new(AtomicBool::new(true));
|
let transmit_active = Arc::new(AtomicBool::new(true));
|
||||||
let frames_sent = Arc::new(AtomicU32::new(0));
|
let frames_sent = Arc::new(AtomicU32::new(0));
|
||||||
|
// Bridge the bench's private mpsc<OutPacket> to the
|
||||||
|
// realtime-thread-safe EncodedVoiceFrameSender that
|
||||||
|
// CaptureState expects. The worker task forwards
|
||||||
|
// encoded Opus frames to `tx` via `frames_sent`.
|
||||||
|
let voice_out_tx =
|
||||||
|
crate::opus_voice::start_out_packet_worker(tx, frames_sent, "cpal-bench")
|
||||||
|
.expect("start_out_packet_worker");
|
||||||
let state = CaptureState::new(
|
let state = CaptureState::new(
|
||||||
encoder,
|
encoder,
|
||||||
in_sample_rate,
|
in_sample_rate,
|
||||||
in_channels,
|
in_channels,
|
||||||
1.0,
|
1.0,
|
||||||
tx,
|
voice_out_tx,
|
||||||
transmit_active.clone(),
|
transmit_active.clone(),
|
||||||
frames_sent,
|
None,
|
||||||
|
Arc::new(std::sync::Mutex::new(
|
||||||
|
crate::AudioProcessingConfig::default(),
|
||||||
|
)),
|
||||||
|
Arc::new(std::sync::Mutex::new(None)),
|
||||||
Arc::new(crate::SharedAudioProcessingStats::default()),
|
Arc::new(crate::SharedAudioProcessingStats::default()),
|
||||||
);
|
);
|
||||||
Self {
|
Self {
|
||||||
|
|||||||
@@ -1,538 +0,0 @@
|
|||||||
//! Optional raw iOS RemoteIO path for the WebRTC APM experimental mode.
|
|
||||||
//!
|
|
||||||
//! Provides an alternative to `ios_voice_unit.rs` for the
|
|
||||||
//! `SonoraExperimental` processing mode. Instead of
|
|
||||||
//! `kAudioUnitSubType_VoiceProcessingIO` (which owns AEC/NS/AGC), it
|
|
||||||
//! opens `kAudioUnitSubType_RemoteIO` with voice processing explicitly
|
|
||||||
//! disabled so WebRTC APM can own the full signal path.
|
|
||||||
//!
|
|
||||||
//! ## Hard invariants enforced here
|
|
||||||
//!
|
|
||||||
//! * INV_009: Rust AEC only active when platform AEC is disabled.
|
|
||||||
//! * INV_010: VoiceProcessingIO and WebRTC APM AEC are mutually exclusive.
|
|
||||||
//! * INV_011: Software AEC backend receives both capture and render-reference.
|
|
||||||
//! * INV_012: Render reference is copied from decoded/mixed remote PCM
|
|
||||||
//! before playout.
|
|
||||||
//!
|
|
||||||
//! ## Fallback
|
|
||||||
//!
|
|
||||||
//! If RemoteIO construction fails, the caller falls back to `IosVoiceUnit`
|
|
||||||
//! (VPIO) and logs the error.
|
|
||||||
//!
|
|
||||||
//! ## Status
|
|
||||||
//!
|
|
||||||
//! Experimental / disabled by default. Only activated when the user
|
|
||||||
//! explicitly selects `SonoraExperimental` mode via the bridge API.
|
|
||||||
//!
|
|
||||||
//! ## Platform
|
|
||||||
//!
|
|
||||||
//! `kAudioUnitSubType_RemoteIO` is only available in the iOS SDK.
|
|
||||||
//! This module is gated to `target_os = "ios"`.
|
|
||||||
|
|
||||||
#[cfg(target_os = "ios")]
|
|
||||||
pub use inner::IosRawUnit;
|
|
||||||
|
|
||||||
#[cfg(target_os = "ios")]
|
|
||||||
mod inner {
|
|
||||||
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
|
||||||
use std::sync::{Arc, Mutex};
|
|
||||||
|
|
||||||
use audiopus::coder::Encoder as OpusEncoder;
|
|
||||||
use coreaudio::audio_unit::audio_format::LinearPcmFlags;
|
|
||||||
use coreaudio::audio_unit::render_callback::{self, data};
|
|
||||||
use coreaudio::audio_unit::IOType;
|
|
||||||
use coreaudio::audio_unit::{AudioUnit, Element, SampleFormat, Scope, StreamFormat};
|
|
||||||
use tracing::{info, warn};
|
|
||||||
|
|
||||||
use crate::mobile_voice_backend::VoiceAudioParams;
|
|
||||||
use crate::processor::AudioProcessor;
|
|
||||||
use crate::AudioError;
|
|
||||||
|
|
||||||
const SAMPLE_RATE_HZ: f64 = 48_000.0;
|
|
||||||
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
// Render-reference ring buffer //
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
|
|
||||||
/// 4-slot ring buffer shared between the render callback (writer) and
|
|
||||||
/// the capture callback (reader for Sonora AEC3). Capacity: 4 × 10 ms
|
|
||||||
/// = 40 ms of headroom.
|
|
||||||
///
|
|
||||||
/// If the capture callback runs before the render callback has written
|
|
||||||
/// a frame it reads zeros (silence reference), which is safe — Sonora
|
|
||||||
/// AEC3 simply skips cancellation for that frame.
|
|
||||||
type RenderReferenceBuffer = crate::render_reference::RenderReferenceBuffer<480, 4>;
|
|
||||||
type RenderReferenceFrameAccumulator =
|
|
||||||
crate::render_reference::RenderReferenceFrameAccumulator<480>;
|
|
||||||
const RAW_RENDER_SCRATCH_FRAMES: usize = 1024;
|
|
||||||
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
// Capture pipeline state //
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
|
|
||||||
struct RawCaptureState {
|
|
||||||
encoder: OpusEncoder,
|
|
||||||
pcm_accum: Vec<i16>,
|
|
||||||
opus_out: [u8; crate::opus_voice::MAX_OPUS_FRAME],
|
|
||||||
voice_out_tx: crate::opus_voice::EncodedVoiceFrameSender,
|
|
||||||
transmit_active: Arc<AtomicBool>,
|
|
||||||
output_muted: Arc<AtomicBool>,
|
|
||||||
mic_gain: f32,
|
|
||||||
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
|
|
||||||
vad_detector: crate::vad::WebRtcFallbackVad,
|
|
||||||
silero_coreml_worker: Option<crate::vad::apple_coreml::AppleCoreMlVadWorker>,
|
|
||||||
current_vad_backend: crate::VadBackend,
|
|
||||||
capture_frame_seq: u64,
|
|
||||||
vad_state: crate::voice_activity::VoiceActivityStateMachine,
|
|
||||||
/// Processing config — retained for route-change reloads.
|
|
||||||
audio_processing_config: Arc<Mutex<crate::AudioProcessingConfig>>,
|
|
||||||
webrtc_apm_processor: crate::processor::WebRtcApmProcessor,
|
|
||||||
audio_processing_stats: Arc<crate::SharedAudioProcessingStats>,
|
|
||||||
render_reference: Arc<RenderReferenceBuffer>,
|
|
||||||
pending_10ms: [i16; crate::frame::FRAME_10MS_SAMPLES],
|
|
||||||
pending_10ms_len: usize,
|
|
||||||
fallback_warned_backend: Option<crate::VadBackend>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl RawCaptureState {
|
|
||||||
fn new(
|
|
||||||
params: &VoiceAudioParams,
|
|
||||||
render_reference: Arc<RenderReferenceBuffer>,
|
|
||||||
) -> Result<Self, AudioError> {
|
|
||||||
let encoder = crate::opus_voice::new_voip_encoder("ios raw")?;
|
|
||||||
let webrtc_apm_config = params
|
|
||||||
.audio_processing_config
|
|
||||||
.lock()
|
|
||||||
.map(|cfg| crate::processor::webrtc_apm::WebRtcApmConfig::from_audio_config(&cfg))
|
|
||||||
.unwrap_or_default();
|
|
||||||
Ok(Self {
|
|
||||||
encoder,
|
|
||||||
pcm_accum: Vec::with_capacity(crate::frame::FRAME_20MS_SAMPLES * 2),
|
|
||||||
opus_out: [0u8; crate::opus_voice::MAX_OPUS_FRAME],
|
|
||||||
voice_out_tx: crate::opus_voice::start_out_packet_worker(
|
|
||||||
params.voice_out_tx.clone(),
|
|
||||||
params.frames_sent.clone(),
|
|
||||||
"ios-raw",
|
|
||||||
)?,
|
|
||||||
transmit_active: params.transmit_active.clone(),
|
|
||||||
output_muted: params.output_muted.clone(),
|
|
||||||
mic_gain: params.mic_gain,
|
|
||||||
voice_activity_selector: params.voice_activity_selector.clone(),
|
|
||||||
vad_detector: crate::vad::WebRtcFallbackVad::default(),
|
|
||||||
silero_coreml_worker: None,
|
|
||||||
current_vad_backend: crate::VadBackend::WebrtcVad,
|
|
||||||
capture_frame_seq: 0,
|
|
||||||
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
|
|
||||||
audio_processing_config: params.audio_processing_config.clone(),
|
|
||||||
webrtc_apm_processor: crate::processor::WebRtcApmProcessor::with_config(
|
|
||||||
webrtc_apm_config,
|
|
||||||
)?,
|
|
||||||
audio_processing_stats: params.audio_processing_stats.clone(),
|
|
||||||
render_reference,
|
|
||||||
pending_10ms: [0_i16; crate::frame::FRAME_10MS_SAMPLES],
|
|
||||||
pending_10ms_len: 0,
|
|
||||||
fallback_warned_backend: None,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fn mark_vad_fallback_active(&mut self, failed_backend: crate::VadBackend) {
|
|
||||||
if self.fallback_warned_backend != Some(failed_backend) {
|
|
||||||
self.fallback_warned_backend = Some(failed_backend);
|
|
||||||
if self.capture_frame_seq < 128 {
|
|
||||||
tracing::info!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
backend = failed_backend.as_str(),
|
|
||||||
seq = self.capture_frame_seq,
|
|
||||||
"VAD backend warming up; using WebRTC fallback"
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
tracing::warn!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
backend = failed_backend.as_str(),
|
|
||||||
"VAD backend unavailable; using WebRTC fallback for runtime detection"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn ingest_i16(&mut self, samples: &[i16]) {
|
|
||||||
// Accumulate into 10 ms frames for VAD / Sonora processing.
|
|
||||||
let mut offset = 0;
|
|
||||||
while offset < samples.len() {
|
|
||||||
let remaining = crate::frame::FRAME_10MS_SAMPLES - self.pending_10ms_len;
|
|
||||||
let take = remaining.min(samples.len() - offset);
|
|
||||||
self.pending_10ms[self.pending_10ms_len..self.pending_10ms_len + take]
|
|
||||||
.copy_from_slice(&samples[offset..offset + take]);
|
|
||||||
self.pending_10ms_len += take;
|
|
||||||
offset += take;
|
|
||||||
|
|
||||||
if self.pending_10ms_len == crate::frame::FRAME_10MS_SAMPLES {
|
|
||||||
let frame = self.pending_10ms;
|
|
||||||
self.process_10ms_capture_frame(&frame);
|
|
||||||
self.encode_complete_20ms_frames();
|
|
||||||
self.pending_10ms_len = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !self.transmit_active.load(Ordering::Relaxed) {
|
|
||||||
self.pcm_accum.clear();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
self.encode_complete_20ms_frames();
|
|
||||||
}
|
|
||||||
|
|
||||||
fn encode_complete_20ms_frames(&mut self) {
|
|
||||||
while self.pcm_accum.len() >= crate::frame::FRAME_20MS_SAMPLES {
|
|
||||||
let mut frame = [0i16; crate::frame::FRAME_20MS_SAMPLES];
|
|
||||||
frame.copy_from_slice(&self.pcm_accum[..crate::frame::FRAME_20MS_SAMPLES]);
|
|
||||||
self.pcm_accum.drain(..crate::frame::FRAME_20MS_SAMPLES);
|
|
||||||
|
|
||||||
match self.encoder.encode(&frame, &mut self.opus_out[..]) {
|
|
||||||
Ok(len) => {
|
|
||||||
crate::opus_voice::send_voip_frame(
|
|
||||||
&self.voice_out_tx,
|
|
||||||
&self.opus_out,
|
|
||||||
len,
|
|
||||||
|| {
|
|
||||||
warn!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
"ios raw: voice_out queue full; dropping frame"
|
|
||||||
);
|
|
||||||
},
|
|
||||||
|| {},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
tracing::error!(target: "chanora_audio",
|
|
||||||
error = %e, "ios raw opus encode failed");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn process_10ms_capture_frame(
|
|
||||||
&mut self,
|
|
||||||
samples: &[i16; crate::frame::FRAME_10MS_SAMPLES],
|
|
||||||
) {
|
|
||||||
let mut frame = [0.0_f32; crate::frame::FRAME_10MS_SAMPLES];
|
|
||||||
for (dst, src) in frame.iter_mut().zip(samples.iter().copied()) {
|
|
||||||
*dst = crate::frame::i16_to_f32(src);
|
|
||||||
}
|
|
||||||
let input_dbfs = crate::frame::dbfs(&frame);
|
|
||||||
|
|
||||||
// Debug WAV mic taps are intentionally unavailable on iOS raw
|
|
||||||
// realtime callbacks until WavDebugRecorder supports a
|
|
||||||
// preallocated handoff path; the current recorder push path
|
|
||||||
// allocates per frame.
|
|
||||||
|
|
||||||
// Feed render reference to WebRTC APM before capture so AEC can adapt.
|
|
||||||
let render_ref = self.render_reference.read_latest();
|
|
||||||
self.webrtc_apm_processor.process_render(&render_ref);
|
|
||||||
self.webrtc_apm_processor.process_capture(&mut frame);
|
|
||||||
|
|
||||||
// Processed-mic debug WAV capture is disabled for the same
|
|
||||||
// realtime allocation reason as the raw-mic tap above.
|
|
||||||
|
|
||||||
let voice_activity_mode = self
|
|
||||||
.voice_activity_selector
|
|
||||||
.as_ref()
|
|
||||||
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
|
|
||||||
.unwrap_or(false);
|
|
||||||
if !voice_activity_mode {
|
|
||||||
self.silero_coreml_worker = None;
|
|
||||||
self.current_vad_backend = crate::VadBackend::Disabled;
|
|
||||||
self.fallback_warned_backend = None;
|
|
||||||
self.audio_processing_stats.set_vad_fallback_active(false);
|
|
||||||
}
|
|
||||||
|
|
||||||
let (vad_backend, vad_hangover) = self
|
|
||||||
.audio_processing_config
|
|
||||||
.try_lock()
|
|
||||||
.map(|cfg| (cfg.vad_backend, cfg.vad_hangover_ms))
|
|
||||||
.unwrap_or((
|
|
||||||
crate::VadBackend::WebrtcVad,
|
|
||||||
crate::voice_activity::VAD_HANGOVER_MS,
|
|
||||||
));
|
|
||||||
if voice_activity_mode {
|
|
||||||
self.vad_state.configure(
|
|
||||||
crate::voice_activity::VAD_OPEN_AFTER_MS,
|
|
||||||
vad_hangover,
|
|
||||||
crate::voice_activity::VAD_MIN_TX_MS,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if voice_activity_mode && vad_backend != self.current_vad_backend {
|
|
||||||
self.current_vad_backend = vad_backend;
|
|
||||||
self.fallback_warned_backend = None;
|
|
||||||
if vad_backend == crate::VadBackend::SileroOnnx {
|
|
||||||
self.silero_coreml_worker = None;
|
|
||||||
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
|
||||||
self.audio_processing_stats.set_vad_fallback_active(true);
|
|
||||||
} else {
|
|
||||||
self.silero_coreml_worker = None;
|
|
||||||
self.audio_processing_stats.set_vad_fallback_active(false);
|
|
||||||
}
|
|
||||||
self.vad_state.reset();
|
|
||||||
}
|
|
||||||
|
|
||||||
let (vad_probability, active) = if voice_activity_mode {
|
|
||||||
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
|
|
||||||
let capture_seq = self.capture_frame_seq;
|
|
||||||
let mut used_fallback_vad = false;
|
|
||||||
let vad = if vad_backend == crate::VadBackend::Disabled {
|
|
||||||
crate::vad::VadOutput {
|
|
||||||
probability: 1.0,
|
|
||||||
speech: true,
|
|
||||||
}
|
|
||||||
} else if vad_backend == crate::VadBackend::SileroOnnx {
|
|
||||||
match crate::vad::callback_vad_worker_policy(
|
|
||||||
voice_activity_mode,
|
|
||||||
vad_backend,
|
|
||||||
self.silero_coreml_worker.is_some(),
|
|
||||||
) {
|
|
||||||
crate::vad::VadWorkerPolicy::UseWorker => {
|
|
||||||
let worker = self
|
|
||||||
.silero_coreml_worker
|
|
||||||
.as_ref()
|
|
||||||
.expect("policy checked worker");
|
|
||||||
let enqueued = worker.try_send(capture_seq, &frame);
|
|
||||||
if !worker.is_stale(capture_seq) {
|
|
||||||
let p = worker.latest_probability();
|
|
||||||
crate::vad::VadOutput {
|
|
||||||
probability: p,
|
|
||||||
speech: p >= 0.5,
|
|
||||||
}
|
|
||||||
} else if enqueued {
|
|
||||||
crate::vad::VadOutput {
|
|
||||||
probability: 0.0,
|
|
||||||
speech: false,
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
used_fallback_vad = true;
|
|
||||||
self.mark_vad_fallback_active(vad_backend);
|
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(
|
|
||||||
&mut self.vad_detector,
|
|
||||||
&frame,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
crate::vad::VadWorkerPolicy::UseFallback => {
|
|
||||||
used_fallback_vad = true;
|
|
||||||
self.mark_vad_fallback_active(vad_backend);
|
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(
|
|
||||||
&mut self.vad_detector,
|
|
||||||
&frame,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
crate::vad::VadWorkerPolicy::NotModelBacked => crate::vad::VadOutput {
|
|
||||||
probability: 1.0,
|
|
||||||
speech: true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
|
|
||||||
};
|
|
||||||
self.audio_processing_stats
|
|
||||||
.set_vad_fallback_active(used_fallback_vad);
|
|
||||||
(vad.probability, self.vad_state.update(vad.speech))
|
|
||||||
} else {
|
|
||||||
(0.0, false)
|
|
||||||
};
|
|
||||||
if let Some(sel) = &self.voice_activity_selector {
|
|
||||||
sel.set_voice_activity_open(voice_activity_mode && active);
|
|
||||||
}
|
|
||||||
self.audio_processing_stats.update_capture(
|
|
||||||
input_dbfs,
|
|
||||||
crate::frame::dbfs(&frame),
|
|
||||||
vad_probability,
|
|
||||||
voice_activity_mode && active,
|
|
||||||
self.transmit_active.load(Ordering::Relaxed),
|
|
||||||
);
|
|
||||||
|
|
||||||
if !self.transmit_active.load(Ordering::Relaxed) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if crate::capture_accumulator::append_processed_i16_bounded(
|
|
||||||
&mut self.pcm_accum,
|
|
||||||
&frame,
|
|
||||||
self.mic_gain,
|
|
||||||
) {
|
|
||||||
self.audio_processing_stats.increment_callback_xrun();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
// IosRawUnit //
|
|
||||||
// ------------------------------------------------------------------ //
|
|
||||||
|
|
||||||
/// Raw iOS RemoteIO audio unit for the Sonora experimental path.
|
|
||||||
pub struct IosRawUnit {
|
|
||||||
unit: AudioUnit,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl IosRawUnit {
|
|
||||||
/// Open a RemoteIO AudioUnit, install render + input callbacks, start.
|
|
||||||
pub(crate) fn start(params: VoiceAudioParams) -> Result<Self, AudioError> {
|
|
||||||
// INV_010: reject if config requests VPIO (that's IosVoiceUnit's job).
|
|
||||||
{
|
|
||||||
let cfg = params.audio_processing_config.lock().unwrap();
|
|
||||||
if cfg.ios_mode == crate::IosVoiceProcessingMode::PlatformVoiceProcessing {
|
|
||||||
return Err(AudioError::InvalidAudioProcessingConfig(
|
|
||||||
"IosRawUnit requires raw WebRTC APM mode".to_string(),
|
|
||||||
));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut unit = AudioUnit::new_uninitialized(IOType::RemoteIO)
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio new: {e}")))?;
|
|
||||||
|
|
||||||
// Enable input on bus 1.
|
|
||||||
const ENABLE_IO: u32 = 2003;
|
|
||||||
let enable: u32 = 1;
|
|
||||||
unit.set_property(ENABLE_IO, Scope::Input, Element::Input, Some(&enable))
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio enable input: {e}")))?;
|
|
||||||
|
|
||||||
// 48 kHz Int16 mono on both buses.
|
|
||||||
let fmt = StreamFormat {
|
|
||||||
sample_rate: SAMPLE_RATE_HZ,
|
|
||||||
sample_format: SampleFormat::I16,
|
|
||||||
flags: LinearPcmFlags::IS_SIGNED_INTEGER | LinearPcmFlags::IS_PACKED,
|
|
||||||
channels: 1,
|
|
||||||
};
|
|
||||||
unit.set_stream_format(fmt, Scope::Input, Element::Output)
|
|
||||||
.map_err(|e| AudioError::StreamConfig(format!("remoteio fmt output: {e}")))?;
|
|
||||||
unit.set_stream_format(fmt, Scope::Output, Element::Input)
|
|
||||||
.map_err(|e| AudioError::StreamConfig(format!("remoteio fmt input: {e}")))?;
|
|
||||||
|
|
||||||
// Shared render-reference buffer (INV_011 / INV_012).
|
|
||||||
let render_ref_buf = RenderReferenceBuffer::new();
|
|
||||||
let render_ref_for_capture = render_ref_buf.clone();
|
|
||||||
|
|
||||||
let mut capture_state = RawCaptureState::new(¶ms, render_ref_for_capture)?;
|
|
||||||
|
|
||||||
unit.set_input_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
|
|
||||||
capture_state.ingest_i16(args.data.buffer);
|
|
||||||
Ok(())
|
|
||||||
})
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio input cb: {e}")))?;
|
|
||||||
|
|
||||||
let mut scratch = [0.0_f32; RAW_RENDER_SCRATCH_FRAMES * 2];
|
|
||||||
let mut mono = [0.0_f32; RAW_RENDER_SCRATCH_FRAMES];
|
|
||||||
let mut render_ref_accum = RenderReferenceFrameAccumulator::new();
|
|
||||||
let handler = params.handler.clone();
|
|
||||||
let output_gain = params.output_gain.clone();
|
|
||||||
let output_muted = params.output_muted.clone();
|
|
||||||
let stats_render = params.audio_processing_stats.clone();
|
|
||||||
|
|
||||||
unit.set_render_callback(move |args: render_callback::Args<data::Interleaved<i16>>| {
|
|
||||||
let out = args.data.buffer;
|
|
||||||
let n = out.len();
|
|
||||||
let process_n = n.min(RAW_RENDER_SCRATCH_FRAMES);
|
|
||||||
let stereo_n = process_n * 2;
|
|
||||||
if n > RAW_RENDER_SCRATCH_FRAMES {
|
|
||||||
stats_render.increment_callback_xrun();
|
|
||||||
}
|
|
||||||
scratch[..stereo_n].fill(0.0);
|
|
||||||
|
|
||||||
match handler.try_lock() {
|
|
||||||
Ok(mut h) => {
|
|
||||||
let _ = h.fill_buffer(&mut scratch[..stereo_n]);
|
|
||||||
}
|
|
||||||
Err(std::sync::TryLockError::WouldBlock) => {
|
|
||||||
stats_render.increment_callback_xrun();
|
|
||||||
}
|
|
||||||
Err(std::sync::TryLockError::Poisoned(e)) => {
|
|
||||||
warn!(target: "chanora_audio",
|
|
||||||
"AudioHandler poisoned (raw render): {e}");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// INV_012: copy render reference BEFORE playout.
|
|
||||||
crate::voice_render::downmix_stereo_f32_to_mono_f32(
|
|
||||||
&scratch[..stereo_n],
|
|
||||||
&mut mono[..process_n],
|
|
||||||
);
|
|
||||||
render_ref_accum.push_mono_samples(&mono[..process_n], |frame| {
|
|
||||||
render_ref_buf.write(frame);
|
|
||||||
});
|
|
||||||
|
|
||||||
let gain = f32::from_bits(output_gain.load(Ordering::Relaxed));
|
|
||||||
let muted = output_muted.load(Ordering::Relaxed);
|
|
||||||
let mix_stats = crate::voice_render::downmix_stereo_f32_to_interleaved_i16(
|
|
||||||
&scratch[..stereo_n],
|
|
||||||
&mut out[..process_n],
|
|
||||||
1,
|
|
||||||
gain,
|
|
||||||
muted,
|
|
||||||
);
|
|
||||||
if process_n < n {
|
|
||||||
out[process_n..].fill(0);
|
|
||||||
}
|
|
||||||
if mix_stats.clipped_samples > 0 {
|
|
||||||
stats_render.add_clipped_samples(mix_stats.clipped_samples);
|
|
||||||
}
|
|
||||||
stats_render.update_render(crate::frame::dbfs(&scratch[..stereo_n]), n as u32);
|
|
||||||
Ok(())
|
|
||||||
})
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio render cb: {e}")))?;
|
|
||||||
|
|
||||||
unit.initialize()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio init: {e}")))?;
|
|
||||||
unit.start()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio start: {e}")))?;
|
|
||||||
|
|
||||||
info!(
|
|
||||||
target: "chanora_audio",
|
|
||||||
sample_rate_hz = SAMPLE_RATE_HZ,
|
|
||||||
"ios RemoteIO (Sonora experimental) started"
|
|
||||||
);
|
|
||||||
Ok(Self { unit })
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Restart the unit after a route change (stop → uninit → init → start).
|
|
||||||
pub fn restart(&mut self) -> Result<(), AudioError> {
|
|
||||||
self.unit
|
|
||||||
.stop()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio restart stop: {e}")))?;
|
|
||||||
self.unit
|
|
||||||
.uninitialize()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio restart uninit: {e}")))?;
|
|
||||||
self.unit
|
|
||||||
.initialize()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio restart init: {e}")))?;
|
|
||||||
self.unit
|
|
||||||
.start()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio restart start: {e}")))?;
|
|
||||||
info!(target: "chanora_audio", "ios RemoteIO restarted");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Pause the unit during an AVAudioSession interruption.
|
|
||||||
pub fn pause(&mut self) -> Result<(), AudioError> {
|
|
||||||
self.unit
|
|
||||||
.stop()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio pause: {e}")))
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Resume the unit after an interruption ends.
|
|
||||||
pub fn resume(&mut self) -> Result<(), AudioError> {
|
|
||||||
self.unit
|
|
||||||
.start()
|
|
||||||
.map_err(|e| AudioError::Backend(format!("remoteio resume: {e}")))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Drop for IosRawUnit {
|
|
||||||
fn drop(&mut self) {
|
|
||||||
if let Err(e) = self.unit.stop() {
|
|
||||||
warn!(target: "chanora_audio", error = %e,
|
|
||||||
"ios RemoteIO stop on drop failed");
|
|
||||||
} else {
|
|
||||||
info!(target: "chanora_audio", "ios RemoteIO stopped");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -406,38 +406,20 @@ impl IosCaptureState {
|
|||||||
speech: true,
|
speech: true,
|
||||||
}
|
}
|
||||||
} else if vad_backend == crate::VadBackend::SileroOnnx {
|
} else if vad_backend == crate::VadBackend::SileroOnnx {
|
||||||
match crate::vad::callback_vad_worker_policy(
|
if let Some(worker) = self.silero_coreml_worker.as_ref() {
|
||||||
voice_activity_mode,
|
let enqueued = worker.try_send(capture_seq, &frame);
|
||||||
vad_backend,
|
if !worker.is_stale(capture_seq) {
|
||||||
self.silero_coreml_worker.is_some(),
|
let p = worker.latest_probability();
|
||||||
) {
|
crate::vad::VadOutput {
|
||||||
crate::vad::VadWorkerPolicy::UseWorker => {
|
probability: p,
|
||||||
let worker = self
|
speech: p >= 0.5,
|
||||||
.silero_coreml_worker
|
|
||||||
.as_ref()
|
|
||||||
.expect("policy checked worker");
|
|
||||||
let enqueued = worker.try_send(capture_seq, &frame);
|
|
||||||
if !worker.is_stale(capture_seq) {
|
|
||||||
let p = worker.latest_probability();
|
|
||||||
crate::vad::VadOutput {
|
|
||||||
probability: p,
|
|
||||||
speech: p >= 0.5,
|
|
||||||
}
|
|
||||||
} else if enqueued {
|
|
||||||
crate::vad::VadOutput {
|
|
||||||
probability: 0.0,
|
|
||||||
speech: false,
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
used_fallback_vad = true;
|
|
||||||
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(
|
|
||||||
&mut self.vad_detector,
|
|
||||||
&frame,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
}
|
} else if enqueued {
|
||||||
crate::vad::VadWorkerPolicy::UseFallback => {
|
crate::vad::VadOutput {
|
||||||
|
probability: 0.0,
|
||||||
|
speech: false,
|
||||||
|
}
|
||||||
|
} else {
|
||||||
used_fallback_vad = true;
|
used_fallback_vad = true;
|
||||||
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(
|
crate::vad::VoiceActivityDetector::process_10ms(
|
||||||
@@ -445,10 +427,13 @@ impl IosCaptureState {
|
|||||||
&frame,
|
&frame,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
crate::vad::VadWorkerPolicy::NotModelBacked => crate::vad::VadOutput {
|
} else {
|
||||||
probability: 1.0,
|
used_fallback_vad = true;
|
||||||
speech: true,
|
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
|
||||||
},
|
crate::vad::VoiceActivityDetector::process_10ms(
|
||||||
|
&mut self.vad_detector,
|
||||||
|
&frame,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
|
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
|
||||||
|
|||||||
@@ -65,10 +65,7 @@ pub(crate) mod voice_render;
|
|||||||
mod sdl_output;
|
mod sdl_output;
|
||||||
|
|
||||||
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
||||||
mod ios_voice_unit;
|
mod ios_voice_unit;
|
||||||
|
|
||||||
#[cfg(target_os = "ios")]
|
|
||||||
pub mod ios_raw_unit;
|
|
||||||
|
|
||||||
#[cfg(target_os = "android")]
|
#[cfg(target_os = "android")]
|
||||||
pub mod android_voice_unit;
|
pub mod android_voice_unit;
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ use std::thread;
|
|||||||
|
|
||||||
use tracing::{info, warn};
|
use tracing::{info, warn};
|
||||||
use windows::core::{w, PCWSTR};
|
use windows::core::{w, PCWSTR};
|
||||||
use windows::Win32::Foundation::{HMODULE, HWND, LPARAM, LRESULT, WPARAM};
|
use windows::Win32::Foundation::{HINSTANCE, HMODULE, HWND, LPARAM, LRESULT, WPARAM};
|
||||||
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
|
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
|
||||||
use windows::Win32::UI::Input::{
|
use windows::Win32::UI::Input::{
|
||||||
GetRawInputData, RegisterRawInputDevices, HRAWINPUT, RAWINPUT, RAWINPUTDEVICE, RAWINPUTHEADER,
|
GetRawInputData, RegisterRawInputDevices, HRAWINPUT, RAWINPUT, RAWINPUTDEVICE, RAWINPUTHEADER,
|
||||||
@@ -35,7 +35,7 @@ use windows::Win32::UI::Input::{
|
|||||||
use windows::Win32::UI::WindowsAndMessaging::{
|
use windows::Win32::UI::WindowsAndMessaging::{
|
||||||
CallNextHookEx, CreateWindowExW, DefWindowProcW, DispatchMessageW, GetMessageW,
|
CallNextHookEx, CreateWindowExW, DefWindowProcW, DispatchMessageW, GetMessageW,
|
||||||
PostThreadMessageW, RegisterClassExW, SetWindowsHookExW, TranslateMessage, UnhookWindowsHookEx,
|
PostThreadMessageW, RegisterClassExW, SetWindowsHookExW, TranslateMessage, UnhookWindowsHookEx,
|
||||||
HC_ACTION, HHOOK, HOOKPROC, KBDLLHOOKSTRUCT, MSG, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
|
HC_ACTION, HOOKPROC, KBDLLHOOKSTRUCT, MSG, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
|
||||||
WINDOW_EX_STYLE, WINDOW_STYLE, WM_INPUT, WM_KEYDOWN, WM_KEYUP, WM_QUIT, WM_SYSKEYDOWN,
|
WINDOW_EX_STYLE, WINDOW_STYLE, WM_INPUT, WM_KEYDOWN, WM_KEYUP, WM_QUIT, WM_SYSKEYDOWN,
|
||||||
WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP, WNDCLASSEXW, XBUTTON1, XBUTTON2,
|
WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP, WNDCLASSEXW, XBUTTON1, XBUTTON2,
|
||||||
};
|
};
|
||||||
@@ -423,7 +423,7 @@ unsafe fn run_raw_input_loop(
|
|||||||
// class.
|
// class.
|
||||||
let _atom = RegisterClassExW(&wc);
|
let _atom = RegisterClassExW(&wc);
|
||||||
|
|
||||||
let hwnd = unsafe {
|
let hwnd = match unsafe {
|
||||||
CreateWindowExW(
|
CreateWindowExW(
|
||||||
WINDOW_EX_STYLE(0),
|
WINDOW_EX_STYLE(0),
|
||||||
class_name,
|
class_name,
|
||||||
@@ -433,13 +433,23 @@ unsafe fn run_raw_input_loop(
|
|||||||
0,
|
0,
|
||||||
0,
|
0,
|
||||||
0,
|
0,
|
||||||
HWND(HWND_MESSAGE_PTR),
|
Some(HWND(HWND_MESSAGE_PTR as *mut core::ffi::c_void)),
|
||||||
None,
|
None,
|
||||||
h_instance,
|
Some(HINSTANCE(h_instance.0)),
|
||||||
None,
|
None,
|
||||||
)
|
)
|
||||||
|
} {
|
||||||
|
Ok(h) => h,
|
||||||
|
Err(_) => {
|
||||||
|
warn!(
|
||||||
|
target: "chanora_audio",
|
||||||
|
"windows ptt: CreateWindowExW(HWND_MESSAGE) returned null"
|
||||||
|
);
|
||||||
|
report!(false);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
};
|
};
|
||||||
if hwnd.0 == 0 {
|
if hwnd.0.is_null() {
|
||||||
warn!(
|
warn!(
|
||||||
target: "chanora_audio",
|
target: "chanora_audio",
|
||||||
"windows ptt: CreateWindowExW(HWND_MESSAGE) returned null"
|
"windows ptt: CreateWindowExW(HWND_MESSAGE) returned null"
|
||||||
@@ -517,13 +527,13 @@ unsafe fn run_raw_input_loop(
|
|||||||
usUsagePage: 0x01,
|
usUsagePage: 0x01,
|
||||||
usUsage: 0x06,
|
usUsage: 0x06,
|
||||||
dwFlags: RIDEV_REMOVE,
|
dwFlags: RIDEV_REMOVE,
|
||||||
hwndTarget: HWND(0),
|
hwndTarget: HWND(std::ptr::null_mut()),
|
||||||
},
|
},
|
||||||
RAWINPUTDEVICE {
|
RAWINPUTDEVICE {
|
||||||
usUsagePage: 0x01,
|
usUsagePage: 0x01,
|
||||||
usUsage: 0x02,
|
usUsage: 0x02,
|
||||||
dwFlags: RIDEV_REMOVE,
|
dwFlags: RIDEV_REMOVE,
|
||||||
hwndTarget: HWND(0),
|
hwndTarget: HWND(std::ptr::null_mut()),
|
||||||
},
|
},
|
||||||
];
|
];
|
||||||
let _ = RegisterRawInputDevices(&undo, std::mem::size_of::<RAWINPUTDEVICE>() as u32);
|
let _ = RegisterRawInputDevices(&undo, std::mem::size_of::<RAWINPUTDEVICE>() as u32);
|
||||||
@@ -544,7 +554,7 @@ unsafe extern "system" fn raw_input_wnd_proc(
|
|||||||
}
|
}
|
||||||
|
|
||||||
unsafe fn handle_wm_input(lparam: LPARAM) {
|
unsafe fn handle_wm_input(lparam: LPARAM) {
|
||||||
let h_raw = HRAWINPUT(lparam.0);
|
let h_raw = HRAWINPUT(lparam.0 as *mut core::ffi::c_void);
|
||||||
let mut size: u32 = 0;
|
let mut size: u32 = 0;
|
||||||
let header_sz = std::mem::size_of::<RAWINPUTHEADER>() as u32;
|
let header_sz = std::mem::size_of::<RAWINPUTHEADER>() as u32;
|
||||||
// First call: query buffer size.
|
// First call: query buffer size.
|
||||||
@@ -841,31 +851,33 @@ unsafe fn run_hook_loop(
|
|||||||
let kbd_proc: HOOKPROC = Some(kbd_hook_proc);
|
let kbd_proc: HOOKPROC = Some(kbd_hook_proc);
|
||||||
let mouse_proc: HOOKPROC = Some(mouse_hook_proc);
|
let mouse_proc: HOOKPROC = Some(mouse_hook_proc);
|
||||||
|
|
||||||
let kbd_hook = match SetWindowsHookExW(WH_KEYBOARD_LL, kbd_proc, h_instance, 0) {
|
let kbd_hook =
|
||||||
Ok(h) => h,
|
match SetWindowsHookExW(WH_KEYBOARD_LL, kbd_proc, Some(HINSTANCE(h_instance.0)), 0) {
|
||||||
Err(e) => {
|
Ok(h) => h,
|
||||||
warn!(
|
Err(e) => {
|
||||||
target: "chanora_audio",
|
warn!(
|
||||||
error = %e,
|
target: "chanora_audio",
|
||||||
"windows ptt: SetWindowsHookExW(WH_KEYBOARD_LL) failed"
|
error = %e,
|
||||||
);
|
"windows ptt: SetWindowsHookExW(WH_KEYBOARD_LL) failed"
|
||||||
report!(false);
|
);
|
||||||
return false;
|
report!(false);
|
||||||
}
|
return false;
|
||||||
};
|
}
|
||||||
let mouse_hook = match SetWindowsHookExW(WH_MOUSE_LL, mouse_proc, h_instance, 0) {
|
};
|
||||||
Ok(h) => h,
|
let mouse_hook =
|
||||||
Err(e) => {
|
match SetWindowsHookExW(WH_MOUSE_LL, mouse_proc, Some(HINSTANCE(h_instance.0)), 0) {
|
||||||
warn!(
|
Ok(h) => h,
|
||||||
target: "chanora_audio",
|
Err(e) => {
|
||||||
error = %e,
|
warn!(
|
||||||
"windows ptt: SetWindowsHookExW(WH_MOUSE_LL) failed"
|
target: "chanora_audio",
|
||||||
);
|
error = %e,
|
||||||
let _ = UnhookWindowsHookEx(kbd_hook);
|
"windows ptt: SetWindowsHookExW(WH_MOUSE_LL) failed"
|
||||||
report!(false);
|
);
|
||||||
return false;
|
let _ = UnhookWindowsHookEx(kbd_hook);
|
||||||
}
|
report!(false);
|
||||||
};
|
return false;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
info!(
|
info!(
|
||||||
target: "chanora_audio",
|
target: "chanora_audio",
|
||||||
@@ -908,7 +920,7 @@ unsafe extern "system" fn kbd_hook_proc(code: i32, wparam: WPARAM, lparam: LPARA
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
CallNextHookEx(HHOOK(0), code, wparam, lparam)
|
CallNextHookEx(None, code, wparam, lparam)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Pure-logic dispatcher for a low-level keyboard hook event (L0
|
/// Pure-logic dispatcher for a low-level keyboard hook event (L0
|
||||||
@@ -943,7 +955,7 @@ unsafe extern "system" fn mouse_hook_proc(code: i32, wparam: WPARAM, lparam: LPA
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
CallNextHookEx(HHOOK(0), code, wparam, lparam)
|
CallNextHookEx(None, code, wparam, lparam)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Pure-logic dispatcher for a low-level mouse hook event (L0
|
/// Pure-logic dispatcher for a low-level mouse hook event (L0
|
||||||
|
|||||||
@@ -8,17 +8,17 @@
|
|||||||
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
#[cfg(any(target_os = "ios", target_os = "macos"))]
|
||||||
pub mod apple_coreml;
|
pub mod apple_coreml;
|
||||||
pub mod resampler;
|
pub mod resampler;
|
||||||
#[cfg(not(target_os = "ios"))]
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
pub mod silero_onnx;
|
pub mod silero_onnx;
|
||||||
|
|
||||||
use std::sync::atomic::{AtomicU64, Ordering};
|
use std::sync::atomic::{AtomicU64, Ordering};
|
||||||
use std::sync::{OnceLock, RwLock};
|
use std::sync::{OnceLock, RwLock};
|
||||||
|
|
||||||
use crate::frame::{f32_to_i16, i16_to_f32};
|
use crate::frame::{f32_to_i16, i16_to_f32};
|
||||||
use crate::{AudioError, VadBackend};
|
use crate::AudioError;
|
||||||
use resampler::{Downsampler48to16, INPUT_FRAME_10MS};
|
use resampler::{Downsampler48to16, INPUT_FRAME_10MS};
|
||||||
|
|
||||||
#[cfg(not(target_os = "ios"))]
|
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
|
||||||
pub use silero_onnx::SileroOnnxVad;
|
pub use silero_onnx::SileroOnnxVad;
|
||||||
|
|
||||||
/// Voice activity detector output for one 10 ms frame.
|
/// Voice activity detector output for one 10 ms frame.
|
||||||
@@ -108,39 +108,12 @@ pub fn process_i16_10ms(detector: &mut dyn VoiceActivityDetector, samples: &[i16
|
|||||||
detector.process_10ms(&frame)
|
detector.process_10ms(&frame)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Callback-side policy for optional model-backed VAD workers.
|
|
||||||
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
|
|
||||||
pub(crate) enum VadWorkerPolicy {
|
|
||||||
/// Keep using the already-available model worker.
|
|
||||||
UseWorker,
|
|
||||||
/// No worker may be constructed on the callback thread; use WebRTC fallback.
|
|
||||||
UseFallback,
|
|
||||||
/// This backend does not need a model worker.
|
|
||||||
NotModelBacked,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Decide whether a realtime callback may use a model-backed VAD worker.
|
|
||||||
///
|
|
||||||
/// Model/worker construction is intentionally absent from this policy: if a
|
|
||||||
/// worker is not already present, callbacks must stay nonblocking and fall back.
|
|
||||||
pub(crate) fn callback_vad_worker_policy(
|
|
||||||
voice_activity_mode: bool,
|
|
||||||
backend: VadBackend,
|
|
||||||
worker_available: bool,
|
|
||||||
) -> VadWorkerPolicy {
|
|
||||||
if !voice_activity_mode || backend != VadBackend::SileroOnnx {
|
|
||||||
return VadWorkerPolicy::NotModelBacked;
|
|
||||||
}
|
|
||||||
if worker_available {
|
|
||||||
VadWorkerPolicy::UseWorker
|
|
||||||
} else {
|
|
||||||
VadWorkerPolicy::UseFallback
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
static SILERO_MODEL_PATH_OVERRIDE: OnceLock<RwLock<Option<String>>> = OnceLock::new();
|
static SILERO_MODEL_PATH_OVERRIDE: OnceLock<RwLock<Option<String>>> = OnceLock::new();
|
||||||
static SILERO_MODEL_EPOCH: AtomicU64 = AtomicU64::new(0);
|
static SILERO_MODEL_EPOCH: AtomicU64 = AtomicU64::new(0);
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(crate) static SILERO_MODEL_PATH_TEST_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
|
||||||
|
|
||||||
fn silero_model_path_override() -> &'static RwLock<Option<String>> {
|
fn silero_model_path_override() -> &'static RwLock<Option<String>> {
|
||||||
SILERO_MODEL_PATH_OVERRIDE.get_or_init(|| RwLock::new(None))
|
SILERO_MODEL_PATH_OVERRIDE.get_or_init(|| RwLock::new(None))
|
||||||
}
|
}
|
||||||
@@ -175,6 +148,19 @@ pub fn silero_model_epoch() -> u64 {
|
|||||||
SILERO_MODEL_EPOCH.load(Ordering::Relaxed)
|
SILERO_MODEL_EPOCH.load(Ordering::Relaxed)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(crate) fn clear_silero_model_path_for_test() {
|
||||||
|
set_silero_model_path_for_test(None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(crate) fn set_silero_model_path_for_test(path: Option<String>) {
|
||||||
|
if let Ok(mut guard) = silero_model_path_override().write() {
|
||||||
|
*guard = path;
|
||||||
|
SILERO_MODEL_EPOCH.fetch_add(1, Ordering::Relaxed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Return the expected path of the Silero VAD v6 ONNX model on
|
/// Return the expected path of the Silero VAD v6 ONNX model on
|
||||||
/// supported platforms.
|
/// supported platforms.
|
||||||
/// The model is shipped as a Flutter asset and copied to the app's
|
/// The model is shipped as a Flutter asset and copied to the app's
|
||||||
@@ -264,13 +250,20 @@ mod tests {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn set_silero_model_path_rejects_missing_file() {
|
fn set_silero_model_path_rejects_missing_file() {
|
||||||
|
let _guard = SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
|
||||||
|
clear_silero_model_path_for_test();
|
||||||
|
|
||||||
let result = set_silero_model_path("/definitely/not/a/silero_vad.onnx");
|
let result = set_silero_model_path("/definitely/not/a/silero_vad.onnx");
|
||||||
|
|
||||||
assert!(result.is_err());
|
assert!(result.is_err());
|
||||||
|
clear_silero_model_path_for_test();
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn set_silero_model_path_updates_override_and_epoch() {
|
fn set_silero_model_path_updates_override_and_epoch() {
|
||||||
|
let _guard = SILERO_MODEL_PATH_TEST_LOCK.lock().unwrap();
|
||||||
|
clear_silero_model_path_for_test();
|
||||||
|
|
||||||
let path =
|
let path =
|
||||||
std::env::temp_dir().join(format!("chanora_test_silero_{}.onnx", std::process::id()));
|
std::env::temp_dir().join(format!("chanora_test_silero_{}.onnx", std::process::id()));
|
||||||
std::fs::write(&path, b"test").unwrap();
|
std::fs::write(&path, b"test").unwrap();
|
||||||
@@ -280,34 +273,7 @@ mod tests {
|
|||||||
|
|
||||||
assert!(silero_model_epoch() > before);
|
assert!(silero_model_epoch() > before);
|
||||||
assert_eq!(silero_model_bundle_path(), path.to_string_lossy());
|
assert_eq!(silero_model_bundle_path(), path.to_string_lossy());
|
||||||
|
clear_silero_model_path_for_test();
|
||||||
let _ = std::fs::remove_file(path);
|
let _ = std::fs::remove_file(path);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn callback_policy_uses_existing_model_worker_only() {
|
|
||||||
assert_eq!(
|
|
||||||
callback_vad_worker_policy(true, VadBackend::SileroOnnx, true),
|
|
||||||
VadWorkerPolicy::UseWorker
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
callback_vad_worker_policy(true, VadBackend::SileroOnnx, false),
|
|
||||||
VadWorkerPolicy::UseFallback
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn callback_policy_keeps_disabled_and_webrtc_paths_worker_free() {
|
|
||||||
assert_eq!(
|
|
||||||
callback_vad_worker_policy(false, VadBackend::SileroOnnx, false),
|
|
||||||
VadWorkerPolicy::NotModelBacked
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
callback_vad_worker_policy(true, VadBackend::Disabled, false),
|
|
||||||
VadWorkerPolicy::NotModelBacked
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
callback_vad_worker_policy(true, VadBackend::WebrtcVad, false),
|
|
||||||
VadWorkerPolicy::NotModelBacked
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -361,6 +361,31 @@ impl SileroOnnxVadWorker {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(crate) fn stale_test_worker() -> Self {
|
||||||
|
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(64);
|
||||||
|
let alive = Arc::new(AtomicBool::new(true));
|
||||||
|
let alive_for_thread = alive.clone();
|
||||||
|
let handle = std::thread::Builder::new()
|
||||||
|
.name("chanora-silero-vad-stale-test".to_string())
|
||||||
|
.spawn(move || {
|
||||||
|
while alive_for_thread.load(Ordering::Relaxed) {
|
||||||
|
match rx.recv_timeout(std::time::Duration::from_millis(10)) {
|
||||||
|
Ok(_) | Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
|
||||||
|
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.ok();
|
||||||
|
Self {
|
||||||
|
tx: Some(tx),
|
||||||
|
latest_probability: Arc::new(AtomicU32::new(0.0_f32.to_bits())),
|
||||||
|
latest_processed_seq: Arc::new(AtomicU64::new(u64::MAX)),
|
||||||
|
alive,
|
||||||
|
handle,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Best-effort enqueue of a 10 ms frame for background inference.
|
/// Best-effort enqueue of a 10 ms frame for background inference.
|
||||||
pub fn try_send(&self, seq: u64, frame: &[f32; super::resampler::INPUT_FRAME_10MS]) -> bool {
|
pub fn try_send(&self, seq: u64, frame: &[f32; super::resampler::INPUT_FRAME_10MS]) -> bool {
|
||||||
let Some(tx) = &self.tx else {
|
let Some(tx) = &self.tx else {
|
||||||
@@ -393,8 +418,15 @@ impl SileroOnnxVadWorker {
|
|||||||
impl Drop for SileroOnnxVadWorker {
|
impl Drop for SileroOnnxVadWorker {
|
||||||
fn drop(&mut self) {
|
fn drop(&mut self) {
|
||||||
self.alive.store(false, Ordering::Relaxed);
|
self.alive.store(false, Ordering::Relaxed);
|
||||||
|
// Drop the sender first so the worker thread's rx.recv() returns
|
||||||
|
// Err and the loop exits promptly.
|
||||||
let _ = self.tx.take();
|
let _ = self.tx.take();
|
||||||
let _ = self.handle.take();
|
// Join the thread instead of detaching. The channel close
|
||||||
|
// unblocks rx.recv() so the join is bounded; it waits at most
|
||||||
|
// until the current in-flight inference completes.
|
||||||
|
if let Some(handle) = self.handle.take() {
|
||||||
|
let _ = handle.join();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -196,8 +196,8 @@ fn windows_exercise() {
|
|||||||
// start/stop lifecycle through the public trait surface so
|
// start/stop lifecycle through the public trait surface so
|
||||||
// any info!/warn! the factory or the backend's `start` path
|
// any info!/warn! the factory or the backend's `start` path
|
||||||
// emits is captured by the layer.
|
// emits is captured by the layer.
|
||||||
use chanora_audio::ptt_backends::{select_ptt_backend, PttBinding, PttInputClass};
|
use chanora_audio::ptt_backends::{PttBinding, PttInputClass};
|
||||||
use chanora_audio::AudioTransmitGate;
|
use chanora_audio::{select_ptt_backend, AudioTransmitGate};
|
||||||
|
|
||||||
let mut backend = select_ptt_backend();
|
let mut backend = select_ptt_backend();
|
||||||
let gate = AudioTransmitGate::new(false);
|
let gate = AudioTransmitGate::new(false);
|
||||||
|
|||||||
@@ -1062,10 +1062,8 @@ pub enum BridgeAudioRoute {
|
|||||||
/// Bridge iOS voice-processing mode.
|
/// Bridge iOS voice-processing mode.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum BridgeIosVoiceProcessingMode {
|
pub enum BridgeIosVoiceProcessingMode {
|
||||||
/// Shipping VPIO path.
|
/// Apple VoiceProcessingIO path.
|
||||||
PlatformVoiceProcessing,
|
PlatformVoiceProcessing,
|
||||||
/// Experimental Sonora path.
|
|
||||||
SonoraExperimental,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Bridge processing backend.
|
/// Bridge processing backend.
|
||||||
@@ -1223,7 +1221,6 @@ impl From<BridgeIosVoiceProcessingMode> for chanora_core::IosVoiceProcessingMode
|
|||||||
fn from(mode: BridgeIosVoiceProcessingMode) -> Self {
|
fn from(mode: BridgeIosVoiceProcessingMode) -> Self {
|
||||||
match mode {
|
match mode {
|
||||||
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => Self::PlatformVoiceProcessing,
|
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => Self::PlatformVoiceProcessing,
|
||||||
BridgeIosVoiceProcessingMode::SonoraExperimental => Self::SonoraExperimental,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1234,7 +1231,6 @@ impl From<chanora_core::IosVoiceProcessingMode> for BridgeIosVoiceProcessingMode
|
|||||||
chanora_core::IosVoiceProcessingMode::PlatformVoiceProcessing => {
|
chanora_core::IosVoiceProcessingMode::PlatformVoiceProcessing => {
|
||||||
Self::PlatformVoiceProcessing
|
Self::PlatformVoiceProcessing
|
||||||
}
|
}
|
||||||
chanora_core::IosVoiceProcessingMode::SonoraExperimental => Self::SonoraExperimental,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -2361,25 +2357,11 @@ pub async fn set_ios_voice_processing_mode(
|
|||||||
let config = BridgeAudioProcessingConfig {
|
let config = BridgeAudioProcessingConfig {
|
||||||
route: BridgeAudioRoute::Speaker,
|
route: BridgeAudioRoute::Speaker,
|
||||||
ios_mode: mode,
|
ios_mode: mode,
|
||||||
processing_backend: match mode {
|
processing_backend: BridgeAudioBackend::PlatformVoiceProcessing,
|
||||||
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => {
|
|
||||||
BridgeAudioBackend::PlatformVoiceProcessing
|
|
||||||
}
|
|
||||||
BridgeIosVoiceProcessingMode::SonoraExperimental => BridgeAudioBackend::WebrtcApm,
|
|
||||||
},
|
|
||||||
vad_backend: BridgeVadBackend::SileroOnnx,
|
vad_backend: BridgeVadBackend::SileroOnnx,
|
||||||
aec: match mode {
|
aec: BridgeEffectOwner::Platform,
|
||||||
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => BridgeEffectOwner::Platform,
|
ns: BridgeEffectOwner::Platform,
|
||||||
BridgeIosVoiceProcessingMode::SonoraExperimental => BridgeEffectOwner::WebrtcApm,
|
agc: BridgeEffectOwner::Platform,
|
||||||
},
|
|
||||||
ns: match mode {
|
|
||||||
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => BridgeEffectOwner::Platform,
|
|
||||||
BridgeIosVoiceProcessingMode::SonoraExperimental => BridgeEffectOwner::WebrtcApm,
|
|
||||||
},
|
|
||||||
agc: match mode {
|
|
||||||
BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => BridgeEffectOwner::Platform,
|
|
||||||
BridgeIosVoiceProcessingMode::SonoraExperimental => BridgeEffectOwner::WebrtcApm,
|
|
||||||
},
|
|
||||||
hpf_enabled: true,
|
hpf_enabled: true,
|
||||||
limiter_enabled: true,
|
limiter_enabled: true,
|
||||||
vad_hangover_ms: 500,
|
vad_hangover_ms: 500,
|
||||||
|
|||||||
@@ -2409,7 +2409,6 @@ impl SseDecode for crate::api::BridgeIosVoiceProcessingMode {
|
|||||||
let mut inner = <i32>::sse_decode(deserializer);
|
let mut inner = <i32>::sse_decode(deserializer);
|
||||||
return match inner {
|
return match inner {
|
||||||
0 => crate::api::BridgeIosVoiceProcessingMode::PlatformVoiceProcessing,
|
0 => crate::api::BridgeIosVoiceProcessingMode::PlatformVoiceProcessing,
|
||||||
1 => crate::api::BridgeIosVoiceProcessingMode::SonoraExperimental,
|
|
||||||
_ => unreachable!(
|
_ => unreachable!(
|
||||||
"Invalid variant for BridgeIosVoiceProcessingMode: {}",
|
"Invalid variant for BridgeIosVoiceProcessingMode: {}",
|
||||||
inner
|
inner
|
||||||
@@ -3445,7 +3444,6 @@ impl flutter_rust_bridge::IntoDart for crate::api::BridgeIosVoiceProcessingMode
|
|||||||
fn into_dart(self) -> flutter_rust_bridge::for_generated::DartAbi {
|
fn into_dart(self) -> flutter_rust_bridge::for_generated::DartAbi {
|
||||||
match self {
|
match self {
|
||||||
Self::PlatformVoiceProcessing => 0.into_dart(),
|
Self::PlatformVoiceProcessing => 0.into_dart(),
|
||||||
Self::SonoraExperimental => 1.into_dart(),
|
|
||||||
_ => unreachable!(),
|
_ => unreachable!(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -4215,7 +4213,6 @@ impl SseEncode for crate::api::BridgeIosVoiceProcessingMode {
|
|||||||
<i32>::sse_encode(
|
<i32>::sse_encode(
|
||||||
match self {
|
match self {
|
||||||
crate::api::BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => 0,
|
crate::api::BridgeIosVoiceProcessingMode::PlatformVoiceProcessing => 0,
|
||||||
crate::api::BridgeIosVoiceProcessingMode::SonoraExperimental => 1,
|
|
||||||
_ => {
|
_ => {
|
||||||
unimplemented!("");
|
unimplemented!("");
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -123,7 +123,7 @@ Runtime event or error
|
|||||||
| Bridge command DTOs | Flutter generated API | `chanora_bridge`, Rust core | Stable typed DTOs; no raw protocol-library types cross to Flutter |
|
| Bridge command DTOs | Flutter generated API | `chanora_bridge`, Rust core | Stable typed DTOs; no raw protocol-library types cross to Flutter |
|
||||||
| Bridge event DTOs | Rust core / bridge | Flutter services/widgets | User-safe errors and capability fields are explicit |
|
| Bridge event DTOs | Rust core / bridge | Flutter services/widgets | User-safe errors and capability fields are explicit |
|
||||||
| Protocol DTOs | `chanora_protocol` | Rust core, state sync | Protocol adapter isolates `tsclientlib` |
|
| Protocol DTOs | `chanora_protocol` | Rust core, state sync | Protocol adapter isolates `tsclientlib` |
|
||||||
| Audio configuration | Flutter settings / Rust core | `chanora_audio` | Voice modes and processing flags are explicit; VAD remains disabled/deferred |
|
| Audio configuration | Flutter settings / Rust core | `chanora_audio` | Voice modes and processing flags are explicit; Windows/Linux desktop VAD-backed `VoiceActivity` is enabled only where runtime evidence exists, with unsupported platforms disabled/deferred |
|
||||||
| Storage records | Storage crate | Rust core / Flutter UI via bridge | Secrets stay behind secure-storage abstraction |
|
| Storage records | Storage crate | Rust core / Flutter UI via bridge | Secrets stay behind secure-storage abstraction |
|
||||||
| Diagnostic bundles | Diagnostics crate | Flutter diagnostics UI | Redaction runs before export or display |
|
| Diagnostic bundles | Diagnostics crate | Flutter diagnostics UI | Redaction runs before export or display |
|
||||||
| Platform capability records | Platform adapters/audio/PTT backends | UI and release record | UI/release wording must not over-claim capability |
|
| Platform capability records | Platform adapters/audio/PTT backends | UI and release record | UI/release wording must not over-claim capability |
|
||||||
@@ -159,7 +159,7 @@ Runtime event or error
|
|||||||
| Secure storage abstraction | Platform storage details do not leak into UI or unrelated crates |
|
| Secure storage abstraction | Platform storage details do not leak into UI or unrelated crates |
|
||||||
| Advisory audio benchmarks | Performance regressions are surfaced without making CI a hard release gate at this stage |
|
| Advisory audio benchmarks | Performance regressions are surfaced without making CI a hard release gate at this stage |
|
||||||
| PTT capability levels | Platform PTT support is represented as capability data and must match release wording |
|
| PTT capability levels | Platform PTT support is represented as capability data and must match release wording |
|
||||||
| VoiceActivity deferral | `VoiceActivity` remains reserved/disabled until a later baseline allocates implementation |
|
| VoiceActivity platform scope | Windows/Linux desktop `VoiceActivity` is implemented through the capture VAD path; unsupported platforms remain disabled/deferred until backend allocation and runtime verification exist |
|
||||||
| No automatic diagnostic upload in MVP | Diagnostics are local and user-initiated unless future approved requirements change policy |
|
| No automatic diagnostic upload in MVP | Diagnostics are local and user-initiated unless future approved requirements change policy |
|
||||||
|
|
||||||
## 11. Verification Handoff
|
## 11. Verification Handoff
|
||||||
|
|||||||
@@ -63,7 +63,7 @@ Design rules:
|
|||||||
| Capture/playback | Platform-specific units handle Android, iOS, desktop/fallback paths behind Rust audio abstractions |
|
| Capture/playback | Platform-specific units handle Android, iOS, desktop/fallback paths behind Rust audio abstractions |
|
||||||
| Codec | Opus encode/decode lives in `opus_voice.rs` and associated audio modules |
|
| Codec | Opus encode/decode lives in `opus_voice.rs` and associated audio modules |
|
||||||
| DSP chain | High-pass filter, noise suppression, echo cancellation, and AGC are represented by audio processing modules/backends |
|
| DSP chain | High-pass filter, noise suppression, echo cancellation, and AGC are represented by audio processing modules/backends |
|
||||||
| Transmit control | `TransmitMode` supports `Ptt`, `Continuous`, and reserved `VoiceActivity`; `VoiceActivity` has no active MVP implementation |
|
| Transmit control | `TransmitMode` supports `Ptt`, `Continuous`, and `VoiceActivity`; `VoiceActivity` is active for Windows/Linux desktop capture when VAD is configured, while mobile, macOS, and unverified-platform enablement remain deferred |
|
||||||
| VoiceActivity gate (capture-side) | `voice_activity::VoiceActivityStateMachine` is the 10 ms-cadence gate for `TransmitMode::VoiceActivity`; open-after 40 ms (debounce), hangover 500 ms (anti-chatter), min-tx 200 ms (anti-flicker), weak-hold 30-100 frames (anti-stale-VAD); live `configure()` re-clamps existing timers on settings change without resetting state; 9 unit tests cover the main paths |
|
| VoiceActivity gate (capture-side) | `voice_activity::VoiceActivityStateMachine` is the 10 ms-cadence gate for `TransmitMode::VoiceActivity`; open-after 40 ms (debounce), hangover 500 ms (anti-chatter), min-tx 200 ms (anti-flicker), weak-hold 30-100 frames (anti-stale-VAD); live `configure()` re-clamps existing timers on settings change without resetting state; 9 unit tests cover the main paths |
|
||||||
| PTT | Desktop/mobile backends expose capability level and active backend; missed-key-up watchdog prevents stuck transmit |
|
| PTT | Desktop/mobile backends expose capability level and active backend; missed-key-up watchdog prevents stuck transmit |
|
||||||
| Release tail | Tail handling prevents abrupt cutoffs after PTT release where configured |
|
| Release tail | Tail handling prevents abrupt cutoffs after PTT release where configured |
|
||||||
|
|||||||
@@ -11,7 +11,7 @@
|
|||||||
| Android Keystore-backed DEK deferred | Secure-storage claim limited | Storage design carries fallback limitation | Platform audit required | Waiver required |
|
| Android Keystore-backed DEK deferred | Secure-storage claim limited | Storage design carries fallback limitation | Platform audit required | Waiver required |
|
||||||
| Full reducer tests incomplete | State verification incomplete | State design remains valid but evidence partial | SWE.4/SWE.5 partial | Blocks full state-sync claim |
|
| Full reducer tests incomplete | State verification incomplete | State design remains valid but evidence partial | SWE.4/SWE.5 partial | Blocks full state-sync claim |
|
||||||
| Desktop/iOS artifacts not release-ready | Platform packaging requirements partial | Release design remains source-build/unsigned | SYS.4 evidence partial | Public binary release No-Go |
|
| Desktop/iOS artifacts not release-ready | Platform packaging requirements partial | Release design remains source-build/unsigned | SYS.4 evidence partial | Public binary release No-Go |
|
||||||
| VAD deferred | VoiceActivity not active | UI must show disabled/coming-soon | No VAD pass claim | No VAD marketing claim |
|
| VAD platform-scoped | VoiceActivity active only for verified Windows/Linux desktop paths | UI must show disabled/unavailable on unsupported platforms | VAD pass claim must name verified platform/runtime evidence | No broad VAD marketing claim without platform scope |
|
||||||
|
|
||||||
## 2. Conclusion
|
## 2. Conclusion
|
||||||
|
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ This record captures the current maintainability review so implementation, verif
|
|||||||
| Android Keystore-backed DEK remains deferred | Android identity/bookmark encryption has weaker fail-safe properties than final target secure-storage design. | Android secure-storage audit or waiver; explicit release-readiness limitation. |
|
| Android Keystore-backed DEK remains deferred | Android identity/bookmark encryption has weaker fail-safe properties than final target secure-storage design. | Android secure-storage audit or waiver; explicit release-readiness limitation. |
|
||||||
| Android permission/audio lifecycle needs deeper route exercise | Build/install/launch smoke now passes on the emulator, but full permission-flow and audio-route lifecycle behavior still need an interactive scenario or device test before release. | Device/emulator scenario covering permission request/denial/grant, voice controls, foreground service, audio focus, and route/SCO transitions. |
|
| Android permission/audio lifecycle needs deeper route exercise | Build/install/launch smoke now passes on the emulator, but full permission-flow and audio-route lifecycle behavior still need an interactive scenario or device test before release. | Device/emulator scenario covering permission request/denial/grant, voice controls, foreground service, audio focus, and route/SCO transitions. |
|
||||||
| iOS device runtime verification not executed in this review | The iOS audio-session error path is hardened, but VoiceProcessingIO/session ordering and runtime audio behavior still need device evidence. | iOS device or simulator build/run plus audio-session smoke evidence before iOS runtime success is claimed. |
|
| iOS device runtime verification not executed in this review | The iOS audio-session error path is hardened, but VoiceProcessingIO/session ordering and runtime audio behavior still need device evidence. | iOS device or simulator build/run plus audio-session smoke evidence before iOS runtime success is claimed. |
|
||||||
| VAD / VoiceActivity wording drift | VAD assets, tests, and scaffolding exist, but product-enabled `VoiceActivity` remains reserved/disabled per DEC-030. | Release, README, and verification wording must distinguish scaffolding/assets/tests from shipped product behavior. |
|
| VAD / VoiceActivity wording drift | VAD assets, tests, and Windows/Linux desktop runtime wiring exist, but product-enabled `VoiceActivity` must remain platform-scoped per DEC-030. | Release, README, and verification wording must distinguish verified desktop behavior from unsupported mobile/macOS/unverified-platform behavior. |
|
||||||
| Protocol voice packet re-export is an intentional exception | Future maintainers may assume complete protocol isolation and accidentally widen the Seam. | Architecture note in SAD/SDD or a decision-register entry. |
|
| Protocol voice packet re-export is an intentional exception | Future maintainers may assume complete protocol isolation and accidentally widen the Seam. | Architecture note in SAD/SDD or a decision-register entry. |
|
||||||
| Bridge DTO mirror drift | Field additions can be missed across Core, Bridge, and Dart generated DTOs. | Bridge generation check plus Flutter analyze/test after bridge DTO changes. |
|
| Bridge DTO mirror drift | Field additions can be missed across Core, Bridge, and Dart generated DTOs. | Bridge generation check plus Flutter analyze/test after bridge DTO changes. |
|
||||||
| Full live reducer integration remains separate from reducer unit coverage | State reducer tests are strong, but runtime UI still has snapshot/probe paths. | SWE.5 integration run proving live protocol events fold through the intended state path, or explicit P1 deferral. |
|
| Full live reducer integration remains separate from reducer unit coverage | State reducer tests are strong, but runtime UI still has snapshot/probe paths. | SWE.5 integration run proving live protocol events fold through the intended state path, or explicit P1 deferral. |
|
||||||
@@ -91,7 +91,7 @@ adb -s emulator-5554 shell pidof app.chanora.chanora_flutter
|
|||||||
|
|
||||||
- Android minimum runtime baseline is API 28 (Android 9.0) per SysRS-288, SRS-187, DEC-004, and the Gradle `minSdk = 28` configuration. Documents must not revive the older API 24 baseline.
|
- Android minimum runtime baseline is API 28 (Android 9.0) per SysRS-288, SRS-187, DEC-004, and the Gradle `minSdk = 28` configuration. Documents must not revive the older API 24 baseline.
|
||||||
- Flutter app version/build is `0.3.0+100` in `apps/chanora_flutter/pubspec.yaml`. Rust workspace package version remains `0.2.0-beta.1`. Release documents must distinguish these values instead of treating them as one candidate version.
|
- Flutter app version/build is `0.3.0+100` in `apps/chanora_flutter/pubspec.yaml`. Rust workspace package version remains `0.2.0-beta.1`. Release documents must distinguish these values instead of treating them as one candidate version.
|
||||||
- The v0.3.0 changelog entry may mention VAD assets/backends only as implementation scaffolding; product-enabled `VoiceActivity` remains disabled/coming-soon until DEC-030 is superseded and runtime verification exists.
|
- The v0.3.0 changelog entry may mention Windows/Linux desktop VAD-backed `VoiceActivity` only with matching runtime evidence; mobile, macOS, and unverified-platform `VoiceActivity` remain disabled/unavailable until DEC-030 is superseded and runtime verification exists.
|
||||||
- README wording must describe the existing Flutter/Rust workspace and app scaffold, not a future scaffold that has not been created.
|
- README wording must describe the existing Flutter/Rust workspace and app scaffold, not a future scaffold that has not been created.
|
||||||
|
|
||||||
## 8. Git Policy
|
## 8. Git Policy
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ This register records product and engineering decisions referenced by the DV doc
|
|||||||
| DEC-012 legal/trademark/OSS review | Open | Blocks public/store release |
|
| DEC-012 legal/trademark/OSS review | Open | Blocks public/store release |
|
||||||
| DEC-020 dual license MIT OR Apache-2.0 | Accepted per README | Supports license posture; dependency notices still require review |
|
| DEC-020 dual license MIT OR Apache-2.0 | Accepted per README | Supports license posture; dependency notices still require review |
|
||||||
| DEC-027 desktop mouse side-button PTT | Accepted by requirements baseline | Verification must not over-claim unsupported platform input classes |
|
| DEC-027 desktop mouse side-button PTT | Accepted by requirements baseline | Verification must not over-claim unsupported platform input classes |
|
||||||
| DEC-030 VAD deferral | Accepted as deferral | VAD scaffolding, assets, and tests may exist, but product `VoiceActivity` remains disabled/coming-soon until a later baseline enables and verifies it |
|
| DEC-030 VAD platform scope | Partially superseded by desktop enablement | Windows/Linux desktop `VoiceActivity` is enabled through the audio capture VAD path and must be claimed only with matching runtime evidence; mobile, macOS, and unverified-platform `VoiceActivity` remain disabled/deferred until a later baseline enables and verifies them |
|
||||||
| DEC-032 Android CMake patch exit path | Active tracking | Patched dependency requires reevaluation |
|
| DEC-032 Android CMake patch exit path | Active tracking | Patched dependency requires reevaluation |
|
||||||
| DEC-033 macOS VPIO ducking configuration | Accepted | Write `kAUVoiceIOProperty_OtherAudioDuckingConfiguration` with `mEnableAdvancedDucking=0` (disables dynamic voice-activity-driven ducking) and `mDuckingLevel=Min` (= 10) to minimise the ducking of other apps' audio during a voice session; property is macOS 14+ only, the macOS 13 set fails silently (debug log) and VPIO uses its default behaviour; matches the iOS `.voiceChat` baseline on macOS 14+ |
|
| DEC-033 macOS VPIO ducking configuration | Accepted | Write `kAUVoiceIOProperty_OtherAudioDuckingConfiguration` with `mEnableAdvancedDucking=0` (disables dynamic voice-activity-driven ducking) and `mDuckingLevel=Min` (= 10) to minimise the ducking of other apps' audio during a voice session; property is macOS 14+ only, the macOS 13 set fails silently (debug log) and VPIO uses its default behaviour; matches the iOS `.voiceChat` baseline on macOS 14+ |
|
||||||
| DEC-034 Android runtime verification gate | Active tracking | Android target compilation, install, and runtime smoke are blocked locally until `aarch64-linux-android-clang` is available and `adb devices -l` shows an authorized target; release docs must not claim Android runtime success |
|
| DEC-034 Android runtime verification gate | Active tracking | Android target compilation, install, and runtime smoke are blocked locally until `aarch64-linux-android-clang` is available and `adb devices -l` shows an authorized target; release docs must not claim Android runtime success |
|
||||||
|
|||||||
@@ -21,7 +21,7 @@
|
|||||||
| Protocol adapter | `chanora_protocol` — `tsclientlib` isolated behind `ProtocolClient`, typed DTOs, `ProtocolError` catalogue |
|
| Protocol adapter | `chanora_protocol` — `tsclientlib` isolated behind `ProtocolClient`, typed DTOs, `ProtocolError` catalogue |
|
||||||
| Connection lifecycle | `chanora_core` — supervisor task, exponential backoff reconnect (1s→60s), user-disconnect suppresses reconnect; branch `simplify-project-review` has started splitting the previous large `lib.rs` into focused internal Modules (`events.rs`, `network_diagnostics.rs`) while preserving public re-exports |
|
| Connection lifecycle | `chanora_core` — supervisor task, exponential backoff reconnect (1s→60s), user-disconnect suppresses reconnect; branch `simplify-project-review` has started splitting the previous large `lib.rs` into focused internal Modules (`events.rs`, `network_diagnostics.rs`) while preserving public re-exports |
|
||||||
| State sync reducer unit | `chanora_state` — `ConnectionState`, `channel_join`, snapshot/delta reducers, reconnect handling, deterministic ordering, malformed duplicate normalization, channel-delete/client cleanup, and reducer unit tests. Runtime core integration still uses snapshot/probe refresh paths and remains separate validation work. |
|
| State sync reducer unit | `chanora_state` — `ConnectionState`, `channel_join`, snapshot/delta reducers, reconnect handling, deterministic ordering, malformed duplicate normalization, channel-delete/client cleanup, and reducer unit tests. Runtime core integration still uses snapshot/probe refresh paths and remains separate validation work. |
|
||||||
| Audio subsystem | `chanora_audio` — Opus encode/decode, HPF/NS/AEC3/AGC2 DSP, PTT backends (Windows/macOS/Linux/focused), iOS VoiceProcessingIO, Android Oboe, jitter buffer via `tsclientlib::audio::AudioHandler`, mixer, mute/deaf gates, release-tail timer, and VAD scaffolding/assets. Product `VoiceActivity` remains disabled per DEC-030. |
|
| Audio subsystem | `chanora_audio` — Opus encode/decode, HPF/NS/AEC3/AGC2 DSP, PTT backends (Windows/macOS/Linux/focused), iOS VoiceProcessingIO, Android Oboe, jitter buffer via `tsclientlib::audio::AudioHandler`, mixer, mute/deaf gates, release-tail timer, and Windows/Linux desktop `VoiceActivity` through the capture VAD path. Mobile, macOS, and unverified-platform `VoiceActivity` remain deferred per DEC-030. |
|
||||||
| Push-to-talk | Per-platform backends: Windows Raw Input + hook fallback, macOS Event Tap, Linux freedesktop portal, focused fallback; `PttCapabilityLevel` (L0–L3); missed-key-up watchdog |
|
| Push-to-talk | Per-platform backends: Windows Raw Input + hook fallback, macOS Event Tap, Linux freedesktop portal, focused fallback; `PttCapabilityLevel` (L0–L3); missed-key-up watchdog |
|
||||||
| Voice controls UI | `voice_bar`, `voice_compact`, `voice_haptics`, `voice_level_meter`, `voice_platform`, `ptt_capability_badge`, `talk_power_warning` |
|
| Voice controls UI | `voice_bar`, `voice_compact`, `voice_haptics`, `voice_level_meter`, `voice_platform`, `ptt_capability_badge`, `talk_power_warning` |
|
||||||
| Storage (non-secret) | `chanora_storage` — `BookmarkRepository` (SQLite/rusqlite bundled, schema v2), ChaCha20-Poly1305 encrypted passwords |
|
| Storage (non-secret) | `chanora_storage` — `BookmarkRepository` (SQLite/rusqlite bundled, schema v2), ChaCha20-Poly1305 encrypted passwords |
|
||||||
@@ -54,7 +54,7 @@
|
|||||||
|---|---|
|
|---|---|
|
||||||
| Event replay tooling | Reducer tests cover the state-sync contract, but standalone replay-file tooling remains a P1 verification gap. |
|
| Event replay tooling | Reducer tests cover the state-sync contract, but standalone replay-file tooling remains a P1 verification gap. |
|
||||||
| Reducer runtime integration evidence | The standalone reducer is unit-tested, but `chanora_core` still refreshes UI state through snapshot/probe paths rather than folding all live protocol events through `chanora_state::reduce`. |
|
| Reducer runtime integration evidence | The standalone reducer is unit-tested, but `chanora_core` still refreshes UI state through snapshot/probe paths rather than folding all live protocol events through `chanora_state::reduce`. |
|
||||||
| Silero VAD | `assets/models/silero_vad.onnx` bundled but DEC-030 defers VAD to P1; `TransmitMode::VoiceActivity` is reserved and disabled in this baseline. |
|
| Mobile/macOS VoiceActivity | `assets/models/silero_vad.onnx` is bundled and used by the desktop VAD path where runtime evidence supports it; mobile, macOS, and unverified-platform `TransmitMode::VoiceActivity` remain disabled/deferred until a later baseline supplies backend enablement and verification evidence. |
|
||||||
| macOS build | Source-buildable only; no public release artifact is approved. |
|
| macOS build | Source-buildable only; no public release artifact is approved. |
|
||||||
| Windows build | Source-buildable only; no public release artifact is approved. |
|
| Windows build | Source-buildable only; no public release artifact is approved. |
|
||||||
| iOS build | Source-buildable/unsigned validation only; no TestFlight/App Store release artifact is approved. |
|
| iOS build | Source-buildable/unsigned validation only; no TestFlight/App Store release artifact is approved. |
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ A waiver records a known gap that reviewers may accept for a limited decision sc
|
|||||||
| DV-WVR-004 | Standalone event replay and runtime reducer-integration evidence not yet complete | `docs/implementation-status-2026-05-28.md`, local `cargo test -p chanora_state --locked` evidence | Reducer unit coverage supports state synchronization, but file-based replay evidence and live-event integration evidence remain open | DV documentation pass and internal validation | Event replay tooling implemented or requirement reprioritized; runtime reducer integration evidence attached |
|
| DV-WVR-004 | Standalone event replay and runtime reducer-integration evidence not yet complete | `docs/implementation-status-2026-05-28.md`, local `cargo test -p chanora_state --locked` evidence | Reducer unit coverage supports state synchronization, but file-based replay evidence and live-event integration evidence remain open | DV documentation pass and internal validation | Event replay tooling implemented or requirement reprioritized; runtime reducer integration evidence attached |
|
||||||
| DV-WVR-005 | Event replay tool not found | `docs/implementation-status-2026-05-28.md` | Limits P1 state verification hooks SRS-061/SRS-098 | Accepted as P1 deferral | Event replay tooling implemented or requirement reprioritized |
|
| DV-WVR-005 | Event replay tool not found | `docs/implementation-status-2026-05-28.md` | Limits P1 state verification hooks SRS-061/SRS-098 | Accepted as P1 deferral | Event replay tooling implemented or requirement reprioritized |
|
||||||
| DV-WVR-006 | Desktop and iOS artifacts are source-buildable or unsigned only | `docs/implementation-status-2026-05-28.md`, `docs/release/ios-build.md` | Blocks packaged public release claims | Internal validation from source/unsigned builds only | Signed/notarized/package artifacts exist and hashes are recorded |
|
| DV-WVR-006 | Desktop and iOS artifacts are source-buildable or unsigned only | `docs/implementation-status-2026-05-28.md`, `docs/release/ios-build.md` | Blocks packaged public release claims | Internal validation from source/unsigned builds only | Signed/notarized/package artifacts exist and hashes are recorded |
|
||||||
| DV-WVR-007 | Silero VAD asset bundled while `VoiceActivity` is deferred | `docs/implementation-status-2026-05-28.md` | Risk that UI/release wording overstates VAD availability | DV may pass if VoiceActivity remains disabled/coming-soon | VAD implementation allocated in a later baseline or asset/wording reconciled |
|
| DV-WVR-007 | Silero VAD asset bundled while `VoiceActivity` is platform-scoped | `docs/implementation-status-2026-05-28.md` | Risk that UI/release wording overstates VAD availability beyond verified Windows/Linux desktop paths | DV may pass if VoiceActivity claims are limited to verified desktop evidence and unsupported platforms remain disabled/unavailable | Mobile/macOS implementation allocated in a later baseline or asset/wording reconciled |
|
||||||
| DV-WVR-008 | Artifact hashes, tag, and candidate run IDs are not recorded in release record | `docs/release/release-readiness-go-nogo-record.md` | Blocks final release approval and reproducibility | DV documentation review only | Candidate build run records, tag, commit SHA, and artifact hashes are recorded |
|
| DV-WVR-008 | Artifact hashes, tag, and candidate run IDs are not recorded in release record | `docs/release/release-readiness-go-nogo-record.md` | Blocks final release approval and reproducibility | DV documentation review only | Candidate build run records, tag, commit SHA, and artifact hashes are recorded |
|
||||||
| DV-WVR-009 | Android target compile and runtime smoke blocked locally | `docs/governance/maintainability-review-2026-06-08.md`, `docs/release/release-readiness-go-nogo-record.md` | Blocks Android runtime, permission-flow, and audio-lifecycle success claims | Documentation review only; internal validation must keep Android limitation stated | Android NDK compiler `aarch64-linux-android-clang` is available, `adb devices -l` shows an authorized target, and Android build/install/smoke evidence is attached |
|
| DV-WVR-009 | Android target compile and runtime smoke blocked locally | `docs/governance/maintainability-review-2026-06-08.md`, `docs/release/release-readiness-go-nogo-record.md` | Blocks Android runtime, permission-flow, and audio-lifecycle success claims | Documentation review only; internal validation must keep Android limitation stated | Android NDK compiler `aarch64-linux-android-clang` is available, `adb devices -l` shows an authorized target, and Android build/install/smoke evidence is attached |
|
||||||
|
|
||||||
|
|||||||
@@ -49,6 +49,11 @@ terms.
|
|||||||
| `flutter` | 0.0.0 | sdk | yes |
|
| `flutter` | 0.0.0 | sdk | yes |
|
||||||
| `flutter_foreground_task` | 9.2.2 | hosted | yes |
|
| `flutter_foreground_task` | 9.2.2 | hosted | yes |
|
||||||
| `flutter_lints` | 6.0.0 | hosted | yes |
|
| `flutter_lints` | 6.0.0 | hosted | yes |
|
||||||
|
| `flutter_local_notifications` | 22.0.0 | hosted | yes |
|
||||||
|
| `flutter_local_notifications_linux` | 8.0.1 | hosted | yes |
|
||||||
|
| `flutter_local_notifications_platform_interface` | 12.0.0 | hosted | yes |
|
||||||
|
| `flutter_local_notifications_web` | 1.0.0 | hosted | yes |
|
||||||
|
| `flutter_local_notifications_windows` | 3.1.0 | hosted | yes |
|
||||||
| `flutter_localizations` | 0.0.0 | sdk | yes |
|
| `flutter_localizations` | 0.0.0 | sdk | yes |
|
||||||
| `flutter_rust_bridge` | 2.12.0 | hosted | yes |
|
| `flutter_rust_bridge` | 2.12.0 | hosted | yes |
|
||||||
| `flutter_test` | 0.0.0 | sdk | yes |
|
| `flutter_test` | 0.0.0 | sdk | yes |
|
||||||
@@ -117,6 +122,7 @@ terms.
|
|||||||
| `string_scanner` | 1.4.1 | hosted | yes |
|
| `string_scanner` | 1.4.1 | hosted | yes |
|
||||||
| `term_glyph` | 1.2.2 | hosted | yes |
|
| `term_glyph` | 1.2.2 | hosted | yes |
|
||||||
| `test_api` | 0.7.11 | hosted | yes |
|
| `test_api` | 0.7.11 | hosted | yes |
|
||||||
|
| `timezone` | 0.11.0 | hosted | yes |
|
||||||
| `typed_data` | 1.4.0 | hosted | yes |
|
| `typed_data` | 1.4.0 | hosted | yes |
|
||||||
| `url_launcher` | 6.3.2 | hosted | yes |
|
| `url_launcher` | 6.3.2 | hosted | yes |
|
||||||
| `url_launcher_android` | 6.3.30 | hosted | yes |
|
| `url_launcher_android` | 6.3.30 | hosted | yes |
|
||||||
@@ -1869,6 +1875,166 @@ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|||||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### flutter_local_notifications 22.0.0
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright 2018 Michael Bui. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of the copyright holder nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### flutter_local_notifications_linux 8.0.1
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright 2018 Michael Bui. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of the copyright holder nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### flutter_local_notifications_platform_interface 12.0.0
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright 2020 Michael Bui. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of the copyright holder nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### flutter_local_notifications_web 1.0.0
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright 2020 Michael Bui. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of the copyright holder nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### flutter_local_notifications_windows 3.1.0
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright 2024 Michael Bui. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of the copyright holder nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
### flutter_localizations 0.0.0
|
### flutter_localizations 0.0.0
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -4673,6 +4839,33 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|||||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### timezone 0.11.0
|
||||||
|
|
||||||
|
```
|
||||||
|
Copyright (c) 2014, timezone project authors.
|
||||||
|
All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer in the
|
||||||
|
documentation and/or other materials provided with the distribution.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||||
|
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||||
|
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||||
|
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
|
||||||
|
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
||||||
|
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
||||||
|
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
|
||||||
|
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||||
|
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
```
|
||||||
|
|
||||||
### typed_data 1.4.0
|
### typed_data 1.4.0
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -291,7 +291,7 @@ those terms.
|
|||||||
| chanora_diagnostics | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
| chanora_diagnostics | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
||||||
| chanora_prefetch | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
| chanora_prefetch | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
||||||
| chanora_protocol | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
| chanora_protocol | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
||||||
| chanora_resolver | 0.1.0 | `Apache License 2.0` | — |
|
| chanora_resolver | 0.2.0-beta.1 | `Apache License 2.0` | — |
|
||||||
| chanora_state | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
| chanora_state | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
||||||
| chanora_storage | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
| chanora_storage | 0.2.0-beta.1 | `Apache License 2.0` | <https://github.com/anomalyco/opencode> |
|
||||||
| alsa | 0.11.0 | `Apache License 2.0` | <https://github.com/diwic/alsa-rs> |
|
| alsa | 0.11.0 | `Apache License 2.0` | <https://github.com/diwic/alsa-rs> |
|
||||||
@@ -10967,3 +10967,4 @@ cargo about generate --output-file docs/security/license-inventory.html about.hb
|
|||||||
|
|
||||||
This artefact supports the DEC-012 legal review handoff at
|
This artefact supports the DEC-012 legal review handoff at
|
||||||
`docs/governance/legal-review-readiness.md`.
|
`docs/governance/legal-review-readiness.md`.
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -2597,7 +2597,7 @@ This section extends the ASPICE SWE.1 Software Requirements Specification. The s
|
|||||||
- Source SysDes: SysDes-150
|
- Source SysDes: SysDes-150
|
||||||
- Verification method: Integration Test, UI Review
|
- Verification method: Integration Test, UI Review
|
||||||
|
|
||||||
**SRS-205**: The software shall represent the user's voice transmit mode as a `TransmitMode` enum with variants `Ptt`, `Continuous`, and `VoiceActivity` (the last reserved with no v1 implementation per DEC-030). The setting shall be persisted per identity via the identity store. The default value for a fresh install shall be `Ptt`. The UI shall render `VoiceActivity` as a disabled "coming soon" option until an implementation is allocated in a later baseline.
|
**SRS-205**: The software shall represent the user's voice transmit mode as a `TransmitMode` enum with variants `Ptt`, `Continuous`, and `VoiceActivity`. `VoiceActivity` shall be selectable only on platforms with an implemented and verified VAD capture path in this baseline (currently Windows/Linux desktop); mobile, macOS, and unverified-platform enablement remain deferred per DEC-030. The setting shall be persisted per identity via the identity store where the selected platform supports it. The default value for a fresh install shall be `Ptt`. The UI shall render `VoiceActivity` as disabled/unavailable on unsupported platforms rather than claiming runtime support.
|
||||||
|
|
||||||
- Status: Baseline Candidate
|
- Status: Baseline Candidate
|
||||||
- Type: Software Interface Requirement
|
- Type: Software Interface Requirement
|
||||||
@@ -2837,7 +2837,7 @@ Consistent with the SysRS-307 / SysRS-308 / SysRS-309 deferrals propagated throu
|
|||||||
|
|
||||||
| Version | Date | Description |
|
| Version | Date | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle software requirements SRS-204 through SRS-207 sourced from SysDes-149..151: bridge surface drops `start_audio` / `stop_audio` and adds `voice_join(channel_id)` / `voice_leave()`, audio engine opens streams on first voice-channel join and closes on last leave with output independent of mic-permission state, `TransmitMode` enum (`Ptt` default, `Continuous`, reserved `VoiceActivity` per DEC-030) persisted per identity, `release_tail_ms` (default 200, range 0–500) gate on the `true → false` transition of `transmit_active` with re-press cancellation, Voice Bar hard-mute override of `transmit_active`. Strict layered sourcing preserved (`SRS -> SysDes` only). |
|
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle software requirements SRS-204 through SRS-207 sourced from SysDes-149..151: bridge surface drops `start_audio` / `stop_audio` and adds `voice_join(channel_id)` / `voice_leave()`, audio engine opens streams on first voice-channel join and closes on last leave with output independent of mic-permission state, `TransmitMode` enum (`Ptt` default, `Continuous`, platform-scoped `VoiceActivity` per DEC-030) persisted per identity where supported, `release_tail_ms` (default 200, range 0–500) gate on the `true → false` transition of `transmit_active` with re-press cancellation, Voice Bar hard-mute override of `transmit_active`. Strict layered sourcing preserved (`SRS -> SysDes` only). |
|
||||||
|
|
||||||
|
|
||||||
## Baseline Candidate 0.9.6 Update
|
## Baseline Candidate 0.9.6 Update
|
||||||
|
|||||||
+2
-2
@@ -2852,7 +2852,7 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through
|
|||||||
- ASPICE SYS.3 alignment: Architecture constraints
|
- ASPICE SYS.3 alignment: Architecture constraints
|
||||||
- Allocated SysRS: SysRS-298
|
- Allocated SysRS: SysRS-298
|
||||||
|
|
||||||
**SysDes-149**: The system architecture shall define a `TransmitMode` element carried as an enum at the audio + bridge + UI boundary with variants `Ptt`, `Continuous`, and a reserved `VoiceActivity` placeholder that has no allocated implementation in this baseline (deferred per DEC-030). The active mode shall be persisted in the identity store; the UI shall surface `VoiceActivity` as a disabled "coming soon" option until an implementation is allocated in a later baseline.
|
**SysDes-149**: The system architecture shall define a `TransmitMode` element carried as an enum at the audio + bridge + UI boundary with variants `Ptt`, `Continuous`, and `VoiceActivity`. `VoiceActivity` shall be surfaced only when the current platform has an allocated, implemented, and verified VAD capture path in this baseline (currently Windows/Linux desktop); mobile, macOS, and unverified-platform enablement remain deferred per DEC-030. The active mode shall be persisted in the identity store where the selected platform supports it; unsupported platforms shall surface `VoiceActivity` as disabled/unavailable rather than claiming runtime support.
|
||||||
|
|
||||||
- Status: Baseline Candidate
|
- Status: Baseline Candidate
|
||||||
- Type: Subsystem Interface
|
- Type: Subsystem Interface
|
||||||
@@ -3029,7 +3029,7 @@ This SysDes version covers all known SysRS requirements from `SysRS-001` through
|
|||||||
|
|
||||||
| Version | Date | Description |
|
| Version | Date | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle allocation SysDes-149 through SysDes-151 sourced from SysRS-303 / SysRS-304: `TransmitMode` enum element (`Ptt` / `Continuous` / reserved `VoiceActivity` per DEC-030) at the audio + bridge + UI boundary, audio engine lifecycle bound to voice-channel membership with no manual start affordance and a listen-only path independent of mic permission, hard-mute override element, and the release-tail timer adapter (default 200 ms, range 0–500 ms) on `transmit_active`. Strict layered sourcing preserved (`SysDes -> SysRS` only). |
|
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle allocation SysDes-149 through SysDes-151 sourced from SysRS-303 / SysRS-304: `TransmitMode` enum element (`Ptt` / `Continuous` / platform-scoped `VoiceActivity` per DEC-030) at the audio + bridge + UI boundary, audio engine lifecycle bound to voice-channel membership with no manual start affordance and a listen-only path independent of mic permission, hard-mute override element, and the release-tail timer adapter (default 200 ms, range 0–500 ms) on `transmit_active`. Strict layered sourcing preserved (`SysDes -> SysRS` only). |
|
||||||
|
|
||||||
|
|
||||||
## Baseline Candidate 0.9.6 Update
|
## Baseline Candidate 0.9.6 Update
|
||||||
|
|||||||
+2
-2
@@ -1924,7 +1924,7 @@ This section converts the baseline product decisions into auditable system-level
|
|||||||
- Priority: P0
|
- Priority: P0
|
||||||
- Verification: Privacy Review, Security Audit, Diagnostic Inspection
|
- Verification: Privacy Review, Security Audit, Diagnostic Inspection
|
||||||
|
|
||||||
**SysRS-303**: The Chanora application system shall define the v1 voice transmit mode set as `Ptt` and `Continuous`, with `VoiceActivity` reserved on the enum surface but unimplemented in this baseline (deferred per DEC-030). The default mode on a fresh install shall be `Ptt`; the user-selected mode shall be persisted per identity. The audio engine lifecycle shall be bound to voice-channel membership: input and output streams shall open on the user's first voice-channel join of the session and shall close on the last voice-channel leave, with no manual start affordance exposed at any system interface. The output stream shall open regardless of microphone-permission state so listen-only is a first-class flow. A user-facing hard-mute toggle shall force the transmit gate closed and shall override the active transmit mode, the PTT key state, and every other internal signal.
|
**SysRS-303**: The Chanora application system shall define the v1 voice transmit mode set as `Ptt`, `Continuous`, and `VoiceActivity`. `VoiceActivity` shall be enabled only on platforms with an implemented and verified VAD capture path in this baseline (currently Windows/Linux desktop); mobile, macOS, and unverified-platform enablement remain deferred per DEC-030. The default mode on a fresh install shall be `Ptt`; the user-selected mode shall be persisted per identity where the selected platform supports it. The audio engine lifecycle shall be bound to voice-channel membership: input and output streams shall open on the user's first voice-channel join of the session and shall close on the last voice-channel leave, with no manual start affordance exposed at any system interface. The output stream shall open regardless of microphone-permission state so listen-only is a first-class flow. A user-facing hard-mute toggle shall force the transmit gate closed and shall override the active transmit mode, the PTT key state, and every other internal signal.
|
||||||
|
|
||||||
- Priority: P0
|
- Priority: P0
|
||||||
- Verification: Functional Test, UX Review
|
- Verification: Functional Test, UX Review
|
||||||
@@ -1989,7 +1989,7 @@ This section converts the baseline product decisions into auditable system-level
|
|||||||
|
|
||||||
| Version | Date | Description |
|
| Version | Date | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle requirements SysRS-303 and SysRS-304 capturing the no-manual-start audio engine bound to voice-channel membership, the v1 transmit-mode set (`Ptt` default + `Continuous`, with `VoiceActivity` reserved per DEC-030), the listen-only flow (output independent of mic permission), the hard-mute override, and the 200 ms (0–500 ms) PTT release tail for word-boundary anti-clipping. |
|
| 0.9.5 | 2026-05-15 | Added v1 audio + PTT lifecycle requirements SysRS-303 and SysRS-304 capturing the no-manual-start audio engine bound to voice-channel membership, the v1 transmit-mode set (`Ptt` default, `Continuous`, and platform-scoped `VoiceActivity` per DEC-030), the listen-only flow (output independent of mic permission), the hard-mute override, and the 200 ms (0–500 ms) PTT release tail for word-boundary anti-clipping. |
|
||||||
|
|
||||||
|
|
||||||
## Baseline Candidate 0.9.9 Update
|
## Baseline Candidate 0.9.9 Update
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
@echo off
|
||||||
|
rem Wrapper around cmake.exe that injects policy/cache variables to force
|
||||||
|
rem the release dynamic CRT (/MD) for Debug configurations. audiopus_sys
|
||||||
|
rem calls cmake::build(opus_path), which invokes CMake configure; the
|
||||||
|
rem upstream Opus CMakeLists.txt does not set CMP0091 or
|
||||||
|
rem CMAKE_MSVC_RUNTIME_LIBRARY, so CMake defaults to /MDd in Debug
|
||||||
|
rem builds. That pulls in __imp__CrtDbgReportW which is unresolved at
|
||||||
|
rem test link time because the rest of the Rust/Cargo graph uses /MD.
|
||||||
|
rem
|
||||||
|
rem This wrapper is only invoked on Windows MSVC targets (see
|
||||||
|
rem .cargo/config.toml [env] CMAKE_<target-triple> entries). Non-Windows
|
||||||
|
rem hosts are unaffected.
|
||||||
|
rem
|
||||||
|
rem cmake-rs invokes this as: cmake-msvc-release-crt.cmd <configure-args>
|
||||||
|
rem and also: cmake-msvc-release-crt.cmd --build ...
|
||||||
|
rem and also: cmake-msvc-release-crt.cmd --version
|
||||||
|
rem and also: cmake-msvc-release-crt.cmd -E ...
|
||||||
|
rem
|
||||||
|
rem Echoes a marker to stdout so the cargo build log proves the wrapper
|
||||||
|
rem was actually invoked. cmake-rs forwards cmake stdout to the build
|
||||||
|
rem script log, so this surfaces in `cargo build -vv` output.
|
||||||
|
setlocal
|
||||||
|
|
||||||
|
echo cmake-msvc-release-crt.cmd: invoked with %*
|
||||||
|
|
||||||
|
rem Pass through non-configure invocations unchanged.
|
||||||
|
if "%~1"=="--build" goto :passthrough
|
||||||
|
if "%~1"=="--version" goto :passthrough
|
||||||
|
if "%~1"=="-E" goto :passthrough
|
||||||
|
if "%~1"=="--install" goto :passthrough
|
||||||
|
if "%~1"=="--open" goto :passthrough
|
||||||
|
|
||||||
|
rem Configure invocation: inject -D flags before user args.
|
||||||
|
echo cmake-msvc-release-crt.cmd: injecting CMP0091=NEW and CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL
|
||||||
|
cmake.exe -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL %*
|
||||||
|
exit /b %ERRORLEVEL%
|
||||||
|
|
||||||
|
:passthrough
|
||||||
|
cmake.exe %*
|
||||||
|
exit /b %ERRORLEVEL%
|
||||||
Reference in New Issue
Block a user