edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #13 from EdisonJwa/feat/apple-coreml-vad

Browse Source 
Add Apple CoreML Silero VAD
This commit is contained in:
Edison Jwa
2026-06-02 20:14:17 +09:00
committed by GitHub
parent 96aa943d5f 1813bbaa0c
commit dab90852a3
24 changed files with 798 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/bench-advisory.yml
+9 -5
View File
@@ -1,7 +1,7 @@
name: bench-advisory
# SDD-120 §6 / SRS-218 — advisory-only realtime-audio bench workflow.
# Runs the criterion bench harness on PR + push events, compares the
# Runs the criterion bench harness on PR + tag events, compares the
# current results against the SAD-089 baseline JSON resolved at the
# merge-base, and renders a markdown report posted (or updated) as a
# single sticky PR comment. The job status is ALWAYS success — this
@@ -11,7 +11,7 @@ on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches: [product/scaffold-v0]
tags: ["**"]
permissions:
pull-requests: write
@@ -25,20 +25,24 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: System deps (cpal / Opus)
- name: System deps (cpal / Opus / SDL2)
run: |
sudo apt-get update
sudo apt-get install -y \
libasound2-dev libpulse-dev pkg-config \
libdbus-1-dev \
libsdl2-dev \
libopus-dev
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Run benchmarks
run: |
cargo bench -p chanora_audio \
if ! cargo bench -p chanora_audio \
--bench realtime_capture \
--bench opus_codec \
--bench resampler
--bench resampler; then
echo "::warning::Benchmark harness failed; continuing advisory workflow per SRS-218 clause 4"
fi
- name: Emit current baseline JSON
run: cargo run --example emit_baseline -p chanora_audio
- name: Resolve merge-base baseline
.github/workflows/ci.yml
+14 -2
View File
@@ -2,7 +2,7 @@ name: ci
on:
push:
branches: ["**"]
tags: ["**"]
pull_request:
jobs:
@@ -11,11 +11,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: System deps (cpal / Opus / SQLite)
- name: System deps (cpal / Opus / SQLite / SDL2)
run: |
sudo apt-get update
sudo apt-get install -y \
libasound2-dev libpulse-dev pkg-config \
libdbus-1-dev \
libsdl2-dev \
libopus-dev
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
@@ -119,6 +121,16 @@ jobs:
- name: flutter pub get
working-directory: apps/chanora_flutter
run: flutter pub get
- name: Check local SileroCoreML package
id: silero-coreml
run: |
if [ -d ../silero-coreml ]; then
echo "available=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::Skipping iOS build because ../silero-coreml is not available on this runner"
echo "available=false" >> "$GITHUB_OUTPUT"
fi
- name: flutter build ios --no-codesign
if: steps.silero-coreml.outputs.available == 'true'
working-directory: apps/chanora_flutter
run: flutter build ios --release --no-codesign
CHANGELOG.md
+11
View File
@@ -6,6 +6,17 @@ This project is expected to follow a Conventional Commits style workflow.
## [Unreleased]
### Added
- Apple CoreML-backed Silero VAD is now the preferred Apple voice
activity detector when the private sibling `silero-coreml` SwiftPM
package is available; WebRTC VAD remains the runtime fallback.
### Changed
- Apple platform floors are raised to iOS 16 and macOS 13 while the
CoreML VAD package is linked.
## [v1.0.0-rc.1] — MVP Public release candidate
This is the first release candidate for the MVP public release per
README.md
+12 -1
View File
@@ -46,13 +46,24 @@ Current platform policy:
| Platform | Baseline |
|---|---|
| iOS / iPadOS runtime target | iOS 13+ unless Flutter, plugin, audio, or product constraints require raising it |
| iOS / iPadOS runtime target | iOS 16+ while Apple CoreML Silero VAD is linked |
| macOS runtime target | macOS 13+ while Apple CoreML Silero VAD is linked |
| App Store Connect upload gate | Xcode 26+ with iOS 26 / iPadOS 26 SDK+ for upload on or after 2026-04-28 |
| Android runtime target | Android API 24+ unless Flutter, plugin, audio, or product constraints require raising it |
| Google Play target API | Target the Google Play-required API level on upload date |
The App Store / Play Store upload gates are release requirements. They are separate from local development and internal testing requirements.
Apple CoreML VAD development requires the private `silero-coreml` SwiftPM package checked out as a sibling of this repository, so the app checkout and package checkout share the same parent directory:
```text
workspace/
chanora/
silero-coreml/
```
The iOS and macOS Xcode projects reference that package via `../../../../silero-coreml` from their project files. GitHub CI skips the unsigned iOS build when the sibling package is unavailable, but local Apple builds need that checkout.
---
## Architecture Overview
apps/chanora_flutter/ios/Podfile
+1 -1
View File
@@ -1,5 +1,5 @@
# Uncomment this line to define a global platform for your project
platform :ios, '15.1'
platform :ios, '16.0'
# CocoaPods analytics sends network stats synchronously affecting flutter build latency.
ENV['COCOAPODS_DISABLE_STATS'] = 'true'
apps/chanora_flutter/ios/Podfile.lock
+2 -2
View File
@@ -55,7 +55,7 @@ EXTERNAL SOURCES:
SPEC CHECKSUMS:
audio_session: 9bb7f6c970f21241b19f5a3658097ae459681ba0
chanora_bridge: e1c7a6f9135400efec036d9df706b2b4b29b2cd5
chanora_bridge: 2ed7c2ba427fab135dd9eab66c507b09cfee113a
connectivity_plus: cb623214f4e1f6ef8fe7403d580fdad517d2f7dd
Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
flutter_foreground_task: a159d2c2173b33699ddb3e6c2a067045d7cebb89
@@ -65,6 +65,6 @@ SPEC CHECKSUMS:
shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb
url_launcher_ios: 7a95fa5b60cc718a708b8f2966718e93db0cef1b
PODFILE CHECKSUM: 50c5b575d74b9daff4bf33213abfd2c15d7f2e41
PODFILE CHECKSUM: e2123068539aeb66d53dc1612b383d13f489ede2
COCOAPODS: 1.16.2
apps/chanora_flutter/ios/Runner.xcodeproj/project.pbxproj
+30 -3
View File
@@ -14,6 +14,8 @@
3EF79A791760D95CE0F41CFF /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 63497078A621E2A73B102C46 /* Pods_RunnerTests.framework */; };
74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74858FAE1ED2DC5600515810 /* AppDelegate.swift */; };
7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */; };
8C5000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */; };
8C5000042DD0000000000001 /* SileroCoreML in Frameworks */ = {isa = PBXBuildFile; productRef = 8C5000032DD0000000000001 /* SileroCoreML */; };
97C146FC1CF9000F007C117D /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FA1CF9000F007C117D /* Main.storyboard */; };
97C146FE1CF9000F007C117D /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FD1CF9000F007C117D /* Assets.xcassets */; };
97C147011CF9000F007C117D /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FF1CF9000F007C117D /* LaunchScreen.storyboard */; };
@@ -57,6 +59,7 @@
74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = "Runner-Bridging-Header.h"; sourceTree = "<group>"; };
74858FAE1ED2DC5600515810 /* AppDelegate.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = "<group>"; };
7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SceneDelegate.swift; sourceTree = "<group>"; };
8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SileroCoreMLBridge.swift; sourceTree = "<group>"; };
7AFA3C8E1D35360C0083082E /* Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = Release.xcconfig; path = Flutter/Release.xcconfig; sourceTree = "<group>"; };
7E043103010958FC2C6CA47F /* Pods-Runner.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.debug.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.debug.xcconfig"; sourceTree = "<group>"; };
89E01DD0E6B92DA93A02E9D6 /* Pods-Runner.profile.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.profile.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.profile.xcconfig"; sourceTree = "<group>"; };
@@ -76,6 +79,7 @@
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
8C5000042DD0000000000001 /* SileroCoreML in Frameworks */,
1E3B5BCCA481234F14E64D44 /* Pods_Runner.framework in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
@@ -164,6 +168,7 @@
1498D2331E8E89220040F4C2 /* GeneratedPluginRegistrant.m */,
74858FAE1ED2DC5600515810 /* AppDelegate.swift */,
7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */,
8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */,
74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */,
1937FD83C5CC909094CDC137 /* PrivacyInfo.xcprivacy */,
);
@@ -210,6 +215,9 @@
dependencies = (
);
name = Runner;
packageProductDependencies = (
8C5000032DD0000000000001 /* SileroCoreML */,
);
productName = Runner;
productReference = 97C146EE1CF9000F007C117D /* Runner.app */;
productType = "com.apple.product-type.application";
@@ -243,6 +251,9 @@
Base,
);
mainGroup = 97C146E51CF9000F007C117D;
packageReferences = (
8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */,
);
productRefGroup = 97C146EF1CF9000F007C117D /* Products */;
projectDirPath = "";
projectRoot = "";
@@ -386,6 +397,7 @@
74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */,
1498D2341E8E89220040F4C2 /* GeneratedPluginRegistrant.m in Sources */,
7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */,
8C5000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
@@ -462,7 +474,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1;
IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = iphoneos;
SUPPORTED_PLATFORMS = iphoneos;
@@ -595,7 +607,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1;
IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = YES;
ONLY_ACTIVE_ARCH = YES;
SDKROOT = iphoneos;
@@ -646,7 +658,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1;
IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = iphoneos;
SUPPORTED_PLATFORMS = iphoneos;
@@ -742,6 +754,21 @@
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */ = {
isa = XCLocalSwiftPackageReference;
relativePath = ../../../../silero-coreml;
};
/* End XCLocalSwiftPackageReference section */
/* Begin XCSwiftPackageProductDependency section */
8C5000032DD0000000000001 /* SileroCoreML */ = {
isa = XCSwiftPackageProductDependency;
package = 8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */;
productName = SileroCoreML;
};
/* End XCSwiftPackageProductDependency section */
};
rootObject = 97C146E61CF9000F007C117D /* Project object */;
}
apps/chanora_flutter/ios/Runner/SileroCoreMLBridge.swift
+98
View File
@@ -0,0 +1,98 @@
import CoreML
import Foundation
import SileroCoreML
private final class ChanoraSileroVadBox {
let vad: SileroVAD
init() throws {
let configuration = MLModelConfiguration()
vad = try SileroVAD(configuration: configuration)
}
}
private let chanoraSileroErrorLock = NSLock()
private var chanoraSileroLastError = ""
private func setChanoraSileroLastError(_ message: String) {
chanoraSileroErrorLock.lock()
chanoraSileroLastError = message
chanoraSileroErrorLock.unlock()
}
@_cdecl("chanora_silero_vad_create")
public func chanoraSileroVadCreate() -> UnsafeMutableRawPointer? {
do {
let box = try ChanoraSileroVadBox()
return Unmanaged.passRetained(box).toOpaque()
} catch {
setChanoraSileroLastError(String(describing: error))
return nil
}
}
@_cdecl("chanora_silero_vad_destroy")
public func chanoraSileroVadDestroy(_ handle: UnsafeMutableRawPointer?) {
guard let handle else { return }
Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).release()
}
@_cdecl("chanora_silero_vad_reset")
public func chanoraSileroVadReset(_ handle: UnsafeMutableRawPointer?) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
box.vad.reset()
return 0
}
@_cdecl("chanora_silero_vad_process")
public func chanoraSileroVadProcess(
_ handle: UnsafeMutableRawPointer?,
_ samples: UnsafePointer<Float>?,
_ sampleCount: Int,
_ probabilityOut: UnsafeMutablePointer<Float>?
) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
guard let samples else {
setChanoraSileroLastError("SileroVAD samples pointer is null")
return -2
}
guard let probabilityOut else {
setChanoraSileroLastError("SileroVAD probability output pointer is null")
return -3
}
guard sampleCount == SileroVAD.chunkSize else {
setChanoraSileroLastError("SileroVAD expected \(SileroVAD.chunkSize) samples, got \(sampleCount)")
return -4
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
do {
let chunk = Array(UnsafeBufferPointer(start: samples, count: sampleCount))
probabilityOut.pointee = try box.vad.process(chunk)
return 0
} catch {
setChanoraSileroLastError(String(describing: error))
return -5
}
}
@_cdecl("chanora_silero_vad_last_error")
public func chanoraSileroVadLastError() -> UnsafeMutablePointer<CChar>? {
chanoraSileroErrorLock.lock()
let message = chanoraSileroLastError
chanoraSileroErrorLock.unlock()
return strdup(message)
}
@_cdecl("chanora_silero_vad_free_string")
public func chanoraSileroVadFreeString(_ string: UnsafeMutablePointer<CChar>?) {
guard let string else { return }
free(string)
}
apps/chanora_flutter/ios/chanora_bridge.podspec
+7 -7
View File
@@ -40,7 +40,7 @@ Pod::Spec.new do |s|
s.license = { :type => 'Apache-2.0 OR MIT', :text => 'See LICENSE-APACHE / LICENSE-MIT at the repo root' }
s.author = { 'EdisonJwa' => 'me@edison.network' }
s.source = { :path => '.' }
s.platform = :ios, '15.1'
s.platform = :ios, '16.0'
# Build the Rust bridge on `pod install`. The script runs under
# bash; we use `set -e` so any failure (cargo missing, target not
@@ -92,9 +92,9 @@ Pod::Spec.new do |s|
RUSTUP_HOME="$USER_HOME/.rustup" \\
RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\
RUSTC="$RUSTC_BIN" \\
IPHONEOS_DEPLOYMENT_TARGET=15.1 \\
IPHONEOS_DEPLOYMENT_TARGET=16.0 \\
CMAKE_POLICY_VERSION_MINIMUM=3.5 \\
CMAKE_OSX_DEPLOYMENT_TARGET=15.1 \\
CMAKE_OSX_DEPLOYMENT_TARGET=16.0 \\
"$CARGO_BIN" build --release --target aarch64-apple-ios -p chanora_bridge
if [ ! -f "$BRIDGE" ]; then
@@ -122,7 +122,7 @@ Pod::Spec.new do |s|
<key>CFBundleShortVersionString</key><string>1.0.0</string>
<key>CFBundleVersion</key><string>1</string>
<key>CFBundleSupportedPlatforms</key><array><string>iPhoneOS</string></array>
<key>MinimumOSVersion</key><string>15.1</string>
<key>MinimumOSVersion</key><string>16.0</string>
</dict>
</plist>
PLIST
@@ -200,9 +200,9 @@ PLIST
RUSTUP_HOME="$USER_HOME/.rustup" \\
RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\
RUSTC="$RUSTC_BIN" \\
IPHONEOS_DEPLOYMENT_TARGET=15.1 \\
IPHONEOS_DEPLOYMENT_TARGET=16.0 \\
CMAKE_POLICY_VERSION_MINIMUM=3.5 \\
CMAKE_OSX_DEPLOYMENT_TARGET=15.1 \\
CMAKE_OSX_DEPLOYMENT_TARGET=16.0 \\
"$CARGO_BIN" build --release --target "$RUST_TARGET" -p chanora_bridge
cd "$REPO_ROOT/apps/chanora_flutter/ios"
@@ -230,7 +230,7 @@ PLIST
<key>CFBundleShortVersionString</key><string>1.0.0</string>
<key>CFBundleVersion</key><string>1</string>
<key>CFBundleSupportedPlatforms</key><array><string>$SUPPORTED_PLATFORM</string></array>
<key>MinimumOSVersion</key><string>15.1</string>
<key>MinimumOSVersion</key><string>16.0</string>
</dict>
</plist>
PLIST
apps/chanora_flutter/macos/Podfile.lock
+23 -4
View File
@@ -7,33 +7,52 @@ PODS:
- FlutterMacOS (1.0.0)
- package_info_plus (0.0.1):
- FlutterMacOS
- share_plus (0.0.1):
- FlutterMacOS
- shared_preferences_foundation (0.0.1):
- Flutter
- FlutterMacOS
- url_launcher_macos (0.0.1):
- FlutterMacOS
DEPENDENCIES:
- audio_session (from `Flutter/ephemeral/.symlinks/plugins/audio_session/macos`)
- chanora_bridge (from `/Users/edison/chanora/apps/chanora_flutter/macos`)
- chanora_bridge (from `/Users/edison/dev/chanora/apps/chanora_flutter/macos`)
- connectivity_plus (from `Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos`)
- FlutterMacOS (from `Flutter/ephemeral`)
- package_info_plus (from `Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos`)
- share_plus (from `Flutter/ephemeral/.symlinks/plugins/share_plus/macos`)
- shared_preferences_foundation (from `Flutter/ephemeral/.symlinks/plugins/shared_preferences_foundation/darwin`)
- url_launcher_macos (from `Flutter/ephemeral/.symlinks/plugins/url_launcher_macos/macos`)
EXTERNAL SOURCES:
audio_session:
:path: Flutter/ephemeral/.symlinks/plugins/audio_session/macos
chanora_bridge:
:path: "/Users/edison/chanora/apps/chanora_flutter/macos"
:path: "/Users/edison/dev/chanora/apps/chanora_flutter/macos"
connectivity_plus:
:path: Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos
FlutterMacOS:
:path: Flutter/ephemeral
package_info_plus:
:path: Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos
share_plus:
:path: Flutter/ephemeral/.symlinks/plugins/share_plus/macos
shared_preferences_foundation:
:path: Flutter/ephemeral/.symlinks/plugins/shared_preferences_foundation/darwin
url_launcher_macos:
:path: Flutter/ephemeral/.symlinks/plugins/url_launcher_macos/macos
SPEC CHECKSUMS:
audio_session: eaca2512cf2b39212d724f35d11f46180ad3a33e
chanora_bridge: 1403e892a388d6bbb751d2d658e539d7a456fb1b
chanora_bridge: 4105993843b5421ee4ce72220a74c63f6fd99103
connectivity_plus: 4adf20a405e25b42b9c9f87feff8f4b6fde18a4e
FlutterMacOS: d0db08ddef1a9af05a5ec4b724367152bb0500b1
package_info_plus: f0052d280d17aa382b932f399edf32507174e870
share_plus: 510bf0af1a42cd602274b4629920c9649c52f4cc
shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb
url_launcher_macos: f87a979182d112f911de6820aefddaf56ee9fbfd
PODFILE CHECKSUM: d2e26b3cd926b9e407faa321206548300d262a08
PODFILE CHECKSUM: 99f0d126cab50f07c488b8550ebf033d2e8bcaeb
COCOAPODS: 1.16.2
apps/chanora_flutter/macos/Runner.xcodeproj/project.pbxproj
+39 -12
View File
@@ -27,6 +27,8 @@
33CC10F32044A3C60003C045 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F22044A3C60003C045 /* Assets.xcassets */; };
33CC10F62044A3C60003C045 /* MainMenu.xib in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F42044A3C60003C045 /* MainMenu.xib */; };
33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33CC11122044BFA00003C045 /* MainFlutterWindow.swift */; };
8C6000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */; };
8C6000042DD0000000000001 /* SileroCoreML in Frameworks */ = {isa = PBXBuildFile; productRef = 8C6000032DD0000000000001 /* SileroCoreML */; };
45F255D1DE0134185DB5423D /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 06E1AA7E1FB968C1D78DA8DE /* PrivacyInfo.xcprivacy */; };
9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = EFEADEEFAB54DFEAAD7A70E9 /* Pods_Runner.framework */; };
C2DC22E19FDCE26B9D79442E /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = FDE04BBB936AA14C2B58FA0E /* Pods_RunnerTests.framework */; };
@@ -74,6 +76,7 @@
33CC10F52044A3C60003C045 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.xib; name = Base; path = Base.lproj/MainMenu.xib; sourceTree = "<group>"; };
33CC10F72044A3C60003C045 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; name = Info.plist; path = Runner/Info.plist; sourceTree = "<group>"; };
33CC11122044BFA00003C045 /* MainFlutterWindow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MainFlutterWindow.swift; sourceTree = "<group>"; };
8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SileroCoreMLBridge.swift; sourceTree = "<group>"; };
33CEB47222A05771004F2AC0 /* Flutter-Debug.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Debug.xcconfig"; sourceTree = "<group>"; };
33CEB47422A05771004F2AC0 /* Flutter-Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Release.xcconfig"; sourceTree = "<group>"; };
33CEB47722A0578A004F2AC0 /* Flutter-Generated.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = "Flutter-Generated.xcconfig"; path = "ephemeral/Flutter-Generated.xcconfig"; sourceTree = "<group>"; };
@@ -105,6 +108,7 @@
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
8C6000042DD0000000000001 /* SileroCoreML in Frameworks */,
9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
@@ -179,6 +183,7 @@
children = (
33CC10F02044A3C60003C045 /* AppDelegate.swift */,
33CC11122044BFA00003C045 /* MainFlutterWindow.swift */,
8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */,
33E51913231747F40026EE4D /* DebugProfile.entitlements */,
33E51914231749380026EE4D /* Release.entitlements */,
33CC11242044D66E0003C045 /* Resources */,
@@ -250,6 +255,9 @@
33CC11202044C79F0003C045 /* PBXTargetDependency */,
);
name = Runner;
packageProductDependencies = (
8C6000032DD0000000000001 /* SileroCoreML */,
);
productName = Runner;
productReference = 33CC10ED2044A3C60003C045 /* chanora_flutter.app */;
productType = "com.apple.product-type.application";
@@ -293,6 +301,9 @@
Base,
);
mainGroup = 33CC10E42044A3C60003C045;
packageReferences = (
8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */,
);
productRefGroup = 33CC10EE2044A3C60003C045 /* Products */;
projectDirPath = "";
projectRoot = "";
@@ -442,6 +453,7 @@
33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */,
33CC10F12044A3C60003C045 /* AppDelegate.swift in Sources */,
335BBD1B22A9A15E00E9071D /* GeneratedPluginRegistrant.swift in Sources */,
8C6000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
@@ -480,7 +492,7 @@
BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)";
@@ -496,7 +508,7 @@
BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)";
@@ -512,7 +524,7 @@
BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)";
@@ -562,7 +574,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = macosx;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
@@ -601,7 +613,7 @@
"$(inherited)",
"@executable_path/../Frameworks",
);
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_VERSION = 5.0;
};
@@ -611,7 +623,7 @@
isa = XCBuildConfiguration;
buildSettings = {
CODE_SIGN_STYLE = Manual;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)";
};
name = Profile;
@@ -663,7 +675,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = YES;
ONLY_ACTIVE_ARCH = YES;
SDKROOT = macosx;
@@ -714,7 +726,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15;
MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = macosx;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
@@ -753,7 +765,7 @@
"$(inherited)",
"@executable_path/../Frameworks",
);
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_OPTIMIZATION_LEVEL = "-Onone";
SWIFT_VERSION = 5.0;
@@ -779,7 +791,7 @@
"$(inherited)",
"@executable_path/../Frameworks",
);
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_VERSION = 5.0;
};
@@ -789,7 +801,7 @@
isa = XCBuildConfiguration;
buildSettings = {
CODE_SIGN_STYLE = Manual;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)";
};
name = Debug;
@@ -798,7 +810,7 @@
isa = XCBuildConfiguration;
buildSettings = {
CODE_SIGN_STYLE = Automatic;
MACOSX_DEPLOYMENT_TARGET = 11.0;
MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)";
};
name = Release;
@@ -847,6 +859,21 @@
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */ = {
isa = XCLocalSwiftPackageReference;
relativePath = ../../../../silero-coreml;
};
/* End XCLocalSwiftPackageReference section */
/* Begin XCSwiftPackageProductDependency section */
8C6000032DD0000000000001 /* SileroCoreML */ = {
isa = XCSwiftPackageProductDependency;
package = 8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */;
productName = SileroCoreML;
};
/* End XCSwiftPackageProductDependency section */
};
rootObject = 33CC10E52044A3C60003C045 /* Project object */;
}
apps/chanora_flutter/macos/Runner/SileroCoreMLBridge.swift
+98
View File
@@ -0,0 +1,98 @@
import CoreML
import Foundation
import SileroCoreML
private final class ChanoraSileroVadBox {
let vad: SileroVAD
init() throws {
let configuration = MLModelConfiguration()
vad = try SileroVAD(configuration: configuration)
}
}
private let chanoraSileroErrorLock = NSLock()
private var chanoraSileroLastError = ""
private func setChanoraSileroLastError(_ message: String) {
chanoraSileroErrorLock.lock()
chanoraSileroLastError = message
chanoraSileroErrorLock.unlock()
}
@_cdecl("chanora_silero_vad_create")
public func chanoraSileroVadCreate() -> UnsafeMutableRawPointer? {
do {
let box = try ChanoraSileroVadBox()
return Unmanaged.passRetained(box).toOpaque()
} catch {
setChanoraSileroLastError(String(describing: error))
return nil
}
}
@_cdecl("chanora_silero_vad_destroy")
public func chanoraSileroVadDestroy(_ handle: UnsafeMutableRawPointer?) {
guard let handle else { return }
Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).release()
}
@_cdecl("chanora_silero_vad_reset")
public func chanoraSileroVadReset(_ handle: UnsafeMutableRawPointer?) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
box.vad.reset()
return 0
}
@_cdecl("chanora_silero_vad_process")
public func chanoraSileroVadProcess(
_ handle: UnsafeMutableRawPointer?,
_ samples: UnsafePointer<Float>?,
_ sampleCount: Int,
_ probabilityOut: UnsafeMutablePointer<Float>?
) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
guard let samples else {
setChanoraSileroLastError("SileroVAD samples pointer is null")
return -2
}
guard let probabilityOut else {
setChanoraSileroLastError("SileroVAD probability output pointer is null")
return -3
}
guard sampleCount == SileroVAD.chunkSize else {
setChanoraSileroLastError("SileroVAD expected \(SileroVAD.chunkSize) samples, got \(sampleCount)")
return -4
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
do {
let chunk = Array(UnsafeBufferPointer(start: samples, count: sampleCount))
probabilityOut.pointee = try box.vad.process(chunk)
return 0
} catch {
setChanoraSileroLastError(String(describing: error))
return -5
}
}
@_cdecl("chanora_silero_vad_last_error")
public func chanoraSileroVadLastError() -> UnsafeMutablePointer<CChar>? {
chanoraSileroErrorLock.lock()
let message = chanoraSileroLastError
chanoraSileroErrorLock.unlock()
return strdup(message)
}
@_cdecl("chanora_silero_vad_free_string")
public func chanoraSileroVadFreeString(_ string: UnsafeMutablePointer<CChar>?) {
guard let string else { return }
free(string)
}
apps/chanora_flutter/macos/macos_deployment_target.rb
+3 -4
View File
@@ -4,9 +4,8 @@
# This value is the macOS SDK floor against which `libchanora_bridge.dylib` is
# compiled — it is NOT the Flutter Runner app's deployment target (which lives
# in Runner.xcodeproj/project.pbxproj at the PBXNativeTarget level and is
# currently `11.0`). The bridge floor is intentionally broader (`10.15`) so the
# cdylib symbols remain link-compatible with any consumer >= 10.15; the Runner
# app's own minimum is independently set at the Xcode-target layer.
# currently `13.0`). The bridge floor matches the Runner app and Apple/CoreML
# package requirement so the bundled SwiftPM dependency can link consistently.
#
# Consumers:
# - apps/chanora_flutter/macos/Podfile (`platform :osx, ...`)
@@ -18,4 +17,4 @@
# To bump the floor, edit ONLY this file. Do not introduce any other literal
# occurrence of the floor string in this directory.
MACOS_BRIDGE_DEPLOYMENT_TARGET = '10.15'.freeze
MACOS_BRIDGE_DEPLOYMENT_TARGET = '13.0'.freeze
apps/chanora_flutter/pubspec.lock
+4 -4
View File
@@ -457,10 +457,10 @@ packages:
dependency: transitive
description:
name: meta
sha256: "23f08335362185a5ea2ad3a4e597f1375e78bce8a040df5c600c8d3552ef2394"
sha256: "1741988757a65eb6b36abe716829688cf01910bbf91c34354ff7ec1c3de2b349"
url: "https://pub.dev"
source: hosted
version: "1.17.0"
version: "1.18.0"
mime:
dependency: transitive
description:
@@ -790,10 +790,10 @@ packages:
dependency: transitive
description:
name: test_api
sha256: "8161c84903fd860b26bfdefb7963b3f0b68fee7adea0f59ef805ecca346f0c7a"
sha256: "949a932224383300f01be9221c39180316445ecb8e7547f70a41a35bf421fb9e"
url: "https://pub.dev"
source: hosted
version: "0.7.10"
version: "0.7.11"
typed_data:
dependency: transitive
description:
crates/chanora_audio/benches/realtime_capture.rs
+19 -17
View File
@@ -61,9 +61,26 @@ fn bench_capture_alloc_count(c: &mut Criterion) {
let blocks_final = stats_final.total_blocks;
let delta = blocks_final - blocks_warm;
// Sidecar file for §5 emit_baseline: criterion's own
// estimates.json carries the no-op closure timing, NOT the
// alloc count, so we write the canonical alloc-count value
// here and the emitter reads it directly. Write before the
// developer-facing assertion so the advisory CI can still report
// the regression after tolerating the bench process failure.
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR")
.map(std::path::PathBuf::from)
.or_else(|_| std::env::current_dir().map(|d| d.join("target")))
{
let path = dir.join("criterion").join("capture_alloc_count.sidecar");
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let _ = std::fs::write(&path, delta.to_string());
}
// Local-developer surface: hard-fail on any regression.
// The CI advisory comparator carries the same rule with a
// markdown 🔴 marker on regression instead of a panic.
// markdown marker on regression instead of a panic.
assert_eq!(
delta, 0,
"post-warmup heap allocation regression: {} blocks (SRS-219 clause a)",
@@ -76,27 +93,12 @@ fn bench_capture_alloc_count(c: &mut Criterion) {
// measurement is the delta computed above; criterion's
// function-time-mean is uninteresting for an alloc-count
// metric. The §5 emitter reads the `capture_alloc_count`
// metric value out of band via a sidecar file written below.
// metric value out of band via the sidecar file written above.
c.bench_function("capture_alloc_count", |b| {
b.iter(|| {
black_box(delta);
});
});
// Sidecar file for §5 emit_baseline: criterion's own
// estimates.json carries the no-op closure timing, NOT the
// alloc count, so we write the canonical alloc-count value
// here and the emitter reads it directly.
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR")
.map(std::path::PathBuf::from)
.or_else(|_| std::env::current_dir().map(|d| d.join("target")))
{
let path = dir.join("criterion").join("capture_alloc_count.sidecar");
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let _ = std::fs::write(&path, delta.to_string());
}
}
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
crates/chanora_audio/src/audio_processing.rs
+3 -9
View File
@@ -100,12 +100,6 @@ pub enum VadBackend {
Disabled,
}
#[cfg(target_os = "ios")]
fn default_vad_backend() -> VadBackend {
VadBackend::WebrtcVad
}
#[cfg(not(target_os = "ios"))]
fn default_vad_backend() -> VadBackend {
VadBackend::SileroOnnx
}
@@ -114,6 +108,9 @@ impl VadBackend {
/// Stable bridge/debug string.
pub fn as_str(self) -> &'static str {
match self {
#[cfg(any(target_os = "ios", target_os = "macos"))]
Self::SileroOnnx => "apple_coreml",
#[cfg(not(any(target_os = "ios", target_os = "macos")))]
Self::SileroOnnx => "silero_vad_onnx",
Self::WebrtcVad => "webrtc_vad",
Self::EnergyDebug => "energy_debug",
@@ -252,9 +249,6 @@ mod tests {
assert_eq!(config.aec, EffectOwner::Platform);
assert_eq!(config.ns, EffectOwner::Platform);
assert_eq!(config.agc, EffectOwner::Platform);
#[cfg(target_os = "ios")]
assert_eq!(config.vad_backend, VadBackend::WebrtcVad);
#[cfg(not(target_os = "ios"))]
assert_eq!(config.vad_backend, VadBackend::SileroOnnx);
}
crates/chanora_audio/src/engine.rs
-1
View File
@@ -312,7 +312,6 @@ pub struct AudioEngine {
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
#[cfg(any(target_os = "ios", target_os = "macos", target_os = "android"))]
mic_gain: f32,
#[cfg(any(target_os = "ios", target_os = "macos"))]
// Streams must be dropped to stop audio. Both are `!Send` because
// cpal's Stream isn't Send on some backends; we keep them in an
// Option wrapped by Mutex so stop() can move them out. On Linux
crates/chanora_audio/src/ios_raw_unit.rs
+42 -5
View File
@@ -116,6 +116,7 @@ mod inner {
mic_gain: f32,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
vad_detector: crate::vad::WebRtcFallbackVad,
silero_coreml_worker: Option<crate::vad::apple_coreml::AppleCoreMlVadWorker>,
current_vad_backend: crate::VadBackend,
capture_frame_seq: u64,
vad_state: crate::voice_activity::VoiceActivityStateMachine,
@@ -152,6 +153,7 @@ mod inner {
mic_gain: params.mic_gain,
voice_activity_selector: params.voice_activity_selector.clone(),
vad_detector: crate::vad::WebRtcFallbackVad::default(),
silero_coreml_worker: None,
current_vad_backend: crate::VadBackend::WebrtcVad,
capture_frame_seq: 0,
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
@@ -272,6 +274,7 @@ mod inner {
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
.unwrap_or(false);
if !voice_activity_mode {
self.silero_coreml_worker = None;
self.current_vad_backend = crate::VadBackend::Disabled;
self.fallback_warned_backend = None;
self.audio_processing_stats.set_vad_fallback_active(false);
@@ -297,9 +300,16 @@ mod inner {
self.current_vad_backend = vad_backend;
self.fallback_warned_backend = None;
if vad_backend == crate::VadBackend::SileroOnnx {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
self.silero_coreml_worker =
crate::vad::apple_coreml::AppleCoreMlVadWorker::try_new();
if self.silero_coreml_worker.is_none() {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
} else {
self.audio_processing_stats.set_vad_fallback_active(false);
}
} else {
self.silero_coreml_worker = None;
self.audio_processing_stats.set_vad_fallback_active(false);
}
self.vad_state.reset();
@@ -307,6 +317,7 @@ mod inner {
let (vad_probability, active) = if voice_activity_mode {
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
let capture_seq = self.capture_frame_seq;
let mut used_fallback_vad = false;
let vad = if vad_backend == crate::VadBackend::Disabled {
crate::vad::VadOutput {
@@ -314,9 +325,35 @@ mod inner {
speech: true,
}
} else if vad_backend == crate::VadBackend::SileroOnnx {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
if let Some(worker) = self.silero_coreml_worker.as_ref() {
let enqueued = worker.try_send(capture_seq, &frame);
if !worker.is_stale(capture_seq) {
let p = worker.latest_probability();
crate::vad::VadOutput {
probability: p,
speech: p >= 0.5,
}
} else if enqueued {
crate::vad::VadOutput {
probability: 0.0,
speech: false,
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else {
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
};
crates/chanora_audio/src/ios_voice_unit.rs
+39 -5
View File
@@ -138,6 +138,7 @@ struct IosCaptureState {
mic_gain: f32,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
vad_detector: crate::vad::WebRtcFallbackVad,
silero_coreml_worker: Option<crate::vad::apple_coreml::AppleCoreMlVadWorker>,
/// Last VAD backend we configured — used to detect backend changes.
current_vad_backend: crate::VadBackend,
fallback_warned_backend: Option<crate::VadBackend>,
@@ -177,6 +178,7 @@ impl IosCaptureState {
mic_gain: params.mic_gain,
voice_activity_selector: params.voice_activity_selector.clone(),
vad_detector: crate::vad::WebRtcFallbackVad::default(),
silero_coreml_worker: None,
current_vad_backend: crate::VadBackend::WebrtcVad,
fallback_warned_backend: None,
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
@@ -350,6 +352,7 @@ impl IosCaptureState {
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
.unwrap_or(false);
if !voice_activity_mode {
self.silero_coreml_worker = None;
self.current_vad_backend = crate::VadBackend::Disabled;
self.fallback_warned_backend = None;
self.audio_processing_stats.set_vad_fallback_active(false);
@@ -373,9 +376,16 @@ impl IosCaptureState {
self.current_vad_backend = vad_backend;
self.fallback_warned_backend = None;
if vad_backend == crate::VadBackend::SileroOnnx {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
self.silero_coreml_worker =
crate::vad::apple_coreml::AppleCoreMlVadWorker::try_new();
if self.silero_coreml_worker.is_none() {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
} else {
self.audio_processing_stats.set_vad_fallback_active(false);
}
} else {
self.silero_coreml_worker = None;
self.audio_processing_stats.set_vad_fallback_active(false);
}
// Reset VAD state machine timers on backend switch.
@@ -418,6 +428,7 @@ impl IosCaptureState {
// VAD: only evaluate while VoiceActivity mode is active.
let (vad_probability, gate_open) = if voice_activity_mode {
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
let capture_seq = self.capture_frame_seq;
let mut used_fallback_vad = false;
let vad = if vad_backend == crate::VadBackend::Disabled {
crate::vad::VadOutput {
@@ -425,9 +436,32 @@ impl IosCaptureState {
speech: true,
}
} else if vad_backend == crate::VadBackend::SileroOnnx {
used_fallback_vad = true;
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
if let Some(worker) = self.silero_coreml_worker.as_ref() {
let enqueued = worker.try_send(capture_seq, &frame);
if !worker.is_stale(capture_seq) {
let p = worker.latest_probability();
crate::vad::VadOutput {
probability: p,
speech: p >= 0.5,
}
} else if enqueued {
crate::vad::VadOutput {
probability: 0.0,
speech: false,
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
}
} else {
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
};
crates/chanora_audio/src/route_policy.rs
+9 -9
View File
@@ -30,7 +30,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::PlatformVoiceProcessing,
vad_backend: VadBackend::WebrtcVad,
vad_backend: VadBackend::SileroOnnx,
aec: EffectOwner::Platform,
// VPIO owns NS and AGC on the shipping default path (IOSP_002/003).
ns: EffectOwner::Platform,
@@ -43,7 +43,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::Noop,
vad_backend: VadBackend::WebrtcVad,
vad_backend: VadBackend::SileroOnnx,
// No AEC needed for wired headset (no acoustic echo path).
aec: EffectOwner::Off,
// Conservative NS/AGC: optional, not forced.
@@ -57,7 +57,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::PlatformVoiceProcessing,
vad_backend: VadBackend::WebrtcVad,
vad_backend: VadBackend::SileroOnnx,
// BT HFP manages its own AEC in the headset firmware.
aec: EffectOwner::Off,
ns: EffectOwner::Conservative,
@@ -87,7 +87,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::Noop,
vad_backend: VadBackend::WebrtcVad,
vad_backend: VadBackend::SileroOnnx,
// Safe fallback: AEC off until route is classified.
aec: EffectOwner::Off,
ns: EffectOwner::Off,
@@ -146,7 +146,7 @@ mod tests {
cfg.ios_mode,
IosVoiceProcessingMode::PlatformVoiceProcessing
);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad);
assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
}
#[test]
@@ -157,7 +157,7 @@ mod tests {
AudioBackend::PlatformVoiceProcessing
);
assert_eq!(cfg.aec, EffectOwner::Platform);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad);
assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
}
#[test]
@@ -165,14 +165,14 @@ mod tests {
let cfg = ios_route_policy(AudioRoute::WiredHeadset);
assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.processing_backend, AudioBackend::Noop);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad);
assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
}
#[test]
fn bluetooth_hfp_disables_app_aec() {
let cfg = ios_route_policy(AudioRoute::BluetoothHfp);
assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad);
assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
}
#[test]
@@ -187,7 +187,7 @@ mod tests {
fn unknown_route_safe_fallback_no_aec() {
let cfg = ios_route_policy(AudioRoute::Unknown);
assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad);
assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
}
#[test]
crates/chanora_audio/src/vad/apple_coreml.rs
+318
View File
@@ -0,0 +1,318 @@
//! Apple/CoreML Silero VAD bridge.
//!
//! The Swift Runner target exports a tiny C ABI around
//! `SileroCoreML.SileroVAD`. This Rust side resolves those symbols at
//! runtime, then runs inference on a background worker so realtime CoreAudio
//! callbacks only enqueue frames and read atomics.
use super::{VadOutput, VoiceActivityDetector};
use std::ffi::{c_char, c_void, CStr};
use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering};
use std::sync::Arc;
use std::thread::JoinHandle;
/// 16 kHz frame size required by `SileroCoreML.SileroVAD.process(_:)`.
pub const SILERO_COREML_FRAME_16K: usize = 512;
/// Maximum lag in 10 ms frames before the realtime callback falls back.
pub const SILERO_COREML_MAX_STALE_FRAMES: u64 = 3;
const SILERO_COREML_THRESHOLD: f32 = 0.5;
const RTLD_DEFAULT: *mut c_void = -2_isize as *mut c_void;
type CreateFn = unsafe extern "C" fn() -> *mut c_void;
type DestroyFn = unsafe extern "C" fn(*mut c_void);
type ResetFn = unsafe extern "C" fn(*mut c_void) -> i32;
type ProcessFn = unsafe extern "C" fn(*mut c_void, *const f32, usize, *mut f32) -> i32;
type LastErrorFn = unsafe extern "C" fn() -> *mut c_char;
type FreeStringFn = unsafe extern "C" fn(*mut c_char);
#[allow(improper_ctypes)]
extern "C" {
fn dlsym(handle: *mut c_void, symbol: *const c_char) -> *mut c_void;
}
#[derive(Clone, Copy)]
struct AppleSileroSymbols {
create: CreateFn,
destroy: DestroyFn,
reset: ResetFn,
process: ProcessFn,
last_error: LastErrorFn,
free_string: FreeStringFn,
}
impl AppleSileroSymbols {
fn resolve() -> Option<Self> {
unsafe {
Some(Self {
create: std::mem::transmute::<*mut c_void, CreateFn>(resolve_symbol(
b"chanora_silero_vad_create\0",
)?),
destroy: std::mem::transmute::<*mut c_void, DestroyFn>(resolve_symbol(
b"chanora_silero_vad_destroy\0",
)?),
reset: std::mem::transmute::<*mut c_void, ResetFn>(resolve_symbol(
b"chanora_silero_vad_reset\0",
)?),
process: std::mem::transmute::<*mut c_void, ProcessFn>(resolve_symbol(
b"chanora_silero_vad_process\0",
)?),
last_error: std::mem::transmute::<*mut c_void, LastErrorFn>(resolve_symbol(
b"chanora_silero_vad_last_error\0",
)?),
free_string: std::mem::transmute::<*mut c_void, FreeStringFn>(resolve_symbol(
b"chanora_silero_vad_free_string\0",
)?),
})
}
}
fn last_error_message(&self) -> String {
unsafe {
let ptr = (self.last_error)();
if ptr.is_null() {
return "unknown SileroCoreML bridge error".to_string();
}
let message = CStr::from_ptr(ptr).to_string_lossy().into_owned();
(self.free_string)(ptr);
message
}
}
}
unsafe fn resolve_symbol(name: &'static [u8]) -> Option<*mut c_void> {
let ptr = dlsym(RTLD_DEFAULT, name.as_ptr().cast());
if ptr.is_null() {
None
} else {
Some(ptr)
}
}
/// 16 kHz detector backed by Swift `SileroCoreML.SileroVAD`.
pub struct AppleCoreMlVad {
handle: *mut c_void,
symbols: AppleSileroSymbols,
accum: Vec<f32>,
frame_scratch: Box<[f32; SILERO_COREML_FRAME_16K]>,
last_probability: f32,
}
impl AppleCoreMlVad {
/// Create a detector when the Swift Runner bridge symbols are linked.
pub fn try_new() -> Option<Self> {
let symbols = AppleSileroSymbols::resolve()?;
Self::try_new_with_symbols(symbols)
}
fn try_new_with_symbols(symbols: AppleSileroSymbols) -> Option<Self> {
let handle = unsafe { (symbols.create)() };
if handle.is_null() {
tracing::warn!(
target: "chanora_audio",
error = %symbols.last_error_message(),
"AppleCoreMlVad: Swift SileroCoreML bridge unavailable; falling back to WebRtcFallbackVad"
);
return None;
}
tracing::info!(
target: "chanora_audio",
backend = "apple_coreml",
model = "silero_vad",
model_version = "6.2.1",
model_resource = "SileroVADModel",
model_artifact = "mlmodelc_or_mlpackage",
sample_rate_hz = 16_000,
chunk_size = SILERO_COREML_FRAME_16K,
threshold = SILERO_COREML_THRESHOLD,
"AppleCoreMlVad: using Apple CoreML Silero VAD"
);
Some(Self {
handle,
symbols,
accum: Vec::with_capacity(SILERO_COREML_FRAME_16K),
frame_scratch: Box::new([0.0; SILERO_COREML_FRAME_16K]),
last_probability: 0.0,
})
}
/// Reset accumulated samples, last probability, and the Swift VAD stream.
pub fn reset_state(&mut self) {
self.accum.clear();
self.last_probability = 0.0;
let rc = unsafe { (self.symbols.reset)(self.handle) };
if rc != 0 {
tracing::warn!(
target: "chanora_audio",
error = %self.symbols.last_error_message(),
"AppleCoreMlVad: reset failed"
);
}
}
fn calc_level(&mut self) -> f32 {
let mut probability = self.last_probability;
let rc = unsafe {
(self.symbols.process)(
self.handle,
self.frame_scratch.as_ptr(),
self.frame_scratch.len(),
&mut probability,
)
};
if rc == 0 {
self.last_probability = probability.clamp(0.0, 1.0);
} else {
tracing::warn!(
target: "chanora_audio",
error = %self.symbols.last_error_message(),
"AppleCoreMlVad: inference failed; holding last probability"
);
}
self.last_probability
}
}
impl VoiceActivityDetector for AppleCoreMlVad {
fn process_10ms(&mut self, samples: &[f32]) -> VadOutput {
debug_assert_eq!(
samples.len(),
super::resampler::OUTPUT_FRAME_10MS,
"AppleCoreMlVad expects 160 samples (16 kHz 10 ms), got {}",
samples.len()
);
self.accum.extend_from_slice(samples);
if self.accum.len() >= SILERO_COREML_FRAME_16K {
self.frame_scratch
.copy_from_slice(&self.accum[..SILERO_COREML_FRAME_16K]);
self.calc_level();
drop(self.accum.drain(..SILERO_COREML_FRAME_16K));
}
VadOutput {
probability: self.last_probability,
speech: self.last_probability >= SILERO_COREML_THRESHOLD,
}
}
}
impl Drop for AppleCoreMlVad {
fn drop(&mut self) {
unsafe { (self.symbols.destroy)(self.handle) };
}
}
// SAFETY: the opaque Swift object is owned by this detector and only used by
// the worker thread after construction. It is never shared concurrently.
unsafe impl Send for AppleCoreMlVad {}
struct SileroFrameMessage {
seq: u64,
frame: [f32; super::resampler::INPUT_FRAME_10MS],
}
/// Background Apple/CoreML Silero worker.
pub struct AppleCoreMlVadWorker {
tx: Option<std::sync::mpsc::SyncSender<SileroFrameMessage>>,
latest_probability: Arc<AtomicU32>,
latest_processed_seq: Arc<AtomicU64>,
alive: Arc<AtomicBool>,
handle: Option<JoinHandle<()>>,
}
impl AppleCoreMlVadWorker {
/// Start the background CoreML worker when the Swift bridge is available.
pub fn try_new() -> Option<Self> {
let symbols = AppleSileroSymbols::resolve()?;
let latest_probability = Arc::new(AtomicU32::new(0.0_f32.to_bits()));
let latest_processed_seq = Arc::new(AtomicU64::new(u64::MAX));
let alive = Arc::new(AtomicBool::new(true));
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(64);
let latest_probability_for_thread = latest_probability.clone();
let latest_processed_seq_for_thread = latest_processed_seq.clone();
let alive_for_thread = alive.clone();
let handle = std::thread::Builder::new()
.name("chanora-apple-silero-vad".to_string())
.spawn(move || {
let Some(vad) = AppleCoreMlVad::try_new_with_symbols(symbols) else {
return;
};
let mut vad = super::Resampled16kHzVad::new(vad);
while alive_for_thread.load(Ordering::Relaxed) {
let message = match rx.recv() {
Ok(message) => message,
Err(_) => break,
};
let output = vad.process_10ms(&message.frame);
latest_probability_for_thread.store(
output.probability.clamp(0.0, 1.0).to_bits(),
Ordering::Relaxed,
);
latest_processed_seq_for_thread.store(message.seq, Ordering::Relaxed);
}
})
.ok()?;
Some(Self {
tx: Some(tx),
latest_probability,
latest_processed_seq,
alive,
handle: Some(handle),
})
}
/// Enqueue one 48 kHz 10 ms frame without blocking the caller.
pub fn try_send(&self, seq: u64, frame: &[f32; super::resampler::INPUT_FRAME_10MS]) -> bool {
let Some(tx) = &self.tx else {
return false;
};
tx.try_send(SileroFrameMessage { seq, frame: *frame })
.is_ok()
}
/// Return the latest probability published by the worker thread.
pub fn latest_probability(&self) -> f32 {
f32::from_bits(self.latest_probability.load(Ordering::Relaxed))
}
/// Return true until the worker has produced a recent probability.
pub fn is_stale(&self, capture_seq: u64) -> bool {
let latest = self.latest_processed_seq.load(Ordering::Relaxed);
latest == u64::MAX || capture_seq.saturating_sub(latest) > SILERO_COREML_MAX_STALE_FRAMES
}
}
impl Drop for AppleCoreMlVadWorker {
fn drop(&mut self) {
self.alive.store(false, Ordering::Relaxed);
let _ = self.tx.take();
// Drop can run from the realtime audio callback during backend changes;
// never join here. Closing tx lets the worker exit and dropping the
// handle detaches the thread without blocking the callback.
let _ = self.handle.take();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn coreml_constants_match_silero_package_contract() {
assert_eq!(SILERO_COREML_FRAME_16K, 512);
assert_eq!(SILERO_COREML_MAX_STALE_FRAMES, 3);
}
#[test]
fn worker_is_unavailable_without_swift_bridge_symbols_on_host_tests() {
assert!(AppleCoreMlVadWorker::try_new().is_none());
}
#[test]
fn vad_is_unavailable_without_swift_bridge_symbols_on_host_tests() {
assert!(AppleCoreMlVad::try_new().is_none());
}
}
crates/chanora_audio/src/vad/mod.rs
+6 -4
View File
@@ -1,10 +1,12 @@
//! Voice activity detection backends and helpers.
//!
//! iOS capture feeds VoiceProcessingIO-processed microphone frames into
//! this module and uses the realtime-safe WebRTC fallback. Other
//! platforms may use a model-backed detector when available so
//! VoiceActivity mode never collapses back to Continuous transmit.
//! Apple capture feeds VoiceProcessingIO/CoreAudio-processed microphone
//! frames into this module and prefers Apple CoreML Silero VAD when the
//! Swift bridge is linked. WebRTC VAD remains the realtime-safe fallback;
//! non-Apple platforms may use ONNX-backed Silero when available.
#[cfg(any(target_os = "ios", target_os = "macos"))]
pub mod apple_coreml;
pub mod resampler;
#[cfg(not(target_os = "ios"))]
pub mod silero_onnx;
deny.toml
+7 -1
View File
@@ -78,7 +78,13 @@ wildcards = "warn"
highlight = "all"
# Crates we never want, regardless of license. Empty by default.
deny = []
skip = []
skip = [
# `tracing-android` still depends on android_log-sys 0.2.x while
# flutter_rust_bridge's `android_logger` uses 0.3.x. Both are
# Android-only logcat bindings; keep this targeted until upstreams
# converge.
{ name = "android_log-sys", version = "0.2.0" },
]
skip-tree = []
# ---------- Sources ----------
docs/security/flutter-license-inventory.md
+4 -4
View File
@@ -74,7 +74,7 @@ terms.
| `logging` | 1.3.0 | hosted | yes |
| `matcher` | 0.12.19 | hosted | yes |
| `material_color_utilities` | 0.13.0 | hosted | yes |
| `meta` | 1.17.0 | hosted | yes |
| `meta` | 1.18.0 | hosted | yes |
| `mime` | 2.0.0 | hosted | yes |
| `native_toolchain_c` | 0.17.6 | hosted | yes |
| `nm` | 0.5.0 | hosted | yes |
@@ -116,7 +116,7 @@ terms.
| `stream_transform` | 2.1.1 | hosted | yes |
| `string_scanner` | 1.4.1 | hosted | yes |
| `term_glyph` | 1.2.2 | hosted | yes |
| `test_api` | 0.7.10 | hosted | yes |
| `test_api` | 0.7.11 | hosted | yes |
| `typed_data` | 1.4.0 | hosted | yes |
| `url_launcher` | 6.3.2 | hosted | yes |
| `url_launcher_android` | 6.3.30 | hosted | yes |
@@ -2813,7 +2813,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
limitations under the License.
```
### meta 1.17.0
### meta 1.18.0
```
Copyright 2016, the Dart project authors.
@@ -4641,7 +4641,7 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```
### test_api 0.7.10
### test_api 0.7.11
```
Copyright 2018, the Dart project authors.