Merge pull request #13 from EdisonJwa/feat/apple-coreml-vad

Add Apple CoreML Silero VAD
This commit is contained in:
Edison Jwa
2026-06-02 20:14:17 +09:00
committed by GitHub
24 changed files with 798 additions and 100 deletions
+9 -5
View File
@@ -1,7 +1,7 @@
name: bench-advisory name: bench-advisory
# SDD-120 §6 / SRS-218 — advisory-only realtime-audio bench workflow. # SDD-120 §6 / SRS-218 — advisory-only realtime-audio bench workflow.
# Runs the criterion bench harness on PR + push events, compares the # Runs the criterion bench harness on PR + tag events, compares the
# current results against the SAD-089 baseline JSON resolved at the # current results against the SAD-089 baseline JSON resolved at the
# merge-base, and renders a markdown report posted (or updated) as a # merge-base, and renders a markdown report posted (or updated) as a
# single sticky PR comment. The job status is ALWAYS success — this # single sticky PR comment. The job status is ALWAYS success — this
@@ -11,7 +11,7 @@ on:
pull_request: pull_request:
types: [opened, synchronize, reopened] types: [opened, synchronize, reopened]
push: push:
branches: [product/scaffold-v0] tags: ["**"]
permissions: permissions:
pull-requests: write pull-requests: write
@@ -25,20 +25,24 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- name: System deps (cpal / Opus) - name: System deps (cpal / Opus / SDL2)
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install -y \ sudo apt-get install -y \
libasound2-dev libpulse-dev pkg-config \ libasound2-dev libpulse-dev pkg-config \
libdbus-1-dev \
libsdl2-dev \
libopus-dev libopus-dev
- uses: dtolnay/rust-toolchain@stable - uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
- name: Run benchmarks - name: Run benchmarks
run: | run: |
cargo bench -p chanora_audio \ if ! cargo bench -p chanora_audio \
--bench realtime_capture \ --bench realtime_capture \
--bench opus_codec \ --bench opus_codec \
--bench resampler --bench resampler; then
echo "::warning::Benchmark harness failed; continuing advisory workflow per SRS-218 clause 4"
fi
- name: Emit current baseline JSON - name: Emit current baseline JSON
run: cargo run --example emit_baseline -p chanora_audio run: cargo run --example emit_baseline -p chanora_audio
- name: Resolve merge-base baseline - name: Resolve merge-base baseline
+14 -2
View File
@@ -2,7 +2,7 @@ name: ci
on: on:
push: push:
branches: ["**"] tags: ["**"]
pull_request: pull_request:
jobs: jobs:
@@ -11,11 +11,13 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: System deps (cpal / Opus / SQLite) - name: System deps (cpal / Opus / SQLite / SDL2)
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install -y \ sudo apt-get install -y \
libasound2-dev libpulse-dev pkg-config \ libasound2-dev libpulse-dev pkg-config \
libdbus-1-dev \
libsdl2-dev \
libopus-dev libopus-dev
- uses: dtolnay/rust-toolchain@stable - uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
@@ -119,6 +121,16 @@ jobs:
- name: flutter pub get - name: flutter pub get
working-directory: apps/chanora_flutter working-directory: apps/chanora_flutter
run: flutter pub get run: flutter pub get
- name: Check local SileroCoreML package
id: silero-coreml
run: |
if [ -d ../silero-coreml ]; then
echo "available=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::Skipping iOS build because ../silero-coreml is not available on this runner"
echo "available=false" >> "$GITHUB_OUTPUT"
fi
- name: flutter build ios --no-codesign - name: flutter build ios --no-codesign
if: steps.silero-coreml.outputs.available == 'true'
working-directory: apps/chanora_flutter working-directory: apps/chanora_flutter
run: flutter build ios --release --no-codesign run: flutter build ios --release --no-codesign
+11
View File
@@ -6,6 +6,17 @@ This project is expected to follow a Conventional Commits style workflow.
## [Unreleased] ## [Unreleased]
### Added
- Apple CoreML-backed Silero VAD is now the preferred Apple voice
activity detector when the private sibling `silero-coreml` SwiftPM
package is available; WebRTC VAD remains the runtime fallback.
### Changed
- Apple platform floors are raised to iOS 16 and macOS 13 while the
CoreML VAD package is linked.
## [v1.0.0-rc.1] — MVP Public release candidate ## [v1.0.0-rc.1] — MVP Public release candidate
This is the first release candidate for the MVP public release per This is the first release candidate for the MVP public release per
+12 -1
View File
@@ -46,13 +46,24 @@ Current platform policy:
| Platform | Baseline | | Platform | Baseline |
|---|---| |---|---|
| iOS / iPadOS runtime target | iOS 13+ unless Flutter, plugin, audio, or product constraints require raising it | | iOS / iPadOS runtime target | iOS 16+ while Apple CoreML Silero VAD is linked |
| macOS runtime target | macOS 13+ while Apple CoreML Silero VAD is linked |
| App Store Connect upload gate | Xcode 26+ with iOS 26 / iPadOS 26 SDK+ for upload on or after 2026-04-28 | | App Store Connect upload gate | Xcode 26+ with iOS 26 / iPadOS 26 SDK+ for upload on or after 2026-04-28 |
| Android runtime target | Android API 24+ unless Flutter, plugin, audio, or product constraints require raising it | | Android runtime target | Android API 24+ unless Flutter, plugin, audio, or product constraints require raising it |
| Google Play target API | Target the Google Play-required API level on upload date | | Google Play target API | Target the Google Play-required API level on upload date |
The App Store / Play Store upload gates are release requirements. They are separate from local development and internal testing requirements. The App Store / Play Store upload gates are release requirements. They are separate from local development and internal testing requirements.
Apple CoreML VAD development requires the private `silero-coreml` SwiftPM package checked out as a sibling of this repository, so the app checkout and package checkout share the same parent directory:
```text
workspace/
chanora/
silero-coreml/
```
The iOS and macOS Xcode projects reference that package via `../../../../silero-coreml` from their project files. GitHub CI skips the unsigned iOS build when the sibling package is unavailable, but local Apple builds need that checkout.
--- ---
## Architecture Overview ## Architecture Overview
+1 -1
View File
@@ -1,5 +1,5 @@
# Uncomment this line to define a global platform for your project # Uncomment this line to define a global platform for your project
platform :ios, '15.1' platform :ios, '16.0'
# CocoaPods analytics sends network stats synchronously affecting flutter build latency. # CocoaPods analytics sends network stats synchronously affecting flutter build latency.
ENV['COCOAPODS_DISABLE_STATS'] = 'true' ENV['COCOAPODS_DISABLE_STATS'] = 'true'
+2 -2
View File
@@ -55,7 +55,7 @@ EXTERNAL SOURCES:
SPEC CHECKSUMS: SPEC CHECKSUMS:
audio_session: 9bb7f6c970f21241b19f5a3658097ae459681ba0 audio_session: 9bb7f6c970f21241b19f5a3658097ae459681ba0
chanora_bridge: e1c7a6f9135400efec036d9df706b2b4b29b2cd5 chanora_bridge: 2ed7c2ba427fab135dd9eab66c507b09cfee113a
connectivity_plus: cb623214f4e1f6ef8fe7403d580fdad517d2f7dd connectivity_plus: cb623214f4e1f6ef8fe7403d580fdad517d2f7dd
Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467 Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
flutter_foreground_task: a159d2c2173b33699ddb3e6c2a067045d7cebb89 flutter_foreground_task: a159d2c2173b33699ddb3e6c2a067045d7cebb89
@@ -65,6 +65,6 @@ SPEC CHECKSUMS:
shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb
url_launcher_ios: 7a95fa5b60cc718a708b8f2966718e93db0cef1b url_launcher_ios: 7a95fa5b60cc718a708b8f2966718e93db0cef1b
PODFILE CHECKSUM: 50c5b575d74b9daff4bf33213abfd2c15d7f2e41 PODFILE CHECKSUM: e2123068539aeb66d53dc1612b383d13f489ede2
COCOAPODS: 1.16.2 COCOAPODS: 1.16.2
@@ -14,6 +14,8 @@
3EF79A791760D95CE0F41CFF /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 63497078A621E2A73B102C46 /* Pods_RunnerTests.framework */; }; 3EF79A791760D95CE0F41CFF /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 63497078A621E2A73B102C46 /* Pods_RunnerTests.framework */; };
74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74858FAE1ED2DC5600515810 /* AppDelegate.swift */; }; 74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74858FAE1ED2DC5600515810 /* AppDelegate.swift */; };
7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */; }; 7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */; };
8C5000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */; };
8C5000042DD0000000000001 /* SileroCoreML in Frameworks */ = {isa = PBXBuildFile; productRef = 8C5000032DD0000000000001 /* SileroCoreML */; };
97C146FC1CF9000F007C117D /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FA1CF9000F007C117D /* Main.storyboard */; }; 97C146FC1CF9000F007C117D /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FA1CF9000F007C117D /* Main.storyboard */; };
97C146FE1CF9000F007C117D /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FD1CF9000F007C117D /* Assets.xcassets */; }; 97C146FE1CF9000F007C117D /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FD1CF9000F007C117D /* Assets.xcassets */; };
97C147011CF9000F007C117D /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FF1CF9000F007C117D /* LaunchScreen.storyboard */; }; 97C147011CF9000F007C117D /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 97C146FF1CF9000F007C117D /* LaunchScreen.storyboard */; };
@@ -57,6 +59,7 @@
74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = "Runner-Bridging-Header.h"; sourceTree = "<group>"; }; 74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = "Runner-Bridging-Header.h"; sourceTree = "<group>"; };
74858FAE1ED2DC5600515810 /* AppDelegate.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = "<group>"; }; 74858FAE1ED2DC5600515810 /* AppDelegate.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = "<group>"; };
7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SceneDelegate.swift; sourceTree = "<group>"; }; 7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SceneDelegate.swift; sourceTree = "<group>"; };
8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SileroCoreMLBridge.swift; sourceTree = "<group>"; };
7AFA3C8E1D35360C0083082E /* Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = Release.xcconfig; path = Flutter/Release.xcconfig; sourceTree = "<group>"; }; 7AFA3C8E1D35360C0083082E /* Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = Release.xcconfig; path = Flutter/Release.xcconfig; sourceTree = "<group>"; };
7E043103010958FC2C6CA47F /* Pods-Runner.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.debug.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.debug.xcconfig"; sourceTree = "<group>"; }; 7E043103010958FC2C6CA47F /* Pods-Runner.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.debug.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.debug.xcconfig"; sourceTree = "<group>"; };
89E01DD0E6B92DA93A02E9D6 /* Pods-Runner.profile.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.profile.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.profile.xcconfig"; sourceTree = "<group>"; }; 89E01DD0E6B92DA93A02E9D6 /* Pods-Runner.profile.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.profile.xcconfig"; path = "Target Support Files/Pods-Runner/Pods-Runner.profile.xcconfig"; sourceTree = "<group>"; };
@@ -76,6 +79,7 @@
isa = PBXFrameworksBuildPhase; isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647; buildActionMask = 2147483647;
files = ( files = (
8C5000042DD0000000000001 /* SileroCoreML in Frameworks */,
1E3B5BCCA481234F14E64D44 /* Pods_Runner.framework in Frameworks */, 1E3B5BCCA481234F14E64D44 /* Pods_Runner.framework in Frameworks */,
); );
runOnlyForDeploymentPostprocessing = 0; runOnlyForDeploymentPostprocessing = 0;
@@ -164,6 +168,7 @@
1498D2331E8E89220040F4C2 /* GeneratedPluginRegistrant.m */, 1498D2331E8E89220040F4C2 /* GeneratedPluginRegistrant.m */,
74858FAE1ED2DC5600515810 /* AppDelegate.swift */, 74858FAE1ED2DC5600515810 /* AppDelegate.swift */,
7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */, 7884E8672EC3CC0400C636F2 /* SceneDelegate.swift */,
8C5000002DD0000000000001 /* SileroCoreMLBridge.swift */,
74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */, 74858FAD1ED2DC5600515810 /* Runner-Bridging-Header.h */,
1937FD83C5CC909094CDC137 /* PrivacyInfo.xcprivacy */, 1937FD83C5CC909094CDC137 /* PrivacyInfo.xcprivacy */,
); );
@@ -210,6 +215,9 @@
dependencies = ( dependencies = (
); );
name = Runner; name = Runner;
packageProductDependencies = (
8C5000032DD0000000000001 /* SileroCoreML */,
);
productName = Runner; productName = Runner;
productReference = 97C146EE1CF9000F007C117D /* Runner.app */; productReference = 97C146EE1CF9000F007C117D /* Runner.app */;
productType = "com.apple.product-type.application"; productType = "com.apple.product-type.application";
@@ -243,6 +251,9 @@
Base, Base,
); );
mainGroup = 97C146E51CF9000F007C117D; mainGroup = 97C146E51CF9000F007C117D;
packageReferences = (
8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */,
);
productRefGroup = 97C146EF1CF9000F007C117D /* Products */; productRefGroup = 97C146EF1CF9000F007C117D /* Products */;
projectDirPath = ""; projectDirPath = "";
projectRoot = ""; projectRoot = "";
@@ -386,6 +397,7 @@
74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */, 74858FAF1ED2DC5600515810 /* AppDelegate.swift in Sources */,
1498D2341E8E89220040F4C2 /* GeneratedPluginRegistrant.m in Sources */, 1498D2341E8E89220040F4C2 /* GeneratedPluginRegistrant.m in Sources */,
7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */, 7884E8682EC3CC0700C636F2 /* SceneDelegate.swift in Sources */,
8C5000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */,
); );
runOnlyForDeploymentPostprocessing = 0; runOnlyForDeploymentPostprocessing = 0;
}; };
@@ -462,7 +474,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1; IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = NO; MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = iphoneos; SDKROOT = iphoneos;
SUPPORTED_PLATFORMS = iphoneos; SUPPORTED_PLATFORMS = iphoneos;
@@ -595,7 +607,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1; IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = YES; MTL_ENABLE_DEBUG_INFO = YES;
ONLY_ACTIVE_ARCH = YES; ONLY_ACTIVE_ARCH = YES;
SDKROOT = iphoneos; SDKROOT = iphoneos;
@@ -646,7 +658,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 15.1; IPHONEOS_DEPLOYMENT_TARGET = 16.0;
MTL_ENABLE_DEBUG_INFO = NO; MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = iphoneos; SDKROOT = iphoneos;
SUPPORTED_PLATFORMS = iphoneos; SUPPORTED_PLATFORMS = iphoneos;
@@ -742,6 +754,21 @@
defaultConfigurationName = Release; defaultConfigurationName = Release;
}; };
/* End XCConfigurationList section */ /* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */ = {
isa = XCLocalSwiftPackageReference;
relativePath = ../../../../silero-coreml;
};
/* End XCLocalSwiftPackageReference section */
/* Begin XCSwiftPackageProductDependency section */
8C5000032DD0000000000001 /* SileroCoreML */ = {
isa = XCSwiftPackageProductDependency;
package = 8C5000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */;
productName = SileroCoreML;
};
/* End XCSwiftPackageProductDependency section */
}; };
rootObject = 97C146E61CF9000F007C117D /* Project object */; rootObject = 97C146E61CF9000F007C117D /* Project object */;
} }
@@ -0,0 +1,98 @@
import CoreML
import Foundation
import SileroCoreML
private final class ChanoraSileroVadBox {
let vad: SileroVAD
init() throws {
let configuration = MLModelConfiguration()
vad = try SileroVAD(configuration: configuration)
}
}
private let chanoraSileroErrorLock = NSLock()
private var chanoraSileroLastError = ""
private func setChanoraSileroLastError(_ message: String) {
chanoraSileroErrorLock.lock()
chanoraSileroLastError = message
chanoraSileroErrorLock.unlock()
}
@_cdecl("chanora_silero_vad_create")
public func chanoraSileroVadCreate() -> UnsafeMutableRawPointer? {
do {
let box = try ChanoraSileroVadBox()
return Unmanaged.passRetained(box).toOpaque()
} catch {
setChanoraSileroLastError(String(describing: error))
return nil
}
}
@_cdecl("chanora_silero_vad_destroy")
public func chanoraSileroVadDestroy(_ handle: UnsafeMutableRawPointer?) {
guard let handle else { return }
Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).release()
}
@_cdecl("chanora_silero_vad_reset")
public func chanoraSileroVadReset(_ handle: UnsafeMutableRawPointer?) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
box.vad.reset()
return 0
}
@_cdecl("chanora_silero_vad_process")
public func chanoraSileroVadProcess(
_ handle: UnsafeMutableRawPointer?,
_ samples: UnsafePointer<Float>?,
_ sampleCount: Int,
_ probabilityOut: UnsafeMutablePointer<Float>?
) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
guard let samples else {
setChanoraSileroLastError("SileroVAD samples pointer is null")
return -2
}
guard let probabilityOut else {
setChanoraSileroLastError("SileroVAD probability output pointer is null")
return -3
}
guard sampleCount == SileroVAD.chunkSize else {
setChanoraSileroLastError("SileroVAD expected \(SileroVAD.chunkSize) samples, got \(sampleCount)")
return -4
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
do {
let chunk = Array(UnsafeBufferPointer(start: samples, count: sampleCount))
probabilityOut.pointee = try box.vad.process(chunk)
return 0
} catch {
setChanoraSileroLastError(String(describing: error))
return -5
}
}
@_cdecl("chanora_silero_vad_last_error")
public func chanoraSileroVadLastError() -> UnsafeMutablePointer<CChar>? {
chanoraSileroErrorLock.lock()
let message = chanoraSileroLastError
chanoraSileroErrorLock.unlock()
return strdup(message)
}
@_cdecl("chanora_silero_vad_free_string")
public func chanoraSileroVadFreeString(_ string: UnsafeMutablePointer<CChar>?) {
guard let string else { return }
free(string)
}
@@ -40,7 +40,7 @@ Pod::Spec.new do |s|
s.license = { :type => 'Apache-2.0 OR MIT', :text => 'See LICENSE-APACHE / LICENSE-MIT at the repo root' } s.license = { :type => 'Apache-2.0 OR MIT', :text => 'See LICENSE-APACHE / LICENSE-MIT at the repo root' }
s.author = { 'EdisonJwa' => 'me@edison.network' } s.author = { 'EdisonJwa' => 'me@edison.network' }
s.source = { :path => '.' } s.source = { :path => '.' }
s.platform = :ios, '15.1' s.platform = :ios, '16.0'
# Build the Rust bridge on `pod install`. The script runs under # Build the Rust bridge on `pod install`. The script runs under
# bash; we use `set -e` so any failure (cargo missing, target not # bash; we use `set -e` so any failure (cargo missing, target not
@@ -92,9 +92,9 @@ Pod::Spec.new do |s|
RUSTUP_HOME="$USER_HOME/.rustup" \\ RUSTUP_HOME="$USER_HOME/.rustup" \\
RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\ RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\
RUSTC="$RUSTC_BIN" \\ RUSTC="$RUSTC_BIN" \\
IPHONEOS_DEPLOYMENT_TARGET=15.1 \\ IPHONEOS_DEPLOYMENT_TARGET=16.0 \\
CMAKE_POLICY_VERSION_MINIMUM=3.5 \\ CMAKE_POLICY_VERSION_MINIMUM=3.5 \\
CMAKE_OSX_DEPLOYMENT_TARGET=15.1 \\ CMAKE_OSX_DEPLOYMENT_TARGET=16.0 \\
"$CARGO_BIN" build --release --target aarch64-apple-ios -p chanora_bridge "$CARGO_BIN" build --release --target aarch64-apple-ios -p chanora_bridge
if [ ! -f "$BRIDGE" ]; then if [ ! -f "$BRIDGE" ]; then
@@ -122,7 +122,7 @@ Pod::Spec.new do |s|
<key>CFBundleShortVersionString</key><string>1.0.0</string> <key>CFBundleShortVersionString</key><string>1.0.0</string>
<key>CFBundleVersion</key><string>1</string> <key>CFBundleVersion</key><string>1</string>
<key>CFBundleSupportedPlatforms</key><array><string>iPhoneOS</string></array> <key>CFBundleSupportedPlatforms</key><array><string>iPhoneOS</string></array>
<key>MinimumOSVersion</key><string>15.1</string> <key>MinimumOSVersion</key><string>16.0</string>
</dict> </dict>
</plist> </plist>
PLIST PLIST
@@ -200,9 +200,9 @@ PLIST
RUSTUP_HOME="$USER_HOME/.rustup" \\ RUSTUP_HOME="$USER_HOME/.rustup" \\
RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\ RUSTUP_TOOLCHAIN="stable-aarch64-apple-darwin" \\
RUSTC="$RUSTC_BIN" \\ RUSTC="$RUSTC_BIN" \\
IPHONEOS_DEPLOYMENT_TARGET=15.1 \\ IPHONEOS_DEPLOYMENT_TARGET=16.0 \\
CMAKE_POLICY_VERSION_MINIMUM=3.5 \\ CMAKE_POLICY_VERSION_MINIMUM=3.5 \\
CMAKE_OSX_DEPLOYMENT_TARGET=15.1 \\ CMAKE_OSX_DEPLOYMENT_TARGET=16.0 \\
"$CARGO_BIN" build --release --target "$RUST_TARGET" -p chanora_bridge "$CARGO_BIN" build --release --target "$RUST_TARGET" -p chanora_bridge
cd "$REPO_ROOT/apps/chanora_flutter/ios" cd "$REPO_ROOT/apps/chanora_flutter/ios"
@@ -230,7 +230,7 @@ PLIST
<key>CFBundleShortVersionString</key><string>1.0.0</string> <key>CFBundleShortVersionString</key><string>1.0.0</string>
<key>CFBundleVersion</key><string>1</string> <key>CFBundleVersion</key><string>1</string>
<key>CFBundleSupportedPlatforms</key><array><string>$SUPPORTED_PLATFORM</string></array> <key>CFBundleSupportedPlatforms</key><array><string>$SUPPORTED_PLATFORM</string></array>
<key>MinimumOSVersion</key><string>15.1</string> <key>MinimumOSVersion</key><string>16.0</string>
</dict> </dict>
</plist> </plist>
PLIST PLIST
+23 -4
View File
@@ -7,33 +7,52 @@ PODS:
- FlutterMacOS (1.0.0) - FlutterMacOS (1.0.0)
- package_info_plus (0.0.1): - package_info_plus (0.0.1):
- FlutterMacOS - FlutterMacOS
- share_plus (0.0.1):
- FlutterMacOS
- shared_preferences_foundation (0.0.1):
- Flutter
- FlutterMacOS
- url_launcher_macos (0.0.1):
- FlutterMacOS
DEPENDENCIES: DEPENDENCIES:
- audio_session (from `Flutter/ephemeral/.symlinks/plugins/audio_session/macos`) - audio_session (from `Flutter/ephemeral/.symlinks/plugins/audio_session/macos`)
- chanora_bridge (from `/Users/edison/chanora/apps/chanora_flutter/macos`) - chanora_bridge (from `/Users/edison/dev/chanora/apps/chanora_flutter/macos`)
- connectivity_plus (from `Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos`) - connectivity_plus (from `Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos`)
- FlutterMacOS (from `Flutter/ephemeral`) - FlutterMacOS (from `Flutter/ephemeral`)
- package_info_plus (from `Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos`) - package_info_plus (from `Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos`)
- share_plus (from `Flutter/ephemeral/.symlinks/plugins/share_plus/macos`)
- shared_preferences_foundation (from `Flutter/ephemeral/.symlinks/plugins/shared_preferences_foundation/darwin`)
- url_launcher_macos (from `Flutter/ephemeral/.symlinks/plugins/url_launcher_macos/macos`)
EXTERNAL SOURCES: EXTERNAL SOURCES:
audio_session: audio_session:
:path: Flutter/ephemeral/.symlinks/plugins/audio_session/macos :path: Flutter/ephemeral/.symlinks/plugins/audio_session/macos
chanora_bridge: chanora_bridge:
:path: "/Users/edison/chanora/apps/chanora_flutter/macos" :path: "/Users/edison/dev/chanora/apps/chanora_flutter/macos"
connectivity_plus: connectivity_plus:
:path: Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos :path: Flutter/ephemeral/.symlinks/plugins/connectivity_plus/macos
FlutterMacOS: FlutterMacOS:
:path: Flutter/ephemeral :path: Flutter/ephemeral
package_info_plus: package_info_plus:
:path: Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos :path: Flutter/ephemeral/.symlinks/plugins/package_info_plus/macos
share_plus:
:path: Flutter/ephemeral/.symlinks/plugins/share_plus/macos
shared_preferences_foundation:
:path: Flutter/ephemeral/.symlinks/plugins/shared_preferences_foundation/darwin
url_launcher_macos:
:path: Flutter/ephemeral/.symlinks/plugins/url_launcher_macos/macos
SPEC CHECKSUMS: SPEC CHECKSUMS:
audio_session: eaca2512cf2b39212d724f35d11f46180ad3a33e audio_session: eaca2512cf2b39212d724f35d11f46180ad3a33e
chanora_bridge: 1403e892a388d6bbb751d2d658e539d7a456fb1b chanora_bridge: 4105993843b5421ee4ce72220a74c63f6fd99103
connectivity_plus: 4adf20a405e25b42b9c9f87feff8f4b6fde18a4e connectivity_plus: 4adf20a405e25b42b9c9f87feff8f4b6fde18a4e
FlutterMacOS: d0db08ddef1a9af05a5ec4b724367152bb0500b1 FlutterMacOS: d0db08ddef1a9af05a5ec4b724367152bb0500b1
package_info_plus: f0052d280d17aa382b932f399edf32507174e870 package_info_plus: f0052d280d17aa382b932f399edf32507174e870
share_plus: 510bf0af1a42cd602274b4629920c9649c52f4cc
shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb
url_launcher_macos: f87a979182d112f911de6820aefddaf56ee9fbfd
PODFILE CHECKSUM: d2e26b3cd926b9e407faa321206548300d262a08 PODFILE CHECKSUM: 99f0d126cab50f07c488b8550ebf033d2e8bcaeb
COCOAPODS: 1.16.2 COCOAPODS: 1.16.2
@@ -27,6 +27,8 @@
33CC10F32044A3C60003C045 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F22044A3C60003C045 /* Assets.xcassets */; }; 33CC10F32044A3C60003C045 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F22044A3C60003C045 /* Assets.xcassets */; };
33CC10F62044A3C60003C045 /* MainMenu.xib in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F42044A3C60003C045 /* MainMenu.xib */; }; 33CC10F62044A3C60003C045 /* MainMenu.xib in Resources */ = {isa = PBXBuildFile; fileRef = 33CC10F42044A3C60003C045 /* MainMenu.xib */; };
33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33CC11122044BFA00003C045 /* MainFlutterWindow.swift */; }; 33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33CC11122044BFA00003C045 /* MainFlutterWindow.swift */; };
8C6000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */; };
8C6000042DD0000000000001 /* SileroCoreML in Frameworks */ = {isa = PBXBuildFile; productRef = 8C6000032DD0000000000001 /* SileroCoreML */; };
45F255D1DE0134185DB5423D /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 06E1AA7E1FB968C1D78DA8DE /* PrivacyInfo.xcprivacy */; }; 45F255D1DE0134185DB5423D /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 06E1AA7E1FB968C1D78DA8DE /* PrivacyInfo.xcprivacy */; };
9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = EFEADEEFAB54DFEAAD7A70E9 /* Pods_Runner.framework */; }; 9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = EFEADEEFAB54DFEAAD7A70E9 /* Pods_Runner.framework */; };
C2DC22E19FDCE26B9D79442E /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = FDE04BBB936AA14C2B58FA0E /* Pods_RunnerTests.framework */; }; C2DC22E19FDCE26B9D79442E /* Pods_RunnerTests.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = FDE04BBB936AA14C2B58FA0E /* Pods_RunnerTests.framework */; };
@@ -74,6 +76,7 @@
33CC10F52044A3C60003C045 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.xib; name = Base; path = Base.lproj/MainMenu.xib; sourceTree = "<group>"; }; 33CC10F52044A3C60003C045 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.xib; name = Base; path = Base.lproj/MainMenu.xib; sourceTree = "<group>"; };
33CC10F72044A3C60003C045 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; name = Info.plist; path = Runner/Info.plist; sourceTree = "<group>"; }; 33CC10F72044A3C60003C045 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; name = Info.plist; path = Runner/Info.plist; sourceTree = "<group>"; };
33CC11122044BFA00003C045 /* MainFlutterWindow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MainFlutterWindow.swift; sourceTree = "<group>"; }; 33CC11122044BFA00003C045 /* MainFlutterWindow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MainFlutterWindow.swift; sourceTree = "<group>"; };
8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SileroCoreMLBridge.swift; sourceTree = "<group>"; };
33CEB47222A05771004F2AC0 /* Flutter-Debug.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Debug.xcconfig"; sourceTree = "<group>"; }; 33CEB47222A05771004F2AC0 /* Flutter-Debug.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Debug.xcconfig"; sourceTree = "<group>"; };
33CEB47422A05771004F2AC0 /* Flutter-Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Release.xcconfig"; sourceTree = "<group>"; }; 33CEB47422A05771004F2AC0 /* Flutter-Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = "Flutter-Release.xcconfig"; sourceTree = "<group>"; };
33CEB47722A0578A004F2AC0 /* Flutter-Generated.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = "Flutter-Generated.xcconfig"; path = "ephemeral/Flutter-Generated.xcconfig"; sourceTree = "<group>"; }; 33CEB47722A0578A004F2AC0 /* Flutter-Generated.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = "Flutter-Generated.xcconfig"; path = "ephemeral/Flutter-Generated.xcconfig"; sourceTree = "<group>"; };
@@ -105,6 +108,7 @@
isa = PBXFrameworksBuildPhase; isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647; buildActionMask = 2147483647;
files = ( files = (
8C6000042DD0000000000001 /* SileroCoreML in Frameworks */,
9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */, 9B86175918197ECC64969956 /* Pods_Runner.framework in Frameworks */,
); );
runOnlyForDeploymentPostprocessing = 0; runOnlyForDeploymentPostprocessing = 0;
@@ -179,6 +183,7 @@
children = ( children = (
33CC10F02044A3C60003C045 /* AppDelegate.swift */, 33CC10F02044A3C60003C045 /* AppDelegate.swift */,
33CC11122044BFA00003C045 /* MainFlutterWindow.swift */, 33CC11122044BFA00003C045 /* MainFlutterWindow.swift */,
8C6000002DD0000000000001 /* SileroCoreMLBridge.swift */,
33E51913231747F40026EE4D /* DebugProfile.entitlements */, 33E51913231747F40026EE4D /* DebugProfile.entitlements */,
33E51914231749380026EE4D /* Release.entitlements */, 33E51914231749380026EE4D /* Release.entitlements */,
33CC11242044D66E0003C045 /* Resources */, 33CC11242044D66E0003C045 /* Resources */,
@@ -250,6 +255,9 @@
33CC11202044C79F0003C045 /* PBXTargetDependency */, 33CC11202044C79F0003C045 /* PBXTargetDependency */,
); );
name = Runner; name = Runner;
packageProductDependencies = (
8C6000032DD0000000000001 /* SileroCoreML */,
);
productName = Runner; productName = Runner;
productReference = 33CC10ED2044A3C60003C045 /* chanora_flutter.app */; productReference = 33CC10ED2044A3C60003C045 /* chanora_flutter.app */;
productType = "com.apple.product-type.application"; productType = "com.apple.product-type.application";
@@ -293,6 +301,9 @@
Base, Base,
); );
mainGroup = 33CC10E42044A3C60003C045; mainGroup = 33CC10E42044A3C60003C045;
packageReferences = (
8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */,
);
productRefGroup = 33CC10EE2044A3C60003C045 /* Products */; productRefGroup = 33CC10EE2044A3C60003C045 /* Products */;
projectDirPath = ""; projectDirPath = "";
projectRoot = ""; projectRoot = "";
@@ -442,6 +453,7 @@
33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */, 33CC11132044BFA00003C045 /* MainFlutterWindow.swift in Sources */,
33CC10F12044A3C60003C045 /* AppDelegate.swift in Sources */, 33CC10F12044A3C60003C045 /* AppDelegate.swift in Sources */,
335BBD1B22A9A15E00E9071D /* GeneratedPluginRegistrant.swift in Sources */, 335BBD1B22A9A15E00E9071D /* GeneratedPluginRegistrant.swift in Sources */,
8C6000012DD0000000000001 /* SileroCoreMLBridge.swift in Sources */,
); );
runOnlyForDeploymentPostprocessing = 0; runOnlyForDeploymentPostprocessing = 0;
}; };
@@ -480,7 +492,7 @@
BUNDLE_LOADER = "$(TEST_HOST)"; BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2; MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests; PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
@@ -496,7 +508,7 @@
BUNDLE_LOADER = "$(TEST_HOST)"; BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2; MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests; PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
@@ -512,7 +524,7 @@
BUNDLE_LOADER = "$(TEST_HOST)"; BUNDLE_LOADER = "$(TEST_HOST)";
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
MARKETING_VERSION = 0.2; MARKETING_VERSION = 0.2;
PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests; PRODUCT_BUNDLE_IDENTIFIER = app.chanora.chanoraFlutter.RunnerTests;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
@@ -562,7 +574,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15; MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = NO; MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = macosx; SDKROOT = macosx;
STRING_CATALOG_GENERATE_SYMBOLS = YES; STRING_CATALOG_GENERATE_SYMBOLS = YES;
@@ -601,7 +613,7 @@
"$(inherited)", "$(inherited)",
"@executable_path/../Frameworks", "@executable_path/../Frameworks",
); );
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = ""; PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_VERSION = 5.0; SWIFT_VERSION = 5.0;
}; };
@@ -611,7 +623,7 @@
isa = XCBuildConfiguration; isa = XCBuildConfiguration;
buildSettings = { buildSettings = {
CODE_SIGN_STYLE = Manual; CODE_SIGN_STYLE = Manual;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
}; };
name = Profile; name = Profile;
@@ -663,7 +675,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15; MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = YES; MTL_ENABLE_DEBUG_INFO = YES;
ONLY_ACTIVE_ARCH = YES; ONLY_ACTIVE_ARCH = YES;
SDKROOT = macosx; SDKROOT = macosx;
@@ -714,7 +726,7 @@
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES; GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES; GCC_WARN_UNUSED_VARIABLE = YES;
MACOSX_DEPLOYMENT_TARGET = 10.15; MACOSX_DEPLOYMENT_TARGET = 13.0;
MTL_ENABLE_DEBUG_INFO = NO; MTL_ENABLE_DEBUG_INFO = NO;
SDKROOT = macosx; SDKROOT = macosx;
STRING_CATALOG_GENERATE_SYMBOLS = YES; STRING_CATALOG_GENERATE_SYMBOLS = YES;
@@ -753,7 +765,7 @@
"$(inherited)", "$(inherited)",
"@executable_path/../Frameworks", "@executable_path/../Frameworks",
); );
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = ""; PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_OPTIMIZATION_LEVEL = "-Onone"; SWIFT_OPTIMIZATION_LEVEL = "-Onone";
SWIFT_VERSION = 5.0; SWIFT_VERSION = 5.0;
@@ -779,7 +791,7 @@
"$(inherited)", "$(inherited)",
"@executable_path/../Frameworks", "@executable_path/../Frameworks",
); );
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PROVISIONING_PROFILE_SPECIFIER = ""; PROVISIONING_PROFILE_SPECIFIER = "";
SWIFT_VERSION = 5.0; SWIFT_VERSION = 5.0;
}; };
@@ -789,7 +801,7 @@
isa = XCBuildConfiguration; isa = XCBuildConfiguration;
buildSettings = { buildSettings = {
CODE_SIGN_STYLE = Manual; CODE_SIGN_STYLE = Manual;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
}; };
name = Debug; name = Debug;
@@ -798,7 +810,7 @@
isa = XCBuildConfiguration; isa = XCBuildConfiguration;
buildSettings = { buildSettings = {
CODE_SIGN_STYLE = Automatic; CODE_SIGN_STYLE = Automatic;
MACOSX_DEPLOYMENT_TARGET = 11.0; MACOSX_DEPLOYMENT_TARGET = 13.0;
PRODUCT_NAME = "$(TARGET_NAME)"; PRODUCT_NAME = "$(TARGET_NAME)";
}; };
name = Release; name = Release;
@@ -847,6 +859,21 @@
defaultConfigurationName = Release; defaultConfigurationName = Release;
}; };
/* End XCConfigurationList section */ /* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */ = {
isa = XCLocalSwiftPackageReference;
relativePath = ../../../../silero-coreml;
};
/* End XCLocalSwiftPackageReference section */
/* Begin XCSwiftPackageProductDependency section */
8C6000032DD0000000000001 /* SileroCoreML */ = {
isa = XCSwiftPackageProductDependency;
package = 8C6000022DD0000000000001 /* XCLocalSwiftPackageReference "silero-coreml" */;
productName = SileroCoreML;
};
/* End XCSwiftPackageProductDependency section */
}; };
rootObject = 33CC10E52044A3C60003C045 /* Project object */; rootObject = 33CC10E52044A3C60003C045 /* Project object */;
} }
@@ -0,0 +1,98 @@
import CoreML
import Foundation
import SileroCoreML
private final class ChanoraSileroVadBox {
let vad: SileroVAD
init() throws {
let configuration = MLModelConfiguration()
vad = try SileroVAD(configuration: configuration)
}
}
private let chanoraSileroErrorLock = NSLock()
private var chanoraSileroLastError = ""
private func setChanoraSileroLastError(_ message: String) {
chanoraSileroErrorLock.lock()
chanoraSileroLastError = message
chanoraSileroErrorLock.unlock()
}
@_cdecl("chanora_silero_vad_create")
public func chanoraSileroVadCreate() -> UnsafeMutableRawPointer? {
do {
let box = try ChanoraSileroVadBox()
return Unmanaged.passRetained(box).toOpaque()
} catch {
setChanoraSileroLastError(String(describing: error))
return nil
}
}
@_cdecl("chanora_silero_vad_destroy")
public func chanoraSileroVadDestroy(_ handle: UnsafeMutableRawPointer?) {
guard let handle else { return }
Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).release()
}
@_cdecl("chanora_silero_vad_reset")
public func chanoraSileroVadReset(_ handle: UnsafeMutableRawPointer?) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
box.vad.reset()
return 0
}
@_cdecl("chanora_silero_vad_process")
public func chanoraSileroVadProcess(
_ handle: UnsafeMutableRawPointer?,
_ samples: UnsafePointer<Float>?,
_ sampleCount: Int,
_ probabilityOut: UnsafeMutablePointer<Float>?
) -> Int32 {
guard let handle else {
setChanoraSileroLastError("SileroVAD handle is null")
return -1
}
guard let samples else {
setChanoraSileroLastError("SileroVAD samples pointer is null")
return -2
}
guard let probabilityOut else {
setChanoraSileroLastError("SileroVAD probability output pointer is null")
return -3
}
guard sampleCount == SileroVAD.chunkSize else {
setChanoraSileroLastError("SileroVAD expected \(SileroVAD.chunkSize) samples, got \(sampleCount)")
return -4
}
let box = Unmanaged<ChanoraSileroVadBox>.fromOpaque(handle).takeUnretainedValue()
do {
let chunk = Array(UnsafeBufferPointer(start: samples, count: sampleCount))
probabilityOut.pointee = try box.vad.process(chunk)
return 0
} catch {
setChanoraSileroLastError(String(describing: error))
return -5
}
}
@_cdecl("chanora_silero_vad_last_error")
public func chanoraSileroVadLastError() -> UnsafeMutablePointer<CChar>? {
chanoraSileroErrorLock.lock()
let message = chanoraSileroLastError
chanoraSileroErrorLock.unlock()
return strdup(message)
}
@_cdecl("chanora_silero_vad_free_string")
public func chanoraSileroVadFreeString(_ string: UnsafeMutablePointer<CChar>?) {
guard let string else { return }
free(string)
}
@@ -4,9 +4,8 @@
# This value is the macOS SDK floor against which `libchanora_bridge.dylib` is # This value is the macOS SDK floor against which `libchanora_bridge.dylib` is
# compiled — it is NOT the Flutter Runner app's deployment target (which lives # compiled — it is NOT the Flutter Runner app's deployment target (which lives
# in Runner.xcodeproj/project.pbxproj at the PBXNativeTarget level and is # in Runner.xcodeproj/project.pbxproj at the PBXNativeTarget level and is
# currently `11.0`). The bridge floor is intentionally broader (`10.15`) so the # currently `13.0`). The bridge floor matches the Runner app and Apple/CoreML
# cdylib symbols remain link-compatible with any consumer >= 10.15; the Runner # package requirement so the bundled SwiftPM dependency can link consistently.
# app's own minimum is independently set at the Xcode-target layer.
# #
# Consumers: # Consumers:
# - apps/chanora_flutter/macos/Podfile (`platform :osx, ...`) # - apps/chanora_flutter/macos/Podfile (`platform :osx, ...`)
@@ -18,4 +17,4 @@
# To bump the floor, edit ONLY this file. Do not introduce any other literal # To bump the floor, edit ONLY this file. Do not introduce any other literal
# occurrence of the floor string in this directory. # occurrence of the floor string in this directory.
MACOS_BRIDGE_DEPLOYMENT_TARGET = '10.15'.freeze MACOS_BRIDGE_DEPLOYMENT_TARGET = '13.0'.freeze
+4 -4
View File
@@ -457,10 +457,10 @@ packages:
dependency: transitive dependency: transitive
description: description:
name: meta name: meta
sha256: "23f08335362185a5ea2ad3a4e597f1375e78bce8a040df5c600c8d3552ef2394" sha256: "1741988757a65eb6b36abe716829688cf01910bbf91c34354ff7ec1c3de2b349"
url: "https://pub.dev" url: "https://pub.dev"
source: hosted source: hosted
version: "1.17.0" version: "1.18.0"
mime: mime:
dependency: transitive dependency: transitive
description: description:
@@ -790,10 +790,10 @@ packages:
dependency: transitive dependency: transitive
description: description:
name: test_api name: test_api
sha256: "8161c84903fd860b26bfdefb7963b3f0b68fee7adea0f59ef805ecca346f0c7a" sha256: "949a932224383300f01be9221c39180316445ecb8e7547f70a41a35bf421fb9e"
url: "https://pub.dev" url: "https://pub.dev"
source: hosted source: hosted
version: "0.7.10" version: "0.7.11"
typed_data: typed_data:
dependency: transitive dependency: transitive
description: description:
@@ -61,9 +61,26 @@ fn bench_capture_alloc_count(c: &mut Criterion) {
let blocks_final = stats_final.total_blocks; let blocks_final = stats_final.total_blocks;
let delta = blocks_final - blocks_warm; let delta = blocks_final - blocks_warm;
// Sidecar file for §5 emit_baseline: criterion's own
// estimates.json carries the no-op closure timing, NOT the
// alloc count, so we write the canonical alloc-count value
// here and the emitter reads it directly. Write before the
// developer-facing assertion so the advisory CI can still report
// the regression after tolerating the bench process failure.
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR")
.map(std::path::PathBuf::from)
.or_else(|_| std::env::current_dir().map(|d| d.join("target")))
{
let path = dir.join("criterion").join("capture_alloc_count.sidecar");
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let _ = std::fs::write(&path, delta.to_string());
}
// Local-developer surface: hard-fail on any regression. // Local-developer surface: hard-fail on any regression.
// The CI advisory comparator carries the same rule with a // The CI advisory comparator carries the same rule with a
// markdown 🔴 marker on regression instead of a panic. // markdown marker on regression instead of a panic.
assert_eq!( assert_eq!(
delta, 0, delta, 0,
"post-warmup heap allocation regression: {} blocks (SRS-219 clause a)", "post-warmup heap allocation regression: {} blocks (SRS-219 clause a)",
@@ -76,27 +93,12 @@ fn bench_capture_alloc_count(c: &mut Criterion) {
// measurement is the delta computed above; criterion's // measurement is the delta computed above; criterion's
// function-time-mean is uninteresting for an alloc-count // function-time-mean is uninteresting for an alloc-count
// metric. The §5 emitter reads the `capture_alloc_count` // metric. The §5 emitter reads the `capture_alloc_count`
// metric value out of band via a sidecar file written below. // metric value out of band via the sidecar file written above.
c.bench_function("capture_alloc_count", |b| { c.bench_function("capture_alloc_count", |b| {
b.iter(|| { b.iter(|| {
black_box(delta); black_box(delta);
}); });
}); });
// Sidecar file for §5 emit_baseline: criterion's own
// estimates.json carries the no-op closure timing, NOT the
// alloc count, so we write the canonical alloc-count value
// here and the emitter reads it directly.
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR")
.map(std::path::PathBuf::from)
.or_else(|_| std::env::current_dir().map(|d| d.join("target")))
{
let path = dir.join("criterion").join("capture_alloc_count.sidecar");
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let _ = std::fs::write(&path, delta.to_string());
}
} }
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))] #[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "android")))]
+3 -9
View File
@@ -100,12 +100,6 @@ pub enum VadBackend {
Disabled, Disabled,
} }
#[cfg(target_os = "ios")]
fn default_vad_backend() -> VadBackend {
VadBackend::WebrtcVad
}
#[cfg(not(target_os = "ios"))]
fn default_vad_backend() -> VadBackend { fn default_vad_backend() -> VadBackend {
VadBackend::SileroOnnx VadBackend::SileroOnnx
} }
@@ -114,6 +108,9 @@ impl VadBackend {
/// Stable bridge/debug string. /// Stable bridge/debug string.
pub fn as_str(self) -> &'static str { pub fn as_str(self) -> &'static str {
match self { match self {
#[cfg(any(target_os = "ios", target_os = "macos"))]
Self::SileroOnnx => "apple_coreml",
#[cfg(not(any(target_os = "ios", target_os = "macos")))]
Self::SileroOnnx => "silero_vad_onnx", Self::SileroOnnx => "silero_vad_onnx",
Self::WebrtcVad => "webrtc_vad", Self::WebrtcVad => "webrtc_vad",
Self::EnergyDebug => "energy_debug", Self::EnergyDebug => "energy_debug",
@@ -252,9 +249,6 @@ mod tests {
assert_eq!(config.aec, EffectOwner::Platform); assert_eq!(config.aec, EffectOwner::Platform);
assert_eq!(config.ns, EffectOwner::Platform); assert_eq!(config.ns, EffectOwner::Platform);
assert_eq!(config.agc, EffectOwner::Platform); assert_eq!(config.agc, EffectOwner::Platform);
#[cfg(target_os = "ios")]
assert_eq!(config.vad_backend, VadBackend::WebrtcVad);
#[cfg(not(target_os = "ios"))]
assert_eq!(config.vad_backend, VadBackend::SileroOnnx); assert_eq!(config.vad_backend, VadBackend::SileroOnnx);
} }
-1
View File
@@ -312,7 +312,6 @@ pub struct AudioEngine {
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>, voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
#[cfg(any(target_os = "ios", target_os = "macos", target_os = "android"))] #[cfg(any(target_os = "ios", target_os = "macos", target_os = "android"))]
mic_gain: f32, mic_gain: f32,
#[cfg(any(target_os = "ios", target_os = "macos"))]
// Streams must be dropped to stop audio. Both are `!Send` because // Streams must be dropped to stop audio. Both are `!Send` because
// cpal's Stream isn't Send on some backends; we keep them in an // cpal's Stream isn't Send on some backends; we keep them in an
// Option wrapped by Mutex so stop() can move them out. On Linux // Option wrapped by Mutex so stop() can move them out. On Linux
+42 -5
View File
@@ -116,6 +116,7 @@ mod inner {
mic_gain: f32, mic_gain: f32,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>, voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
vad_detector: crate::vad::WebRtcFallbackVad, vad_detector: crate::vad::WebRtcFallbackVad,
silero_coreml_worker: Option<crate::vad::apple_coreml::AppleCoreMlVadWorker>,
current_vad_backend: crate::VadBackend, current_vad_backend: crate::VadBackend,
capture_frame_seq: u64, capture_frame_seq: u64,
vad_state: crate::voice_activity::VoiceActivityStateMachine, vad_state: crate::voice_activity::VoiceActivityStateMachine,
@@ -152,6 +153,7 @@ mod inner {
mic_gain: params.mic_gain, mic_gain: params.mic_gain,
voice_activity_selector: params.voice_activity_selector.clone(), voice_activity_selector: params.voice_activity_selector.clone(),
vad_detector: crate::vad::WebRtcFallbackVad::default(), vad_detector: crate::vad::WebRtcFallbackVad::default(),
silero_coreml_worker: None,
current_vad_backend: crate::VadBackend::WebrtcVad, current_vad_backend: crate::VadBackend::WebrtcVad,
capture_frame_seq: 0, capture_frame_seq: 0,
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(), vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
@@ -272,6 +274,7 @@ mod inner {
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity) .map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
.unwrap_or(false); .unwrap_or(false);
if !voice_activity_mode { if !voice_activity_mode {
self.silero_coreml_worker = None;
self.current_vad_backend = crate::VadBackend::Disabled; self.current_vad_backend = crate::VadBackend::Disabled;
self.fallback_warned_backend = None; self.fallback_warned_backend = None;
self.audio_processing_stats.set_vad_fallback_active(false); self.audio_processing_stats.set_vad_fallback_active(false);
@@ -297,9 +300,16 @@ mod inner {
self.current_vad_backend = vad_backend; self.current_vad_backend = vad_backend;
self.fallback_warned_backend = None; self.fallback_warned_backend = None;
if vad_backend == crate::VadBackend::SileroOnnx { if vad_backend == crate::VadBackend::SileroOnnx {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx); self.silero_coreml_worker =
self.audio_processing_stats.set_vad_fallback_active(true); crate::vad::apple_coreml::AppleCoreMlVadWorker::try_new();
if self.silero_coreml_worker.is_none() {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
} else {
self.audio_processing_stats.set_vad_fallback_active(false);
}
} else { } else {
self.silero_coreml_worker = None;
self.audio_processing_stats.set_vad_fallback_active(false); self.audio_processing_stats.set_vad_fallback_active(false);
} }
self.vad_state.reset(); self.vad_state.reset();
@@ -307,6 +317,7 @@ mod inner {
let (vad_probability, active) = if voice_activity_mode { let (vad_probability, active) = if voice_activity_mode {
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1); self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
let capture_seq = self.capture_frame_seq;
let mut used_fallback_vad = false; let mut used_fallback_vad = false;
let vad = if vad_backend == crate::VadBackend::Disabled { let vad = if vad_backend == crate::VadBackend::Disabled {
crate::vad::VadOutput { crate::vad::VadOutput {
@@ -314,9 +325,35 @@ mod inner {
speech: true, speech: true,
} }
} else if vad_backend == crate::VadBackend::SileroOnnx { } else if vad_backend == crate::VadBackend::SileroOnnx {
used_fallback_vad = true; if let Some(worker) = self.silero_coreml_worker.as_ref() {
self.mark_vad_fallback_active(vad_backend); let enqueued = worker.try_send(capture_seq, &frame);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame) if !worker.is_stale(capture_seq) {
let p = worker.latest_probability();
crate::vad::VadOutput {
probability: p,
speech: p >= 0.5,
}
} else if enqueued {
crate::vad::VadOutput {
probability: 0.0,
speech: false,
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(vad_backend);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else { } else {
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame) crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
}; };
+39 -5
View File
@@ -138,6 +138,7 @@ struct IosCaptureState {
mic_gain: f32, mic_gain: f32,
voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>, voice_activity_selector: Option<Arc<crate::TransmitModeSelector>>,
vad_detector: crate::vad::WebRtcFallbackVad, vad_detector: crate::vad::WebRtcFallbackVad,
silero_coreml_worker: Option<crate::vad::apple_coreml::AppleCoreMlVadWorker>,
/// Last VAD backend we configured — used to detect backend changes. /// Last VAD backend we configured — used to detect backend changes.
current_vad_backend: crate::VadBackend, current_vad_backend: crate::VadBackend,
fallback_warned_backend: Option<crate::VadBackend>, fallback_warned_backend: Option<crate::VadBackend>,
@@ -177,6 +178,7 @@ impl IosCaptureState {
mic_gain: params.mic_gain, mic_gain: params.mic_gain,
voice_activity_selector: params.voice_activity_selector.clone(), voice_activity_selector: params.voice_activity_selector.clone(),
vad_detector: crate::vad::WebRtcFallbackVad::default(), vad_detector: crate::vad::WebRtcFallbackVad::default(),
silero_coreml_worker: None,
current_vad_backend: crate::VadBackend::WebrtcVad, current_vad_backend: crate::VadBackend::WebrtcVad,
fallback_warned_backend: None, fallback_warned_backend: None,
vad_state: crate::voice_activity::VoiceActivityStateMachine::default(), vad_state: crate::voice_activity::VoiceActivityStateMachine::default(),
@@ -350,6 +352,7 @@ impl IosCaptureState {
.map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity) .map(|selector| selector.mode() == crate::TransmitMode::VoiceActivity)
.unwrap_or(false); .unwrap_or(false);
if !voice_activity_mode { if !voice_activity_mode {
self.silero_coreml_worker = None;
self.current_vad_backend = crate::VadBackend::Disabled; self.current_vad_backend = crate::VadBackend::Disabled;
self.fallback_warned_backend = None; self.fallback_warned_backend = None;
self.audio_processing_stats.set_vad_fallback_active(false); self.audio_processing_stats.set_vad_fallback_active(false);
@@ -373,9 +376,16 @@ impl IosCaptureState {
self.current_vad_backend = vad_backend; self.current_vad_backend = vad_backend;
self.fallback_warned_backend = None; self.fallback_warned_backend = None;
if vad_backend == crate::VadBackend::SileroOnnx { if vad_backend == crate::VadBackend::SileroOnnx {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx); self.silero_coreml_worker =
self.audio_processing_stats.set_vad_fallback_active(true); crate::vad::apple_coreml::AppleCoreMlVadWorker::try_new();
if self.silero_coreml_worker.is_none() {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
self.audio_processing_stats.set_vad_fallback_active(true);
} else {
self.audio_processing_stats.set_vad_fallback_active(false);
}
} else { } else {
self.silero_coreml_worker = None;
self.audio_processing_stats.set_vad_fallback_active(false); self.audio_processing_stats.set_vad_fallback_active(false);
} }
// Reset VAD state machine timers on backend switch. // Reset VAD state machine timers on backend switch.
@@ -418,6 +428,7 @@ impl IosCaptureState {
// VAD: only evaluate while VoiceActivity mode is active. // VAD: only evaluate while VoiceActivity mode is active.
let (vad_probability, gate_open) = if voice_activity_mode { let (vad_probability, gate_open) = if voice_activity_mode {
self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1); self.capture_frame_seq = self.capture_frame_seq.wrapping_add(1);
let capture_seq = self.capture_frame_seq;
let mut used_fallback_vad = false; let mut used_fallback_vad = false;
let vad = if vad_backend == crate::VadBackend::Disabled { let vad = if vad_backend == crate::VadBackend::Disabled {
crate::vad::VadOutput { crate::vad::VadOutput {
@@ -425,9 +436,32 @@ impl IosCaptureState {
speech: true, speech: true,
} }
} else if vad_backend == crate::VadBackend::SileroOnnx { } else if vad_backend == crate::VadBackend::SileroOnnx {
used_fallback_vad = true; if let Some(worker) = self.silero_coreml_worker.as_ref() {
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx); let enqueued = worker.try_send(capture_seq, &frame);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame) if !worker.is_stale(capture_seq) {
let p = worker.latest_probability();
crate::vad::VadOutput {
probability: p,
speech: p >= 0.5,
}
} else if enqueued {
crate::vad::VadOutput {
probability: 0.0,
speech: false,
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
crate::vad::VoiceActivityDetector::process_10ms(
&mut self.vad_detector,
&frame,
)
}
} else {
used_fallback_vad = true;
self.mark_vad_fallback_active(crate::VadBackend::SileroOnnx);
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
}
} else { } else {
crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame) crate::vad::VoiceActivityDetector::process_10ms(&mut self.vad_detector, &frame)
}; };
+9 -9
View File
@@ -30,7 +30,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route, route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing, ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::PlatformVoiceProcessing, processing_backend: AudioBackend::PlatformVoiceProcessing,
vad_backend: VadBackend::WebrtcVad, vad_backend: VadBackend::SileroOnnx,
aec: EffectOwner::Platform, aec: EffectOwner::Platform,
// VPIO owns NS and AGC on the shipping default path (IOSP_002/003). // VPIO owns NS and AGC on the shipping default path (IOSP_002/003).
ns: EffectOwner::Platform, ns: EffectOwner::Platform,
@@ -43,7 +43,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route, route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing, ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::Noop, processing_backend: AudioBackend::Noop,
vad_backend: VadBackend::WebrtcVad, vad_backend: VadBackend::SileroOnnx,
// No AEC needed for wired headset (no acoustic echo path). // No AEC needed for wired headset (no acoustic echo path).
aec: EffectOwner::Off, aec: EffectOwner::Off,
// Conservative NS/AGC: optional, not forced. // Conservative NS/AGC: optional, not forced.
@@ -57,7 +57,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route, route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing, ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::PlatformVoiceProcessing, processing_backend: AudioBackend::PlatformVoiceProcessing,
vad_backend: VadBackend::WebrtcVad, vad_backend: VadBackend::SileroOnnx,
// BT HFP manages its own AEC in the headset firmware. // BT HFP manages its own AEC in the headset firmware.
aec: EffectOwner::Off, aec: EffectOwner::Off,
ns: EffectOwner::Conservative, ns: EffectOwner::Conservative,
@@ -87,7 +87,7 @@ pub fn ios_route_policy(route: AudioRoute) -> AudioProcessingConfig {
route, route,
ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing, ios_mode: IosVoiceProcessingMode::PlatformVoiceProcessing,
processing_backend: AudioBackend::Noop, processing_backend: AudioBackend::Noop,
vad_backend: VadBackend::WebrtcVad, vad_backend: VadBackend::SileroOnnx,
// Safe fallback: AEC off until route is classified. // Safe fallback: AEC off until route is classified.
aec: EffectOwner::Off, aec: EffectOwner::Off,
ns: EffectOwner::Off, ns: EffectOwner::Off,
@@ -146,7 +146,7 @@ mod tests {
cfg.ios_mode, cfg.ios_mode,
IosVoiceProcessingMode::PlatformVoiceProcessing IosVoiceProcessingMode::PlatformVoiceProcessing
); );
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad); assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
} }
#[test] #[test]
@@ -157,7 +157,7 @@ mod tests {
AudioBackend::PlatformVoiceProcessing AudioBackend::PlatformVoiceProcessing
); );
assert_eq!(cfg.aec, EffectOwner::Platform); assert_eq!(cfg.aec, EffectOwner::Platform);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad); assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
} }
#[test] #[test]
@@ -165,14 +165,14 @@ mod tests {
let cfg = ios_route_policy(AudioRoute::WiredHeadset); let cfg = ios_route_policy(AudioRoute::WiredHeadset);
assert_eq!(cfg.aec, EffectOwner::Off); assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.processing_backend, AudioBackend::Noop); assert_eq!(cfg.processing_backend, AudioBackend::Noop);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad); assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
} }
#[test] #[test]
fn bluetooth_hfp_disables_app_aec() { fn bluetooth_hfp_disables_app_aec() {
let cfg = ios_route_policy(AudioRoute::BluetoothHfp); let cfg = ios_route_policy(AudioRoute::BluetoothHfp);
assert_eq!(cfg.aec, EffectOwner::Off); assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad); assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
} }
#[test] #[test]
@@ -187,7 +187,7 @@ mod tests {
fn unknown_route_safe_fallback_no_aec() { fn unknown_route_safe_fallback_no_aec() {
let cfg = ios_route_policy(AudioRoute::Unknown); let cfg = ios_route_policy(AudioRoute::Unknown);
assert_eq!(cfg.aec, EffectOwner::Off); assert_eq!(cfg.aec, EffectOwner::Off);
assert_eq!(cfg.vad_backend, VadBackend::WebrtcVad); assert_eq!(cfg.vad_backend, VadBackend::SileroOnnx);
} }
#[test] #[test]
@@ -0,0 +1,318 @@
//! Apple/CoreML Silero VAD bridge.
//!
//! The Swift Runner target exports a tiny C ABI around
//! `SileroCoreML.SileroVAD`. This Rust side resolves those symbols at
//! runtime, then runs inference on a background worker so realtime CoreAudio
//! callbacks only enqueue frames and read atomics.
use super::{VadOutput, VoiceActivityDetector};
use std::ffi::{c_char, c_void, CStr};
use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering};
use std::sync::Arc;
use std::thread::JoinHandle;
/// 16 kHz frame size required by `SileroCoreML.SileroVAD.process(_:)`.
pub const SILERO_COREML_FRAME_16K: usize = 512;
/// Maximum lag in 10 ms frames before the realtime callback falls back.
pub const SILERO_COREML_MAX_STALE_FRAMES: u64 = 3;
const SILERO_COREML_THRESHOLD: f32 = 0.5;
const RTLD_DEFAULT: *mut c_void = -2_isize as *mut c_void;
type CreateFn = unsafe extern "C" fn() -> *mut c_void;
type DestroyFn = unsafe extern "C" fn(*mut c_void);
type ResetFn = unsafe extern "C" fn(*mut c_void) -> i32;
type ProcessFn = unsafe extern "C" fn(*mut c_void, *const f32, usize, *mut f32) -> i32;
type LastErrorFn = unsafe extern "C" fn() -> *mut c_char;
type FreeStringFn = unsafe extern "C" fn(*mut c_char);
#[allow(improper_ctypes)]
extern "C" {
fn dlsym(handle: *mut c_void, symbol: *const c_char) -> *mut c_void;
}
#[derive(Clone, Copy)]
struct AppleSileroSymbols {
create: CreateFn,
destroy: DestroyFn,
reset: ResetFn,
process: ProcessFn,
last_error: LastErrorFn,
free_string: FreeStringFn,
}
impl AppleSileroSymbols {
fn resolve() -> Option<Self> {
unsafe {
Some(Self {
create: std::mem::transmute::<*mut c_void, CreateFn>(resolve_symbol(
b"chanora_silero_vad_create\0",
)?),
destroy: std::mem::transmute::<*mut c_void, DestroyFn>(resolve_symbol(
b"chanora_silero_vad_destroy\0",
)?),
reset: std::mem::transmute::<*mut c_void, ResetFn>(resolve_symbol(
b"chanora_silero_vad_reset\0",
)?),
process: std::mem::transmute::<*mut c_void, ProcessFn>(resolve_symbol(
b"chanora_silero_vad_process\0",
)?),
last_error: std::mem::transmute::<*mut c_void, LastErrorFn>(resolve_symbol(
b"chanora_silero_vad_last_error\0",
)?),
free_string: std::mem::transmute::<*mut c_void, FreeStringFn>(resolve_symbol(
b"chanora_silero_vad_free_string\0",
)?),
})
}
}
fn last_error_message(&self) -> String {
unsafe {
let ptr = (self.last_error)();
if ptr.is_null() {
return "unknown SileroCoreML bridge error".to_string();
}
let message = CStr::from_ptr(ptr).to_string_lossy().into_owned();
(self.free_string)(ptr);
message
}
}
}
unsafe fn resolve_symbol(name: &'static [u8]) -> Option<*mut c_void> {
let ptr = dlsym(RTLD_DEFAULT, name.as_ptr().cast());
if ptr.is_null() {
None
} else {
Some(ptr)
}
}
/// 16 kHz detector backed by Swift `SileroCoreML.SileroVAD`.
pub struct AppleCoreMlVad {
handle: *mut c_void,
symbols: AppleSileroSymbols,
accum: Vec<f32>,
frame_scratch: Box<[f32; SILERO_COREML_FRAME_16K]>,
last_probability: f32,
}
impl AppleCoreMlVad {
/// Create a detector when the Swift Runner bridge symbols are linked.
pub fn try_new() -> Option<Self> {
let symbols = AppleSileroSymbols::resolve()?;
Self::try_new_with_symbols(symbols)
}
fn try_new_with_symbols(symbols: AppleSileroSymbols) -> Option<Self> {
let handle = unsafe { (symbols.create)() };
if handle.is_null() {
tracing::warn!(
target: "chanora_audio",
error = %symbols.last_error_message(),
"AppleCoreMlVad: Swift SileroCoreML bridge unavailable; falling back to WebRtcFallbackVad"
);
return None;
}
tracing::info!(
target: "chanora_audio",
backend = "apple_coreml",
model = "silero_vad",
model_version = "6.2.1",
model_resource = "SileroVADModel",
model_artifact = "mlmodelc_or_mlpackage",
sample_rate_hz = 16_000,
chunk_size = SILERO_COREML_FRAME_16K,
threshold = SILERO_COREML_THRESHOLD,
"AppleCoreMlVad: using Apple CoreML Silero VAD"
);
Some(Self {
handle,
symbols,
accum: Vec::with_capacity(SILERO_COREML_FRAME_16K),
frame_scratch: Box::new([0.0; SILERO_COREML_FRAME_16K]),
last_probability: 0.0,
})
}
/// Reset accumulated samples, last probability, and the Swift VAD stream.
pub fn reset_state(&mut self) {
self.accum.clear();
self.last_probability = 0.0;
let rc = unsafe { (self.symbols.reset)(self.handle) };
if rc != 0 {
tracing::warn!(
target: "chanora_audio",
error = %self.symbols.last_error_message(),
"AppleCoreMlVad: reset failed"
);
}
}
fn calc_level(&mut self) -> f32 {
let mut probability = self.last_probability;
let rc = unsafe {
(self.symbols.process)(
self.handle,
self.frame_scratch.as_ptr(),
self.frame_scratch.len(),
&mut probability,
)
};
if rc == 0 {
self.last_probability = probability.clamp(0.0, 1.0);
} else {
tracing::warn!(
target: "chanora_audio",
error = %self.symbols.last_error_message(),
"AppleCoreMlVad: inference failed; holding last probability"
);
}
self.last_probability
}
}
impl VoiceActivityDetector for AppleCoreMlVad {
fn process_10ms(&mut self, samples: &[f32]) -> VadOutput {
debug_assert_eq!(
samples.len(),
super::resampler::OUTPUT_FRAME_10MS,
"AppleCoreMlVad expects 160 samples (16 kHz 10 ms), got {}",
samples.len()
);
self.accum.extend_from_slice(samples);
if self.accum.len() >= SILERO_COREML_FRAME_16K {
self.frame_scratch
.copy_from_slice(&self.accum[..SILERO_COREML_FRAME_16K]);
self.calc_level();
drop(self.accum.drain(..SILERO_COREML_FRAME_16K));
}
VadOutput {
probability: self.last_probability,
speech: self.last_probability >= SILERO_COREML_THRESHOLD,
}
}
}
impl Drop for AppleCoreMlVad {
fn drop(&mut self) {
unsafe { (self.symbols.destroy)(self.handle) };
}
}
// SAFETY: the opaque Swift object is owned by this detector and only used by
// the worker thread after construction. It is never shared concurrently.
unsafe impl Send for AppleCoreMlVad {}
struct SileroFrameMessage {
seq: u64,
frame: [f32; super::resampler::INPUT_FRAME_10MS],
}
/// Background Apple/CoreML Silero worker.
pub struct AppleCoreMlVadWorker {
tx: Option<std::sync::mpsc::SyncSender<SileroFrameMessage>>,
latest_probability: Arc<AtomicU32>,
latest_processed_seq: Arc<AtomicU64>,
alive: Arc<AtomicBool>,
handle: Option<JoinHandle<()>>,
}
impl AppleCoreMlVadWorker {
/// Start the background CoreML worker when the Swift bridge is available.
pub fn try_new() -> Option<Self> {
let symbols = AppleSileroSymbols::resolve()?;
let latest_probability = Arc::new(AtomicU32::new(0.0_f32.to_bits()));
let latest_processed_seq = Arc::new(AtomicU64::new(u64::MAX));
let alive = Arc::new(AtomicBool::new(true));
let (tx, rx) = std::sync::mpsc::sync_channel::<SileroFrameMessage>(64);
let latest_probability_for_thread = latest_probability.clone();
let latest_processed_seq_for_thread = latest_processed_seq.clone();
let alive_for_thread = alive.clone();
let handle = std::thread::Builder::new()
.name("chanora-apple-silero-vad".to_string())
.spawn(move || {
let Some(vad) = AppleCoreMlVad::try_new_with_symbols(symbols) else {
return;
};
let mut vad = super::Resampled16kHzVad::new(vad);
while alive_for_thread.load(Ordering::Relaxed) {
let message = match rx.recv() {
Ok(message) => message,
Err(_) => break,
};
let output = vad.process_10ms(&message.frame);
latest_probability_for_thread.store(
output.probability.clamp(0.0, 1.0).to_bits(),
Ordering::Relaxed,
);
latest_processed_seq_for_thread.store(message.seq, Ordering::Relaxed);
}
})
.ok()?;
Some(Self {
tx: Some(tx),
latest_probability,
latest_processed_seq,
alive,
handle: Some(handle),
})
}
/// Enqueue one 48 kHz 10 ms frame without blocking the caller.
pub fn try_send(&self, seq: u64, frame: &[f32; super::resampler::INPUT_FRAME_10MS]) -> bool {
let Some(tx) = &self.tx else {
return false;
};
tx.try_send(SileroFrameMessage { seq, frame: *frame })
.is_ok()
}
/// Return the latest probability published by the worker thread.
pub fn latest_probability(&self) -> f32 {
f32::from_bits(self.latest_probability.load(Ordering::Relaxed))
}
/// Return true until the worker has produced a recent probability.
pub fn is_stale(&self, capture_seq: u64) -> bool {
let latest = self.latest_processed_seq.load(Ordering::Relaxed);
latest == u64::MAX || capture_seq.saturating_sub(latest) > SILERO_COREML_MAX_STALE_FRAMES
}
}
impl Drop for AppleCoreMlVadWorker {
fn drop(&mut self) {
self.alive.store(false, Ordering::Relaxed);
let _ = self.tx.take();
// Drop can run from the realtime audio callback during backend changes;
// never join here. Closing tx lets the worker exit and dropping the
// handle detaches the thread without blocking the callback.
let _ = self.handle.take();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn coreml_constants_match_silero_package_contract() {
assert_eq!(SILERO_COREML_FRAME_16K, 512);
assert_eq!(SILERO_COREML_MAX_STALE_FRAMES, 3);
}
#[test]
fn worker_is_unavailable_without_swift_bridge_symbols_on_host_tests() {
assert!(AppleCoreMlVadWorker::try_new().is_none());
}
#[test]
fn vad_is_unavailable_without_swift_bridge_symbols_on_host_tests() {
assert!(AppleCoreMlVad::try_new().is_none());
}
}
+6 -4
View File
@@ -1,10 +1,12 @@
//! Voice activity detection backends and helpers. //! Voice activity detection backends and helpers.
//! //!
//! iOS capture feeds VoiceProcessingIO-processed microphone frames into //! Apple capture feeds VoiceProcessingIO/CoreAudio-processed microphone
//! this module and uses the realtime-safe WebRTC fallback. Other //! frames into this module and prefers Apple CoreML Silero VAD when the
//! platforms may use a model-backed detector when available so //! Swift bridge is linked. WebRTC VAD remains the realtime-safe fallback;
//! VoiceActivity mode never collapses back to Continuous transmit. //! non-Apple platforms may use ONNX-backed Silero when available.
#[cfg(any(target_os = "ios", target_os = "macos"))]
pub mod apple_coreml;
pub mod resampler; pub mod resampler;
#[cfg(not(target_os = "ios"))] #[cfg(not(target_os = "ios"))]
pub mod silero_onnx; pub mod silero_onnx;
+7 -1
View File
@@ -78,7 +78,13 @@ wildcards = "warn"
highlight = "all" highlight = "all"
# Crates we never want, regardless of license. Empty by default. # Crates we never want, regardless of license. Empty by default.
deny = [] deny = []
skip = [] skip = [
# `tracing-android` still depends on android_log-sys 0.2.x while
# flutter_rust_bridge's `android_logger` uses 0.3.x. Both are
# Android-only logcat bindings; keep this targeted until upstreams
# converge.
{ name = "android_log-sys", version = "0.2.0" },
]
skip-tree = [] skip-tree = []
# ---------- Sources ---------- # ---------- Sources ----------
+4 -4
View File
@@ -74,7 +74,7 @@ terms.
| `logging` | 1.3.0 | hosted | yes | | `logging` | 1.3.0 | hosted | yes |
| `matcher` | 0.12.19 | hosted | yes | | `matcher` | 0.12.19 | hosted | yes |
| `material_color_utilities` | 0.13.0 | hosted | yes | | `material_color_utilities` | 0.13.0 | hosted | yes |
| `meta` | 1.17.0 | hosted | yes | | `meta` | 1.18.0 | hosted | yes |
| `mime` | 2.0.0 | hosted | yes | | `mime` | 2.0.0 | hosted | yes |
| `native_toolchain_c` | 0.17.6 | hosted | yes | | `native_toolchain_c` | 0.17.6 | hosted | yes |
| `nm` | 0.5.0 | hosted | yes | | `nm` | 0.5.0 | hosted | yes |
@@ -116,7 +116,7 @@ terms.
| `stream_transform` | 2.1.1 | hosted | yes | | `stream_transform` | 2.1.1 | hosted | yes |
| `string_scanner` | 1.4.1 | hosted | yes | | `string_scanner` | 1.4.1 | hosted | yes |
| `term_glyph` | 1.2.2 | hosted | yes | | `term_glyph` | 1.2.2 | hosted | yes |
| `test_api` | 0.7.10 | hosted | yes | | `test_api` | 0.7.11 | hosted | yes |
| `typed_data` | 1.4.0 | hosted | yes | | `typed_data` | 1.4.0 | hosted | yes |
| `url_launcher` | 6.3.2 | hosted | yes | | `url_launcher` | 6.3.2 | hosted | yes |
| `url_launcher_android` | 6.3.30 | hosted | yes | | `url_launcher_android` | 6.3.30 | hosted | yes |
@@ -2813,7 +2813,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
limitations under the License. limitations under the License.
``` ```
### meta 1.17.0 ### meta 1.18.0
``` ```
Copyright 2016, the Dart project authors. Copyright 2016, the Dart project authors.
@@ -4641,7 +4641,7 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
``` ```
### test_api 0.7.10 ### test_api 0.7.11
``` ```
Copyright 2018, the Dart project authors. Copyright 2018, the Dart project authors.