edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(perf,benchmark-infra): criterion bench harness + advisory CI workflows (SDD-120)

Browse Source 
Implementation of SDD-120 §1-§8:

Bench harness (crates/chanora_audio/benches/):
- common.rs: deterministic synthetic audio (440 Hz sine, no RNG).
- realtime_capture.rs: bench_capture_alloc_count (dhat) +
  bench_capture_callback_wall_clock (criterion).
- opus_codec.rs: bench_opus_encode_latency + bench_opus_decode_latency
  (direct audiopus, not AudioHandler — SDD-120 §3 item 4).
- resampler.rs: bench_resampler_throughput across 44.1->48 /
  16->48 / 48->48 passthrough.

CI tooling (crates/chanora_audio/examples/):
- emit_baseline.rs: aggregates criterion estimates.json outputs
  into the SRS-217 baseline schema.
- compare_baseline.rs: applies SRS-219 tolerance, renders markdown
  table with 🟢/🟡/🔴 markers + yellow simpler-form realization per
  SDD-120 §8.

  Deviation from SDD-120 §2 / §5 / §7 placement: these tools live
  under examples/, not benches/ or src/bin/. Rationale: they must
  consume serde_json (a dev-only dep — production builds must not
  pull it). Cargo only resolves dev-dependencies for [[test]],
  [[bench]], and [[example]] targets; [[bin]] targets under
  src/bin/ see only regular [dependencies]. examples/ keeps the
  binaries out of the production dep tree while still giving them
  cargo run --example invocation. An SDD-120 amendment should
  reflect this.

Workflows (.github/workflows/):
- bench-advisory.yml: PR + push triggers; runs benches; posts a
  sticky PR comment via actions/github-script@v7; job status is
  always success (SRS-218 clause 4 — non-blocking).
- bench-baseline-update.yml: workflow_dispatch only; runs benches;
  opens PR via peter-evans/create-pull-request@v6 (sole writer of
  the SAD-089 baseline JSON).

Cargo.toml additions ([dev-dependencies] only — verified excluded
from --release builds): criterion 0.5, dhat 0.3, serde_json 1.

Source-code seam: minimal pub-but-#[doc(hidden)] bench_seam module
in chanora_audio (engine.rs + lib.rs re-export) so the criterion
bench harness can construct a CaptureState and drive
CaptureState::ingest without re-implementing the engine (SDD-120
§3). Non-iOS targets only — CaptureState itself is iOS-gated.

Initial baseline seed: crates/chanora_audio/benches/baselines/
x86_64-unknown-linux-gnu.json = {}. compare_baseline handles the
missing-baseline case gracefully and emits a 'no red markers'
report; the first manual dispatch of bench-baseline-update.yml
after merge establishes the real values.

Out of scope per SDD-120 §10: production telemetry export,
build-failing hard CI gate, multi-host benchmarking, IDE
integration, Dart-side bridge round-trip bench.

Verification:
- cargo check --workspace --all-targets: PASS.
- cargo bench --bench realtime_capture --no-run: PASS.
- cargo bench --bench opus_codec --no-run: PASS.
- cargo bench --bench resampler --no-run: PASS.
- cargo build --example emit_baseline --example compare_baseline
  -p chanora_audio: PASS.
- cargo test --workspace: 106 passed, 0 failed, 3 ignored — no
  regression from prior count.
This commit is contained in:
EdisonJwa
2026-05-18 13:52:15 +08:00
parent 575a6cbc5c
commit 7188a5a69d
13 changed files with 1248 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+258 -4
View File
@@ -115,6 +115,18 @@ dependencies = [
"log",
]
[[package]]
name = "anes"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b46cbb362ab8752921c97e041f5e366ee6297bd428a31275b9fcf1e380f7299"
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anyhow"
version = "1.0.102"
@@ -317,6 +329,12 @@ version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
[[package]]
name = "cast"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5"
[[package]]
name = "cc"
version = "1.2.62"
@@ -390,6 +408,8 @@ dependencies = [
"chanora_protocol",
"coreaudio-rs",
"cpal",
"criterion",
"dhat",
"futures-util",
"jni 0.21.1",
"ndk-context",
@@ -397,6 +417,7 @@ dependencies = [
"rand 0.8.6",
"reqwest",
"sdl2",
"serde_json",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -490,6 +511,33 @@ dependencies = [
"zeroize",
]
[[package]]
name = "ciborium"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42e69ffd6f0917f5c029256a24d0161db17cea3997d185db0d35926308770f0e"
dependencies = [
"ciborium-io",
"ciborium-ll",
"serde",
]
[[package]]
name = "ciborium-io"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05afea1e0a06c9be33d539b876f1ce3692f4afea2cb41f740e7743225ed1c757"
[[package]]
name = "ciborium-ll"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57663b653d948a338bfb3eeba9bb2fd5fcfaecb9e199e87e1eda4d9e8b240fd9"
dependencies = [
"ciborium-io",
"half",
]
[[package]]
name = "cipher"
version = "0.4.4"
@@ -501,6 +549,31 @@ dependencies = [
"zeroize",
]
[[package]]
name = "clap"
version = "4.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51"
dependencies = [
"clap_builder",
]
[[package]]
name = "clap_builder"
version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
dependencies = [
"anstyle",
"clap_lex",
]
[[package]]
name = "clap_lex"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "cmac"
version = "0.7.2"
@@ -644,6 +717,42 @@ dependencies = [
"libc",
]
[[package]]
name = "criterion"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2b12d017a929603d80db1831cd3a24082f8137ce19c69e6447f54f5fc8d692f"
dependencies = [
"anes",
"cast",
"ciborium",
"clap",
"criterion-plot",
"is-terminal",
"itertools 0.10.5",
"num-traits",
"once_cell",
"oorandom",
"plotters",
"rayon",
"regex",
"serde",
"serde_derive",
"serde_json",
"tinytemplate",
"walkdir",
]
[[package]]
name = "criterion-plot"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b50826342786a51a89e2da3a28f1c32b06e387201bc2d19791f622c673706b1"
dependencies = [
"cast",
"itertools 0.10.5",
]
[[package]]
name = "critical-section"
version = "1.2.0"
@@ -659,6 +768,16 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9dd111b7b7f7d55b72c0a6ae361660ee5853c9af73f70c3c2ef6858b950e2e51"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
@@ -674,6 +793,12 @@ version = "0.8.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
[[package]]
name = "crunchy"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5"
[[package]]
name = "crypto-bigint"
version = "0.5.5"
@@ -836,6 +961,22 @@ dependencies = [
"serde_core",
]
[[package]]
name = "dhat"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98cd11d84628e233de0ce467de10b8633f4ddaecafadefc86e13b84b8739b827"
dependencies = [
"backtrace",
"lazy_static",
"mintex",
"parking_lot",
"rustc-hash 1.1.0",
"serde",
"serde_json",
"thousands",
]
[[package]]
name = "digest"
version = "0.9.0"
@@ -1350,6 +1491,17 @@ dependencies = [
"tracing",
]
[[package]]
name = "half"
version = "2.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ea2d84b969582b4b1864a92dc5d27cd2b77b622a8d79306834f1be5ba20d84b"
dependencies = [
"cfg-if",
"crunchy",
"zerocopy",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
@@ -1755,6 +1907,26 @@ dependencies = [
"serde",
]
[[package]]
name = "is-terminal"
version = "0.4.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3640c1c38b8e4e43584d8df18be5fc6b0aa314ce6ebf51b53313d4306cca8e46"
dependencies = [
"hermit-abi",
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "itertools"
version = "0.10.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0fd2260e829bddf4cb6ea802289de2f86d6a7a690192fbe91b3f46e0f2c8473"
dependencies = [
"either",
]
[[package]]
name = "itertools"
version = "0.14.0"
@@ -2027,6 +2199,12 @@ dependencies = [
"adler2",
]
[[package]]
name = "mintex"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c505b3e17ed6b70a7ed2e67fbb2c560ee327353556120d6e72f5232b6880d536"
[[package]]
name = "mio"
version = "1.2.0"
@@ -2334,6 +2512,12 @@ dependencies = [
"portable-atomic",
]
[[package]]
name = "oorandom"
version = "11.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6790f58c7ff633d8771f42965289203411a5e5c68388703c06e14f24770b41e"
[[package]]
name = "opaque-debug"
version = "0.3.1"
@@ -2488,6 +2672,34 @@ version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "plotters"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5aeb6f403d7a4911efb1e33402027fc44f29b5bf6def3effcc22d7bb75f2b747"
dependencies = [
"num-traits",
"plotters-backend",
"plotters-svg",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "plotters-backend"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df42e13c12958a16b3f7f4386b9ab1f3e7933914ecea48da7139435263a4172a"
[[package]]
name = "plotters-svg"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51bae2ac328883f7acdfea3d66a7c35751187f870bc81f94563733a154d7a670"
dependencies = [
"plotters-backend",
]
[[package]]
name = "poly1305"
version = "0.8.0"
@@ -2599,7 +2811,7 @@ dependencies = [
"pin-project-lite",
"quinn-proto",
"quinn-udp",
"rustc-hash",
"rustc-hash 2.1.2",
"rustls",
"socket2",
"thiserror 2.0.18",
@@ -2620,7 +2832,7 @@ dependencies = [
"lru-slab",
"rand 0.9.4",
"ring",
"rustc-hash",
"rustc-hash 2.1.2",
"rustls",
"rustls-pki-types",
"slab",
@@ -2741,6 +2953,26 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rayon"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
dependencies = [
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
@@ -2890,6 +3122,12 @@ version = "0.1.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b50b8869d9fc858ce7266cce0194bd74df58b9d0e3f6df3a9fc8eb470d95c09d"
[[package]]
name = "rustc-hash"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.2"
@@ -3435,6 +3673,12 @@ dependencies = [
"syn",
]
[[package]]
name = "thousands"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3bf63baf9f5039dadc247375c29eb13706706cfde997d0330d05aa63a77d8820"
[[package]]
name = "thread_local"
version = "1.1.9"
@@ -3494,6 +3738,16 @@ dependencies = [
"zerovec",
]
[[package]]
name = "tinytemplate"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be4d6b5f19ff7664e8c98d03e2139cb510db9b0a60b55f8e8709b689d939b6bc"
dependencies = [
"serde",
"serde_json",
]
[[package]]
name = "tinyvec"
version = "1.11.0"
@@ -3762,7 +4016,7 @@ source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917a
dependencies = [
"base64",
"heck",
"itertools",
"itertools 0.14.0",
"num-derive",
"num-traits",
"serde",
@@ -3786,7 +4040,7 @@ dependencies = [
"git-testament",
"hickory-net",
"hickory-resolver",
"itertools",
"itertools 0.14.0",
"num-traits",
"pin-utils",
"rand 0.10.1",