edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
23bd1c7930aa58b7b8df3d73b737dccfe11dfefd
chanora/docs/verification/macos-p0-acceptance.md
T
EdisonJwa 41d5a4b91b docs(verification,index): macOS + iOS p0 acceptance checklists + 0.9.8 index row 
New controlled documents:

  * docs/verification/macos-p0-acceptance.md
    15-row human-must checklist for Apple Silicon macOS P0 sign-off.
    Targets the SDD-085 CGEventTap backend (now fully live) +
    SDD-094..097 audio lifecycle. Auto-test rows pre-filled from the
    M1 Mac verification pass: chanora_audio 34 / 0 / 0 (Linux: 32;
    macOS delta is the keymap + Send-bound + descriptor-builder
    tests). Pre-flight covers the manual framework-wrap +
    install_name_tool + ad-hoc codesign step via the new
    tools/macos-postbuild.sh. TC-3 walks the Input Monitoring grant
    + descriptor watch transition timing (~1.5 s).

  * docs/verification/ios-p0-acceptance.md
    12-row human-must checklist for iOS P0 sign-off on a physical
    iPhone via the developer's free Apple Personal Team. TC-3
    documents the Focused-only PTT capability iOS gives us (no
    global event tap analogue exists). TC-8 covers UIBackgroundModes
    = audio. TC-9 covers AVAudioSession routing — phone-call
    interruption, AirPods route, etc.

docs/governance/document-index.md
  Bumped to 0.9.8 with a change-history row covering both new
  acceptance documents. No spec items added; the existing SDD-085
  (macOS) and SDD-094..097 (audio lifecycle) are what these
  documents sign off.

python3 tools/validate_docs.py: clean (pre-existing 35-filename
[FAIL] retained, unchanged).
2026-05-16 13:50:57 +08:00

223 lines
13 KiB
Markdown
Raw Blame History

# macOS P0 acceptance — human-must test cases (v1.0.0-rc.8)
This document captures the human-side verification pass for the
macOS P0 ship of Chanora. The officially-tested environment is
**Apple Silicon macOS 14+** (DEC-025). Intel macOS is best-effort —
the same build runs but is not signed-off for this release.
Automated tests (`cargo check`, `cargo test --workspace --lib`,
the live `IOHIDCheckAccess` permission query path) sign off the
mechanical side. This checklist signs off the user-experience
side. A failure on any row blocks promotion of the macOS build
from rc.8.
Source: this checklist mirrors `linux-p0-acceptance.md` and
`windows-p0-acceptance.md`. The auto-test sign-off lives in this
document's "Auto-test sign-off" section. SDD references: SDD-081
(trait), SDD-085 (macOS CGEventTap backend), SDD-094..097 (v1
audio + PTT lifecycle), DEC-025 (target environment), DEC-027
(diagnostic privacy invariant), SRS-197 (iOS/macOS audio routing
contract), SRS-198 (honest capability advertising under runtime
permission state).
## 0. Pre-flight
| # | Step | Expected |
|---|---|---|
| 0.1 | Build host: M1 / M2 / M3 Mac running macOS 14 or later. From a graphical login (NOT SSH alone) open Terminal and confirm: `sw_vers -productVersion` is `14.x` or higher; `uname -m` is `arm64`. | Apple Silicon macOS confirmed. |
| 0.2 | Wipe stale install: `rm -rf "~/Library/Application Support/app.chanora.chanoraFlutter" ~/Library/Logs/app.chanora.chanora_flutter ~/Library/Containers/com.example.chanoraFlutter`. (App-Sandbox-aware container path; macOS quietly relocates `Application Support` under `Containers/<bundle-id>/Data/` when the app is sandboxed.) | First-launch from clean state. |
| 0.3 | Build the bridge: `cd ~/chanora && cargo build --release -p chanora_bridge`. | `target/release/libchanora_bridge.dylib` ≈ 15 MB. |
| 0.4 | Build the Flutter app: `cd apps/chanora_flutter && flutter build macos --release`. | `build/macos/Build/Products/Release/chanora_flutter.app` exists, ≈ 60 MB. |
| 0.5 | Wrap + sign: `cd ~/chanora && tools/macos-postbuild.sh Release`. | Last line reads `[postbuild] OK: …chanora_flutter.app is launchable`. The `chanora_bridge.framework` is now inside `Contents/Frameworks/`. |
| 0.6 | Launch from a graphical session: `open ~/chanora/apps/chanora_flutter/build/macos/Build/Products/Release/chanora_flutter.app`. The first launch will pop a Gatekeeper warning because the app is ad-hoc signed. Click "Open Anyway" in System Settings → Privacy & Security if prompted. | Window opens; no crash. |
| 0.7 | About / version label | Shows **`v1.0.0-rc.8`**. |
| 0.8 | Open `~/Library/Logs/app.chanora.chanora_flutter/chanora.log` | Contains `bridge initialised`, `log file path …`, `identity store initialised`, `bookmark store initialised`. File < 1 MB. The PTT controller line should read `backend_id="event-tap"` and capability `L0Focused` (because Input Monitoring has not been granted yet). |
## TC-1: Connect + identity persistence (DEC-006, SRS-201)
| # | Step | Expected |
|---|---|---|
| 1.1 | Enter a reachable TS3 server address, a nickname, blank password | "Connect" button enabled |
| 1.2 | Click Connect | Server snapshot appears: channel tree on left, client list visible |
| 1.3 | Note the connected server name + channel tree | Tree is sorted (parents above children); depth indent visible up to 6 levels |
| 1.4 | Disconnect, reconnect with the same nickname | Server-visible UID is the same as previous session (identity persisted via `Application Support/.../identity.tskey`) |
| 1.5 | Disconnect | Returns to connect form, no error |
## TC-2: Voice channel join (SRS-204, SDD-094)
| # | Step | Expected |
|---|---|---|
| 2.1 | After connecting, observe the UI | **No "Start audio" button anywhere.** Voice Bar visible at the bottom. |
| 2.2 | Double-click a voice channel | First time only: macOS shows a system prompt requesting microphone access; click **Allow**. Subsequent joins do not re-prompt. Client moves into the channel; voice bar shows the channel name. |
| 2.3 | Double-click a different voice channel | Client moves; no audio glitch; the second `voice_join` is logged cleanly with no panic. |
| 2.4 | Double-click the same channel again quickly (concurrency) | Either: no-op move, or single clean move; no duplicate audio engine spawn. |
## TC-3: PTT press / release (SDD-083, SDD-085, SDD-096) — Input Monitoring granted
| # | Step | Expected |
|---|---|---|
| 3.1 | Open Voice settings (gear icon), select **Push to talk** | Bind-key row + Release-tail slider become visible. |
| 3.2 | Click "Bind key" → press a key (e.g. Right Option) | Flutter captures the key label. macOS may show a system prompt requesting **Input Monitoring** access. Open System Settings → Privacy & Security → Input Monitoring, toggle Chanora ON. |
| 3.3 | Return to Chanora; within ~1.5 s the capability badge transitions from `L0Focused` to `L2 GlobalHoldToTalk` (or `L3` for a mouse side-button). | The transition fires WITHOUT restarting Chanora — the macOS `permission re-query worker` polls every 1.5 s and re-publishes the descriptor. |
| 3.4 | With another window focused, hold the bound key | Mic indicator turns "Mic on"; `transmit_active` true in stats. |
| 3.5 | Release the key | After the configured release-tail, indicator returns to "Mic off". |
| 3.6 | Switch focus several times while pressing the key | PTT still fires from another window — confirms the CGEventTap is reading session-global events. |
## TC-4: Release tail (SDD-097)
| # | Step | Expected |
|---|---|---|
| 4.1 | Set release-tail to 0 ms in Voice settings | Slider updates immediately. |
| 4.2 | Tap PTT key briefly | Mic on/off transition is sharp (no tail). |
| 4.3 | Set release-tail to 500 ms | Slider updates immediately. |
| 4.4 | Tap PTT key briefly | Mic stays "on" for ~500 ms after release before flipping to off. |
## TC-5: Continuous mode (SDD-094)
| # | Step | Expected |
|---|---|---|
| 5.1 | Switch transmit mode to **Continuous** | Bind-key + release-tail rows disappear; capability badge greys out. |
| 5.2 | Talk into mic | "Mic on" stays solid; no self-disable bug. |
| 5.3 | Switch back to PTT | Previously bound key still works without re-binding (binding persisted in `audio_meta.json`). |
## TC-6: Mic mute (SDD-097)
| # | Step | Expected |
|---|---|---|
| 6.1 | Press mic-mute toggle in Voice Bar | Mic icon shows muted state; selector hard-mute engaged. |
| 6.2 | Press PTT (still muted) | No "Sending audio while muted" spam in log. Mic indicator does not flip on. |
| 6.3 | Un-mute | PTT resumes working normally. |
## TC-7: Speaker mute
| # | Step | Expected |
|---|---|---|
| 7.1 | Press speaker-mute toggle | Other clients in channel are silenced locally; server still sees you in channel. |
## TC-8: Binding persistence across restart
| # | Step | Expected |
|---|---|---|
| 8.1 | With a working PTT binding, quit Chanora | `audio_meta.json` next to `identity.tskey` shows the `ptt_*` fields populated. |
| 8.2 | Relaunch | Binding restored without re-running the bind flow. Hold key — mic flips on. |
## TC-9: Binding before voice-join
| # | Step | Expected |
|---|---|---|
| 9.1 | Connect to server, do **not** join a voice channel yet. Open Voice settings, bind a PTT key. | Binding succeeds. |
| 9.2 | Now double-click a voice channel | Voice engine starts with the existing binding active immediately. |
## TC-10: No-permission channel rejection (typed `ServerRejected` propagation)
| # | Step | Expected |
|---|---|---|
| 10.1 | Try to join a channel you lack permissions on | Red SnackBar with a human message derived from the TS3 error code (e.g. `permissions_client_insufficient` → "You do not have permission to join this channel."). Move does not silently fail. |
## TC-11: Reconnect
| # | Step | Expected |
|---|---|---|
| 11.1 | Toggle Wi-Fi off for 10 s, then reconnect | UI shows red "Connection lost" banner, then recovers. Transient `Operation timed out` blips are logged but **not** shown as a banner. |
## TC-12: Diagnostic export + privacy (DEC-027, SAD-077, SDD-090)
| # | Step | Expected |
|---|---|---|
| 12.1 | Open Diagnostics, export a bundle | Bundle contains tracing records with allow-listed fields only: `backend_id`, `bound_input_class`, `capability_level`, `host_id`, `timeout_secs`, `error`, `event`, `message`, `target`, `level`. **No raw key codes, no bound key labels, no mouse-button numbers, no IOHIDAccessType raw values.** |
| 12.2 | Grep the bundle for the human-readable PTT key label | No hit. |
## TC-13: PTT bind dialog labels
| # | Step | Expected |
|---|---|---|
| 13.1 | Open Voice settings; the Bind-key row displays the Flutter-supplied label (e.g. "Space", "F10"). | The translated string never appears in `chanora.log`. |
## TC-14: Capability badge gating
| # | Step | Expected |
|---|---|---|
| 14.1 | Revoke Input Monitoring in System Settings while Chanora is running. Within ~1.5 s the badge downgrades from `L2 GlobalHoldToTalk` to `L0Focused`. The Bind-key UI hides if it would be unusable; PTT then only works when Chanora is the foreground window. | The transition is logged: `macos ptt: permission state transition from=Granted to=Denied`. |
| 14.2 | Re-grant; within ~1.5 s the badge upgrades back to L2. | Same transition log line reversed. |
## TC-15: Indicator label
| # | Step | Expected |
|---|---|---|
| 15.1 | In any mode, look at the stats line | Reads "Mic on" / "Mic off" (mode-neutral); **not** "PTT on" / "PTT off". |
## Auto-test sign-off
These rows do not require human interaction.
| Layer | Result on Apple Silicon macOS (M1 host) |
|---|---|
| `cargo check --workspace` | clean |
| `cargo test --workspace --lib` | 74 baseline + 6 new macOS keymap/state tests = **80 / 0 / 1** target; currently `chanora_audio` reports 34 / 0 / 0 on macOS (Linux had 32) |
| `cargo build --release -p chanora_bridge` | clean — produces `libchanora_bridge.dylib` ≈ 15 MB |
| `flutter build macos --release` | clean — produces `chanora_flutter.app` ≈ 60 MB |
| `tools/macos-postbuild.sh Release` | exit 0; framework-wrap + install_name_tool + ad-hoc codesign succeed |
| `chanora_flutter.app` headless launch | bridge / identity / bookmark store all initialise; `~/Library/Logs/app.chanora.chanora_flutter/chanora.log` captures clean boot |
| Live `IOHIDCheckAccess(kIOHIDRequestTypeListenEvent)` | returns `Unknown` (2) on a fresh-install box — expected; transitions to Granted/Denied after the user responds to the system prompt |
The macOS host runs 6 more `chanora_audio` tests than Linux because of the macOS-only PTT keymap + descriptor builder + RunLoopHandle Send-bound tests. Linux runs 4 portal-specific tests that macOS skips. Net cross-platform delta: `+2` macOS.
## Sign-off form
```
chanora v1.0.0-rc.8 — macOS P0 acceptance
Host: Apple Silicon macOS (M1/M2/M3)
Tester: ______________________
Date: ______________________
Build: apps/chanora_flutter/build/macos/Build/Products/Release/chanora_flutter.app
@ ______________________
chanora_bridge.framework/chanora_bridge inside Contents/Frameworks/
@ ______________________
TC-1 Connect + identity persistence [ pass / fail ] notes:
TC-2 Voice channel join [ pass / fail ] notes:
TC-3 PTT press / release (Input Monitor) [ pass / fail ] notes:
TC-4 Release tail [ pass / fail ] notes:
TC-5 Continuous mode [ pass / fail ] notes:
TC-6 Mic mute [ pass / fail ] notes:
TC-7 Speaker mute [ pass / fail ] notes:
TC-8 Binding persistence across restart [ pass / fail ] notes:
TC-9 Binding before voice-join [ pass / fail ] notes:
TC-10 No-permission channel rejection [ pass / fail ] notes:
TC-11 Reconnect [ pass / fail ] notes:
TC-12 Diagnostic export + privacy [ pass / fail ] notes:
TC-13 PTT bind dialog labels [ pass / fail ] notes:
TC-14 Capability badge gating [ pass / fail ] notes:
TC-15 Indicator label [ pass / fail ] notes:
Overall result: [ macOS rc.8 PASS / macOS rc.8 FAIL ]
Sign: ______________________
```
## Known gaps (not blockers)
- **Intel macOS** (`x86_64-apple-darwin`): the same Rust build targets it cleanly but no acceptance pass; ship as best-effort.
- **macOS Gatekeeper + notarization**: ad-hoc-signed `chanora_flutter.app` requires the user to right-click → Open the first time (or approve in Privacy & Security). Developer ID + notarization for distribution is a P1 follow-up.
- **`tools/macos-postbuild.sh` is a separate manual step**: Xcode build-phase auto-integration (so a plain `flutter build macos --release` produces a launchable bundle) is a P1 carryover, same shape as the analogous Windows + Linux gaps.
- **VAD** (`TransmitMode::VoiceActivity`) is intentionally disabled in P0 (DEC-030) on macOS as on every platform.
- **iOS** P0 acceptance is NOT covered here; see `ios-p0-acceptance.md`.
Reference in New Issue View Git Blame Copy Permalink