Edison Jwa
bba6273af7
- Move ASPICE docs to chanoraapp/docs submodule at docs/ - Move development docs to dev-docs/ (superpowers, offline-knowledge, impl-mapping) - Add AGENTS.md with project conventions for AI agents - Add impl-mapping.md (SAD component → source file mapping) - Archive completed plans to dev-docs/superpowers/plans/_archived/ - Remove AGENTS.md from .gitignore (now tracked)
136 lines
5.3 KiB
Markdown
136 lines
5.3 KiB
Markdown
# Review: Test & Document Coverage Analysis
|
|
|
|
**Reviewer:** opencode (automated)
|
|
**Reviewed file:** `docs/offline-knowledge/coverage-analysis.md`
|
|
**Date:** 2026-06-13
|
|
**Method:** Spot-checked 5 random test files, verified aggregate counts via grep/find, cross-referenced directory listings
|
|
|
|
---
|
|
|
|
## Verdict: Significant inaccuracies found
|
|
|
|
The document has **3 critical counting errors**, **2 factual errors about file existence**, and **several minor issues**. The per-file Rust test counts are mostly accurate, but the aggregate totals are wrong.
|
|
|
|
---
|
|
|
|
## Critical Errors
|
|
|
|
### 1. Total Rust test count is wrong by 40%
|
|
|
|
| Metric | Document | Actual | Delta |
|
|
|--------|----------|--------|-------|
|
|
| Inline `#[test]` | 220 | 309 | +89 |
|
|
| Integration tests | 2 | 2 | 0 |
|
|
| **Total** | **222** | **312** | **+90** |
|
|
|
|
The per-crate sums also don't reconcile: the document's own per-file tables sum to ~202 for chanora_audio (plus 2 integration = 204), but `grep -c '#\[test\]'` across `crates/chanora_audio/src/` yields **219** inline tests (+ 2 integration = 221). The document undercounts chanora_audio by 17 tests.
|
|
|
|
### 2. chanora_resolver test count off by 1
|
|
|
|
| Crate | Document | Actual |
|
|
|-------|----------|--------|
|
|
| chanora_resolver | 12 | 13 |
|
|
|
|
The extra test is in `examples/cli.rs` (documented separately as 1 example test, but the crate header total should be 13, not 12).
|
|
|
|
### 3. Doc file count is ambiguous and inaccurate
|
|
|
|
| Scope | Document says | Actual |
|
|
|-------|---------------|--------|
|
|
| All docs/ .md files | 55 | 75 |
|
|
| Excluding superpowers/ | — | 66 |
|
|
| Excluding superpowers/ + offline-knowledge/ | — | 51 |
|
|
|
|
The "55" figure doesn't match any reasonable scope calculation. The document also doesn't clarify whether superpowers/ plans/specs are included.
|
|
|
|
---
|
|
|
|
## Factual Errors
|
|
|
|
### 4. `poke_active_chat.dart` does not exist as a source file
|
|
|
|
The document lists `poke_active_chat.dart` as a tested service (line 163), and `poke_active_chat_test.dart` does exist under `test/services/`. However, **no corresponding source file** exists in `lib/services/`. This is either:
|
|
- An orphaned test for a deleted/moved source file, or
|
|
- The source file is located elsewhere (not in `lib/services/`)
|
|
|
|
The document should flag this as an anomaly, not list it as "Tested".
|
|
|
|
### 5. `audio_device_list_tile_test.dart` exists but is not counted
|
|
|
|
The document marks `audio_device_list_tile.dart` as "UNTESTED" (line 189), but `apps/chanora_flutter/test/widgets/audio_device_list_tile_test.dart` **does exist**. This means:
|
|
- Widget test file count should be **14**, not 13
|
|
- Widget coverage should be **14/24 (58%)**, not 13/24 (54%)
|
|
|
|
---
|
|
|
|
## Section Header vs. Content Mismatches
|
|
|
|
### 6. Architecture section: header says "4 files", lists 7
|
|
|
|
The header on line 221 reads "Architecture (4 files)" but the table contains 7 entries. The actual `docs/architecture/` directory has 7 files.
|
|
|
|
### 7. Governance section: header says "11 files", lists 12
|
|
|
|
The header on line 264 reads "Governance (11 files)" but the table contains 12 entries. The actual `docs/governance/` directory has 12 files.
|
|
|
|
---
|
|
|
|
## Spot-Check Results (5 Random Test Files)
|
|
|
|
| File | Document Count | Actual | Match? |
|
|
|------|---------------|--------|--------|
|
|
| `chanora_audio/src/ptt_backends/windows.rs` | 44 | 44 | ✅ |
|
|
| `chanora_audio/src/engine.rs` | 7 | 7 | ✅ |
|
|
| `chanora_state/src/lib.rs` | 18 | 18 | ✅ |
|
|
| `chanora_storage/src/lib.rs` | 15 | 15 | ✅ |
|
|
| `chanora_audio/src/route_policy.rs` | 8 | 8 | ✅ |
|
|
|
|
Per-file Rust test counts are **accurate**. The error is in the aggregation.
|
|
|
|
---
|
|
|
|
## Dart/Flutter Section: Mostly Accurate
|
|
|
|
| Metric | Document | Actual | Match? |
|
|
|--------|----------|--------|--------|
|
|
| `test()` calls | 155 | 155 | ✅ |
|
|
| `testWidgets()` calls | 66 | 66 | ✅ |
|
|
| Total Dart tests | 221 | 221 | ✅ |
|
|
| Service source files | 21 | 21 | ✅ |
|
|
| Widget source files | 24 | 24 | ✅ |
|
|
| Service test files | — | 20 | ⚠️ Not stated |
|
|
| Widget test files | 13 | 14 | ❌ |
|
|
|
|
---
|
|
|
|
## Missing Crates / Scope Issues
|
|
|
|
The document covers all 9 crates under `crates/` plus `chanora_core` under `core/`. No crates are missing. However:
|
|
|
|
- The document doesn't clearly explain that `chanora_core` lives under `core/`, not `crates/`
|
|
- The "Crates with tests: 7/9" metric (line 15) excludes `chanora_core`, which has 11 tests. If counted, it should be **8/10**
|
|
|
|
---
|
|
|
|
## Documentation Gap Analysis: Mostly Complete
|
|
|
|
The gap analysis (lines 333-354) correctly identifies undocumented modules. One omission:
|
|
|
|
- **Flutter test infrastructure** — no doc for the test helper setup, mock patterns, or test utilities used across 37 test files
|
|
|
|
---
|
|
|
|
## Summary of Required Corrections
|
|
|
|
| # | Issue | Severity | Fix |
|
|
|---|-------|----------|-----|
|
|
| 1 | Total Rust tests: 220 → 312 | Critical | Re-count and update |
|
|
| 2 | chanora_audio tests: 204 → 221 | Critical | Re-count and update |
|
|
| 3 | chanora_resolver tests: 12 → 13 | Minor | Update count |
|
|
| 4 | Doc file count: 55 → clarify scope | Minor | State scope explicitly |
|
|
| 5 | `poke_active_chat.dart` doesn't exist | Critical | Remove or flag as anomaly |
|
|
| 6 | `audio_device_list_tile_test.dart` exists | Major | Update widget test count to 14 |
|
|
| 7 | Architecture header: 4 → 7 | Minor | Fix header |
|
|
| 8 | Governance header: 11 → 12 | Minor | Fix header |
|
|
| 9 | Widget coverage: 54% → 58% | Major | Recalculate |
|