feat(poc/diagnostics): add diagnostics-redaction spike

Proof-of-concept proving the diagnostics-redaction exit criterion from
docs/architecture/proof-of-concept-plan.md §2:
  "Password and identity-secret samples are redacted."

Full coverage of the audit-report test matrix in
docs/security/diagnostic-redaction-audit-report.md §4
(REDACT-TC-001..010), plus two sanity tests.

The spike ships:
  - RedactionPolicy: typed catalogue of regex rules
    (identity-base64-blob, password-kv, ts3server-url-password,
     authorization-bearer, linux/windows/macos user-path) with
    optional capture-group narrowing.
  - Structured-field redaction keyed on case-insensitive name
    substrings (password, secret, token, ...).
  - Bundle-level switches: chat and channel tree excluded by
    default per audit-report §5.
  - KnownSecretRegistry: literal-substring scrub for secrets the
    host application has already loaded into memory (defense in
    depth that regexes alone cannot guarantee — closes the gap
    behind REDACT-TC-002).
  - Length cap (MAX_PROTOCOL_STRING_LEN = 256) with truncation
    marker for REDACT-TC-009.
  - UTF-8 preserved in non-sensitive fields per REDACT-TC-010 /
    ADR-008.

Test suite (12/12 PASS on 2026-05-13):
  REDACT-TC-001 server password in connection data
  REDACT-TC-002 identity secret in storage error (via KnownSecretRegistry)
  REDACT-TC-003 server URL with password field
  REDACT-TC-004 chat text excluded by default
  REDACT-TC-005 channel name with Unicode excluded by default
  REDACT-TC-006 nickname with Unicode preserved in safe field
  REDACT-TC-007 local file path user segment minimized
  REDACT-TC-008 mixed sensitive bundle (whole-bundle JSON scan)
  REDACT-TC-009 long hostile protocol string truncated
  REDACT-TC-010 multilingual safe text preserved
  + known-secret literal scrub
  + empty registered secret ignored

Out of scope: tracing-subscriber integration, diagnostic export
file format, memory/core dumps, performance, adversarial regex
evasion beyond trivial cases. These belong to chanora_diagnostics.

Authority: PoC plan §2, docs/security/diagnostic-redaction-audit-report.md,
SRS-093, SysRS-152/154/155.
Not product code; not promoted into chanora_diagnostics.
This commit is contained in:
EdisonJwa
2026-05-14 12:26:49 +08:00
parent 52e8d43f69
commit 06ec6f2965
10 changed files with 1073 additions and 0 deletions
@@ -0,0 +1,96 @@
//! Structured diagnostic bundle and the redaction pass over it.
//!
//! Mirrors `docs/security/diagnostic-redaction-audit-report.md` §5
//! "Export Bundle Contents". Fields default to safe values; sensitive
//! categories are excluded by default.
use serde::{Deserialize, Serialize};
use crate::redactor::Redactor;
use crate::REDACTION_MARKER;
/// Raw bundle assembled by the application before redaction. The
/// caller MUST pass this through `Redactor::redact_bundle` before any
/// disk write or user-visible export.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct DiagnosticBundle {
pub app_version: String,
pub build_number: String,
pub platform_info: String,
pub connection_state: String,
pub server_address: Option<String>,
pub channel_tree: Vec<String>,
pub chat_history: Vec<String>,
pub log_lines: Vec<String>,
pub audio_device_names: Vec<String>,
/// Free-form extras keyed by field name. Sensitive field names
/// (matched against `RedactionPolicy::redact_field_substrings`)
/// are redacted wholesale; everything else flows through
/// `redact_text` for value-level scrubbing.
pub extras: std::collections::BTreeMap<String, String>,
}
/// Final, redacted bundle suitable for export. Produced by
/// `Redactor::redact_bundle`. The shape is intentionally the same
/// type so callers can re-serialize as JSON without conversion.
pub type RedactedBundle = DiagnosticBundle;
impl Redactor {
/// Apply the redaction policy to `bundle`, returning a copy safe
/// for export.
pub fn redact_bundle(&self, bundle: &DiagnosticBundle) -> RedactedBundle {
let policy = self.policy();
let chat_history = if policy.include_chat {
bundle
.chat_history
.iter()
.map(|m| self.redact_text(m))
.collect()
} else {
// Chat is excluded by default (audit report §5 / REDACT-TC-004).
Vec::new()
};
let channel_tree = if policy.include_channel_tree {
bundle
.channel_tree
.iter()
.map(|c| self.redact_text(c))
.collect()
} else {
Vec::new()
};
let mut redacted_extras = std::collections::BTreeMap::new();
for (k, v) in &bundle.extras {
if self.is_sensitive_field(k) {
redacted_extras.insert(k.clone(), REDACTION_MARKER.to_string());
} else {
redacted_extras.insert(k.clone(), self.redact_text(v));
}
}
RedactedBundle {
app_version: bundle.app_version.clone(),
build_number: bundle.build_number.clone(),
// Platform info gets a soft scrub (paths, usernames).
platform_info: self.redact_text(&bundle.platform_info),
connection_state: self.redact_text(&bundle.connection_state),
server_address: bundle.server_address.as_ref().map(|s| self.redact_text(s)),
channel_tree,
chat_history,
log_lines: bundle
.log_lines
.iter()
.map(|l| self.redact_text(l))
.collect(),
audio_device_names: bundle
.audio_device_names
.iter()
.map(|d| self.redact_text(d))
.collect(),
extras: redacted_extras,
}
}
}
@@ -0,0 +1,31 @@
//! Chanora PoC — diagnostics redaction spike.
//!
//! Authority:
//! * `docs/architecture/proof-of-concept-plan.md` §2 — Diagnostics
//! redaction spike. Exit criterion: "Password and identity-secret
//! samples are redacted."
//! * `docs/security/diagnostic-redaction-audit-report.md` — policy
//! in §2, surfaces in §3, test matrix REDACT-TC-001..010 in §4,
//! bundle policy in §5.
//! * SRS-093 / SysRS-152 / SysRS-154 / SysRS-155 — software-level
//! redaction obligations for logs and diagnostic exports.
//!
//! Production code will own this redactor inside the `chanora_diagnostics`
//! crate and wire it as a `tracing` layer. The PoC is the same shape in
//! miniature: a typed policy, a redactor that applies it to free text
//! and to structured diagnostic bundles, plus an explicit list of
//! known-literal secrets the host application can register at runtime.
pub mod bundle;
pub mod policy;
pub mod redactor;
pub use bundle::{DiagnosticBundle, RedactedBundle};
pub use policy::{RedactionPolicy, RedactionRule};
pub use redactor::{KnownSecretRegistry, Redactor};
/// The replacement string used for redacted segments.
///
/// Chosen distinctly so audit tests can grep for either presence
/// (correct redaction occurred) or absence (no leak).
pub const REDACTION_MARKER: &str = "[REDACTED]";
@@ -0,0 +1,30 @@
//! CLI driver — accepts mixed sensitive input on stdin, prints the
//! redacted version on stdout, and prints a summary on stderr.
use std::io::Read;
use std::process::ExitCode;
use diagnostics_redaction_spike::Redactor;
fn main() -> ExitCode {
let mut input = String::new();
if std::io::stdin().read_to_string(&mut input).is_err() {
eprintln!("failed to read stdin");
return ExitCode::from(1);
}
let redactor = Redactor::with_default_policy();
// The host application would normally register live secrets here
// (e.g. as the secure-storage layer materialises them).
// For the CLI demo, register a fixed plaintext so the audience
// can see literal scrubbing work.
redactor
.known_secrets()
.register("CHANORA_LITERAL_DEMO_SECRET");
let out = redactor.redact_text(&input);
println!("{out}");
eprintln!("--- redaction applied; rules in policy: {} ---", redactor.policy().rules.len());
ExitCode::SUCCESS
}
@@ -0,0 +1,153 @@
//! Redaction policy: the catalogue of patterns and structural rules
//! the redactor applies.
use once_cell::sync::Lazy;
use regex::Regex;
/// A single regex-driven rule.
#[derive(Debug, Clone)]
pub struct RedactionRule {
pub id: &'static str,
pub description: &'static str,
pub pattern: Regex,
/// If `Some(n)`, the rule replaces capture-group `n` rather than
/// the whole match. Useful for "ts3server://host?password=XXXX"
/// where the host should be preserved but `XXXX` redacted.
pub redact_group: Option<usize>,
}
impl RedactionRule {
pub fn new(
id: &'static str,
description: &'static str,
pattern: &str,
) -> Self {
Self {
id,
description,
pattern: Regex::new(pattern).expect("static rule regex must compile"),
redact_group: None,
}
}
pub fn new_group(
id: &'static str,
description: &'static str,
pattern: &str,
group: usize,
) -> Self {
Self {
id,
description,
pattern: Regex::new(pattern).expect("static rule regex must compile"),
redact_group: Some(group),
}
}
}
/// The full redaction policy.
#[derive(Debug, Clone)]
pub struct RedactionPolicy {
pub rules: Vec<RedactionRule>,
/// Structured-field redaction: any key whose lower-cased name
/// contains one of these substrings is fully redacted in
/// diagnostic bundles. Mirrors the bundle policy in
/// `diagnostic-redaction-audit-report.md` §5.
pub redact_field_substrings: Vec<&'static str>,
/// Bundle-level: include chat messages? Audit-report default = No.
pub include_chat: bool,
/// Bundle-level: include channel tree details?
pub include_channel_tree: bool,
}
/// Hard upper bound on protocol-string lengths kept in diagnostics.
///
/// REDACT-TC-009 ("long hostile protocol string"): truncate or safely
/// escape. The PoC truncates with a clear marker.
pub const MAX_PROTOCOL_STRING_LEN: usize = 256;
impl RedactionPolicy {
/// The default policy used by the PoC. Mirrors the audit-report
/// catalogue. Production code (`chanora_diagnostics`) will own the
/// canonical version of this list.
pub fn default_policy() -> Self {
Self {
rules: DEFAULT_RULES.clone(),
redact_field_substrings: vec![
"password",
"secret",
"private_key",
"private-key",
"identity_key",
"identity-key",
"token",
"credential",
"passphrase",
],
include_chat: false,
include_channel_tree: false,
}
}
}
static DEFAULT_RULES: Lazy<Vec<RedactionRule>> = Lazy::new(|| {
vec![
// Identity secret as it appears in `tsclientlib` style:
// long base64-ish run preceded by a known marker.
// We err on the side of catching base64 blobs >= 64 chars.
RedactionRule::new(
"identity-base64-blob",
"Long base64-like run; matches identity secrets and tokens.",
r"\b[A-Za-z0-9+/]{64,}={0,2}\b",
),
// `password = "..."` / `password: "..."` / `password=...`
// Captures the value in group 1 so the *key* name is kept.
// The value excludes separators commonly found in URL query
// strings and config lines (`& , ; " whitespace`).
RedactionRule::new_group(
"password-kv",
"key=value style password assignment.",
r#"(?i)\b(?:password|passwd|pass|pwd)\s*[:=]\s*"?([^"\s,;&]+)"?"#,
1,
),
// ts3server://host?password=XXXX&...
RedactionRule::new_group(
"ts3server-url-password",
"Password embedded in a ts3server:// URL query string.",
r"(?i)(?:[?&])password=([^&\s]+)",
1,
),
// Generic Authorization: Bearer ...
RedactionRule::new_group(
"authorization-bearer",
"Authorization header bearer token.",
r"(?i)Authorization:\s*Bearer\s+(\S+)",
1,
),
// Linux user home: /home/<user>/... → minimize the username
// segment. Captures group 1 = username.
RedactionRule::new_group(
"linux-home-path",
"Linux home-directory path; minimizes the username segment.",
r"(/home/)([^/\s]+)",
2,
),
// Windows user: C:\Users\<user>\...
RedactionRule::new_group(
"windows-user-path",
"Windows user-profile path; minimizes the username segment.",
r"(?i)([A-Z]:\\Users\\)([^\\\s]+)",
2,
),
// macOS user: /Users/<user>/...
RedactionRule::new_group(
"macos-user-path",
"macOS user-profile path; minimizes the username segment.",
r"(/Users/)([^/\s]+)",
2,
),
]
});
@@ -0,0 +1,127 @@
//! The redactor — applies the policy to free text and to structured
//! diagnostic data.
use std::collections::HashSet;
use std::sync::{Arc, RwLock};
use crate::policy::{RedactionPolicy, MAX_PROTOCOL_STRING_LEN};
use crate::REDACTION_MARKER;
/// Runtime registry of *literal* known secrets. The host application
/// (e.g. the secure-storage layer) registers a secret here when it
/// loads one into memory, so even if it slips into a log line verbatim
/// it gets scrubbed.
///
/// This is the strongest defense for SS-AUD-003 and REDACT-TC-002 —
/// regexes can miss; literal substring matches cannot.
#[derive(Debug, Clone, Default)]
pub struct KnownSecretRegistry {
inner: Arc<RwLock<HashSet<String>>>,
}
impl KnownSecretRegistry {
pub fn new() -> Self {
Self::default()
}
/// Register a literal secret string. Empty strings are ignored
/// (to avoid degenerate `s.replace("", _)` behaviour).
pub fn register(&self, secret: &str) {
if secret.is_empty() {
return;
}
self.inner.write().unwrap().insert(secret.to_string());
}
pub fn forget(&self, secret: &str) {
self.inner.write().unwrap().remove(secret);
}
pub fn snapshot(&self) -> Vec<String> {
self.inner.read().unwrap().iter().cloned().collect()
}
}
pub struct Redactor {
policy: RedactionPolicy,
known: KnownSecretRegistry,
}
impl Redactor {
pub fn new(policy: RedactionPolicy, known: KnownSecretRegistry) -> Self {
Self { policy, known }
}
pub fn with_default_policy() -> Self {
Self::new(RedactionPolicy::default_policy(), KnownSecretRegistry::new())
}
pub fn known_secrets(&self) -> &KnownSecretRegistry {
&self.known
}
pub fn policy(&self) -> &RedactionPolicy {
&self.policy
}
/// Redact `input` according to the policy + known secret registry.
///
/// Order:
/// 1. Known literal secrets (defence in depth — never miss a value).
/// 2. Regex rules (with optional capture-group narrowing).
/// 3. Length cap for hostile / oversized strings (REDACT-TC-009).
pub fn redact_text(&self, input: &str) -> String {
// 1. Literal known secrets.
let mut text = input.to_string();
for known in self.known.snapshot() {
if !known.is_empty() {
text = text.replace(&known, REDACTION_MARKER);
}
}
// 2. Regex rules.
for rule in &self.policy.rules {
text = match rule.redact_group {
None => rule
.pattern
.replace_all(&text, REDACTION_MARKER)
.into_owned(),
Some(group) => rule
.pattern
.replace_all(&text, |caps: &regex::Captures<'_>| {
let full = caps.get(0).map(|m| m.as_str()).unwrap_or("");
match caps.get(group) {
Some(target) => full.replace(target.as_str(), REDACTION_MARKER),
None => full.to_string(),
}
})
.into_owned(),
};
}
// 3. Length cap.
if text.len() > MAX_PROTOCOL_STRING_LEN {
// Find the last char boundary inside the budget so we
// never split a UTF-8 codepoint.
let mut cut = MAX_PROTOCOL_STRING_LEN;
while cut > 0 && !text.is_char_boundary(cut) {
cut -= 1;
}
let mut truncated = text[..cut].to_string();
truncated.push_str("…[truncated]");
text = truncated;
}
text
}
/// True if the policy classifies `field_name` as a sensitive
/// structured field that must be redacted wholesale.
pub fn is_sensitive_field(&self, field_name: &str) -> bool {
let lower = field_name.to_lowercase();
self.policy
.redact_field_substrings
.iter()
.any(|needle| lower.contains(needle))
}
}