feat(poc/diagnostics): add diagnostics-redaction spike
Proof-of-concept proving the diagnostics-redaction exit criterion from
docs/architecture/proof-of-concept-plan.md §2:
"Password and identity-secret samples are redacted."
Full coverage of the audit-report test matrix in
docs/security/diagnostic-redaction-audit-report.md §4
(REDACT-TC-001..010), plus two sanity tests.
The spike ships:
- RedactionPolicy: typed catalogue of regex rules
(identity-base64-blob, password-kv, ts3server-url-password,
authorization-bearer, linux/windows/macos user-path) with
optional capture-group narrowing.
- Structured-field redaction keyed on case-insensitive name
substrings (password, secret, token, ...).
- Bundle-level switches: chat and channel tree excluded by
default per audit-report §5.
- KnownSecretRegistry: literal-substring scrub for secrets the
host application has already loaded into memory (defense in
depth that regexes alone cannot guarantee — closes the gap
behind REDACT-TC-002).
- Length cap (MAX_PROTOCOL_STRING_LEN = 256) with truncation
marker for REDACT-TC-009.
- UTF-8 preserved in non-sensitive fields per REDACT-TC-010 /
ADR-008.
Test suite (12/12 PASS on 2026-05-13):
REDACT-TC-001 server password in connection data
REDACT-TC-002 identity secret in storage error (via KnownSecretRegistry)
REDACT-TC-003 server URL with password field
REDACT-TC-004 chat text excluded by default
REDACT-TC-005 channel name with Unicode excluded by default
REDACT-TC-006 nickname with Unicode preserved in safe field
REDACT-TC-007 local file path user segment minimized
REDACT-TC-008 mixed sensitive bundle (whole-bundle JSON scan)
REDACT-TC-009 long hostile protocol string truncated
REDACT-TC-010 multilingual safe text preserved
+ known-secret literal scrub
+ empty registered secret ignored
Out of scope: tracing-subscriber integration, diagnostic export
file format, memory/core dumps, performance, adversarial regex
evasion beyond trivial cases. These belong to chanora_diagnostics.
Authority: PoC plan §2, docs/security/diagnostic-redaction-audit-report.md,
SRS-093, SysRS-152/154/155.
Not product code; not promoted into chanora_diagnostics.
This commit is contained in:
@@ -0,0 +1,96 @@
|
||||
//! Structured diagnostic bundle and the redaction pass over it.
|
||||
//!
|
||||
//! Mirrors `docs/security/diagnostic-redaction-audit-report.md` §5
|
||||
//! "Export Bundle Contents". Fields default to safe values; sensitive
|
||||
//! categories are excluded by default.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::redactor::Redactor;
|
||||
use crate::REDACTION_MARKER;
|
||||
|
||||
/// Raw bundle assembled by the application before redaction. The
|
||||
/// caller MUST pass this through `Redactor::redact_bundle` before any
|
||||
/// disk write or user-visible export.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
|
||||
pub struct DiagnosticBundle {
|
||||
pub app_version: String,
|
||||
pub build_number: String,
|
||||
pub platform_info: String,
|
||||
pub connection_state: String,
|
||||
pub server_address: Option<String>,
|
||||
pub channel_tree: Vec<String>,
|
||||
pub chat_history: Vec<String>,
|
||||
pub log_lines: Vec<String>,
|
||||
pub audio_device_names: Vec<String>,
|
||||
/// Free-form extras keyed by field name. Sensitive field names
|
||||
/// (matched against `RedactionPolicy::redact_field_substrings`)
|
||||
/// are redacted wholesale; everything else flows through
|
||||
/// `redact_text` for value-level scrubbing.
|
||||
pub extras: std::collections::BTreeMap<String, String>,
|
||||
}
|
||||
|
||||
/// Final, redacted bundle suitable for export. Produced by
|
||||
/// `Redactor::redact_bundle`. The shape is intentionally the same
|
||||
/// type so callers can re-serialize as JSON without conversion.
|
||||
pub type RedactedBundle = DiagnosticBundle;
|
||||
|
||||
impl Redactor {
|
||||
/// Apply the redaction policy to `bundle`, returning a copy safe
|
||||
/// for export.
|
||||
pub fn redact_bundle(&self, bundle: &DiagnosticBundle) -> RedactedBundle {
|
||||
let policy = self.policy();
|
||||
|
||||
let chat_history = if policy.include_chat {
|
||||
bundle
|
||||
.chat_history
|
||||
.iter()
|
||||
.map(|m| self.redact_text(m))
|
||||
.collect()
|
||||
} else {
|
||||
// Chat is excluded by default (audit report §5 / REDACT-TC-004).
|
||||
Vec::new()
|
||||
};
|
||||
|
||||
let channel_tree = if policy.include_channel_tree {
|
||||
bundle
|
||||
.channel_tree
|
||||
.iter()
|
||||
.map(|c| self.redact_text(c))
|
||||
.collect()
|
||||
} else {
|
||||
Vec::new()
|
||||
};
|
||||
|
||||
let mut redacted_extras = std::collections::BTreeMap::new();
|
||||
for (k, v) in &bundle.extras {
|
||||
if self.is_sensitive_field(k) {
|
||||
redacted_extras.insert(k.clone(), REDACTION_MARKER.to_string());
|
||||
} else {
|
||||
redacted_extras.insert(k.clone(), self.redact_text(v));
|
||||
}
|
||||
}
|
||||
|
||||
RedactedBundle {
|
||||
app_version: bundle.app_version.clone(),
|
||||
build_number: bundle.build_number.clone(),
|
||||
// Platform info gets a soft scrub (paths, usernames).
|
||||
platform_info: self.redact_text(&bundle.platform_info),
|
||||
connection_state: self.redact_text(&bundle.connection_state),
|
||||
server_address: bundle.server_address.as_ref().map(|s| self.redact_text(s)),
|
||||
channel_tree,
|
||||
chat_history,
|
||||
log_lines: bundle
|
||||
.log_lines
|
||||
.iter()
|
||||
.map(|l| self.redact_text(l))
|
||||
.collect(),
|
||||
audio_device_names: bundle
|
||||
.audio_device_names
|
||||
.iter()
|
||||
.map(|d| self.redact_text(d))
|
||||
.collect(),
|
||||
extras: redacted_extras,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
//! Chanora PoC — diagnostics redaction spike.
|
||||
//!
|
||||
//! Authority:
|
||||
//! * `docs/architecture/proof-of-concept-plan.md` §2 — Diagnostics
|
||||
//! redaction spike. Exit criterion: "Password and identity-secret
|
||||
//! samples are redacted."
|
||||
//! * `docs/security/diagnostic-redaction-audit-report.md` — policy
|
||||
//! in §2, surfaces in §3, test matrix REDACT-TC-001..010 in §4,
|
||||
//! bundle policy in §5.
|
||||
//! * SRS-093 / SysRS-152 / SysRS-154 / SysRS-155 — software-level
|
||||
//! redaction obligations for logs and diagnostic exports.
|
||||
//!
|
||||
//! Production code will own this redactor inside the `chanora_diagnostics`
|
||||
//! crate and wire it as a `tracing` layer. The PoC is the same shape in
|
||||
//! miniature: a typed policy, a redactor that applies it to free text
|
||||
//! and to structured diagnostic bundles, plus an explicit list of
|
||||
//! known-literal secrets the host application can register at runtime.
|
||||
|
||||
pub mod bundle;
|
||||
pub mod policy;
|
||||
pub mod redactor;
|
||||
|
||||
pub use bundle::{DiagnosticBundle, RedactedBundle};
|
||||
pub use policy::{RedactionPolicy, RedactionRule};
|
||||
pub use redactor::{KnownSecretRegistry, Redactor};
|
||||
|
||||
/// The replacement string used for redacted segments.
|
||||
///
|
||||
/// Chosen distinctly so audit tests can grep for either presence
|
||||
/// (correct redaction occurred) or absence (no leak).
|
||||
pub const REDACTION_MARKER: &str = "[REDACTED]";
|
||||
@@ -0,0 +1,30 @@
|
||||
//! CLI driver — accepts mixed sensitive input on stdin, prints the
|
||||
//! redacted version on stdout, and prints a summary on stderr.
|
||||
|
||||
use std::io::Read;
|
||||
use std::process::ExitCode;
|
||||
|
||||
use diagnostics_redaction_spike::Redactor;
|
||||
|
||||
fn main() -> ExitCode {
|
||||
let mut input = String::new();
|
||||
if std::io::stdin().read_to_string(&mut input).is_err() {
|
||||
eprintln!("failed to read stdin");
|
||||
return ExitCode::from(1);
|
||||
}
|
||||
|
||||
let redactor = Redactor::with_default_policy();
|
||||
|
||||
// The host application would normally register live secrets here
|
||||
// (e.g. as the secure-storage layer materialises them).
|
||||
// For the CLI demo, register a fixed plaintext so the audience
|
||||
// can see literal scrubbing work.
|
||||
redactor
|
||||
.known_secrets()
|
||||
.register("CHANORA_LITERAL_DEMO_SECRET");
|
||||
|
||||
let out = redactor.redact_text(&input);
|
||||
println!("{out}");
|
||||
eprintln!("--- redaction applied; rules in policy: {} ---", redactor.policy().rules.len());
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
//! Redaction policy: the catalogue of patterns and structural rules
|
||||
//! the redactor applies.
|
||||
|
||||
use once_cell::sync::Lazy;
|
||||
use regex::Regex;
|
||||
|
||||
/// A single regex-driven rule.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RedactionRule {
|
||||
pub id: &'static str,
|
||||
pub description: &'static str,
|
||||
pub pattern: Regex,
|
||||
/// If `Some(n)`, the rule replaces capture-group `n` rather than
|
||||
/// the whole match. Useful for "ts3server://host?password=XXXX"
|
||||
/// where the host should be preserved but `XXXX` redacted.
|
||||
pub redact_group: Option<usize>,
|
||||
}
|
||||
|
||||
impl RedactionRule {
|
||||
pub fn new(
|
||||
id: &'static str,
|
||||
description: &'static str,
|
||||
pattern: &str,
|
||||
) -> Self {
|
||||
Self {
|
||||
id,
|
||||
description,
|
||||
pattern: Regex::new(pattern).expect("static rule regex must compile"),
|
||||
redact_group: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn new_group(
|
||||
id: &'static str,
|
||||
description: &'static str,
|
||||
pattern: &str,
|
||||
group: usize,
|
||||
) -> Self {
|
||||
Self {
|
||||
id,
|
||||
description,
|
||||
pattern: Regex::new(pattern).expect("static rule regex must compile"),
|
||||
redact_group: Some(group),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The full redaction policy.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RedactionPolicy {
|
||||
pub rules: Vec<RedactionRule>,
|
||||
|
||||
/// Structured-field redaction: any key whose lower-cased name
|
||||
/// contains one of these substrings is fully redacted in
|
||||
/// diagnostic bundles. Mirrors the bundle policy in
|
||||
/// `diagnostic-redaction-audit-report.md` §5.
|
||||
pub redact_field_substrings: Vec<&'static str>,
|
||||
|
||||
/// Bundle-level: include chat messages? Audit-report default = No.
|
||||
pub include_chat: bool,
|
||||
|
||||
/// Bundle-level: include channel tree details?
|
||||
pub include_channel_tree: bool,
|
||||
}
|
||||
|
||||
/// Hard upper bound on protocol-string lengths kept in diagnostics.
|
||||
///
|
||||
/// REDACT-TC-009 ("long hostile protocol string"): truncate or safely
|
||||
/// escape. The PoC truncates with a clear marker.
|
||||
pub const MAX_PROTOCOL_STRING_LEN: usize = 256;
|
||||
|
||||
impl RedactionPolicy {
|
||||
/// The default policy used by the PoC. Mirrors the audit-report
|
||||
/// catalogue. Production code (`chanora_diagnostics`) will own the
|
||||
/// canonical version of this list.
|
||||
pub fn default_policy() -> Self {
|
||||
Self {
|
||||
rules: DEFAULT_RULES.clone(),
|
||||
redact_field_substrings: vec![
|
||||
"password",
|
||||
"secret",
|
||||
"private_key",
|
||||
"private-key",
|
||||
"identity_key",
|
||||
"identity-key",
|
||||
"token",
|
||||
"credential",
|
||||
"passphrase",
|
||||
],
|
||||
include_chat: false,
|
||||
include_channel_tree: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static DEFAULT_RULES: Lazy<Vec<RedactionRule>> = Lazy::new(|| {
|
||||
vec![
|
||||
// Identity secret as it appears in `tsclientlib` style:
|
||||
// long base64-ish run preceded by a known marker.
|
||||
// We err on the side of catching base64 blobs >= 64 chars.
|
||||
RedactionRule::new(
|
||||
"identity-base64-blob",
|
||||
"Long base64-like run; matches identity secrets and tokens.",
|
||||
r"\b[A-Za-z0-9+/]{64,}={0,2}\b",
|
||||
),
|
||||
// `password = "..."` / `password: "..."` / `password=...`
|
||||
// Captures the value in group 1 so the *key* name is kept.
|
||||
// The value excludes separators commonly found in URL query
|
||||
// strings and config lines (`& , ; " whitespace`).
|
||||
RedactionRule::new_group(
|
||||
"password-kv",
|
||||
"key=value style password assignment.",
|
||||
r#"(?i)\b(?:password|passwd|pass|pwd)\s*[:=]\s*"?([^"\s,;&]+)"?"#,
|
||||
1,
|
||||
),
|
||||
// ts3server://host?password=XXXX&...
|
||||
RedactionRule::new_group(
|
||||
"ts3server-url-password",
|
||||
"Password embedded in a ts3server:// URL query string.",
|
||||
r"(?i)(?:[?&])password=([^&\s]+)",
|
||||
1,
|
||||
),
|
||||
// Generic Authorization: Bearer ...
|
||||
RedactionRule::new_group(
|
||||
"authorization-bearer",
|
||||
"Authorization header bearer token.",
|
||||
r"(?i)Authorization:\s*Bearer\s+(\S+)",
|
||||
1,
|
||||
),
|
||||
// Linux user home: /home/<user>/... → minimize the username
|
||||
// segment. Captures group 1 = username.
|
||||
RedactionRule::new_group(
|
||||
"linux-home-path",
|
||||
"Linux home-directory path; minimizes the username segment.",
|
||||
r"(/home/)([^/\s]+)",
|
||||
2,
|
||||
),
|
||||
// Windows user: C:\Users\<user>\...
|
||||
RedactionRule::new_group(
|
||||
"windows-user-path",
|
||||
"Windows user-profile path; minimizes the username segment.",
|
||||
r"(?i)([A-Z]:\\Users\\)([^\\\s]+)",
|
||||
2,
|
||||
),
|
||||
// macOS user: /Users/<user>/...
|
||||
RedactionRule::new_group(
|
||||
"macos-user-path",
|
||||
"macOS user-profile path; minimizes the username segment.",
|
||||
r"(/Users/)([^/\s]+)",
|
||||
2,
|
||||
),
|
||||
]
|
||||
});
|
||||
@@ -0,0 +1,127 @@
|
||||
//! The redactor — applies the policy to free text and to structured
|
||||
//! diagnostic data.
|
||||
|
||||
use std::collections::HashSet;
|
||||
use std::sync::{Arc, RwLock};
|
||||
|
||||
use crate::policy::{RedactionPolicy, MAX_PROTOCOL_STRING_LEN};
|
||||
use crate::REDACTION_MARKER;
|
||||
|
||||
/// Runtime registry of *literal* known secrets. The host application
|
||||
/// (e.g. the secure-storage layer) registers a secret here when it
|
||||
/// loads one into memory, so even if it slips into a log line verbatim
|
||||
/// it gets scrubbed.
|
||||
///
|
||||
/// This is the strongest defense for SS-AUD-003 and REDACT-TC-002 —
|
||||
/// regexes can miss; literal substring matches cannot.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct KnownSecretRegistry {
|
||||
inner: Arc<RwLock<HashSet<String>>>,
|
||||
}
|
||||
|
||||
impl KnownSecretRegistry {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Register a literal secret string. Empty strings are ignored
|
||||
/// (to avoid degenerate `s.replace("", _)` behaviour).
|
||||
pub fn register(&self, secret: &str) {
|
||||
if secret.is_empty() {
|
||||
return;
|
||||
}
|
||||
self.inner.write().unwrap().insert(secret.to_string());
|
||||
}
|
||||
|
||||
pub fn forget(&self, secret: &str) {
|
||||
self.inner.write().unwrap().remove(secret);
|
||||
}
|
||||
|
||||
pub fn snapshot(&self) -> Vec<String> {
|
||||
self.inner.read().unwrap().iter().cloned().collect()
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Redactor {
|
||||
policy: RedactionPolicy,
|
||||
known: KnownSecretRegistry,
|
||||
}
|
||||
|
||||
impl Redactor {
|
||||
pub fn new(policy: RedactionPolicy, known: KnownSecretRegistry) -> Self {
|
||||
Self { policy, known }
|
||||
}
|
||||
|
||||
pub fn with_default_policy() -> Self {
|
||||
Self::new(RedactionPolicy::default_policy(), KnownSecretRegistry::new())
|
||||
}
|
||||
|
||||
pub fn known_secrets(&self) -> &KnownSecretRegistry {
|
||||
&self.known
|
||||
}
|
||||
|
||||
pub fn policy(&self) -> &RedactionPolicy {
|
||||
&self.policy
|
||||
}
|
||||
|
||||
/// Redact `input` according to the policy + known secret registry.
|
||||
///
|
||||
/// Order:
|
||||
/// 1. Known literal secrets (defence in depth — never miss a value).
|
||||
/// 2. Regex rules (with optional capture-group narrowing).
|
||||
/// 3. Length cap for hostile / oversized strings (REDACT-TC-009).
|
||||
pub fn redact_text(&self, input: &str) -> String {
|
||||
// 1. Literal known secrets.
|
||||
let mut text = input.to_string();
|
||||
for known in self.known.snapshot() {
|
||||
if !known.is_empty() {
|
||||
text = text.replace(&known, REDACTION_MARKER);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Regex rules.
|
||||
for rule in &self.policy.rules {
|
||||
text = match rule.redact_group {
|
||||
None => rule
|
||||
.pattern
|
||||
.replace_all(&text, REDACTION_MARKER)
|
||||
.into_owned(),
|
||||
Some(group) => rule
|
||||
.pattern
|
||||
.replace_all(&text, |caps: ®ex::Captures<'_>| {
|
||||
let full = caps.get(0).map(|m| m.as_str()).unwrap_or("");
|
||||
match caps.get(group) {
|
||||
Some(target) => full.replace(target.as_str(), REDACTION_MARKER),
|
||||
None => full.to_string(),
|
||||
}
|
||||
})
|
||||
.into_owned(),
|
||||
};
|
||||
}
|
||||
|
||||
// 3. Length cap.
|
||||
if text.len() > MAX_PROTOCOL_STRING_LEN {
|
||||
// Find the last char boundary inside the budget so we
|
||||
// never split a UTF-8 codepoint.
|
||||
let mut cut = MAX_PROTOCOL_STRING_LEN;
|
||||
while cut > 0 && !text.is_char_boundary(cut) {
|
||||
cut -= 1;
|
||||
}
|
||||
let mut truncated = text[..cut].to_string();
|
||||
truncated.push_str("…[truncated]");
|
||||
text = truncated;
|
||||
}
|
||||
|
||||
text
|
||||
}
|
||||
|
||||
/// True if the policy classifies `field_name` as a sensitive
|
||||
/// structured field that must be redacted wholesale.
|
||||
pub fn is_sensitive_field(&self, field_name: &str) -> bool {
|
||||
let lower = field_name.to_lowercase();
|
||||
self.policy
|
||||
.redact_field_substrings
|
||||
.iter()
|
||||
.any(|needle| lower.contains(needle))
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user