docs: evaluate tsdeclarations codegen, map permissions, verify encoding (TODO-063,068,069)
tsdeclarations eval: direct codegen redundant (already via tsproto), but permission ID generation and error code enum have high value. Permission mapping: 253 TS3 permissions identified, 7 P1 for chat/moderation. Encoding verification: tsclientlib uses strict UTF-8, UCS-2 is edge case.
This commit is contained in:
@@ -0,0 +1,102 @@
|
||||
# tsclientlib Encoding Handling Verification
|
||||
|
||||
**Date:** 2026-06-11
|
||||
**Scope:** Verification of encoding handling in Chanora's tsclientlib fork
|
||||
**Checkout analyzed:** `~/.cargo/git/checkouts/tsclientlib-b9d0d0212095a0c5/3fbfa26/`
|
||||
|
||||
---
|
||||
|
||||
## 1. How tsclientlib Handles Encoding
|
||||
|
||||
### Command Layer (`tsproto-packets/src/commands.rs`)
|
||||
|
||||
The `CommandParser` operates on raw `&[u8]` bytes. String conversion happens at the value level:
|
||||
|
||||
- **`CommandArgumentValue::get_str()`** (line 157): Returns `Result<Cow<'a, str>>` using `str::from_utf8()` on raw bytes or `String::from_utf8()` on unescaped bytes.
|
||||
- **`CommandArgumentValue::get_parse()`** (line 166): Calls `get_str()` then `.parse()` for typed values.
|
||||
- Both methods return `Utf8Error` on invalid UTF-8 — **strict validation, not lossy**.
|
||||
|
||||
### Packet Layer (`tsproto/src/`)
|
||||
|
||||
| File | Line | Usage | Behavior |
|
||||
|------|------|-------|----------|
|
||||
| `license.rs` | 482, 550 | `str::from_utf8(&data[...])` | Strict — returns `Error::DeserializeString` on failure |
|
||||
| `log.rs` | 84, 101 | `if let Ok(s) = str::from_utf8(packet.content())` | Silent skip — invalid UTF-8 packets are not logged |
|
||||
| `client.rs` | 363 | `str::from_utf8(name)` | Used in error message formatting only |
|
||||
| `algorithms.rs` | 450 | `String::from_utf8_lossy(&dec)` | Lossy — only in debug `println!` |
|
||||
|
||||
### Bookkeeping Layer (`utils/ts-bookkeeping/`)
|
||||
|
||||
- All string fields in `Connection`, `Server`, `Client`, `Channel` structs are `String` (Rust's UTF-8 type).
|
||||
- Generated message parsers (via `MessageDeclarations.tt` line 47) use `str::from_utf8(s)?.to_string()` — strict UTF-8 validation.
|
||||
- `ParseError::StringParse` wraps `std::str::Utf8Error` (messages.rs:26).
|
||||
|
||||
---
|
||||
|
||||
## 2. UCS-2 and CESU-8 Handling
|
||||
|
||||
**No UCS-2 or CESU-8 code exists anywhere in the fork.**
|
||||
|
||||
Search for `ucs`, `UCS`, `cesu`, `CESU`, `utf16`, `UTF16`, `utf-16`, `UTF-16` across all `.rs` files returned zero results.
|
||||
|
||||
The fork treats all protocol data as UTF-8 bytes. There is no:
|
||||
- UCS-2 to UTF-8 conversion
|
||||
- CESU-8 decoding
|
||||
- UTF-16 handling
|
||||
- Encoding detection or negotiation
|
||||
|
||||
---
|
||||
|
||||
## 3. What Happens with Non-UTF-8 Data
|
||||
|
||||
If the server sends non-UTF-8 bytes (e.g., UCS-2 encoded emoji from an older client):
|
||||
|
||||
| Layer | Behavior |
|
||||
|-------|----------|
|
||||
| `CommandParser::get_str()` | Returns `Err(Utf8Error)` — call site decides |
|
||||
| License parsing | Returns `Err(Error::DeserializeString)` — connection fails |
|
||||
| Log parsing | `if let Ok(s)` — silently skipped, packet not logged |
|
||||
| Message parsing | Returns `Err(ParseError::StringParse)` — command rejected |
|
||||
|
||||
**Critical:** There is no graceful fallback. Non-UTF-8 data causes errors at the protocol layer.
|
||||
|
||||
---
|
||||
|
||||
## 4. What the TS3 Protocol Actually Sends
|
||||
|
||||
Per ReSpeak protocol reference §1.8.3: "Command string encoded in UTF-8"
|
||||
|
||||
The TS3 protocol itself uses UTF-8 for command strings. The UCS-2 concern applies to:
|
||||
- **Older mobile clients** (pre-3.2.0) that may send CESU-8 encoded supplementary characters
|
||||
- **Server-side storage** where some servers may store UCS-2 for certain fields
|
||||
- **Emoji support** added in Server 3.2.0+ (UTF-8 with full Unicode)
|
||||
|
||||
In practice, the protocol wire format is UTF-8. The UCS-2 issue is a server/client implementation detail, not a protocol encoding issue.
|
||||
|
||||
---
|
||||
|
||||
## 5. Risks for Chanora
|
||||
|
||||
### Low Risk
|
||||
- **Normal operation:** Modern TS3 servers and clients send valid UTF-8. No issues expected.
|
||||
- **Chanora's own strings:** Always valid UTF-8 (Rust `String` type guarantees this).
|
||||
|
||||
### Medium Risk
|
||||
- **Legacy clients:** Older mobile clients sending CESU-8 will cause `Utf8Error` in command parsing. Commands will be rejected, not silently corrupted.
|
||||
- **Server names/channel names with emoji:** If a server stores names in UCS-2 format (some older databases), Chanora will fail to parse them.
|
||||
|
||||
### Mitigation
|
||||
- The fork uses **strict** UTF-8 validation (not lossy). This is correct behavior — rejecting malformed data is safer than silently corrupting it.
|
||||
- If UCS-2 compatibility is needed, the fix belongs in the `EdisonJwa/tsclientlib` fork, not in Chanora's protocol layer.
|
||||
|
||||
---
|
||||
|
||||
## 6. Conclusion
|
||||
|
||||
tsclientlib handles encoding correctly for the TS3 protocol:
|
||||
- All string parsing uses `str::from_utf8()` — strict UTF-8 validation
|
||||
- No UCS-2/CESU-8 code exists — the fork assumes UTF-8 wire format
|
||||
- Invalid UTF-8 causes errors, not silent corruption
|
||||
- The protocol itself is UTF-8; UCS-2 is an edge case from legacy implementations
|
||||
|
||||
**Status:** VERIFIED — encoding handling is correct as-is.
|
||||
Reference in New Issue
Block a user