# tsclientlib Encoding Handling Verification

**Date:** 2026-06-11
**Scope:** Verification of encoding handling in Chanora's tsclientlib fork
**Checkout analyzed:** `~/.cargo/git/checkouts/tsclientlib-b9d0d0212095a0c5/3fbfa26/`

---

## 1. How tsclientlib Handles Encoding

### Command Layer (`tsproto-packets/src/commands.rs`)

The `CommandParser` operates on raw `&[u8]` bytes. String conversion happens at the value level:

- **`CommandArgumentValue::get_str()`** (line 157): Returns `Result<Cow<'a, str>>` using `str::from_utf8()` on raw bytes or `String::from_utf8()` on unescaped bytes.
- **`CommandArgumentValue::get_parse()`** (line 166): Calls `get_str()` then `.parse()` for typed values.
- Both methods return `Utf8Error` on invalid UTF-8 — **strict validation, not lossy**.

### Packet Layer (`tsproto/src/`)

| File | Line | Usage | Behavior |
|------|------|-------|----------|
| `license.rs` | 482, 550 | `str::from_utf8(&data[...])` | Strict — returns `Error::DeserializeString` on failure |
| `log.rs` | 84, 101 | `if let Ok(s) = str::from_utf8(packet.content())` | Silent skip — invalid UTF-8 packets are not logged |
| `client.rs` | 363 | `str::from_utf8(name)` | Used in error message formatting only |
| `algorithms.rs` | 450 | `String::from_utf8_lossy(&dec)` | Lossy — only in debug `println!` |

### Bookkeeping Layer (`utils/ts-bookkeeping/`)

- All string fields in `Connection`, `Server`, `Client`, `Channel` structs are `String` (Rust's UTF-8 type).
- Generated message parsers (via `MessageDeclarations.tt` line 47) use `str::from_utf8(s)?.to_string()` — strict UTF-8 validation.
- `ParseError::StringParse` wraps `std::str::Utf8Error` (messages.rs:26).

---

## 2. UCS-2 and CESU-8 Handling

**No UCS-2 or CESU-8 code exists anywhere in the fork.**

Search for `ucs`, `UCS`, `cesu`, `CESU`, `utf16`, `UTF16`, `utf-16`, `UTF-16` across all `.rs` files returned zero results.

The fork treats all protocol data as UTF-8 bytes. There is no:
- UCS-2 to UTF-8 conversion
- CESU-8 decoding
- UTF-16 handling
- Encoding detection or negotiation

---

## 3. What Happens with Non-UTF-8 Data

If the server sends non-UTF-8 bytes (e.g., UCS-2 encoded emoji from an older client):

| Layer | Behavior |
|-------|----------|
| `CommandParser::get_str()` | Returns `Err(Utf8Error)` — call site decides |
| License parsing | Returns `Err(Error::DeserializeString)` — connection fails |
| Log parsing | `if let Ok(s)` — silently skipped, packet not logged |
| Message parsing | Returns `Err(ParseError::StringParse)` — command rejected |

**Critical:** There is no graceful fallback. Non-UTF-8 data causes errors at the protocol layer.

---

## 4. What the TS3 Protocol Actually Sends

Per ReSpeak protocol reference §1.8.3: "Command string encoded in UTF-8"

The TS3 protocol itself uses UTF-8 for command strings. The UCS-2 concern applies to:
- **Older mobile clients** (pre-3.2.0) that may send CESU-8 encoded supplementary characters
- **Server-side storage** where some servers may store UCS-2 for certain fields
- **Emoji support** added in Server 3.2.0+ (UTF-8 with full Unicode)

In practice, the protocol wire format is UTF-8. The UCS-2 issue is a server/client implementation detail, not a protocol encoding issue.

---

## 5. Risks for Chanora

### Low Risk
- **Normal operation:** Modern TS3 servers and clients send valid UTF-8. No issues expected.
- **Chanora's own strings:** Always valid UTF-8 (Rust `String` type guarantees this).

### Medium Risk
- **Legacy clients:** Older mobile clients sending CESU-8 will cause `Utf8Error` in command parsing. Commands will be rejected, not silently corrupted.
- **Server names/channel names with emoji:** If a server stores names in UCS-2 format (some older databases), Chanora will fail to parse them.

### Mitigation
- The fork uses **strict** UTF-8 validation (not lossy). This is correct behavior — rejecting malformed data is safer than silently corrupting it.
- If UCS-2 compatibility is needed, the fix belongs in the `EdisonJwa/tsclientlib` fork, not in Chanora's protocol layer.

---

## 6. Conclusion

tsclientlib handles encoding correctly for the TS3 protocol:
- All string parsing uses `str::from_utf8()` — strict UTF-8 validation
- No UCS-2/CESU-8 code exists — the fork assumes UTF-8 wire format
- Invalid UTF-8 causes errors, not silent corruption
- The protocol itself is UTF-8; UCS-2 is an edge case from legacy implementations

**Status:** VERIFIED — encoding handling is correct as-is.