docs: add TSDNS/file transfer designs, review p256 PR draft (TODO-057,066,067)
TSDNS: verified fully implemented in chanora_resolver (port 41144, magic bytes, TCP query). File transfer: verified fully implemented for avatars/icons (ftinitdownload, TCP data, cacache). p256 PR draft reviewed and corrected (function path, probability figure).
This commit is contained in:
Edison Jwa
@@ -0,0 +1,169 @@
|
||||
# File Transfer Protocol Design
|
||||
|
||||
**Date:** 2026-06-11
|
||||
**Status:** Implemented
|
||||
**Location:** `core/chanora_core/src/file_transfer.rs`
|
||||
|
||||
## Overview
|
||||
|
||||
TeamSpeak 3 file transfer protocol for downloading avatars and icons. Uses a two-phase approach: command phase over encrypted UDP, then raw TCP data transfer.
|
||||
|
||||
## Protocol Specification
|
||||
|
||||
### Two-Phase Transfer
|
||||
|
||||
1. **Command Phase** — Client sends `ftinitdownload` over main encrypted UDP connection
|
||||
2. **Transfer Phase** — Client opens TCP connection to server's file transfer port (default 30033), sends `ftkey`, receives raw bytes
|
||||
|
||||
### Relevant Commands
|
||||
|
||||
| Command | Direction | Purpose |
|
||||
|---|---|---|
|
||||
| `ftinitdownload` | Client → Server | Initialize download, returns `ftkey`, `port`, `size` |
|
||||
| `ftgetfileinfo` | Client → Server | Get file metadata |
|
||||
| `ftgetfilelist` | Client → Server | List files in channel repository |
|
||||
| `ftinitupload` | Client → Server | Initialize upload |
|
||||
| `ftdeletefile` | Client → Server | Delete a file |
|
||||
| `ftcreatedir` | Client → Server | Create directory |
|
||||
| `ftrenamefile` | Client → Server | Rename/move file |
|
||||
|
||||
### `ftinitdownload` Command
|
||||
|
||||
```
|
||||
ftinitdownload clientftfid={id} name={path} cid={channelId} cpw={password} seekpos={seek} proto=0
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|---|---|---|
|
||||
| `clientftfid` | `u16` | Client-side transfer ID |
|
||||
| `name` | `string` | File path (e.g., `/avatar_abcdef`) |
|
||||
| `cid` | `ChannelId` | Channel scope (0 = server-level) |
|
||||
| `cpw` | `string` | Channel password (empty for server-level) |
|
||||
| `seekpos` | `u64` | Resume offset (0 for fresh download) |
|
||||
| `proto` | `u8` | Protocol version (always 0) |
|
||||
|
||||
**Server Response:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `clientftfid` | `u16` | Echo of client transfer ID |
|
||||
| `serverftfid` | `u16` | Server-side transfer ID |
|
||||
| `ftkey` | `string` | One-time transfer key (hex) |
|
||||
| `port` | `u16` | File transfer TCP port (usually 30033) |
|
||||
| `size` | `u64` | File size in bytes |
|
||||
| `proto` | `u8` | Protocol version echo |
|
||||
| `ip` | `string` (optional) | Override IP for TCP connection |
|
||||
|
||||
### TCP Transfer Flow
|
||||
|
||||
1. Open TCP connection to `server_ip:port`
|
||||
2. Send `ftkey` followed by newline
|
||||
3. Read exactly `size` bytes of raw file data
|
||||
4. Close TCP connection
|
||||
|
||||
### File Paths
|
||||
|
||||
Files are addressed by path scoped to channel ID:
|
||||
- `cid=0` — Server-level repository (avatars, icons live here)
|
||||
- `cid=N` — Channel-specific repository
|
||||
|
||||
**Avatar path:** `/avatar_<hex>` where `<hex>` is derived from client UID:
|
||||
```rust
|
||||
fn uid_to_avatar_path(uid_b64: &str) -> String {
|
||||
let decoded = BASE64_STANDARD.decode(uid_b64).unwrap_or_default();
|
||||
let mut rendered = String::with_capacity(decoded.len() * 2);
|
||||
for byte in decoded {
|
||||
rendered.push((b'a' + (byte >> 4)) as char);
|
||||
rendered.push((b'a' + (byte & 0x0f)) as char);
|
||||
}
|
||||
rendered
|
||||
}
|
||||
```
|
||||
|
||||
**Icon path:** `/icon_<id>` where `<id>` is unsigned CRC32 of icon bytes.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Bridge (get_avatar/get_icon)
|
||||
↓
|
||||
FileTransferService
|
||||
↓
|
||||
1. Check BlobCache → hit? return bytes
|
||||
2. Check negative cache → hit? return None
|
||||
3. Check in-flight map → already downloading? await existing
|
||||
4. Acquire concurrency semaphore (max 2)
|
||||
5. Download via ProtocolClient
|
||||
6. Store in BlobCache
|
||||
7. Notify all waiters
|
||||
```
|
||||
|
||||
### Code Location
|
||||
|
||||
- `core/chanora_core/src/file_transfer.rs` — `FileTransferService` (344 lines)
|
||||
- `crates/chanora_cache/src/lib.rs` — `BlobCache` (cacache-backed)
|
||||
- `crates/chanora_protocol/src/adapter.rs` — `uid_to_avatar_path()` (line 1561)
|
||||
|
||||
### Features Implemented
|
||||
|
||||
- [x] `ftinitdownload` command over encrypted UDP
|
||||
- [x] TCP data transfer with `ftkey` authentication
|
||||
- [x] Avatar download (`/avatar_<hex>`)
|
||||
- [x] Icon download (`/icon_<crc32>`)
|
||||
- [x] Content-addressed blob cache (cacache-backed)
|
||||
- [x] Request coalescing (multiple requests for same hash = 1 download)
|
||||
- [x] Rate limiting (max 2 concurrent downloads)
|
||||
- [x] Negative cache (5-minute TTL for 404s)
|
||||
- [x] Cache integrity verification (SSRI)
|
||||
- [x] Cross-server dedup (same content = same blob)
|
||||
|
||||
### Cache Architecture
|
||||
|
||||
**Storage:** `<cache_dir>/chanora/blobs/` managed by `cacache`
|
||||
|
||||
**Key mapping:**
|
||||
| Protocol key | cacache key | Example |
|
||||
|---|---|---|
|
||||
| Avatar MD5 | `av_<md5hex>` | `av_a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6` |
|
||||
| Icon CRC32 | `ic_<crc32u>` | `ic_123456789` |
|
||||
|
||||
**Why content-addressed:**
|
||||
- `client_flag_avatar` = MD5 of avatar bytes (computed by uploader)
|
||||
- `icon_id` = CRC32 of icon bytes (computed by uploader)
|
||||
- Same content on any server = same hash = stored once
|
||||
|
||||
### Error Handling
|
||||
|
||||
| tsclientlib error | ProtocolError | Action |
|
||||
|---|---|---|
|
||||
| Permission denied | `ServerRejected { code, message }` | Negative cache (5 min) |
|
||||
| File not found | `ServerRejected { code, message }` | Negative cache (5 min) |
|
||||
| Network/TCP failure | `Backend(String)` | Retry with backoff |
|
||||
| Timeout | `Timeout` | Retry with backoff |
|
||||
| Connection lost | `Lost(String)` | Fail all pending downloads |
|
||||
|
||||
## Anti-Flood Strategy
|
||||
|
||||
- Max 2 concurrent downloads per server
|
||||
- 5-second pause on flood error, then resume at reduced rate
|
||||
- Lazy download (only when UI needs to display)
|
||||
- No proactive download of all avatars on connect
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests in `core/chanora_core/src/file_transfer.rs`:
|
||||
- `returns_cached_avatar_without_connection` (line 295)
|
||||
- `negative_cache_short_circuits_not_connected` (line 313)
|
||||
- `returns_cached_icon_without_connection` (line 333)
|
||||
|
||||
## References
|
||||
|
||||
- YaTQA Reference §8.3: File transfer protocol details
|
||||
- `docs/architecture/file-transfer-design.md` — Full design document (737 lines)
|
||||
- `docs/architecture/file-transfer-research.md` — Research findings (770 lines)
|
||||
- `docs/architecture/file-transfer-implementation-plan.md` — Implementation plan (1315 lines)
|
||||
- `core/chanora_core/src/file_transfer.rs` — Implementation (344 lines)
|
||||
@@ -0,0 +1,87 @@
|
||||
# TSDNS Protocol Design
|
||||
|
||||
**Date:** 2026-06-11
|
||||
**Status:** Implemented
|
||||
**Location:** `crates/chanora_resolver/src/lib.rs`
|
||||
|
||||
## Overview
|
||||
|
||||
TSDNS (TeamSpeak DNS) is a lightweight DNS-like protocol for resolving TeamSpeak server addresses. It operates over TCP port 41144 and provides a simple query-response mechanism.
|
||||
|
||||
## Protocol Specification
|
||||
|
||||
### Query Format
|
||||
|
||||
1. Convert input to lowercase
|
||||
2. Encode as UTF-8/CESU-8
|
||||
3. Append magic bytes: `0x0A 0x0D 0x0D 0x0D 0x0A`
|
||||
4. Send via TCP to port 41144
|
||||
|
||||
### Response Format
|
||||
|
||||
- **Success:** IP address or hostname (optionally with port)
|
||||
- **Not found:** Literal string `404`
|
||||
- **Port placeholder:** `$PORT` means "use the port from the user's input"
|
||||
|
||||
### Example
|
||||
|
||||
```
|
||||
Query: "voice.example.com\n\r\r\r\n"
|
||||
Response: "185.250.249.77:9987"
|
||||
```
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Constants
|
||||
|
||||
```rust
|
||||
const TSDNS_PORT: u16 = 41144;
|
||||
const TSDNS_TERMINATOR: &[u8] = b"\n\r\r\r\n";
|
||||
const TSDNS_TIMEOUT: Duration = Duration::from_secs(3);
|
||||
```
|
||||
|
||||
### Resolution Flow
|
||||
|
||||
1. **TSDNS SRV lookup** (`_tsdns._tcp.DOMAIN`) - checks for SRV records first
|
||||
2. **TSDNS TCP fallback** - direct connection to port 41144
|
||||
3. **Candidate hosts** - tries parent domain then full domain (e.g., `teamspeak.com` then `voice.teamspeak.com`)
|
||||
|
||||
### Code Location
|
||||
|
||||
- `query_tsdns_socket()` (lines 529-563) - core protocol implementation
|
||||
- `resolve_tsdns_tcp_candidates()` (lines 454-476) - TCP fallback resolution
|
||||
- `resolve_tsdns_srv_candidates()` (lines 414-433) - SRV-based resolution
|
||||
- `tsdns_candidate_hosts()` (lines 977-997) - generates candidate hostnames
|
||||
|
||||
### Features Implemented
|
||||
|
||||
- [x] TCP port 41144 communication
|
||||
- [x] Lowercase domain normalization
|
||||
- [x] Magic bytes terminator (`0x0A 0x0D 0x0D 0x0D 0x0A`)
|
||||
- [x] TSDNS SRV record support (`_tsdns._tcp.DOMAIN`)
|
||||
- [x] TCP fallback when no SRV records
|
||||
- [x] `$PORT` placeholder support
|
||||
- [x] 3-second timeout per connection
|
||||
- [x] `404` not-found handling
|
||||
- [x] IPv4 and IPv6 address support
|
||||
- [x] Port parsing from response
|
||||
|
||||
## DNS Resolution Order (per spec)
|
||||
|
||||
1. **SRV TS3:** `_ts3._udp.INPUT` — port overrides user input
|
||||
2. **SRV TSDNS:** `_tsdns._tcp.DOMAIN`
|
||||
3. **TSDNS:** Port 41144
|
||||
4. **DNS:** AAAA, A records (CNAME implicit)
|
||||
|
||||
First complete resolution wins. No fallback on connection failure.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests in `crates/chanora_resolver/src/lib.rs`:
|
||||
- `tsdns_candidates_include_parent_then_full_host` (line 1384)
|
||||
- `tsdns_endpoint_preserves_srv_method_metadata` (line 1252)
|
||||
|
||||
## References
|
||||
|
||||
- YaTQA Reference §8.4: TSDNS protocol details
|
||||
- YaTQA Reference §8.5: DNS resolution order
|
||||
@@ -18,25 +18,33 @@ P-256 coordinates must always be exactly 32 bytes (the field element size). Stri
|
||||
|
||||
## Fix
|
||||
|
||||
Zero-pad P-256 coordinates to 32 bytes after `BigInt::to_bytes_be()` serialization.
|
||||
In `EccKeyPubP256::from_tomcrypt` (`utils/tsproto-types/src/crypto.rs`), replace the `WrongPublicKeyLength` error returns with zero-padding to `field_size`.
|
||||
|
||||
```rust
|
||||
// Before (buggy):
|
||||
let x_bytes = x.to_bytes_be();
|
||||
let y_bytes = y.to_bytes_be();
|
||||
if x_bytes.len() != field_size {
|
||||
return Err(Error::WrongPublicKeyLength {
|
||||
expected: field_size,
|
||||
got: x_bytes.len(),
|
||||
});
|
||||
}
|
||||
if y_bytes.len() != field_size {
|
||||
return Err(Error::WrongPublicKeyLength {
|
||||
expected: field_size,
|
||||
got: y_bytes.len(),
|
||||
});
|
||||
}
|
||||
|
||||
// After (fixed):
|
||||
let x_bytes = {
|
||||
let raw = x.to_bytes_be();
|
||||
let mut padded = vec![0u8; 32];
|
||||
padded[32 - raw.len()..].copy_from_slice(&raw);
|
||||
padded
|
||||
let mut buf = vec![0u8; field_size.saturating_sub(x_bytes.len())];
|
||||
buf.extend_from_slice(&x_bytes);
|
||||
buf
|
||||
};
|
||||
let y_bytes = {
|
||||
let raw = y.to_bytes_be();
|
||||
let mut padded = vec![0u8; 32];
|
||||
padded[32 - raw.len()..].copy_from_slice(&raw);
|
||||
padded
|
||||
let mut buf = vec![0u8; field_size.saturating_sub(y_bytes.len())];
|
||||
buf.extend_from_slice(&y_bytes);
|
||||
buf
|
||||
};
|
||||
```
|
||||
|
||||
@@ -49,17 +57,17 @@ fix: zero-pad P-256 ECDH coordinates to 32 bytes
|
||||
|
||||
#### What
|
||||
|
||||
Zero-pad P-256 public key coordinates (x, y) to exactly 32 bytes after `BigInt::to_bytes_be()` serialization.
|
||||
In `EccKeyPubP256::from_tomcrypt`, replace `WrongPublicKeyLength` rejection of short P-256 coordinates with zero-padding to the field size.
|
||||
|
||||
#### Why
|
||||
|
||||
`BigInt::to_bytes_be()` strips leading zero bytes. When a P-256 coordinate happens to have leading zeros (probability ~1/256 per coordinate), the resulting byte array is shorter than 32 bytes. This violates SEC 1 uncompressed point encoding and causes intermittent ECDH handshake failures with TeamSpeak 3 servers.
|
||||
`BigInt::to_bytes_be()` strips leading zero bytes. When a P-256 coordinate happens to have leading zeros (probability ~0.8% per coordinate), the ASN.1-decoded integer becomes shorter than the expected 32-byte field size, causing `WrongPublicKeyLength` errors during the init-server handshake.
|
||||
|
||||
#### Impact
|
||||
|
||||
- Fixes non-deterministic connection failures (~0.4% of connections affected)
|
||||
- Ensures compliance with P-256 field element encoding (RFC 6979 / SEC 1)
|
||||
- No behavioral change for the ~99.6% of connections where coordinates don't have leading zeros
|
||||
- Fixes non-deterministic connection failures (~0.8% of connections affected)
|
||||
- Ensures compliance with P-256 field element encoding (SEC 1)
|
||||
- No behavioral change for the ~99.2% of connections where coordinates don't have leading zeros
|
||||
|
||||
#### Testing
|
||||
|
||||
@@ -70,3 +78,9 @@ Zero-pad P-256 public key coordinates (x, y) to exactly 32 bytes after `BigInt::
|
||||
#### Notes
|
||||
|
||||
This fix is currently carried in the Chanora fork (`EdisonJwa/tsclientlib`). Upstreaming reduces fork maintenance burden and benefits all tsclientlib users.
|
||||
|
||||
---
|
||||
|
||||
## Status
|
||||
|
||||
**Ready for submission** — PR draft reviewed and verified against actual fork commit `8b7a322` (branch `fix/p256-short-coordinate-pad`). Code examples, file paths, and probability figures have been corrected to match the implementation.
|
||||
|
||||
Reference in New Issue
Block a user