* docs(architecture): add file transfer design, research, and implementation plan * feat(cache): add chanora_cache crate with cacache-backed blob cache - New chanora_cache crate: content-addressed blob store wrapping cacache - BlobCache API: async put/get/remove/clear/total_size/evict - Key validation: av_ prefix (32 hex chars), ic_ prefix (decimal digits) - Cacache provides crash safety, SSRI integrity, content dedup - Mtime-based eviction via cacache::list_sync + sort by timestamp - 7 unit tests all passing - Added to workspace members * feat(protocol): add file download support for avatars and icons - Add Request::DownloadFile variant with oneshot reply - Add ProtocolClient::download_avatar(client_uid) and download_icon(icon_id) - Track pending file downloads by FiletransferHandle - Handle StreamItem::FileDownload: read bytes from TCP stream - Handle StreamItem::FiletransferFailed: map to ProtocolError - Add ProtocolError::FileTransfer(String) variant - Add path helper tests for avatar/icon download paths - No tsclientlib types leak across the adapter boundary * feat(core): add blob cache wiring and avatar download orchestration - Add chanora_cache dependency to Cargo.toml - Add blob_cache field to ChanoraSession (Arc<Mutex<Option<BlobCache>>>) - Add init_cache() method: creates BlobCache, runs eviction - Add get_avatar() method: cache-first, download on miss, store in cache - Add clear_cache() and cache_size() methods for cache management - Add CoreError::Cache variant for BlobCacheError conversion - Add avatar_cache integration test * feat(bridge): add init_cache, download_avatar, and cache management functions - Add init_cache(dir) bridge function - Add download_avatar(avatar_hash, client_uid) bridge function - Add clear_file_cache() and file_cache_size() bridge functions - Map CoreError::Cache and ProtocolError::FileTransfer in BridgeError * feat(flutter): add cache initialization wiring and avatar download shims - Add wireCache() to app_bootstrap using getApplicationCacheDirectory() - Call wireCache() after wireStorage() in main bootstrap flow - Add Dart-side initCache and downloadAvatar wrapper shims in api.dart - Update Cargo.lock for new chanora_cache dependency * feat(core): FileTransferService with coalescing, throttling, negative cache - New file_transfer module with FileTransferService struct - Semaphore(2) throttles concurrent downloads - In-flight HashMap coalesces duplicate avatar requests - 5-min negative cache short-circuits ServerRejected misses - ChanoraSession delegates get_avatar through the service - connect/disconnect update shared protocol handle - clear_cache/cache_size delegate to service - 2 new unit tests (cached hit, negative cache) * feat(core,bridge): add get_icon with coalescing and negative cache - FileTransferService::get_icon() mirrors get_avatar pattern - ChanoraSession::get_icon() delegates through FileTransferService - Bridge download_icon() exposed for Flutter - Dart downloadIcon() shim added - Uses PREFIX_ICON (ic_<crc32u>) cache key format - 1 new unit test (cached icon hit) * fix(core,protocol): simplify store_protocol and add download size cap - store_protocol: always write to shared Arc<Mutex<Option<ProtocolClient>>>; the FileTransferService holds the same Arc so it sees updates automatically - read_download_bytes: reject downloads exceeding 10 MB to prevent malicious servers from causing OOM
37 KiB
File Transfer Cache Research
Date: 2026-06-10
Status: Research complete, design implications noted (updated with TeaSpeak server findings)
Companion to: docs/architecture/file-transfer-design.md
Purpose: Factual findings from protocol analysis, existing client implementations, and cross-platform research that inform the cache architecture decision.
1. TS3 Protocol Identity Semantics
1.1 Avatar Identity
| Aspect | Value |
|---|---|
| Protocol field | client_flag_avatar |
| Type | TYPE_STRING (TeaSpeakLibrary PropertyDefinition.h:200) |
| Meaning | MD5 hash of the avatar file bytes |
| Scope | Per-client per-server — a user can have different avatars on different servers |
| Freshness | Automatically up-to-date for any client "in view" (FLAG_CLIENT_VIEW) |
| Empty value | No avatar set |
Key fact: Identical avatar image bytes produce the same client_flag_avatar hash on any TS3 server. The hash is a content fingerprint, not a server-assigned identifier.
Download path on server: /avatar_<base64HashClientUID> — the filename is derived from the client's unique identifier (UID), not from the content hash. The content hash is communicated separately via client_flag_avatar.
Sources:
- TeaSpeakLibrary
PropertyDefinition.h:200:PropertyDescription{CLIENT_FLAG_AVATAR, "client_flag_avatar", "", TYPE_STRING, FLAG_CLIENT_VIEW | FLAG_SAVE | FLAG_USER_EDITABLE} - TS3AudioBot avatar upload: computes MD5 of image bytes, then sets
client_flag_avatarto that hash (TS3AudioBot/TSLib/TsBaseFunctions.cs:324-341) - TS3 NodeJS Library: avatar filename is
avatar_${clientBase64HashClientUID}(TS3-NodeJS-Library/src/node/Client.ts:300-313) - TS3 PHP Framework: avatar name derivation from UID (
ts3phpframework/src/Node/Client.php:288-307)
1.2 Icon Identity
| Aspect | Value |
|---|---|
| Protocol fields | channel_icon_id, client_icon_id, virtualserver_icon_id |
| Type | TYPE_UNSIGNED_NUMBER (TeaSpeakLibrary PropertyDefinition.h:83,146,217) |
| Meaning | CRC32 (unsigned) of the icon file bytes |
| Scope | Per-entity per-server — but CRC32 is content-derived |
| Download path | /icon_<unsigned_crc32> |
Key fact: Identical icon bytes produce the same CRC32 on any TS3 server. The icon ID is a content fingerprint. The upload process computes crc32.unsigned(data) and stores at /icon_<id>.
CRC32 collision caveat: CRC32 is only 32 bits. Different icon content can theoretically produce the same CRC32. Qint's filecache.rs explicitly notes this: "there could be collisions because only CRC-32 is used." ForChanora's purposes (small icons, not security-critical), this is acceptable.
Sources:
- TeaSpeakLibrary
PropertyDefinition.h:83,146,217: all icon IDs areTYPE_UNSIGNED_NUMBER - TS3 NodeJS Library
uploadIcon(): computescrc32.unsigned(data), uploads to/icon_<id>(TS3-NodeJS-Library/src/TeamSpeak.ts:2234-2241) - TS3 PHP Framework: icon path uses
/icon_<unsigned id>(ts3phpframework/src/Node/Node.php:137-146) - TS3 community forum: "The filename itself is the result of the CRC32 checksum" (TeamSpeak staff)
1.3 Implication for Cache Design
Both avatars and icons are content-addressed by the protocol itself:
| Asset | Content hash source | Same content across servers? |
|---|---|---|
| Avatar | client_flag_avatar = MD5 of bytes |
Same bytes → same hash → same ID |
| Icon | icon_id = CRC32 of bytes |
Same bytes → same CRC32 → same ID |
This means a flat content-addressed blob store can achieve zero-duplication without any per-server directories, hardlinks, or ref-counting.
2. Virtual Server Identity
2.1 Server UID
| Aspect | Value |
|---|---|
| Protocol field | virtualserver_unique_identifier |
| Type | TYPE_STRING (TeaSpeakLibrary PropertyDefinition.h:22) |
| Generated by | The server instance, on creation |
| Globally unique? | Not guaranteed — locally generated, no central registry |
| Stable? | Yes — persists across restarts of the same virtual server |
Key fact: virtualserver_unique_identifier is generated by each TS3 server. Two physically different servers could theoretically produce the same UID. It is not safe as a global cache key.
2.2 What Chanora Currently Tracks
| Layer | Server identity fields | Source |
|---|---|---|
Protocol adapter (adapter.rs:1752-1755) |
server_name, welcome_message, platform, version |
state.server.* from tsclientlib |
DTO (ServerSnapshot) |
server_name, welcome_message, platform, version |
No UID field |
Bridge (BridgeSnapshot) |
Same as DTO | Same |
| Storage (bookmarks) | Keyed by host (hostname:port) |
SQLite WHERE host = ?1 |
| Core (recent servers) | cfg.address as host |
Auto-saved on connect |
Chanora does not currently plumb virtualserver_unique_identifier through the DTO stack. The field exists in tsclientlib's state but is not extracted.
2.3 Implication for Cache Design
Using virtualserver_unique_identifier as the sole cache key is risky (not globally unique). Using connection address (host:port) is safe but duplicates cache entries when the same server is accessed via different addresses.
Recommendation: For a content-addressed blob store, server identity is only needed for per-server metadata (eviction, "clear cache for this server"), not for the blob key itself. The blob key is the content hash.
3. Existing TS3 Client Cache Implementations
3.1 Qint (tsclientlib-based, Tauri + Rust)
Architecture: Per-server directory with SQLite metadata.
<cache>/files/<server-uid>/<channel-id>/<base64(path)>
Avatar handling:
- Avatar state stored per
(server, client)row in SQLite - On avatar hash change, deletes the cached
/avatar_<uid>file for that server - Avatar download path:
/avatar_<uid_base64>
Icon handling:
- Icons path-cached with CRC32
- Code comments note CRC32 collisions and freshness by mtime
- Qint explicitly deletes and re-downloads when icon mtime changes
Dedup: None. Same avatar on 5 servers = 5 stored copies.
Sources:
Qint/proxy/src/filecache.rs: "Stores files transferred via the TS3 file transfer protocol. This includes icons and avatars."Qint/proxy/src/db/mod.rs: avatar hash change triggers deleteQint/src-tauri/src/cmd.rs: file download command
3.2 TS3 Official Client (closed source)
Architecture: Lazy cache with SDK callbacks.
getAvatar()returns cached path if present; otherwise triggers downloadonAvatarUpdatedcallback fires when avatar is downloaded or deleted- Cache paths (from community documentation):
- Windows:
%LOCALAPPDATA%\TeamSpeak\Cache\Default - Linux:
~/.cache/TeamSpeak/Default - macOS:
~/Library/Caches/TeamSpeak/Default
- Windows:
- SDK also exposes
CLIENT_MYTS_AVATAR/client_myteamspeak_avatarfor cross-server myTeamSpeak avatars
Sources:
ts3client-pluginsdk/src/plugin.c:getAvatar()andonAvatarUpdatedts3client-pluginsdk/public_rare_definitions.h:CLIENT_FLAG_AVATAR,CLIENT_MYTS_AVATAR- Community: clear cache, broken icons
3.3 TeaSpeak Client (TypeScript + C++ native)
Architecture: Browser Cache API for images, per-server own-avatar storage.
Key components (from .d.ts type declarations):
-
AvatarManager— per-connection (FileManager) avatar handlercachedAvatars(private) — in-memory cache ofClientAvatarobjectsupdateCache(clientAvatarId, clientAvatarHash)— updates cache when hash changesresolveAvatar(clientAvatarId, avatarHash?, cacheOnly?)— resolves avatar by IDflush_cache()— clears cachecreate_avatar_download(client_avatar_id)— initiates file transfer
-
ClientAvatar— tracks individual avatar stateclientAvatarId— derived from client UID viauniqueId2AvatarId()currentAvatarHash— theclient_flag_avatarvalue- State machine:
unset→loading→loaded/errored loadingTimestamp— when download started
-
ImageCache— generic image cache using browser Cache APIresolveCached(key, maxAge?)— check if cachedputCache(key, value, type?, headers?)— storecleanup(maxAge)— evict old entriesreset()— clear allisPersistent()— whether cache persists to disk
-
OwnAvatarStorage— user's own avatar, keyed byserverUniqueId + modeloadAvatarImage(serverUniqueId, mode)— load own avatar for a serverupdateAvatar(serverUniqueId, mode, target)— update own avataravatarUploadSucceeded(serverUniqueId)— move from "uploading" to "server" state- Stores
LocalAvatarInfo: fileName, fileSize, fileHashMD5, timestamps, contentType
-
FileManager— per-connection file transfer managerMAX_CONCURRENT_TRANSFERS— transfer concurrency limitavatars: AvatarManager— avatar subsysteminitializeFileDownload(options)— start download (path, name, channel, target)deleteIcon(iconId: number)— delete icon by ID
-
FileTransfer— transfer state machine- States:
PENDING → INITIALIZING → CONNECTING → RUNNING → FINISHED / ERRORED / CANCELED InitializedTransferProperties: serverTransferId, transferKey, addresses[], protocol, seekOffset, fileSize- Multiple addresses returned by server for file transfer (failover)
- States:
-
localIconCache: ImageCache— global icon cache (singleton)
Sources:
- TeaSpeak-Client
imports/shared-app/file/Avatars.d.ts— ClientAvatar, AbstractAvatarManager - TeaSpeak-Client
imports/shared-app/file/LocalAvatars.d.ts— AvatarManager - TeaSpeak-Client
imports/shared-app/file/LocalIcons.d.ts— localIconCache - TeaSpeak-Client
imports/shared-app/file/ImageCache.d.ts— ImageCache (browser Cache API) - TeaSpeak-Client
imports/shared-app/file/FileManager.d.ts— FileManager, transfer API - TeaSpeak-Client
imports/shared-app/file/Transfer.d.ts— FileTransfer, state machine, error types - TeaSpeak-Client
imports/shared-app/file/OwnAvatarStorage.d.ts— own avatar per-server storage - TeaSpeak-Client
native/serverconnection/test/js/ft.ts— file transfer test (TCP + ftkey protocol)
3.4 TS3AudioBot (C#)
Architecture: No local avatar cache. Avatar upload is hash-driven.
- Uploads avatar bytes to
/avatar, computes MD5, setsclient_flag_avatarto that hash - Bot avatar selection reads local files from an
avatars/directory - No caching of other users' avatars
Sources:
3.5 TS3 NodeJS Library
Architecture: No local cache. Downloads on demand.
- Avatar filename:
avatar_${clientBase64HashClientUID} getAvatar()downloads directly — no caching layer- Tests assert the exact
/avatar_<base64uid>path
Sources:
3.6 Summary Table
| Client | Cache Key Strategy | Dedup Across Servers? | Icon Cache |
|---|---|---|---|
| Qint | <server-uid>/<channel-id>/<path> |
No | Yes (CRC32, mtime freshness) |
| TS3 Official | Lazy cache (path-based) | Unknown | Yes |
| TeaSpeak | UID-derived avatar ID + browser Cache API | Implicit (same hash = same cache) | Yes (global ImageCache) |
| TS3AudioBot | None | N/A | No |
| TS3 NodeLib | None | N/A | No |
| Chanora (decided) | Content hash (MD5/CRC32), flat blobs/ in chanora_cache crate |
Yes | Yes |
4. TeaSpeak Protocol Definitions (Authoritative)
From TeaSpeakLibrary src/PropertyDefinition.h — the most complete open-source reference for TS3 protocol property types:
4.1 Avatar Properties
// Line 200
PropertyDescription{CLIENT_FLAG_AVATAR, "client_flag_avatar", "",
TYPE_STRING, FLAG_CLIENT_VIEW | FLAG_SAVE | FLAG_USER_EDITABLE}
// "automatically up-to-date for any manager 'in view', this manager got an avatar"
4.2 Icon Properties
// Line 83 — server icon
PropertyDescription{VIRTUALSERVER_ICON_ID, "virtualserver_icon_id", "0",
TYPE_UNSIGNED_NUMBER, FLAG_SERVER_VVSS | FLAG_USER_EDITABLE}
// Line 146 — channel icon
PropertyDescription{CHANNEL_ICON_ID, "channel_icon_id", "0",
TYPE_UNSIGNED_NUMBER, FLAG_CHANNEL_VIEW | FLAG_SS | FLAG_USER_EDITABLE}
// Line 217 — client icon
PropertyDescription{CLIENT_ICON_ID, "client_icon_id", "0",
TYPE_UNSIGNED_NUMBER, FLAG_CLIENT_VIEW | FLAG_CLIENT_VARIABLE}
4.3 Server Identity
// Line 22
PropertyDescription{VIRTUALSERVER_UNIQUE_IDENTIFIER,
"virtualserver_unique_identifier", "",
TYPE_STRING, FLAG_SERVER_VV | FLAG_SNAPSHOT}
4.4 File Transfer Permissions
// From PermissionManager.cpp
PermissionType::i_client_max_avatar_filesize // "Max avatar filesize in bytes"
PermissionType::b_client_avatar_delete_other // "Allow deletion of avatars from other clients"
PermissionType::b_ft_transfer_list // "Retrieve list of running filetransfers"
4.5 File Transfer Error Codes
// From Error.h
channel_no_filetransfer_supported = 0x30C
file_transfer_connection_timeout = 0x80E
file_transfer_complete = 0x811
file_transfer_canceled = 0x812
file_transfer_interrupted = 0x813
file_transfer_server_quota_exceeded = 0x814
file_transfer_client_quota_exceeded = 0x815
file_transfer_reset = 0x816
file_transfer_limit_reached = 0x817
5. Cross-Platform Filesystem Research
5.1 Hardlink Support
| Platform | Filesystem | Hardlinks in App-Private Storage? | Gotcha |
|---|---|---|---|
| Android (API 28+) | ext4 / f2fs | Yes | Rust uses libc::link; not FUSE-mounted; same-filesystem only |
| iOS | APFS | Yes (writable sandbox dirs) | App bundle is read-only; avoid hardlinks to bundle assets |
| macOS | APFS | Yes | — |
| Linux | ext4 / btrfs / xfs | Yes | — |
| Windows | NTFS | Yes | Rust uses CreateHardLinkW |
std::fs::hard_link gotchas (all platforms):
- Same filesystem required
- Destination must not exist (returns error)
- Symlink behavior is platform-specific
- All hardlinks share the same inode — modifying one modifies all
- Files must be treated as immutable for hardlink safety
Sources:
- Rust stdlib:
hard_linkmaps tolibc::link(Unix),CreateHardLinkW(Windows) (Rust source) - Android: internal storage uses ext4/f2fs, not FUSE (Android scoped storage docs)
- iOS: APFS supports hardlinks; writable sandbox directories work (Apple FileSystem basics)
5.2 Rust Cache Libraries
cacache (MIT licensed, production-ready):
- Content-addressed disk cache
- Automatic dedup, atomic writes, integrity verification
- Exposes
hard_link,copy, andreflinkpaths for retrieval - On-disk layout:
content-v2/sha512/... - Could replace a custom implementation, but adds a dependency
Sources:
5.3 Flutter Cache Patterns
Common Flutter packages use cache-dir + metadata DB, not hardlink dedup:
flutter_cache_manager: files in cache dir +sqflitemetadatasuper_cache_disk: file-per-entry (.dat+.meta) in app cache dir
Sources:
6. Chanora Codebase Context
6.1 Storage Patterns
| Component | Pattern | Location |
|---|---|---|
| Identity storage | Atomic write (temp + sync_all + rename), mode 0600 on Unix |
chanora_storage/src/lib.rs:446-476 |
| Metadata | Same atomic write pattern | chanora_storage/src/lib.rs:480-515 |
| Bookmarks | SQLite at <storage_dir>/chanora.db, keyed by host |
chanora_storage/src/lib.rs:782-928 |
| Storage root | getApplicationSupportDirectory() from Flutter |
app_bootstrap.dart:19-24 |
| Bridge init | rust.initStorage(dir: dir) |
app_bootstrap.dart:23-25,108-133 |
6.2 Existing Avatar Handling
| Component | What | Location |
|---|---|---|
| UID to avatar path | uid_to_avatar_path() — base64 decode UID, encode each byte as 2 chars (a-p) |
adapter.rs:1561-1568 |
| Client profile DTO | avatar_path field — set when client.avatar_hash and unique_id non-empty |
dto.rs:135-136, adapter.rs:1323-1332 |
| No download | Currently no file download implementation exists | — |
| No icon handling | No icon field/path in protocol DTO or adapter | — |
6.3 Server Identity in Chanora
Chanora currently tracks servers by connection address (host:port), not by server UID:
- Bookmarks:
WHERE host = ?1 - Recent servers: auto-saved by
cfg.address - Prefetch cache: keyed by normalized host
- ServerSnapshot: has
server_namebut noserver_uid
The virtualserver_unique_identifier field is available from tsclientlib's state but is not extracted by the adapter.
6.4 tsclientlib File Transfer API
| Aspect | Detail |
|---|---|
| Download method | Connection::download_file() |
| Stream items | StreamItem::FileDownload(FileDownloadResult { size, stream }) |
| Failure | StreamItem::FiletransferFailed(handle, error) |
| TCP handling | tsclientlib handles TCP connection + ftkey writing automatically |
| Chanora's job | Read size bytes from the returned TcpStream |
| Async behavior | StreamItem::FileDownload fires asynchronously, not inline with the request |
6.5 ts-bookkeeping Generated Fields
From the generated parser in target/debug/build/ts-bookkeeping-*/out/:
virtual_server_id: u64— numeric, per-virtual-server, may change across restartsvirtual_server_uid— string, thevirtualserver_unique_identifier
Both are available in the InInitServer struct from the init handshake but are not currently plumbed through.
7. TeaSpeak Server Internals (Authoritative)
Source: TeaSpeak Server at https://git.did.science/TeaSpeak/Server/Server (branch new-groups, commit b54c6d4e).
7.1 Avatar ID Derivation — Server Side
The server derives the avatar filename from the client UID, not from the avatar content:
// DataClient.cpp:242-244
std::string DataClient::getAvatarId() {
return hex::hex(base64::validate(this->getUid()) ? base64::decode(this->getUid()) : this->getUid(), 'a', 'q');
}
The same transform produces client_base64HashClientUID (shown to other clients):
// client.cpp:1113-1114
bulk.put_unchecked("client_base64HashClientUID",
hex::hex(base64::validate(info->client_unique_id) ? base64::decode(info->client_unique_id) : info->client_unique_id, 'a', 'q'));
This matches Chanora's existing uid_to_avatar_path() in adapter.rs:1561-1568.
7.2 Avatar Upload Path
When a client uploads an avatar, the server stores it as /avatar_<avatarId>:
// file.cpp:696-702
} else if (cmd["path"].as<std::string>().empty() && cmd["name"].string() == "/avatar") {
...
info.file_path = "/avatar_" + this->getAvatarId();
transfer_response = file::server()->file_transfer().initialize_avatar_transfer(...);
}
The avatar file path is identity-based (from UID), not content-based.
7.3 client_flag_avatar — Who Computes the Hash?
The CLIENT computes the MD5 and sends it to the server during upload. The server stores it as a string property (FLAG_USER_EDITABLE). The server does NOT compute or verify the hash.
This means client_flag_avatar is:
- Set by the uploading client
- Stored verbatim by the server
- Broadcast to other clients as part of the client properties
- A reliable content fingerprint: same avatar bytes → same MD5 → same
client_flag_avataron any server
7.4 Icon IDs — Server Does NOT Compute CRC32
Icon IDs are permission values, not content hashes computed by the server:
// ConnectedClient.cpp:186-210 — client icon ID from permissions
auto permission_flags = local_permissions->permission_flags(permission::i_icon_id);
new_icon_id = value.value;
updated_client_properties.emplace_back(property::CLIENT_ICON_ID);
// channel.cpp:1495-1504 — channel icon ID
if(key == property::CHANNEL_ICON_ID) {
auto icon_id = converter<uint32_t>::from_string_view(value);
channel->permissions()->set_permission(permission::i_icon_id, { ... icon_id ... });
}
// server.cpp:76-89 — server icon ID
SERVEREDIT_CHK_PROP_CACHED("virtualserver_icon_id", permission::b_virtualserver_modify_icon_id, int64_t)
The CLIENT computes the CRC32 during upload and uses it as the filename /icon_<crc32>. The server stores the file and records the ID as a permission value. No server-side CRC32 or MD5 computation exists.
7.5 Per-Server Storage Layout
Avatars and icons are stored per virtual server on the server's filesystem:
// LocalFileSystem.cpp:39-45
fs::path LocalFileSystem::server_path(const std::shared_ptr<VirtualFileServer> &server) {
return fs::u8path(this->root_path_) / fs::u8path("server_" + std::to_string(server->server_id()));
}
// target_path = this->server_path(server) / "icons" / path;
// target_path = this->server_path(server) / "avatars" / path;
<server_root>/
server_<sid>/
avatars/
/avatar_<avatarId> ← one per client who uploaded
icons/
/icon_<id> ← one per unique icon
7.6 File Transfer Protocol (Server Side)
Upload/delete/query routing:
// file.cpp:273-341 — delete routing
if (first_entry_name.find("/icon_") == 0 && file_path.empty()) { ... delete_icons(...); }
else if (first_entry_name.starts_with("/avatar_") && file_path.empty()) { ... delete_avatars(...); }
// file.cpp:483-523 — query routing
if (first_entry_name.find("/icon_") == 0 && file_path.empty()) { ... query_icon_info(...); }
else if (first_entry_name.starts_with("/avatar_") && file_path.empty()) { ... query_avatar_info(...); }
Transfer initialization returns ftkey and metadata:
// file.cpp:759-761
result.put_unchecked(0, "ftkey", transfer->transfer_key);
result.put_unchecked(0, "seekpos", transfer->file_offset);
// file.cpp:887-899
result.put_unchecked(0, "ftkey", transfer->transfer_key);
result.put_unchecked(0, "proto", "1");
result.put_unchecked(0, "size", transfer->expected_file_size);
7.7 Key Takeaway for Chanora
| Who does what | Avatar | Icon |
|---|---|---|
| Uploader (client) computes | MD5 of avatar bytes → sets client_flag_avatar |
CRC32 of icon bytes → filename /icon_<crc32> |
| Server does | Stores file as /avatar_<uid>, saves property |
Stores file as /icon_<id>, saves permission |
| Other clients receive | client_flag_avatar (MD5) as a property update |
icon_id (CRC32) as a property update |
| Chanora cache key | av_<md5>.dat — content fingerprint |
ic_<crc32>.dat — content fingerprint |
The content hash is computed once (by the uploader) and then broadcast as a property. Chanora never needs to hash anything — it just uses the protocol-provided values as cache keys.
8. Design Implications
8.1 The Core Insight
The TS3 protocol provides content hashes as part of normal server-to-client updates:
| Event | Data provided by server | What Chanora gets for free |
|---|---|---|
| Client enters view | client_flag_avatar = MD5 of avatar bytes |
Content key for blob store |
| Channel update | channel_icon_id = CRC32 of icon bytes |
Content key for blob store |
| Server update | virtualserver_icon_id = CRC32 of icon bytes |
Content key for blob store |
No hashing needed on the client side. The protocol is already content-addressed.
8.2 Recommended Cache Architecture
<app_cache_dir>/chanora/
blobs/ ← cacache content store root
content-v2/ ← content-addressed by SHA-512
<sha512-hex>/data ← raw blob bytes
index-v2/ ← key → content mapping
Where <app_cache_dir> is the platform cache directory (not the support directory used by chanora_storage). Chanora's BlobCache maps protocol keys (av_<md5>, ic_<crc32>) to cacache string keys. Physical layout is managed by cacache.
Lookup flow:
- Server sends
client_flag_avatar = "a1b2c3d4..."for user X - Check: does
blobs/av_a1b2c3d4....datexist? - Yes → use it, zero downloads (works for ANY server)
- No → download from
/avatar_<uid_base64>→ save asblobs/av_a1b2c3d4....dat
Same for icons with ic_<crc32>.dat.
8.3 Why This Beats Alternatives
| Approach | Dedup | Globally unique key | Needs server UID plumbing | Needs hardlinks | Complexity |
|---|---|---|---|---|---|
<server_uid>/<hash>.dat |
No | No (UID not guaranteed unique) | Yes | Optional | Medium |
<host>_<port>/<hash>.dat |
No | Yes | No | Optional | Medium |
<host>_<port>/<hash>.dat + hardlinks |
Yes | Yes | No | Yes | Medium-High |
blobs/av_<hash>.dat (flat) |
Yes | Yes (content hash) | No | No | Low |
8.4 Trade-offs
| Pro | Con |
|---|---|
| Zero duplication across all servers | "Clear cache for server X only" requires metadata layer (Phase 3+) |
| No hardlinks needed | Orphan cleanup requires scanning for unreferenced blobs |
| No server UID plumbing needed | Cannot distinguish same-hash-different-content for icons (CRC32 collision) |
| Simplest possible implementation | — |
| Freshness = hash change = different filename (automatic) | — |
| Cross-platform (just file I/O) | — |
8.5 Phased Implementation
| Phase | What | Delivers |
|---|---|---|
| 1 | Protocol download (raw bytes via adapter, no cache) | Working download pipeline |
| 2 | Bridge + Flutter display (Image.memory()) |
Visible avatars in UI |
| 3 | chanora_cache crate: cacache-backed blob cache, separate crate, cache dir, mtime eviction |
Zero re-downloads, zero duplication |
| 4 | Session orchestration (coalescing, rate limiting, negative cache) | Anti-flood, robustness |
| 5 | Cache management (clear all, orphan cleanup, optional per-server metadata) | User control |
9. Resolved Questions
Q1: Icon CRC32 Collisions — Accept with Size Guard
Risk assessment: CRC32 produces a 32-bit hash. For N unique icons, the Birthday paradox gives collision probability ≈ N² / (2 × 2³²).
| Icons (N) | Collision probability |
|---|---|
| 100 | ~0.0001% (negligible) |
| 1,000 | ~0.01% (negligible) |
| 10,000 | ~1.2% (marginal) |
| 65,536 | ~50% (likely) |
A single user typically encounters fewer than 1,000 unique icons across all servers. The practical collision risk is negligible.
What happens on collision: Wrong icon displayed for a channel/client/server. This is a visual glitch, not a security issue. The icon will appear incorrect until the cache is cleared.
Existing practice: Qint explicitly notes CRC32 collisions (filecache.rs:4) but does NOT guard against them — they only refresh icons by mtime. No other TS3 client guards against CRC32 collisions.
Recommendation: Accept CRC32 as the cache key. Add a lightweight file size guard: when downloading an icon, if ic_<crc32>.dat already exists but has a different size than the ftinitdownload response reported, re-download. File size is available from the protocol (msg.size in InFileDownloadPart). This catches most collisions (different content = different size with high probability) without computing a secondary hash.
Decision: CRC32 + file size guard. No SHA256 overhead needed.
Q2: client_myteamspeak_avatar — Defer Indefinitely
What it is: A string property (Option<String> in ts-bookkeeping) broadcast alongside client_flag_avatar. It represents a myTeamSpeak cross-server avatar — a user linked to a myTeamSpeak account can set a global avatar that follows them across all servers.
Current state in Chanora's dependency chain:
- ts-bookkeeping exposes it:
InInitServerhasmy_team_speak_avatar: Option<String> - TeaSpeakLibrary tracks
client_myteamspeak_idbut not the avatar - tsclientlib exposes it as a property on client state
Value for Chanora:
- myTeamSpeak is a TeamSpeak-specific cloud service (account sync, cross-server features)
- Chanora is an independent client — no myTeamSpeak account integration is planned
- The property may contain a URL or identifier that requires myTeamSpeak API access to resolve
- Without myTeamSpeak integration, the avatar cannot be fetched
Recommendation: Defer indefinitely. If Chanora ever integrates myTeamSpeak accounts, this can be handled as a separate avatar source (URL-based HTTP download) alongside the existing protocol-based avatar download. The cache architecture supports this — just add a different blob prefix (e.g., mt_<hash>.dat).
Decision: Out of scope for MVP and foreseeable roadmap.
Q3: Cache Backing Store — cacache Wrapper in Separate chanora_cache Crate
Decision: Use cacache as the backing store inside chanora_cache. Not a custom flat-file implementation.
Why cacache won over custom:
-
Crash safety is production-tested.
cacachehandles partial writes, power loss, crash mid-write. A custom implementation would need to getsync_all+ atomic rename right — one bug = corrupted cache. Even though cache data is disposable (reconstructible from server),cacacheeliminates this entire class of bugs. -
Less code to maintain. ~120 LOC wrapper vs ~200 LOC custom implementation. The hard parts (atomic writes, integrity, content dedup) are owned by
cacache, tested by the npm ecosystem. -
Integrity verification on every read. SSRI verification detects corruption, bit rot, partial writes automatically. A custom impl would need to add this separately or accept silent corruption.
-
Content dedup by SHA-512. Same avatar on two servers = stored once automatically. The protocol's MD5/CRC32 keys map to
cacachestring keys; content dedup happens at the SHA-512 layer underneath.
What about the downsides:
| Concern | Assessment |
|---|---|
| ~6 transitive deps | sha2 already in tree via chacha20poly1305. serde_json, tempfile, digest are lightweight. Acceptable for the safety benefit. |
| SHA-512 overhead on every write/read | For <100KB avatars, SHA-512 takes ~0.1ms. Negligible. |
| Opaque on-disk format | cacache provides ls() API for enumeration and inspection. Not as simple as ls blobs/ but adequate. |
cacache has no built-in LRU eviction |
We write a custom eviction pass using cacache::ls() + timestamp sort. ~20 lines. Same complexity as custom impl's eviction. |
Separate crate rationale:
chanora_cacheis separate fromchanora_storagebecause cache data has different durability semantics (disposable vs persistent), different backup semantics (excluded vs included), and different directory placement (cache dir vs support dir).chanora_cachelives in the platform's cache directory (getApplicationCacheDirectory()).chanora_storagelives in the support directory (getApplicationSupportDirectory()).- Bridge init is separate:
init_cache(cache_dir)vsinit_storage(support_dir).
API design:
pub struct BlobCache { cache_dir: PathBuf, max_bytes: u64 }
impl BlobCache {
pub fn new(cache_dir: impl AsRef<Path>, max_bytes: u64) -> Result<Self, BlobCacheError>;
pub async fn put(&self, prefix: &str, key: &str, data: &[u8]) -> Result<(), BlobCacheError>;
pub async fn get(&self, prefix: &str, key: &str) -> Result<Option<Vec<u8>>, BlobCacheError>;
pub async fn remove(&self, prefix: &str, key: &str) -> Result<(), BlobCacheError>;
pub async fn clear(&self) -> Result<(), BlobCacheError>;
pub async fn total_size(&self) -> Result<u64, BlobCacheError>;
pub async fn evict(&self) -> Result<(), BlobCacheError>;
}
All methods are async (cacache is async-native). Key validation at API boundary (av_ = 32 hex chars, ic_ = decimal digits).
Q4: Per-Blob Metadata — No Metadata Sidecars (Resolved)
Original options:
| Approach | Pros | Cons |
|---|---|---|
| SQLite (chanora.db) | ACID, queryable, already in use | Schema migration, couples cache to bookmark DB |
| JSON sidecar files | Simple, self-contained, easy to debug | Write amplification (2 files per blob), concurrent write risk |
| In-memory only | Simplest | Lost on restart, can't do orphan cleanup offline |
| No metadata (mtime-based) | Simplest, zero write amplification, 1 file per blob | No per-blob metadata beyond mtime |
Why no metadata is sufficient:
-
Content is immutable. A given hash (MD5 or CRC32) always maps to the same bytes. There is no "stale content" problem — if the hash changes, it's a new file with a new name. No invalidation needed.
-
mtime = insertion time. Since content is never modified after write, the filesystem mtime equals the time the blob was cached. This is sufficient for "delete oldest files first" eviction.
-
Write amplification avoided. One file per blob (just the data) instead of two (data + JSON sidecar). For a cache that may hold thousands of small files, this matters.
-
Eviction is simple.
walk dir → stat → sort by mtime → delete oldest. No JSON parsing, no schema, no migration. -
Per-server metadata deferred. "Clear cache for server X only" and orphan cleanup are post-MVP features. If needed, a refs-layer can be added later without changing the blob layout.
Oracle consultation: Oracle recommended this approach explicitly — no metadata files, mtime-based eviction, separate crate. The immutability guarantee makes metadata redundant.
Decision: No metadata sidecars. One file per blob. Mtime-based eviction. Per-server metadata deferred to post-MVP.
Q5: File Transfer Address Failover — Not Needed
What the protocol provides:
The TeaSpeak client's InitializedTransferProperties returns addresses[] — an array of {serverAddress, serverPort}. The official TS3 client can try multiple addresses for failover.
What tsclientlib provides:
// tsclientlib/src/lib.rs:1373-1375
let ip = msg.ip.unwrap_or_else(|| self.client.address.ip());
let addr = SocketAddr::new(ip, msg.port);
TcpStream::connect(&addr).await
tsclientlib's InFileDownloadPart has ip: Option<IpAddr> — single IP only, not an array. If the server provides an IP, it uses that. Otherwise, it falls back to the connection address. No multi-address failover.
What ts-bookkeeping parses:
pub struct InFileDownloadPart {
pub client_filetransfer_id: u16,
pub server_filetransfer_id: u16,
pub filetransfer_key: String,
pub port: u16,
pub size: u64,
pub protocol: u8,
pub ip: Option<IpAddr>, // ← single optional IP
}
The server's notifystartdownload response sends ip as an optional single value, not an array. The TeaSpeak client's addresses[] is a higher-level abstraction (likely the client's own fallback logic), not a protocol feature.
Recommendation: Chanora follows tsclientlib's existing behavior — use msg.ip or fallback to connection address. No custom failover logic needed. If the TCP connection fails, the download fails and retries follow the exponential backoff strategy from the design doc.
Decision: Single address (from tsclientlib). No failover needed.