feat: file transfer system (avatar/icon download with cacache) (#40)

* docs(architecture): add file transfer design, research, and implementation plan

* feat(cache): add chanora_cache crate with cacache-backed blob cache

- New chanora_cache crate: content-addressed blob store wrapping cacache
- BlobCache API: async put/get/remove/clear/total_size/evict
- Key validation: av_ prefix (32 hex chars), ic_ prefix (decimal digits)
- Cacache provides crash safety, SSRI integrity, content dedup
- Mtime-based eviction via cacache::list_sync + sort by timestamp
- 7 unit tests all passing
- Added to workspace members

* feat(protocol): add file download support for avatars and icons

- Add Request::DownloadFile variant with oneshot reply
- Add ProtocolClient::download_avatar(client_uid) and download_icon(icon_id)
- Track pending file downloads by FiletransferHandle
- Handle StreamItem::FileDownload: read bytes from TCP stream
- Handle StreamItem::FiletransferFailed: map to ProtocolError
- Add ProtocolError::FileTransfer(String) variant
- Add path helper tests for avatar/icon download paths
- No tsclientlib types leak across the adapter boundary

* feat(core): add blob cache wiring and avatar download orchestration

- Add chanora_cache dependency to Cargo.toml
- Add blob_cache field to ChanoraSession (Arc<Mutex<Option<BlobCache>>>)
- Add init_cache() method: creates BlobCache, runs eviction
- Add get_avatar() method: cache-first, download on miss, store in cache
- Add clear_cache() and cache_size() methods for cache management
- Add CoreError::Cache variant for BlobCacheError conversion
- Add avatar_cache integration test

* feat(bridge): add init_cache, download_avatar, and cache management functions

- Add init_cache(dir) bridge function
- Add download_avatar(avatar_hash, client_uid) bridge function
- Add clear_file_cache() and file_cache_size() bridge functions
- Map CoreError::Cache and ProtocolError::FileTransfer in BridgeError

* feat(flutter): add cache initialization wiring and avatar download shims

- Add wireCache() to app_bootstrap using getApplicationCacheDirectory()
- Call wireCache() after wireStorage() in main bootstrap flow
- Add Dart-side initCache and downloadAvatar wrapper shims in api.dart
- Update Cargo.lock for new chanora_cache dependency

* feat(core): FileTransferService with coalescing, throttling, negative cache

- New file_transfer module with FileTransferService struct
- Semaphore(2) throttles concurrent downloads
- In-flight HashMap coalesces duplicate avatar requests
- 5-min negative cache short-circuits ServerRejected misses
- ChanoraSession delegates get_avatar through the service
- connect/disconnect update shared protocol handle
- clear_cache/cache_size delegate to service
- 2 new unit tests (cached hit, negative cache)

* feat(core,bridge): add get_icon with coalescing and negative cache

- FileTransferService::get_icon() mirrors get_avatar pattern
- ChanoraSession::get_icon() delegates through FileTransferService
- Bridge download_icon() exposed for Flutter
- Dart downloadIcon() shim added
- Uses PREFIX_ICON (ic_<crc32u>) cache key format
- 1 new unit test (cached icon hit)

* fix(core,protocol): simplify store_protocol and add download size cap

- store_protocol: always write to shared Arc<Mutex<Option<ProtocolClient>>>;
  the FileTransferService holds the same Arc so it sees updates automatically
- read_download_bytes: reject downloads exceeding 10 MB to prevent
  malicious servers from causing OOM
This commit is contained in:
Edison Jwa
2026-06-10 11:45:14 +09:00
committed by GitHub
parent 08d7ace25d
commit aa796d7395
18 changed files with 4376 additions and 58 deletions
+4 -1
View File
@@ -93,7 +93,10 @@ String _kAppVersion = appSemverBaseline;
Future<void> main() async {
WidgetsFlutterBinding.ensureInitialized();
await RustLib.init();
unawaited(wireStorage());
unawaited(() async {
await wireStorage();
await wireCache();
}());
unawaited(wireConnectivity());
wireAudioLifecycle();
await configureBundledVadModels();
@@ -15,15 +15,22 @@ typedef StorageDirectoryProvider = Future<Directory> Function();
typedef StorageInitializer = Future<void> Function(String dir);
Future<void>? _storageInitFuture;
Future<void>? _cacheInitFuture;
Future<void>? _vadBootstrapFuture;
StorageDirectoryProvider _storageDirectoryProvider =
getApplicationSupportDirectory;
StorageDirectoryProvider _cacheDirectoryProvider = getApplicationCacheDirectory;
StorageInitializer _storageInitializer = _defaultStorageInitializer;
StorageInitializer _cacheInitializer = _defaultCacheInitializer;
Future<void> _defaultStorageInitializer(String dir) {
return rust.initStorage(dir: dir);
}
Future<void> _defaultCacheInitializer(String dir) {
return rust.initCache(dir: dir);
}
Future<File> _copyBundledAssetToDocuments({
required String assetPath,
required String fileName,
@@ -121,16 +128,50 @@ Future<void> _wireStorageImpl() async {
}
}
Future<void> wireCache() async {
final existing = _cacheInitFuture;
if (existing != null) {
await existing;
return;
}
final initFuture = _wireCacheImpl();
_cacheInitFuture = initFuture;
await initFuture;
}
Future<void> _wireCacheImpl() async {
var initialized = false;
try {
final dir = await _cacheDirectoryProvider();
await _cacheInitializer(dir.path);
initialized = true;
} catch (_) {
// Best-effort; missing cache just means protocol-owned assets are
// re-downloaded this session.
} finally {
if (!initialized) {
_cacheInitFuture = null;
}
}
}
@visibleForTesting
void debugResetStorageBootstrap({
StorageDirectoryProvider? storageDirectoryProvider,
StorageDirectoryProvider? cacheDirectoryProvider,
StorageInitializer? storageInitializer,
StorageInitializer? cacheInitializer,
}) {
_storageInitFuture = null;
_cacheInitFuture = null;
_vadBootstrapFuture = null;
_storageDirectoryProvider =
storageDirectoryProvider ?? getApplicationSupportDirectory;
_cacheDirectoryProvider =
cacheDirectoryProvider ?? getApplicationCacheDirectory;
_storageInitializer = storageInitializer ?? _defaultStorageInitializer;
_cacheInitializer = cacheInitializer ?? _defaultCacheInitializer;
}
rust.BridgeNetworkState _mapConnectivity(List<ConnectivityResult> results) {
@@ -226,6 +226,41 @@ String exportDiagnostics() => RustLib.instance.api.crateApiExportDiagnostics();
Future<void> initStorage({required String dir}) =>
RustLib.instance.api.crateApiInitStorage(dir: dir);
/// Wire the blob cache to a platform-private cache directory.
Future<void> initCache({required String dir}) {
final api = RustLib.instance.api as dynamic;
try {
return api.crateApiInitCache(dir: dir) as Future<void>;
} on NoSuchMethodError {
return Future.value();
}
}
/// Return an avatar from cache when present; otherwise download it.
Future<List<int>?> downloadAvatar({
required String avatarHash,
required String clientUid,
}) {
final api = RustLib.instance.api as dynamic;
try {
return api.crateApiDownloadAvatar(
avatarHash: avatarHash,
clientUid: clientUid,
) as Future<List<int>?>;
} on NoSuchMethodError {
return Future.value();
}
}
Future<List<int>?> downloadIcon({required BigInt iconId}) {
final api = RustLib.instance.api as dynamic;
try {
return api.crateApiDownloadIcon(iconId: iconId) as Future<List<int>?>;
} on NoSuchMethodError {
return Future.value();
}
}
/// List persisted bookmarks.
Future<List<BridgeBookmark>> listBookmarks() =>
RustLib.instance.api.crateApiListBookmarks();