edison/chanora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: file transfer system (avatar/icon download with cacache) (#40)

Browse Source 
* docs(architecture): add file transfer design, research, and implementation plan

* feat(cache): add chanora_cache crate with cacache-backed blob cache

- New chanora_cache crate: content-addressed blob store wrapping cacache
- BlobCache API: async put/get/remove/clear/total_size/evict
- Key validation: av_ prefix (32 hex chars), ic_ prefix (decimal digits)
- Cacache provides crash safety, SSRI integrity, content dedup
- Mtime-based eviction via cacache::list_sync + sort by timestamp
- 7 unit tests all passing
- Added to workspace members

* feat(protocol): add file download support for avatars and icons

- Add Request::DownloadFile variant with oneshot reply
- Add ProtocolClient::download_avatar(client_uid) and download_icon(icon_id)
- Track pending file downloads by FiletransferHandle
- Handle StreamItem::FileDownload: read bytes from TCP stream
- Handle StreamItem::FiletransferFailed: map to ProtocolError
- Add ProtocolError::FileTransfer(String) variant
- Add path helper tests for avatar/icon download paths
- No tsclientlib types leak across the adapter boundary

* feat(core): add blob cache wiring and avatar download orchestration

- Add chanora_cache dependency to Cargo.toml
- Add blob_cache field to ChanoraSession (Arc<Mutex<Option<BlobCache>>>)
- Add init_cache() method: creates BlobCache, runs eviction
- Add get_avatar() method: cache-first, download on miss, store in cache
- Add clear_cache() and cache_size() methods for cache management
- Add CoreError::Cache variant for BlobCacheError conversion
- Add avatar_cache integration test

* feat(bridge): add init_cache, download_avatar, and cache management functions

- Add init_cache(dir) bridge function
- Add download_avatar(avatar_hash, client_uid) bridge function
- Add clear_file_cache() and file_cache_size() bridge functions
- Map CoreError::Cache and ProtocolError::FileTransfer in BridgeError

* feat(flutter): add cache initialization wiring and avatar download shims

- Add wireCache() to app_bootstrap using getApplicationCacheDirectory()
- Call wireCache() after wireStorage() in main bootstrap flow
- Add Dart-side initCache and downloadAvatar wrapper shims in api.dart
- Update Cargo.lock for new chanora_cache dependency

* feat(core): FileTransferService with coalescing, throttling, negative cache

- New file_transfer module with FileTransferService struct
- Semaphore(2) throttles concurrent downloads
- In-flight HashMap coalesces duplicate avatar requests
- 5-min negative cache short-circuits ServerRejected misses
- ChanoraSession delegates get_avatar through the service
- connect/disconnect update shared protocol handle
- clear_cache/cache_size delegate to service
- 2 new unit tests (cached hit, negative cache)

* feat(core,bridge): add get_icon with coalescing and negative cache

- FileTransferService::get_icon() mirrors get_avatar pattern
- ChanoraSession::get_icon() delegates through FileTransferService
- Bridge download_icon() exposed for Flutter
- Dart downloadIcon() shim added
- Uses PREFIX_ICON (ic_<crc32u>) cache key format
- 1 new unit test (cached icon hit)

* fix(core,protocol): simplify store_protocol and add download size cap

- store_protocol: always write to shared Arc<Mutex<Option<ProtocolClient>>>;
  the FileTransferService holds the same Arc so it sees updates automatically
- read_download_bytes: reject downloads exceeding 10 MB to prevent
  malicious servers from causing OOM
This commit is contained in:
Edison Jwa
2026-06-10 11:45:14 +09:00
committed by GitHub
parent 08d7ace25d
commit aa796d7395
18 changed files with 4376 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+367 -15
View File
@@ -148,12 +148,111 @@ version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "435a87a52755b8f27fcf321ac4f04b2802e337c8c4872923137471ec39c37532"
dependencies = [
"event-listener",
"event-listener 5.4.1",
"event-listener-strategy",
"futures-core",
"pin-project-lite",
]
[[package]]
name = "async-channel"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81953c529336010edd6d8e358f886d9581267795c61b19475b71314bffa46d35"
dependencies = [
"concurrent-queue",
"event-listener 2.5.3",
"futures-core",
]
[[package]]
name = "async-channel"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2"
dependencies = [
"concurrent-queue",
"event-listener-strategy",
"futures-core",
"pin-project-lite",
]
[[package]]
name = "async-executor"
version = "1.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a"
dependencies = [
"async-task",
"concurrent-queue",
"fastrand",
"futures-lite",
"pin-project-lite",
"slab",
]
[[package]]
name = "async-global-executor"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05b1b633a2115cd122d73b955eadd9916c18c8f510ec9cd1686404c60ad1c29c"
dependencies = [
"async-channel 2.5.0",
"async-executor",
"async-io",
"async-lock",
"blocking",
"futures-lite",
"once_cell",
]
[[package]]
name = "async-io"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc"
dependencies = [
"autocfg",
"cfg-if",
"concurrent-queue",
"futures-io",
"futures-lite",
"parking",
"polling",
"rustix",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-lock"
version = "3.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311"
dependencies = [
"event-listener 5.4.1",
"event-listener-strategy",
"pin-project-lite",
]
[[package]]
name = "async-process"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75"
dependencies = [
"async-channel 2.5.0",
"async-io",
"async-lock",
"async-signal",
"async-task",
"blocking",
"cfg-if",
"event-listener 5.4.1",
"futures-lite",
"rustix",
]
[[package]]
name = "async-recursion"
version = "1.1.1"
@@ -165,6 +264,57 @@ dependencies = [
"syn",
]
[[package]]
name = "async-signal"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485"
dependencies = [
"async-io",
"async-lock",
"atomic-waker",
"cfg-if",
"futures-core",
"futures-io",
"rustix",
"signal-hook-registry",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-std"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c8e079a4ab67ae52b7403632e4618815d6db36d2a010cfe41b02c1b1578f93b"
dependencies = [
"async-channel 1.9.0",
"async-global-executor",
"async-io",
"async-lock",
"async-process",
"crossbeam-utils",
"futures-channel",
"futures-core",
"futures-io",
"futures-lite",
"gloo-timers",
"kv-log-macro",
"log",
"memchr",
"once_cell",
"pin-project-lite",
"pin-utils",
"slab",
"wasm-bindgen-futures",
]
[[package]]
name = "async-task"
version = "4.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de"
[[package]]
name = "async-trait"
version = "0.1.89"
@@ -257,6 +407,12 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c7f02d4ea65f2c1853089ffd8d2787bdbc63de2f0d29dedbcf8ccdfa0ccd4cf"
[[package]]
name = "base64"
version = "0.21.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"
[[package]]
name = "base64"
version = "0.22.1"
@@ -308,6 +464,19 @@ dependencies = [
"objc2",
]
[[package]]
name = "blocking"
version = "1.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21"
dependencies = [
"async-channel 2.5.0",
"async-task",
"futures-io",
"futures-lite",
"piper",
]
[[package]]
name = "build-target"
version = "0.4.0"
@@ -352,6 +521,32 @@ version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
[[package]]
name = "cacache"
version = "13.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c5063741c7b2e260bbede781cf4679632dd90e2718e99f7715e46824b65670b"
dependencies = [
"async-std",
"digest 0.10.7",
"either",
"futures",
"hex",
"libc",
"memmap2",
"miette",
"reflink-copy",
"serde",
"serde_derive",
"serde_json",
"sha1",
"sha2",
"ssri",
"tempfile",
"thiserror 1.0.69",
"walkdir",
]
[[package]]
name = "cast"
version = "0.3.0"
@@ -475,11 +670,23 @@ dependencies = [
"tracing-subscriber",
]
[[package]]
name = "chanora_cache"
version = "0.2.0-beta.1"
dependencies = [
"cacache",
"tempfile",
"thiserror 2.0.18",
"tokio",
"tracing",
]
[[package]]
name = "chanora_core"
version = "0.2.0-beta.1"
dependencies = [
"chanora_audio",
"chanora_cache",
"chanora_diagnostics",
"chanora_prefetch",
"chanora_protocol",
@@ -515,7 +722,7 @@ name = "chanora_protocol"
version = "0.2.0-beta.1"
dependencies = [
"async-trait",
"base64",
"base64 0.22.1",
"chanora_resolver",
"futures",
"reqwest 0.13.4",
@@ -554,7 +761,7 @@ dependencies = [
name = "chanora_storage"
version = "0.2.0-beta.1"
dependencies = [
"base64",
"base64 0.22.1",
"chacha20poly1305",
"keyring",
"rand 0.8.6",
@@ -1239,6 +1446,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "event-listener"
version = "2.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0206175f82b8d6bf6652ff7d71a1e27fd2e4efde587fd368662814d6ec1d9ce0"
[[package]]
name = "event-listener"
version = "5.4.1"
@@ -1256,7 +1469,7 @@ version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93"
dependencies = [
"event-listener",
"event-listener 5.4.1",
"pin-project-lite",
]
@@ -1559,6 +1772,18 @@ dependencies = [
"time",
]
[[package]]
name = "gloo-timers"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbb143cf96099802033e0d4f4963b19fd2e0b728bcf076cd9cf7f6634f092994"
dependencies = [
"futures-channel",
"futures-core",
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "group"
version = "0.13.0"
@@ -1837,7 +2062,7 @@ version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"futures-channel",
"futures-util",
@@ -2142,6 +2367,15 @@ dependencies = [
"zeroize",
]
[[package]]
name = "kv-log-macro"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de8b303297635ad57c9f5059fd9cee7a47f8e8daa09df0fcd07dd39fb22977f"
dependencies = [
"log",
]
[[package]]
name = "lazy_static"
version = "1.5.0"
@@ -2226,6 +2460,9 @@ name = "log"
version = "0.4.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "113b30b4cd05f7c06868fdb2854f66a7b9fece9a48425351cd532e810d74024f"
dependencies = [
"value-bag",
]
[[package]]
name = "lru-slab"
@@ -2274,6 +2511,15 @@ version = "2.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8"
[[package]]
name = "memmap2"
version = "0.5.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83faa42c0a078c393f6b29d5db232d8be22776a891f8f56e5284faee4a20b327"
dependencies = [
"libc",
]
[[package]]
name = "memoffset"
version = "0.9.1"
@@ -2283,6 +2529,29 @@ dependencies = [
"autocfg",
]
[[package]]
name = "miette"
version = "5.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59bb584eaeeab6bd0226ccf3509a69d7936d148cf3d036ad350abe35e8c6856e"
dependencies = [
"miette-derive",
"once_cell",
"thiserror 1.0.69",
"unicode-width",
]
[[package]]
name = "miette-derive"
version = "5.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49e7bc1560b95a3c4a25d03de42fe76ca718ab92d1a22a55b9b4cf67b3ae635c"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "mime"
version = "0.3.17"
@@ -2813,6 +3082,17 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "piper"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1"
dependencies = [
"atomic-waker",
"fastrand",
"futures-io",
]
[[package]]
name = "pkcs8"
version = "0.10.2"
@@ -2857,6 +3137,20 @@ dependencies = [
"plotters-backend",
]
[[package]]
name = "polling"
version = "3.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218"
dependencies = [
"cfg-if",
"concurrent-queue",
"hermit-abi",
"pin-project-lite",
"rustix",
"windows-sys 0.61.2",
]
[[package]]
name = "poly1305"
version = "0.8.0"
@@ -3183,6 +3477,18 @@ dependencies = [
"syn",
]
[[package]]
name = "reflink-copy"
version = "0.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13362233b147e57674c37b802d216b7c5e3dcccbed8967c84f0d8d223868ae27"
dependencies = [
"cfg-if",
"libc",
"rustix",
"windows",
]
[[package]]
name = "regex"
version = "1.12.3"
@@ -3218,7 +3524,7 @@ version = "0.12.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"futures-core",
"http",
@@ -3256,7 +3562,7 @@ version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"encoding_rs",
"futures-core",
@@ -3672,6 +3978,17 @@ dependencies = [
"digest 0.10.7",
]
[[package]]
name = "sha1"
version = "0.10.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
]
[[package]]
name = "sha2"
version = "0.10.9"
@@ -3851,6 +4168,23 @@ dependencies = [
"der",
]
[[package]]
name = "ssri"
version = "9.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da7a2b3c2bc9693bcb40870c4e9b5bf0d79f9cb46273321bf855ec513e919082"
dependencies = [
"base64 0.21.7",
"digest 0.10.7",
"hex",
"miette",
"serde",
"sha-1",
"sha2",
"thiserror 1.0.69",
"xxhash-rust",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
@@ -4349,7 +4683,7 @@ name = "ts-bookkeeping"
version = "0.1.0"
source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917abbf6a0c8442a79742d6d2d40ecf71e"
dependencies = [
"base64",
"base64 0.22.1",
"heck",
"itertools 0.14.0",
"num-derive",
@@ -4370,7 +4704,7 @@ version = "0.2.0"
source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917abbf6a0c8442a79742d6d2d40ecf71e"
dependencies = [
"audiopus",
"base64",
"base64 0.22.1",
"futures",
"git-testament",
"hickory-net",
@@ -4398,7 +4732,7 @@ version = "0.2.0"
source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917abbf6a0c8442a79742d6d2d40ecf71e"
dependencies = [
"aes",
"base64",
"base64 0.22.1",
"curve25519-dalek-ng",
"eax",
"futures",
@@ -4427,7 +4761,7 @@ name = "tsproto-packets"
version = "0.1.0"
source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917abbf6a0c8442a79742d6d2d40ecf71e"
dependencies = [
"base64",
"base64 0.22.1",
"bitflags 2.12.1",
"num-derive",
"num-traits",
@@ -4442,7 +4776,7 @@ name = "tsproto-structs"
version = "0.2.0"
source = "git+https://github.com/EdisonJwa/tsclientlib.git?branch=fix%2Fp256-short-coordinate-pad#8b7a3226c692319b714ea1d32fd5ded05911aa40"
dependencies = [
"base64",
"base64 0.22.1",
"csv",
"heck",
"once_cell",
@@ -4455,7 +4789,7 @@ name = "tsproto-structs"
version = "0.2.0"
source = "git+https://github.com/ReSpeak/tsclientlib.git?rev=04aa2491#04aa24917abbf6a0c8442a79742d6d2d40ecf71e"
dependencies = [
"base64",
"base64 0.22.1",
"csv",
"heck",
"once_cell",
@@ -4468,7 +4802,7 @@ name = "tsproto-types"
version = "0.1.0"
source = "git+https://github.com/EdisonJwa/tsclientlib.git?branch=fix%2Fp256-short-coordinate-pad#8b7a3226c692319b714ea1d32fd5ded05911aa40"
dependencies = [
"base64",
"base64 0.22.1",
"bitflags 2.12.1",
"curve25519-dalek-ng",
"elliptic-curve",
@@ -4512,6 +4846,12 @@ version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-width"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
[[package]]
name = "unicode-xid"
version = "0.2.6"
@@ -4570,6 +4910,12 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "value-bag"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ba6f5989077681266825251a52748b8c1d8a4ad098cc37e440103d0ea717fc0"
[[package]]
name = "vcpkg"
version = "0.2.15"
@@ -5256,6 +5602,12 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "xxhash-rust"
version = "0.8.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fdd20c5420375476fbd4394763288da7eb0cc0b8c11deed431a91562af7335d3"
[[package]]
name = "yoke"
version = "0.8.2"
@@ -5289,7 +5641,7 @@ dependencies = [
"async-recursion",
"async-trait",
"enumflags2",
"event-listener",
"event-listener 5.4.1",
"futures-core",
"futures-lite",
"hex",