Optimizing encrypted P2P file transfer - from 225 to 441 MB/s

Part of the KEIBIDROP development blog. KEIBIDROP is in active development. Release is coming soon. KEIBIDROP transfers files between two peers over encrypted gRPC. The full stack: Disk I/O -> FUSE kernel -> FUSE daemon -> gRPC framing -> ChaCha20-Poly1305 -> TCP -> Peer We built micro-benchmarks for each layer and measured throughput with 1GB files on an Intel MacBook Pro. Baseline numbers Layer Throughput Overhead Raw disk (SSD) ~5 GB/s -- Raw gRPC (no encryption) 981 MB/s 5x vs disk Encrypted gRPC (ChaCha20) 437 MB/s 2.2x vs raw gRPC FUSE end-to-end 225 MB/s 1.9x vs encrypted gRPC The encryption layer costs 2.2x. FUSE adds another 1.9x. Six optimizations 1. Cache the AEAD cipher. The original code created a new ChaCha20-Poly1305 cipher for every message. Caching it in the constructor is safe because the nonce is a monotonic counter. // Before: creating cipher per-message aead , _ := chacha20poly1305 . NewX ( s . key ) // expensive! // After: created once in constructor type SecureWr