CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving was presented at ACM SIGCOMM’24 .