Speculative KV coding: losslessly compressing KV cache by up to ~4×

Posted 3 days ago by kkm
123 points

https://fergusfinn.com/blog/kv-entropy-coder/

9 comments

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..