AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization

  • Posted 2 hours ago by khalidkhair
  • 2 points
https://nvidia.github.io/Model-Optimizer/announcements/autoquant-kv.html

0 comments