Sors: a Rust proxy that reorders prompts to maximize vLLM prefix cache hits

• Posted 1 hour ago by flaccount
• 2 points