Ask HN: 2x Arc A770 or 1x Radeon 7900 XTX for llama.cpp

  • Posted 3 days ago by danielEM
  • 5 points
Can't find "apple to apple" comparison on performance on QWQ 32b (4bit), can anyone help me with decision on which solution to pick?

From what I dig so far it looks like dual Arc A770 is supported by llama.cpp. And saw some reports that llama.cpp on top of IPEX-LLM is fastest way for inference on intel card.

On the other end there is more expensive 7900 XTX on which AMD claims (Jan '25) that inference is faster than on 4090.

So - what is the state of the art as of today, how does one compare to another (apple to apple)? What is tokens/s diff?

3 comments

    Loading..
    Loading..
    Loading..