> The Pro 9000 WX-Series processors are also designed for local AI deployments, including fine-tuning models, inference, and AI application development. “When running a context-based prompting inference test using DeepSeek R1 32B, we are seeing a 49 percent better performance of Threadripper Pro 9000 over Intel,” claims AMD.
Yup, that's a thing. Mostly along the lines of "CPU memory is cheap, GPU memory is expensive, if CPU performance is made good enough we can run really really big models for much less because of cheaper RAM"
Because it is the same chip, with higher clock speeds and different guarantees. The Threadripper WX series have a lot more bandwidth than the Threadripper X chips which will be considerably cheaper.
> The Pro 9000 WX-Series processors are also designed for local AI deployments, including fine-tuning models, inference, and AI application development. “When running a context-based prompting inference test using DeepSeek R1 32B, we are seeing a 49 percent better performance of Threadripper Pro 9000 over Intel,” claims AMD.
Are they using the CPU as a GPU here?
Yup, that's a thing. Mostly along the lines of "CPU memory is cheap, GPU memory is expensive, if CPU performance is made good enough we can run really really big models for much less because of cheaper RAM"
I'm all for it. It enables those really big models with a relatively small number of active parameters.
Pricing is disappointing - no cheaper than the existing AMD server equivalent Epyc Turin 9655 ($11.5k for 96 cores). https://www.amd.com/en/products/processors/server/epyc/9005-...
Significantly worse price/core than the earlier Threadripper 64 core 3990x for $3900.
Because it is the same chip, with higher clock speeds and different guarantees. The Threadripper WX series have a lot more bandwidth than the Threadripper X chips which will be considerably cheaper.
3
Just curious, are the older versions just... gone? It seems their availability is quite limited.
Also, does Intel have something comparable?
> does Intel have something comparable
Not for the last 7 years.
Sapphire/Emerald Rapids and Xeon Max (HBM) do come somewhat close, if you don’t mind absurd TDP.