AMDs hardware teams have tried to redefine AI inferencing with powerful chips like the Ryzen AI Max and Threadripper. But in software, the company has been largely absent where PCs are concerned. Thats changing, AMD executives say.
AMDs Advancing AI event Thursday focused on enterprise-class GPUs like its Instinct lineup. But its a software platform you may not have heard of, called ROCm, that AMD depends upon just as much. AMD is releasing ROCm 7 today, which the company says can boost AI inferencing by three times through the software alone. And its finally coming to Windows to battle Nvidias CUDA supremacy.
Radeon Open Compute (ROCm) is AMDs open software stack for AI computing, with drivers and tools to run AI workloads. Remember the Nvidia GeForce RTX 5060 debacle of a few weeks back? Without a software driver, Nvidias latest GPU was a lifeless hunk of silicon.
Early on, AMD was in the same pickle. Without the limitless coffers of companies like Nvidia, AMD made a choice: it would prioritize big businesses with ROCm and its enterprise GPUs instead of client PCs. Ramine Roane, corporate vice president of the AI solutions group, called that a sore point: We focused ROCm on the cloud GPUs, but it wasnt always working on the endpoint so were fixing that.
Mark Hachman / Foundry
In todays world, simply shipping the best product isnt always enough. Capturing customers and partners willing to commit to the product is a necessity. Its why former Microsoft CEO Steve Ballmer famously chanted Developers developers developers on stage; when Sony built a Blu-ray drive into the PlayStation, movie studios gave the new video format a critical mass that the rival HD-DVD format didnt have.
Now, AMDs Roane said that the company belatedly realized that AI developers like Windows, too. It was a decision to basically not use resources to port the software to Windows, but now we realize that, hey, developers actually really care about that, he said.
ROCm will be supported by PyTorch in preview in the third quarter of 2025, and by ONNX-EP in July, Roane said.
All this means is that AMD processors will finally gain a much larger presence in AI applications, which means that if you own a laptop with a Ryzen AI processor, a desktop with a Ryzen AI Max chip, or a desktop with a Radeon GPU inside, it will have more opportunities to tap into AI applications. PyTorch, for example, is a machine-learning library that popular AI models like Hugging Faces Transformers run on top of. It should mean that it will be much easier for AI models to take advantage of Ryzen hardware.
ROCm will also be added to in box Linux distributions, too: Red Hat (in the second half of 2025), Ubuntu (the same) and SuSE.
Roane also helpfully provided some context over what model size each AMD platform should be able to run, from a Ryzen AI 300 notebook on up to a Threadripper platform.
Mark Hachman / Foundry
The AI performance improvements that ROCm 7 adds are substantial: a 3.2X performance improvement in Llama 3.1 70B, 3.4X in Qwen2-72B, and 3.8X in DeepSeek R1. (The B stands for the number of parameters, in billions; the higher the parameters, the generally higher the quality of the outputs.) Today, those numbers matter more than they have in the past, as Roane said that inferencing chips are showing steeper growth than processors used for training.
(Training generates the AI models used in products like ChatGPT or Copilot. Inferencing refers to the actual process of using AI. In other words, you might train an AI to know everything about baseball; when you ask it if Babe Ruth was better than Willie Mays, youre using inferencing.)
Mark Hachman / Foundry
AMD said that the improved ROCm stack also offered the same training performance, or about three times the previous generation. Finally, AMD said that its own MI355X running the new ROCm software would outperfom an Nvidia B200 by 1.3X on the DeepSeek R1 model, with 8-bit floating-point accuracy.
Again, performance matters in AI, the goal is to push out as many AI tokens as quickly as possible; in games, its polygons or pixels instead. Simply offering developers a chance to take advantage of the AMD hardware you already own is a win-win, for you and AMD alike.
The one thing that AMD doesnt have is a consumer-focused application to encourage users to use AI, whether it be LLMs, AI art, or something else. Intel publishes AI Playground, and Nvidia (though it doesnt own the technology) worked with a third-party developer for its own application, LM Studio. One of the convenient features of AI Playground is that every model available has been quantized, or tuned, for Intels hardware.
Roane said that similarly-tuned models exist for AMD hardware like the Ryzen AI Max. However, consumers have to go to repositories like Hugging Face and download them themselves.
Roane called AI Playground a good idea. No specific plans right now, but its definitely a direction we would like to move, he said, in response to a question from PCWorld.com.
