Let’s be honest – we’re all getting sick of seeing AI plastered over every tech product. A trend that will not be slowing down any time soon. A recent victim of this trend is the PC market, as AMD, Intel, Microsoft, and Qualcomm have been talking about AI PCs for the last year or so. Microsoft will be hosting an event on March 21st that is titled The New Era of Work. AMD, Intel, and Qualcomm will have dueling keynotes for their respective CEOs at Computex in Taipei, Taiwan. Be prepared for a flood of AI PCs this year.
In all honesty, Tirias Research has been a promoter of AI processing as the next big wave of computing – using trained data to better process predictive models and user interfaces. The use of AI processing has made major improvements to such PC tasks as voice recognition, video upscaling, video call optimization, microphone noise reduction, and power/battery management. The role of Large Language Models (LLMs) to build AI that can generate novel material/content from text prompts (Generative AI or just GenAI) has unleased another level of applications for AI. With GenAI, some tasks such as image development, creative and business writing, chatbot assistants, and now even video creation are possible with minimal user input. But to date, GenAI has run in cloud datacenters with some limited client device examples. The processing requirements and the power requirements to run the ever-increasing demand for GenAI is threatening to break cloud data centers.
The next generation of AI PCs aim to change the run-away cloud data center requirements by making PCs capable of running GenAI workloads locally with reduced or no cloud support.
Who Defines an AI PC?
Recent reports have Microsoft setting a minimum requirement for its AI PCs as having at least 16GB of DRAM and a neural processing unit (NPU) with 40 TOPS (trillions of operations per second) or more. No Windows PC processor shipping today meets the 40 TOPS requirement. Qualcomm’s Snapdragon X Elite platform, which is expected to ship in the second half of 2024, should meet Microsoft’s performance requirements with around 45 TOPS and support for at least 16 GB of DRAM. Intel’s Meteor Lake platform is just a bit under those requirements at around 34 TOPS, but Intel’s Lunar Lake platform has a chance to hit the requirement when it ships later in 2024. AMD’s next Ryzen processor (Strix Point) may also meet or exceed the 40 TOPS requirement. With keynotes from AMD, Intel, and Qualcomm at this year’s Computex, we’ll know where they all stand in the race to meet Microsoft’s requirements.
This begs the question: should I wait for these new processors in the second half of the year or is Microsoft even the final arbiter of what makes an AI PC? AMD and Intel are showing some early examples of the capabilities of their platforms shipping today. Qualcomm has had an NPU in its Snapdragon processors for PCs since the Snapdragon 850 in 2018.
Apple M processors for MacBooks have had neural processors (referred to as the Neural Engine) since the release of the M1 in 2020 and it has embraced the idea of running more AI workloads on clients. From the recent MacBook Air product launch press release: “With the transition to Apple silicon, every Mac is a great platform for AI. M3 includes a faster and more efficient 16-core Neural Engine, along with accelerators in the CPU and GPU to boost on-device machine learning, making MacBook Air the world’s best consumer laptop for AI. Leveraging this incredible AI performance, macOS delivers intelligent features that enhance productivity and creativity, so users can enable powerful camera features, real-time speech to text, translation, text predictions, visual understanding, accessibility features, and much more.”
Nvidia would argue that any PC with an RTX 20 series discrete graphics processing unit (GPU) or later is an AI PC because you can run AI workloads on the Tensor Units in the GPU. Nvidia uses AI to scale up video game textures and display resolution for running raytracing graphics. The fact that most AI workloads are trained on Nvidia GPUs helps bolster their argument. Running inference workloads on GeForce RTX GPUs is relatively easy. The problem is that it’s not very power efficient, as compared with running AI models on dedicated NPUs.
The argument for running AI inference workloads on dedicated NPUs is that it is the most power efficient way to run AI workloads without killing the notebook’s battery life and notebooks outsell desktop PCs, especially for businesses.
How will AI PCs progress through the rest of 2024?
Microsoft is expected to make the push to support its AI CoPilot on client PCs, not just as a cloud service. CoPilot can change the way people use PCs and can boost productivity. It may become the biggest change to the PCs since the text-based DOS to the graphical user interface of Windows. While today’s PC processors with NPUs may offer some AI capability, we are very close to a step function increase in AI processing capabilities.
Intel has been active in developing relevant software with its AI PC Acceleration Program launched last year. The company claims it will enable AI on more than 100 million PCs through 2025, based on its Intel Core Ultra processors. The company is working with more than 100 ISVs and hopes to have more than 300 AI-accelerated features to enhance PC experiences across audio effects, content creation, gaming, security, streaming, video collaboration and more. Companies already involved in the program include Adobe, Audacity, BlackMagic, BufferZone, CyberLink, DeepRender, Fortemedia, MAGIX, Rewind AI, Skylum, Topaz, VideoCom, Webex, Wondershare Filmora, XSplit, and Zoom. Meanwhile AMD has focused more on the large ISV such as Microsoft and Adobe. Qualcomm hopes its AI Stack and AI Hub and the years of work it put into its mobile platforms will give it a jump start.
All this activity leads up to expected major developments from all of the players at the Microsoft event in late March and especially at Computex in June where we will learn more on how AI PCs will develop for the balance of the year. You will hear a lot about AI PCs throughout the year and it will become pervasive over the next two years. The benefits of AI PCs will be greatest for creative work initially, but AI capabilities will be embedded in many applications. Some AI capabilities will remain cloud-based, but many will benefit from local AI. The only question on whether to buy an AI PC this year is if you to be on the leading edge of this change.
Tirias Research tracks and consults for companies throughout the electronics ecosystem from semiconductors to systems and sensors to the cloud. Members of the Tirias Research team have consulted for AMD, Arm, Intel, Nvidia, Qualcomm, and other companies throughout the CPU and GPU ecosystems.
