Qualcomm recently made several major AI announcements at the Mobile World Congress Barcelona event. Each announcement supports Qualcomm’s goal of running AI at scale everywhere, including the edge. The announcements included:
- At-scale AI running on a wide range of next-generation edge devices including PCs, cars, phones, industrial IoT devices, Wi-Fi access points and cellular infrastructure.
- Qualcomm AI Hub providing resources for developers, including more than 75 optimized AI models for Snapdragon and other Qualcomm platforms.
- Qualcomm researchers developing large multimodal models and customized large vision models that can run on Android smartphones and Windows PCs.
These advances rely on new Qualcomm hardware. The company’s Snapdragon 8 Gen 3 SoC for mobile devices was announced at MWC, while the Snapdragon X Elite for PCs was previously announced a few months ago. These chipsets enable on-device AI for a number of devices such as the Samsung Galaxy S24 smartphone, the latest iPhone and many phones manufactured in China. Within the next few months, Qualcomm is expected to introduce its AI PC SoCs equipped with a CPU, a GPU and an NPU capable of processing speeds of 45 TOPS.
Qualcomm also made announcements about the AI-powered Snapdragon X80 5G Modem and the FastConnect 7900 next-generation Wi-Fi 7 chip. Those products have been covered in detail by my colleague Anshel Sag.
Here’s a more detailed look at the AI-related announcements.
Qualcomm AI Hub
AI-capable phones, PCs and other devices need new applications to make full use of their AI capabilities. Qualcomm created its AI Hub to provide developers with resources needed to create and deploy AI applications on Snapdragon or other Qualcomm platforms. One of the key AI Hub enablers is a library of AI models quantized and optimized for high performance on these platforms.
One thing that makes the hub attractive is its simplicity of use. A developer simply selects the desired model from a selection of 75-plus common AI and generative AI models, then chooses the framework reference—either TensorFlow, PyTorch or Onyx. The models are also available on Hugging Face and GitHub.
Once the specific model is chosen, the developer can select the target platform and the device. Then for deployment, the developer inserts a few lines of code to integrate the optimized model into the workflow to take advantage of on-device AI capabilities running on one of Qualcomm’s platforms.
In addition to a large number of capabilities such as image and text generation, the hub provides up to four times faster inference on devices powered by Snapdragon or other Qualcomm platforms.
Large AI Models Can Now Run On Android
At the event, Qualcomm successfully demonstrated the world’s first large language model and large multi-modal model running on an Android phone. The 7-billion-parameter LLM can accept text, images and voice, which allows a user to have a multi-turn conversation with their phone using multiple back-and-forth exchanges.
When users ask questions or provide inputs, the LMM AI responds accordingly. Multi-turn conversations require AI to have contextual memory so it can remember the context and a complete history of information exchanged between the user and AI. Multi-turn LLM has the power to handle complex questions needed for sophisticated interactive applications such as customer service.
LoRA For Android Phones
Qualcomm also achieved another technical milestone with its first low rank adaptation to run on an Android phone. LoRA was originally developed by Microsoft to reduce model training complexity in terms of latency, cost and hardware requirements. LoRA provides a novel approach for generating images and differs from generative AI tools such as DALL-E. By reducing model complexity, LoRA reduces memory requirements and increases efficiency so that lightweight versions of models can be customized and adapted without requiring the full model to be downloaded or fine-tuned.
Snapdragon X Elite Platform
Qualcomm built the Snapdragon X Elite to run AI. Its design includes a 4nm SoC architecture, a 12-core Qualcomm Oryon CPU with dual-core boost and an integrated Qualcomm Adreno GPU for graphics. The device eliminates the nuisance of frequent charging because a single battery charge supports multi-day usage. One of its most important advantages is that the Snapdragon X Elite is also capable of running generative AI LLM models with 13 billion parameters.
To demonstrate the SoC’s capabilities, Qualcomm ran a performance comparison between the Snapdragon X Elite PC and a competitive x86 equipped with its most efficient GPU, CPU and NPU configuration. The comparison used GIMP, a powerful open-source raster graphics editor equipped with a Stable Diffusion plugin for GAI image generation. As indicated by the graphic, the Snapdragon X Elite with a 45 TOPS NPU provided image generation 3 times faster than the x86 competitor.
Qualcomm’s Role In Localizing AI
Qualcomm’s AI announcements at MWC Barcelona all represented important breakthroughs. The company has managed to push AI and computing across previous device barriers, from smartphones to PCs. Its Snapdragon 8 Gen 3 SoC for mobile devices and Snapdragon X Elite PC platform can carry out various AI functions across generative AI, computer vision and natural language processing. And Qualcomm has also empowered developers with enhanced AI capabilities.
The ability to run large multimodal models and lightweight versions of models on Android phones opens up new capabilities across a wide range of domains. Android users will be able to interact with virtual assistants for applications such as financial services, healthcare and e-commerce, to name just a few. AI also gives a completely new dimension to gaming. AI gaming agents not only make for challenging opponents, but they can also generate new game content or dynamically adjust rules and game play according to the player’s expertise or desire for a greater challenge. Virtually every game becomes more challenging and realistic when AI is applied to it. By being able to run complex AI models efficiently, Android phones can provide better user experiences and improved functionality through natural language processing and computer vision.
Full democratization of AI will be heavily influenced by Qualcomm’s commercialization of on-device AI at scale at the edge and elsewhere, a concept that supports the industry’s goal of intelligent computing everywhere. Collectively, the MWC announcements have positioned Qualcomm as a key player in the AI landscape. With these capabilities, it should be able to drive AI adoption on a massive scale.
