Vultr, one of the world’s largest privately held cloud computing platform, is diversifying its AI cloud inference offering with the introduction of AMD Instinct MI300X accelerators.
The AMD Instinct MI300X GPUs are designed to deliver competitive performance, particularly in inference tasks, which are critical for deploying AI models in real-world applications. AMD’s latest MLPerf inference results showcase the MI300X’s ability to compete closely with Nvidia’s H100, although Nvidia’s H200 still holds a performance edge.
The MI300X’s performance in MLPerf benchmarks, particularly with the Llama2-70B model, validates its potential in real-world applications. The GPU’s high compute power, extensive memory and optimized ROCm software stack contribute to its strong competitive performance. These attributes are crucial as large language models continue to scale up, necessitating efficient and cost-effective solutions for both inference and training.
One of the standout features of the MI300X is its impressive memory capacity. With 192 GB of HBM3 memory and a peak memory bandwidth of 5.3 TB/s, the MI300X is well-equipped to handle state-of-the-art LLMs like Meta’s Llama and even larger models in the future. This extensive memory capacity not only supports the growing size and complexity of large language models but also ensures cost-effective performance, making it a no-brainer for enterprises looking to scale their AI capabilities.
Vultr’s partnership with AMD capitalizes on the composable cloud infrastructure to unlock new frontiers of GPU-accelerated workloads, spanning from the data center to the edge. This partnership is particularly significant as it combines Vultr’s scalable cloud platform with AMD’s advanced processing capabilities, providing businesses with the flexibility to scale their compute resources seamlessly as demand grows. The integration of AMD ROCm open software further enhances this environment, offering an industry-leading platform for AI development and deployment.
Vultr’s offering promises improved price-to-performance ratios, scalable compute resources and optimized workload management. For enterprises engaged in AI research and development, this translates to accelerated discovery cycles and more efficient innovation processes. Moreover, the platform is specifically optimized for AI inference, enabling faster, more scalable and energy-efficient processing of AI models.
Vultr is not the only one acknowledging the potential of AMD’s latest offering with this announcement. Microsoft Azure launched its ND MI300X v5 VM series powered by these GPUs earlier this year. Oracle Cloud has also added the MI300X to its lineup, expanding the choice of available AI accelerators to customers.
As cloud providers look to diversify their GPU portfolios, Vultr’s collaboration with AMD stands out as an example of how the industry is evolving to meet the diverse needs of AI workloads. It is a clear indication that the future of enterprise AI is in open, flexible environments that can adapt to the rapidly changing artificial intelligence landscape.
