Artificial intelligence applications and tools are rapidly dominating the healthcare delivery space, and the rush has triggered a massive demand cycle and significant need for quick capital to acquire advanced computing hardware. Technology companies are rapidly purchasing high-performance graphics processing units (GPUs) and high-bandwidth memory (HBM) to power massive commercial large language models and applications to serve their end-users. In a similar fashion, visionary healthcare organizations and leaders are recognizing that with increasing AI powered medical models and applications, they too will require access to significant amounts of compute and memory infrastructure. While continuing to invest in existing cloud players may work for some, others are moving to develop their own sovereign, on-premise compute infrastructure.
Reliance on one of the large cloud players has been normative for years, especially as they often come with a significant amount of bundled services and support. But now, with rapidly increasing cloud and compute costs, the idea of building a sovereign hospital data center has started to gain popularity. DataBank describes why some leaders are choosing to invest in their own compute infrastructure, with lowered costs for observability and monitoring being one key reason. Specifically, healthcare requires a lot of steady-state and somewhat predictable computing workloads (e.g., medical AI diagnostics, imaging analytics, etc.). Since these healthcare models are often also high-stakes, sovereign compute infrastructure allows organizations to have a higher degree of audibility and observability as a means to ensure high degrees of patient safety and efficacy.
Other advantages include the fact that ownership of compute may reduce dependency on price fluctuations and gouging. A significant portion of the AI compute market is currently driven by the millions, if not billions, of daily active users for the common AI frontier models. With the rise in demand by retail users, compute providers are also facing significant shortages, raising their prices, and are rushing to develop more hardware. For healthcare organizations, investing in their own infrastructure may help reduce some of the dependencies and price fluctuations that may otherwise exist when relying on public cloud infrastructure.
Furthermore, healthcare leaders are also increasingly discussing how compute sovereignty may also lead to higher levels of privacy and data sovereignty. When organizations are able to become wholly vertically integrated, that is, provide end-to-end services from application to hardware for their entire service lifecycle, they ultimately control who owns their data and how it is used. This significantly minimizes cybersecurity risks by decreasing the number of outside players that may have access to the data through cloud or hardware infrastructure. The HIPAA Journal recently published that nearly 75,000 data breaches happened in 2024 alone, with a steady year-over-year increase since 2023. With how rapidly healthcare organizations are relying on AI applications, data federation and new tools that are being intricately weaved through core data streams, the number of cybersecurity incidents is sure to increase in the coming decade.
However, owning sovereign compute is not as easy as it sounds. Many healthcare organizations still choose to delegate out their compute and infrastructure needs for a simple reason: to let the experts handle what they are best at managing. While hospital based data centers may make sense for many different reasons, there are also other reasons why they are challenging. For one, owning compute infrastructure also requires maintenance and constant upkeep, meaning that specialists have to be brought on board. They are also often capital expense heavy, requiring massive amounts of upfront cash investments and significant real-estate and physical infrastructure needs. Finally, they take time. Even with data center experts and how common the trade has become now, the average hospital data center may take anywhere from two to five years. For many organizations, working with an experienced technology or cloud provider that can provide turn-key service and the resources to rapidly deploy applications at scale makes a lot of sense, especially given that they don’t have to concern themselves with maintenance costs or issues.
Without a doubt, with rising memory and chip prices, healthcare organizations that are increasingly leveraging the best of AI applications will face significant cost and infrastructure decisions. By recognizing that computing power is just as vital to 21st-century medicine as electricity or clean water, hardware capabilities are going to be one of the most important aspects in the coming decades.
