This is my fourth and last blog on digital storage and memory projections for 2025. The first two articles focused on digital storage and memory devices including magnetic tape, HDDs, SSDs as well as NAND, DRAM and emerging non-volatile memories. The third was an update on optical storage, since several optical storage startups targeting the archiving and digital preservation market came to the forefront in 2024 and appear to be poised to deliver prototype products in 2025 and beyond. This article will focus on developments in storage systems and software and their use in workflows as well as additional insights on the future of storage devices and systems.
2024 saw the recovery of the overall storage and memory markets, although NAND flash and SSD markets have had only a partial recovery this year with most growth in the enterprise and data center markets. In September 2024 IDC released a report stating that the worldwide enterprise external OEM storage systems are expected to grow an average of 4.3% over a 5-year period.
IDC attributed the growth to, “Positive developments and drivers for growing demand include increasing demand for flash storage (typically, all-flash arrays) to support projects related to artificial intelligence, both for training and inferencing” and “increasing demand for flash media across both external storage systems and servers.” They also say that IaaS solutions are leading more companies to repatriate or planning to repatriate workloads from shared cloud to dedicated IT environments.
On the other hand, IDC, in another September report, forecast that 2024 cloud infrastructure spending (computing and storage) will grow 48.8% over 2023. Much of this spending growth was due to the increasing costs for GPU servers. Actual unit cloud growth in the same period was 17.7%.
Shared cloud infrastructure spending is expected to grow 57.9% Y/Y with dedicated cloud infrastructure spending growth projected at 20.4% Y/Y. Non-cloud infrastructure spending is expected to grow 11.7% Y/Y. Long term, IDC predicts spending on cloud infrastructure (compute and storage) to have a compound growth rate 18.1 from 2023-2028 and accounting for 76.4% of total compute and storage infrastructure spending by 2028 with shared cloud infrastructure spending being 78.6% of total cloud spending by 2028.
Developing AI workloads will have an impact on storage and memory demand. Eric Herzog, CMO at Infinidat, said that Enterprise storage infrastructure will take on a new role as the foundation for retrieval-augmented generation, RAG. RAG is a GenAI-centric framework for augmenting, refining and optimizing the output of AI models including large language models, LLMs, and small language models, SLMs). RAG can eliminate the need for continually re-training models, reducing the costs of these models and can reduce the incidence of AI hallucinations. Infinidat announced its RAG workflow deployment architecture in November 2024.
Infinidat is also joining efforts to defend data stores from cyberattacks. As part of its data protection capabilities. At the first sign of a cyberattack an immutable snapshot is taken of data to automatically reduce the impact of an attack. Infinidat also projects growth in hybrid multi-cloud storage in 2025. This brings together on-premises/private cloud and public cloud storage resources to enable high levels of flexibility, cost efficiency and use case-specific solutions. AWS and Azure are able to assist with such efforts. Infinidat announced such services with AWS and Microsoft in 2023 using the company’s InfuzeOS software defined storage, SDS, solution. Infinidat also says that non-VM based virtualization and Kubernetes/container deployments will increase in 2025.
Steve Leeper, VP of Product Marketing at Datadobi, says that, “The amount of unstructured data stored in both public cloud and private cloud environments will continue to grow. The impact of unstructured data management solutions that give customers the ability to manage data no matter where it is located will increase as the data in multiple environments accumulates.
It’s no longer realistic to ignore the fact that, in most organizations, data lives in a hybrid environment and global data management is required.” In addition, he indicated that with the growth of unstructured data there is a greater need for data insights to create GENAI-ready data.
Don Boxley, CEO and co-founder of DH2i says that AI can be used to create self-optimized high availability, HA, clusters, where, “AI eliminates…inefficiencies by continuously analyzing workloads and resource usage, allowing clusters to self-optimize and maintain peak performance without manual oversight.”
He also says that AI-driven HA clustering can help maintain HA across different cloud environments by managing clusters spanning multiple providers. Boxley says that, “AI simplifies cross-cloud HA by dynamically analyzing traffic and distributing workloads intelligently across providers, ensuring seamless performance and responsiveness.”
AI data demand will increase the need for archiving data. Gal Naor, CEO of StorONE says that, “The exponential growth of data in 2025 will significantly increase storage costs as organizations face the challenge of retaining cold data for extended periods. Although rarely accessed, this data must remain secure, easily accessible and cost-efficient.”
In addition, “Auto Tiering storage solutions will dynamically migrate inactive data to low-cost drives while ensuring rapid access for future analysis, reducing overall costs without compromising efficiency.” Also, “With rising cyber threats, fast and cost-effective recovery will be critical. Intelligent architectures will store snapshots on economical tiers while ensuring immediate availability for recovery, enhancing both preparedness and cost management.”
Skip Levens, Product Leader and AI Strategist for Media and Entertainment at Quantum, had some comments on AI growth in 2025 and its impact on digital storage demand. He says that, “In 2025, organizations that take a more pragmatic approach to AI—and its underlying data infrastructure—will be best prepared to fuel new insights and power discovery.”
He also talks about who the winners will be, “Those who are leading the data race are the ones who are not only leveraging every scrap of their collected data for differentiated AI outcomes, but those who have an infrastructure and process in place for effectively doing so—managing, organizing, indexing, and cataloging every piece of it. They’ll produce more, faster, and better results than their competitors. In 2025, we’ll start to see who leaps ahead in this new ‘data and algorithm arms race.”
Members of the Active Archive Alliance also had comments related to the growth of archive data to support 2025 workflows. Rich Godomski, Head of Tape Evangelism with FujiFilm NA Corp., Data Storage Solutions, say that, “Sustainable active archive solutions with intelligent data management capabilities can leverage ultra energy efficient and extremely cost-effective tiers of storage such as S3 compatible object-based tape libraries. This will be needed to offset the voracious energy consumption of truly cutting-edge and breakthrough AI applications as the AI age evolves in 2025 and beyond.”
Paul Luppino, Director of Global Digital Solutions at Iron Mountain, said that, “Artificial intelligence (AI) has the potential to revolutionize data storage and active archives by enhancing efficiency and accessibility. As data volumes soar, we can optimize storage management by predicting usage patterns and minimizing costs, potentially making decisions about how and where to store data at the point of creation.
In the realm of active archives, AI can analyze and prioritize data, ensuring frequently accessed information is readily available while less critical data is stored cost-effectively. Automated classification, tagging, and indexing could simplify the search process, allowing for intelligent data handling.”
Mark Pastor with Platform Product Management at Western Digital said that, “…disaggregated storage…has been proven to deliver the performance and capacity required to meet the requirements of demanding GPU-related workloads which are at the heart of AI and machine learning processes. Disaggregating storage from the server accomplishes two key things: (1) it enables storage to be shared across multiple servers offering greater flexibility and utilization of storage resources, and (2) demonstrations show that disaggregated storage delivers the performance needed to keep GPU processing fully saturated.
Over time these external storage architectures will become standard with HDD for active archives and with flash for performance workloads and will ultimately migrate to fabric as opposed to SAS given the convenience and distance benefits of fabrics.”
Jason Lohrey, CEO of Arcitecta also emphasized the value of fabric shared storage, saying that, “Businesses can maximize their existing investments and avoid vendor lock-in by leveraging a data fabric—an architecture that unifies cloud, disk, tape, and flash storage into a single, logical namespace. This trend towards virtualization allows for a more flexible approach to data management, enabling businesses to mix and match technologies to meet specific needs.”
Ted Oade, Director of Product Marketing at Spectra Logic also talks about how archive storage practices can help create more sustainable AI workloads and create competitive advantages, “Modern tape storage is not only highly durable but also incredibly energy-efficient, particularly when compared to disk storage. By offloading cold data to tape in an active archive, data centers can free up energy for AI workloads, maximizing efficiency. As energy becomes a factor potentially limiting the growth of AI, businesses that embrace sustainable practices will gain a competitive edge in 2025 and beyond”
2025 promises increased demand for storage devices, systems and software to support the growth of AI data processing. AI will increasingly be used to make digital storage more efficient and safer. Digital storage and memory architectures may play an important role in more sustainable AI data centers.
