Businesses pouring resources into artificial intelligence (AI) need a backbone of scalable infrastructure to ensure their investment delivers success.
IDC predicts spending on AI will grow 31.9% per year between 2025 and 2029 to $1.3 trillion, with agentic AI the key driver.
This scale, and the rapid evolution of the underlying technology, may seem overwhelming for technology leaders. But as IDC notes: “Informed leadership will be critical to success during these next several years.”
A clear AI vision and a wealth of data to draw on are essential. But tech leaders also need to understand how to build a future-proof infrastructure.
For example, the rapid development of AI has become synonymous with accelerated compute in the form of cutting edge but expensive and power-hungry graphical processing units (GPUs), as well as field programmable gate arrays and other specialist processors.
Deploying GPUs and other accelerators in house is not impossible, but executives need to consider whether this allows them to scale up quickly or adopt new architectures as needed. This could be as their projects move from development to deployment, or as they adopt new AI models or exploit larger datasets.
And while GPUs are the workhorses of model training, other architectures, including traditional central processing units (CPUs), may come into play for other tasks in the AI workflow.
Compute is not the only element leaders need to consider. Having GPUs sitting idle waiting for data is a waste of investment and power. This means optimised networking is essential. Model training facilities typically run interconnects at 400Gbps, and 800Gpbs is on the horizon. Inference doesn’t necessarily require such high-speed interconnect. But as inference accounts for more of the AI workflow, latency between the data centre and customers becomes critical. Remote data centres make sense for training models. But when it comes to delivering answers to customers in real time, the edge will become increasingly important.
And of course, this data needs to be stored. Blisteringly fast solid-state drives are a key part of the model training process, keeping GPUs fully supplied with data. But they may be just one of multiple tiers – from flash to large capacity hard disk drives –to keep data lakes topped up with cold data.
All of this generates further decisions, from the AI models and workflow management tools enterprises work with, and the optimal file formats for storage to the cooling and power infrastructure required, not to mention the cyber and physical security needed to protect all this.
It’s inevitable the cloud will play at least some role in most enterprises’ AI infrastructure. Service providers are expected to account for 80% of the enormous infrastructure spend IDC has predicted. The question for CIOs is how to best leverage what cloud operators can provide.
So, every tech leader needs to think beyond their short-term AI goals. They need to consider the scale they will need to operate at in the years to come and choose underlying infrastructure, tooling and providers accordingly. Their future will depend on it.
Find out how to transform your infrastructure for the AI era with Amazon Web Services (AWS).
