You may or may not have heard this phrase before: “the backbone of AI.” What does it mean? What is the “backbone” of an AI system?
I found this from Copilot:
“In modern artificial intelligence, the backbone typically refers to the core neural network architecture responsible for extracting meaningful features from raw input data. This concept is especially common in computer vision, where a backbone is a pre-trained Convolutional Neural Network (CNN) used as the feature extractor for more complex tasks like object detection, segmentation, or image captioning.”
That’s one way to describe it, and I’ve often said that the convolutional neural network is central to AI, as it allows the entity to “see” what’s around it, to start to process environmental data, which makes deployed AI an order of magnitude stronger. I don’t know if I’d call it the backbone, though.
Here’s something from a writer self-identified as “Pete the Momentum” identifying Coreweave systems as a “backbone” for AI operators, though I can’t tell if this is a puff piece or something else.
I found this to be a little more helpful: a Data Center Knowledge piece from Ivo Ivanov of Industry Perspectives, who cited our own MIT publication, writing:
“According to a 2024 MIT Technology Review survey, a staggering 95% of businesses are already utilizing AI in some way, and more than half are aiming for full-scale integration in the next two years. The momentum behind AI is nothing short of remarkable, but as with any emerging technology, there are peaks and troughs before a state of blissful equilibrium is reached.”
Boston Discussion at Imagination in Action
Then there’s this from our April event at MIT from Imagination in Action, an organization I am affiliated with. My colleague Dave Blundin interviewed Jeremy Kepner, head of the MIT Lincoln Lab supercomputing center, Libby Wayman of Breakthrough Energy Ventures, and Chase Lockmiller of Crusoe, who zoomed in via teleconference.
The Energy Equation
“I think AI is driving a multiple order of magnitude change in the amount of power requirement to run global computing infrastructure,” Lockmiller said, elaborating on data center plans around the country, in places like Northern Virginia and Denver, Colorado. “It creates this huge, huge opportunity to create an abundance of not just intelligence, but also of energy.”
Keep in mind that Lockmiller’s company is involved, as he mentioned, in the Abilene, Texas plant that is a part of the Operation Stargate project with a staggering $50 billion allocation, that was announced in the White House.
Wayman, calling the AI arms race a “U.S. and China phenomenon,” spoke to some of the challenges involved in getting us to where we want to be.
“In the United States, about 4.4% of U.S. electricity consumption went to data centers by the end of 2024, so zooming out from the specific data points that Chase shared, the trajectory that we’re on is that that data centers could consume about 20% of U.S. electricity, which is basically expanding the electric grid and electric consumption in the U.S. by 20%, by the end of this decade,” she said. “This constraint that we’re feeling right now is really just from a couple percent of increase, and to take that up to 20% increase is going to put a massive strain on the system. What that’s looking like in individual local economies is: you’re starting to see some modeling that indicates that electricity prices will go up, emissions will go up.”
MIT Contributions
“We observed this energy importance in computing, I think, around 2002,” Kepner said, “and it was fairly obvious to us here at MIT, that energy is going to be very important.”
He talked about standing up a “next-gen” data center in a former industrial town in Holyoke, Massachusetts, next to a hydroelectric plant.
“We wanted it to be, not just great for computing, but we were really thinking long term,” he continued. “We knew no one was going to buy us another one of these, and we could tell that the technology was going to be changing, this GPU technology was going to be changing, so we designed an extremely flexible data center, so that we could accommodate essentially any generation of technology.”
Language, and Progress
Kepner had two more interesting things to say: the first was around the industry of supercomputing, where most people assume that progress has been meteoric.
“People often ask me, like, hasn’t this changed so much over the years?” he said. “And I’ve been in supercomputing for a very long time, and as far as we’re concerned, nothing has changed. These computers do the exact same mathematical operation, which is called matrix multiply, that they’ve done for 40 years. It is the only operation we know how to accelerate. We don’t know how to accelerate anything else. And in fact, if AI could not take advantage of that operation, we are not talking about it. We are talking about something else. And so that has been the fundamental driver of this industry for four decades.
Interesting.
The other point is around where investors can put their backing and effort, as Kepner urged us to “bet on math.”
“Math is as much of a human language as English or French or anything, and the AIs understand it far better than English,” he said.
Making Things Go
Toward the end of the talk, Lockmiller gave us a pretty granular view of how management happens around these cutting-edge projects, as speed becomes a top priority.
“Libby mentioned the speed of things,” he said. “You know, when we were building out the initial phases for Abilene, I committed to us being able to do it in a year. And we were going through this process of trying to order a lot of different things from vendors.”
He cited something called a Power Distribution Center, which uses medium voltage power, doling it out to low voltage transformers.
“The lead time of that was over 100 weeks,” Lockmiller said, “and this was from multiple different vendors. And I said, ‘Okay, well, that doesn’t work for me, because I said I was going to get this done in a year.’ So we actually had a facility where we’re manufacturing some components. And I turned to my team and I said, ‘how quickly can we make this?’ and we eventually got that done in 22 weeks.”
That’s a little bit about what’s going on at the top of some of these big projects. Stay tuned for more.
