Powering the Machine
Tales of a Time Travelling Computer Architect
Disclaimer: Opinions shared in this, and all my posts are mine, and mine alone. They do not reflect the views of my employer(s) and are not investment advice.
In a recent post (well not so recent anymore, sorry for the long break), I presented my case for why the story of ENIAC is relevant in the context of today’s AI datacenter buildout. Check it out if you already haven’t.
I left the post with two questions I wanted to explore:
Which compute infrastructure can last the longest before we need a clean reset?
How effectively can we map future workloads onto the infrastructure we are building today?
At first glance, these questions seem to be about chips, algorithms, and software. But history suggests that the answer often lies elsewhere. Every major shift in computing eventually collides with the physical realities of the world around it. How is it powered? How is it cooled? Where is it housed? What infrastructure must exist before the machine can deliver any useful work?
That is why I want to begin this exploration not with computation itself, but with the infrastructure that enables it.
In the 1940s, the builders of ENIAC were not just inventing a new computer. They were discovering what it meant to power, cool, and operate a machine unlike anything that had existed before. Today, as AI companies race to build ever larger datacenters, similar questions are once again coming into focus. The answers, however, may be very different.
To understand why, I’d like to tell a story.
Setup complete after rewiring. Preparing for Test #23 for the evaluation of errors in axially symmetric supersonic airflow.
As Homer Spence wrote down these words in the ENIAC service log book, he felt a sense of relief, with a slight dose of anxiety. He had just fixed another loose blob of solder on the ENIAC’s Accumulator 14 panel, and was one step closer to running the ENIAC without errors.
The irony of the situation always made Homer chuckle. The fate of the world’s most complex electronic device, what many considered an “artificial brain”, relied on such craft work. But this craft work had been going on for a week now, and Homer really hoped this was it.
He held up the control box and started ENIAC while an operator monitored the panels. The machine slowly came alive. Thousands of vacuum tubes began to glow with a familiar orange hue, and the room grew noticeably warmer. Homer smiled. After weeks of troubleshooting, the glow was reassuring. It meant the machine was alive.
Then something felt wrong.
The brightness kept increasing.
The room became hotter.
Homer was sweating profusely as if he was standing in front of a raging forest fire. As his hand reached for the stop button on the control box, his entire field of view turned white.
The next time Homer opened his eyes, the bright white light was replaced by complete darkness. As he looked around, the familiar contours calmed him down. He was laying on the sofa-bed he had placed in his office for all the late nights he spent at work. Maybe he was completely exhausted and blacked out during the last test, and someone helped him to his office.
Homer checked for burn wounds and was relieved to find none. The concern was not irrational. ENIAC consumed more than 150 kilowatts during normal operation, and everyone on the project knew what that could mean. A fire the previous year had damaged one of the panels after insulation around a power wire ignited. Ever since, discussions about transformers, fuses, and power distribution had become almost as common as discussions about computation itself.
It was late, and Homer decided to head home. But first, he wanted to check that everything was fine in the ENIAC room.
He grabbed his notebook and opened the office door.
Homer expected to see the familiar rows of ENIAC panels stretching across the room.
Instead, he froze.
A modern datacenter is an engineering marvel. The space itself is nothing to write home about. White walls, bright lights, and a constant hum from equipment hidden somewhere beyond sight. Yet Homer immediately sensed that this was not a building designed for people.
The room was filled with tall rectangular cabinets arranged in perfectly ordered rows. (Each of these rectangular containers, not too dissimilar to a refrigerator, is called a server rack.) They stood shoulder to shoulder, forming long corridors that stretched farther than Homer could see. At first glance, they reminded him of ENIAC’s panels. But where ENIAC’s cabinets exposed switches, cables, and glowing vacuum tubes, these appeared sealed and anonymous.
From a distance, the fronts of the cabinets looked alive. Hundreds of tiny lights blinked continuously, each following a rhythm known only to the machine. Homer found himself staring at them. They reminded him of the warm orange glow of vacuum tubes, except these lights were colder, sharper, and far more numerous. A closer look revealed multiple rows, each holding a small rectangular box. (Each of these is called a server.) A thick bunch of cables could also be seen leaving the top of each server rack and disappearing into vents in the ceiling.
Homer Spence had seen nothing like this before. The ENIAC room he expected to find on the other side of the door had vanished, replaced by what appeared to be an endless maze of cabinets, cables, and blinking lights. For a brief moment, he wondered whether he was still dreaming. As he stood frozen in the middle of the aisle, a voice interrupted his thoughts.
“Hi there. Can I help you?”
Homer turned to find a bespectacled man wearing a black leather jacket. Still trying to make sense of his surroundings, Homer skipped the pleasantries.
“Where am I?”
The man, now smiling like a proud father, responded.
“You’re standing in one of the world’s largest AI datacenters.”
Homer looked around once more.
“This entire building?”
The man laughed.
“No. This is just one hall.”
For the first time that night, Homer felt genuinely unsettled.
Back at the Moore School, ENIAC occupied a single room. Every panel, every accumulator, every cable was within walking distance. Here, the machine seemed to extend far beyond the limits of his vision.
The man continued.
“This facility has a power capacity of five gigawatts. When fully populated, it will contain more than five hundred thousand Rubin Ultra processors. Right now, we’re using it to train one of the largest artificial intelligence models ever built.”
Homer barely registered anything after “five gigawatts”. The number echoed in his head.
When ENIAC was being designed, supplying 150 kilowatts of power had proven to be a formidable challenge.
“How is this facility powered?” Homer asked.
“Powering this facility is no mean feat. We started with utility power from the grid - the same thing that’s used in homes around the world. But the local grid simply cannot support our capacity yet. So we had to improvise and build our own power plants right next to this datacenter. They are mostly turbines powered by natural gas, which is often how the central grid is powered anyway. But we plan to move to renewable sources like solar and wind, and are also exploring nuclear power as an option.”
The detailed answer surprised him. When ENIAC was being designed, power infrastructure was largely a means to an end. The objective was to build the computer. Everything else followed from that. Only later did the team discover how difficult it would be to supply 150 kilowatts reliably to a machine filled with nearly eighteen thousand vacuum tubes.
The challenge became apparent when the time came to procure the transformers. Different parts of ENIAC operated at different voltage levels, ultimately requiring twenty-eight separate transformer units. Securing them proved far more difficult than anyone anticipated. General Electric could not deliver one soon due to other military commitments, so they paid three times as much to have Maguire Industries expedite the process. Maguire only shipped 4 out of the 28 units promised, and the rushed timelines also meant that certain specifications about the insulation were ignored. Eventually, the order was fulfilled by J. J. Nothelfer Winding Laboratories and the testing could only begin after a long wait.
Even then, the challenges were not over. The reliability of ENIAC depended heavily on the quality of the incoming power. Voltage fluctuations from the utility grid affected both the accuracy and lifetime of the vacuum tubes, forcing the engineers to develop custom equipment to stabilize the supply. Years later, they would go even further, installing a dynamo coupled to a massive flywheel to smooth out power disturbances. Yet even this system remained connected to the city’s electrical grid. The objective was never to generate power independently. It was simply to make utility power good enough for computation.
Homer realized that the ENIAC team had treated power as a dependency of the computer. The engineers of the future (where he was now convinced he was) appeared to be doing the opposite.
As the conversation continued, Homer became increasingly bothered by something. Standing in the ENIAC room for extended periods was an uncomfortable ordeal. ENIAC’s vacuum tubes, not too dissimilar to incandescent light bulbs, produced enormous amounts of heat and quickly transformed the room into something resembling a tropical summer afternoon. Yet he was standing in a facility that consumed over 30,000 times more power, and the air felt no different than an office building.
Where was all the heat going?
“How do you keep this place cool?” Homer finally asked.
The man smiled and led him down one of the narrow aisles between the server racks. At first glance, the racks appeared quiet and uneventful. But as Homer looked more closely, he noticed thick pipes running overhead. Some disappeared directly into the racks, while others connected to even larger pipes that stretched across the building.
The man pointed toward them.
“Air is simply not efficient enough at these power levels,” he explained. “Traditional datacenters for cloud storage and web hosting typically operated at power densities of five to ten kilowatts per rack, where air cooling was sufficient. These racks contain many high-performance processors operating together and can exceed one hundred kilowatts per rack. At that point, liquid cooling becomes unavoidable.”
He paused before continuing.
“But that’s only part of the story.”
Homer listened carefully.
“Building this cooling system was enormously expensive. The pipes, pumps, heat exchangers, cooling towers, and control systems cost far more than the cooling equipment found in traditional datacenters. But once the infrastructure is in place, liquid cooling removes heat much more efficiently. Over the lifetime of the facility, the operating costs can be significantly lower.”
Homer was unfamiliar with this way of thinking. During the development of ENIAC, cooling was never approached this way. The machine was cooled by moving air through the room because it worked and because it was inexpensive. If the room became too warm, you opened a vent, added another fan, or simply endured the discomfort. Most of the cost was incurred only when the machine was operating - which was not very common due to various other limitations.
Here, the philosophy seemed reversed. The engineers had spent enormous sums of money even before a single calculation was performed. This facility had been designed around an expectation that workloads would run continuously. This would justify the initial costs.
For several minutes, neither man spoke. Homer couldn’t help but be impressed by everything he heard. He had spent many nights with the ENIAC team worrying about the challenges of power delivery, cooling, and maintenance. This often slowed the progress of making ENIAC better at the computations it was built for.
By planning for power and cooling years in advance, these engineers were building computers that could scale far beyond anything imaginable in ENIAC’s era. But Homer couldn’t shake a lingering thought. Every transformer ordered for ENIAC had a purpose. Every vacuum tube installed would eventually perform a calculation. The luxury of building infrastructure years ahead of demand did not exist.
He turned to the man with the question that started everything.
“Where am I?”
A few concrete takeaways
I have written this post in an experimental format - some of you may like it, others won’t. (I’m happy to hear from you in the comments either way.)
But I want to explicitly call out a few lessons that stood out for me:
1. Compute does not scale alone
Power and cooling are first-order design variables. It is becoming increasingly difficult to separate the computer architecture from the infrastructure. A specific architecture in the datacenter exists due to its supporting infrastructure, and vice-versa.
2. Infrastructure choices are really workload assumptions in disguise.
ENIAC’s infrastructure was built around a machine that spent much of its life idle, being debugged, rewired, or waiting for the next problem. Most of the cost was incurred when the machine was actually running.
Modern AI datacenters are built around the opposite assumption. Training workloads can keep hundreds of thousands of processors busy at near 100% utilization for weeks or months at a time - this has made it seem more rational to spend enormous amounts on power delivery and liquid cooling infrastructure upfront.
3. The bottleneck is shifting from execution to foresight.
ENIAC engineers had to solve engineering problems in real time. Modern datacenter builders increasingly have to solve prediction problems: how much power, cooling, and demand will exist years before workloads arrive.
That concludes Part 2 of this series. If you like what I just said, follow along for upcoming posts in this series by subscribing below:
References
My primary reference for this study is the book “ENIAC in Action” which goes into an impressive level of detail about many aspects of this computer.
Here are some other references I used for this post:
Inside the world’s most powerful AI datacenter - The Official Microsoft Blog
Mark Zuckerberg says Meta is building a 5GW AI data center | TechCrunch
(2) How AI Labs Are Solving the Power Crisis: The Onsite Gas Deep Dive
Power for AI Data Centers: Energy Demand, Grid Impacts, Challenges and Perspectives