In April, AWS gave us a peek behind the curtain of their data center engineering. To handle the thermal output of the latest AI clusters, they unveiled Direct-to-Chip Liquid Cooling.

This is more than just a hardware curiosity. From a performance standpoint, liquid cooling allows for 3x higher compute density per rack. It ensures that custom silicon like Trainium and Inferentia can maintain peak clock speeds for long-running training jobs without the thermal throttling that often plagues air-cooled environments.

We also saw the launch of EC2 UltraClusters 2.0, which supports non-blocking petabit-scale networking via Elastic Fabric Adapter (EFA). If you’re distributed-training models with trillions of parameters, the communication bottleneck between nodes has been virtually eliminated.

April reminded us that “the cloud” is ultimately limited by the laws of physics, and AWS is literally re-engineering the plumbing of the internet to keep up with AI.