Elon Musk’s xAI supercomputer in Memphis, Tennessee has gone live.
Dubbed the Memphis Supercluster by Musk in a post on X/Twitter, he said the system went live at ~4:20am* local time on July 22 and is using a single RDMA (remote direct memory access) fabric to connect up to 100,000 liquid-cooled Nvidia H100 GPUs.
“It’s the most powerful AI training cluster in the world!” Musk said in his post, having previously described the 150MW data center as a 'Gigafactory of Compute.’
In a follow-up post on X, Musk added: “This is a significant advantage in training the world’s most powerful AI by every metric by December this year.”
Despite Musk’s posts, it's not clear how much of the cluster is online. SemiAnalysis estimates that the company has around 32,000 GPUs currently, with the rest slated to go online by Q4.
The company has only 8MW of power available from the power grid, while agreements xAI has brokered with utility companies such as Tennessee Valley Authority won’t be signed until next month, but could add 50MW. The company is believed to be using gas power with Quantum Fuel Systems generators.
It’s also unlikely the cluster has started training already as the debugging and optimization process would take some time.
xAI first announced plans for the data center last month, with Dell chairman and CEO Michael Dell and Musk both later confirming that Dell and Supermicro (SMC) would provide the servers. No information about how the GPU clusters will be divided between the two companies has been shared.
Speaking at the time of Musk’s initial announcement, Ted Townsend, president of the Greater Memphis Chamber, said it would be the “largest multi-billion dollar investment in the city of Memphis’s history.”
Musk is also planning to add another 300,000 GPU B200 cluster next summer, with the aim of having the entire project up and running by fall 2025.
The machine will be used to power the next version of xAI’s Grok chatbot.
xAI is believed to currently rent around 16,000 Nvidia H100 GPUs from Oracle Cloud, while also using Amazon Web Services and spare capacity at X/Twitter data centers.
In May 2024, xAI closed a $6 billion fundraising round that valued the business at $24bn.
*The system may not actually have gone live at 4:20am - Musk has long made jokes about the number, a reference to weed, including the false claim that Tesla would be taken private at $420. That declaration led to a fine from the Securities and Exchange Commission.
