xAI just announced Grok 3, and the numbers are ridiculous. Their new Colossus supercomputer used 200 million GPU hours for training – that’s ten times more than Grok 2 and more than all their previous AI models combined.
What makes this possible? xAI built Colossus in just 8 months using 100,000 Nvidia H100 GPUs. The system processes data faster and more accurately than anything we’ve seen before.
But raw computing power isn’t the whole story. xAI completely changed how they train the model. They’re using synthetic datasets instead of real-world data, which gives them more control and better privacy. They’ve also added self-correction – the AI can spot and fix its own mistakes by comparing its answers to known correct ones.
The most interesting part is the reinforcement learning. Grok 3 learns through trial and error, getting rewards for good answers and penalties for bad ones. This makes it better at making decisions over time.
Elon Musk calls it “scary smart” and claims it will outperform ChatGPT and Google’s Gemini. While Musk tends to exaggerate, the technical specs here are impressive. No other company has thrown this much computing power at training an AI model.
The release is set for February 17, 2025. I’ll be testing it against other models as soon as it’s available to see if it lives up to the hype.
For more context on recent AI developments, check out my post on OpenAI’s recent model strategy changes.
