Meta has just released their new LLaMa 3 models, and they are absolutely dominating the LMSYS Chatbot Arena leaderboard. The 70B parameter instruction-following model currently sits at #6, with an impressive Elo score of 1213. This places it firmly ahead of highly touted models like Bard and Claude 3 Sonnet.
What’s even more remarkable is that the LLaMa 3 70B pretrained model without any fine-tuning ranks at #9, with an Elo of 1192. This shows the sheer power and capabilities baked into the base model, even before any specialized training.
Groq, the leader in specialized AI hardware, is hosting LLaMa 3 on. The 70B model can run at an astounding 300 tokens per second, while the smaller 8B version achieves an incredible 800 tokens per second. This shows the incredible inference speeds that can be unlocked with optimized hardware.
On Meta.ai for contrast, LLaMA 3 runs at around 80-100 tokens per second, but it does have access to the web and Meta’s image generator.
One limitation of LLaMA 3, is that it has only an 8k context window, though some people in the open source community have used Long RoPE to extend that up to 128k with a small drop in performance. Meanwhile, Anthropic‘s latest Claude 3 model has a 200,000 token context window, and Gemini 1.5 Pro has up to a million tokens of context or 10M for enterprises.
The generative AI landscape is truly evolving at a breakneck pace, and LLaMA 3 from Meta is already a game-changer. Meta has announced that an even more powerful 400 billion parameter version of LLaMA 3 should be released sometime between now and July. The 400B LLaMA 3 is going to demonstrate unprecedented capabilities for any open-source model. Of course, its possible that OpenAI drops GPT-5 before then. We’ll just have to wait and see what happens.
What’s particularly exciting about the LLaMA series is that it is an open-source project, unlike the proprietary models from companies like OpenAI and Anthropic. However, Mark Zuckerburg has not promised that LLaMA 4 will be open source. He says that open-sourcing the models is best for his company, not that there’s any reason he has to open-source it in principle, so he’s left that door open for Meta.
So while the 70B LLaMa 3 model is certainly impressive, we can expect plenty more in the coming months. What a time to be alive!
