Google’s slashing prices on Gemini 1.5 Pro starting October 1st, 2024. Here’s the breakdown: Input costs: – Up to 128K tokens: $1.25 per million (down from $3.50) – Over 128K tokens: $5.00 per million (down from $7.00) Output costs: – Up to 128K tokens: $2.50 per million (down from $10.50) – Over 128K tokens: $10.00 per million (down from $21.00) These are massive cuts, especially on the output side. But how does Gemini 1.5 Pro stack up performance-wise? Compared to Gemini 1.5 Flash, it’s better across the board: – General knowledge (MMLU-Pro): 75.8% vs 67.3% – Coding (Natural2Code): 85.4% vs 79.8% – Math: 86.5% vs 77.9% – Advanced math: 52.0% vs 47.2% – Reasoning: 56.0% vs 51.0% – Multilingual: 75.1% vs 73.9% – Long context: 73.7% vs 71.9% – Image understanding: 66.5% vs 62.3% – Visual chat: 53.9% vs 48.9% – Math with visuals: 68.1% vs 65.8% – Audio: 10.0% vs 9.6% – Video: 78.6% vs 76.1% – Safety: 98.9% vs 97.0% Gemini 1.5 Pro isn’t top dog on the MMLU-Pro leaderboard. It scores 0.7276, behind o1-preview (0.9104), GPT-4o (0.7942), and Grok-2 (0.7927). But with these price cuts, it’s punching above its weight class. From July to September 2024, Gemini models got faster and snappier too. Output speed went up, latency went down. My take? Gemini 1.5 Pro isn’t the best for writing or complex stuff. But for basic API tasks? It’s solid. These price cuts make it way more competitive, especially for high-volume use. If you’re doing bulk API work, Gemini 1.5 Pro is worth a serious look. Just don’t expect it to replace your writers or tackle super complex problems. Want to dive deeper into AI model comparisons? Check out my post on Claude vs GPT: Which AI to Use and When. It breaks down the strengths and weaknesses of two other major AI players.
Menu
