In December, OpenAI introduced its o3 reasoning AI model, showcasing its ARC-AGI benchmark results—one of the most challenging tests for AI capabilities. However, the revised results now present a different picture: the model is significantly more expensive to operate than initially thought.
Soaring Costs of Running o3
Last week, the Arc Prize Foundation, which oversees ARC-AGI, updated its estimated compute costs for OpenAI o3. Initially, the most powerful configuration, o3 high, was believed to cost around $3,000 per ARC-AGI task. Now, revised calculations suggest the cost could be as high as $30,000 per task. This highlights the substantial expenses associated with running advanced AI models, particularly in their early stages.
OpenAI has not yet disclosed pricing details for o3, nor has it made the model publicly available. However, the Arc Prize Foundation pointed to OpenAI’s o1-pro figures as a possible benchmark for estimating o3’s real costs.
Compute Demand and Performance Challenges
The Arc Prize Foundation noted that o3 high used 172 times more compute power in ARC-AGI tests compared to o3 low, the weakest model in the lineup. Additionally, OpenAI’s most sophisticated models are known for their high operational costs—previous reports indicated that specialized AI agents could cost up to $20,000 per month to run.
Beyond cost concerns, the performance of o3 high also raises questions, notes NIXSOLUTIONS. It reportedly required 1,024 attempts to solve each ARC-AGI problem before achieving a better result. This suggests that even the most advanced AI systems remain prone to inefficiencies.
As OpenAI continues refining its models, the true cost-effectiveness of o3 remains uncertain. Yet, we’ll keep you updated as more details emerge.