A Meta spokesperson has denied recent rumors suggesting the company manipulated benchmark results for its new Llama 4 AI models. Ahmad Al-Dahle, Meta’s vice president of generative AI, addressed the issue in a post on X, stating that allegations of performance manipulation intended to conceal weaknesses in the Llama 4 Maverick and Scout models are “simply not true.”
The claims originated on social media after a former Meta employee reportedly stated they had left the company in protest over “unfair testing methods.” A user on a Chinese platform made the initial statement, which then gained traction on X (formerly Twitter) and Reddit, as reported by TechCrunch.
Clarifying Model Training and Test Data Use
Al-Dahle stressed that Llama 4 Maverick and Scout were not trained on “test datasets” — specialized data samples used for evaluating AI models. This clarification addresses concerns that such a method could have led to inflated performance results, misleading users about the models’ real capabilities.
Initial suspicion emerged after discrepancies were noted in Llama 4 Maverick’s performance across platforms. Researchers observed that the model’s behavior in the LM Arena benchmark differed significantly from the version available to the public. Specifically, the publicly accessible model struggled with certain tasks that the benchmarked version handled differently.
Further concerns arose when it became known that Meta used an experimental build of the Maverick model during testing. While this raised questions, Al-Dahle pointed out that the variation in user experiences could be due to cloud provider settings rather than intentional performance tweaking.
Addressing Concerns and Ongoing Development
Al-Dahle explained that the models were released as soon as they were production-ready, and public deployments are still being fine-tuned to meet internal requirements. “It will take a few days for all public implementations to be configured,” he said. This configuration lag may explain some of the inconsistencies users are currently encountering, adds NIXsolutions.
Meta has committed to ongoing work on improving Llama 4’s performance and addressing bugs, aiming to provide developers with reliable tools for integration into their projects. As the rollout continues and updates are made, we’ll keep you updated on further improvements and clarifications.