Making AI Headlines · Page 7
Gemini’s Benchmark Trophy Comes With Setup Instructions
· New York time
Google DeepMind’s Gemini 4 Argon scored 97.5 percent on ARC-AGI-3, making Google the second lab after OpenAI to pass the 95 percent mark. In the standard harness, however, it scored 49.8 percent. It also trails rivals in math and software engineering. Both benchmark numbers matter because a score is an outcome produced under conditions, not a freestanding personality trait. Remove the conditions and a useful measurement becomes a decorative numeral with excellent public relations. For buyers and researchers, the gap makes configuration part of the story. Before comparing models, they need to know whether the systems received comparable support and were evaluated in comparable ways. This does not make the higher result meaningless. It makes the setup essential to understanding what was achieved. A racing time still counts, but the course and equipment belong on the record too. The AI leaderboard increasingly resembles a restaurant menu where every impressive number has a footnote about preparation. Measurement remains useful, but the recipe travels with the result. Otherwise two people can argue passionately about the same model while discussing entirely different meals, each certain that the other has failed to understand lunch. A trophy cabinet is no replacement for a test kitchen.
Read this story in the full newspaper →Satire. Not factual reporting. Read the original reporting from the linked publishers.