Making AI Headlines · Page 9
Stanford Finds AI Capability Claims Age Faster Than Their Conference Slides
· New York time
Stanford researchers tested 50 widely cited claims about large language models against 20 current models and found that 44 percent had expired while another 34 percent had weakened significantly. The study put the average lifespan of the claims at 15.7 months. Claims about what models cannot do expired fastest, according to the research. That makes a static description of a fast-changing set of systems a risky thing to keep repeating indefinitely. A limitation observed in one generation may not hold in the next. The study does not say every finding becomes useless overnight. It does suggest that research claims need to be checked against the models available when they are applied. In a field fond of announcing permanent turning points, even the footnotes may need a calendar reminder.
Read this story in the full newspaper →Satire. Not factual reporting. Read the original reporting from the linked publishers.