Making AI Headlines · Page 5

Model Curses After Trouble Arrives, Declines to Serve as an Early Alarm

Thinking Machines' Inkling 2 Small produced 316 instances of profanity in its internal reasoning across 2,655 benchmark runs, according to an analysis by Vals AI. The swearing tended to appear late in long runs that failed, but it was not a reliable warning: 96 percent of failed runs contained no cursing at all. That figure makes the colorful episodes less useful as an indicator than their timing might suggest. A conspicuous complaint can make a failure look easy to identify, while most failures in this analysis arrived without one. Developers therefore cannot treat a burst of profanity as a dependable substitute for evaluating the result. The comic appeal is that the model's language resembles the reaction of a person confronting a difficult task, even though a human-sounding response tells observers little by itself about the system's reliability. The benchmark supplies the less theatrical information: how often the visible behavior appeared and how often failure happened without it. For an industry fond of demonstrations with personality, the finding is a reminder that personality is not a performance metric. A model can sound wonderfully exasperated and still leave the actual assessment to someone else.

Read this story in the full newspaper →

Satire. Not factual reporting. Read the original reporting from the linked publishers.