METR: AI models still lag at judgment
An update to AI's time horizon graph
ACN Wirevia MIT Technology Review
Written with AI assistance, reviewed by the ACN desk.
A new report from AI evaluation nonprofit METR finds that some AI capabilities are continuing to improve along an exponential trajectory, but models are not as good at tasks that require judgment as they are at coding, according to MIT Technology Review. The update to METR’s time horizon plot adds longer and more open-ended tasks, including designing new AI evaluations and scoring philosophical arguments. None of the AI-designed evaluations was good enough for METR employees to consider useful for their research, the publication reported, though newer models such as Opus 5.5 reached what METR calls “borderline above the bar” performance.
Spot an error? Email corrections@aicodingnews.si. Fixes are logged on the corrections page.
Sponsor
An update to AI's time horizon graph
The Riemann proof passes its formal check; understanding it comes next
Levent Alpöge · proof posted in Lean · not yet verified
Told House committees of "geopolitical risk" · CNBC
Sponsor
Sponsor
Sponsor
Sponsor