News
November 22, 2026 · METR

METR's new capability report finds AI agents lag experts on judgment tasks

METR's November update says model time horizons double roughly every 3.6 months, but agents still show weaker judgment than human experts on open-ended tasks.

METR, a nonprofit that evaluates AI models, published an update to its capability measurements on November 20 covering public models up to Opus 5.5 and GPT-6 Astra. The report extends its well-known time horizon chart with harder and more open-ended tasks. MIT Technology Review, which covered the release, describes it as a major update to one of the most misread graphs in AI.

What's new

  • Faster doubling: on its combined task suites, METR says model time horizons are doubling roughly every 3.6 months. MIT Technology Review notes the original plot showed a doubling about every four months.
  • Coding and optimisation: according to METR, models usually match or beat skilled humans on these tasks, and its recruited human testers typically could not outperform frontier models.
  • Weaker judgment: on tasks that require choices without clear feedback, METR says agents appear to have worse judgment than human experts. It also found that model judges overrated outputs from similarly strong or stronger models.
  • New task types: MIT Technology Review reports that METR asked models to design new AI evaluations and to score philosophical arguments. None of the model-designed evaluations was judged useful for METR's research, though some newer models reached what METR calls borderline above the bar.
  • Measurement worries: METR says it is concerned that easy-to-run evaluations may overstate real-world ability because similar environments are now common in training.

Why it matters

Analysis: for people choosing AI tools, the report supports a familiar pattern. Agents are strong at well-defined coding and optimisation work with quick feedback, and much less reliable where success is subjective or slow to judge. That argues for keeping a human reviewer on decisions such as research direction, product strategy or quality assessment, and for being cautious when one model is asked to grade another model's work. METR itself stresses large uncertainty, so the doubling figure is best read as a trend estimate, not a forecast.

Source: METR