METR

Where the book uses it

• Chapter 3 — The Missing Researcher

  The dated promise of an AI that does its own research, what the laboratories have published, and what the public record can and cannot yet check.

Where the record uses it

• Scorecard indicator METR 50%-success horizon
• Scorecard indicator METR 80%-success horizon
• Scorecard indicator Post-2024 METR horizon doubling time
• Dated test Chapter 3, “attempt” and “leave with it”, due 31 August 2027
• Person Cotra, Ajeya

Sources that use it

• Anthropic, “An update on independent evaluators,” 15 October 2026 (“We also signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators”; “We’ve onboarded six METR employees with company hardware, office space, and access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to”; “METR will publish its analysis of our risk levels, incidents, or practices. Anthropic can’t veto publication, but we can require them to redact information if it falls into narrow, predefined categories like the ones above.”; the post does not mention the Federal Trade Commission; “We’re also working with SecureBio and Accenture as evaluators”; of an independent evaluator, “they shouldn’t hold equity in the company they’re evaluating, or be financially or operationally dependent on it”). In the Sources
• METR, “Our approach to working with AI companies,” metr.org, 14 October 2026 (its terms for evaluations, reviews and investigations with AI companies, and its funding: “to date we have chosen not to accept payment or donations from frontier AI companies”; its risk-assessment page, read 30 October 2026: “METR does not accept compensation for this work”) In the Sources
• OpenAI, “The Hugging Face incident and the road ahead,” 26 August 2026 (technical report; independent METR/Redwood Research report; “warning shot”; stricter sandboxes, weight-access controls, “significantly more compute” into chain-of-thought monitoring; “pacing capabilities when needed”; “If our currently deployed CoT monitoring system was running at the time of the incident, it would have caught the initial relevant activity and paged our security team more than a day before models breached Hugging Face systems”). In the Sources
• Ajeya Cotra (METR; co-author of the METR–Redwood investigation of the OpenAI/Hugging Face incident; the page’s transcript is the host’s, which he describes as reworded, and the quoted words were read against the recording: the warning-shot sentence exact at 2:16:12, prefaced as what feels especially concerning to her; 30 to 40 percent exact on the recording, which says 70,000 messages, the report supplying over and files; the “four hours” on the recording: “within four hours of the message board being established. So within four hours, these agents had a totally workable universal cheat”, and “one hour into this message board being established” for the proposal; on the episode: “this might be the clearest warning shot we ever get for loss of control” (2:16:12)) on the Dwarkesh Podcast, 1 September 2026. In the Sources
• Joel Becker (METR), “Reconciling Impressive AI Benchmark Performance, Limited Developer Productivity Impacts,” Stanford Digital Economy Lab seminar, published 17 March 2026 (“the title’s a bit over-promising with ‘limited developer productivity impact’ … probably developer productivity impacts look pretty impressive today”, “impact” at 1:16 read singular on the 13 September pair and plural on the 24 September pair, so whether he misquoted his own plural title is unsettled, “impact” kept as printed under the set’s standard; “maybe accidentally” at 45:16; “I think it’s something like 80 to 90% of these people are continuing to use Cursor.” at 45:30, the caption track prints “custom” there where large-v3 has no word, the room audio gating to silence, and renders Cursor correctly fifteen times elsewhere, the word “Cursor” unsettled and kept as printed under the set’s standard; the caption’s repeated these at 45:30 rendered once; “we traditionally measure AIs under conditions that are optimized for them to be most performant, and then we deploy them under conditions optimized for humans”; read against the caption track and two decodes, 13 September 2026; he calls it “an opinionated perspective” and prefaces it “I think”, offering it conditional on “we haven’t messed something up here” at 49:26–49:32; the caption’s let’s let’s and on on at 49:28–49:30 sit before that quotation, outside it, read on the word-timed track on 24 September 2026; the um at 49:35, between I think and we in the sentence book 1 prints, dropped from that quotation, one um on the loudness profile where large-v3 hears two, the 24 September pair held in the set’s Quote audio checks folder of that day with their md5 in its manifest; here at the end of we haven’t messed something up here on large-v3 where medium.en reads yet, kept as printed and unsettled; on the two conditions, from his I think: “I think we traditionally measure AIs under conditions that are optimized for them to be most performant, and then we deploy them under conditions optimized for humans” (49:35.0–49:41.3)). In the Sources

And 17 more: search the Sources.

Related terms

• Mythos
• Time horizon
• Adoption clock
• Agent
• AGI
• Alignment
• ARC-AGI-3
• Astra
• Automated AI researcher
• Benchmark
• Brake, the
• Capability clock