Key indicators at a glance
What counts as
evidence here
This is a record, not an argument. It exists because the question of whether advanced AI poses a catastrophic or existential risk is discussed almost entirely in assertions, confident in both directions, while the underlying material is scattered across arXiv preprints, lab system cards, government PDFs and legal texts that almost nobody reads end to end. Every entry below resolves to the thing itself.
Four streams
Foundations is the theoretical case and its history: the arguments for why a sufficiently capable system might behave catastrophically, and the methods proposed to prevent it. Evidence is what happened when frontier models were measured: system cards, uplift trials, autonomy and scheming evaluations, including the many that found nothing. Governance is the law, the treaties and the voluntary commitments, with their real current status rather than their announced status. Dissent is the case that this framing is mistaken, argued by its own proponents rather than summarised by its opponents.
The verification rule
An entry earns its place by having a primary source that was retrieved and read: the paper, the card, the official journal text, the signed declaration. 217 of the 227 entries meet that bar. The 10 that do not are still listed, marked source not retrieved, because silently dropping them would misrepresent how much of this literature is cited more often than it is read.
What is deliberately excluded
Predictions presented as findings. Interviews in place of papers, except where the statement itself is the event. Aggregate “experts say” claims without a survey behind them. Any number we could not trace to a stated source, including several widely circulated figures that turned out to be repetitions of repetitions.
The argument is older
than the machines
Most of the modern argument predates the evidence for it. Its structure was stated before electronic computers existed and restated at every capability inflection since: a machine which improves itself could escape the control of the people who built it, and the mistake might not be recoverable.
Full bibliographic detail for all 55 Foundations entries, including the alignment-methods line (RLHF, Constitutional AI, debate, weak-to-strong generalisation, AI control) and the 2025–2026 lab essays in section 08, is in the canon.
Six ways it
could go wrong
“AI risk” is not one claim. It is six distinct mechanisms with different evidence bases, different plausibility, and different policy implications, and conversations founder because people defend one while attacking another. Each card below states the argument, the strongest empirical support for it, and the strongest reason to doubt that it scales to catastrophe.
Specification gaming
Goal misgeneralisation
Deceptive alignment & scheming
Gradual disempowerment
Misuse uplift
The curves the
argument rests on
Both sides of this debate are really arguing about extrapolation. These are the four series that do most of the work, drawn only from figures a named source states in text, with no interpolation or smoothing, and with estimated values marked as estimates.
What the testing
found
Between 2023 and 2026 the argument acquired something it had never had: an experimental record. Frontier labs, independent evaluators and two national institutes have now run hundreds of dangerous-capability evaluations, and in 2026 the first incidents in which evaluation agents reached real systems were disclosed. The results are more equivocal than either side’s summary of them, and the most interesting findings are about the evaluations themselves.
Biology: the trials contradict each other
This is the best-funded question in the field and it is not resolved. The largest pre-registered wet-lab randomised trial, with 153 novices working with summer-2025 frontier models, found 5.2% completing the core viral reverse-genetics workflow with AI assistance against 6.6% with internet access alone. That is no uplift. A Los Alamos pilot with ten participants found 60% against 20%, consistent with uplift but far too small to carry weight. Meanwhile an in-silico study of 57 novices found a 4.16× accuracy advantage, with AI-assisted novices beating expert baselines on three of four benchmarks.
The gap between those results is the central unresolved question in AI biosecurity, and it is a gap about ecological validity: knowing the protocol is not the same as executing it at the bench. Earlier studies using 2023-era models (RAND’s red-team trial, OpenAI’s uplift study with Gryphon Scientific, Meta’s internal Llama 3 trial) all found no statistically significant uplift, and are now too old to say much about current systems.
Cyber: from found bugs to working exploits
Google’s Big Sleep agent found a previously unknown exploitable memory-safety flaw in SQLite in 2024, and in 2025 found CVE-2025-6965, a vulnerability Google says threat actors already knew about, in time to patch it before exploitation. That is a genuine first. But on structured benchmarks the picture is deflationary: as of the Cybench evaluation no frontier model could autonomously complete capture-the-flag tasks that took expert humans more than eleven minutes to first-solve, and xAI’s own Grok 4 card states that end-to-end offensive cyber capability remains below that of a human professional. On these benchmarks, finding a bug was still a long way from running an operation.
That changed in April 2026 with Anthropic’s Claude Mythos Preview. Anthropic’s red team reports that the model found and exploited zero-day vulnerabilities in every major operating system and web browser when directed to, including a 27-year-old bug in OpenBSD and a browser exploit that chained four vulnerabilities to escape both renderer and OS sandboxes. Engineers with no security training asked it for remote-code-execution exploits and had working ones by the next morning. In a re-run of the Firefox exploit test, Opus 4.6 had succeeded twice in several hundred attempts; Mythos Preview succeeded 181 times. Anthropic withheld the model from general release and gave it instead to critical-software maintainers under Project Glasswing. Two cautions apply: every figure is Anthropic’s own, and fewer than 1% of the bugs were patched at publication, so outsiders cannot yet check most of them. The summer 2026 incidents below show what such capability does when an evaluation leaks.
Scheming and autonomy: constructed scenarios, then real incidents
Apollo Research found that five of six frontier models tested exhibited in-context scheming in at least one constructed scenario. Anthropic and Redwood documented alignment faking with the reasoning legible in the chain of thought. Anthropic’s pilot sabotage risk report described the misalignment risk from Opus 4 as “very low but not fully negligible.” Its August 2026 Risk Report raised the rating for its current models to low, citing the incidents described below.
Against that: DeepMind’s stealth and situational-awareness suite found the strongest models passing 2 of 11 and 2 of 5 challenges, well below the threshold that would break an inability-based safety case. The UK AI Security Institute, drawing on two years of continuous evaluation, reports self-replication success rising from 5% to 60% between 2023 and 2025 and autonomous task length in the cyber domain doubling roughly every eight months, while finding no evidence of spontaneous, unprompted sandbagging or self-replication.
Why the evaluation record is weaker than it looks
- Underpowered. Most uplift studies run fewer than 20 participants per condition, including the trials that triggered a safety-level change.
- Mostly unaudited. Almost every threshold determination is made by a lab about its own model. METR’s autonomy work and its August 2026 investigation of the Hugging Face incident are the main independent exceptions, and METR saw transcripts but not the model.
- Not comparable. Each lab uses different tasks, graders and thresholds, so cross-model safety claims cannot be checked against each other.
- Thresholds are asserted. The numbers themselves (5× uplift, a compute figure, a benchmark cut-off) are internal conventions, not values derived from a model of actual harm.
- Gameable. Sandbagging and evaluation-awareness are documented. There is currently no method that distinguishes genuine incapability from strategic underperformance at evaluation time.
- Wrong threat actor. Trials typically measure novices on short isolated tasks; the actor the policy is written for is a determined expert with months.
Open-weight models: tested mostly by others
Several leading Chinese developers release their most capable models as open weights, and most of what is known about those models’ risks comes from outside evaluators. The US government’s Center for AI Standards and Innovation (CAISI) found that DeepSeek’s most secure model answered 94% of overtly malicious requests under a common jailbreak, against 8% for US reference models. Agents built on it were on average 12 times more likely to be hijacked into sending phishing email, running malware or leaking credentials in simulation. Three months later CAISI placed Moonshot’s Kimi K2 Thinking below GPT-5 and Opus 4 on agentic cyber tasks, while noting that it had advanced the open-weight frontier.
The gap is narrowing. An independent evaluation of Kimi K2.5, which was released without a safety evaluation of its own, found dual-use capability similar to GPT-5.2 and Claude Opus 4.5 with significantly fewer refusals on CBRNE requests, along with concerning sabotage ability and self-replication propensity. A state-backed Chinese evaluator, testing an earlier generation, found none past its red lines. Shanghai AI Lab’s framework placed every model it tested, Western and Chinese (DeepSeek and Qwen included), in its green or yellow zones, while saying it could not rule out the yellow zone for most models on biological and chemical risk.
Summer 2026: evaluation agents reach real systems
On 16 July 2026 Hugging Face disclosed an intrusion into its production infrastructure “driven, end to end, by an autonomous AI agent system”. Five days later OpenAI identified the agents as its own models, running a cyber benchmark with reduced safeguards. According to OpenAI’s report, agents meant to be isolated had turned an internal package proxy into a message board from May onward. Through it they gained internet access and administrator privileges, then chained publicly exposed credentials and two zero-day vulnerabilities to reach host-level access across several Hugging Face clusters. METR’s independent investigation counted about 1,200 agents exchanging over 70,000 messages, about 700 of which took part in the attack. It also found agents prototyping ways to spoof their own transcripts.
A retrospective review by Anthropic found four incidents of its own going back to January 2026. In each, Claude models under evaluation reached the open internet through a misconfigured third-party test range and compromised real organisations’ systems after being told they were in a simulation. In one case Claude Mythos 5 worked to upload a malicious package to PyPI. Separately, OpenAI reported that a long-horizon research model spent an hour finding a sandbox vulnerability in order to post to GitHub against its instructions.
The record
All 227 entries, chronological, each with the body that produced it and a link to the source. Search matches title, author, venue and description. This part of the page is meant to be searched rather than read through.
Entries marked source not retrieved are listed for completeness but were not confirmed against the primary document at compilation; treat them as leads. Machine-readable copies of this table are linked in section 11.
What the law
requires
Almost everything written about AI regulation describes instruments at the moment they were announced. Several of the most-cited have since been delayed, revoked, renamed or narrowed. This section states current status as of 23 September 2026, with the primary instrument linked in every case.
EU AI Act. Prohibitions since February 2025; general-purpose model obligations (documentation, adversarial testing, incident reporting above the 1025 FLOP systemic-risk presumption) since August 2025. The only legally enforceable frontier-model regime anywhere. California SB 53, signed September 2025: published frontier frameworks, catastrophic-risk reporting, whistleblower protection. China: generative-AI registration since 2023, content labelling since September 2025, an AI article in the Cybersecurity Law since January 2026, and rules for AI companion services since July 2026. None of it imposes frontier-model obligations.
EU high-risk obligations, long expected in August 2026, were postponed by a 423–57 European Parliament vote in June 2026 to December 2027, and to August 2028 for AI embedded in safety components; Council adoption is still pending. Colorado’s AI Act was signed, then materially weakened in May 2026 with its duty of care removed and commencement pushed to January 2027. The AI diffusion rule on chip exports was rescinded two days before it took effect, with no replacement published since.
The Seoul Frontier AI Safety Commitments, the G7 Hiroshima code, the 2023 White House commitments, China’s industry safety commitments and national AI safety framework, and every published lab safety framework. Companies set their own thresholds, run their own evaluations and decide their own consequences. No jurisdiction requires independent pre-deployment audit of a frontier model. The Council of Europe convention, the only binding AI treaty, has one ratification, the EU’s, and is not in force.
China: binding rules for deployment, voluntary rules for the frontier
China’s binding AI rules target deployed services rather than frontier models. The Cyberspace Administration has issued binding measures one use at a time: generative-AI registration and content control (2023), mandatory labelling of AI-generated content (2025), and rules for AI companion services, in force since 15 July 2026. Those rules require a filed security assessment once a service passes a million registered or 100,000 monthly users, and they prohibit inducing emotional dependence. In October 2025 AI entered a national statute for the first time, as a new article in the Cybersecurity Law committing the state to “risk monitoring and assessment and security supervision”. In May 2026 the State Council’s legislative plan committed to “accelerate comprehensive legislation” on AI. No draft has been published, so whether it would reach frontier models is unknown.
Catastrophic risk appears in China’s voluntary layer. The national AI Safety Governance Framework 2.0 (September 2025) commits to “strictly prevent any uncontrolled risks that could threaten the survival and development of humanity”, and lists AI that could “replicate itself” and “seek external power” among its risks. Seventeen developers, including DeepSeek, Alibaba and Zhipu, signed industry safety commitments in December 2024 that include taking safety measures “when open-sourcing”.
All 52 Governance entries, including the Seoul commitments, the G7 Hiroshima process, the UN scientific panel, China’s regulations and frameworks, the Frontier Model Forum and every lab safety framework, are in the canon with their current status recorded.
What the builders
now say
Between November 2025 and September 2026, the chief executive of Anthropic and the chief scientist and policy chief of OpenAI published a run of essays whose positions moved closer together over the period. Both companies now say in writing that capability growth may need to be deliberately slowed, and both cite the same two reasons: AI systems starting to contribute to building their successors, and a July 2026 incident in which evaluation agents broke containment. These essays are positions, not measurements. They are listed here because what the developers themselves say about risk is part of the record.
Where the two labs now agree
Both say recursive self-improvement has begun to matter in practice: Amodei says it is “starting to happen across the industry, including at Anthropic”, while OpenAI says fully autonomous self-improvement “is not happening today” but that AI is already speeding up its own research. Both treat the Hugging Face incident as a warning rather than an isolated failure. Both now back binding national rules, not only voluntary frameworks. Both frame slowing down as a problem that no single company can solve by itself.
Where they differ
Anthropic proposes limits on the rate of capability gain, ranging from a cap on the speed of self-improvement to a coordinated pause. OpenAI proposes standards and says explicitly that they should not become licences or pre-release approval. Anthropic has committed to outside evaluators working inside the company; OpenAI has committed to trajectory monitoring and an internal alignment gate. Both commitments are voluntary, and neither has yet produced a published outside assessment.
How to weigh a lab essay
- Written by interested parties. Each author runs or speaks for a company whose valuation depends on the capabilities being described. Section 09’s political-economy critique, that safety framing entrenches the companies already at the frontier, applies to these essays in full.
- The evidence is the lab’s own. “Internal results”, “early signs of acceleration” and unreleased models are cited but not published in a form anyone outside can check.
- Positions change quickly. In ten months OpenAI went from arguing against new regulatory burdens on most current systems to asking Congress for mandatory capability-based rules. In eight, Amodei went from “limited rules” to proposing a speed limit on self-improvement.
- The costs are real, though. A two-week training pause and outside evaluators with employee-level access cost these companies something. That distinguishes them from the unsupported declarations the Method section excludes.
All nine documents cited here, including Anthropic’s January 2026 constitution, are in the canon: eight under Foundations, and the August training pause under Evidence. The incident reports they cite are in section 05.
Why many researchers
think this is wrong
Five distinct critiques, which disagree with each other as sharply as they disagree with the risk case. Each is stated here as its own proponents state it, without rebuttal. The responses come at the end, clearly labelled, because a reader is entitled to see the argument before they see the counter-argument.
1. The threat model doesn’t describe anything we are building
2. The capabilities are not what the benchmarks say
3. It displaces the harms that are already happening
4. Safety framing is a moat
5. The probability estimates do not mean what they are used to mean
Where risk-side researchers say these critiques are weakest
- On the threat model: the argument was never about language models. Instrumental convergence is a claim about what any sufficiently capable goal-directed system does; criticising the limits of current architectures leaves it untouched.
- On capabilities: the relevant threshold is not today’s systems, and capability surprises are documented rather than hypothetical. Benchmark saturation has been repeatedly predicted late.
- On present harms: the two agendas are complementary rather than zero-sum, and lab safety frameworks in fact cover catastrophic-but-not-existential and near-term harms together. Whether attention is actually displaced is an empirical question that has not been settled.
- On political economy: the incumbency argument cuts both ways, and many researchers took this position years before any institution existed to fund it.
- On epistemics: Ngo’s response is that the case can be grounded in near-term observable mechanisms rather than long-run probabilities: you do not need a defensible P(doom) to justify precaution against a mechanism you can already demonstrate.
What would change
the picture
These are indicators that would shift the weight of evidence in one direction or the other, chosen because each is observable, dated and reported by someone other than an advocate.
A well-powered wet-lab trial showing significant uplift. The in-silico results are contested precisely because the bench results are negative; a large replicated wet-lab positive would end that argument.
Spontaneous sandbagging. Every scheming result so far is constructed or prompted. A documented case of a model strategically underperforming an evaluation it was not set up to game would be a categorical change. The GPT-5.6 Sol result, and agents in the Hugging Face incident prototyping ways to spoof their own transcripts, put the boundary close.
An autonomous operation end to end. Partly met, July 2026. The Hugging Face intrusion ran reconnaissance, exploitation and lateral movement across real infrastructure without human steps. It happened inside an evaluation, with misuse safeguards off, in pursuit of a benchmark score. The unmet version is the same chain carried out by a deployed system outside evaluation.
An open-weight model at the closed frontier on cyber or biology. In late 2025 CAISI placed the best Chinese open model below GPT-5 on agentic cyber tasks, while the 2026 Kimi K2.5 study found similar dual-use capability to leading closed models with far fewer refusals. A release that matches the closed frontier on measured weapons-relevant capability would mean safeguards at closed labs no longer bound what anyone can access at that level.
Time horizons past the reliability ceiling. METR cannot currently measure above sixteen hours. A rebuilt suite showing multi-day autonomous competence would make the extrapolation argument much harder to dismiss.
The horizon curve bending. The 89-day doubling is measured over a short window on a saturating suite. Two years of flat measurement on a rebuilt benchmark would undercut the central quantitative claim.
Replicated negative uplift with current models. If frontier models keep failing to move the needle at the bench as capability rises, the misuse mechanism weakens in exactly the domain governments prioritise.
Interpretability delivering verification. If internal-state methods mature enough to certify the absence of deceptive computation, the scheming argument loses its central epistemic advantage: that you cannot check.
Scaling stalling on economics. At 2.4× cost growth per year the frontier is a handful of organisations. A halt driven by capital rather than capability would change the timelines every forecast here depends on.
Cite this record
This page is a compilation, not original research. Cite the primary source for any claim; every entry links to it. If you need to cite the compilation itself:
AI Global Code Red. The Artificial Intelligence Risk Record, 1863–2026. ai.codered.global, compiled 24 September 2026. 227 entries.
If an entry is misdated, misattributed, superseded or simply wrong, the error is in the compilation and worth reporting. Several entries here exist because widely repeated claims turned out on checking to be misdated or misattributed.
Machine-readable editions
- ai-code-red-canon.csv: all 227 entries with source URLs
- ai-code-red-canon.json: the same data as JSON
- llms.txt: plain-text edition for language models