AI Risk Management: A Pre-Mortem on the First AI-Enabled Terrorist Attack

Harvard’s Kennedy School recently published a paper titled “A Pre-mortem on The First Major AI-Enabled Terrorism Event and The Choices That Could Enable It.” The paper addresses how a major AI-enabled cyber, bio or chemical terrorism event might happen and how the choices made across an AI model’s lifecycle might contribute to the event.

The paper says that the risk profile and vulnerabilities of closed weight AI models – those whose parameters are proprietary and never disclosed by the developer – and open weight AI models – those whose parameters are publicly available – differ significantly. The paper addresses how dangerous capabilities in each of these models might be elicited and how their respective detection efforts and safeguards would fail. This excerpt addresses how a closed weight model might be compromised in a terrorist attack:

Overarching prediction 1. If a major AI-enabled cyber, biological, or chemical terror event occurs in the foreseeable future, it will have been enabled either by a closed-weight model deployed without state-ofthe-art safeguards, or by an open-weight model whose safeguards (if significant ones exist) could not survive downstream tampering.

If it was due to a closed-weight model:

2. We predict the model would not have been implemented with input, output, and trajectory-level safety classifiers up to par with the state of the art as of summer 2026. Anthropic’s constitutional classifiers withstood over 1,700 hours of red-teaming while maintaining a 0.5% improper-filtering rate (Cunningham et al., 2026), and subsequent staged attacks required query volumes high enough that user monitoring would flag them (Davies et al., 2026). A frontier developer choosing to ship a closedweight model without such a safety filter stack collapses the defense-in-depth advantage closed deployment otherwise offers.

3. We predict the model would not have implemented state-of-the-art safety training and adversarial fine-tuning as of summer 2026. By state-of-the-art we mean scalable automated adversarial red-teaming at scale, of the kind introduced in Wallace et al. (2026), combined with capability suppression methods and hazardous content pretraining data-filtering discussed in Section 3.4. Section 4 shows that refusal training raises the cost of misuse. Effortful, state-of-the-art model safety training has proven useful for closed-weight models. For example, the UK AI Security Institute reported a 40x increase in the required jailbreaking time between two models released six months apart (AISI, 2025).

4. We predict the model would not have had user-monitoring and account-restriction protocols to par with the state of the art as of summer 2026. By state-of-the-art, we mean monitoring instrumented to catch the high query volumes that current state-of-the-art jailbreak attacks require (Davies et al., 2026), user chat multi-turn monitoring to catch task decomposition attacks, detection that links activity across accounts belonging to the same actor, and enforcement that acts on what it finds. Clymer et al. (2025) illustrate through an example safety case for misuse safeguards.

This has led to documented success across the deepfake and cybercrime ecosystem as described in Section 4.1 and Section 4.2. In the deepfake ecosystem, closed-weight image and video model providers were able to monitor users and enforce their policies which led to no documented misuse for nonconsensual intimate imagery at scale. In the cybercrime ecosystem, model developers have been able to detect various state and non-state actors attempting to use their models for cybercrime and shut them down accordingly (Section 4).

The paper provides a similar analysis of how open weight AI models might be compromised, discusses two hypothetical scenarios involving terrorism events. It concludes with recommendations for action by developers, distributors, policy makers and researchers and funders to mitigate the risks posed by both closed weight and open weight models.