AI Risk Management: Catastrophic Risks

Here’s a new report from MIT that highlights the need to prioritize and address the risks posed by AI, and the potentially catastrophic consequences if the parties in the best position to address those risks continue with a “business as usual” approach.  I’m guessing it will ruin your whole day.

Anyway, the report classifies “catastrophic risks” as those that could result in the deaths of more than 1 million people, more than $100 billion in damage, or civilization-scale intangible harms (e.g., democratic norms, privacy).  Experts gave 18 of 24 identified risks at least a 10% probability of catastrophic outcomes over the next 5 years under a business as usual scenario.  This excerpt discusses the most significant of these risks and the experts’ assessment of their probability:

Dangerous capabilities.  AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system. (21.5% probability)

Competitive dynamics. Competition by AI developers or state-like actors in an AI “race” by rapidly developing, deploying, and applying AI systems to maximize strategic or economic advantage, increasing the risk they release unsafe and error-prone systems. (16.6% probability)

Weapons & Cyberattacks. Using AI systems to develop cyber weapons (e.g., by coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or chemical, biological, radiological, nuclear, and high-yield explosives), or use weapons to cause mass harm. (21.0% probability)

Power centralization. AI-driven concentration of power and resources within certain entities or groups, especially those with access to or ownership of powerful AI systems, leading to inequitable distribution of benefits and increased societal inequality. (18.0% probability)

False information. AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional, or material harms. (12.8% probability)

The good news is that the report says that pragmatic mitigation efforts can reduce these risks. The bad news is that even with these efforts, all 24 risks retained at least a 5% catastrophic probability, and that many remained above a 10% probability of causing catastrophic harm.  Sleep tight.