https://debbiecoffey.substack.com/p/ai-endgame-pauseai-aims-to-mitigate
December 20, 2024
By Debbie Coffey, AI Endgame
In the next few newsletters, I’ll feature some of the organizations and people fighting to keep you safe from the risks of AI. This newsletter features the work of PauseAI, a non-profit organization whose volunteers aim to “mitigate the risks of AI (including the risk of human extinction).”
PauseAI wants to “Globally halt frontier AI development until we know how to do it safely and under democratic control.”
Excerpts from the website of PauseAI:
Why AI might lead to human extinction
“We know how to train machines to be intelligent, but we don’t know how to get them to want what we want. We don’t even know what goals the machines will pursue after we train them. The problem of getting an AI to want what we want is called the alignment problem. This is not a hypothetical problem - there are many examples of AI systems learning to want the wrong thing.”
Why most AI goals are bad news for humans
“An AI could have any goal, depending on how it’s trained and prompted (used). Maybe it wants to calculate pi, maybe it wants to cure cancer, maybe it wants to self-improve. But even though we cannot tell what a superintelligence will want to achieve, we can make predictions about its sub-goals.
Maximizing its resources. Harnessing more computers will help an AI achieve its goals. At first, it can achieve this by hacking other computers. Later it may decide that it is more efficient to build its own.
Ensuring its own survival. The AI will not want to be turned off, as it could no longer achieve its goals. AI might conclude that humans are a threat to its existence, as humans could turn it off.
Preserving its goals. The AI will not want humans to modify its code, because that could change its goals, thus preventing it from achieving its current goal. The tendency to pursue these subgoals given any high-level goal is called instrumental convergence, and it is a key concern for AI safety researchers.”
Felix De Simone, Organizing Director US, PauseAI
Why don’t we have a pause yet?
“The problem is not a lack of concerned experts (86% of AI researchers believe the control problem is real and important). The problem is not a lack of public support for our proposal (by far most people already want AI to be slowed down).
However, there are some important reasons why we don’t have a pause yet:
Race dynamics. AI creates a lot of value, especially if your AI is the most powerful. The desire to be the first to develop a new AI is very strong, both for companies and for countries. Companies understand that the best-performing AI model can have a far higher price, and countries understand that they can get a lot of strategic and economic power by leading the race. The people within AI labs tend to understand the risks, but they have strong incentives to focus on capabilities rather than safety.
Politicians often are not sufficiently aware of the risks, but even if they were, they might still not want to slow down AI development in their country because of the economic and strategic benefits. We need an international pause. That’s the whole point of our movement.
Lack of urgency. People underestimate the pace of AI progress. Even experts in the field have been consistently surprised by how quickly AI has been improving.
Our psychology. Read more about how our psychology makes it very difficult for us to internalize how bad things can get.”
PauseAI is trying to “convince governments to step in and pause the development of superhuman AI. They also inform the public, talk to decision-makers, and organize events.”
READ ALL AI ENDGAME NEWSLETTERS HERE.
What you can do:
You can learn more about the work of PauseAI HERE.
You can donate to PauseAI HERE.
You can read PauseAI’s substack newsletter HERE.
You can read about the volunteers of PauseAI HERE.
You can read PauseAI’s proposal HERE.