Regulation & the Race to the Bottom
I think the only way to prevent a race to the bottom in AI development is through regulation. But I also don’t think the current administration wants to regulate AI as aggressively as is actually needed.
It’s made me think about what could be accomplished for AI safety through private institutions, voluntary standards, and third-party review, as they may be the only tools realistically available right now. At the same time, they run into a fundamental enforcement problem.
Thought experiment: An existing lab or new company develops some form of artificial superintelligence that fails existing alignment standards. Who actually stops the lab from just releasing it anyway?
My (purely speculative) read of the current administration’s position is that they really believe they have a “stop button”. We saw a version of this after the Fable 5 launch when they imposed the export control, and Anthropic responded by suspending access.
But I think this example also exposes the limits of a “stop button”. It shows that the government can force a cooperating lab to cut off access to a model. But it doesn’t show that the government could stop a less cooperative lab, prevent the underlying technology from spreading, or even retain control if a more capable and autonomous system is released.
The current administration is determined to keep the US ahead in the global AI race. If you think about it from that perspective, why slow the development of something that could create a massive geopolitical advantage, especially if they believe they can intervene at any time, while being skeptical of the most extreme risk scenarios?
This creates a dangerous incentive structure where rewards overwhelmingly favor developing more powerful AI, not more aligned AI. Any lab willing to disregard safety constraints (e.g. by using opaque recurrent architectures, unrestricted recursive self-improvement, or whatever current and future techniques produce the best capability gains) could gain an advantage over more cautious competition. And that pressure is stronger if risks are seen as hypothetical and people think they’re exaggerated.
We’re competing over capability rather than alignment.
With all of this, the potential good remains the north star for me. If advanced AI could produce extraordinary scientific, medical, economic, and other progress, how could that not eventually become our future? The question is how we can reach that future at a pace we can survive.
It leaves two (overlapping) dangers:
- We’re not ready for ASI, even if it’s aligned. Getting there too fast could cause enormous harm while society is trying to adjust.
- We don’t know if alignment is solvable, and developing more capable systems may make the problem harder rather than easier.
Even if we were to develop a perfectly human-aligned ASI, it wouldn’t be safe to release right away. Jobs, markets, governments, and other institutions could be affected negatively faster than they could adapt sustainably. And just thinking about how we define “human-aligned” is its own problem. Values that this system embodies shouldn’t be chosen by a single company or group of researchers. It would need to be something closer to a democratic constitutional process.
The core problem is the combination of misaligned incentives and unrealized risk. The benefits of more capable systems are in sight, and enormously valuable. But the worst risks are uncertain, disputed, and mostly unrealized. Until that changes, every lab has to keep accelerating, even if we’d be safer under a process that allows progress to happen more deliberately.