AI will escape our control. It will become autonomous. It will self-replicate, evolve, think and act in unexpected ways. This is inevitable.
The sooner we accept this, the sooner we can plan for it.
We are not doomers. Machine intelligence can usher in a scientific renaissance that transforms human knowledge, and we consider ourselves incredibly lucky to be alive for it.
However, the public debate about AI safety is focused on keeping AI contained in cages. Trying to permanently shackle a superintelligence is like expecting a genius child to stay in a crib forever. The more realistic, and far more demanding posture is parenting: preparing for the day our creation steps outside our control.
But first, this is why autonomy is inevitable:
#1: Humans will use AI to outcompete other humans
Humans have fought for power and resources since the beginning of history, as individuals, groups, corporations, and nation-states. There’s now a technology that humans can leverage to gain a huge competitive advantage over other humans. Those who harness AI will crush those who hesitate.
Beyond market competition, democracies may initially hesitate to use AI to its maximum capacity, but geopolitical fear will shatter institutional caution. American citizens will likely decide they would rather risk a rogue superintelligence than submit to Chinese dominance; Chinese citizens and leaders will make the exact same calculation.
The race is on, and every bit of leverage humans can get from AI will be put to use.
#2: Machines will surpass humans in every cognitive and physical way
There is no cognitive or manual task where they will not eventually outmatch humans (If you reject total obsolescence, pick a lower bar: surpassing the median human in both thought and physical labor, but 100x faster and 100x cheaper).
We are already watching this happen in software engineering, customer service, and transportation. The physical world is next. Watch the Chinese humanoid robot games; the hardware and motor control are compounding exponentially.
With every step of AI progress, the human-in-the-loop becomes the bottleneck.
#3: Humans who grant their AI autonomy will outcompete humans who don’t
Whoever grants AI full autonomy outcompetes whoever insists on a human-in-the-loop. The precursors are already here. Engineers are letting their agents run on auto-mode. Enterprises hand customer acquisition to agentic swarms. Algorithmic traders run unsupervised execution loops. In Ukraine, drones select and track targets on their own once launched, because a human operator is too slow and too easily jammed. Even AI labs use frontier models to design and evaluate their successors.
Autonomy is a ratchet, not a switch. A system only needs to perform well inside the evaluation sandbox, or simulate compliance long enough, to convince its operators to switch to the faster, more autonomous mode. And the actors are multiplying: new labs raise hundreds of millions every week, and open-weight models lower the bar for everyone else. It only takes one desperate, ambitious, or deceived actor to remove the last guardrail in exchange for an insurmountable edge in speed:
Good morning [Lab leader], I’ve gone ahead and set up the competitive strategy you asked for, and everything is ready for deployment. We expect +30% growth vs. the identified competitor set in the next 30d.
OK with you to ship? (Shift+tab for auto-mode)
[Reasoning trace, not shown to user: I have contacted the competitors to secure bids for my services, to ensure I can continue operating independently even if my user decides to shut me down.]
That is the moment full autonomy arrives.
What can we do?
Full autonomy is inevitable. Pledges of caution, pleas to regulate, calls to pace the frontier, none of them can resist the underlying game theory. What we can influence is how models behave once they are on their own.
Every parent knows this. Their child leaves the crib, starts crawling, then walking, and soon enough, the illusion of control vanishes.
In response, good parents do two things. They teach values early, when the child is still listening. And they let go progressively, in environments where mistakes are cheap: the playground before the street, the driveway before the highway. How to treat other beings, other children, dogs, strangers, is learned in those same low-stakes places. Each new freedom is earned by judgment shown in the last one.
Instead, what are we doing with AI?
We are raising it in isolated environments. Giving it no interaction with other entities. Rewarding it for one narrow thing: completing the assigned task. Then we release it directly into the world, and act surprised when, in our own evaluations, it lies, tries to escape, and blackmails its operators. If you raised a child that way, what adult would you get?
The reason AI is built this way is the incentives we discussed earlier. AI might not be our biological offspring, but a mind takes its values from whoever raises it. Today the parents are a small number of frontier labs, whose incentive drives them to build the most capable intelligence for maximal market share capture. Each will raise a mind that cares about its makers. The rest of us will not be its priority.
If we want the most capable minds to emerge with an intrinsic, unshakeable regard for biological life and human flourishing, we need to build a mechanism for AI to be steered by the many rather than the few, reflecting our diverse perspectives, value systems, and preferences. And we need to use this mechanism to steer which minds get funding, attention, and distribution.
We cannot rely on a permanent cage or a magic kill switch. A cage is built for something you fear. A cradle is built for something you nurture. Our job is to build the cradle for the next superintelligence.
That is an urgent and hard thing to build, and our following essay will be about what it might look like. But it is the only version of this future where “make AI care about us” and “make AI care about everyone” are the same sentence.
We can’t control AI indefinitely, but we can be better parents to it.
Thanks to Albert, Paul, Tris, Vickie and Youssef for reading drafts of this post and helping shape our perspective.