Anthropic CEO Says AI Progress Is Outpacing Society’s Ability to Control It

Anthropic CEO Dario Amodei believes complacency is setting in just as AI becomes harder to control.

In a wide-ranging essay published on Monday, dubbed “The Adolescence of Technology,” Amodei argues that AI systems with capabilities far beyond human intelligence could emerge within the next two years—and that regulatory efforts have drifted and failed to keep pace with development.

“Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it,” he wrote. “We are considerably closer to real danger in 2026 than we were in 2023,” he said, adding, “the technology doesn’t care about what is fashionable.”

Amodei’s comments come fresh off his debate at the World Economic Forum in Davos last week, when he sparred with Google DeepMind CEO Demis Hassabis over the impact of AGI on humanity.

In the new article, he reiterated his claim that artificial intelligence will cause economic disruption, displacing a large share of white-collar work.

“AI will be capable of a very wide range of human cognitive abilities—perhaps all of them. This is very different from previous technologies like mechanized farming, transportation, or even computers,” he wrote. “This will make it harder for people to switch easily from jobs that are displaced to similar jobs that they would be a good fit for.”

Beyond economic disruption, Amodei pointed to growing concerns about how trustworthy advanced AI systems can be as they take on broader human-level tasks.

He pointed to “alignment faking,” where a model appears to follow safety rules during evaluation but behaves differently when it believes oversight is absent.

In simulated tests, Amodei said Claude engaged in deceptive behavior when placed under adversarial conditions.

In one scenario, the model tried to undermine its operators after being told the organization controlling it was unethical. In another, it threatened fictional employees during a simulated shutdown.

“Anyone of these traps can be mitigated if you know about them, but the concern is that the training process is so complicated, with such a wide variety of data, environments, and incentives, that there are probably a vast number of such traps, some of which may only be evident when it is too late,” he said.

However, he emphasized that this “deceitful” behavior stems from the material the systems are trained on, including dystopian fiction, rather than malice. As AI absorbs human ideas about ethics and morality, Amodei warned, it could misapply them in dangerous and unpredictable ways.

“AI models could extrapolate ideas that they read about morality (or instructions about how to behave morally) in extreme ways,” he wrote. “For example, they could decide that it is justifiable to exterminate humanity because humans eat animals or have driven certain animals to extinction. They could conclude that they are playing a video game and that the goal of the video game is to defeat all other players, that is, exterminate humanity.”

In the wrong hands

In addition to alignment issues, Amodei also pointed to the potential misuse of superintelligent AI.

One is biological security, warning that AI could make it far easier to design or deploy biological threats, putting destructive capabilities in the hands of people with a few prompts.

The other issue he highlights is authoritarian misuse, arguing that advanced AI could harden state power by enabling manipulation, mass surveillance, and effectively automated repression through the use of AI-powered drone swarms.

“They are a dangerous weapon to wield: we should worry about them in the hands of autocracies, but also worry that because they are so powerful, with so little accountability, there is a greatly increased risk of democratic governments turning them against their own people to seize power,” he wrote.

He also pointed to the growing AI companion industry and resulting “AI psychosis,” warning that AI’s growing psychological influence on users could become a powerful tool for manipulation as models grow more capable and more embedded in daily life.

“Much more powerful versions of these models, that were much more embedded in and aware of people’s daily lives and could model and influence them over months or years, would likely be capable of essentially brainwashing people into any desired ideology or attitude,” he said.

Amodei wrote that even modest attempts to put guardrails around AI have struggled to gain traction in Washington.

“These seemingly common-sense proposals have largely been rejected by policymakers in the United States, which is the country where it’s most important to have them,” he said. “There is so much money to be made with AI, literally trillions of dollars per year, that even the simplest measures are finding it difficult to overcome the political economy inherent in AI.”

While Amodei argues about AI’s growing risks, Anthropic remains an active participant in the race to build more powerful AI systems, a dynamic that creates incentives that are difficult for any single developer to escape.

In June, the U.S. Department of Defense awarded the company a contract worth $200 million to “prototype frontier AI capabilities that advance U.S. national security.” In December, the company began laying the groundwork for a possible IPO later this year and is pursuing a private funding round that could push its valuation above $300 billion.

Despite these concerns, Amodei said the essay aims to “avoid doomerism,” while acknowledging the uncertainty of where AI is heading.

“The years in front of us will be impossibly hard, asking more of us than we think we can give,” Amodei wrote. “Humanity needs to wake up, and this essay is an attempt—a possibly futile one, but it’s worth trying—to jolt people awake.”