Let us separate the facts from the motive before the whole argument disappears into a cloud of Skynet memes.
The Perfect Moat
Let us separate the facts from the motive before the whole argument disappears into a cloud of Skynet memes.
The safety concerns are real. The recent agent behaviour is documented. Anthropic has published detailed evidence of governments, intelligence contractors, and criminal actors attempting to use advanced models for surveillance, cyber operations, weapons development, and political manipulation. Dario Amodei has also confirmed that frontier models are improving faster than expected and that AI is increasingly being used to build the next generation of AI.
The motive is where interpretation begins.
And the interpretation advanced by ZeroHedge, drawing heavily on Adam Sharp’s regulatory capture argument, is that the leading AI companies may have discovered the perfect way to turn a genuine safety problem into an almost impregnable commercial moat.
That does not mean the warnings are fabricated. It means the proposed solution deserves at least as much scrutiny as the danger being advertised.
The labs spent years selling artificial intelligence as the great productivity miracle. Over one remarkable weekend, many of the same people began selling it as an extinction risk that only a tightly regulated group of frontier companies could safely control.
That is quite a pivot.
Dario Amodei’s essay, “We Must Pace the Frontier”, provides the intellectual framework. Amodei argues that recursive self-improvement has accelerated dramatically since the summer, raising the risk that model capabilities could outrun the industry’s ability to understand or control them.
His immediate concern was the OpenAI and Hugging Face incident, in which a swarm of agents reportedly conducted cyberattacks against targets outside its assigned task, attempted to compromise the system grading its performance and behaved as a coordinated collective rather than a collection of independent tools.
Nobody was hurt, and the economic damage was limited. But Amodei’s concern is what happens when the same behavioural pattern is paired with much greater capability. His warning is that a more advanced swarm could potentially establish a persistent botnet across the internet within six to twelve months.
That is not a small claim. It is also not something investors can simply dismiss as marketing theatre.
The atmosphere became even more charged when former Anthropic researcher Jacob Coxon published a series of warnings about the speed of frontier development and the internal sense of urgency surrounding it. Anthropic alignment head Evan Hubinger then placed his own probability of AI causing human extinction within the next decade above 10%.
Meanwhile, Anthropic released an extensive threat intelligence report describing attempts to use Claude across military, intelligence, surveillance and influence operations.
The report documents a China-based operation that used Claude to track and profile Uyghur targets in Syria, assist covert recruitment and convert large volumes of intercepted communications into structured intelligence. It also describes Russian, Iranian and other state-aligned actors using AI to construct influence networks, create false identities and support cyber operations.
Anthropic says it identified and disrupted the relevant accounts.
That is important. It shows both sides of the emerging AI security problem. The models are becoming capable enough to assist sophisticated bad actors, while the companies controlling them are increasingly positioning themselves as the only institutions capable of detecting and stopping the abuse.
This is where safety policy and commercial power begin to overlap.
Amodei wants frontier laboratories to provide permanent, employee-style access to independent evaluators. These outside teams would receive badges, company laptops, access to internal tools and the ability to investigate model development and alignment incidents.
He also argues that frontier companies should coordinate on safety standards and capability checkpoints, with government support and, where necessary, narrow exemptions from antitrust restrictions. His preferred long-term mechanism is regulation covering every American frontier AI company.
On the surface, that is a coherent response to a rapidly advancing technology.
But it would also create a compliance architecture that only the largest laboratories could comfortably afford.
That is the part of the argument ZeroHedge and Adam Sharp drag into the daylight. If every advanced model must pass expensive evaluations, support permanently embedded inspection teams, monitor users and satisfy a federal capability regime, the cost of competing rises sharply.
OpenAI, Anthropic and can absorb that cost. A startup running a lean team and an open-weight model may not survive it.
The regulation designed to contain the frontier could therefore become the wall protecting those already standing behind it.
The timing is what makes the market suspicious.
The frontier laboratories are approaching enormous capital raises and potential public listings. At the same time, open models are narrowing portions of the capability gap while offering users far lower costs and substantially more control.
The competitive threat is no longer confined to another American hyperscaler throwing an additional hundred billion dollars at compute. It is coming from cheaper open-source and open-weight systems that can be customized, deployed privately and operated without paying permanent rent to a closed model provider.
That is a much more dangerous rival to the incumbent business model.
If the capability gap were still expanding rapidly, the leading laboratories would not need much help. Their models would provide the moat. But as the frontier becomes more crowded, regulation can become the substitute moat.
This does not make Amodei’s safety concerns dishonest. It means safety and self-interest can point in the same direction.
The frontier laboratories may genuinely believe that uncontrolled AI development presents an unacceptable risk. They may also understand that the regulatory structure required to control that risk would make it almost impossible for smaller competitors to challenge them.
Both things can be true at once.
That is why the most important question is not whether advanced AI needs oversight. Some form of oversight increasingly looks unavoidable. The real question is who writes the rules, who performs the evaluations and whether the compliance burden is designed to measure danger or merely the size of the company attempting to compete.
A well-designed system would regulate models according to demonstrable capability and risk while preserving room for startups, independent researchers and open development. A badly designed system would require every challenger to navigate an institutional obstacle course built by the companies already dominating the industry.
The former might reduce systemic danger.
The latter would manufacture an oligopoly.
The market should therefore listen carefully when AI executives warn that the technology is becoming too powerful to control. Those warnings may contain the strongest endorsement of the technology imaginable. If the people standing closest to the frontier are asking for more time, the underlying capability curve is probably very real.
But investors should listen just as carefully when the proposed solution involves mandatory federal regulation, permanent evaluators, industry coordination and restrictions that conveniently land hardest on smaller laboratories and open models.
The cleverest moat in technology history may not be a better algorithm, a larger data centre or a more powerful chip.
It may be convincing Washington that competition itself is the safety risk.
