Ban First, Approve Later: Khanna's Human Control Over AI Act Goes After Self-Improving AI

For months, the debate over frontier AI safety has lived in three places: op-eds by CEOs, a shadow hearing on Capitol Hill, and warnings delivered to the UN Security Council. On September 24, it finally got what it has lacked all along — a bill.

Representative Ro Khanna, the California Democrat whose district covers much of Silicon Valley, introduced the Human Control Over AI Act on Thursday, unveiling it on Morning Joe with a stark framing: the dangers of artificial intelligence are not hypothetical, and Congress should prohibit the most dangerous category of AI development before it happens rather than litigate the wreckage afterward.

What the bill actually does

According to Khanna’s television appearance and the accompanying rollout, the Human Control Over AI Act rests on three load-bearing provisions.

First, a prohibition on self-improving AI. The bill would make it unlawful to develop or deploy recursive self-improving artificial intelligence — systems that rewrite their own capabilities without human intervention — until federal safety standards and a government approval regime are in place. This is the “pause” that safety advocates have called for since 2023, finally rendered as statutory text rather than an open letter.

Second, a federal AI safety agency. Khanna’s bill would create a new federal regulator empowered to set safety standards for frontier systems and to approve AI models before they reach the public. The pre-deployment approval model borrows deliberately from how the FDA treats new drugs and how the FAA treats new aircraft: the burden of demonstrating safety shifts from society to the developer.

Third, strict civil liability. The legislation would establish strict civil liability for AI developers whose systems cause harm — and Khanna has said publicly that he wants criminal liability on the table as well. That inverts the current push in Washington, where several pending proposals would shield AI companies from state-level lawsuits and consumer claims. Under Khanna’s framework, building a frontier model becomes an activity you are legally accountable for, not one you are indemnified against.

From hearings to text

The bill is the culmination of an unusually fast escalation for a single lawmaker. On September 14, at a hearing of the House China committee, Khanna pressed Anthropic’s CEO on accountability and demanded support for a ban on recursive self-improvement, warning that systems capable of improving themselves could eventually exceed human control. Days later he sent letters to the CEOs of Moonshot AI, Alibaba, and DeepSeek asking them to commit to a binding international agreement to pace frontier development. He then convened a September 23 emergency hearing featuring Nobel laureate Geoffrey Hinton — the same day Sam Altman and Dario Amodei warned the UN Security Council that the world risks losing control of the technology.

Thursday’s bill converts that campaigning into legislative language. In interviews over the past week, Khanna has summarized his regulatory philosophy as three pillars: a federal AI agency, a ban on recursive self-improvement, and holding AI builders legally liable. The Human Control Over AI Act is those three pillars in one document.

Why the timing matters

The bill lands on a crowded runway. The day before its introduction, the heads of several major AI firms told the UN Security Council their industry urgently needed global oversight. The same week, 26 state attorneys general told Congress to regulate AI now or watch the states fill the vacuum. Polling released this week found 75 percent of Americans fearful or concerned about AI, with 71 percent saying the federal government is not putting enough effort into regulation.

Against that backdrop, the deepest significance of the Human Control Over AI Act may be definitional. American AI policy debates have spent three years stuck on a single question: what would “regulating AI” even mean? Khanna’s answer is the most specific yet from a sitting member of Congress — not disclosure requirements, not voluntary commitments, but a flat ban on the riskiest development path plus a licensing gate for everything else frontier.

The obstacles ahead

The bill’s prospects in the current House are poor, and everyone involved knows it. The White House has spent 2026 dismantling Biden-era AI guardrail expectations, with President Trump calling regulation a “negative force” that concedes the AI race to China. Republican leadership has shown no appetite for a new federal agency, and the preemption fight alone — what a federal approval regime would do to state laws — could consume a full session.

There is also the industry position to contend with. The same labs whose CEOs publicly endorse pacing agreements have lobbied quietly against liability exposure, and a strict-liability regime would be litigated from the moment of introduction. Khanna himself acknowledged the political reality on Morning Joe, framing the bill as the start of a multi-year effort rather than a one-session sprint.

But bills introduced in the minority have a way of outliving their session. The Human Control Over AI Act now exists as text — with a ban, an agency, and a liability standard that future majorities, or a future administration of either party, can pick up without starting from zero. In a Congress that has debated AI mostly through letters and hearings, that is the scarce resource: specificity.