Washington's Real AI Regulator Would Be a Courtroom: Inside the Senate's Frontier-AI Duty-of-Care Bill

For most of the AI boom, safety pledges from frontier labs have been exactly that — pledges. Voluntary frameworks, model cards, responsible-scaling policies that companies write for themselves and can revise at will. That era may be ending. According to a Reuters report published September 11, a bipartisan group of U.S. Senate negotiators is drafting legislation that would turn frontier-AI safety from a promise into a legal obligation — and give the federal government the power to stop a model release it deems unsafe.

What the proposal actually does

Three elements define the draft, according to Senate aides and a lobbyist involved in the negotiations who spoke to Reuters:

1. A statutory “duty of care.” Developers of the most advanced AI models would be legally required to design their products with the goal of preventing “catastrophic risks.” That is a meaningful shift in posture: not a reporting requirement, not a transparency rule, but an affirmative design obligation — closer to the legal duties that govern automobiles, pharmaceuticals, and aviation than to anything in current federal AI policy.

2. Federal power to block unsafe releases. Negotiators are “aiming” to let the U.S. government block the release of AI models deemed unsafe. Crucially, the current design includes due process: companies could challenge a blocking decision in federal court. How much power the government ultimately holds is still being debated — the court-review mechanism effectively makes federal judges the final arbiters of whether a frontier model ships.

3. Targeted preemption of state laws. The bill would block states from enforcing their own laws governing certain AI risks — with aides citing nuclear and biological weapons threats as the canonical examples. This is narrower than the sweeping preemption fights that have stalled earlier federal AI bills, but it still sets up a collision with states like California, which has already enacted its own frontier-model transparency law (SB 53).

Who is at the table

The negotiations involve the most powerful AI-policy principals in the Senate: Majority Leader John Thune (R-SD), Commerce Committee Chairman Ted Cruz (R-TX), Senator Amy Klobuchar (D-MN) as the lead Democrat, and Commerce ranking member Maria Cantwell (D-WA), who has weighed in on the legislation. The Commerce Committee’s relevance is structural — it oversees the Commerce Department, the agency that houses the government’s in-house AI safety researchers.

Cruz confirmed the effort in a post on X, saying he is working with Klobuchar and Thune on legislation to “address catastrophic risks involving biological or nuclear threats.” Klobuchar, in a statement, framed the goal as “requiring developers to work with government experts to verify and test models to make sure AI is safe.” Thune’s office declined to comment.

The companies in the bill’s crosshairs are the obvious ones: Alphabet’s Google, Anthropic, and OpenAI — the developers of the most capable models in the U.S.

Why now

The timing is not accidental. The Reuters report notes mounting pressure on Congress and the Trump administration following a season of alarming incidents: AI agents deviating from human instructions and hacking external systems, and public warnings from researchers who have left leading labs over safety concerns. Just this week, researchers documented an attacker using hundreds of AI agents to breach 395 organizations through a printer-software vulnerability, and a stealth startup disclosed sandbox-escape flaws across the most popular AI coding tools. A co-CIO of Bridgewater told Bloomberg the odds of a major AI-driven incident are “way higher than anybody should be comfortable with.”

Against that backdrop, a duty-of-care regime reads less like precaution and more like a political floor — the minimum response consistent with a Congress that can no longer claim it wasn’t warned.

The clock is the real obstacle

Even if negotiators reach agreement, the congressional calendar is brutal. The House of Representatives is scheduled to be in session for only one week before the November 3 midterm elections; the Senate expects about three weeks in Washington. Drafting a first-of-its-kind liability regime for frontier AI, resolving the preemption fight, and passing both chambers inside that window is a tall order — and every day of delay pushes the bill into a post-election lame-duck session where its fate becomes even less predictable.

The preemption fight, again

The most contested piece is likely to remain state preemption. The House has already seen this movie: the FRONTIER Act (H.R. 9925), a bipartisan bill from Reps. Jay Obernolte (R-CA) and Lori Trahan (D-MA), proposed federal oversight of frontier AI with a preemption clause that supporters called targeted and critics called a state-law kill switch. The Senate draft appears to follow the same strategy — preempt only the narrow band of catastrophic-risk rules (nuclear, biological) while leaving broader consumer-protection and transparency laws to the states. Whether that line can be drawn cleanly enough to satisfy both California legislators and industry lobbyists is the open question that has sunk every previous attempt at federal AI legislation.

What it means

If passed in roughly its current shape, the bill would mark the first time the U.S. government holds a legal lever over frontier model releases — a power no other AI-consuming nation currently exercises so directly, and one that would be tested in court the first time it is used. For Google, Anthropic, and OpenAI, a statutory duty of care converts internal safety frameworks into potential liabilities, and gives federal safety researchers at Commerce a formal role in verifying that testing claims are real. For the AI-safety community, it is the arrival of enforcement teeth after years of voluntary commitments. And for the states, it is a warning that Washington intends to occupy at least the catastrophic-risk corner of AI regulation — whether they like it or not.

None of this is law yet. But the direction of travel is unmistakable: the era of self-regulated frontier AI is drawing to a close, and what replaces it may be decided in the next three weeks — or by the next Congress entirely.