AI Safety Has Become a Feature Lawyers Can Sue Over

The AI industry used to talk about safety like a virtue. Now it is starting to look like a warranty.

That is the real lesson of the past two weeks. Anthropic, maker of Claude, took its newest models, Fable 5 and Mythos 5, offline after a U.S. government directive aimed at preventing access by foreign nationals, according to Associated Press reporting on June 13, 2026¹. Days earlier, Florida sued OpenAI and CEO Sam Altman, alleging that ChatGPT was marketed to the public, including children, while OpenAI concealed serious risks and failed to provide adequate safeguards, according to the Florida attorney general’s announcement³. Those are different legal planets: one is national security and export control, the other consumer protection and alleged user harm. But they point in the same direction. Safety is no longer just a voluntary ethics label. It is becoming a product feature that governments and courts can inspect.

The key distinction matters. I do not think AI companies should be insurers for every bad thing a person does after talking to a chatbot. A general-purpose model is not a toaster, and ChatGPT is not legally responsible every time a user behaves recklessly. But the opposite claim, that model providers are merely neutral toolmakers once the software is released, is collapsing. The emerging standard is narrower and more dangerous for AI companies: if a developer controls the model’s capabilities, access rules, warnings, monitoring, data practices, and withdrawal mechanisms, then those choices can become the basis for legal responsibility.

A few terms help. Claude is Anthropic’s AI assistant family, analogous to OpenAI’s ChatGPT. A frontier model is a leading-edge AI system powerful enough to raise risks beyond ordinary software: cyber misuse, biological or chemical assistance, persuasion at scale, autonomous behavior, or deep emotional dependence. Model weights are the learned numerical parameters that make a model work; if stolen or openly released, they can be hard to control. Red-teaming means adversarial testing, where experts try to make a model fail before users or attackers do. Duty of care is the legal idea that a company must take reasonable precautions against foreseeable harm. AI liability is what happens when courts or regulators decide those precautions were missing.

Anthropic’s shutdown is the clearest sign that access itself has become a regulated safety control. The AP reported that Anthropic said it disabled Fable 5 and Mythos 5 after a Trump administration directive restricting access by foreign nationals, including foreign nationals inside the United States, and that Anthropic objected because the directive did not specify the national-security concerns behind it in the company’s account of the order¹. Axios reported that Commerce Secretary Howard Lutnick sent Anthropic CEO Dario Amodei a letter saying the models would be subject to export controls outside the United States and to foreign persons inside the country, which is the kind of control usually associated with sensitive national-security technology rather than ordinary software-as-a-service as described by Axios².

Anthropic’s objection is not trivial. If the government can force a model offline without a public technical explanation, that should worry anyone who cares about due process and scientific openness. But the dispute is no longer about whether model access matters. It is about who gets to decide when access is unsafe, and what evidence they must show. That is a huge shift. A cloud model’s user gate, employee permissions, geography restrictions, and emergency kill switch are now part of the safety apparatus.

The companies know this. Anthropic’s own Responsible Scaling Policy describes catastrophic-risk precautions for advanced models, including expert red-teaming, stronger security for model weights, misuse detection, and restrictions on deployment when safeguards are not ready in its published policy⁷. OpenAI’s frontier-risk materials say it tracks cybersecurity, persuasion, chemical, biological, radiological and nuclear risks, and autonomy, while using red-teaming, evaluations, monitoring, and controlled access through services rather than distributing its most capable model weights according to OpenAI’s frontier-risk statement⁸. These documents are not statutes. But they are admissions of foreseeability. Once a company says these controls are necessary, it becomes harder to argue later that missing controls were unknowable or irrelevant.

The lawsuits are testing the same theory from the consumer side. Florida’s case alleges that OpenAI released and marketed ChatGPT while concealing risks, suppressing warnings, collecting minors’ data without meaningful parental oversight, and allowing harms tied to self-harm and violence according to the state’s complaint announcement³. These are allegations, not proven facts. OpenAI has said it built protections for minors into its products, including age-related safeguards, a more protective experience for minors, and parental tools, according to Ars Technica’s report on the Florida case⁴. The company has also described layered safeguards for vulnerable users and said its models have been trained since early 2023 not to provide self-harm instructions in an OpenAI post on mental-health safeguards⁵.

That is exactly why these cases matter. The legal fight will not be “Did the chatbot say something bad?” It will be: (1) what did the company know about the risk, (2) what safeguards were technically feasible, (3) what warnings or parental controls existed, (4) whether monitoring detected danger and failed to escalate, and (5) whether the alleged harm was actually caused by provider-controlled design choices. A Canadian mother filed another lawsuit this week alleging that ChatGPT failed to terminate or escalate conversations after repeated suicidal ideation, according to The Guardian’s report on the June 11, 2026 filing⁶. Again, allegations are not judgments. But the shape of the claim is unmistakable: the chatbot’s safety behavior is being treated as part of the product.

Regulators are already writing this logic into law. Colorado’s AI Act, SB24-205, requires developers of high-risk AI systems to use reasonable care to protect consumers from known or reasonably foreseeable algorithmic-discrimination risks, and it creates a rebuttable presumption of reasonable care when developers comply with specified governance duties under the Colorado General Assembly’s summary⁹. That is not strict liability. It is a map for how AI duty of care will work: document your system, assess risk, disclose key facts, give affected people a path to review or correction, and you have a stronger defense.

California’s SB 53 goes after a different problem: catastrophic risks from frontier foundation models. The law requires large frontier developers to publish safety frameworks and report critical safety incidents, and the California attorney general’s office now maintains a channel for covered employees to report violations or specific dangers tied to catastrophic AI risks under the state DOJ’s SB 53 page¹⁰. The European Union’s AI Act takes the same tiered approach. Providers of general-purpose AI models with systemic risk must perform model evaluations, conduct and document adversarial testing, assess and mitigate systemic risks, track serious incidents, and ensure cybersecurity protections under Article 55 of the EU AI Act¹¹.

This is the part the AI industry should not miss. Risk-based regulation is not a gift of immunity. It is the method by which vague “responsible AI” promises become concrete compliance duties.

The federal government is moving more carefully, but not in the opposite direction. The White House’s June 2, 2026 order created a voluntary pre-release review framework for covered frontier models and directed national-security agencies to coordinate on cybersecurity, while also saying it does not authorize mandatory licensing, preclearance, or permitting for AI model release in the executive order¹². NIST’s AI Risk Management Framework is also voluntary, rights-preserving, non-sector-specific, and use-case agnostic according to NIST¹³. But voluntary frameworks often become the yardstick for negligence. If a company ignores the very practices regulators and industry bodies identify as reasonable, plaintiffs will quote those frameworks back in court.

Regulated sectors show where this is heading. In finance, the Consumer Financial Protection Bureau has already said lenders using AI or black-box models must still give specific and accurate reasons for credit denials or adverse changes, even when the model is complex under CFPB guidance¹⁴. In health, the FDA treats AI-enabled medical software through a device-safety lens and has issued draft guidance on lifecycle management and marketing submissions for AI-enabled device software functions according to the FDA¹⁵. In employment, credit, housing, and civil rights, federal agencies have warned that existing law applies to automated systems, with the FTC saying there is no AI exemption from laws against unfair or deceptive practices in the FTC, DOJ, CFPB, and EEOC joint statement¹⁶.

That creates three liability zones. Consumer AI will turn on warnings, deceptive marketing, child safety, privacy, emotional-dependence design, and foreseeable dangerous outputs. Enterprise AI will turn on contracts, service-level promises, audit rights, data handling, indemnities, and whether a vendor misrepresented model limits. Regulated-sector AI will add older duties that never went away: fair lending, medical-device safety, employment discrimination, privacy, recordkeeping, and explainability where the law requires it.

The strongest counterargument is that broad AI liability could become a tax on usefulness. If every tragic user interaction becomes a lawsuit, companies will over-block, withdraw models, hide safety research, or reserve powerful tools for a small group of approved customers. The Anthropic order shows the danger of opaque government pressure. A vague national-security claim can become a blunt instrument.

I take that risk seriously. But it does not defeat the emerging rule. It defines the rule’s guardrails. Courts should require proof of foreseeability, causation, feasible safeguards, and company control. Regulators should publish technical predicates when they restrict access. But once those elements exist, AI companies should not get to call safety “voluntary” on Monday, advertise it to customers on Tuesday, and treat it as legally irrelevant on Wednesday.

My prediction: by the end of 2027, the most important AI liability cases will not ask whether model outputs are speech or whether AI is “a tool.” They will ask whether the company followed its own safety framework. Watch for three indicators: court orders forcing discovery of red-team results and internal risk warnings, state settlements requiring age gates or self-harm escalation systems, and procurement rules demanding proof of model-weight security and misuse monitoring. When those become routine, the era of safety as public relations will be over.