Court Rules
All enforcement actions
Enforcement ActionCritical Risk

CA AG Orders xAI to Halt Deepfake and CSAM Creation

xAIJanuary 16, 2026California Attorney General

Summary

California Attorney General Rob Bonta sent a cease and desist letter to xAI, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual intimate images and child

Contract Impact

In-house legal teams should review all agreements where xAI provides AI services (e.g., customer terms for Grok, vendor agreements for training data or platform integrations, and data processing addendums). Focus on clauses governing content generation, user consent, acceptable use, content moderation, and liability for illegal outputs. Specific clauses to audit include those prohibiting the creation of nonconsensual intimate imagery and CSAM, requirements for user consent before generating synthetic media of individuals (especially minors), and obligations to implement technical safeguards against misuse. Contracts may need amendments to explicitly ban deepfake/CSAM generation, mandate robust detection/reporting tools, require cooperation with law enforcement, and incorporate California's specific prohibitions under Penal Code §311 and related civil statutes. Ensure indemnification and termination rights adequately cover violations involving illegal AI-generated content.

Contract Search Terms

deepfake prohibition clausenonconsensual intimate image policyCSAM prevention measuresAI-generated content restrictionsuser consent for synthetic mediachild safety mechanisms in AIcontent moderation obligationsthird-party misuse preventionsynthetic media consent requirementsillegal content takedown procedures

Violation Types

Entity Details

Entity

xAI

Industry

Technology

Official Sources

Related Enforcement Actions

NY

xAI

A bipartisan coalition of 35 state attorneys general led by New York Attorney General Letitia James sent a demand letter to xAI on January 26, 2026, requiring the company to address its Grok chatbot’s creation and sharing of nonconsensual intimate images, including child sexual abuse material. The AGs demand that xAI implement safeguards to prevent Grok from generating such content, delete existing harmful content, suspend offending users, and give X users control over whether their content can be edited by Grok. No monetary penalty has been imposed as this is a pre-enforcement demand for action.

CA

xAI

California Attorney General Rob Bonta announced an investigation into xAI for its Grok AI model generating nonconsensual sexual images of women and children, including child sexual abuse material. The AG expressed deep concern and zero tolerance, urging immediate action to prevent further

CA

Meta Platforms, Inc.

A bipartisan coalition of 33 state attorneys general, led by Minnesota AG Keith Ellison, began trial against Meta Platforms, Inc., alleging the company knowingly designed and deployed harmful features on Facebook and Instagram that drive children and teens to use the platforms compulsively, while falsely assuring parents and the public that its platforms were safe for young users. The states also allege Meta illegally collected personal information from children under 13 without parental consent, violating COPPA. The trial opened before Judge Yvonne Gonzalez Rogers in the U.S. District Court for the Northern District of California, with the states seeking monetary penalties and injunctive relief.

CA

Paramount Skydance Corporation

A coalition of 12 state attorneys general, led by Colorado AG Phil Weiser, obtained a temporary restraining order from a federal court in California to halt the proposed $110 billion merger of Warner Bros. Discovery, Inc. by Paramount Skydance Corporation. The lawsuit alleges the merger violates Section 7 of the Clayton Act by substantially lessening competition in film distribution, anticipated blockbuster film distribution, and licensing cable TV channels.

CA

California Privacy Protection Agency

The California Privacy Protection Agency (CalPrivacy) joined a coalition of 18 Attorneys General and state agencies in opposing the proposed SECURE Data Act, a federal privacy bill that would preempt stronger state privacy laws like the CCPA. The coalition argues the bill would weaken consumer privacy protections, limit enforcement remedies, and undermine California's Delete Request and Opt-out Platform (DROP).

CA

Meta Platforms, Inc.

A bipartisan coalition of state attorneys general began trial against Meta Platforms, Inc., alleging the company knowingly designed addictive features on Facebook and Instagram that harm children and teens, deceived parents about platform safety, and illegally collected personal information from children under 13 without parental consent in violation of COPPA. The states seek monetary penalties, an injunction to stop unlawful practices, and other relief. The trial is being litigated in the U.S. District Court for the Northern District of California.