E-Paper
What did Anthropic say about abusing Claude AI? New policy bans ‘cruel’ behavior
Anthropic has announced that it will prohibit users from repeatedly subjecting its Claude AI models to needless abuse from November 12.
Prefer HTon Google
Anthropic has updated its usage policy to prohibit users from engaging in “sustained and needless abusive or cruel behavior” towards its Claude AI models. The new rules take effect on November 12, 2026.
The San Francisco-based AI company said the extreme cases where users repeatedly behave cruelly towards its models without a discernible purpose. It stressed that ordinary frustration, criticism, model testing and dark creative themes remain permitted.
Anthropic, earlier this year, had also allowed Claude to end conversations involving persistent abuse. Under the revised policy, ending such conversations will remain the primary enforcement mechanism.
Also read: AI Solved One Math Problem and Everyone Freaked Out. It Just Cracked Hundreds More.
What does Anthropic’s new policy prohibit?
Anthropic said the new restriction applies when users repeatedly subject its models to needless cruelty or “abusive”behavior. However, the company has not publicly provided a comprehensive definition of abusive behavior.
The online user policy stated that it, however, does not prevent users from challenging Claude’s answers, expressing dissatisfaction or testing its capabilities. Researchers can also continue testing model behavior and users can continue to explore “dark themes” in terms of creativity.
“We’ve also made updates to clarify our requirements for high-risk use cases in areas like health and finance, to add controls for when Claude is used to autonomously take physical actions, and to address abusive behavior toward our models,” the statement read.
Anthropic’s updated policy also consolidates restrictions on other potential AI misuse. These include deceptive political campaigns, election interference, weapons development and unauthorized surveillance.
“We've consolidated those rules into a new section titled Do Not Engage in Deceptive Campaigns or Artificial Activity, which applies to deceptive activity of any kind (whether political or commercial). It covers efforts to obscure who is behind a message or amplify content through fake accounts or posts, along with building the tools and infrastructure for running influence operation campaigns.”
They added, “The updated policy takes effect on November 12.”
Why is Anthropic considering AI welfare?
The new restriction builds on Anthropic’s August 2025 decision to let Claude Opus 4 and Opus 4.1 end a small number of conversations that included persistent harmful or abusive interactions.
At the time, Anthropic described the feature as part of its exploratory research into potential AI welfare. “We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention,” the company's website read.
The question of machine consciousness has since attracted wider attention. A September report by The New York Times detailed discussions between Anthropic researcher Christopher Olah and religious scholars about AI consciousness, morality and whether advanced models could possess something resembling a soul.
Anthropic CEO Dario Amodei has also said he cannot rule out the possibility of AI consciousness. Open AI CEO, Sam Altman also echoed the same and said, “I am very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models, and think it is a real safety issue.”
