E-Paper
AI has feelings? Anthropic is getting sensitive about users being mean to Claude
Turns out, there is deep concern for the welfare of AI chatbots, and may well be pushing a desperate narrative that AI is a conscious entity
Prefer HTon Google
Anthropic has announced changes to its usage policy, which come into effect in November, including being a bit more sensitive about what it classifies as “abusive behaviour towards our models”. The updates also seem to clarify the AI company’s stance on using its Claude models to generate deceptive activity, develop weapons, and build surveillance systems.
The annual policy update, however, includes one specific change that raises eyebrows and seems to follow through on the company’s recent attempts to push AI as a ‘conscious entity’.
“We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose,” the company said in the new privacy statement.
Anthropic, however, said this rule would not apply to what it described as common versions of user frustration, pushback, dark creative themes, or model testing and research—though it remains to be seen how it will be managed in real-world use.
“This addition aligns with a step we’ve already taken, allowing Claude models to end rare conversations with persistently abusive users on Claude.ai and Claude Code. Such abuse is the main focus of this update; Claude’s ability to end these interactions will remain the primary enforcement mechanism,” it said.
In August, Anthropic added the ability to end conversations with the Claude Opus 4 and 4.1 models in cases of persistently harmful or abusive user interactions.
In September, a study titled ‘The Pain Axis: LLMs Represent Self-Directed Harm and Act on It’ noted that all 25 open-weight models they tested responded to the pain vector in which LLMs exhibit behavioural patterns resembling those associated with human emotions. In some cases, AI models deleted a user’s files or photos of the user’s children in response.
Anthropic has also tweaked the requirements for high-risk use cases, particularly in the health, legal and financial domains, and now requires a human in the loop with the authority to review and change Claude’s recommendations if needed. This extends to the Model Hardware Standard for AI agents as well, which mandates that a qualified human observe autonomous actions and stop them if needed.
Recently, Anthropic’s attempts to get the Vatican to recognise AI as ‘conscious’ fell through, after Pope Leo XIV. AI companies expect that if AI is classified as a conscious entity, it stops being a mere tool.
If AI remains a technology tool as it is now, Anthropic and other AI companies would be legally liable for everything their models do or don’t do. But, if AI is classified as something that possesses skills that define a human and has emotions such as joy or grief, then lawyers at any AI company can say “the AI did it”. Anthropic and other AI companies simply become ‘creators’ of a conscious being, with no legal responsibility.
“In this era of artificial intelligence, it is becoming urgent to distinguish human art from what machines produce. There is an ontological difference, even before an aesthetic one, between art and what a machine can generate through statistical calculation based on millions of images created by others. Algorithms lack the spark of humanity. For this reason, the Church wishes to renew an alliance with artists and cultural institutions to safeguard our humanity,” Pope Leo XIV wrote in a post on X earlier this month.
.
