Skip to content
context Security

Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm

only one source so far

Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm: a classifier on Claude.ai displays a banner with links to crisis lines, and the model is trained using reinforcement learning with human preferences.

Anthropic has described the measures it has implemented in the Claude product for safely handling conversations about suicide and self-harm. A classifier – a smaller AI model that monitors active conversations and detects moments when additional resources might be needed, including fictional scenarios focused on suicide or self-harm – runs on Claude.ai. When detected, a banner is displayed directing the user to professional help, crisis lines, or country-specific resources. These resources are supplied by ThroughLine, a crisis support provider with a vetted network in 170+ countries (e.g., the 988 line in the US and Canada, Samaritans in the UK, Life Link in Japan). Anthropic is also working with the International Association for Suicide Prevention (IASP) on recommendations for training and product design.

The model's behavior is shaped by a combination of a publicly available system prompt and reinforcement learning, in which the model is rewarded for appropriate responses based on human preference data and internally created data about the model's desired character.

According to Anthropic, the Claude Opus 4.5, Sonnet 4.5, and Haiku 4.5 models respond appropriately to clearly risky messages in 98.6%, 98.7%, and 99.3% of cases respectively (the earlier Opus 4.1 model achieved 97.2%), with a low rate of refusing harmless queries. In evaluations of longer, multi-turn conversations – conducted without a system prompt – the Opus 4.5 and Sonnet 4.5 models achieved an appropriate response in 86% and 78% of scenarios respectively, compared to 56% for the Opus 4.1 model. For details on reducing sycophancy and the rest of the data, see the source article.

What changed

Why it matters

For regular users, it is important to know that Claude.ai now actively recognizes sensitive conversations and directs them to real professional help, while still remaining only a supplement, not a substitute for professional care. For companies deploying AI chatbots in contact with users, this is a documented example of measures (classifier, system prompt, evaluation) and concrete success rate figures that can be used in their own risk and compliance assessment.

Two audiences, two different impacts

What this means

01

For individuals

Users of Claude.ai who mention self-harm or suicidal thoughts in a conversation may now encounter a banner with links to crisis lines and professionals; however, Claude remains only a supplement, not a substitute for professional care.

What to do In the case of serious mental health difficulties, whether your own or someone else's, seek professional help or a crisis line, and do not rely on an AI chatbot as a substitute for care.
More practical updates →
02

For a business

Companies deploying the Claude product or similar models in contact with end users can use the published evaluation data and the described mechanism (classifier, banner, system prompt) as a reference when assessing risks and compliance in sensitive conversations.

Risks and compliance
What to decide Verify whether the deployment of the Claude product in conversations with end users complies with the safety measures and system prompt described by Anthropic, especially in sensitive areas such as healthcare or support…
More business impacts →
Anthropic security Claude safeguards suicide prevention sycophancy

Check the original

Event sources

only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.

1
Anthropic News primary source · first detected Protecting the wellbeing of our users