Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm
Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm: a classifier on Claude.ai displays a banner with links to crisis lines, and the model is trained using reinforcement learning with human preferences.
Anthropic has described the measures it has implemented in the Claude product for safely handling conversations about suicide and self-harm. A classifier – a smaller AI model that monitors active conversations and detects moments when additional resources might be needed, including fictional scenarios focused on suicide or self-harm – runs on Claude.ai. When detected, a banner is displayed directing the user to professional help, crisis lines, or country-specific resources. These resources are supplied by ThroughLine, a crisis support provider with a vetted network in 170+ countries (e.g., the 988 line in the US and Canada, Samaritans in the UK, Life Link in Japan). Anthropic is also working with the International Association for Suicide Prevention (IASP) on recommendations for training and product design.
The model's behavior is shaped by a combination of a publicly available system prompt and reinforcement learning, in which the model is rewarded for appropriate responses based on human preference data and internally created data about the model's desired character.
According to Anthropic, the Claude Opus 4.5, Sonnet 4.5, and Haiku 4.5 models respond appropriately to clearly risky messages in 98.6%, 98.7%, and 99.3% of cases respectively (the earlier Opus 4.1 model achieved 97.2%), with a low rate of refusing harmless queries. In evaluations of longer, multi-turn conversations – conducted without a system prompt – the Opus 4.5 and Sonnet 4.5 models achieved an appropriate response in 86% and 78% of scenarios respectively, compared to 56% for the Opus 4.1 model. For details on reducing sycophancy and the rest of the data, see the source article.
Why it matters
For regular users, it is important to know that Claude.ai now actively recognizes sensitive conversations and directs them to real professional help, while still remaining only a supplement, not a substitute for professional care. For companies deploying AI chatbots in contact with users, this is a documented example of measures (classifier, system prompt, evaluation) and concrete success rate figures that can be used in their own risk and compliance assessment.
Two audiences, two different impacts
What this means
For individuals
Users of Claude.ai who mention self-harm or suicidal thoughts in a conversation may now encounter a banner with links to crisis lines and professionals; however, Claude remains only a supplement, not a substitute for professional care.
For a business
Companies deploying the Claude product or similar models in contact with end users can use the published evaluation data and the described mechanism (classifier, banner, system prompt) as a reference when assessing risks and compliance in sensitive conversations.
Risks and complianceCheck the original
Event sources
only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.