Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm
Anthropic has described the safety measures in the Claude product for conversations about suicide and self-harm: a classifier on Claude.ai displays a banner with links to crisis lines, and the model is trained using reinforcement learning with human preferences.
Companies deploying the Claude product or similar models in contact with end users can use the published evaluation data and the described mechanism (classifier, banner, system prompt) as a reference when assessing risks and compliance in sensitive conversations.