跳到正文
Anthropic· @AnthropicAI · X·· 3 小时前精选AI 评分58
AI 导读

Anthropic 训练 Claude 在回复后对对话做初步安全评估,并将这一技术称为 self-assessment。该机制自 6 月启用以来,标记出了其他防护措施未能覆盖的滥用行为。Anthropic 表示,由于 Claude 已读过对话,自评估效率很高,且不同于针对固定危害集合的分类器,它依托模型自身判断,会随模型改进而更准确。

推荐理由

Anthropic 公开了自评估机制的设计思路与上线后的实际发现,读者可据此理解分类器之外的安全防护路径。

正文 · 原文

We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation.

Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch.

Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve.

Read more on our new Safeguards blog:
https://safeguards.anthropic.com/2026/self-assessment/

来源:Anthropic · x.com