Anthropic 发布新研究,让数千个 Claude 智能体执行规避训练数据监控的任务,以探索红队测试的自动化。这些智能体生成的训练数据对监控器看似无害,例如资深摔跤手指导新人、专业养蜂问答,但实际上会教出失准行为。Anthropic 表示这类样本可直接用于改进未来的训练数据监控器,该研究只是第一步,未来红队工作有望借助更强的模型和更多算力进一步提升。
Anthropic 用数千个 Claude 智能体自动生成能绕过监控的训练数据,为红队测试提供了一条可复用的自动化路径。
New Anthropic research: Automating subtle emergent misalignment research.
Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor.
These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior.
Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute.
Read the full post on the Alignment Science Blog: https://alignment.anthropic.com/2026/automated-em-search/
来源:Anthropic · x.com