Response to Anthropic’s Update on Independent Evaluators
Yesterday, Anthropic published an update on its work with independent evaluators that mentions SecureBio. In this update, Anthropic defined a tiered system of evaluator access, and described SecureBio as being “in the first tier” of this system. This description did not reflect the agreement we have with Anthropic, and Anthropic has since revised it. For clarity, we describe our collaboration with Anthropic below, as well as our approach to independent evaluations more broadly.
SecureBio has not agreed to any formal tier designation with Anthropic. We have one ongoing assessment of Anthropic’s work, which is an independent review of the chemical and biological weapons chapter of Anthropic’s August 2026 Risk Report. Under the terms of this assessment, we may publish our results and methodology without Anthropic’s approval. Anthropic may designate sensitive material for redaction at its discretion; our public report will state whether any redactions were made and whether they affected our findings. This meaningfully differs from Anthropic’s description of a “first tier” engagement, where findings may only be published with Anthropic’s approval.
In general, we partner with AI developers on assessments for a range of purposes. We may sometimes accept engagements with limited publication rights, such as early-stage or pilot work where the primary goal is to inform and strengthen a model developer’s safeguards against biological misuse, rather than to provide independent accountability. We may also withhold certain details where responsible disclosure practices require it, such as a vulnerability in a developer’s biological safeguards where disclosure could lead to real-world harm.
Our independent assessments are guided by our principles and practices for model assessment and by the AEF-1 standard, which we co-authored as part of the AI Evaluator Forum, rather than by any model developer’s access framework. We have recently adopted the practice of including an AEF-1 checklist in each public report, alongside a checklist against our own principles, noting any criteria the assessment did not meet.
Best practices and standards for independent evaluation are still evolving, and our practices will develop with them. In September, SecureBio and other members of the AI Evaluator Forum published an open letter outlining minimum conditions for credible embedded evaluations, since signed by more than 200 researchers and practitioners. We will continue to push for independence and publication rights for third-party evaluators.