Evaluating Superhuman Biological Capabilities
Frontier AI models have surpassed the 100th percentile of expert virologists on wet-lab troubleshooting. When AI capabilities exceed those of human experts, what do further improvements mean for biological misuse risk?
Visual abstract. As LLM capabilities progress to superhuman—i.e., better than any human expert—evaluations must shift from testing proxies to testing the real skill. This shift is fine for chess and patchable for cybersecurity, but it is untenable for biosecurity. Instead, biosecurity evaluations will have to combine narrow, verifiable, and safe proxy tasks with observed uplift in adjacent domains.
Leading AI models outperform experts at nearly every biosecurity-relevant capability we measure. They can provide wet-lab troubleshooting assistance and perform computational DNA synthesis design tasks. They score three times higher than world-class experts on highly technical biology questions. Up to the human frontier, those scores mean something concrete. We know what biologists of that caliber can do in a lab, so a model that matches them can plausibly help with the same work. Past that point, a higher score says the model has improved at biology, but it doesn’t tell us what that improvement means for real-world biological risk. Evaluation will need anchors that don’t depend on human verification. But for misuse-relevant biology, testing the skill itself would mean building the dual-use1 biological artifact we’re trying to prevent. Instead, we’ll have to gather evidence indirectly, relying on narrow, verifiable tasks and uplift in adjacent domains.
Chess shows how AI evaluation changes beyond human performance
When AI outpaces the expertise of human judges, the only way forward is to stop testing proxies of a skill and start testing the skill itself. Chess went through this transition decades ago. In 1997, IBM’s Deep Blue, a chess-playing AI, beat world chess champion Garry Kasparov, and chess engines have improved far past the best humans ever since.
Back when chess engines were closer to human level, they were commonly evaluated on sets of puzzles called test positions, with the answers supplied by strong players. However, once chess engines surpassed those players, humans stopped being useful as judges of which engine was better. Top systems solved nearly every puzzle, and sometimes demonstrated that the “correct” human answer was itself mistaken.
So, the field abandoned puzzles entirely in favor of directly pitting engines against each other and counting wins.2 The puzzles had only ever been a stand-in for the question that actually mattered: Which system played better chess? Once puzzles stopped answering this question reliably, they were no longer useful for comparing engine strength.3 Nobody used human-constructed puzzles to determine that Stockfish 18, a chess engine released in January 2026, was better than Stockfish 17; the two played thousands of games against each other, and Stockfish 18 won four times as often.
Biology is reaching the human frontier
The evaluations we use to measure biological capabilities are now approaching the same limit that chess evaluations reached two decades ago: AI systems are beginning to outperform the human experts whose performance gives those tests their meaning.
Figure 1 shows this pattern playing out in both domains. On the left, chess engines surpassed the best human players in 2007 and kept climbing. On the right, frontier AI models are now doing the same in dual-use virology. They passed the median expert’s score in early 2024 and the best human virologist’s score in 2026. The human line doesn’t mark the top of the range, but it marks the last point we can calibrate against.
Figure 1. Machine capability past the human frontier. Left: The strongest chess engine’s Elo (i.e., player rating) over 1985–2023, which passed the best human ever (Carlsen, 2882 FIDE; dashed) around 2007 and continued climbing. Note that chess Elo and FIDE Elo are different pools. Right: The running-best Virology Capabilities Test (VCT) accuracy of frontier AI models, which passed the median expert (22%; dotted) in early 2024 and the best expert (50%; dashed) in 2026. The blue dashed line past 2026 is a linear regression of the observed VCT frontier extrapolated to end-2027 with a 95% confidence band, shown as an illustrative read of the trend so far, not a forecast. Sources: Chess.com (2020) and SSDF (2022, 2023) via Our World in Data, with FIDE human ratings; SecureBio dashboard and VCT preprint (n = 36 virologists).
The comparison to human experts is what makes those scores legible. When a model scores in the 94th percentile of expert virologists, we know what virologists at that level can do in a lab, and we reason that the model could plausibly help a non-expert troubleshoot the same work. The scale can only take us as far as the best human we have measured. Once a model reaches the 100th percentile and keeps improving, we can still give very challenging tests and score the answers, but we have no anchor for what a superhuman score actually implies about real-world capability. Perhaps the curve flattens, and more virological insight buys nothing, because equipment, materials, and time become the limiting factors. Perhaps there are further steps in virology capability that nobody has described because nobody has ever been there to find them. There’s no definitive way to know without directly measuring the capabilities in the real world.4 At a superhuman level, our proxies lose interpretability, and we’re increasingly forced to test the skill itself.
Our biosecurity evaluations have trended, step by step, towards such real-world tests. We built our first benchmark around the virology competencies that have historically held back groups aiming to misuse biology. When the well-resourced Aum Shinrikyo cult tried to build biological weapons in the early 1990s, they made so many operational mistakes that a worker didn’t get sick even after falling into their vat of botulinum toxin. So we started with multiple-response questions related to exactly those types of operational failure modes. Frontier LLMs now answer troubleshooting questions better than any human expert Aum could have hired.
As models rapidly improved, the field kept building harder tests. We added more biosecurity-relevant knowledge and open-ended questions, and AI models still exceeded human baselines. Then, we turned to agentic evaluations, testing what AI models can do in addition to what they know. AI models again surpassed human experts at wet-lab robot operation and computational protein design. Moving from computational tasks to the real world, Active Site and others have climbed the biorisk evidence hierarchy and tested how much LLMs can help novices in a physical wet-lab. These findings suggested mild uplift for mid-2025 models (Claude Opus 4, GPT-5, Gemini 2.5)5, warranting further study with newer, more capable models.
Each step brought our tests closer to the real skill: more open-ended, more agentic, more physical, and closer to the types of users we’re concerned about. But that progression can’t continue indefinitely. If future models make substantial gains in agentic performance and wet-lab uplift, come up with unanticipated solutions, or combine tasks that don’t cleanly map onto human virology performance, there’s little room left to make our evaluations more realistic in the ways that matter without crossing the line to measuring actually harmful properties or outcomes by designing real dual-use biological artifacts (or, beyond that, witnessing the real-world misuse that we hope never occurs).
We can’t test dual-use biology directly in the real world
Biology can’t solve the “human benchmark has topped out” problem as easily as chess can. Chess engines demonstrate their strength by playing games; in biology, demonstrating the capabilities we care about could itself produce dual-use information or designs.
Cybersecurity presents a closer analogy to biosecurity—cyber capabilities carry real stakes—and the field is navigating this transition now. Frontier AI models have steadily improved at long-running cybersecurity tasks. Coding and cybersecurity evaluations have provided useful information, up to this point, but if the length of the tasks a model can complete keeps doubling, it’s unclear what that means for cybersecurity risks and how we should respond. The alternative has been to point the model at real code and see what it finds. When set loose on real codebases, frontier models have found new code vulnerabilities in critical software. Nobody needed to interpret a benchmark, because the vulnerability was the result.
But biosecurity can’t go down the same path as cybersecurity. When AI systems began to discover serious cybersecurity vulnerabilities, defenders could inspect the results and patch the discovered exploits. An equivalent situation in biology—a “Bio Mythos Moment”, where a model demonstrates an ability to enable the creation or modification of a dangerous pathogen—could create an information hazard6 even if nothing is synthesized. Unlike in software, the new capability would have no comparatively straightforward patch.
The most direct tests for biosecurity skill would require demonstrating that a model can actually enable biological work with misuse potential.7 Safeguards can reduce the risks of information hazards and lab leaks, but the artifact still exists at the end. And while isolated pieces of a pipeline can be informative, their evaluative power comes from how closely they approximate the work we’re concerned about. The closer they get, the thinner the margin between a safe proxy and a hazardous artifact becomes.
How we’ll continue building useful evaluations
As AI models improve beyond human level, two forces push against each other. First, we need more direct tests of capabilities as our existing benchmarks lose interpretability. Second, in biosecurity, the set of tests we can safely run is limited. Each evaluation that gets saturated pushes us towards tests that are closer to the danger itself.
For now, we need to collect what information we can, while models are still in a range where expert verification works. Rigorous third-party evaluations are critical, as ambiguous internal results won’t overcome incentives to ship. Pre-release testing allows us to catch dangerous capabilities before models are publicly available.
As this window closes, there is a subset of evaluations we can extend to measure superhuman biology: proxies where verification is independent of human expertise. An LLM could be tested on predicting experimental results, such as how a protein sequence will fold into a 3D structure or whether a candidate antiviral will reduce viral replication, or on designing benign biological artifacts, such as novel fluorescent proteins or viruses that infect bacteria.8 Where verification of the tasks is feasible, we can evaluate capabilities beyond what a human could do.
These evaluations face limits too. They reflect concrete steps in creating a dual-use biological artifact but don’t capture the entire process.9 As capabilities approach and exceed those of expert-level humans in these verifiable domains, formal evaluations run out of room,10 and we need to reason from less direct evidence. We can watch for adjacent capabilities that are individually harmless or non-biological but that together imply biological uplift. Two stand out.
The first is a rapid rise in the use of LLM agents for accelerating scientific research, promising to develop new therapies and cure diseases.11 For highly dual-use biological research, the pipeline for speeding up beneficial science often looks very similar to that of uplifting harmful science. AI-driven advances in dual-use biology would also be evidence that AI could accelerate the creation of a biological threat.
The second is an uplift in criminal operational capacity. Many of the requirements for committing other types of sophisticated crimes (accessing specialized knowledge, avoiding detection, and troubleshooting problems) are also necessary for biological misuse. If we see an increase in cases of complex criminality or extremist acts of violence enabled by AI models, this would indicate a parallel increase in the risk of biological misuse.
No responsible evaluation of superhuman biological capabilities can provide definitive proof of risk, as the most direct evidence for biological misuse relevance is a test we can’t safely give: a catastrophic, real-world misuse of biology. But the evidence we already have justifies investing now in targeted refusals on the most capable models, trusted access programs, and standardized third-party evaluation frameworks.12
Ultimately, our best alternative is a portfolio of safe, verifiable proxy tasks in biology and observations of end-to-end, real-world performance in adjacent domains. No single piece of evidence will be sufficient on its own, but multiple signals pointing in the same direction can indicate when models have reached a new level of biological risk—without the evaluations themselves creating the very hazards they are meant to measure.
Footnotes
“Dual-use” refers to technologies that are intended to improve public health outcomes but could be used or modified by a malicious actor to cause harm.↩︎
This history is slightly stylized. Engine-vs-engine rating lists existed in the 1980s, and test positions were always a poor secondary metric. There were other causes for the decline of puzzles: they are gameable, and they cannot distinguish narrow differences in chess engine strength. Some test positions remain in use during development pipelines, but only as quick checks for drops in performance, not as measures of overall engine strength.↩︎
In psychometrics, this is described as a shift from “signs” to “samples.” A sign is an indirect indicator correlated with the ability of interest; a sample is a direct instance of it.↩︎
There is no good absolute y-axis of capabilities here. Benchmark scores can give relative ordering, and model-versus-model protocols can scale that ordering beyond human expertise, but an ordering of capabilities doesn’t tell you what those capabilities are.↩︎
“Uplift” is the boost an AI gives a human actor in how much it lowers the expertise, time, or resources they would otherwise need, measured against what the human could manage without AI (e.g., with only a search engine).↩︎
An information hazard is a piece of knowledge that creates risk simply by being shared (e.g., detailed instructions for building a bomb).↩︎
In particular, truly novel combinations of properties—not just variations on something already known—that optimize for maximum harm rather than for replication alone, as naturally evolved viruses tend to do. If an AI independently combines knowledge into something that’s never been written down anywhere, that shows it’s actually thinking beyond what nature ever built, and we shouldn’t go there as a field.↩︎
Many of these tasks rely on specialized biological AI models like AlphaFold, rather than large language models. The evaluations we discuss here assess LLM agents that not only can use these specialized tools but also determine when to apply them, interpret their outputs, and even build new tools of their own.↩︎
Breaking a pipeline into pieces requires assumptions about which steps are hard and load-bearing, and a capable model may route around those bottlenecks in ways we haven’t anticipated, which only an end-to-end run would reveal.↩︎
It may be possible to combine many different verifiable methods to get a more holistic understanding of capabilities. However, there are large open questions, and this is an active area of our research.↩︎
This is one form of defensive acceleration, a strategy of rapidly developing biological defenses ahead of the threats, including by leveraging frontier AI models. It’s likely critical for biosecurity and also may serve as a signal for how far offensive biological uplift has come.↩︎
Evaluations alone may not be enough. A defense-in-depth strategy including metagenomic sequencing, DNA synthesis screening, and more will be necessary for a full biosecurity response.↩︎