We put Anthropic's Claude-optimized protein models on our production cluster overnight

As developers of Dyno's AI model platform we have a strong interest in any advancement in protein structure modeling that let our users design real therapeutic molecules easier, faster and more safely. So when we received a ping from our frontier lab collaborators on the evening of Thursday, September 17 that researchers had released a set of Claude-optimized protein structure models, we knew users would expect validation of the claims in the report. By the next morning, Dyno's AI agents had added the optimized models into our AI platform, called Dyno Phi (about 600M tokens well-spent).

Here's what we learned.

Modern protein models are becoming a foundational part of how researchers design new medicine. These models predict how proteins will fold or interact. They can generate new proteins with desired interactions or properties. But these models are computationally expensive: a drug-design campaign will run these models thousands or millions of times to explore possible candidates. Making them several times faster means we can test more ideas and iterate faster, which reduces the cost of designing better medicines.

AI has made an important class of tools dramatically faster. We tested whether those improvements actually translate from research benchmarking into our production drug-design workflows, and are reporting what we found.

The Claude optimizations are real but with notable differences on production systems

Anthropic recently released optimized packages for a large set of open-source biomolecular models. Each package can expose up to three optimized modes:

• exact, which aims to keep the original outputs bitwise-identical
• fast, which allows small numerical differences for more speed
• big, which trades some runtime for lower peak memory.

The Claude optimization took just under four weeks, with claimed speed ups across each model reaching roughly 4x on average.

The point of this independent benchmarking is not to refute Anthropic's numbers. Their analysis is rigorous, and we see genuine improvements across key model families. Their benchmark isolates the model: exclusive H100, dedicated host, a tuned baseline, warm process, forward-call clock, geometric mean over a 200 to 1,400 token ladder. That is the right way to measure a kernel contribution.

Our analysis uses a production cluster: dynamically scheduled Kubernetes pods, NFS-mounted weights with a cold page cache, two GPU families (A100-40GB and H100), and a driver and CUDA generation that predate the kit's pins. Our jobs start, do one unit of work, and exit.

So the comparison we are making is admittedly apples-to-oranges, and deliberately so: do these optimizations hold when moved from a research VM to a production system, where version constraints exist, GPUs are dynamically allocated, and the use case involves distributed protein design pipelines? And is it worth it for our users to incorporate these models into their existing workflows?

The answers are yes, with caveats worth understanding.

On a real multi-tenant Kubernetes cluster, measured end-to-end, the kits deliver a 1.05x to 4.14x speed increase depending on model and GPU family. The gap between our numbers and the published figures lies largely in fixed costs that a production scheduler cannot amortize. For inverse folding models like ESM-IF1 and ProteinMPNN the speedup was close to as claimed. For Boltz-2, ColabFold and OpenDDE the improvements were more marginal, and in some cases represented a regression from our Dyno Phi native implementation.

Reading our numbers

The single biggest difference between our results and the headline figures is what the clock measures. Anthropic define a different clock per model family. For inverse folding (ESM-IF1 and ProteinMPNN), they report end-to-end wall, process launch to the last output written. That is the same clock we use, so those numbers are directly comparable.

For the structure prediction models (ColabFold, Boltz-2, OpenDDE), the headline is time inside the forward call at steady state, excluding warm-up, compilation, featurization and output writing. To their credit, the report also publishes a whole-pass wall-clock companion and says plainly that it is lower. Those companion figures are what our measurements are set against. This is important because in some cases the optimizations trade start-up time for inference time in a way that looks great in a benchmark but reduces the optimization's effectiveness in a real production system.

Against the matched clock the picture is mixed rather than uniformly short. OpenDDE lands above Anthropic's published band on both modes (1.27x and 1.55x against their 1.16x and 1.15x at comparable input size). Boltz-2 exact also clears it, at 1.92x against a 1.41 to 1.86x band, while Boltz-2 fast sits just under at 2.03x against 2.14 to 3.48x. ColabFold is the one that falls clearly short on both modes. (We also tested additional protein design models such as RFDiffusion but with marginal speedups that were largely confirmed; this summary touches on where interesting differences exist.)

To deep dive a key sequence generation tool, we swept ProteinMPNN across batch sizes, changing nothing but how much work each invocation does. The same code goes from 1.00x at 2 model calls to 5.55x at 128. The kernel did not change. The only variable is how much work there is to amortize the fixed cost against. The implication for anyone running these models in a pipeline is direct:

• Optimization payoff on ephemeral infrastructure is a function of batch size, not of the kernel.
• A stage that does one unit of work per pod can get nothing from a 5x kernel.
• The lever available to us is scheduling and batching, not more kernels.

Where the kit is slower on our system

To their credit, Anthropic reported a regression of their own: OpenDDE big drops to 0.63x at 1,400 tokens once the pair-offload path engages, trading forward-call time for a 52% memory cut. We found four places the kit cost us time:

• ProteinMPNN exact at small batch is a loss. At two model calls it measured 0.86x in the first sweep and 1.00x in the re-run. Its ~6 s startup cost, which matches the report's own figure, is not repaid until roughly 8 to 32 calls. Dyno Phi stages that design a handful of sequences per pod get nothing, or lose slightly.
• Boltz-2 fast costs more on its first call than stock does. 117 s cold against 92 s for off. The compile has to be paid before fast wins, so a single-shot pod is a net loss.
• Boltz-2 exact is not predictably safe on A100. Because of the way Claude decided to optimize the Boltz-2 fused pair transition, it became highly sensitive to the size of the input, sometimes failing on jobs that the default implementation didn't. Whether a job is safe depends on its token count, which is not a property you want to reason about per input.
• ColabFold exact gets worse as inputs grow. At 199 tokens, exact is 1.05x against Anthropic's published whole-pass 1.91 to 2.18x. At 4,778 residues it is 0.87x, 1,062 s against stock's 924 s, running at 78.69 GiB with zero rematerialization headroom. Batching cannot rescue this one, because it grows with input size rather than amortizing away.

What we are doing about it

The results point at scheduling rather than kernels, so that is where the work goes:

• Batching. The ProteinMPNN curve says a stage doing one unit of work per pod captures none of a 5x kernel. Raising work-per-invocation is the highest-leverage change available to us, and it requires no new kernels.
• Compile cache persistence. Most of the fixed cost in the faded bars is compilation. Persisting the JAX cache to shared storage amortizes it across pods instead of across calls within a pod.
• Per-stage mode selection. exact is bitwise-safe and worth promoting wherever it is not a regression. This excludes ColabFold at scale and ProteinMPNN at small batch. This manifests as per-stage decisions we are helping our users understand and incorporate into their workflows.
• Big mode for inputs that require multi-GPU structure prediction or design.

What uniquely enabled this result

Dyno's engineering team hosts a unique agentic development platform capable of long-running, multi-step implementations. This system runs in secure sandbox VMs and coordinates to complete large tasks, quickly, rigorously and safely. Pointing it at the Anthropic blog and our expectations for the benchmarking results led to an initial 8-hour implementation phase, followed by a more open-ended exploration and benchmarking phase. Further, we already had every model in the Anthropic catalog deployed on Dyno Phi in a dynamically scaled production system, meaning we could demonstrate a fair benchmarking within hours of the implementation landing.

Lessons learned

The broader lesson is one that generalizes past these particular kits. A kernel speedup is measured in a regime where the fixed costs have been engineered away. Production pipelines are frequently the opposite regime, and the same optimization can be worth 5x or nothing at all depending on how much work you actually need to accomplish using these models for real therapeutic outcomes. Measuring that is cheap, and it changes what you build next.

Interested in trying the Dyno Phi model platform? Contact david.levy-booth@dynotx.com.

Sources

Accelerating open-source biomolecular models with Claude, the Anthropic blog post and its v3 technical report (139 pp.). All Phi measurements were taken on our production GKE cluster between September 18 and 20, 2026.

Hardware and environment