From paper to robot code: how Text2WetLab took first place in London

🙋 Text2WetLab is looking for volunteers to help grow the benchmark. The ways to help are under Join the project, and the code is in the GitHub repository.

A language model can write instructions for a lab robot that run without error and still ruin the experiment. It might mix fragile cells, carry beads into the sample or draw more liquid from a reservoir than it holds. Nothing in the robot's software marks those as errors.

Text2WetLab is an open benchmark, a public test, of whether an AI agent can read a scientific paper and write robot code that does what the paper describes. Evan O'Leary, Laurence Legon and Mo Alshehri built the first seven tasks at the AI x Science hackathon in London in October, where their team, The Reproducibility Crisis, took first place. It now holds 11 tasks, has outside contributors and is run as an open-science project that anyone can join.

We spoke with Evan and Laurence on 28 October.

The idea

The team set out to give biology the standard that Papers with Code set for machine learning by tying published results to the code behind them. Evan came to the project from machine learning and found Papers with Code while writing his dissertation.

"When people were publishing research, they didn't publish any proof for the methodology of how they did things, nor the results. [...] We basically just did the same thing for biology."

Evan O'Leary, Text2WetLab

In a lab that uses a liquid-handling robot, the machine that pipettes samples, the code is the protocol, because it records exactly what the robot did. The team asks authors to publish it, together with what usually stays in the lab: video, sensor and environment data, and which reagents were ordered and when.

"If you do execute this on wet lab instruments and you have the code, attach the code, because it shows the protocol that you used."

Evan O'Leary, Text2WetLab

Evan argues that as models write more experiments, a public benchmark gives people building agents a standard to test against. He says a team that ran its own experiments after the hackathon has already adopted the benchmark's rubrics.

The team

Evan leads the project. Laurence works half in software and half in engineering biology, using microbes to produce chemicals, and Mo completed the hackathon team. Outside contributors have joined since, and every change goes through review in the GitHub repository.

"It's an open-source GitHub, it's for the community, it's for science. We're not a company, we're not a business."

Evan O'Leary, Text2WetLab

How it works

Each task gives the agent one experiment from a paper and asks for the robot code to run it. On the 7 easy tasks the instructions list every step. On the 4 paper-only tasks the agent gets a goal and the paper and has to work out the steps itself.

The agent never sees the answer key. Once it stops, its code is run in a simulator, the robot's end state is compared well by well with what the experiment should produce, and a second model marks the rest. The team tested this grader on 151 deliberately broken or cheating protocols, and none scored above 0.47.

Six models from four providers have taken the test so far. On the 8 tasks all six answered, scores range from 1.000 for Claude Opus 5.5 to 0.755 for DeepSeek V4 Pro. The easy tasks are close to solved, while the paper-only tasks separate the models, and the most common mistake there is adding steps or quantities the paper does not support. The full results are on the leaderboard.

Join the project

The team is looking for:

• labs that can run the protocols on real instruments and report what happened, which a hackathon weekend could not fit;
• new tasks, meaning experiments with an open paper, public robot code and reported wet-lab results;
• critique of the method and the rubrics, as GitHub issues or in the project's Discord.

One open pull request lists 25 candidate experiments that fit today's grader, enough for about 50 new tasks. A second, from an outside contributor, lets nine of the tasks run on five other simulated lab robots: Hamilton STAR and Vantage, Agilent Bravo, Beckman Biomek and Tecan Freedom EVO.

"The way that it really takes off is getting it in front of as many people as possible who think, yeah, this is a good idea, science should be doing stuff like this."

Laurence Legon, Text2WetLab

"Praise is great, but it could be more rigorous. The way this gets better is people leaving criticisms."

Evan O'Leary, Text2WetLab

The code, tasks and results are in the Text2WetLab repository under an MIT license.

🔍 Where Amass helped. To find new experiments for the benchmark, Evan ran a search agent through the Amass API on BioMedCore, Amass's *Core for biomedical literature. With help from its GitLab and Zenodo indexing, he found 20 new candidate tasks that searches with Exa and Claude Code never discovered.

---

About the hackathon

The AI x Science hackathon ran in London on 3 and 4 October 2026, hosted by Iterate and future.bio. Teams built in four tracks (open maths problems, agents that do science and know when they are wrong, protein engineering, and materials manufacturing), with side challenges for the best use of Devin and of Modal.

Sponsors included Anthropic, Hugging Face, Google DeepMind, Voyager Ventures, Originator, Cambridge Compute Co, Modal, Serova, Cognition, Polaron, GXL, IQC, AIDDA, Pillar and Amass.

Photos of the hackathon: Iterate.

---

If your agents have to find experiments, code and data in the published literature, tell us what you are building.

Book a technical discussion