Experimental reproducibility statistic. Choosing reliable protocols for new experiments is a problem laboratories routinely face. Experimental practices differ immensely across laboratories, and precise details of these practices may be lost or forgotten as skilled members leave the lab. Such fragmentation in protocol methods and their documentation often hampers scientific progress. Indeed, there are few well-defined protocols that are generally agreed upon by the scientific community, in part due to the lack of a system that measures a protocol’s success. In turn, the lack of commonly accepted protocols and inadequate documentation affects experimental reproducibility through method inconsistencies across laboratories.

To attempt to address these problems, we set out to build a database that integrates a crowdsourced ratings and comments system to clearly document, rate, elaborate, review, and organize variants of experimental protocols. Before starting, we designed a survey to poll a range of scientific researchers on their experiences trying new protocols. We found that among a diverse and experienced set of respondents, every single scientist has struggled with replicating protocols from other experimenters, with >50% of respondents having difficulty more than 25% of the time.


Additionally, our survey identified that unclear language and missing steps of protocols as the greatest contributors to the irreproducibility of protocols. Furthermore, 100% of respondents indicated they would use a database like this to browse and download protocols, and over 85% indicated they would upload and maintain their own protocols if such a site existed. With these data and interest in hand, we set out to build ProtoCat.