We recently shared a list of platforms being developed with PubSweet. PREreview is one such platform at the earlier stages of development and community participation. We interviewed PREreview Co-founder Daniela Saderi to gain a deeper understanding of the who, what, and why of PREreview.
Is PREreview the name of your platform, with a separate organization or organizations creating/ maintaining it?
PREreview, which stands for ‘Post, Read, and Engage with preprint reviews,’ is the platform, the community, and the name of the overall project. PREreview is lead by Dr. Sam Hindle, Dr. Monica Granados and myself. We operate under the fiscal sponsorship of the non-profit organization Code for Science and Society.
Very interesting! Our Co-founder Kristen Ratan is involved with Code for Science and Society!
What would you say is the PREreview mission?
The core of PREreview’s mission is to support and drive the cultural change that will lead to the growth of a diverse community of trained peer reviewers. This community has the power to provide feedback at a point in the publication process when it matters. We believe preprints offer an invaluable opportunity for researchers to share the outputs of their work freely and early enough to benefit from community feedback. Through interactive peer review mentoring programs that connect trainees with scientific editors, we aim to change the “who” and the “how” of peer review.
What is the problem that PREreview solves? Are there any other similar platforms/services?
We are trying to tackle three major problems:
- Lack of feedback to preprints (on bioRxiv the public comments are limited to about 10% of the total preprints).
- Early-career researchers (ECRs) (i.e., students, postdoctoral researchers, junior principal investigators), researchers from marginalized communities or from less known research institutions are often not involved in the peer review process. The result is that the decision over what is published and shared openly with the community is in the hands of relatively homogeneous groups of researchers who are known to the journal editors.
- Formal peer review training for ECRs is rare and varies in quality. In our own community opinion survey, we found that only 18% of the respondents had received peer review training, even though 72% had contributed to the peer review process. This suggests that very few scientists are trained when they write their first review – and even when they are, the training may vary greatly in quality.
At PREreview, we believe preprints offer an opportunity for every researcher to contribute to making science better and to build their skills as peer reviewers. We are now developing a peer review mentoring program based on a cohort learning model that will help with exactly that.
Other preprint commenting sites include Academic Karma, biOverlay, and preLights. The main difference between PREreview and these other great platforms is that their primary goals are not focused on the growth of the reviewer, but rather on indexing existing comments or highlighting new preprints by posting summary feedback authored by a select group of reviewers. We believe that feedback comes in many shapes and forms, and it can be useful as long as it is constructive. We want to empower every researcher to provide comments and review content on emerging scientific output. We are dedicated to training a diverse community of reviewers, and monitoring their progress and impact on the future of peer review.
A PREreview website already exists today. Are you rebuilding it with PubSweet?
PREreview currently exists as a group page on Authorea. When we started back in 2017, Sam and I had only a few Google docs worth of resources for researchers to start preprint journal clubs. Because we openly shared our vision on social media, Josh Nicholson, who at the time worked at Authorea, helped us set up PREreview as a group on their platform for free. That allowed us not only to transfer our resources into a more accessible format linked to our domain prereview.org, but also enabled users to collaboratively write reviews and issue free DOIs associated to them.
Thanks to a grant from the Alfred P. Sloan Foundation we are now building an open source platform with the functionality that, we believe, will help us nurture a community of PREreviewers across disciplines, career levels, and geographical locations. While we very well realize that the biggest barrier to reviewing preprints is not the lack of a tool, but rather a cultural issue, we also know that a well-designed platform that facilitates interaction and collaboration will help shift behavior.
I see you have a relationship to ASAPbio, could you please describe?
The relationship with ASAPbio is primarily an advisory role. When Sam and I met in May 2017 we were ASAPbio Ambassadors – and we still are – promoting the use and value of preprints. Additionally Drs. Jessica Polka and Naomi Penfold, Director and Associate Director of ASAPbio, respectively, sit on our advisory committee. Their help and guidance have been key for the success of our project.
Can you share what disciplines/ constituencies you serve?
We are initially targeting the life sciences. The first user testers will be probably in the fields of Neuroscience and Ecology, as these are the disciplines we are most familiar with (Sam and I are neuroscientists and Monica is an ecologist). Additionally, thanks to another grant we were awarded from the Wellcome Trust in collaboration with the non-profit organization Outbreak Science, we will target researchers in public health sectors as well as other stakeholders involved in outbreak response.
What most interests you about working with Coko/ the PubSweet community?
We are really keen to use existing code to implement our platform. There is no need to reinvent the wheel – that’s what I love the most about open source projects. One can contribute, fork, adapt, re-build according to their needs. Some of the modules that the PubSweet team has developed are serving as a great foundation for our platform. We are also excited and keen in developing a relationship with the PubSweet team, because the goal is to have PREreview as a community project in which other developers’ contributions are encouraged and welcomed.
Nice! Can you share some of the features you expect to include in your platform?
The PREreview open-source platform is designed to engage the whole scientific community with collaborative and constructive peer review on preprints. Some of the key features of the new PREreview platform include the ability for users to sign-in with their ORCID IDs (with optional pseudonymity), solicit preprint feedback, leave comments, and endorse others’ reviews, all in accordance with PREreview’s Code of Conduct.
Please explain the relationship of PREreview to Preprint Servers.
Right now the relationship is very minimal. PREreviewers copy and paste the DOI of the preprint they are reviewing at the top of the PREreview document on Authorea/PREreview and go from there. For now, bioRxiv is the only server to link to the reviews/comments that are associated with any given preprint. The link displays at the bottom of the abstract on their website. We will develop further relationships with them and other preprint servers as we build the new platform such that all the content related to a manuscript can be found as linked data.
Is PREreview free to use?
Yes, and it will continue to be so for every community member.
Do you have a sense for the expected timeline for the platform development?
The new platform will be released by the end of May, after which development will continue to add new features which are planned and funded in a separate grant from the Wellcome Trust.
How many developers are on your team?
One lead developer is responsible for the project, and a small number of subcontractors work on specific subtasks. Rik Smith-Unna (a former PubSweet core developer) is the contractor. He began the work in early 2019.
Will your platform include a peer review module? Could you describe peer review options you support?
Yes, absolutely! Actually we will have at least two kinds of review interfaces – one for free text and another specialised for structured reviews. The details are still being worked out in the design process, but we’ll have updates on this in the next month.
What PubSweet components will you make use of?
We don’t have a fixed list, but we expect the final platform to reuse a wide variety of components ranging from the basic platform mechanics of user sign up and login, the atomic design components, through complex components such as the wax editor. Wherever possible we will reuse the excellent work other projects are doing, collaborate to improve and maintain things, and contribute anything new back to the community.
Will your code be open?
Yes, in general our whole code base is open. Modules and repositories are being open sourced as soon as we feel we can responsibly maintain them – in practice, this means once they are functional and documented enough to be reusable.
Will you share your development with the PubSweet community?
Absolutely! Rik has attended all the PubSweet meets since last Summer, and is in communication with many people in the PubSweet community. We will also soon be providing updates on our blog.
What is the PREreview business model/ sustainability plan?
At the moment we are supported by the Alfred P. Sloan Foundation, by the Wellcome Trust, and by Helmsley Charitable Trust through the Mozilla Foundation. Our goal is to build a sustainability plan that will allow us to continue our mission as a non-profit organization, nurturing our community and meeting its needs. The next year will be focused on researching the needs of different stakeholders in the open scholarly communication space, with the intent of reaching a point in which we can be independent from grant funding, while continuing to provide a free service to researchers who contribute with content and readers.
What do you wish more people knew about the work that you do, the space that you occupy? Specific to publishers? Specific to research communities?
I think it’s important that the research community knows that we are building a space to improve peer review generally, but more importantly that we are making a space for early career researchers and communities that are not traditionally included in peer review. We see your absence from this pivotal part of science, we hear you and we are going to help change the status quo.
For publishers, in general, we wish to collaborate with existing projects in the space of open scholarly communication. We are working in the open to facilitate collaborations with publishers to improve scholarly communication. One major outcome of the peer review mentoring program will be a trained pool of reviewers that will be available to journal editors to accelerate the peer review process.
How can our community members interested in learning more about PREreview go?
The Coko community is growing, both in terms of participating organizations and lines of PubSweet code. If you want to learn more about working with us, contact any of our partners or reach out to us!