Written by Paul Shannon, John Chodacki, Nokome Bentley, Adam Hyde, Ryan Dix-Peek, Yannis Barlas and Ben Whitmore
Recently, a team of technologists from across the globe came together in New Zealand to brainstorm how to integrate AI into the peer review process using Kotahi responsibly. We recognize the potential benefits that AI can bring, such as increased efficiency and accuracy, but we also acknowledge the need to be thoughtful about how we implement this technology. As a result, we engaged in a collaborative brainstorming session to explore how AI could be integrated into our platform responsibly and effectively. In this article, we will share some of our key insights and considerations from this session.
As the scientific publishing landscape continues to evolve, many are looking to AI as a potential solution to streamline the publishing process. At Kotahi, we’ve been thinking about how AI can be integrated into our platform to improve the efficiency and accuracy of the whole process.
Types of AI contributions
One question we’ve been grappling with is how to be transparent and honest about AI’s contribution and quantify its impact. Our approach was to first distinguish between different applications of AI technologies so we can better distinguish boundaries for where they are appropriate:
- Computer-assisted humans use AI to help with certain tasks, such as identifying potential conflicts of interest or suggesting potential reviewers.
- Generative AI, on the other hand, can create original content, such as writing summaries or even entire manuscripts.
By establishing this foundational distinction between these two approaches, we have a clearer understanding of the role that AI should play in the publication process.
Publishing is collaboration
While AI can help with certain tasks, such as identifying potential reviewers, it is important to remember that humans must continue to play the primary role in the publishing process. Authors and reviewers provide invaluable feedback and insights that cannot be replicated by AI alone. Therefore, we need to find a way to integrate AI into the publishing process transparently that also does not diminish the importance of human input.
One potential solution that maintains this distinction is to, for example, offer AI assistance to a reviewer directly in the reviewer form (as an opt-in) to help them turn their review notes (possibly in bullet points) into readable sentences and paragraphs that use a constructive, respectful tone suitable for a review. This would allow reviewers to choose whether or not they want to use AI to assist them in their reviews while also offering transparency about the use of AI in the review process.
This example illustrates how the process can be viewed as a kind of human-machine collaboration that improves efficiency and desired outcomes.
Always opt-in
Finally, we need to consider customization. Each group of researchers has unique needs and preferences, and AI should be customizable to meet their needs. By allowing researchers to customize the AI tools they use, we can ensure that they are getting the most out of this technology. Our approach would be to offer an opt-in process or button at the set-up of a journal to ensure the journal’s team is interested in AI-assisted features. On top of that, we would also want to ensure all authors are asked whether they would like to have their manuscripts handled with AI-supported features.
Where do we go from here?
Overall, integrating AI into Kotahi has the potential to greatly improve the efficiency and accuracy of the publishing process. However, we need to be thoughtful about how we implement this technology and be transparent about its contribution. By distinguishing between computer-assisted humans and generative AI, considering the role of manual input and allowing for customization, we can ensure that AI is integrated responsibly and effectively into the process.
With all this being said, we are all in. AI can and will greatly improve publishing, and Kotahi is set up to experiment with and implement AI seamlessly and responsibly. We are here to help you revolutionise scientific publishing with AI as a key component.
