AI and Peer Reviewer Selection

In a recent study by Hamilton (2020), of 196 surveyed journal editors, only 35% reported satisfaction with their journal’s peer review process. They named, among other challenges, difficulty recruiting reviewers as one desired area of improvement. Publishers, meanwhile, have shown an interest in diversifying their author, editor, and reviewer pools, as evidenced by the proliferation of webinars, committees, projects, and coalitions dedicated to increasing Diversity, Equity, and Inclusion—notably, C4DISC, the Royal Society of Chemistry’s Joint Commitment, last year’s Peer Review Week theme of Identity in Peer Review, and the many excellent posts on the Scholarly Kitchen that address this topic. In recent years, companies like Clarivate and Unsilo have introduced AI-driven tools that claim to meet the editors’ need by suggesting relevant reviewers. In this poster, we will explore the viability of using these and similar technologies to improve the diversity of reviewer pools by connecting editors with reviewers outside of their usual professional networks. These technologies are exciting, but have the potential to perpetuate existing biases. Artificial intelligence has the potential to improve the peer-review system; however, Checco et al. (2021) found major pitfalls and implications of using AI tools in terms of biases and ethics.

To what extent can technology counterbalance human bias by suggesting early career, non-male, or international reviewersü What biases might an AI-driven reviewer search replicate, and can they be avoidedü How do design choices by submission management systems affect an editor’s reviewer selection processü Are there problems that can’t be solved by techü