Crowd Sourced Review Probably Can’t Replace the Journals | The Road Goes On, Thoughts and Essays by Daniel Bingham

"Two years ago, I started a journey into academic publishing. I imagined using a reputation system to replace the journals with crowd sourcing. The reputation system would match reviewers to papers, incentivize good faith review, and identify bad actors. It wasn’t clear whether it would work in practice, but I wanted to find out.

I spent a year doing user research, building a beta (peer-review.io), and working to get people to give it a shot.

I am now convinced that it’s not going to work.

While some of the reasons are specific to the reputation based approach, most impact any attempt to crowd source peer review. There’s a brick wall standing in the way of any attempt to move academic publishing outside of the journals...."