Opening Up Peer Review

Yesterday, I had the opportunity to speak as part of a workshop held in conjunction with the annual meeting of the European Network for Cinema and Media Studies (NECS). A NECS working group had drafted a Statement on Open Scholarship that was under consideration for adoption by the membership, and the workshop was intended to provide an opportunity to dig into some of the issues raised by the statement. I took advantage of the opportunity to do some thinking about open peer review, which I hadn’t written about since before Generous Thinking. I’m glad to have had the opportunity to have put these thoughts together, especially regarding the questions of community and equity that have become so important to my recent work.

Thanks so much for asking me to participate in this workshop. One of the things that most excites me about the NECS statement on open scholarship is that it treats open access as a step toward creating greater equity in our fields, but it does not stop there. The statement recognizes that disseminating our work in open venues is just one important move toward a much larger and more important rethinking of the ways that we work and the values that we both bring to and uphold through that work.

That rethinking includes moving away from treating scholarly work as a production line, turning out an endless supply of new products, and instead understanding scholarly work as an ongoing process of discovery and exchange and conversation that benefits from openness in fostering greater collaboration and dialogue.

At the center of that process lies peer review, a form of sharing and discussion with colleagues that is designed to ensure that the work we produce is as good as it can be. Conventionally, that process or review has been handled through an intermediary, with a high degree of secrecy: editors select reviewers, who usually remain anonymous to the author, who in turn is often anonymous to the reviewers. The reviewers evaluate the work on behalf of the editor and submit reports to the editor, which the editor may or may not pass on to the author, and which are frequently redacted. The author then typically responds to the editor with information about how they will address the reviewers’ concerns in revision.

The anonymity and third-party mediation of this process evolved out of a desire for objectivity and impartiality in peer review — a laudable aim, if arguably an impossible one. If critical theory over the last fifty years has taught us nothing else, it has shown that we are all deeply subjective beings, and perhaps especially at those moments when we think we’re being most objective. Even more, those categories of identity that go unmarked — for gender, for race, for sexuality, for class, and so on — have close associations with what we define as “objective,” making minoritized perspectives always already “subjective.” As a result, our conventionally mediated anonymous forms of peer review sacrifice the potential for a highly productive set of exchanges among colleagues in service to an ideal that promotes and prolongs the status quo.

Opening up peer review is not a simple matter, of course. Scholars who act as reviewers in open processes need to find constructive ways of conveying critical responses — which often takes more thoughtful, careful work than does reflexive dismissal and rejection. They may also need to find the wherewithal to “speak truth to power” in cases where an author outranks them in the academic hierarchy — something that always feels risky, and especially so for early career scholars. Authors similarly need to confront their own feelings of vulnerability in making the bumps and foibles involved in the drafting process visible, and they need to be prepared to engage thoughtfully with critical commentary, perhaps especially when they disagree.

But all of this, as I hope you hear, is not about our publications, or about our publishing systems, but about us — about how we relate to one another, about how we engage with one another as we discuss our work. And thus all of it is within our power to improve — especially if we act as a community of practice, with an emphasis on community. We’ll need to establish standards and expectations for how collegial, constructive, and yet critical conversations can be carried out, and we’ll need to hold ourselves and one another accountable for adhering to those standards and expectations. But if we can do that, there is an enormous potential benefit for all of us, and for students and scholars yet to come, in getting to see and be part of the conversations that form a crucial part of the scholarly process.

Over the last dozen years, I’ve constructed and engaged in a range of open review processes. The first of these was a process I held in 2009 in conjunction with the submission of my second book, Planned Obsolescence, to NYU Press. While the press sent the manuscript to two anonymous reviewers, as usual, I posted the entire manuscript in CommentPress for discussion. The two reviewers from the press gave my editor, and through him, me, very thoughtful suggestions about how the manuscript might be strengthened, but the nearly 40 reviewers in the open process actively discussed those suggestions with one another, and with me, allowing me a much richer sense of what was just an idiosyncratic opinion and what was a real problem I needed to contend with — and even more, what that problem meant in the context of my argument. That open process also drew in a far broader range of readers and perspectives, including folks outside my immediate field whose opinions would never have been consulted in a conventional review process. And having those reviews as part of the public record of the manuscript’s development allowed me both to give credit to those reviewers whose ideas were particularly formative in my thinking and to allow the genealogy of the eventual book to remain visible to students and other readers curious about how the arguments evolved.

Since that time, I’ve replicated the process with a number of other projects, including a couple of journal articles and another book project. And in each case, the community of readers helped me to find means of rethinking and clarifying my arguments and their expression. I do want to acknowledge, though, that it hasn’t been all rainbows and unicorns. First off, this process has required a lot more labor, both from me, in encouraging and engaging with readers, and from the readers themselves. And parts of these processes have been difficult, including a few places where I wish the flaws in my drafts were perhaps a little less public, and a few comments that stung. But all of that — including the vulnerability and the exposure I felt — has both made the work better and made me a more generous scholar, recognizing as I do the enormous generosity readers extended to me in taking the time to read my work and to share their responses to it.

So what I hope that this workshop and the NECS statement on open scholarship will help the field develop is more such practices that are designed to highlight and reward the generosity of the scholarly community, that enable us to explore and expand on our processes of research and communication by calling attention to the work of peer review as a crucial contribution to scholarly conversations, enabling all of us to pursue both the goals that we have for our individual work as well as the collective goals to which our work contributes.