A praise of post-publication review

We have all suffered from the inadequacy of the current publication system. As readers, we often cannot freely access papers; when we can, papers often become available a very long time after having been written, sometimes years. As authors, we spend a large amount of our time submitting the same paper to many journals, reformatting, addressing criticisms that are sometimes contradictory, only to be rejected for subjective reasons (“readership interest”). As reviewers, multiple submissions mean that the same paper gets evaluated multiple times only to eventually put most reports in the trash. Our reviewing work is free and we do not get any credit for it. As hiring committee members, we have very little information to judge CVs because we do not have access to expert reviews on the papers, which are kept confidential by the journals. As a result, we use journal names as a proxy for quality. As candidates, we suffer from the randomness of editorial selection and the fact that committee members use journal names as a proxy for quality. As tax payers, we pay for an extremely inefficient, redundant and expensive publication system, largely to for-profit journals that make huge margins.

There is a solution to these problems; it’s called post-publication review. Journals belong to the past. We do not need journals to publish anymore. To publish means to make your work public. You publish when you decide to put your work online. Nowadays we call this a “pre-print”. The term itself is awfully misleading: is printing a paper really the goal of the scientific publication system? When was the last time you read a paper in actual printed journal? If I put my paper on arxiv, bioarxiv or another site then it is published, because one can publicly access it.

An objection is: scientific papers must be peer-reviewed. But why? Because we want to know what the community, in particular the most expert scientists, think of the paper. Does the current system allow that? No, because in almost all journals, reviews and author responses are confidential. Peer review can be also useful to correct errors. Great, but why should this correction process stop at publication time? If you want to let other scientists make useful suggestions, it makes so much more sense to publish your paper so that anyone can do this. If all the process is public, then we do not need the journal name to judge the quality of a paper; we can directly read what the community thinks, objects, whether there might be alternative interpretations, whether the authors have counter-arguments, all useful things that are currently hidden by an opaque review process. Finally, journal names are used as a quality label by hiring committees and grant agencies. But that’s bad; we know that journal impact factor and paper influence (eg future citations) are poorly correlated. Committees use it because they are often not in the exact field of the candidate and therefore have little information to judge the quality of papers. If reviews were public, and ideally signed, each candidate’s paper would come with several detailed recommendations and criticisms, much more informative than journal name.

The new publication process would be: publish online, then ask for public reviews, answer and revise. How would you get to know the important papers in your field if journals disappear? Well, there are many ways: keyword or author-based email alerts, most positively reviewed papers, papers liked or reviewed by peers you respect, blogs that make a monthly selection of papers (= journals), PageRank algorithm, etc.

This is not a utopia: it is coming, although quite slowly because we scientists appear to be a very conservative group of people – still happily reviewing for private journals that make huge profits from our free work, most of which is thrown away. There have been many initiatives these last years. PLoS has pioneered open access. eLife publishes reviews and responses after publication. Nature communications just started the same thing. Pubpeer allows post-publication comments. F1000Research publishes papers and then does post-publication reviews. And now the Wellcome trust implements this same scheme for its grantees.

The question then is not, can we realistically switch to a post-publication process, but rather: what can we do to accelerate this switch?

There are a few ideas in Michael Eisen (founder of PLoS) “Mission Bay Manifesto on Science Publishing”. Some are more difficult than others. Here are some ideas that anyone could easily implement:

  • Remove journal names from your online publication lists, and from your presentations. Anyone interested can find out anyway.
  • Start writing post-publication reviews, for example in Pubpeer, but on papers you are interested in – not just pointing out flaws.
  • Post every paper on arxiv, bioarxiv , before sending it to a journal.
  • Think about what journal to submit to. Whenever possible, submit to open-access non-profit journals instead of for-profit journals. There is for example the PLoS journals. As much as you can, submit to journals with public reviews. For example, eLife.
  • If you are in hiring committees, try to resist the journal name criterion.
  • Start a blog and make an annotated selection of papers you like.

Journals belong to the past. We have the opportunity now to build a much better scientific publication system for all parties involved, and we should do it.


2 réflexions au sujet de « A praise of post-publication review »

  1. A few comments:
    - https://publons.com/ is a great way to gain credit for peer review
    - https://peerj.com/ is a journal with open peer review. As far as I can see, it is cheaper and offers better service than Plos and Frontiers. Unfortunately it does not (yet) have the popularity it deserves.
    - It is not clear to me we can entirely get rid of pre-publication peer review. It seems to me that we may still need a filter for entry into the official scientific literature, to keep it separate from all the junk out there (think of the medical literature and all the non-scientists who read it). Of course there are loopholes and the peer-reviewed literature is full of junk too, but still it's not the same level of junk as outside the peer-reviewed literature.
    - Furthermore I'm not so sure that there will be enough incentives for post-publication peer review to keep the system running and ensure most of the papers posted get reviewed. We may end up with just a tiny fraction of papers reviewed, what should we do with the rest?
    - Post-publication peer reviews and selections should not remain on blogs, but should be posted on well-indexed databases, otherwise they will be lost for most purposes.

    • I think F1000 Research adresses some of these concerns. There are still invited reviewers, who sign their review and might endorse the paper, and the paper gets indexed when it is approved by a couple of reviewers.
      I agree that that reviews should not remain on blogs, but directly accessible from the place where the paper is published, or some well-indexed database. Some ongoing experiments are pubpeer and pubmed commons.

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *