In 2016, the academic publication system worked mostly as follows :

1. Scientists do some research and write a paper about it to share their findings with the world.
2. The paper is submitted to a journal, chosen for its prestige.
3. The editor, possibly a professional editor (ie not a professional scientist), judges that the paper is not worth of review: go back to (2).
4. The editor sends the paper to 2-3 reviewers, asking them to review the paper quickly (the more prestigious, the quicker). Reviewers are chosen based on the editor’s expertise and network.
5. Reviewers have too many reviews to write already: go back to (4).
6. The reviewers (or one reviewer) find the paper uninteresting, or bad, or they “don’t believe it”: go back to (2).
7. The reviews are mixed: a third reviewer is asked; go back to (4).
8. The reviewers find the paper interesting enough but have requests for the authors; they should entirely rewrite their paper; they should do new experiments; they should cite the reviewers; they should remove an uninteresting figure; they should add some discussion points.
9. After some work, the authors submit a revised paper.
10. Actually, reviewers didn’t think the paper was that interesting anyway: go back to (2).
11. The reviewers have more requests: go back to (9).
12. The paper is accepted: congratulations!
13. The format of references does not follow the journal’s style: please rewrite the reference list.
14. The author’s paper now belongs to Elsevier/Springer Nature/etc. It will be sold by the publisher to academic institutions, with a margin of 30-40%.
15. (recent alternative option) The authors pay publication fees of about $2000. The paper is now freely accessible.
16. A knowledgeable reader disagrees with the main conclusion of the paper: she writes a letter to the editor.
17. After careful consideration, the letter is rejected.
18. Another reader, not an expert of the field, reads the paper in a prestigious journal and believes it is highly regarded in the community, until she talks with colleagues and finds out that the paper is very criticized.
19. The prestigious paper is commented in mainstream journals. Experts rant about the journalists.
20. The first author applies for a position. The committee examines CVs of all candidates and selects those with publications in prestigious journals. The senior author sends an excellent letter of recommendation. The committee values it if the senior author is famous.
21. The senior author applies for a grant. The committee selects candidates with a good “track record”, ie prestigious publications.

This publication model is wasteful, full of perverse incentives, opaque, and misleading. What I want to do here is not one more critique of this system, but rather give a vision of what the publication system could be and, in my opinion, will be, perhaps within a decade. This is a vision of a publication system centered on individual articles, rather than on journals. It is made possible today by information technology (aka “the internet”). What stands in its way is, in a word: conservatism. It works as follows:

1. Scientists do some research and write a paper about it to share their findings with the world.
2. The paper is uploaded to an archive site, together with the data. All is freely accessible to anyone.
3. The authors invite an unlimited number of professional scientists (say, authors of at least one paper) to write a formal, signed, review. The reviews are public.
4. Any other scientist can contribute a signed or anonymous review or comment, including comments on the public reviews (aka “scientific debate”).
5. Authors respond to the reviews and may submit a revised version.
6. Authors decide when to stop the revision process; reviewers express their satisfaction, or not.
7. Any scientist can recommend the paper if she finds it worth reading. Tools are made available to search for papers, possibly using the number of recommendations, possibly weighted by scientist reputation (all data are freely accessible and third parties provide searching tools).
8. A knowledgeable reader disagrees with the main conclusion of the paper: she writes a public critique, signed or anonymous, after reexamining the paper’s raw data. The authors respond publicly.
9. Another reader, not an expert of the field, reads the paper. She notices that many knowledgeable scientists disagree with the conclusions and can read the detailed arguments and the authors’ response.
10. The first author applies for a position. For every publication, the committee reads the reaction of the community (formal reviews, comments, recommendations) and uses it to evaluate the value of the work along all the dimensions addressed in the reactions (importance, originality, technicality, etc). The committee also asks the candidate what her main contributions are in her view and why. The candidate can point to published reactions to her work that support her claims.
11. The senior author applies for a grant or a promotion. The committee reads the reaction of the community to the candidate’s previous work. They might also look at review invitations and reviewing activity.

In the post-journal world, a paper is submitted a single time (but of course it can be revised), and in contrast with the journal model, the reaction of the community is known. As a consequence, quality stamps (journal names) are obsolete. No review is thrown away. As a consequence, scientists save time on both submitting and reviewing. Scientists are not incentivized to make grand claims. In addition, the system saves several billions of dollars each year (eg US libraries spend $2 billion each year on subscriptions), not counting the time saved on wasted reviews and submissions.

All of this is clearly possible with today’s technology. The next question I will try to address is: how to move quickly to the post-journal world?

Where to send computational and theoretical neuroscience papers? A list of journals can be found on Jim Perlewitz’s page. First, many journals belong to one of the scientific publishing companies who make huge profits from the free work of academics (margins around 30-40%), with questionable added value. In computational neuroscience, we have:

• Journal in Computational Neuroscience (Springer Nature)

• Frontiers in Computational Neuroscience; in Neuroinformatics (Springer Nature)

• Biological Cybernetics (Springer Nature)

• The Journal of Mathematical Neuroscience (Springer Nature)

• Cognitive Neurodynamics (Springer Nature)

• Neuroinformatics (Springer Nature)

• Network: Computation in Neural Systems (Taylor & Francis)

Then we have Neural Computation, run by MIT Press, which is non-profit. Unfortunately, papers are behind paywalls. In my case, the library does not have a subscription.

Finally, there is PLoS Computational Biology, a highly respected journal in the field. However, even in this non-profit journal, open access is very expensive ($2250 per paper). I find these costs hard to justify, when any scientist can publish on archives for free (eg arxiv or biorxiv). The solution to this is overlay journals: the papers are linked to an archive site; the journal only deals with peer-review.

I was also recently rather disappointed by PLoS CB, after a paper got rejected 10 month after submission on the sole basis of interest. Based on the reactions I received, this is not an isolated case. Anyway, PLoS CB practices editorial pre-selection, which is highly subjective. As for other prestigious journals, its prestige is precisely based on its relatively low acceptance rate. But this is a highly questionable model, because it is wasteful: wasting the time of authors who submit the same paper to many journals, wasting the time of reviewers who write reports that are thrown away. In my opinion, it is also perverse. It gives a label of quality to individual papers and scientists based on the opinion of just a couple of scientists, sampled by one editor, and this insignificant label is then used by various committees (grants, promotions, hiring). It also pushes authors into boosting their claims to pass the editorial barrier. The solution to this is post-publication review (see also Michael Eisen’s post).

So ideally, I would like to see an overlay journal practicing post-publication review in my field. The question now is: should we create one? What would be needed is the technical infrastructure, but I am not sure it is fully available yet (as regards the post-publication review system, indexing etc).

Alternatively, we could use a general post-publication review journal. There is for example F1000 Research. PeerJ also does this partially (publishing the reviews is an option). Both cost about $1000 per article; both are run by companies. But they are less expensive than alternative open access journals (PLoS ONE: $1495; eNeuro: $1950 for members, $2925 otherwise), and more open. None of these two journals is ideal, but they represent some progress over conventional journals.

There are two issues with choosing a general journal. First, there is the question of editors. I have no experience with these journals so I am not sure there is the relevant expertise in the editorial board. I have seen a couple of computational neuroscientists in PeerJ though. The second issue is visibility. One idea could be to manage a list of computational neuroscience papers, to which anyone could submit a reference or preprint, in addition to automatic additions (say from relevant journals + keyword alerts). One would simply subscribe to the list (RSS feed or email alert).

Any good ideas are welcome!

Update (6.1.2017)

Some more information and thought following comments on this text:

• One issue for an overlay journal is to get it indexed in Pubmed. This seems quite important for a neuroscience journal, but the indexing requirements seem to be pretty much contradictory with the idea of an overlay journal, as it requires XML-tagged data of its bibliographic citations. But maybe this could be negotiable? (see "Optional arrangement"). Some more general information about indexing here.
• Other issue is more technical; to get an automatic post-publication review system.
• Finally a general thought; there is no reason why this should be specific to computational neuroscience. However, this could be a first step towards a more generalized system.

A lot has been written about impact factors, and why they are a poor indicator of article quality (just google it). Even for the one thing that the impact factor is meant to measure, ie, future citations, it performs badly. There are now many empirical studies on this. Sadly, in 2016, scientists in hiring or promotion committees still value impact factors very highly, especially in life sciences. Yes, in a domain of science that is meant to be highly empirical, scientists still prefer to look away from compelling evidence that impact factors should not be used for evaluating research. Recently, in a PhD thesis committee, we were discussing the grade and one member, an experimental biologist, raised the issue that the candidate had not published in the highest ranked journals. We opposed that we were there precisely to evaluate the manuscript itself as a whole, not some elements of CV. He admitted later that he had not read the thesis.

I could simply cite the overwhelming evidence. But I want to make a simpler point of statistics. Many things are wrong with using impact factors, but let us just consider the one thing it is meant for, ie, predicting the number of future citations of papers. We know that for any journal, the distributions of citations is very broad and highly skewed. According to Seglen (1997), “articles in the most cited half of articles in a journal are cited 10 times as often as the least cited half”. In practice, if you are given the name of the journal in which a paper has been published, you will not know its future number of citations. From a purely statistical point of view, journal name is just not very significant.

Examples are sometimes more convincing that hard facts. So from the paper I just cited, number of citations vs. impact factor for all publications of 4 scientists:

Now look at my own publication statistics. To be more rigorous, I should calculate the citations per year, but the point is obvious enough. My top 3 cited papers were published in: J. Neurophys (IF 2.8); J. Comp. Neuro (IF 1.8); Frontiers in Neuroinformatics(IF 1.3). This third paper (on the Brian simulator) was published in 2008, and the same year I had a paper in Neuron, which has gathered about 4 times fewer citations. And this is in spite of the fact that papers published in big journals are obviously much more visible (ie there is an expected boost in citations that is unrelated to article quality).

Is this really surprising? In a big journal, just like in a small journal, decisions are based on n = 2 or 3 reviewers, selected by one editor, i.e. a biased sample of size 2/3; just after an editorial selection based on n = 1 editor, in many cases not a professional scientist.

It’s 2016 and life scientists have been recently warned repeatedly that good statistics require large unbiased samples, but impact factors, totally insignificant pieces of information, are still broadly used to distribute grants and make careers. So yes, using impact factors as a proxy to evaluate the quality of papers or of their authors is a display of poor scientific practice.

To me the problem with this is so not much that bad decisions are being made, but that it has a huge influence on the way science is done. Scientists know that they are being evaluated in this way, and this influences their research (especially for students and postdocs) so that it meets the editorial biases of the big journals. In 1952, Hodgkin and Huxley published 120 pages in Journal of Physiology, basically defining the field of electrophysiology. Who would take the risk of doing this today at the age of 35, instead of a career-making 4-page Nature paper?

We have all suffered from the inadequacy of the current publication system. As readers, we often cannot freely access papers; when we can, papers often become available a very long time after having been written, sometimes years. As authors, we spend a large amount of our time submitting the same paper to many journals, reformatting, addressing criticisms that are sometimes contradictory, only to be rejected for subjective reasons (“readership interest”). As reviewers, multiple submissions mean that the same paper gets evaluated multiple times only to eventually put most reports in the trash. Our reviewing work is free and we do not get any credit for it. As hiring committee members, we have very little information to judge CVs because we do not have access to expert reviews on the papers, which are kept confidential by the journals. As a result, we use journal names as a proxy for quality. As candidates, we suffer from the randomness of editorial selection and the fact that committee members use journal names as a proxy for quality. As tax payers, we pay for an extremely inefficient, redundant and expensive publication system, largely to for-profit journals that make huge margins.

There is a solution to these problems; it’s called post-publication review. Journals belong to the past. We do not need journals to publish anymore. To publish means to make your work public. You publish when you decide to put your work online. Nowadays we call this a “pre-print”. The term itself is awfully misleading: is printing a paper really the goal of the scientific publication system? When was the last time you read a paper in actual printed journal? If I put my paper on arxiv, bioarxiv or another site then it is published, because one can publicly access it.

An objection is: scientific papers must be peer-reviewed. But why? Because we want to know what the community, in particular the most expert scientists, think of the paper. Does the current system allow that? No, because in almost all journals, reviews and author responses are confidential. Peer review can be also useful to correct errors. Great, but why should this correction process stop at publication time? If you want to let other scientists make useful suggestions, it makes so much more sense to publish your paper so that anyone can do this. If all the process is public, then we do not need the journal name to judge the quality of a paper; we can directly read what the community thinks, objects, whether there might be alternative interpretations, whether the authors have counter-arguments, all useful things that are currently hidden by an opaque review process. Finally, journal names are used as a quality label by hiring committees and grant agencies. But that’s bad; we know that journal impact factor and paper influence (eg future citations) are poorly correlated. Committees use it because they are often not in the exact field of the candidate and therefore have little information to judge the quality of papers. If reviews were public, and ideally signed, each candidate’s paper would come with several detailed recommendations and criticisms, much more informative than journal name.

The new publication process would be: publish online, then ask for public reviews, answer and revise. How would you get to know the important papers in your field if journals disappear? Well, there are many ways: keyword or author-based email alerts, most positively reviewed papers, papers liked or reviewed by peers you respect, blogs that make a monthly selection of papers (= journals), PageRank algorithm, etc.

This is not a utopia: it is coming, although quite slowly because we scientists appear to be a very conservative group of people – still happily reviewing for private journals that make huge profits from our free work, most of which is thrown away. There have been many initiatives these last years. PLoS has pioneered open access. eLife publishes reviews and responses after publication. Nature communications just started the same thing. Pubpeer allows post-publication comments. F1000Research publishes papers and then does post-publication reviews. And now the Wellcome trust implements this same scheme for its grantees.

The question then is not, can we realistically switch to a post-publication process, but rather: what can we do to accelerate this switch?

There are a few ideas in Michael Eisen (founder of PLoS) “Mission Bay Manifesto on Science Publishing”. Some are more difficult than others. Here are some ideas that anyone could easily implement:

• Remove journal names from your online publication lists, and from your presentations. Anyone interested can find out anyway.
• Start writing post-publication reviews, for example in Pubpeer, but on papers you are interested in – not just pointing out flaws.
• Post every paper on arxiv, bioarxiv , before sending it to a journal.
• Think about what journal to submit to. Whenever possible, submit to open-access non-profit journals instead of for-profit journals. There is for example the PLoS journals. As much as you can, submit to journals with public reviews. For example, eLife.
• If you are in hiring committees, try to resist the journal name criterion.
• Start a blog and make an annotated selection of papers you like.

Journals belong to the past. We have the opportunity now to build a much better scientific publication system for all parties involved, and we should do it.