When a university or research institution wants to hire a professor or other permanent academic position (e.g. full-time research scientist in CNRS in France), a committee is appointed (or elected) to decide who to hire. From my experience as a candidate and as a member of such committees, the way committees work is very interesting from the point of view of psychology. It seems like the perfect illustration of many known cognitive biases in decision making (see the excellent book of Nobel prize psychologist Daniel Kahneman, “Thinking, fast and slow”). I believe it also applies to grant selection committees.

Scientists tend to see them as rational beings. But it is now well established that 1) humans (including “experts”) are not rational at all in many situations, 2) we tend to think of ourselves as more rational than we actually are (i.e., we are confident in our biased judgments). Selecting the right candidate for a position is a difficult decision making problem, and most scientists in hiring committees do not know much about either decision making or psychology. Therefore it is likely that they are subject to cognitive biases. In addition, these are collective decisions, which come with additional biases described in social psychology.

I will focus on one particular situation, the selection of junior scientists by national research organizations in France, the largest one being the CNRS. I will also briefly mention a couple of other cases. After reading this text, you will probably feel that I have exposed some serious problems in the hiring process. However, the aim of this text is not to point fingers at individuals. On the contrary, my aim is to provide a psychological perspective on the process, that is to show that these problems reflect general human cognitive biases, which are well documented. Of course, I also believe that there are ways to reduce these problems, but this means changing the processes, not the individuals (who will still be humans). This text focuses on the psychological side of the problem (explaining what happens), not on its political side (changing the processes).

1. The situation

1.1. The decision to be taken

In France, each year, the CNRS offers a number of permanent positions to junior scientists (“junior” meaning generally in their 30s) in all academic disciplines. The call is national, not tied to a particular university (which have their own system). There is a general call for each entire discipline (say computer science) and there are many, many candidates.

Given that the positions are permanent (i.e., about 30-40 years of employment), the goal is to select the most promising young scientists, those who will have the most brilliant career. It is therefore a judgment on the expected future scientific output of the candidates, which I call the “target attribute”. Note that this is quite different from to the decision to a hire a postdoc for a specific 2-3 years project, where the target attribute is the correct accomplishment of the project.

The judgment is based on information available at the time of the decision, which is the past scientific career of the candidate, education and any other element available at that time.

1.2. The committee

For each discipline there is a committee of about 20 scientists (2/3 are elected, 1/3 are nominated). A large part of these scientists are junior scientists themselves, therefore they do not have either a long scientific experience or much experience in hiring people (e.g. postdocs). There is no external review, meaning that all applications are reviewed internally by some of these 20 scientists. Each candidate has a referee in the committee who assesses the application in detail.

1.3. Information and (lack of) feedback

The committee has to select a small number of candidates from a very large and diverse set of applications. For most of these candidates, there is no expert in the committee. The application consists of a CV (in particular list of publications), research project and report on previous work. There may also be reference letters, although there is no consistent rule across committees. The candidate also makes a short oral presentation and is interviewed for a short time by a subset of the committee (for practical reasons).

Part of the committee has no experience in hiring. How much can the committee learn from experience? First of all, the composition of committees changes every 4 years – even though some members can remain. To learn from experience requires feedback on decisions. A decision is: select a candidate, or reject a candidate. The target attribute to be judged is the future scientific output of the candidate during his/her 30-40 years contract.

First option: the candidate is selected, he or she goes to a lab. The lab is in general not the lab of one of the committee members. As far as I know, there is no follow-up to the decisions, e.g. to see how well the selected scientist does. If there was some, it would in any case be very limited in time, since the lifetime of a committee is 1/10th the duration of the scientist's career.

Second option: the candidate is rejected. Candidates that are rejected may 1) quit science, 2) find a position elsewhere, 3) apply again the next year. In the first two cases, the event is generally not known to the committee. But in any case, it is not possible to know how well the candidate would have done if he/she had been selected. So there is no feedback on the decision in these two cases. In the third case, there is still little useful feedback on the decision not to hire the person the previous year, given that the target attribute is the life-long career of an individual.

Finally, discussions and reports of the committee are not public, in fact they are strictly confidential. Therefore there can be no external feedback on the committee, and committees of different disciplines cannot exchange information.

1.4. Summary

In summary, the situation is that of a group of people that must take a very difficult decision, whose good or bad outcome can be assessed only in the long run, with limited information, and who have no opportunity to learn from experience. Therefore, decisions are based not on experience, but on the self-confidence of the committee in their own judgments. Additionally, information is unevenly distributed across the committee, because one member (the referee) examines the application in detail, and a subset of the committee is present for the oral presentation and interview.

The setting is perfect for all sorts of interesting cognitive phenomena. In the next posts, I will discuss in particular attribute substitution, cognitive dissonance, the halo effect, the illusion of validity, obedience to authority.