First I will comment on the criteria that corresponds to the question “what is the experience of the candidate?”, rather than the original question that the committee is supposed to answer: “how likely is the candidate to have a brilliant scientific career over the next 30-40 years”. There is no doubt that the candidate's experience is one factor that should be taken into account to make the hiring decision. Attribute substitution is when that factor is not simply taken into account, but mistaken for the target attribute that constitutes the object of the judgment.

Here are a few criteria that are unofficially used to assess the experience of the candidate:

- The number of publications; in particular, there is often an unofficial threshold for that number.

- The age of the candidate (older is better, but not too old).

- In teaching positions (not CNRS), whether the candidate has already taught quite a bit.

- Whether the candidate did a postdoc abroad (rather than in the same country, which would be bad).

The key here is to note that these criteria are about the substituted attribute, the experience of the candidate at the time of the application. They are not directly about the target attribute, the future career of the scientist. But they are assessed as if they were actually directly about that target attribute.

The number of publications

Consider the number of publications. Quite obviously, a candidate who has not published (not as a main author), should not be hired. Even if the candidate is brilliant, there is simply no information to know it. But using the number of publications as a proxy for “scientific excellence” (one of the official criteria) is another story. “Scientific excellence” is about productivity. All other things being equal (e.g. quality of the publications), more is better: it is better a hire a scientist who will publish in his/her entire life 100 high-quality papers than 10 papers of the same quality. The problem is that “number of publications at the time of application” is a rather poor substitute for future productivity. Imagine you have all the information you need, that is, the number of future publications of the candidate if he/she is hired. For a given productivity, the number of publications at time of application obviously correlates with the time for which the candidate has been publishing. Every year the substituted attribute (number of publications) increases with no change in the target attribute (productivity). This leads to paradoxical decisions: a candidate who has published 10 papers in 15 years will be ranked higher than a candidate who has published 8 papers in 4 years (again, all other things being equal - I am only considering the number of publications). The substituted attribute has no direct relationship with the target attribute. Yet it seems to be used as such (at least in biology sections).

The age of the candidate

Over the last few decades, the average hiring age in CNRS junior positions has increased very substantially. There was a time when scientists were hired right after they obtained their PhD. This would seem almost crazy now, yet a number of the older committee members got their position at that time, and would now argue (cognitive dissonance!, I will come back to that) that it would not be reasonable to hire scientists so young. Why did hiring age increase so much? A major fact to consider is that the number of applications has considerably increased without a corresponding increase in the number of positions, but let us consider a few hypotheses.

Hypothesis 1: age is a reasonable criterion, but and committee members failed to recognize it before. Given that the way committees are composed has not changed and that, as I noted above, committees have no feedback to learn from errors, this seems unlikely.

Hypothesis 2: young scientists were much better in the old times, and so informed decisions could be taken at an earlier age. This would require that young scientists published more before (otherwise there can be no informed decision), but the trend is in fact opposite.

Hypothesis 3: the way science is done has completely changed, and so now people need much more experience. I have heard this hypothesis, in particular to explain why young biologists must now do several postdocs before finding a position. Note that the argument is about experience, a substituted attribute, and not about future scientific career. I observe that: 1) one can learn things before or after being permanently hired, 2) during the time when they learn the things that are now required for being a good permanent scientist, the candidate is a non-permanent scientist, i.e. a scientist on a different type of contract; so it would seem that the requirement if for having a permanent contract, not for doing science. In my field, which is interdisciplinary, I know of many examples of older renown scientists who made a career in a different field (e.g. physics) before changing field (e.g. biology/neuroscience) while being on permanent positions. So empirically, having a permanent position does not seem to prevent one to learn new things and even change fields. Take Gerald Edelman: Nobel prize for his immunology work, then changed fields to work in neuroscience. He was not a postdoc when he got the Nobel prize. Therefore, this hypothesis does not seem to have a clear rational or empirical basis.

Hypothesis 4: finally, the explanation I have heard most often from committee members (so, people who have a role in this age increase) is the following: the number of candidates has increased, and so “mechanically” hiring age increases. More recently, there has been another increase in mean hiring age after the legal limit on application age was raised, which was explained to me as follows: now there is a competition with older candidates who necessarily have a better application, and since we take the best application, we have to take the oldest candidates. “Better application” means more publications here. So quite explicitly, it appears that the committee substitutes experience for excellence. I note again that the older scientist once was a younger scientist, with fewer publications (= substituted attribute), and yet both have the same scientific career (= target attribute).

Teaching experience

Another typical case of substituting experience is in hiring assistant professors in universities. As I have been in a few such committees, I can tell that one strong criterion (which is a threshold, i.e., a pass/fail criterion) is the number of hours that the candidate has taught, and it has to be sufficient. In general, lectures are taught only by permanent faculty (assistant professors and professors), and so candidates have taught tutorials, which is considered fine. At first sight, it seems to make sense to consider teaching experience for a teaching position, and this is why this attribute is used as a substitute, without even necessarily noting the substitution. However, the target attribute is not how much one as taught, but how good the candidate will be as a teacher. The number of hours that someone has taught is essentially irrelevant, since it gives no indication as to whether the candidate is a good or a bad teacher. In general, students teach during their PhD because either it is mandatory or they get a substantial amount of money. Students who do not teach during their PhD have a contract that does require them to teach, and these students want to spend more time on their research. It doesn't make necessarily them bad teachers.

Now one could (and would) oppose the following argument: candidates who have more teaching experience will be better teachers. First of all, since we are talking of a permanent position, the few hours one has taught at the time of application will have little impact on the timescale of a 30-40 years career. But more importantly, this argument is illogical. Candidates who have taught before being hired, and who are supposedly better teachers now, were then worse teachers when they started (which apparently was not an issue then). The decision of permanently hiring when they start teaching or after they start teaching has no impact whatsoever on their teaching experience over the course of their teaching career. Experience in teaching at the time of application should not be relevant to the hiring decision if there is no feedback on the teaching experience. This illustrates a clear case of attribute substitution.

Unfortunately, this substituted criterion is in broad use and prevents a number of good young scientists to apply. Indeed, in France, to be authorized to locally apply to a faculty position in a university, one must first pass a national screening stage (“qualification”) in which a national committee decides, with their own criteria, whether the candidate has the required credentials to apply to faculty positions. In general, committees require candidates to have taught a certain number of hours. The committees have no information on the quality of teaching, only on what was taught and for how many hours. So let me be clear: an explicit requirement to be allowed to teach on a permanent contract is to have taught on a fixed-term contract (e.g. PhD or ATER, which is temporary assistant professor), in which case there is no requirement.

It gets better: there is one committee for each discipline, but if this screening stage is passed, then the candidate can apply to any faculty position in any other discipline. I have a persona example. I did a PhD in computational neuroscience, an interdisciplinary field, during which I gave math tutorials. After I got my diploma, I applied to that screening stage in different committees. At the time, I already had a few published papers and had taught about 200 hours. But the computer science committee decided not to grant me the authorization, because (I assume) I had not taught computer science, or perhaps they considered the research not relevant to their discipline, even though it was quite obvious that I had the required credentials. This has happened again this year for former PhD student of mine: he had taught the right number of hours, but both in maths and computer science instead of only in computer science, so they rejected the application. Fortunately for me, the neuroscience committee did grant me the authorization. The next year, I got a position in the computer science department of Ecole Normale Supérieure, thanks to the authorization from the neuroscience committee. It is clear here that the question that the computer science committee answered was not, as it should have been: “should this guy be authorized to apply for a faculty position in some discipline?” but a substituted question, “did that guy teach 200 hours of computer science?”. This is particularly problematic for interdisciplinary science.

Postdoc experience

In many committees, in particular in biology, but also in many others, it is considered that one must have done a postdoc abroad to be seriously considered. It acts as a first screening: no postdoc abroad = fail. In some committees, the candidate must also have published during that postdoc, so as to show that the PhD publications were not just because of the supervisor, who is generally a co-author – although the postdoc is generally not an independent position either.

I also have a personal example. When I applied after the PhD, I failed. I asked one committee member for feedback. He told me: your application was considered excellent, so you should do a postdoc abroad and then next year you will get the position. For personal reasons, I did not want to move far, and I could not understand the logic behind that requirement. I already had two publications as a single author, and in fact my supervisor did not sign any of my publications, so the argument I previously mentioned did not apply. But in any case, as it was phrased, what the committee wanted was not that I publish in another lab, they just wanted me to check the box “postdoc abroad” (there actually is a box in the CNRS application forms). Also, the requirement was not simply: postdoc in another lab, but in a different country. I also did not understand that geographic requirement: I had spent an extended period in England during studies, which the committee knew, and so the linguistic argument did not apply either. It turned out that what I wanted to do was best done in another lab in France, but it didn't help me check the required box. So apparently, the single fact that a postdoc was done abroad, without knowing how the postdoc actually went, was a decisive criterion in the ranking, independently of any other consideration. Imagine I had actually planned to do a postdoc abroad, and already had made arrangements to do it at the time of application. Then, given the information, the committee would have known with certainty that the next year, I would have done a postdoc abroad and therefore that I would definitely pass that criterion. So actually doing the postdoc abroad was apparently irrelevant to their decision. So there was no rational basis to that requirement.

The committee member had given me that piece of information without blushing, and he did not seem to be embarrassed by the fact that the committee insisted on such an irrational criterion. It surprised me at first that people involved in irrational decision making seem at the same time to be very confident about of the correctness of their decisions. This occurs even though it is clear that the decision to be taken is difficult and there is considerable uncertainty in the choice. But in fact this is a very well established psychological phenomenon, which is explained for example in Daniel Kahneman's book. The degree of confidence one has in a decision or judgment is essentially determined not by the rationality of that decision or objective facts, but by the consistence of the causal story that one makes to explain the decision. So one would say: it is clear that Peter should be ranked before Paul, since he has two more publications; we could not do otherwise. But the story neglects the fact that the number of publications is actually a substituted attribute, not the target attribute of the judgment. The same goes for other substituted attributes. People involved in the decision can hear the objective elements that contradict the decision, but if they cannot be fit in the story, they are essentially ignored. This is related to cognitive dissonance theory, certainly one of the most brilliant theories in psychology, developed by Festinger in the 1950s. I will talk about it later.