The current discussions about AI are plagued with anthropomorphism. Of course, the name “artificial intelligence” has probably something to do with the matter. A conscious artificial intelligence might sound like science-fiction, but a sentient statistical model certainly sounds a bit more bizarre. In this post, I want to address the following question: are large language models (LLM), such as ChatGPT, or more generally deep learning models, conscious? (perhaps just a little bit?)

The first thing that must be realized is that the question of what it takes for something to be conscious is a very old question, which has been the subject of considerable work in the past. The opinions and arguments that we read today are not original. Very often, they unknowingly repeat common arguments that have been discussed at length in the past. It might seem for the general public or AI engineers that the spectacular successes of modern AI programs should bring a new light on these old questions, but that is not really the case. These spectacular programs already existed 50 years ago, just as thought experiments: which for logical arguments makes no difference at all. For example, in 1980, Searle articulated an argument against the idea that a program that seems to behave like a human would necessarily have a mind. The argument was based on a thought experiment, where an average English-speaking person would apply the operations of a sophisticated program that takes a question written in Chinese as input and outputs an answer in Chinese. Searle argues that while that person is able to have a conversation in Chinese by mechanically applying rules, she does not understand Chinese. Therefore, he concludes, understanding cannot be identified with the production of acceptable answers. Now such programs exist, and their existence does not change anything to either the argument or the objections that have been raised then. All these discussions were precisely based on the premises that such systems would exist.

This is an important preamble: a lot has been said and written about consciousness, what it means, what it takes for something to be conscious, etc. Answers to these questions are not to be found in current AI research but in the philosophy literature, and whether a program produces convincing answers to questions, or can correctly categorize images, bears very little relevance to any of these discussions. A corollary is that AI engineering skills provide no particular authority on the subject of AI consciousness, and indeed, as far as I can tell, many arguments we hear in the AI community on this subject tend be rather naïve, or at least not original and with known weaknesses.

Another important remark is that, while there is a lot of literature on the subject, there is no consensus whatsoever in the specialized community on those questions. Views on consciousness go from consciousness does not exist (eliminativism/illusionism) to everything is conscious (panpsychism(s)). In my view, there is no consensus because there is currently no convincing theory of consciousness (some say that there can be no theory of consciousness, which is another proposition on which there is no consensus). There are good reasons for this state of affair, which I scratched here and there (and others have too, obviously).

This remark is important because I have seen it become an argument for why a particular artefact might be conscious: we don’t know what consciousness is exactly, or what it takes for something to be conscious, therefore we cannot rule out the possibility that something in particular is conscious.

However, this is a fallacy. We do not know what it takes for an entity to be conscious, but we certainly do know that certain entities cannot be conscious, if the meaning we give to this notion must have some vague resemblance with its usual meaning.

Now, I will make a few general points against the idea that LLMs, or more generally formal deep neural networks, are conscious, by discussing the concept of a “conscious state” (i.e., “how it feels like now”).

Once we remove the buzzwords, such as “intelligence”, “learning”, etc., a modern deep learning model is essentially a massively parallel differentiable program. In effect, it is essentially a tensor calculator, the state of which is updated by a series of iterations.

The default assumption in the computational view of mind is that a mental state is something like a program state. But a conscious state is a very particular kind of state. First of all, a conscious state has a subject (the entity that is conscious) and an object (what it is about). Both relate to well-known topics in philosophy of mind, namely the unity of consciousness and the intentionality of consciousness.

When we wonder whether something is conscious, that thing is typically an organism, or a machine, or even, if you adopt panpsychism, a rock or an atom. But we could consider many other combinations of molecules and ask whether they are conscious. How about the entity made of a piece of my finger (still attached to my body) plus two coins in a fountain in Rome? This seems absurd, but why? There is a reason why: two objects that do not interact at all have the same properties whether we look at them one by one or together. New properties above the individual elements can only arise if there is some kind of interaction between the elements. So, if neither my finger nor any of the coins is conscious, then the combination is not conscious either. Thus, we can say that a necessary condition for a set of components to constitute a subject is that there is some causal integration between the components. This is actually the basis of one particular theory of consciousness, Integrated Information Theory (which I have criticized here, there and there, essentially because a necessary condition is not a sufficient condition). If a deep learning network is conscious, then which layer is the subject?

I leave this tricky question hanging there to address the more critical ones (but in case you want to dig, look up these keywords: autonomy of the living, biological organism, biological agency). One is the question of intentionality: the fact that a conscious state is about something: I am conscious of something in particular. A large language model is a symbol processing system. The problem is that it is humans who give meaning to symbols. The program is fed with sequences of bits, and outputs sequences of bits. If one asks an LLM “What is the color of a banana?” and the LLM replies “yellow”, does the program understand what a banana is? Clearly, it doesn’t have the visual experience of imagining a banana. It has never seen a banana. It doesn’t experience the smell or touch of a banana. All it does is output 0100, which we have decided stands for the word banana, when we input a particular stream of numbers. But the particular numbers are totally arbitrary: we could choose to assign a different sequence of numbers to each of the words and we would still interpret the computer’s output in exactly the same way, even though the program would now have different inputs and outputs. So, if the program has a conscious experience, then it is about nothing in particular: therefore, there is no conscious experience at all.

This is known as the “symbol grounding problem”, again a well-known problem (which I examined in the context of “neural codes”). Many people consider that a necessary ingredient has to do with embodiment, that is, the idea that the machine has to have a body interacting with the world (but again, careful, necessary does not mean sufficient). It seems that it is Yann Le Cun’s position, for example. Again, it is not clear at all what it takes for something to be conscious, but it is clear that there can be no conscious experience at all unless that experience is somehow about something, and so the symbol grounding problem must be addressed.

These are well-known problems in philosophy of mind. The final point I want to discuss here is more subtle. It relates to the notion of “state”. In computationalism (more generally, functionalism), a conscious state is a sort of program state. But there is a big confusion here. Both terminologies use the word “state”, but their meaning is completely different. A state of mind, an experience, is not at all a “state” in the physical sense of pressure, volume and so on, that is, the configuration of a system. Consider a visual experience. Can it be the case that to see something is to have the brain in a certain state? For example, some neurons in the inferotemporal cortex fire when a picture of Jennifer Anniston is presented: is the visual experience of Jennifer Anniston the same as the active state of the neuron? If the answer is positive, then why should you have the experience, rather than that neuron? Otherwise, perhaps the neuron triggers the experience, but then we need to assign the experience to the state of some downstream neurons and we face the problem of infinite regress.

The issue is that an experience is simply not a physical state; to treat it as such is a category error. To see this, consider this thought experiment, which I called the “Bewitched thought experiment” (self-quote):

In the TV series Bewitched, Samantha the housewife twitches her nose and everyone freezes except her. Then she twitches her nose and everyone unfreezes, without noticing that anything happened. For them, time has effectively stopped. Was anyone experiencing anything during that time? According to the encoding view of conscious experience, yes: one experiences the same percept during the entire time, determined by the unchanging state of the brain. But this seems wrong, and indeed in the TV series the characters behave as if there had been no experience at all during that time.

It would seem bizarre that people experience something when their brain state is maintained fixed. That is because to experience is an activity, not a thing. Therefore, a system cannot experience something just by virtue of being in some state. The vector of activation states of a neural network is not a conscious “state”. It is not even false: it is a category error. So, when a deep network outputs a symbol that we associate to Jennifer Anniston, it doesn’t actually see Jennifer Anniston. Having the output “neuron” in a particular state is not an experience, let alone an experience of seeing Jennifer Anniston.

All these remarks, which tend to show that the application of tensor calculus does not produce conscious experience, can be perplexing because it is hard to imagine what else could possibly produce conscious experience. Current “artificial intelligence” is our best shot so far at mimicking consciousness, and I am saying that it is not even a little bit conscious. So, what else could consciousness be then?

My honest answer is: I don’t know. Will we know someday? I also don’t know. Should it be possible, hypothetically, to build a conscious artifact? Maybe, but I don’t know (but why not? I don’t know, how then? I also don’t know). I wish I did know, but I don’t find it particularly bizarre that certain things are still unexplained and unclear. It doesn’t mean that there’s nothing interesting to be said on consciousness.

But in any case, I want to point out that the “what else” argument is a self-destructive argument. Asking “what else?” is just admitting that you have no logical argument that allows you to prove your proposition. This is not an argument, but rather an admission of failure (which is fine).

Anyway: I don’t know what it takes for something to be conscious, but I think we can be fairly confident that LLMs or deep networks are not conscious.

I recently read Philip Goff’s book, “Galileo’s error”, where he advocates for the panpsychist view that consciousness is a widespread element of reality, so that atoms are very slightly conscious (note that we should talk of panpsychisms as there are certainly variations on this theme). The book is well written and does a good job at explaining the issues with the different brands of materialism and dualism. It also pushed me to refine some concepts and revisit some issues, which is good.

However, I was not convinced by his proposal. To me, it seems that it boils down to another “what else?” argument: materialism is flawed, dualism is false, therefore panpsychism is true. This cannot be a reasonable response to the fallacious “what else?” argument of materialists: if you are not a materialist, then you are a dualist.

The argument in a few words (certainly a bit caricatured) is that consciousness cannot be reduced to the physical properties of matter, and therefore it must be an additional fundamental property of matter, in some way what atoms are “made of”. Then comes the so-called “combination problem”, which is to understand how atoms get together to form a single conscious entity, a problem that Goff admits is unsolved. This is a strikingly reductionist account of consciousness: if we are conscious, it must be because parts of us are conscious.

As mentioned above, there are different panpsychisms. Integrated Information Theory (IIT), for example, considers consciousness as a property of certain kinds of systems, not a property of atoms. What they have in common, however, is that they postulate consciousness beyond life (of course, some materialist accounts also do). Towards the end of the book, Goff speculates on the ethical issues raised by the possibility that plants might be conscious, and so is it ok to have a vegan diet? But on his account, atoms are also conscious so maybe nuclear fission makes atoms suffer? Maybe nuclear fusion feels like love to atoms and the sun is a sort of gigantic hippie paradise? Who knows.

The fundamental problem I see with the argumentation is that “being conscious” is used as a sort of empty placeholder. The theory is meant to explain subjective perspective, but never asks what a subject might be. Free will is presented as one of the difficult things to explain, but the proposal offers no account of agency. And if atoms are conscious, do they have emotions? But what is an emotion? It seems to me that a theory of consciousness should try to address at least some basic aspects of the phenomenon to explain, beyond the mere fact that it exists.

These questions about subjects and agency are addressed in particular in some corners of theoretical biology, which try to develop the concept of a living organism. Typically, theories of life describe a living organism as a set of processes whose operation ensures their own continued existence; some form of organizational closure that resists entropic decay (see e.g. Varela & Maturana; Kauffman; Rosen; more recently Moreno, Mossio, Montevil, to name a few). The concept of organism is what allows developing derived concepts such as agency, autonomy, but also some aspects of perception and emotion (namely the valence of emotion).

Goff proposes that free will is not action free from determinism, but action that has reasons. But reasons imply goals, and goals imply autonomous agents, concepts that panpsychism, whether Goff’s or IIT’s version, offers no account of. No amount of integrated information (“Phi”) provides any goal to the system, and so this theoretical construct cannot account for either free will or emotion.

I will now discuss a few more specific points made in the book.

The hard problem is harder than you might think

Everything starts with the “hard problem” of consciousness, a problem that has been discussed by many thinkers, the most well-known arguments being probably those of Chalmers and Nagel. Basically, the problem is that experience is not something that can be derived deductively from the properties of matter. There is no logical reasoning that would start with a mechanistic description of a material system, and that would end up with “therefore, this system feels pain”. Of course, there could be a “therefore” that means: whenever a material system has some property P, it feels pain, but this would be an empirical relation (e.g. the “neural correlates of consciousness”), not a logical relation. Basically, you cannot deduce pain from Newton laws.

In the book, the author takes it as a flaw of reductive materialism, the idea that consciousness is a state of matter. It certainly is a flaw of materialism, but it is much worse than that.

In his famous paper (What is it like to be a bat?), Nagel hits the more general problem: experience is something that you live, not something that you describe. Of course, you could try to describe an experience, but understanding the words is not the same thing as having the experience. Philosophers like to speak of the redness of red, but I find this a rather poor choice. Everyone (with some exceptions) has the feeling of red whenever something with the right spectral properties is presented, so it is easy to conflate objective properties of objects with subjective experience.

But consider taste, for example. Taste is interesting because not everyone has the same tastes, meaning that people can have different experiences for the same stuff, and taste can change over time. There is no need to try to imagine the ultrasonic world of bats. Consider for example coriander. Many people hate coriander, and this has been linked to some genetic variant in olfactory receptor genes. Now try to describe the taste of coriander to someone who does not like it. I personally would have no idea where to start; someone might find words to describe it, but we can be sure that the person would be essentially clueless about how it might taste. One might say that the taste of coriander is the taste caused by the activation of the corresponding olfactory receptor; one might describe the odorant molecules, and so on: none of that conveys the taste. There is just no alternative to actually experiencing an experience.

So the problem is much deeper than a flaw of materialism: the problem is that experience is not something that can be explained, whatever the number of words, it is something that is lived. Postulating the existence of an extra non-physical law, which says for example that the universe is made of consciousness, or that certain systems are conscious, will not solve this problem. We might want to postulate the existence of coriander taste as an extra law of nature, it would still not explain what it feels like to taste coriander. Thus, if panpsychism pretends to explain the quality of experience, then it commits the exact same error as reductive materialism. This should make us highly skeptical of claims such as “the quality of experience is specified by the informational relationships [the complex] generates” (Balduzzi & Tononi, 2009) - all the more that it reduces experience to its epistemic value while neglecting the hedonic value (see below).

Of course, this does not mean that nothing interesting can be said about experience, including its relation with matter. But panpsychism is not at all necessary for that. For example, sensorimotor theories of perception have something to say about the structure of experience, for example about the spatialization of pain (see my short series on that). Thus, panpsychism does not solve the hard problem. It does not dissolve it either: the problem still exists even if you consider that materialism is flawed, because it is not an issue with materialist explanations but with explanations in general.

Before I move on to the next topic, I would like to point out that there are certain ways to communicate experiences, but those are not descriptive. More accurately, there are ways to induce experiences in people. These are art, poetry, or guided meditation. A musical experience is not the end point of a logical reasoning. Guided meditation practices, or poetry, use words to put you into certain states of mind, but the words are only there to guide you (“imagine you are a mountain”), they do not constitute the experience.

The case against dualism

In his book, Goff also rejects dualism, but I found the argument rather peculiar. He claims that the main objection against dualism is empirical: we understand a lot about how the brain works, so surely we would have noticed if there were an intervention by some kind of mental entity - some scientific law would be broken, basically. This is a peculiar argument because virtually all scientific findings in neuroscience are of a statistical nature, so there is no way we would notice an intervention rather than categorize it as a random event. Our understanding of the brain is not that extensive.

But we can take the argument as a thought experiment. Suppose we have perfect understanding of the physical working of the brain, so we could for example predict the evolution of brain states from the current state. We could know before the subject herself what she is going to talk about, or the voluntary movements she is going to make. In other words, we would know perfectly how the “philosophical zombie” works. This knowledge is precisely the premises of the hard problem: we have a perfect mechanistic understanding of physiology and behavior, but none of this understanding implies that the person is conscious, or explains the taste of coriander.

If it is the case that we can have this kind of understanding (even as a theoretical possibility), then indeed dualism is false, at least the kind of dualism that supposedly explains free will and agency (as Penfield or Eccles thought).

But it also implies that there cannot be any empirical support for any panpsychist theory, since all the empirical data are already supposedly accounted for by a physicalist account. Therefore, I find it odd that Goff supports the empirical argument against dualism and at the same time claims that there is empirical support for IIT. By his own argument, that cannot be the case. If we have a physicalist account of whatever a person might say or do in different circumstances, then this will include any kind of empirical evidence that we might think of. All the empirical evidence would already be accounted for, and therefore could not count as evidence for the panpsychist theory.

To claim otherwise is to believe that consciousness is required to do certain things, such as speaking of what one sees. But this is contradictory with the claim that a physicalist framework can explain behavior but not consciousness. This is to give a causal role to consciousness, above the causal role of physical processes, and that is precisely Goff’s objection against dualism.

The epistemic and the hedonic

To end these notes, I would like to come back to the remark I made at the beginning, that a theory of consciousness should try to address at least some basic aspects of the phenomenon to explain, and not just acknowledge that it exists.

Ned Block pointed out that there are two aspects of consciousness that need to be accounted for: access consciousness, i.e. its functional aspects, and phenomenal consciousness or “what it is like”, the latter being notoriously more complicated than the former.

I would like to rephrase it as follows. Experience typically comes in at least two dimensions, an epistemic and a hedonic dimension. For example, consider toothache. That experience is informative: it aches somewhere, and perhaps you notice that it aches when you drink something cold and so on. But pain is not just information: it is also something you would like to get rid of. It has a hedonic value, namely a bad one.

Both IIT and global workspace theory focus on the epistemic aspect, but fail to account for the hedonic aspect (for the epistemic aspect as well, but at least they try). This might come from the failure to include life into the picture. For a living organism, there is some sense in saying that an event or an act is good or bad for the organism, according to whether it helps or threatens the organism’s self-maintenance. In contrast, there is no sense in saying that some event is good or bad for a photodiode. The photodiode has been built by some (other) conscious entity to perform a certain function, and so only that external entity can assess whether some transformation of the system is good or bad, according to whether it continues to perform the ascribed function.

Of course, none of this explains why an organism feels bad, rather than just acts accordingly, but it is certainly an aspect of the phenomenon that things that are bad for the organism also tend to feel bad. Without an organism, some autonomous entity for which “bad” can mean something, it seems difficult to theorize about the hedonic aspect of experience. This makes me skeptical of claims that entities that are not living organisms (atoms, photodiodes and so on) are conscious.

Integrated Information Theory (IIT) is a theory of consciousness that assigns a certain degree of consciousness (called Phi) to any system. Philosophically, it is a kind of property dualism: mental states are not reduced to states of matter, but assigned to arrangements of matter that have certain formal properties.

One of the most criticized aspects of the theory is that it embraces panpsychism: for example, it assigns consciousness to elementary systems such as a photodiode. However, the objection is easy to bypass: a theory does not have to produce intuitive claims. It is certainly not intuitive that the passing of time depends on how fast you move, yet it seems to be true.

But in the case of relativity, the counter-intuitive prediction can be tested empirically (and it has been). In the case of IIT, it cannot: there is no independent way to assess whether a photodiode is conscious. This is the fundamental epistemological issue with trying to explain a subjective phenomenon: there is no other way to directly establish the reality of the phenomenon than to experience it yourself. You will never have first-hand evidence that your stomach is another conscious entity in your body.

However, there are other ways to assess a theory, for example to exhibit incoherencies, contradictions or logical fallacies.

Before I begin, I want to point out that property dualism is much closer to materialism than to religious kinds of dualism. In IIT, mental states are really properties of matter, just not physical properties. This means that mental states depend only on the arrangement of matter: the physical system is not just a host for the mind, as in Cartesian dualism. In Buddhism, the mind can reincarnate in another body, and bring memories with it. This is impossible with property dualism, because mental states are an intrinsic property of the system.

It is precisely this point that is inconsistent with the formulation of IIT. The central concept that I will discuss is information. IIT builds on the classical notion of information as differentiation: if a state can take values A or B, then observing A is informative, because it could have been B. If we assign equal probabilities to the two alternatives, then observing the state brings exactly 1 bit of information. Information is the reduction of uncertainty. This is why a binary photodiode exposed to a visual scene can get no more than 1 bit of information, while you would get much more information from the same scene: it could have been so many different other scenes.

At first sight, this seems like a reasonable characterization of information, so that IIT considers it as a self-evident axiom. However, there is a big problem here, one that will lead to major inconsistencies. State A is informative because it excludes possibility B. But this assumes that the entity that gets informed is aware that B was a possibility. How could that be possible if it has never encountered B before, and has no cognitive resources to imagine that B could exist? The reduction in uncertainty is relative to prior knowledge, but what is the prior knowledge of a photodiode?

The confusion, of course, is that the reduction of uncertainty occurs for the observer who formalized the range of possibilities, but it is ascribed to the system itself. There is some magic operating here, where we ascribe information and uncertainty to a physical system, without ever defining what it means for a physical system to have knowledge.

Let us be more concrete. Suppose the photodiode starts its life covered by a box, but then the experimenter removes the box and puts it back from times to times. In terms of information, the state of the photodiode will hold some information about light, or if it consists of a sensor and a detector, about its previous state. According to IIT, this will make the photodiode minimally conscious (in the last version of IIT, the photodiode must have some recurrent connectivity, e.g. reciprocal connection between a sensor and a detector, but this makes no difference to the present discussion).

But consider the very beginning of the photodiode’s life. It has always been in the dark, never exposed to light. In terms of information, the state is still informative because it excludes the possibility of light, but what does that mean for the photodiode that has never seen the light?

Now consider another photodiode. It is physically exactly the same photodiode, but this time the box permanently covers it. Thus, there is only one possible state for that photodiode. According to IIT, this photodiode is not conscious. Yet, it is physically identical to the photodiode at the beginning of the previous scenario.

Thus, we have exhibited two situations where the same physical system in the exact same state is assigned two different states of consciousness. This contradicts the claim that consciousness is an intrinsic property of the system.

The problem arises from the fact that consciousness is defined not from what happens presently with the system, but from what could happen, and it is very unclear what “could” might mean. “Could” the box collapse and expose the photodiode to light? Would that make the photodiode conscious? From that event? Or from the start of the photodiode’s life? Or from the moment when that event “could” happen? What does it mean exactly that an event that does not happen “could” have happened? (if what?)

Let us now turn to brains. According to Tononi, “IIT predicts that a particular brain area can contribute to experience even if it is inactive, but not if it is inactivated.” Specifically, he contrasts the case of a person seeing a gray picture with the case where a colored picture is presented but color-sensitive neurons are lesioned. In the first case, the person perceives the picture as gray. In the second case, there is no experience of color at all. The difference in experience occurs because in the first case, the color-sensitive neurons could have been in a different state, but not in the second case.

“Could have” is very problematic here, because it does not refer to the present state of the system, but to hypothetical states.

Suppose the color area of the brain is not lesioned but reversibly inactivated by cooling, for a random duration. In this case, the person should be color conscious, because at any time, the neurons could have been in a different state.

An important precision in these thought experiments: the subject does not know what the experimenter is doing. Thus, from the viewpoint of the subject, all that happens is the neurons are inactive, and the cause of this inactivity leaves no trace in the subject. Yet, according to IIT, mental states depend on that cause. If it is a gray picture: color-conscious. Lesioning: color-unconscious. Reversible cooling: conscious. Irreversible cooling with a refrigerating device permanently implanted in the brain: unconscious. The fridge breaks down: conscious. The fridge might break down, but actually doesn’t: conscious from the time of cooling.

In summary, IIT fails to consistently assign consciousness to a system, because the definition is based on hypotheticals, which by definition are not instantiated by the system. Deep down, the troubles arise from the circularity of the definition of information as reduction of uncertainty. Uncertainty refers to a prior state of knowledge, but the notion of knowledge for the subject is never defined. In practice, the knowledge underlying the reduction of uncertainty is the knowledge of the observer who formalizes scenarios and quantifies probabilities of events that the system itself has never lived.

This post is an elaboration of the following commentary: Does the present moment depend on the moments not lived?

A theory of consciousness initially proposed by David Chalmers (in his book the Conscious Mind) is that consciousness (or experience) is a property of information processing systems. It is an additional property, not logically implied by physical laws; a new law of nature. The theory was later formalized by Giulio Tononi into Integrated Information Theory, based on Shannon’s mathematical concept of information. One important feature of this theory is it is a radical form of panpsychism: it assigns consciousness (to different degrees) to virtually anything in the world, including a thermostat.

The Bewitched experiment of thought

I have criticized IIT previously on the grounds that it fails to define in a sensible way what makes a conscious subject (eg a subsystem of a conscious entity would be another conscious entity, so for example your brain would produce an infinite number of minds). But here I want to comment specifically on the example of the thermostat. It is an interesting example brought up by Chalmers in his book. The reasoning is as follows: a human brain is conscious; a mouse brain is probably conscious, but with a somewhat lower degree (for example, no self-consciousness). As we go down the scale of information-processing systems, the system might be less and less conscious, but why would it be that there is a definite threshold for consciousness? Why would a billion neurons be conscious but not a million? Why would a million neurons be conscious but not one thousand? And how about just one neuron? How about a thermostat? A thermostat is an elementary information-processing system with just two states, so maybe, Chalmers argue, the thermostat has a very elementary form of experience.

To claim that a thermostat is conscious defies intuition, but I would not follow Searle on insisting that the theory must be wrong because it assigns consciousness to things that we wouldn’t intuitively think are conscious. As I argued in a previous post, to claim that biology tells us that only brains are conscious is to use circular arguments. We don’t know whether anything else than a brain is conscious, and since consciousness is subjective, to decide whether anything is conscious is going to involve some theoretical aspects. Nonetheless, I am skeptical that a thermostat is conscious.

I propose to examine the Bewitched experiment of thought. In the TV series Bewitched, Samantha the housewife twitches her nose and everyone freezes except her. Then she twitches her nose and everyone unfreezes, without noticing that anything happened. For them, time has effectively stopped. The question is: was anyone experiencing anything during that time? To me, it is clear that no one can experience anything if time is frozen. In fact, that whole time has not existed at all for the conscious subject. It follows that a substrate with a fixed state (e.g. hot/cold) cannot experience anything, because time is effectively frozen for that substrate. Experience requires a flow of time, a change in structure through time. I leave it open whether the interaction of the thermostat with the room might produce experience for that coupled system (see below for some further thoughts).

What is “information”?

In my view, the fallacy in the initial reasoning is to put the thermostat and the brain in the same scale. That scale is the set of information-processing systems. But as I have argued before (mostly following Gibson’s arguments), it is misleading to see the brain an information-processing system. The brain can only be seen to transform information of one kind into information of another kind by an external observer, because the very concept of information is something that makes sense to a cognitive/perceptual system. The notion of information used by IIT is Shannon information, a notion from communication theory. This is an extrinsic notion of information: for example, neural activity is informative about objects in the world in the sense that properties of those objects can be inferred from neural activity. But this is totally unhelpful to understand how the brain, which only ever gets to deal with neural signals and not things in the world, sees the world (see this argument in more detail in my paper Is coding a relevant metaphor for the brain?).

Let’s clarify with a concrete case: does the thermostat perceive temperature? The thermostat can be in different states depending on temperature, but from its perspective, there is no temperature. There are changes in state that seems to be unrelated to anything else (there is literally nothing else for the thermostat). One could replace the temperature sensor with some other sensor, or with a random number generator, and there would be literally no functional change in the thermostat itself. Only an external observer can link the thermostat’s state with temperature, so the thermostat cannot possibly be conscious of temperature.

Thus, Shannon’s notion of information is inappropriate to understand consciousness. Instead of extracting information in the sense of communication theory, what the brain might do is build models of sensory (sensorimotor) signals from its subjective perspective, in the same way as scientists make models of the world with observations (=sensory signals) and experiments (=actions). But this intrinsic notion of information, which corresponds eg to laws of physics, is crucially not what Shannon’s notion of information is. And it is also not the kind of information that a thermostat is dealing with.

This inappropriate notion of information leads to what in my view is a rather absurd quantitative scale of consciousness, according to which entities are more or less conscious along a graded scale (phi). Differences in consciousness are qualitative, not quantitative: there is dreaming, being awake, being self-conscious or not, etc. These are not different numbers. This odd analog scale arises because Shannon information is counted in bits. But information in the sense of knowledge (science) is not counted in bits; there are different kinds of knowledge, they have different structure, relations between them etc.

Subjective physics of a thermostat

But let us not throw away Chalmers’ interesting experiment of thought just now. Let us ask, following Chalmers: what does it feel like to be a thermostat? We will examine it not with Shannon’s unhelpful notion of information but with what I called “subjective physics”: the laws that govern sensory signals and their relations to actions, from the perspective of the subject. This will define my world from a functional viewpoint. Let’s say I am a conscious thermostat; a homunculus inside the thermostat. All I can observe is a binary signal. Then there is a binary action that I can make, which for an external observer corresponds to turning on the heat. What kind of world does that make to me? Let’s say I’m a scientist homunculus, what kind of laws about the world can I infer?

If I’m a conventional thermostat, then the action will be automatically triggered when the signal is in a given state (“cold”). After some time, the binary signal will switch and so will the action. So in fact there is an identity between signal and action, which means that all I really observe is just the one binary signal, switching on and off, probably with some kind of periodicity. This is the world I might experience, as a homunculus inside the thermostat (note that to experience the periodicity requires memory, which a normal thermostat doesn’t have). In a way, I’m a “locked-in” thermostat: I can make observations, but I cannot freely act.

Let’s say that I am not locked-in and have a little more free will, so I can decide whether to act (heat) or not. If I can, then my world is a little bit more interesting: my action can trigger a switch of the binary signal, after some latency (again requiring some memory), and then when I stop, the binary signal switches back, after a time that depends on how much time my previous action lasted. So here I have a world that is much more structured, with relatively complex laws which in a way defines the concept of “temperature” from the perspective of the thermostat.

So if a thermostat were conscious, then we have a rough idea of the kind of world it might experience (although not how it feels like), and even in this elementary example, you can’t measure these experiences in bits - let alone the fact that a thermostat is not conscious anyway.

Here I want to stir some ideas about the substrate of consciousness. Let us start with a few intuitive ideas: a human brain is conscious; an animal brain is probably conscious; a stone is not conscious; my stomach is not conscious; a single neuron or cell is not conscious; the brainstem or the visual cortex is not a separate conscious entity; two people do not form a single conscious entity.

Many of these ideas are in fact difficult to justify. Let us start with single cells. To see the problem, think first of organisms that consist of a single cell. For example, bacteria, or ciliates. In this video, an amoeba’s engulfs and then digests two paramecia. At some point, you can see the paramecia jumping all around as if they were panicking. Are these paramecia conscious, do they feel anything? If I did not know anything about their physiology or size, my first intuition would be that they do feel something close to fear. However, knowing that these are unicellular organisms and therefore do not have a nervous system, my intuition is rather that they are not actually conscious. But why?

Why do we think a nervous system is necessary for consciousness? One reason is that organisms to which we ascribe consciousness (humans and animals, or at least some animals) all have a nervous system. But it’s a circular argument, which has no logical validity. A more convincing reason is that in humans, the brain is necessary and sufficient for consciousness. A locked-in patient is still conscious. On the other hand, any large brain lesion has an impact on conscious experience, and specific experiences can be induced by electrical stimulation of the brain.

However, this tends to prove that the brain is the substrate of my experience, but it says nothing about, say, the stomach. The stomach also has a nervous system, it receives sensory signals and controls muscles. If it were conscious, I could not experience it, by definition, since you can only experience your own consciousness. So it could also be, just as for the brain, that the stomach is sufficient and necessary for consciousness of the gut mind: perhaps if you stimulate it electrically, it triggers some specific experience. As ridiculous as it might sound, I cannot discard the idea that the stomach is conscious just because I don’t feel that it’s conscious; I will need arguments of a different kind.

I know I am conscious, but I do not know whether there are other conscious entities in my body. Unfortunately, this applies not just to the stomach, but more generally to any other component of my body, whether it has a nervous system or not. What tells me that the liver is not conscious? Imagine I am a conscious liver. From my perspective, removing one lung, or a foot, or a large part of the visual cortex, has no effect on my conscious experience. So the fact that the brain is necessary and sufficient for your conscious experience doesn’t rule out the fact that some other substrate is necessary and sufficient for the conscious experience of another entity in your body. Now I am not saying that the question of liver consciousness is undecidable, only that we will need more subtle arguments than those exposed so far (discussed later).

Let us come back to the single cell. Although I feel that a unicellular organism is not conscious because it doesn’t have a nervous system, so far I have no valid argument for this intuition. In addition, it turns out that Paramecium, as many other unicellular organism including (at least some) bacteria, is an excitable cell with voltage-gated channels, structurally very similar to a neuron. So perhaps it has some limited form of consciousness after all. If this is true, then I would be inclined to say that all unicellular organisms are also conscious, for example bacteria. But then what about a single cell (eg a neuron) in your body, is it conscious? One might object that a single cell in a multicellular organism is not an autonomous organism. To address this objection, I will go one level below the cell.

Eukaryotic cells (eg your cells) have little energy factories called mitochondria. It turns out that mitochondria are in fact bacteria which have been engulfed in cells a very long (evolutionary) time ago. They have their own DNA, but they now live and reproduce inside cells. This is a case of endosymbiosis. If mitochondria were conscious before they lived in cells, why would they have lost consciousness when they started living in cells? So if we think bacteria are conscious, then we must admit that we have trillions of conscious entities in the cells of our body – not counting the bacteria in our digestive system. The concept of an autonomous organism is an illusion: any living organism depends on interactions with an ecosystem, and that ecosystem might well be a cell or a multicellular organism.

By the same argument, if we think unicellular organisms are conscious, then single neurons should be conscious, as well as all single cells in our body. This is not exclusive of the brain being conscious as a distinct entity.

A plausible alternative, of course, is that single cells are not conscious, although I have not yet proposed a good argument for this alternative. Before we turn to a new question, I will let you contemplate the fact that bacteria can form populations that are tightly coupled by electrical communication. Does this make a bacteria colony conscious?

Let us now turn to another question. We can imagine that a cell is somehow minimally conscious, and that at the same time a brain forms a conscious entity of a different nature. Of course it might not be true, but there is a case for that argument. So now let us consider two people living their own life on opposite sides of the planet. Can this pair form a new conscious entity? Here, there are arguments to answer negatively. This is related to a concept called the unity of consciousness.

Suppose I see a red book. In the brain, some areas might respond to the color and some other areas might respond to the shape. It could be then that the color area experiences redness, and the shape area experience bookness. But I, as a single conscious unit, experiences a red book as a whole. Now if we consider two entities that do not interact, then there cannot be united experiences: somehow the redness and the bookness must be put together. So the substrate of a conscious entity cannot be made of parts that do not interact with the rest. Two separated people cannot form a conscious entity. But this does not rule out the possibility that two closely interacting people may not form a conscious superentity. Again, I do not believe this is the case, but we need to find new arguments to rule this out.

Now we finally have something a little substantial: a conscious entity must be made of components in interaction. From this idea follow a few remarks. First, consciousness is not a property of a substrate, but of the activity of a substrate (see a previous blog post on this idea). For example, if we freeze the brain in a particular state, it is not conscious. This rules out a number of inanimate objects (rocks) as conscious. Second, interactions take place in time. For example, it takes some time, up to a few tens of ms, for an action potential to travel from one neuron to another. This implies that a 1 ms time window cannot enclose a conscious experience. The “grain” of consciousness for a human brain should thus be no less than a few tens of milliseconds. In the same way, if a plant is conscious, then that consciousness cannot exist on a short timescale. This puts a constraint on the kind of experiences that can be ascribed to a particular substrate. Does consciousness require a nervous system? Maybe it doesn’t, but at least for large organisms, a nervous system is required to produce experiences on a short timescale.

I want to end with a final question. We are asking what kind of substrate gives rise to consciousness. But does consciousness require a fixed substrate? After all, the brain is dynamic. Synapses appear and disappear all the time, all the proteins get renewed regularly. The brain is literally a different set of molecules and a different structure from one day to the next. But the conscious entity remains. Or at least it seems so. This is what Buddhists call the illusion of self: contrary to your intuition, you are not the same person today and ten years ago; the self has no objective permanent existence. However, we can say that there is a continuity in conscious experience. That continuity, however, does not rely on a fixed material basis but more likely on some continuity of the underlying activity. Imagine for example a fictional worm that is conscious, but the substrate of consciousness is local. At some point it is produced by the interaction of neurons at some particular place of the nervous system, then that activity travels along the worm’s spine. The conscious entity remains and doesn’t feel like it’s travelling, it is simply grounded on a dynamic substrate.

Now I don’t think that this is true of the brain (or of the worm), but rather that long-range synchronization has something to do with the generation of a global conscious entity. However, it is conceivable that different subsets of neurons, even though they might span the same global brain areas, are involved in conscious experience at different times. In fact, this is even plausible. Most neurons don’t fire much, perhaps a few Hz on average. But one can definitely have a definite conscious experience over a fraction of second, and that experience thus can only involve the interaction of a subset of all neurons. We must conclude that the substrate of consciousness is actually not fixed but involve dynamic sets of neurons.

A summary of these remarks. I certainly have raised more questions than I have answered. In particular, it is not clear whether a single cell or a component of the nervous system (stomach, brainstem) is conscious. However, I have argued that: 1) any conscious experience requires the interaction of the components that produce it, and this interaction takes place in time; 2) the set of components that are involved in any particular experience is dynamic, despite the continuity in conscious experience.

By using the word “sex” in the title, I am obviously trying to increase the number of my readers. But not only. I noticed that many people, in particular scientists, do not seem to get why there is a problem at all. Philosophers like to refer to the experience of color, and the question is then: why is it that red feels like red, and not like something else, and why does it feel like anything at all? Why is it not just information as it is for a computer, why does the experience have a particular quality? Somehow this example does not speak to everyone, so I came with another example: why is it that sex is associated with pleasure, in particular, why is male ejaculation generally associated with orgasm?

You will hear several kinds of explanation. One might tell you: your body secretes some particular molecules at that moment, or some particular neuron gets excited, and that's what produces the sensation of pleasure. But of course, this explanation just pushes the problem a little further, by replacing “ejaculation” by “molecule” or “neuron”. Why is it that a particular molecule produces pleasure? Certainly, when it is not in the brain, the molecule does not provide pleasure to a Petri dish.

Another kind of explaining away is the functionalist or behaviorist view: if everything appears the same from an external observer point of view, then it is the same; there is nothing more than what can be seen. In other words, male orgasm is the act of ejaculating, end of the story. But sex does not fit so well with that view. First, it is well known that female orgasm can be faked; second, male orgasm can be dissociated from ejaculation (in tantric sex).

And finally there is the evolutionary explanation: we feel pleasure because it motivates us for sex, which is necessary for reproduction. But the logic is flawed: we only need a mechanism to make us have sex, but there is no intrinsic reason why that mechanism should be accompanied with any feeling. Why are we not reproducing machines without feelings?

Here comes the example. The axolotl is a kind of salamander that retains its larval traits throughout its adult life. It's basically a cute baby salamander. Nevertheless, it is able to reproduce. The way it reproduces is interesting (probably not different from other salamanders, but cuter). The male lays a sort of jelly full of spermatozoids on the floor. And then later the female comes and inserts the jelly in her belly. After a few days, the female produces eggs.

Now the big question: does the male axolotl have an orgasm when it lays its eggs on the floor?

In the last decade, a number of neuroscientists have become interested in the question of consciousness. For example Christof Koch, Stanilas Dehaene, Gerald Edelman, and many others. There have been a number of interesting new insights on this old subject, mostly focused on the so-called “neural correlates of consciousness”, that is, the properties of neural activity that are associated with conscious states, as opposed to say coma. However, to my mind there is no convincing theory that explains what is consciousness, why we are conscious at all and why we feel anything at all (phenomenal consciousness). But there have been attempts. A recent one is the integrated information theory (IIT) proposed by Tononi, which proposes that consciousness is a property of all systems that have a high level of “integrated information”. In a nutshell, such a system is a dynamical system that cannot be divided into smaller independent (or weakly dependent) systems. The term “information” should be understood in the sense of information theory: how much information (uncertainty reduction) there is in the state of a subsystem about the future of another subsystem. In a nutshell, the problem with this theory is that it is as much about consciousness as information theory is about information.

Christof Koch is a notorious fan of IIT. He describes the theory in a popular science article entitled “Ubiquitous minds” (available on his web page). I should point out that it is not an academic paper, so maybe my criticisms will seem unfair. So to be fair, let us say that what follows is a criticism of the arguments in that article, but perhaps not of Koch's thought in general (admittedly I have not read his book yet).

Koch correctly presents IIT as a modern form of panpsychism, that is, the idea that lots of things are conscious to some level. Animals, of course, but also any kind of system, living or not, that has high “integrated information” (named “phi”). On his blog, Scott Aaronson, a theoretical computer scientist, gives an example of a matrix multiplication system that has this property and therefore should be highly conscious according to IIT if it where physically implemented. Now Tononi and Koch do not see this counter-intuitive implication as a problem with the theory, but on the contrary they embrace it as a highly interesting implication. Koch speculates for example that the internet might be conscious.

Koch starts by describing the naïve version of panpsychism, which indeed can easily be dismissed. Naïve panpsychism states that everything is conscious, to different levels: a brain, a tree, a rock. This immediately raises a big problem (refered to as the “problem of aggregates” in the article): you might claim that everything is conscious, but then you need to define what a “thing” is. Is half a rock conscious? Then which half? Is any set of 1000 particles randomly chosen in the universe conscious? Is half of my brain plus half of your stomach a conscious entity?

IIT is more restricted than naïve panpsychism, but it suffers from the same problem: how do you define a “system”? Wouldn't a subsystem of a conscious system also be conscious, according to the theory? As Koch writes, the theory offers no intrinsic solution to this problem, it must be augmented by an ad hoc postulate (“that only “local maxima” of integrated information exist”). What puzzles me is that the paper ends on the claim that IIT offers an “elegant explanation for [the existence of] subjective experience”. What I have read here is an interesting theory of interdependence in systems, and then a claim that systems made of interdependent parts are conscious. Where is the explanation in that? A word (“consciousness”) was arbitrarily put onto this particular property of systems, but no hint was provided at any point about a connection between the meaning of that word and the property those systems. Why would this property produce consciousness? No explanation is given by the theory.

If it is not an explanation, then it must simply be a hypothesis; the hypothesis that systems with high integrated information are conscious. That is, it is a hypothesis about which systems are conscious and which are not. As we noted above, this hypothesis assigns consciousness to non-living things, possibly including the internet, and definitely including some rather stupid machines that no one would consider conscious. I would consider this a problem, but tenants of IIT would simply adopt panpsychism and consider that, counter-intuitively, those things are actually conscious. But then this means admitting that no observation whatsoever can give us any hint about what systems are conscious (contrary to the first pages of Koch's article, where he argues that animals are conscious on those grounds); in other words, that the hypothesis is metaphysical and not testable. So the hypothesis is either unscientific or wrong.

Now I am not saying that the theory is uninteresting. I simply think that it is a theory about consciousness and not of consciousness. What about is it exactly? Let us go back to what integrated information is supposed to mean. Essentially, high integrated information means that the system cannot be subdivided in two independent systems – the future state of system A depends on the current state of system B and conversely. This corresponds to an important property of consciousness: the unity of consciousness. You experience a single stream of consciousness that integrates sound, vision, etc. Sound and vision are not experienced by two separate minds but by a single one. Yet this is what should happen if there were two unconnected brain areas dealing with sound and light. Thus a necessary condition for a unique conscious experience is that the substrate of consciousness cannot be divided into causally independent subsets. This is an important requirement, and therefore I do think that the theory has interesting things to say about consciousness, in particular what its substrate is, but it explains nothing about why there is a conscious experience at all. It provides a necessary condition for consciousness – and that's already quite good for a theory about consciousness.

But that's it. It does not explain why an interdependent system should be conscious – and in fact, given some examples of such systems, it seems unlikely that it is the case. What is missing in the theory? I hinted at it in my introduction: the problem with integrated information theory is that it is as much about consciousness as information theory is about information. The word “information” in information theory has little to do with information in the common sense of the word, that is, something that carries meaning for the receiver. But information theory is actually better described as a theory of communication. In fact, one should remember that Shannon's seminal paper was entitled “A Mathematical Theory of Communication”, not of information. In a communication channel, A is encoded into B by a dictionary, and B carries information about A insofar as one can recover A from B. But of course it only makes sense for the person sitting at the receiving end of the communication channel if 1) she has the dictionary, 2) A already makes sense to her. “Information” theory says nothing about how A acquires any meaning at all, it is just about the communication of information. For this reason, “integrated information” fails to address another important aspect of consciousness, which in philosophy is named “intentionality”: the idea that one is always conscious of something, i.e. consciousness has a “content” - not just a “quantity of consciousness”. Any theory that is solely based on information in Shannon's sense (dictionary) cannot say much about phenomenonal consciousness (how it feels like).

For the end of this post, I will simply quote Scott Aaronson:
“But let me end on a positive note. In my opinion, the fact that Integrated Information Theory is wrong—demonstrably wrong, for reasons that go to its core—puts it in something like the top 2% of all mathematical theories of consciousness ever proposed. Almost all competing theories of consciousness, it seems to me, have been so vague, fluffy, and malleable that they can only aspire to wrongness.”

Perceiving space is knowing where things are in the world. Or is it?

I am sitting in my living room, and there are big windows on a courtyard. The windows are sound-proof and so if I open just one, acoustical waves mostly enter the room through that window. Now someone enters the courtyard on the right, walks across it and arrives at the door on the left. If I close my eyes, I know that the person is walking from right to left. However, what I hear is the sound of someone walking, always coming from the same direction, that of the window. If someone asks me where the person is at a given moment time, I could point to the more or less correct direction, by inference. But this is not what I perceive. I always perceive the sound coming from the same direction. There is a difference between perceiving (phenomenological) and knowing (conceptual). And there is a difference between phenomenology and behavior.

Another striking example is referred pain. Referred pain is a pain that one feels at a location away from the cause of the injury. For example, in a heart attack, one may feel pain in the arm rather than in the chest. This is a known phenomenon and if you know it, you may correctly identify the location of injury in the heart when you feel pain in the arm. But it doesn't change the fact that you feel pain in the arm. You may entirely convince yourself that the injury is in the heart, and all your behavior might be consistent with that belief, but still you will feel the pain in the arm.

There are several interesting conclusions we can draw from these remarks. First, perception is not entirely reducible to behavior. Here we are touching the hard problem of consciousness (qualia): you could observe a cat turning its head to a sound source and you would think that the cat perceives that the sound came from the source, but in reality you don't know. Maybe the cat perceives it somewhere else but it corrects its movement because it knows its perception tends to be biased. With humans, you could perhaps distinguish between these possibilities because humans speak. But without this option, a purely functionalist approach to perception (in terms of relationships between sensory stimuli and behavior) misses part of the phenomenon.

Second, inference is not the same as perception. Spatial perception is not just the process of inferring where something is from sensory inputs. There is also the experience of perception, which is not captured by the objectivist view.