Notes on consciousness. (VIII) Is a thermostat conscious?

A theory of consciousness initially proposed by David Chalmers (in his book the Conscious Mind) is that consciousness (or experience) is a property of information processing systems. It is an additional property, not logically implied by physical laws; a new law of nature. The theory was later formalized by Giulio Tononi into Integrated Information Theory, based on Shannon’s mathematical concept of information. One important feature of this theory is it is a radical form of panpsychism: it assigns consciousness (to different degrees) to virtually anything in the world, including a thermostat.

The Bewitched experiment of thought

I have criticized IIT previously on the grounds that it fails to define in a sensible way what makes a conscious subject (eg a subsystem of a conscious entity would be another conscious entity, so for example your brain would produce an infinite number of minds). But here I want to comment specifically on the example of the thermostat. It is an interesting example brought up by Chalmers in his book. The reasoning is as follows: a human brain is conscious; a mouse brain is probably conscious, but with a somewhat lower degree (for example, no self-consciousness). As we go down the scale of information-processing systems, the system might be less and less conscious, but why would it be that there is a definite threshold for consciousness? Why would a billion neurons be conscious but not a million? Why would a million neurons be conscious but not one thousand? And how about just one neuron? How about a thermostat? A thermostat is an elementary information-processing system with just two states, so maybe, Chalmers argue, the thermostat has a very elementary form of experience.

To claim that a thermostat is conscious defies intuition, but I would not follow Searle on insisting that the theory must be wrong because it assigns consciousness to things that we wouldn’t intuitively think are conscious. As I argued in a previous post, to claim that biology tells us that only brains are conscious is to use circular arguments. We don’t know whether anything else than a brain is conscious, and since consciousness is subjective, to decide whether anything is conscious is going to involve some theoretical aspects. Nonetheless, I am skeptical that a thermostat is conscious.

I propose to examine the Bewitched experiment of thought. In the TV series Bewitched, Samantha the housewife twitches her nose and everyone freezes except her. Then she twitches her nose and everyone unfreezes, without noticing that anything happened. For them, time has effectively stopped. The question is: was anyone experiencing anything during that time? To me, it is clear that no one can experience anything if time is frozen. In fact, that whole time has not existed at all for the conscious subject. It follows that a substrate with a fixed state (e.g. hot/cold) cannot experience anything, because time is effectively frozen for that substrate. Experience requires a flow of time, a change in structure through time. I leave it open whether the interaction of the thermostat with the room might produce experience for that coupled system (see below for some further thoughts).

What is “information”?

In my view, the fallacy in the initial reasoning is to put the thermostat and the brain in the same scale. That scale is the set of information-processing systems. But as I have argued before (mostly following Gibson’s arguments), it is misleading to see the brain an information-processing system. The brain can only be seen to transform information of one kind into information of another kind by an external observer, because the very concept of information is something that makes sense to a cognitive/perceptual system. The notion of information used by IIT is Shannon information, a notion from communication theory. This is an extrinsic notion of information: for example, neural activity is informative about objects in the world in the sense that properties of those objects can be inferred from neural activity. But this is totally unhelpful to understand how the brain, which only ever gets to deal with neural signals and not things in the world, sees the world (see this argument in more detail in my paper Is coding a relevant metaphor for the brain?).

Let’s clarify with a concrete case: does the thermostat perceive temperature? The thermostat can be in different states depending on temperature, but from its perspective, there is no temperature. There are changes in state that seems to be unrelated to anything else (there is literally nothing else for the thermostat). One could replace the temperature sensor with some other sensor, or with a random number generator, and there would be literally no functional change in the thermostat itself. Only an external observer can link the thermostat’s state with temperature, so the thermostat cannot possibly be conscious of temperature.

Thus, Shannon’s notion of information is inappropriate to understand consciousness. Instead of extracting information in the sense of communication theory, what the brain might do is build models of sensory (sensorimotor) signals from its subjective perspective, in the same way as scientists make models of the world with observations (=sensory signals) and experiments (=actions). But this intrinsic notion of information, which corresponds eg to laws of physics, is crucially not what Shannon’s notion of information is. And it is also not the kind of information that a thermostat is dealing with.

This inappropriate notion of information leads to what in my view is a rather absurd quantitative scale of consciousness, according to which entities are more or less conscious along a graded scale (phi). Differences in consciousness are qualitative, not quantitative: there is dreaming, being awake, being self-conscious or not, etc. These are not different numbers. This odd analog scale arises because Shannon information is counted in bits. But information in the sense of knowledge (science) is not counted in bits; there are different kinds of knowledge, they have different structure, relations between them etc.

Subjective physics of a thermostat

But let us not throw away Chalmers’ interesting experiment of thought just now. Let us ask, following Chalmers: what does it feel like to be a thermostat? We will examine it not with Shannon’s unhelpful notion of information but with what I called “subjective physics”: the laws that govern sensory signals and their relations to actions, from the perspective of the subject. This will define my world from a functional viewpoint. Let’s say I am a conscious thermostat; a homunculus inside the thermostat. All I can observe is a binary signal. Then there is a binary action that I can make, which for an external observer corresponds to turning on the heat. What kind of world does that make to me? Let’s say I’m a scientist homunculus, what kind of laws about the world can I infer?

If I’m a conventional thermostat, then the action will be automatically triggered when the signal is in a given state (“cold”). After some time, the binary signal will switch and so will the action. So in fact there is an identity between signal and action, which means that all I really observe is just the one binary signal, switching on and off, probably with some kind of periodicity. This is the world I might experience, as a homunculus inside the thermostat (note that to experience the periodicity requires memory, which a normal thermostat doesn’t have). In a way, I’m a “locked-in” thermostat: I can make observations, but I cannot freely act.

Let’s say that I am not locked-in and have a little more free will, so I can decide whether to act (heat) or not. If I can, then my world is a little bit more interesting: my action can trigger a switch of the binary signal, after some latency (again requiring some memory), and then when I stop, the binary signal switches back, after a time that depends on how much time my previous action lasted. So here I have a world that is much more structured, with relatively complex laws which in a way defines the concept of “temperature” from the perspective of the thermostat.

So if a thermostat were conscious, then we have a rough idea of the kind of world it might experience (although not how it feels like), and even in this elementary example, you can’t measure these experiences in bits - let alone the fact that a thermostat is not conscious anyway.

Notes on consciousness. (VII) The substrate of consciousness

Here I want to stir some ideas about the substrate of consciousness. Let us start with a few intuitive ideas: a human brain is conscious; an animal brain is probably conscious; a stone is not conscious; my stomach is not conscious; a single neuron or cell is not conscious; the brainstem or the visual cortex is not a separate conscious entity; two people do not form a single conscious entity.

Many of these ideas are in fact difficult to justify. Let us start with single cells. To see the problem, think first of organisms that consist of a single cell. For example, bacteria, or ciliates. In this video, an amoeba’s engulfs and then digests two paramecia. At some point, you can see the paramecia jumping all around as if they were panicking. Are these paramecia conscious, do they feel anything? If I did not know anything about their physiology or size, my first intuition would be that they do feel something close to fear. However, knowing that these are unicellular organisms and therefore do not have a nervous system, my intuition is rather that they are not actually conscious. But why?

Why do we think a nervous system is necessary for consciousness? One reason is that organisms to which we ascribe consciousness (humans and animals, or at least some animals) all have a nervous system. But it’s a circular argument, which has no logical validity. A more convincing reason is that in humans, the brain is necessary and sufficient for consciousness. A locked-in patient is still conscious. On the other hand, any large brain lesion has an impact on conscious experience, and specific experiences can be induced by electrical stimulation of the brain.

However, this tends to prove that the brain is the substrate of my experience, but it says nothing about, say, the stomach. The stomach also has a nervous system, it receives sensory signals and controls muscles. If it were conscious, I could not experience it, by definition, since you can only experience your own consciousness. So it could also be, just as for the brain, that the stomach is sufficient and necessary for consciousness of the gut mind: perhaps if you stimulate it electrically, it triggers some specific experience. As ridiculous as it might sound, I cannot discard the idea that the stomach is conscious just because I don’t feel that it’s conscious; I will need arguments of a different kind.

I know I am conscious, but I do not know whether there are other conscious entities in my body. Unfortunately, this applies not just to the stomach, but more generally to any other component of my body, whether it has a nervous system or not. What tells me that the liver is not conscious? Imagine I am a conscious liver. From my perspective, removing one lung, or a foot, or a large part of the visual cortex, has no effect on my conscious experience. So the fact that the brain is necessary and sufficient for your conscious experience doesn’t rule out the fact that some other substrate is necessary and sufficient for the conscious experience of another entity in your body. Now I am not saying that the question of liver consciousness is undecidable, only that we will need more subtle arguments than those exposed so far (discussed later).

Let us come back to the single cell. Although I feel that a unicellular organism is not conscious because it doesn’t have a nervous system, so far I have no valid argument for this intuition. In addition, it turns out that Paramecium, as many other unicellular organism including (at least some) bacteria, is an excitable cell with voltage-gated channels, structurally very similar to a neuron. So perhaps it has some limited form of consciousness after all. If this is true, then I would be inclined to say that all unicellular organisms are also conscious, for example bacteria. But then what about a single cell (eg a neuron) in your body, is it conscious? One might object that a single cell in a multicellular organism is not an autonomous organism. To address this objection, I will go one level below the cell.

Eukaryotic cells (eg your cells) have little energy factories called mitochondria. It turns out that mitochondria are in fact bacteria which have been engulfed in cells a very long (evolutionary) time ago. They have their own DNA, but they now live and reproduce inside cells. This is a case of endosymbiosis. If mitochondria were conscious before they lived in cells, why would they have lost consciousness when they started living in cells? So if we think bacteria are conscious, then we must admit that we have trillions of conscious entities in the cells of our body – not counting the bacteria in our digestive system. The concept of an autonomous organism is an illusion: any living organism depends on interactions with an ecosystem, and that ecosystem might well be a cell or a multicellular organism.

By the same argument, if we think unicellular organisms are conscious, then single neurons should be conscious, as well as all single cells in our body. This is not exclusive of the brain being conscious as a distinct entity.

A plausible alternative, of course, is that single cells are not conscious, although I have not yet proposed a good argument for this alternative. Before we turn to a new question, I will let you contemplate the fact that bacteria can form populations that are tightly coupled by electrical communication. Does this make a bacteria colony conscious?

Let us now turn to another question. We can imagine that a cell is somehow minimally conscious, and that at the same time a brain forms a conscious entity of a different nature. Of course it might not be true, but there is a case for that argument. So now let us consider two people living their own life on opposite sides of the planet. Can this pair form a new conscious entity? Here, there are arguments to answer negatively. This is related to a concept called the unity of consciousness.

Suppose I see a red book. In the brain, some areas might respond to the color and some other areas might respond to the shape. It could be then that the color area experiences redness, and the shape area experience bookness. But I, as a single conscious unit, experiences a red book as a whole. Now if we consider two entities that do not interact, then there cannot be united experiences: somehow the redness and the bookness must be put together. So the substrate of a conscious entity cannot be made of parts that do not interact with the rest. Two separated people cannot form a conscious entity. But this does not rule out the possibility that two closely interacting people may not form a conscious superentity. Again, I do not believe this is the case, but we need to find new arguments to rule this out.

Now we finally have something a little substantial: a conscious entity must be made of components in interaction. From this idea follow a few remarks. First, consciousness is not a property of a substrate, but of the activity of a substrate (see a previous blog post on this idea). For example, if we freeze the brain in a particular state, it is not conscious. This rules out a number of inanimate objects (rocks) as conscious. Second, interactions take place in time. For example, it takes some time, up to a few tens of ms, for an action potential to travel from one neuron to another. This implies that a 1 ms time window cannot enclose a conscious experience. The “grain” of consciousness for a human brain should thus be no less than a few tens of milliseconds. In the same way, if a plant is conscious, then that consciousness cannot exist on a short timescale. This puts a constraint on the kind of experiences that can be ascribed to a particular substrate. Does consciousness require a nervous system? Maybe it doesn’t, but at least for large organisms, a nervous system is required to produce experiences on a short timescale.

I want to end with a final question. We are asking what kind of substrate gives rise to consciousness. But does consciousness require a fixed substrate? After all, the brain is dynamic. Synapses appear and disappear all the time, all the proteins get renewed regularly. The brain is literally a different set of molecules and a different structure from one day to the next. But the conscious entity remains. Or at least it seems so. This is what Buddhists call the illusion of self: contrary to your intuition, you are not the same person today and ten years ago; the self has no objective permanent existence. However, we can say that there is a continuity in conscious experience. That continuity, however, does not rely on a fixed material basis but more likely on some continuity of the underlying activity. Imagine for example a fictional worm that is conscious, but the substrate of consciousness is local. At some point it is produced by the interaction of neurons at some particular place of the nervous system, then that activity travels along the worm’s spine. The conscious entity remains and doesn’t feel like it’s travelling, it is simply grounded on a dynamic substrate.

Now I don’t think that this is true of the brain (or of the worm), but rather that long-range synchronization has something to do with the generation of a global conscious entity. However, it is conceivable that different subsets of neurons, even though they might span the same global brain areas, are involved in conscious experience at different times. In fact, this is even plausible. Most neurons don’t fire much, perhaps a few Hz on average. But one can definitely have a definite conscious experience over a fraction of second, and that experience thus can only involve the interaction of a subset of all neurons. We must conclude that the substrate of consciousness is actually not fixed but involve dynamic sets of neurons.

A summary of these remarks. I certainly have raised more questions than I have answered. In particular, it is not clear whether a single cell or a component of the nervous system (stomach, brainstem) is conscious. However, I have argued that: 1) any conscious experience requires the interaction of the components that produce it, and this interaction takes place in time; 2) the set of components that are involved in any particular experience is dynamic, despite the continuity in conscious experience.

Notes on consciousness. (VI) The hard problem of consciousness explained with sex

By using the word “sex” in the title, I am obviously trying to increase the number of my readers. But not only. I noticed that many people, in particular scientists, do not seem to get why there is a problem at all. Philosophers like to refer to the experience of color, and the question is then: why is it that red feels like red, and not like something else, and why does it feel like anything at all? Why is it not just information as it is for a computer, why does the experience have a particular quality? Somehow this example does not speak to everyone, so I came with another example: why is it that sex is associated with pleasure, in particular, why is male ejaculation generally associated with orgasm?

You will hear several kinds of explanation. One might tell you: your body secretes some particular molecules at that moment, or some particular neuron gets excited, and that's what produces the sensation of pleasure. But of course, this explanation just pushes the problem a little further, by replacing “ejaculation” by “molecule” or “neuron”. Why is it that a particular molecule produces pleasure? Certainly, when it is not in the brain, the molecule does not provide pleasure to a Petri dish.

Another kind of explaining away is the functionalist or behaviorist view: if everything appears the same from an external observer point of view, then it is the same; there is nothing more than what can be seen. In other words, male orgasm is the act of ejaculating, end of the story. But sex does not fit so well with that view. First, it is well known that female orgasm can be faked; second, male orgasm can be dissociated from ejaculation (in tantric sex).

And finally there is the evolutionary explanation: we feel pleasure because it motivates us for sex, which is necessary for reproduction. But the logic is flawed: we only need a mechanism to make us have sex, but there is no intrinsic reason why that mechanism should be accompanied with any feeling. Why are we not reproducing machines without feelings?

Here comes the example. The axolotl is a kind of salamander that retains its larval traits throughout its adult life. It's basically a cute baby salamander. Nevertheless, it is able to reproduce. The way it reproduces is interesting (probably not different from other salamanders, but cuter). The male lays a sort of jelly full of spermatozoids on the floor. And then later the female comes and inserts the jelly in her belly. After a few days, the female produces eggs.

Now the big question: does the male axolotl have an orgasm when it lays its eggs on the floor?

Notes on consciousness. (V) 4 key questions about consciousness and the mind-body problem

It is fair to say that we have little idea about how neural activity gives rise to consciousness, and about the relationship between neural activity and conscious states (i.e., what you are experiencing). This is the mind-body problem. In my opinion, there has been relatively little fundamental progress on this question because it has been addressed mainly within the computationalist framework (ie in terms of information processing), which is very inappropriate for this question (this is partly Chalmers' criticism). So below I am listing a number of unanswered questions on this matter, which I believe requires a very different kind of approach. First of all, let me remark that because being conscious is always being conscious of something, understanding consciousness is largely about understanding perception at the phenomenal level (perception in the broadest sense, e.g., perceiving your thoughts).

1) How can perception be stable?

Why is it that a pure tone feels like a stable percept when 1) the acoustic wave is time-varying, 2) the activity of neurons everywhere in the brain is dynamic? The same can be said of all senses; in vision, the eyes move at high frequency even when fixating an object, and there is no visual percept if they are forced to be still. More generally: if there is a mapping between states of the brain and percepts, then why is it that percepts are not changing all the time?

A thought experiment. Imagine the state of the brain is held fixed. Someone scratches her nose and time is stopped. Would you still experience something? Any conscious experience seems to require a change, not just a state. This suggests that the relevant mapping is actually not from brain states to percepts, but from brain activity to percepts. This immediately raises a problem, because a conscious state can be defined at any point in time, but it is not immediate that brain activity can (as this would reduce activity to state). This is not a fatal problem, though, for there is a precedent in physics: a gas is composed of individual particles, but the pressure of a gas at a given instant cannot be defined as a function of the state of the particles at that moment, because pressure corresponds to the force exerted by the particles impacting a surface. It might be that the relation between neural activity and conscious states is of a similar kind as the relation between mechanics and thermodynamics.

Two more thoughts experiments. 1) Record the firing of all neurons in the brain, then play them on a set of unconnected light diodes, does that set feel the same experience? 2) (adapted from Chalmers) Replace randomly every other neuron in the brain by an artificial neuron that interacts with other neurons in exactly same way as the neuron it replaces, would there be a conscious experience? My personal answers would be: (1) no and (2) yes, and this suggests to me that the right substrate to look at is not neural activity as a state (e.g. firing rates of all neurons) but neural activity as an interaction between neurons.


2) What is time for a conscious brain?

A fundamental property of consciousness is its unity: a single conscious entity sees, hears and thinks. If visual and auditory areas where independent and, say, control speech, then one conscious entity would report visual experience and another conscious entity would report auditory experience. It could not be a single conscious entity since the two relevant parts are physically disconnected. Thus the unity of consciousness requires an interdependence between all the elements that compose it. This is, as I understand it, the issue that is addressed by a number of biological theories of consciousness, for example Edelman's “reentrant loops” or Tononi's integrated information theory.

However, as far as I know, there is another crucial aspect to this problem, which is the unity of consciousness, or lack of it, in time. There is no general unity of consciousness across time: two things that happen at, say, 1 minute of interval produce distinct percepts, not a single one. Clearly, consciousness is dynamic. But the big question is: how can there be a unique conscious state at any given moment in time when all the elements of the conscious network interact with some delay (since they are physical elements), typically of a few milliseconds? And what is time for such a network? Imagine there is a (physical) visual event at time t1 and an auditory event at time t2. At what time do they occur for the network, as they are sensed at different times by all its elements?Why is it that electricity changes on a millisecond timescale in the brain but conscious states seem to change at a much slower rate?


3) How can there be an intrinsic relation between neural activity and percepts?

Why is it that a particular pattern of neural activity produces the experience of redness? Most biological explanations are of this kind: I experience redness because when some red object is presented, neurons fire in that specific way. This is the coding perspective. The problem in the coding perspective is of course: who decodes the code? Ultimately, this kind of explanation is strongly dualist: it is implicitly assumed that, at some point, neural activity is transformed into the redness experience by some undetermined process that must be of a very different nature.

I would like to point out that proposals in which perception lies in the interaction between the organism and the environment (e.g. the sensorimotor theory) do not solve this problem either. I can close my eyes and imagine something red. It could be that redness corresponds to a particular way in which visual inputs change when I move my eyes or the surface, which I am anticipating or imagining, but this does not explain what is intrinsically red about the pattern of neural activity now. If we cannot explain it without referring to what happened before, then we are denying that the pattern of neural activity itself determines experience, and again this is a strong dualist view.

An experiment of thought. Consider two salamanders, and each of them has only one neuron, which is both a sensory neuron and motor neuron; say, its firing produces a particular movement. The salamanders are very similar, but their visual receptors are tuned to different wavelengths. In the first salamander, the neuron reacts to red stimuli; in the second salamander, the neuron reacts to blue stimuli. What might happen in terms of visual experience when the neuron fires? Does the first salamander see red and the other see blue? If we think that neural activity alone determines experience, then in fact the two salamanders should experience exactly the same thing – and this is also independent of the sensorimotor contingencies in this case.


4) What is the relationship between the structure of experience and the structure of neural activity?

Subjective experience is highly structured. There might be some dispute about how rich it actually is, but it is at least as rich as what you can describe with words. A striking fact about language is that the meaning of sentences is not only implied by the words but also by the relations between them, i.e., the syntax. For example, a visual scene is composed of objects with spatial relations between them, and with attributes (a red car in front of a small house). In fact, there must be more to it than syntax, there must also be semantics: if neural activity completely determines subjective experience, it must not only specify that there is a car, but also what a car is. A useful notion in psychology of perception is the concept of “affordance” introduced by James Gibson: the affordance of an object is what it allows you to do (e.g. a car affords driving). Affordances are potentialities of interaction, and they gives some meaning (rather than labels) to perceptual objects. This brings an inferential structure to experience (if I did that, this would happen).

This stands in sharp contrast with the central perceptual concept in neuroscience, the notion that “cell assemblies” represent particular percepts. A cell assembly is simply a set of neurons, and their co-activation represents a particular percept (say, a particular face). Let us say that one neuron represents “red”, another represents “car”, then the assembly of the two neurons represents the red car. The problem with this concept is that it is very poorly structured. It cannot represent relations between objects, for example. This type of representation is known as the “bag-of-words” model in language processing: a text is represented by its set of words, without any syntactic relationship; clearly, the meaning of the text is quite degraded. The concept of cell assembly is simply too unstructured to represent experience.

If we are looking for a mapping between neural activity and percepts, then 1) we must find a way to define some structure on neural activity, and 2) the mapping must preserve that structure (in mathematical terms, we are looking for a morphism, not a simple mapping).

I can summarize this discussion by pointing out that to make progress on the mind-body problem, there are two crucial steps: 1) to understand the articulation between physical time and the time of consciousness, 2) to understand the articulation between the structure of neural activity and the structure of phenomenal experience.

Notes on consciousness. (IV) The phenomenal content of neural activity

This post is about the mind-body problem. Specifically, what is the relationship between the activity of the brain and the phenomenal content of conscious experience? It is generally thought that experience is somehow produced by the electrical activity of neurons. The caricatural example of this idea is the concept of the “grandmother cell”: a neuron lights up when you think of your grandmother, or conversely the activation of that neuron triggers the experience of, say, the vision of your grandmother's face. The less caricatural version is the concept of cell assemblies, where the single cell is replaced by a set of neurons. There are variations around this theme, but basically, the idea is that subjective experience is produced by the electrical activity of neurons. There actually is some experimental evidence for this idea, coming from the electrical stimulation of the brain of epileptic patients (read any book by Oliver Sacks). Electrical stimulation is used to locate the epileptic focus in those patients, and depending on where the electrode is in the brain, electrical stimulation can trigger various types of subjective experiences. Epileptic seizures themselves can produce such experiences, for example auditory experiences of hearing specific musics. Migraines can also trigger perceptual experiences (called “aura”), in particular visual hallucinations. So there is some support for the idea of a causal relationship between neural activity and subjective experience.

The obvious question, of course, is: why? At this moment, I have no idea why neural activity should produce any conscious experience at all. We do not believe that the activity of the stomach causes any subjective experience for the stomach, or the activity of any set of cells, including cardiac cells, which also have an electrical activity (but of course, maybe we are wrong to hold this belief).

I propose to start with a slighly more specific question: why does neural activity cause subjective experience of a particular quality? Any conscious experience is an experience of something (a property called intentionality in philosophy), for example the vision of your grandmother's face. Why is it that a particular spatio-temporal pattern of activity in a neural network produces, for that neural network, the experience of seeing a face? One type of answer is to say that this particular pattern has been associated with the actual visual stimulus of the face, ie, it “encodes” the face, and so the meaning of those neurons lighting up is the presence of that visual stimulus. This is essentially the “neural coding” perspective. But there is a big logical problem here. What if the visual stimulus is not present, but the neurons that “encode” the face light up either naturally (memory, dream) or by electrode stimulation? Why would that produce a visual experience rather than anything else? If experience is produced by neural activity alone, then it should not matter what external stimulus might cause those neurons to fire, or what happened in the past to those neurons, or even what world the neurons live in, but only which neurons fire now. Which neurons fire now should entirely determine, by itself, the content of subjective experience. Again the problem with the neural coding perspective is that it is essentially dualist: at some stage, there is some other undefined process that “reads the code” and produces subjective experience. The problem we face here is that the firing of neurons itself must intrinsically specify the experience of seeing a face, independent of the existence of an outside world.

I will try to be more specific, with a very simple example. Imagine there is just one neuron, and two stimuli in the world, A and B. Now suppose, by conditioning or even simply by anatomical assumption, that stimulus A makes the neuron fire. A neural coder would say: this neuron codes for stimulus A, and therefore this neuron's firing causes the experience of A. But you could also assume a different situation, maybe a different organism or the same organism conditioned in a different way, where stimulus B, and not A, makes the neuron fire. If neural activity is what causes subjective experience, then this neuron's firing should produce exactly the same experience in both cases, even though different stimuli cause them to fire. This example can be vastly generalized, and the implication is that any two patterns of neural activity that are identical up to a permutation of neurons should produce the same subjective experience for that set of neurons.

As if all this were not puzzling enough, I will now end on a disturbing experiment of thought. Imagine we measure the entire pattern of neural activity of someone experiencing the vision of his grandmother. Then we build a set of blinking red lights, one for each neuron, programmed so as to light up at the same time as the neurons did. The red lights don't even need to be connected to each other. The electrical activity of this set of lights is thus the same as the activity of the neural network. Therefore, by the postulate that electrical activity is what causes subjective experience, the set of lights should experience the sight of the grandmother, with the impression of being the grandson. Would it?

Notes on consciousness. (III) Modern panpsychism: about the integrated information theory of consciousness

In the last decade, a number of neuroscientists have become interested in the question of consciousness. For example Christof Koch, Stanilas Dehaene, Gerald Edelman, and many others. There have been a number of interesting new insights on this old subject, mostly focused on the so-called “neural correlates of consciousness”, that is, the properties of neural activity that are associated with conscious states, as opposed to say coma. However, to my mind there is no convincing theory that explains what is consciousness, why we are conscious at all and why we feel anything at all (phenomenal consciousness). But there have been attempts. A recent one is the integrated information theory (IIT) proposed by Tononi, which proposes that consciousness is a property of all systems that have a high level of “integrated information”. In a nutshell, such a system is a dynamical system that cannot be divided into smaller independent (or weakly dependent) systems. The term “information” should be understood in the sense of information theory: how much information (uncertainty reduction) there is in the state of a subsystem about the future of another subsystem. In a nutshell, the problem with this theory is that it is as much about consciousness as information theory is about information.

Christof Koch is a notorious fan of IIT. He describes the theory in a popular science article entitled “Ubiquitous minds” (available on his web page). I should point out that it is not an academic paper, so maybe my criticisms will seem unfair. So to be fair, let us say that what follows is a criticism of the arguments in that article, but perhaps not of Koch's thought in general (admittedly I have not read his book yet).

Koch correctly presents IIT as a modern form of panpsychism, that is, the idea that lots of things are conscious to some level. Animals, of course, but also any kind of system, living or not, that has high “integrated information” (named “phi”). On his blog, Scott Aaronson, a theoretical computer scientist, gives an example of a matrix multiplication system that has this property and therefore should be highly conscious according to IIT if it where physically implemented. Now Tononi and Koch do not see this counter-intuitive implication as a problem with the theory, but on the contrary they embrace it as a highly interesting implication. Koch speculates for example that the internet might be conscious.

Koch starts by describing the naïve version of panpsychism, which indeed can easily be dismissed. Naïve panpsychism states that everything is conscious, to different levels: a brain, a tree, a rock. This immediately raises a big problem (refered to as the “problem of aggregates” in the article): you might claim that everything is conscious, but then you need to define what a “thing” is. Is half a rock conscious? Then which half? Is any set of 1000 particles randomly chosen in the universe conscious? Is half of my brain plus half of your stomach a conscious entity?

IIT is more restricted than naïve panpsychism, but it suffers from the same problem: how do you define a “system”? Wouldn't a subsystem of a conscious system also be conscious, according to the theory? As Koch writes, the theory offers no intrinsic solution to this problem, it must be augmented by an ad hoc postulate (“that only “local maxima” of integrated information exist”). What puzzles me is that the paper ends on the claim that IIT offers an “elegant explanation for [the existence of] subjective experience”. What I have read here is an interesting theory of interdependence in systems, and then a claim that systems made of interdependent parts are conscious. Where is the explanation in that? A word (“consciousness”) was arbitrarily put onto this particular property of systems, but no hint was provided at any point about a connection between the meaning of that word and the property those systems. Why would this property produce consciousness? No explanation is given by the theory.

If it is not an explanation, then it must simply be a hypothesis; the hypothesis that systems with high integrated information are conscious. That is, it is a hypothesis about which systems are conscious and which are not. As we noted above, this hypothesis assigns consciousness to non-living things, possibly including the internet, and definitely including some rather stupid machines that no one would consider conscious. I would consider this a problem, but tenants of IIT would simply adopt panpsychism and consider that, counter-intuitively, those things are actually conscious. But then this means admitting that no observation whatsoever can give us any hint about what systems are conscious (contrary to the first pages of Koch's article, where he argues that animals are conscious on those grounds); in other words, that the hypothesis is metaphysical and not testable. So the hypothesis is either unscientific or wrong.

Now I am not saying that the theory is uninteresting. I simply think that it is a theory about consciousness and not of consciousness. What about is it exactly? Let us go back to what integrated information is supposed to mean. Essentially, high integrated information means that the system cannot be subdivided in two independent systems – the future state of system A depends on the current state of system B and conversely. This corresponds to an important property of consciousness: the unity of consciousness. You experience a single stream of consciousness that integrates sound, vision, etc. Sound and vision are not experienced by two separate minds but by a single one. Yet this is what should happen if there were two unconnected brain areas dealing with sound and light. Thus a necessary condition for a unique conscious experience is that the substrate of consciousness cannot be divided into causally independent subsets. This is an important requirement, and therefore I do think that the theory has interesting things to say about consciousness, in particular what its substrate is, but it explains nothing about why there is a conscious experience at all. It provides a necessary condition for consciousness – and that's already quite good for a theory about consciousness.

But that's it. It does not explain why an interdependent system should be conscious – and in fact, given some examples of such systems, it seems unlikely that it is the case. What is missing in the theory? I hinted at it in my introduction: the problem with integrated information theory is that it is as much about consciousness as information theory is about information. The word “information” in information theory has little to do with information in the common sense of the word, that is, something that carries meaning for the receiver. But information theory is actually better described as a theory of communication. In fact, one should remember that Shannon's seminal paper was entitled “A Mathematical Theory of Communication”, not of information. In a communication channel, A is encoded into B by a dictionary, and B carries information about A insofar as one can recover A from B. But of course it only makes sense for the person sitting at the receiving end of the communication channel if 1) she has the dictionary, 2) A already makes sense to her. “Information” theory says nothing about how A acquires any meaning at all, it is just about the communication of information. For this reason, “integrated information” fails to address another important aspect of consciousness, which in philosophy is named “intentionality”: the idea that one is always conscious of something, i.e. consciousness has a “content” - not just a “quantity of consciousness”. Any theory that is solely based on information in Shannon's sense (dictionary) cannot say much about phenomenonal consciousness (how it feels like).

For the end of this post, I will simply quote Scott Aaronson:
“But let me end on a positive note. In my opinion, the fact that Integrated Information Theory is wrong—demonstrably wrong, for reasons that go to its core—puts it in something like the top 2% of all mathematical theories of consciousness ever proposed. Almost all competing theories of consciousness, it seems to me, have been so vague, fluffy, and malleable that they can only aspire to wrongness.”

Notes on consciousness. (II) Perceiving and knowing

Perceiving space is knowing where things are in the world. Or is it?

I am sitting in my living room, and there are big windows on a courtyard. The windows are sound-proof and so if I open just one, acoustical waves mostly enter the room through that window. Now someone enters the courtyard on the right, walks across it and arrives at the door on the left. If I close my eyes, I know that the person is walking from right to left. However, what I hear is the sound of someone walking, always coming from the same direction, that of the window. If someone asks me where the person is at a given moment time, I could point to the more or less correct direction, by inference. But this is not what I perceive. I always perceive the sound coming from the same direction. There is a difference between perceiving (phenomenological) and knowing (conceptual). And there is a difference between phenomenology and behavior.

Another striking example is referred pain. Referred pain is a pain that one feels at a location away from the cause of the injury. For example, in a heart attack, one may feel pain in the arm rather than in the chest. This is a known phenomenon and if you know it, you may correctly identify the location of injury in the heart when you feel pain in the arm. But it doesn't change the fact that you feel pain in the arm. You may entirely convince yourself that the injury is in the heart, and all your behavior might be consistent with that belief, but still you will feel the pain in the arm.

There are several interesting conclusions we can draw from these remarks. First, perception is not entirely reducible to behavior. Here we are touching the hard problem of consciousness (qualia): you could observe a cat turning its head to a sound source and you would think that the cat perceives that the sound came from the source, but in reality you don't know. Maybe the cat perceives it somewhere else but it corrects its movement because it knows its perception tends to be biased. With humans, you could perhaps distinguish between these possibilities because humans speak. But without this option, a purely functionalist approach to perception (in terms of relationships between sensory stimuli and behavior) misses part of the phenomenon.

Second, inference is not the same as perception. Spatial perception is not just the process of inferring where something is from sensory inputs. There is also the experience of perception, which is not captured by the objectivist view.

Notes on consciousness. (I) Time

What is time and how is it perceived? This is of course a vast philosophical question, which I will only scratch.

1) Time, space and existence

It is customary to describe time as “the fourth dimension”. This point of view comes from the equations of mechanics and is highly misleading, because it seems to imply that time is of the same kind as space. A century ago, Henri Poincaré noted that our concept of space, both perceptually and scientifically, derives from our physical interactions with the world. That is to say, knowing where something is is knowing how to get there. Space is defined by the laws that govern movements in the physical world and the structure of these laws (Euclidean geometry). A law, some property that does not change, can only be defined with respect to something that changes. Therefore, time, defined as the source of change in the world, is a prerequisite to space. Space exists only by its persistence through the passing of time.

2) Time and change

In fact, nothing exists without the passing of time, because the essence is precisely what does not change through the flow of time. If we see someone throwing a ball, that ball is moving. Our visual sensations change, but we see a ball in movement: this is to say that there is something in the visual signals that does not change, which characterizes the ball as such. We do not see an object in the flickering white noise of a TV set.

In the TV series Bewitched, Samantha the housewife twitches her nose and everyone freezes except her. Then she twitches her nose and everyone unfreezes, without noticing that anything happened. For them, time has effectively stopped. This is to say that time is not perceived as such, but only through the changes it causes in our body. It is these changes that are perceived, not time per se (i.e., not time as in the variable in the equations of mechanics).

3) Irreversibility of time

From the fact that time is the perceived cause of changes, it follows that time has a direction, because physical processes are generally irreversible. This is also related to the theorem in information theory that states that information can only be lost, and never gained, when a process is applied to a variable. The current state of a physical system results from previous processes only, which constitutes “the past”.

A physical system in which events occur (our body) can be seen as a dynamical system, or series of processes that make the state of the system evolve. From one state s, the system changes subsequently to state s’. There is a direction to this change: s -> s’. This is the action of time on the system, and it is directed (the “arrow of time”). If the system where isolated, then time would be arbitrary. One could consider any dimension that is isomorphic to time and preserves directionality, and call it “time”, without changing the organization of changes within the system. It would make no difference for the system.

4) The unity of time

This raises the question of the perceptual unity of time: if time is perceived through changes in our body, then why do we feel that time is a single thing, when lots of different things change in our body? How is it that an auditory event and a visual event can appear to occur “at the same time”, given that they impact different receptors? Why isn’t there a different time for each process in our body? What does it mean that an event occurs “before” another one?

Imagine two independent processes that are spatially separated. From the perspective of these processes, it would make no difference if time passed at a different pace. The unity of time must come from an interaction between processes. The interaction between different processes defines a common flow of time.

Going further would probably require a discussion of consciousness and working memory, so I will leave these questions mostly unanswered for now.

5) The grain of time

How fine is our perception of time? When one listens to an auditory click played through headphones with 500 µs delay between the two ears, we do not hear two clicks. We hear a single click, lateralized towards one side. If we repeatedly play clicks at 50 Hz (every 20 ms), we do not hear a series of clicks. We hear a single continuous sound. When we listen to a pure tone at 50 Hz, the amplitude of the tone varies all the time but we do not hear this variation of amplitude. On the contrary, it feels like the tone has constant loudness.

These remarks suggest that our perception of time has a “grain” of a few tens of ms. That is, processes occurring within a few tens of ms are perceived as being caused by the same event, and the temporal occurrence of events within that time window is not perceived as time. Why?

To see how tricky this is, consider again the first example, when we listen to two clicks delayed by 500 µs between the two ears. The temporal order of the clicks can be clearly distinguished: if the click is first played in the left earphone, the sound is perceived as coming from the left, and conversely if the click is first played in the right earphone. In addition, if the delay between the two clicks is changed, then the sound is perceived as coming from a different direction (usually somewhere between the two ears), in a way that is reproducible. Such changes are perceived when the delay is changed by about 20 µs.

So from a computational point of view, time is processed with a grain of 20 µs. But phenomenologically, time appears to have a grain about a thousand times larger. Why such a difference? The perceptual grain of time does not appear to reflect the precision of neural processing, or in other words, the timescale at which states of the brain seem constant.

6) Duration

This post probably raised more questions than I could answer. I will end it with a discussion of the concept of duration. Spinoza described it as follows: “Duration is an attribute under which we conceive the existence of created things insofar as they persevere in their actuality”. This is essentially the point I have developed at the beginning of this post. In contrast with, say, color and pitch, duration is not a quality of things. Duration is about existence (the fact that a thing exists), while color or pitch is about essence (what this thing is). Properties of objects are defined by their persistence through time, but duration does not persist through time. Rather, duration quantifies how much time some properties exist. For example, it can be said that a musical note has a timbre (the instrument), a pitch and a duration. These are not three independent qualities: duration is about the pitch and timbre (for how much time they can be said to exist), but timbre is not about duration.

In summary: time is about existence, space is about essence.