A brief critique of predictive coding

Predictive coding is becoming a popular theory in neuroscience (see for example Clark 2013). In a nutshell, the general idea is that brains encode predictions of their sensory inputs. This is an appealing idea because superficially, it makes a lot of sense: functionally, the only reason why you would want to process sensory information is if it might impact your future, so it makes sense to try to predict your sensory inputs.

There are substantial problems in the details of predictive coding theories, for example with the arbitrariness of the metric by which you judge that your prediction matches sensory inputs (what is important?), or the fact that predictive coding schemes encode both noise and signal. But I want to focus on the more fundamental problems. One has to with “coding”, the other with “predictive”.

It makes sense that brains anticipate. But does it make sense that brains code? Coding is a metaphor of a communication channel, and this is generally not a great metaphor for what the brain might do, unless you fully embrace dualism. I discuss this at length in a recent paper (Is coding a relevant metaphor for the brain?) so I won’t repeat the entire argument here. Predictive coding is a branch of efficient coding, so the same fallacy underlies its logic: 1) neurons encode sensory inputs; 2) living organisms are efficient; => brains must encode efficiently. (1) is trivially true in the sense that one can define a mapping from sensory inputs to neural activity. (2) is probably true to some extent (evolutionary arguments). So the conclusion follows. Critiques of efficient coding have focused on the “efficient” part: maybe the brain is not that efficient after all. But the error is elsewhere: living organisms are certainly efficient, but it doesn’t follow that they are efficient at coding. They might be efficient at surviving and reproducing, and it is not obvious that it entails coding efficiency (see the last part of the abovementioned paper for a counter-example). So the real strong assumption is there: the main function of the brain is to represent sensory inputs.

The second problem has to with “predictive”. It makes sense that an important function of brains, or in fact of any living organism, is to anticipate (see the great Anticipatory Systems by Robert Rosen). But to what extent do predictive coding schemes actually anticipate? First, in practice, those are generally not prediction schemes but compression schemes, in the sense that they do not tell us what will happen next but what happens now. This is at least the case of the classical Rao & Ballard (1999). Neurons encode the difference between expected input and actual input: this is compression, not prediction. It uses a sort of prediction in order to compress: other neurons (in higher layers) produce predictions of the inputs to those neurons, but the term prediction is used in the sense that the inputs are not known to the higher layer neurons, not that the “prediction” occurs before the inputs. Thus the term “predictive” is misleading because it is not used in a temporal sense.

However, it is relatively easy to imagine how predictive coding might be about temporal predictions, although the neural implementation is not straightforward (delays etc). So I want to make a deeper criticism. I started by claiming that it is useful to predict sensory inputs. I am taking this back (I can because I said it was superficial reasoning). It is not useful to know what will happen. What is useful is to know what might happen, depending on what you do. If there is nothing you can do about the future, what is the functional use of predicting it? So what is useful is to predict the future conditionally to a different set of potential actions. This is about manipulating models of the world, not representing the present.

2 réflexions au sujet de « A brief critique of predictive coding »

  1. Hi Romain,

    Thanks for the post. I too have critiques of Predictive Coding, but different ones from yours, and I disagree with some of your comments.

    0) coding: I disagree that coding is only a useful metaphor for the experimentalist. Encoding is the relationship between neural activity and causal variables in the world. I agree they're not measurable, but after some interactions with the world we can narrow the range of possibilities, and still end up encoding a posterior. If an experimentalist finds an encoding, it might never be used by the animal, and I agree then it is useless to talk about this as a valid neural encoding. But if the encoded information is decoded — i.e. used to drive behavior corresponding to those causal variables, then it is a valid neural encoding.

    1) The essence of prediction is the absence of data. It doesn't matter if it's a prediction over time or space or frequency, past or future. (e.g. flip a coin, cover it with your hand, and predict if it's heads or tails.)
    2) A prediction can be a point estimate or a range or a probability distribution.
    3) Predictive coding is about sending the prediction and encoding the error. I agree that this is compression. But the compression is only what is done _with_ the prediction. And the compression can pertain to any time or any signal, it just depends on how observations are used.
    4) My main critique is that Predictive Coding need not be efficient at all. The error itself may indeed make better use of a channel with limited capacity (but that depends on the SNR). But even in the best case, conveying the prediction back to form the error becomes then a poor use of that channel.

    Perhaps you've seen the opinion piece from Mate Lengyel's lab: https://www.ncbi.nlm.nih.gov/pubmed/28942084. It's relevant to this topic.

    Best,
    -xaq

    • Hi Xaq,
      Thanks for the comment. A quick reply to (0): I address these points in the paper I mention at the beginning. One brief point maybe: there is no decoding in the brain, because by definition decoding is mapping to some property outside the brain. What you (and others) mean is rather that there is behavior that depends on the encoded thing, and we abusively call this "decoding". But the decoding per se is done by an observer (who defines a mapping from either neural activity or behavior to the encoded property).
      Best,
      Romain

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *