A pervasive metaphor in neuroscience is the idea that neurons “encode” stuff: some neurons encode pain; others encode the location of a sound; maybe a population of neurons encode some other property of objects. What does this mean? In essence, that there is a correspondence between some objective property and neural activity: when I feel pain, this neuron spikes; or, the image I see is “represented” in the firing of visual cortical neurons. The mapping between the objective properties and neural activity is the “code”. How insightful is this metaphor?
An encoded message is understandable to the extent that the reader knows the code. But the problem with applying this metaphor to the brain is only the encoded message is communicated, not the code, and not the original message. Mathematically, original message = encoded message + code, but only one term is communicated. This could still work if there were a universal code that we could assume all neurons can read, the “language of neurons”, or if somehow some information about the code could be gathered from the encoded messages themselves. Unfortunately, this is in contradiction with the main paradigm in neural coding theory, “efficient coding”.
The efficient coding hypothesis stipulates that neurons encode signals into spike trains in an efficient way, that is, it uses a code such that all redundancy is removed from the original message while preserving information, in the sense that the encoded message can be mapped back to the original message (Barlow, 1961; Simoncelli, 2003). This implies that with a perfectly efficient code, encoded messages are undistinguishable from random. Since the code is determined on the statistics of the inputs and only the encoded messages are communicated, a code is efficient to the extent that it is not understandable by the receiver. This is the paradox of the efficient code.
In the neural coding metaphor, the code is private and specific to each neuron. If we follow this metaphor, this means that all neurons speak a different language, a language that allows expressing concepts very concisely but that no one else can understand. Thus, according to the coding metaphor, the brain is a Tower of Babel.
Can this work?
"Since the code is determined on the statistics of the inputs and only the encoded messages are communicated, a code is efficient to the extent that it is not understandable by the receiver."
Its not understood why the code will not be understandable by the receiver?
Because if it's efficient then it's random, and if it's random then you can't understand it.
I am a bit confused with this.
Say I have a "pain-neuron" if I feel pain it is 1 and if I do not feel pain it is 0. Then it is perfectly efficient. But it is not random?
Imagine you're a worker who received training before starting your job with instructions like "respond with y when you see x." After some training, y is no longer provided, and your official job is to predict y based on x. There are no globally common rules here, no manager telling you what to do; all you have to do is manage yourself. So, you learn the probability distributions of x and y. The issue now is that the initial x and y you receive are entirely random and illogical, much like initializing neural network parameters. Thus, you can only form your own private language.
Lastly, here's my personal viewpoint: I actually believe that global encoding is possible to some extent. Just as different regions of the brain respond to different types of information, residents of a country may share some kind of "national character."