Misconception #3: “Neural codes can only be based on rates because neural networks are chaotic”. Whether this claim is true or not (and I will comment on it below), chaos does not imply that spike timing is irrelevant. To draw this conclusion is to commit the same category error as I discussed in the previous post, i.e., confusing rate vs. timing and stochastic vs. deterministic.

In a chaotic system, nearby trajectories quickly diverge. This means that it is not possible to predict the future state from the present state, because any uncertainty in estimating the present state will result in large changes in future state. For this reason, the state of the system at a distant time in the future can be seen as stochastic, even though the system itself is deterministic.

Specifically, in vitro experiments suggest that individual neurons are essentially deterministic devices (Mainen and Sejnowski 1995) – at least the variability seen in *in vitro* recordings is often orders of magnitude lower than *in vivo*. But a system composed of interacting neurons can be chaotic, and therefore for all practical aspects their state can be seen as random, so the chaos argument goes.

The fallacy of this argument can be seen by considering the prototypical chaotic system, climate. It is well known that the weather cannot be predicted more than 15 days in the future, because even tiny uncertainties in measurements make the climate models diverge very quickly. But this does not mean that all you can do is pick a random temperature according to the seasonal distribution. It is still possible to make short term predictions, for example. It also does not mean that climate dynamics can be meaningfully described only in terms of mean temperatures (and other mean parameters). For example, there are very strong correlations between weather events occurring at nearby geographical locations. Chaos implies that it is not possible to make accurate predictions in the distant future. It does not imply that temperatures are random.

In the same way, the notion that neural networks are chaotic only implies that one cannot predict the state of the network in the distant future. This has nothing to do with the distinction between rate and spike timing. Rate (as mean seasonal temperature) may still be inadequate to describe the dynamics of the system, and firing may still be correlated across neurons.

In fact the chaos argument is an argument *against* rate-based theories, precisely because a chaotic system is *not* a random system. In particular, in a chaotic system, there are lawful relationships between the different variables. Taking the example of climate again, the solutions of the Lorenz equations (a model of atmostpheric convection) live in a low-dimensional manifold with a butterfly shape known as the Lorenz attractor. Even though one cannot predict the values of the variables in the distant future, these variables evolve in a very coordinated way. It would be a mistake to replace them by their average values. Therefore, if it is true than neural networks are chaotic, then it is probably not true that their dynamics can be described in terms of rates only.

I will end this post by commenting on the notion that neural networks are chaotic. I very much doubt that chaos is an adequate concept to describe spiking dynamics. There are different definitions of a chaotic system, but essentially they state that a chaotic system is a system that is very sensitive to initial conditions, in the sense that two trajectories that are initially very close can be very far apart after a relatively short time. Now take a neuron and inject a constant current: it will fire regularly. In the second trial, inject the exact same current but 1 ms later. Initially the state of the neuron is almost identical in both trials. But when the neuron fires in the first trial, its membrane potential diverges very quickly from the trajectory of the second trial. Is this chaos? Of course not, because the trajectories meet again about 1 ms later. In fact, I showed in a study of spike time reliability in spiking models (Brette and Guigon, 2003) that even if the trajectories diverge between spikes (such as with the model dv/dt=v/tau), spike timing can still be reliable in the long run in response to fluctuating inputs. This counter-intuitive property can be seen as nonlinear entrainment.

In summary, 1) chaos does not support rate-based theories, it rather invalidates them, and 2) chaos is probably not a very meaningful concept to describe spiking dynamics.

Romain,

Have a look at the very recent Litwin-Kumar and Doiron paper in Nature Neuro: http://dx.doi.org/10.1038/nn.3220

and the associated commentary by Churchland and Abbott: http://dx.doi.org/10.1038/nn.3247

All about variability, chaos, determinism, rate vs timing etc...

Cheers,

watson

Thanks, Watson, it was on my list!

Romain

Hi Romain,

Very interesting series of posts, thanks a lot.

About the chaos story, I just wanted to add two things:

1) As you said, in vitro experiments suggest that individual neurons are essentially deterministic devices (eg Mainen and Sejnowski 1995). But these experiments use direct current injection, bypassing the synaptic transmission, which is believed to be highly stochastic.

2) It is likely that recurrent connectivity makes a network chaotic, that is highly sensitive to small perturbations, because these can be amplified by positive feedback. Note that in feedforward networks, this problem is avoided. Postsynaptic spikes can be more reliable and precise than presynaptic ones (law of large numbers). Consequently, spike patterns can be reliably transmitted, without jitter accumulation (eg Diesmann et al 1999).

But even if recurrent networks are chaotic, I agree with your post: it does not mean that the rate is the only meaningful quantity.

Cheers,

Tim Masquelier

Hi Tim,

1) Yes, I am going to write a post about it. Synaptic unreliability is probably the main source of intrinsic noise in vivo. But I think it tends to be overrated. There are generally multiple release sites, so the law of large numbers also applies there.

2) Possibly. But note that whereas we tend to reason in terms of closed systems, in vivo these networks are driven by sensory inputs. See e.g. DeWeeze, Wehr and Zador (2003) for reproducibibility in vivo in the auditory cortex.

Cheers

Romain

And here's the post: Rate vs. timing (VI) Synaptic unreliability

Romain