Misconception #3: “Neural codes can only be based on rates because neural networks are chaotic”. Whether this claim is true or not (and I will comment on it below), chaos does not imply that spike timing is irrelevant. To draw this conclusion is to commit the same category error as I discussed in the previous post, i.e., confusing rate vs. timing and stochastic vs. deterministic.

In a chaotic system, nearby trajectories quickly diverge. This means that it is not possible to predict the future state from the present state, because any uncertainty in estimating the present state will result in large changes in future state. For this reason, the state of the system at a distant time in the future can be seen as stochastic, even though the system itself is deterministic.

Specifically, in vitro experiments suggest that individual neurons are essentially deterministic devices (Mainen and Sejnowski 1995) – at least the variability seen in in vitro recordings is often orders of magnitude lower than in vivo. But a system composed of interacting neurons can be chaotic, and therefore for all practical aspects their state can be seen as random, so the chaos argument goes.

The fallacy of this argument can be seen by considering the prototypical chaotic system, climate. It is well known that the weather cannot be predicted more than 15 days in the future, because even tiny uncertainties in measurements make the climate models diverge very quickly. But this does not mean that all you can do is pick a random temperature according to the seasonal distribution. It is still possible to make short term predictions, for example. It also does not mean that climate dynamics can be meaningfully described only in terms of mean temperatures (and other mean parameters). For example, there are very strong correlations between weather events occurring at nearby geographical locations. Chaos implies that it is not possible to make accurate predictions in the distant future. It does not imply that temperatures are random.

In the same way, the notion that neural networks are chaotic only implies that one cannot predict the state of the network in the distant future. This has nothing to do with the distinction between rate and spike timing. Rate (as mean seasonal temperature) may still be inadequate to describe the dynamics of the system, and firing may still be correlated across neurons.

In fact the chaos argument is an argument against rate-based theories, precisely because a chaotic system is not a random system. In particular, in a chaotic system, there are lawful relationships between the different variables. Taking the example of climate again, the solutions of the Lorenz equations (a model of atmostpheric convection) live in a low-dimensional manifold with a butterfly shape known as the Lorenz attractor. Even though one cannot predict the values of the variables in the distant future, these variables evolve in a very coordinated way. It would be a mistake to replace them by their average values. Therefore, if it is true than neural networks are chaotic, then it is probably not true that their dynamics can be described in terms of rates only.

I will end this post by commenting on the notion that neural networks are chaotic. I very much doubt that chaos is an adequate concept to describe spiking dynamics. There are different definitions of a chaotic system, but essentially they state that a chaotic system is a system that is very sensitive to initial conditions, in the sense that two trajectories that are initially very close can be very far apart after a relatively short time. Now take a neuron and inject a constant current: it will fire regularly. In the second trial, inject the exact same current but 1 ms later. Initially the state of the neuron is almost identical in both trials. But when the neuron fires in the first trial, its membrane potential diverges very quickly from the trajectory of the second trial. Is this chaos? Of course not, because the trajectories meet again about 1 ms later. In fact, I showed in a study of spike time reliability in spiking models (Brette and Guigon, 2003) that even if the trajectories diverge between spikes (such as with the model dv/dt=v/tau), spike timing can still be reliable in the long run in response to fluctuating inputs. This counter-intuitive property can be seen as nonlinear entrainment.

In summary, 1) chaos does not support rate-based theories, it rather invalidates them, and 2) chaos is probably not a very meaningful concept to describe spiking dynamics.