A number of thinkers oppose the notion of pictorial representations, or even of any sort of representation, in the brain. In robotics, Rodney Brooks is often quoted for this famous statement: “the world is its own best model”. In a previous post, I commented on the fact that slime molds can solve complex spatial navigation problems without an internal representation of space – in fact, without a brain! It relies on using the world as a sort of outside memory: the slime mold leaves some extracellular trace on the floor, where it has previously been, so that it avoids being stuck in any one place.

This idea is also central in the sensorimotor theory of perception, and in fact Kevin O’Regan argued about “the world as an outside memory” in an early paper. This is related to a number of psychological findings about change blindness, but I will rephrase the argument from a more computational perspective. Imagine you are making a robot with a moveable eye that has a fovea. At any given moment, you only have a limited view of the world. You could obtain a detailed representation of the visual scene by scanning the scene with your eye and storing the images in memory. This memory would then be a highly detailed pictorial representation of the world. When you want to know some information about an object in any part of the visual scene, you can then look at the right place in the memory. But then why look at the memory if you can directly look at the scene? If moving the eye is very fast, which is the case for humans, then from an operational point of view, there is no difference between the two. It is then simply useless and inefficient to store the information in memory if the information is immediately available in the world. What might need to be stored, however, is some information about how to find the relevant information (what eye movements to produce), but this is not a pictorial representation of the visual scene.

Despite what the title of this post might suggest, I am not going to contradict this view. But we also know that visual memory exists: for example, we can remember a face, or we can remember what is behind us if we have seen it before (although it is not highly detailed). Now I am throwing an idea here, or perhaps an analogy, which might initially sound a bit crazy: how about if memory were like an inside world? In other words, how about interpreting the metaphor “looking at something in memory” in a literal way?

The idea of the world as an external memory implicitly relies on a boundary between mind and world that is put at the interface of our sensors (say, the retina). Certainly this is a conceptual boundary. Our brain interacts with the environment through an interface (sensors/muscles), but we could equally say that any part of the brain interacts with its environment, made of everything outside it, including other parts of the brain. So let us imagine for a while that we put the mind-world boundary in such a way that the hippocampus, which is involved in memory (working memory and spatial memory), is outside it. Then the mind can request information about the world from the sensory system (moving the eyes, observing the visual inputs), or in the same way from the hippocampus (making some form of action on the hippocampus, observing the hippocampal inputs).

Perhaps this might seem somehow like a homunculus thinking exercise, but I think there is something interesting in this perspective. In particular, it puts memory and perception at the same level of description, in terms of sensorimotor interaction. This is interesting because from a phenomenological point of view, there is a similarity between memory and perception: the memory of an image feels (a bit) like an image, or one can say that she “hears a melody in her head”. At the same time, memory has distinct phenomenal properties, for example one cannot interact with memory in the same way as with the physical world, it is also less detailed, and finally there are no “events” in memory (something unpredictable happening).

In other words, this view may suggest a sensorimotor account of memory (where “sensorimotor” must be understood in a very broad sense).