Imagine browsing through a friend's holiday pictures: a trekking path crossing a stream in the Alps, a snapshot of a relaxed afternoon in a Budapest spa, or a punting scene on the Cam. Recognizing the people and objects in the photos seems to be automatic for us. However, as any engineer developing computer vision algorithms would attest, despite our effortless ability in interpreting images, image understanding is a complex and hard-to-imitate process. A study published in Science by Gergő Orbán and Máté Lengyel from Engineering, and their colleagues at Brandeis University, offer some clues to the brain machinery underlying our superior visual abilities. Interestingly, they found that the clue to how the brain interprets images is not what happens when we look at pleasant and colorful images, but what happens when we view a pitch-black image.

So what does a black image tell us about our visual system? Naively, without visual stimulation, one would expect very little activity in the visual part of the brain. Surprisingly however, when the activity of neurons in the dark (so-called spontaneous activity) was first analysed, researchers found very strong, coordinated neural responses. This observation has been equally puzzling from a neuroscience and an engineering perspective. Why would the brain waste precious resources to process a dark image? Does the stimulus matter to the brain at all? The answer to these questions turns out to concern the role of the visual system in processing not what there is, but rather what there may be in an image.

To gain some insight as to why this is the case, a closer examination of the photographs reveals several difficult problems that need to be solved for us to correctly interpret the images. In the picture of the punting scene, the punter in the foreground is higher and occupies a larger area than the colleges and bridges in the background, yet we percieve that she is in reality smaller than them. Her friend in the boat is partially hidden by her pole and is thus visually split into two halves, yet we perceive her as a single person. These and countless other examples reveal that our visual system needs to complement the information contained in any image with our internal model of the world, in order to fill-in the missing information and find the interpretations that are most consistent with reality. It is exactly the sophistication of these internal models that sets apart biological and artificial visual systems; our brains develop internal models of the environment whose complexity is unmatched by present-day visual object recognition algorithms.

So what does a black image tell us about our internal model for visual scenes? In their study, Drs. Orbán and Lengyel and their colleagues took inspiration from modern statistical machine vision algorithms and predicted that neural activity in the dark will reflect aspects of this internal model. For example, imagine reducing the brightness of a photograph. As the details begin to fade, the visual system needs to make increasing use of its internal model to make sense of the information it receives. Most of its activity will be dominated by the internal model. The researchers reasoned that if this is the explanation behind spontaneous activity, the patterns of neural activity in the dark should correspond to possible contents of images. These should then be similar to the patterns of neural activity in response to natural images, as opposed to patterns produced by stimuli that are very unlikely to occur in real situations.

The authors of the study analysed neural activity, previously recorded by their American collaborators, in the primary visual cortex of ferrets. These ferrets were of differing ages and were either sitting in darkness, watching movies or watching a sequence of artificial patterns on a computer screen. In the young animals, that had little or no experience of the world, neural activity in darkness was found to be very different from visually stimulated neural activity. However, as the animals became more mature, spontaneous activity became increasingly similar to neural activity recorded in response to visual stimuli. Moreover, spontaneous activity bore a greater similarity to the neural activity recorded in response to the movies than to the artificial stimuli, just as the researchers predicted for an internal model that is adapted to the regularities of the natural visual environment. Thus, the activity of neurons in the dark is carefully orchestrated as the brain considers possible natural scenes that are compatible with the blank, featureless input stimulus.

In conclusion, it seems that understanding the structure of neural activity in darkness can indeed help us learn about the powerful internal models of the brain. Learning about internal models may in turn be useful for developing more intelligent computer vision algorithms as well as for healing mental disorders, such as schizophrenia, that are associated with disrupted internal models.