Deep Learning to Study the Brain to Improve…Deep Learning

The interplay between AI and neuroscience is one of my favorite topics, and one that several of us on the AI team at Intel have personally worked on. I’m excited to share news about a new collaboration between Intel and MIT research labs designed to advance both of these fields. Before I do, I’d like to set the scene:

The relationship between neural networks and the brain has been a fascinating dialogue over the last fifty years. One side of this dialogue—the flow of knowledge from neuroscience to AI—is well known. The machine learning community has imported several concepts from neuroscience, starting, of course, with the (simplified) neuron, with linear inputs and a non-linear output, as an atomic unit of computation. Another idea from neuroscience is the arrangement of neurons into layers, analogous to the cytoarchitecture of mammalian neocortex. Recently, even more features of the brain have been transplanted into machine learning models. Some of these have been deliberate, such as the use of experience replay buffers for reinforcement learning, inspired by the observation that sequences of pyramidal cell activity are replayed in the hippocampus during sleep and periods of inactivity. Other features have appeared due to resource constraints, such as reduced bit-widths for weights, analogous to the limited precision of biological synapses. (This could be seen as an example of convergent evolution between brains and computers.)

But the other side of the dialogue, where machine learning informs neuroscience, is less well known and equally riveting. Much of this activity has been in the use of connectionist models that are complex enough to imitate features of brain activity and behavior and yet still mathematically tractable. These models provide an intellectual framework around experimental observations, make testable predictions, and suggest new experiments. One example is the Hopfield model of associative memory formation, which combines the recurrent (loopy) connectivity patterns found in the hippocampus with Hebbian learning (“neurons that fire together, wire together”). Two simple equations suffice to replicate phenomena such as auto-associative memory (using part of a memory to recall the rest), false memories, and attractor patterns found in the activity of hippocampal place cells. Another is Olshausen’s model of neural responses in the visual cortex, which combines the variational principle of sparsity with the concept of overcompleteness from linear algebra to explain why edge detectors (and not, say, Fourier bases) are learned in the brain from the statistics of natural images.

This brings us to our new project with MIT. An exciting thread in the AI-neuroscience dialogue has recently emerged where deep learning is serving neuroscience in a new way—namely as a tool for analyzing vast quantities of neuroscience data. In short, researchers are using brain-inspired algorithms to study the brain that inspired them. One example is in the field of connectomics, a recent effort to map large volumes of brain at nanometer scale. At this scale, neurons are huge, and one can even trace all of their input and output connections—that is, their axons, dendrites, and synapses. Assembling these connectomes—3D wiring diagrams of the brain—requires three major steps: (1) create a series of brain slices around 30 nanometers thick; (2) image this stack with an electron microscope; and (3) extract contours of neurons from the images and fuse them across the slices into a 3D reconstruction—a tour de force of experimental and analytical skill. It is also a big data problem; at this resolution, the neocortex of a rat would generate about an exabyte of image data.

We have just started collaborating with Prof. Nir Shavit’s lab at MIT CSAIL, which has been a leader in the area of computational connectomics, the analytics required to achieve step 3. His lab has pioneered ways of accelerating a novel deep learning-based image processing pipeline through clever use of multi-threaded programming and vector instructions on Intel® Xeon® architectures. We are excited to help him develop his next-gen processing pipeline.

What will we researchers learn from these studies? How will revealing the complete wiring diagram of the brain (or brain regions) help us design better neural networks? One possibility is in the area of novel architectures. Many significant performance jumps in neural networks have been due to innovations in connectivity patterns, such as inception modules and skip connections. Once researchers gain access to the full wiring diagram of the brain, and how it varies across brain regions and individuals, it is possible that new connectivity motifs will become apparent and inspire even more productive architectural variety in engineered systems. There are some caveats, of course: the connectome is only part of the story of how the brain computes, since a static wiring diagram doesn’t easily show how connections change with time and experience. Nevertheless, given how far deep learning has progressed so far using only a few key concepts from neuroscience, I look forward to the imminent explosion of neuroscience discovery that will hopefully lead us to vastly more powerful learning systems.

Image credit: Mouse nerve bundle, Y. Meirovitch and K. Kang from N. Shavit and J. Lichtman labs.