Neuroscience to Computer Science: An Update from AI Research at Intel

Artificial Intelligence (AI) is poised to have a transformative effect on human civilization. Enabled by decades of research, AI adoption is now accelerating due to the availability of exascale computing, the explosion of big data, and the emergence of algorithms that can take advantage of these compute and data resources. The disruptive transformation may manifest in several ways. For example:

  1. Augmenting the “Inside-Out” paradigm (observations > laws > predictions) with “Outside-In” paradigms where data-driven models are created to generate new knowledge
  2. Disruption of the traditional balance of division of labor between human and machines where machines crunch numbers and intelligent decision making is the prerogative of solely humans.

This leads to a new class of applications and services powered by machine learning-based solutions. AI Diagnosing Heart Disease & Cancer better than humans or the IBM Watson supercomputer defeating human “Jeopardy” champions are just the tip of the iceberg in terms of what is now computationally feasible. AI extends the reach of computing to largely untapped sectors of modern society: health, education, farming, and transportation—all of which are often operating well below the desired levels of efficiency. According to Intel Fellow Dr. Pradeep Dubey, “traditionally, there has been a division of labor between computers and humans…we are now on the cusp of a major transformation that can disrupt this balance. This disruption is triggered by an unprecedented convergence of massive compute with massive data and some recent algorithmic advances. This confluence has the potential to spur a virtuous cycle of compute.”

To unleash this virtuous cycle of computing, Intel researchers seek to delve deep into these emerging paradigms with a broad span of focus ranging from neuroscience and AI applications to computer science and systems.

Intel’s collaboration with the Princeton Neuroscience Institute aims to map the human mind in real time and develop the next generation of brain imaging analysis. This joint research led to a paper published by the Nature Neuroscience journal that details how fMRI data and AI techniques can enable an open source tool for cognitive neuroscientists called Brain Imaging Analysis Kit (BRAINIAK). Intel is also working with marine researchers in the Parley for the Oceans initiative to deploy advanced drone technology, AI, and machine learning tools to collect biological samples from whales and analyze data in real time.

On the computer science front, we have seen similar exciting results in advancing the state of the art in AI. For example, we have made significant progress on speeding up deep learning training. Our accepted paper at ICLR 2017, “On Large Batch Training for Deep Learning: Generalization Gap and Sharp Minima” illuminated the role noise plays in generalization and how a larger mini-batch size in training converges to a sharper minima due to loss of noise that causes lack of generalization. This work contributed to recent breakthroughs on training with large batches. State-of-the-art CNNs like ResNet-50 are now being trained in under an hour on Intel® processor-based clusters. Our accepted paper at SysML 2018, “On Scale-out Deep Learning Training for Cloud and HPC, detailed parallelization techniques needed to scale deep learning training. Along with our collaborators at NERSC, Stanford and the University of Montreal, we demonstrated 1000x faster deep learning at petascale on science applications focused on pattern discovery in climate data and signal vs. background classification for large hadron collider datasets. This year, we detailed the first ever 2bit inference on ResNet-50 at SysML, and ICLR will feature our work on mixed precision training of CNNs using integer operations.

At Intel, we are committed to technology evolution and bringing the capabilities of that technology to every human.  We are excited to be at the forefront of disruptive AI technologies that help solve problems and positively impact our globe.

Hear more from Bharat Kaul as he discusses the AI opportunity at a recent conference