Reinforcement Learning Coach v0.9
Dec 19, 2017
Dec 19, 2017
Since the release of Coach a couple of months ago, we have been working hard to push it into new frontiers that will improve its usability for real world applications. In this release, we are introducing several new features that will move Coach forward in this direction.
First, we added several convenient tools for imitation, along with the basic behavioral cloning imitation algorithm. Imitation learning can often be very efficient for achieving very good behavior fast, and is an important addition to Coach’s toolbox. Coach now allows users to interact with the simulation environments and collect data from human examples. Additionally, it supports loading a previously collected dataset of experience and training an agent to imitate the behavior in the given dataset. As a starting point, we added a few presets and datasets for several environments in Doom and Gym.
A Doom agent and a Montezuma Revenge agent trained using Behavioral Cloning
The second addition is a built-in support for the recently released CARLA simulator. CARLA is an open-source urban driving simulator developed as a collaboration between Intel Labs and the Computer Vision Center (CVC) that includes realistic urban environments. CARLA enables the training of autonomous driving agents and is now integrated with Coach. We also added several presets for training both reinforcement learning and imitation learning agents for simple driving behaviors.
A CARLA agent trained using reinforcement learning
Finally, to keep up with the state-of-the-art in the field of reinforcement learning, we recently added the Quantile Regression DQN algorithm, which was shown to achieve superior results over the Categorical DQN algorithm on the Atari benchmark.
To conclude, we believe that the CARLA simulator, along with tools for imitation learning, open a new world of possibilities for users that are interested in applying reinforcement learning to real world applications. Go ahead and try it out by following the instructions on our GitHub repository.
 CARLA: An Open Urban Driving Simulator Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez and Vladlen Koltun. CoRR, abs/1711.03938, 2017.
 Distributional Reinforcement Learning with Quantile Regression Will Dabney, Mark Rowland, Marc G. Bellemare and Rémi Munos. CoRR, abs/1710.10044, 2017.
 A Distributional Perspective on Reinforcement Learning Marc G. Bellemare, Will Dabney and Rémi Munos. CoRR, abs/1707.06887, 2017.
Today, Intel is announcing the release of our Reinforcement Learning Coach — an open source research framework for training and evaluating reinforcement learning (RL) agents by harnessing the power of multi-core CPU processing to achieve state-of-the-art results. Coach contains multi-threaded implementations for some of today’s leading RL algorithms, combined with various games and robotics environments.…
This is part 3 of a blog series on deep reinforcement learning. See “Part 1: Demystifying Deep Reinforcement Learning” for an introduction to the topic and “Part 2: Deep Reinforcement Learning with Neon” for the original implementation in Simple-DQN. In this blog post we will extend a Simple-DQN to work with OpenAI Gym, a new…
This is part 2 of a blog series on deep reinforcement learning. See part 1 “Demystifying Deep Reinforcement Learning” for an introduction to the topic. The first time we read DeepMind’s paper “Playing Atari with Deep Reinforcement Learning” in our research group, we immediately knew that we wanted to replicate this incredible result. It was…
Keep tabs on all the latest news with our monthly newsletter.