a3c

English | 中文

An implementation of the A3C algorithm using PyTorch, referencing the TensorFlow implementation from openai/universe-starter-agent and the PyTorch implementation from ikostrikov/pytorch-a3c.
Based primarily on ikostrikov's work with some modifications, aiming to create concise and easily understandable code.

Dependencies

• Python 3.6
• Anaconda
• PyTorch
• gym
• gym[atari]
• opencv-python

Getting Started

The following instructions are based on Ubuntu 16.04 LTS environment. When installing Anaconda, press Enter and Yes for all prompts.

wget https://repo.continuum.io/archive/Anaconda3-4.4.0-Linux-x86_64.sh
bash Anaconda3-4.4.0-Linux-x86_64.sh
source .bashrc
conda install pytorch torchvision -c soumith
conda install opencv
conda install libgcc
conda install -c conda-forge ffmpeg
pip install gym[Atari]
sudo apt-get update
sudo apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

Rendering on a server

When running on a server, you need to use xvfb to create a virtual display for rendering support.

xvfb-run -s "-screen 0 1400x900x24" python main.py --env-name "Pong-v0" --num-processes 8

Architecture

Result

Trained for 2 hours on GCP using 8 CPU cores.

References

Asynchronous Methods for Deep Reinforcement Learning
openai/universe-starter-agen
ikostrikov/pytorch-a3c