gym-turtlebot

A custom Gymnasium environment to get started with deep reinforcement learning (DRL) using ROS 2 and Gazebo.

---

Overview

gym-turtlebot provides a lightweight, modular environment for training reinforcement learning agents using a TurtleBot4 robot simulated in ROS 2 Jazzy and Gazebo Harmonic. It wraps the simulation in a standard Gymnasium interface to enable fast prototyping and evaluation of DRL algorithms in realistic robotics environment.

---

Features

• ROS 2–based robot control integration
• Customizable Gazebo simulation worlds
• Gymnasium-compliant API
• Supports headless and visual simulation
• Easily extendable for custom robots

---

Installation

Using Docker

Prerequisites

• Docker
• VSCode with remote containers plugin installed.

Note

The Docker setup is adapted from vscode_ros2_workspace.

Refer to the template repository for more detailed instructions on how to use VS-code and docker for ROS.

Steps

1. Build

# Clone the repo into your ROS 2 workspace
mkdir ~/ros2_ws
cd ~/ros2_ws
git clone https://github.com/anurye/gym-turtlebot.git

# Install dependencies
cd ~/ros2_ws
./setup.sh

# Build the workspace and source the setup script
cd ~/ros2_ws
./build.sh
source install/local_setup.bash

2. Simulation

cd ~/ros2_ws
ros2 launch tb4_gz_sim simulation.launch.py

3. Deep reinforcement training example

Important

Gazebo Sim does not support model reset at the moment. To work around this limitation, we reset both the robot's odometry and its pose by using:

1. The SetPose service from the robot localization package to reinitialize odometry, and
2. The SetEntityPose service from the ros_gz_interfaces package to reposition entities in Gazebo.

python3 src/tb4_drl_navigation/tb4_drl_navigation/examples/sac.py train

Local Setup

Prerequisites

• Ubuntu 24.04
• ROS 2 Jazzy
• Gazebo Harmonic

Steps

The steps are the same as above.