gazebo ignition simulation crashes when trying to simulate multiple turtlebot4

[abdosalem490]

Please provide the following information:

• OS: Ubuntu 22.04
• ROS Distro: Humble
• Built from source or installed: built from source
• Package version: e21bd2c

Expected behaviour
multiple Turtlebot4 spawned in the simulation

Actual behaviour
the simulation crashes

To Reproduce
Provide the steps to reproduce:

I am trying to simulate multiple turtlebot4 on Gazebo Fortress using ROS2 humble on my Ubuntu 22.04 but when I tried to run these commands, each in alone terminal:

ros2 launch turtlebot4_ignition_bringup turtlebot4_ignition.launch.py namespace:=salem
ros2 launch turtlebot4_ignition_bringup turtlebot4_spawn.launch.py namespace:=/robot2 x:=1.0 y:=0.0
ros2 launch turtlebot4_ignition_bringup turtlebot4_spawn.launch.py namespace:=/robot3 x:=2.0 y:=0.0

the simulation crashes and I get the following logs:

[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [ground_plane] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [warehouse] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [pallet_box_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_big_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_big_1] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_big_2] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_3] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_4] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_5] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_6] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_7] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_big_3] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_big_4] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_1] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [shelf_2] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [barrier_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [barrier_1] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [barrier_2] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [barrier_3] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [chair_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [chair_1] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [fchair_0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [fchair1] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [table0] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [Person 0 - Sitting] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:187] Parent entity with Id: [137] not found. Not adding model visual with ID[141]  and name [artifact_proximity_detector] to the rendering scene.
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [Person 1 - Standing] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [Person 2 - Walking] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [salem/standard_dock] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [salem/turtlebot4] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [/robot2/standard_dock] already exists
[ruby $(which ign) gazebo-1] [Err] [SceneManager.cc:201] Visual: [/robot2/turtlebot4] already exists
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::oakd_rgb_camera_frame::rgbd_camera] for entity [224]. Parent not found with ID[223].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::oakd_rgb_camera_frame::rgbd_camera] for entity [358]. Parent not found with ID[357].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/standard_dock::green_buoy_link::green_buoy] for entity [155]. Parent not found with ID[154].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/standard_dock::halo_link::halo] for entity [157]. Parent not found with ID[156].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/standard_dock::red_buoy_link::red_buoy] for entity [159]. Parent not found with ID[158].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/standard_dock::yellow_buoy_link::yellow_buoy] for entity [161]. Parent not found with ID[160].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::base_link::cliff_front_left] for entity [172]. Parent not found with ID[167].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::base_link::cliff_front_right] for entity [173]. Parent not found with ID[167].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::base_link::cliff_side_left] for entity [174]. Parent not found with ID[167].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::base_link::cliff_side_right] for entity [175]. Parent not found with ID[167].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_front_center_left::ir_intensity_front_center_left] for entity [194]. Parent not found with ID[193].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_front_center_right::ir_intensity_front_center_right] for entity [196]. Parent not found with ID[195].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_front_left::ir_intensity_front_left] for entity [198]. Parent not found with ID[197].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_front_right::ir_intensity_front_right] for entity [200]. Parent not found with ID[199].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_left::ir_intensity_left] for entity [202]. Parent not found with ID[201].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_right::ir_intensity_right] for entity [204]. Parent not found with ID[203].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::ir_intensity_side_left::ir_intensity_side_left] for entity [206]. Parent not found with ID[205].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[salem/turtlebot4::rplidar_link::rplidar] for entity [237]. Parent not found with ID[234].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/standard_dock::green_buoy_link::green_buoy] for entity [289]. Parent not found with ID[288].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/standard_dock::halo_link::halo] for entity [291]. Parent not found with ID[290].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/standard_dock::red_buoy_link::red_buoy] for entity [293]. Parent not found with ID[292].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/standard_dock::yellow_buoy_link::yellow_buoy] for entity [295]. Parent not found with ID[294].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::base_link::cliff_front_left] for entity [306]. Parent not found with ID[301].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::base_link::cliff_front_right] for entity [307]. Parent not found with ID[301].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::base_link::cliff_side_left] for entity [308]. Parent not found with ID[301].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::base_link::cliff_side_right] for entity [309]. Parent not found with ID[301].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_front_center_left::ir_intensity_front_center_left] for entity [328]. Parent not found with ID[327].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_front_center_right::ir_intensity_front_center_right] for entity [330]. Parent not found with ID[329].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_front_left::ir_intensity_front_left] for entity [332]. Parent not found with ID[331].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_front_right::ir_intensity_front_right] for entity [334]. Parent not found with ID[333].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_left::ir_intensity_left] for entity [336]. Parent not found with ID[335].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_right::ir_intensity_right] for entity [338]. Parent not found with ID[337].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::ir_intensity_side_left::ir_intensity_side_left] for entity [340]. Parent not found with ID[339].
[ruby $(which ign) gazebo-1] [Err] [RenderUtil.cc:1277] Failed to create sensor with name[/robot2/turtlebot4::rplidar_link::rplidar] for entity [371]. Parent not found with ID[368].

Other notes
Add anything else you thing is important.

[hilary-luo]

One thing you can do to make two robots work better is pass the namespace in without a slash prepended. For example:

ros2 launch turtlebot4_ignition_bringup turtlebot4_ignition.launch.py namespace:=salem
ros2 launch turtlebot4_ignition_bringup turtlebot4_spawn.launch.py namespace:=robot2 x:=1.0 y:=0.0

Does this make the system functional for your use case?

When I do this I am able to have the topics show up seemingly properly (I haven't gone through all of the topics or done that much in terms of testing. That being said, it does still seem to be trying to respawn elements from the world, I am not sure why that is. I will add this to my list of things to look into but anyone else feel free to look into this and contribute as well.

I am unable to spawn 3 robots although it is hard to say right now if this is a limitation because of the issue aforementioned or if it is just about the pure number of nodes / topics. You could try disabling sensors that you aren't using and see if that allows you to spawn 3 robots.

[abdosalem490]

The problem is that I needed at least 4 robots to perform navigation and mapping of an area for a school project and the turtlebot4 simulator doesn't support that feature yet without producing errors in the console.
would it work with turtlebot3 or shall I search for another robot simulator?

[msminirobot]

The same error occurs for me, having the Debian package installed.

[msminirobot]

The simulation also crashes when adding two robots and one object with inertia. It does not crash when adding one robot and two objects (boxes) with inertia.
Adding several boxes with inertia and one robot is working fine though.

[hilary-luo]

@abdosalem490 I cannot speak to the capabilities of Turtlebot3 as I have not used it before. You could inquire on the Turtlebot3 repo or robotics stack exchange.

@msminirobot Are you able to simulate two robots and with no extra objects? As you work through your troubleshooting feel free to make contributions back about this issue.

[msminirobot]

Yes, I am able to simulate two robots with no extra objects. It will show the error messages shown above though. I am trying to spawn a third robot will result in a segmentation fault that crashes the whole simulation. I also tested removing several topics I will not use, which did not help. In addition, trying to spawn more than two Create 3 (without the TurtleBot4 add ons) does not work either.

[msminirobot]

@abdosalem490 It is possible to simulate several TurtleBot3s in Gazebo. We used it with up to 9 robots so far.

[msminirobot]

The problem seems to be connected with the plugins and especially the sensors plugin. When including the turtlebot model in the SDF world file, I can spawn more than two robots when I remove the sensors-systems.so plugin but the simulation crashes if I include the sensors plugin again.

<plugin name='ignition::gazebo::systems::Sensors' filename='libignition-gazebo-sensors-system.so'>
      <render_engine>ogre</render_engine>
 </plugin>

[msminirobot]

Excluding the sensors-system plugin from the robot model and including it only once in the world file fixes the problem. For this purpose, the create3 description (create3_sim/irobot_create_common/irobot_create_description/urdf/create3.urdf.xacro) has to be updated by removing the sensors-system plugin (in addition, I removed the contact-system plugin). Both plugins should be included in the world file, e.g. just after the other plugins. From my tests, the second robot is then spawned without the error messages reported above and I also was able to include a third robot without crashing the simulation.

When using all sensors, the performance of the simulation will significantly drop when adding several robots. Removing unused sensor nodes from the create3_ros_ignition_bridge.launch.py can improve this.

[Kazimbalti]

Yes I have tried many times, for two turtlebot4 it works fine but when you want to add more than two turtlebot4, then the gazebo ignition crashes.

But I developed swarm of turtlebot3 in classic gazebo and implemented NAV2 and it works perfectly fine.

Yes the turtlebot4 simulation and hardware implementation for multiple turtlebot4 is much difficult.
I have two turtlebot4, and I am trying to implement some navigation but still trying my best.

[msminirobot]

@Kazimbalti Please see me comment above on how to fix the issue. I hope that there will be an official bug fix, but for now the workaround is well enough to get the simulation running.

[msminirobot]

@Kazimbalti I am talking about the TB4 since this issue and repository is about the TB4. I know that the TB3 works fine (see my previous comment: #60 (comment))