Integration with Airsim

[OasisArtisan]

Integration with Airsim

This thread will discuss the effort to integrate Airstack with Airsim.

Why add Airsim when we already have IsaacSim?

Pros:

• Seems much lighter than IsaacSim as it relies on precompiled binaries.
• Airsim is built on unreal engine and since most of our assets come from unreal engine anyways this may speed up scene editing and porting.
• Bigger community. More tutorials, more readily available benchmarks (E.g UAV-ON).
• Easier learning curve than Isaac Sim.
• Integration of an additional sim will allow Airstack to serve a wider range of people.

Cons:

• Airsim has a very small number of precompiled binaries (~12 in the older version and ~3 in the current ProjectAirsim). Adding more requires compiling an unreal engine scene with the Airsim plugin. Doesn't seem like it can be automated.
• Scenes are precompiled, unless you compile the ability to add certain objects / assets and bake it into the binary, there is no way to modify the scene except by editing it in unreal engine and recompiling. Let's say we want to add the ability to spawn dear, we will need to go to each scene we have, add that ability and recompile each scene.

Of course, this is an addition not a replacement, so a user can choose which sim to use based on their needs.

Which Airsim to integrate ?

Option	PX4	Lidar	ROS 2	Pros	Cons
Legacy AirSim (archived 2022)	Yes	Yes	Via wrapper	Well-documented; large community	Archived; known bugs; UE4 only
Project AirSim (IAMAI Simulations)	Yes	Yes	Via wrapper	UE5; MIT license; actively maintained	Smaller community; less field-tested
Colosseum / Cosys-AirSim	Yes	Yes	Yes (native C++)	UE5; best ROS 2 integration; active community	Fragmented forks; smaller team

I am leaning more towards integrating the legacy Airsim just to get instant access to the UAV-ON benchmark. But Ideally, it would be great if we can integrate both Airsim and Project Airsim (Legacy Airsim for established benches, and Project Airsim for long-term support and new environments). Ideally if done right the integration should be almost the same, perhaps they can even share the same docker and same files with just some env var changes. Although not sure if this is possible.

It would be nice to start having standard benchmarks in Airstack that people can quickly evaluate on. If we standardize RAVEN's benchmark, and we have UAV-ON benchmark, then we can have a nice set for object search. Evaluation eventually should be part of Airstack I feel but that's another discussion.

Next I will discuss concrete integration plans.

[AviDube]

As someone who has supported/worked on AirSim for 4 years, here are my thoughts:

The Colosseum project is no longer maintained unless the community contributes updates. The core team that worked on it has moved on. If you're looking for similar improvements and active contributions, I recommend using Cosys—one of the developers who helped with Colosseum actually started that branch after we ceased working on it. We invested considerable effort into enhancing the legacy AirSim repository, which is quite messy. I haven't explored Project AirSim extensively, but it might be a better option for integration.
Additionally, the jump from UE4 to UE5 is significant; with UE5, we successfully trained a camera model that wouldn't have been feasible in UE4. We were also developing a rewritten physics engine to incorporate PyBullet physics, especially since legacy AirSim's physics could break when sped up from real-time. Unfortunately, we didn't complete that work. Another thing to consider is that I really appreciated the binaries, which allowed me to work from computers even with UE5 and limited computing power. We had a dedicated team of level and object designers following our specifications; their game development experience made this easier. We implemented a slider GUI at the start of each level, enabling us to adjust some parameters without recompiling binaries for minor changes. One major point, in my opinion, is that AirSim is not optimized at all. Isaac Sim, with its CUDA optimizations, will perform better when training multiple agents simultaneously. However, for fine-tuning, AirSim can suffice. Overall, I believe it depends on the case. After working with Isaac Sim, I think the ideal solution would combine the strengths of both. If you have questions, I'm happy to chat. I've spent so much time on this that I don't want it to go to waste, LOL.

[OasisArtisan]

Thanks a lot for the comments.
I've read in multiple places how IsaacSim is more optimized and scalable. My first hand experience shows IsaacSim as being very heavy and slow though whereas a precompiled Airsim binary is quite light. Multiple people in our lab @YunwooLee94 @caomuqing @seungchan-kim share the same experience. I don't know why there is this discrepancy.