Help about Use Case: Hide and Seek

[kaanoz1]

I am interested in implementing a collaborative/competitive use case using the RAI: a multi-robot hide-and-seek game. This scenario aims to test reasoning, multi-agent communication, and autonomous decision-making.

Scenario workflow that I planned:

1. Mapping & Discovery Phase: Both robots (Seeker and Hider) explore the environment using SLAM/Navigation tools to identify "hiding spots" (corners, behind furniture, etc.) and build a shared or individual understanding of the map.

2. Game Initialization: Once the command is given by user via GUI, the game starts.

3. The Waiting Period: The Seeker robot must remain stationary at a "home base" for a set duration (for example 60 seconds) with its "eyes closed" (disabling/ignoring camera input), simulating the counting phase.

4. Hiding Phase: During this 60 seconds, the Hider robot must autonomously select the best spot (based on distance or occlusion) and navigate there.

5. The Search: The Seeker activates its sensors and begins searching for the Hider based on its reasoning of potential hiding spots.

So its a common hide and seek game.

Questions about "How I can implement it?":

1. Inter-Agent Communication: Can I use ROS2Connector or ROS2HRIConnector to facilitate direct messaging between agents (e.g., "I found you", "I am hidden")? Is there a recommended pattern for agent-to-agent topic communication within RAI?

2. Agent Architecture: Is the existing ReAct Agent suitable for this loop, or should I define a custom agent class?

3. Extending ReAct: If I stay with ReActAgent, what is the best way to extend its logic to handle state-dependent behaviors (like the Seeker waiting/ignoring input during the countdown)? Should this be handled via a complex SystemPrompt or by overriding the input_callback?

4. Multi-Instance Support: Are there any known limitations when running two distinct RAI agent instances (each with its own LLM session and tools) in the same ROS 2 domain?

I want to implement this use case scenario without reinvent the wheel.

Thank you

[maciejmajek]

Hi! Very cool use case!

1. The HRI module was primarily intended for human-robot interaction, but it works perfectly fine for agent-to-agent communication as well. To keep things simple and reduce friction, I’d start with a basic std_msgs/msg/String message type. You can later move to more structured messages if needed.

2. The ReAct agent is essentially a "think-act" loop, but for your hide and seek scenario, it may struggle with continuous or state-driven loops because its reasoning is mostly context-driven rather than time/state-enforced. A more sophisticated structure, either a custom agent or a state-aware wrapper around ReAct would likely perform better.

Note: For your particular use case, one friction point I can already envision is making the LLM control the exploration points. We’ve tested map understanding (e.g., giving a basic NAV2 map as an image to a VLM with a grid), and it wasn’t that great. It’s generally hard for the LLM to reliably connect image-based maps with real-world coordinates.

3. Relying on the LLM alone for timing or state logic (like enforcing the Seeker to wait 60 seconds) is tricky. I’d recommend overriding input_callback or implementing a state machine that tracks game phases (hiding, seeking, etc.). This keeps the timing and behavior logic deterministic while still allowing ReAct or LLM-based reasoning for decision-making.

4. RAI fully supports running multiple agent instances in the same ROS 2 domain. Each agent can have its own LLM session and toolset without conflicts.

[kaanoz1]

Thank you for your answer. But we need help about a stage that we need to implement in order to accomplish this use case. We need to find "exact" coordinates (I will dwell on why we need 'exact' ones) where our robot can "hide". It might be behind the bed, under the table, or next to the sofa, doesnt matter. Our plan is that first we need to wander map so that we can detect spots that our robot can hide. Therefore we can store these positions and whenever user wants our robot to hide, it can select random or specific one in line with user prompts, it can go that coordinate with navigate_to_pose tool. So our core problem is how we can get 'exact' coordinates. May you help on that? Why we need exact coordinates? Because most of "estimated" coordinates are not working, too far from the object (bed, couch etc..) or coordinates fall inside an object so robot cannot go provided point. How can we fix this issue?