Should I define ROS2Message msg type?

[kumaemon0903]

I am applying RAI to my ROS robotic arm.
I have a trouble about creating my new agent.
This is the code where I have a problem.

self.connector.register_callback("/panda_status", self.monitor_status_topic)

def monitor_status_topic(self, msg: ROS2Message):
    """Update the arm readiness status based on the status message."""
    if msg.payload.status == "ready":
        self.arm_ready = True
    else:
        self.arm_ready = False

This means this part needs a /panda_state topic in the shape of ROS2Message.
I created the python script that publishes the topic.

import rclpy
from rclpy.node import Node

module_path = os.path.join("~", "rai_directory", "rai", "src", "rai_core")
sys.path.insert(0, module_path)

from rai.communication.ros2.connectors import ROS2Connector
from rai.communication.ros2.messages import ROS2Message, ROS2HRIMessage

connector = ROS2Connector()

try:
    while True:
        msg_instance = ROS2HRIMessage(payload={"text": "ready"})
        connector.send_message(
            message=msg_instance,  # ROS2Message
            target="/niryoone_status",
            msg_type="rai_interfaces/msg/HRIMessage"
        )
        print('publishing topics')
except KeyboardInterrupt:
    print('ctrl+C')

However, when I use connector.send_message method, it required me a message type so I tried to use ROS2Message type.
But there wasn't ROS2Message.msg file.
Should I make new ROS2Message.msg file?

[maciejmajek]

Hi, thank you for your interest in rai.

ROS2Message is a python class (not a ROS2 message) which allows transferring all sorts of ROS2 messages in similar API.

To use it (e.g., send message) please pass it to the method with payload={} dict of your message.

For example:

connector.send_message(ROS2Message(payload={"data": "Hello from RAI", ...))

The additional required arguments of send message are target (topic name) and msg_type e.g., "std_msgs/msg/String".

Not every part of the api requires explicit message type- receive_message does not need this.

[kumaemon0903]

Thank you for your reply.
I modified the code, which is show below.

connector.send_message(
            ROS2Message(payload={"status": "ready"}),
            target="/niryoone_status",
            msg_type="std_msgs/msg/String"
            )

Then I got an Python Error.

Traceback (most recent call last):
  File "/home/user/rai_directory/rai/niryoone/src/publish_ready/publish_ready/pub_ready_script.py", line 16, in <module>
    connector.send_message(
  File "/home/user/rai_directory/rai/src/rai_core/rai/communication/ros2/connectors/base.py", line 268, in send_message
    self._topic_api.publish(
  File "/home/user/rai_directory/rai/src/rai_core/rai/communication/ros2/api/topic.py", line 165, in publish
    msg = self.build_ros2_msg(msg_type, msg_content)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/rai_directory/rai/src/rai_core/rai/communication/ros2/api/base.py", line 112, in build_ros2_msg
    rosidl_runtime_py.set_message.set_message_fields(msg, msg_args)
  File "/opt/ros/jazzy/lib/python3.12/site-packages/rosidl_runtime_py/set_message.py", line 101, in set_message_fields
    set_message_fields_internal(msg, values, timestamp_fields)
  File "/opt/ros/jazzy/lib/python3.12/site-packages/rosidl_runtime_py/set_message.py", line 64, in set_message_fields_internal
    field = getattr(msg, field_name)
            ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'String' object has no attribute 'status'

Is it occured because of the message type of "std_msgs/msg/String"?
How can I publish /robot_status topic, or should I publish the status topic on my own?

[kumaemon0903]

I understood the usage of ROS2Message Class.
It just creates a topic from the inputs of pay_load, target, msg_type and confirms if the msg_type has a key that is specified in the payload.
So, if I want to publish a topic which has a status key, I should use a message type with a status key or create a new message type if there is no standard message that has a status key.
That's my understanding.
Is that correct?

[Juliaj]

Correct, the msg_type needs to be a valid ROS 2 message type. In your case, you would need to define a custom type with a status field. Find more info in ROS 2 tutorial https://docs.ros.org/en/jazzy/Tutorials/Beginner-Client-Libraries/Custom-ROS2-Interfaces.html. Cheers!

[kumaemon0903]

Thank you very much!
It worked for me.

[maciejmajek]

To circumvent the need of creating a dict with correct keys, you can use this.

With this you can do

msg = String()
msg.data = "Hello"
connector.send_message(ros2_message_to_dict(msg), target="...", msg_type="std_msgs/msg/String"))

This is something I'd love to see rai do automatically down the road. The problem is that ROS2 messages do not share base class, which means we need some other type of checking if the object is an actual ROS 2 message.

[Juliaj]

Hi @kumaemon0903, we're glad to hear that resolved your issue! 🎉. From the code snippet in your first message, it looks like you were following this walkthrough: https://robotecai.github.io/rai/tutorials/walkthrough. If that's correct, I'd love to hear your thoughts on it. For example, what parts did you find straightforward, and were there any areas that could use some improvement? If you have the time and feel inclined, it would be wonderful if you could capture any improvement ideas in a GitHub issue. And if you're interested in contributing directly, make a PR.

I also noticed you're working on running RAI with your ROS robotic arm—that sounds exciting! If you're open to it, I'd love to see updates on your progress in the RAI Discord channel: https://discord.com/channels/1309212732578336808/1309212732578336813. The community would really enjoy following along 😊!

[kumaemon0903]

Yes!
I followed that tutorial.
There was nothing difficult to understand if I have had a knowledge about langgraph.
The difficulties are mainly caused due to the lack of my knowledge about it.
Now I managed to move my manipulator using RAI framework, but LLM Agent rarely execute the correct motion as my natural language message.
Is it because of the bad of my system prompt?
Can I improve the conversion of message to correct motion by modification of the system prompt?

[kumaemon0903]

It is like when I enter a prompt of "Move the tool center point to the coordinate x=0.2, y=0.2, z=0.1 and grab something there.", the LLM Agent call the moveit! service to move the manipulator to the point x=0, y=0, z=0.
When I used openai api, the Agent tried to move the manpulator to the specified position but returned that the position is out of motion range of the robot. However it is in the motion range, so this is weird.