uORB docs parser

[hamishwillee]

Now that we have a docs standard for UORB messages, it makes sense to generate better docs from them.

This parser splits out the fields, constants etc and generates docs that look like this: #24977 (comment)

It is relatively easy to run and test. In ubuntu from the root do the following to generate all the docs into a folder msg_output2:

python3 Tools/msg/generate_msg_docs.py -d msg_output2

If you are interested in getting logs you can append -e -m ActuatorMotors Wind to generate output that shows any problems with particular files - in this case Wind or ActuatorMotors. If you have e but omit m you get a log of all errors.
The logs show stuff like "multiple sequential spaces".
Note though, nothing is an error. It will generate the docs it can from the data it has, and that is FINE.

You could then copy all the files form msg_output2 into /docs/en/msg_docs/ to preview.

Where are we at:

• Generation looks good EXCEPT for VehicleCommand. More work needed. My view is that even without VehicleCommand, this is pretty good.
• Reporting probably could use more work - I'll have to go through the standard and check I have everything. BUT it is pretty good.

It will be tough to review because the code is non-trivial.
Essentially it is a line parser

• first detects the top comments section, if present, and finds the short and long descripton. Then goes to new state for parsing fields, constants, topics, internal comments.
• Splits up fields/constants and saves values to objects it can later render
• Also adds errors to an error variable by type. This is output if you specify e flag, and can be filtered by message using the m flag and a space separated name list.

[hamishwillee]

@MaEtUgR FYI only, I'm still tweaking this, but for the latest Battery Status proposed, the message would look like this.

---

BatteryStatus (UORB message)

Battery status.

Battery status information for up to 3 battery instances.
These are populated from power module and smart battery device drivers, and one battery updated from MAVLink.
Battery instance information is also logged and streamed in MAVLink telemetry.

TOPICS: battery_status

Fields

Name (type)	Unit [Frame]	Values	Description
timestamp (uint64)	us		Time since system start
connected (bool)			Whether or not a battery is connected. For power modules this is based on a voltage threshold.
voltage_v (float32)	V		Battery voltage (Invalid: 0)
current_a (float32)	A		Battery current (Invalid: -1)
current_average_a (float32)	A		Battery current average (for FW average in level flight) (Invalid: -1)
discharged_mah (float32)	mAh		Discharged amount (Invalid: -1)
remaining (float32)		range: 0 to 1	Remaining capacity (Invalid: -1)
scale (float32)		min: 1	Scaling factor to compensate for lower actuation power caused by voltage sag (Invalid: -1)
time_remaining_s (float32)	s		Predicted time remaining until battery is empty under previous averaged load (Invalid: NaN)
temperature (float32)	°C		Temperature of the battery (Invalid: NaN)
cell_count (uint8)			Number of cells (Invalid: 0)
source (uint8)		SOURCE	Battery source
priority (uint8)			Zero based priority is the connection on the Power Controller V1..Vn AKA BrickN-1
capacity (uint16)	mAh		Capacity of the battery when fully charged
cycle_count (uint16)			Number of discharge cycles the battery has experienced
average_time_to_empty (uint16)	minutes		Predicted remaining battery capacity based on the average rate of discharge
manufacture_date (uint16)			Manufacture date, part of serial number of the battery pack. Formatted as: Day + Month×32 + (Year–1980)×512
state_of_health (uint16)	%	range: 0 to 100	State of health. FullChargeCapacity/DesignCapacity
max_error (uint16)	%	range: 1 to 100	Max error, expected margin of error in the state-of-charge calculation
id (uint8)			ID number of a battery. Should be unique and consistent for the lifetime of a vehicle. 1-indexed
interface_error (uint16)			Interface error counter
voltage_cell_v (float32[14])	V		Battery individual cell voltages (Invalid: 0)
max_cell_voltage_delta (float32)	V		Max difference between individual cell voltages
is_powering_off (bool)			Power off event imminent indication, false if unknown
is_required (bool)			Set if the battery is explicitly required before arming
warning (uint8)		WARNINGSTATE	Current battery warning
faults (uint16)		FAULT	Smart battery supply status/fault flags (bitmask) for health indication
full_charge_capacity_wh (float32)	Wh		Compensated battery capacity
remaining_capacity_wh (float32)	Wh		Compensated battery capacity remaining
over_discharge_count (uint16)			Number of battery overdischarge
nominal_voltage (float32)	V		Nominal voltage of the battery pack
internal_resistance_estimate (float32)	Ohm		Internal resistance per cell estimate
ocv_estimate (float32)	V		Open circuit voltage estimate
ocv_estimate_filtered (float32)	V		Filtered open circuit voltage estimate
volt_based_soc_estimate (float32)		range: 0 to 1	Normalized volt based state of charge estimate
voltage_prediction (float32)	V		Predicted voltage
prediction_error (float32)	V		Prediction error
estimation_covariance_norm (float32)			Norm of the covariance matrix

Enums

SOURCE {#SOURCE}

Name (type)	Value	Description
SOURCE_POWER_MODULE (uint8)	0	Power module (analog ADC or I2C power monitor)
SOURCE_EXTERNAL (uint8)	1	External (MAVLink, CAN, or external driver)
SOURCE_ESCS (uint8)	2	ESCs (via ESC telemetry)

WARNING {#WARNING}

Name (type)	Value	Description
WARNING_NONE (uint8)	0	No battery low voltage warning active
WARNING_LOW (uint8)	1	Low voltage warning
WARNING_CRITICAL (uint8)	2	Critical voltage, return / abort immediately
WARNING_EMERGENCY (uint8)	3	Immediate landing required
WARNING_FAILED (uint8)	4	Battery has failed completely

STATE {#STATE}

Name (type)	Value	Description
STATE_UNHEALTHY (uint8)	6	Battery is diagnosed to be defective or an error occurred, usage is discouraged / prohibited. Possible causes (faults) are listed in faults field
STATE_CHARGING (uint8)	7	Battery is charging

FAULT {#FAULT}

Name (type)	Value	Description
FAULT_DEEP_DISCHARGE (uint8)	0	Battery has deep discharged
FAULT_SPIKES (uint8)	1	Voltage spikes
FAULT_CELL_FAIL (uint8)	2	One or more cells have failed
FAULT_OVER_CURRENT (uint8)	3	Over-current
FAULT_OVER_TEMPERATURE (uint8)	4	Over-temperature
FAULT_UNDER_TEMPERATURE (uint8)	5	Under-temperature fault
FAULT_INCOMPATIBLE_VOLTAGE (uint8)	6	Vehicle voltage is not compatible with this battery (batteries on same power rail should have similar voltage)
FAULT_INCOMPATIBLE_FIRMWARE (uint8)	7	Battery firmware is not compatible with current autopilot firmware
FAULT_INCOMPATIBLE_MODEL (uint8)	8	Battery model is not supported by the system
FAULT_HARDWARE_FAILURE (uint8)	9	Hardware problem
FAULT_FAILED_TO_ARM (uint8)	10	Battery had a problem while arming
FAULT_COUNT (uint8)	11	Counter. Keep this as last element

Constants

Name (type)	Value	Description
MESSAGE_VERSION (uint32)	1
MAX_INSTANCES (uint8)	3

Source Message

Source file (GitHub)

Details

# Battery status
#
# Battery status information for up to 3 battery instances.
# These are populated from power module and smart battery device drivers, and one battery updated from MAVLink.
# Battery instance information is also logged and streamed in MAVLink telemetry.

uint32 MESSAGE_VERSION = 1
uint8 MAX_INSTANCES = 3

uint64 timestamp  # [us] Time since system start

bool connected             # Whether or not a battery is connected. For power modules this is based on a voltage threshold.
float32 voltage_v          # [V] [@invalid 0] Battery voltage
float32 current_a          # [A] [@invalid -1] Battery current
float32 current_average_a  # [A] [@invalid -1] Battery current average (for FW average in level flight)
float32 discharged_mah     # [mAh] [@invalid -1] Discharged amount
float32 remaining          # [@range 0,1] [@invalid -1] Remaining capacity
float32 scale              # [-] [@range 1,] [@invalid -1] Scaling factor to compensate for lower actuation power caused by voltage sag
float32 time_remaining_s   # [s] [@invalid NaN] Predicted time remaining until battery is empty under previous averaged load
float32 temperature        # [°C] [@invalid NaN] Temperature of the battery
uint8 cell_count           # [-] [@invalid 0] Number of cells


uint8 source                   # [@enum SOURCE] Battery source
uint8 SOURCE_POWER_MODULE = 0  # Power module (analog ADC or I2C power monitor)
uint8 SOURCE_EXTERNAL = 1      # External (MAVLink, CAN, or external driver)
uint8 SOURCE_ESCS = 2          # ESCs (via ESC telemetry)

uint8 priority                # [-] Zero based priority is the connection on the Power Controller V1..Vn AKA BrickN-1
uint16 capacity               # [mAh] Capacity of the battery when fully charged
uint16 cycle_count            # [-] Number of discharge cycles the battery has experienced
uint16 average_time_to_empty  # [minutes] Predicted remaining battery capacity based on the average rate of discharge
uint16 manufacture_date       # [-] Manufacture date, part of serial number of the battery pack. Formatted as: Day + Month×32 + (Year–1980)×512
uint16 state_of_health        # [%] [@range 0, 100] State of health. FullChargeCapacity/DesignCapacity
uint16 max_error              # [%] [@range 1, 100] Max error, expected margin of error in the state-of-charge calculation
uint8 id                      # [-] ID number of a battery. Should be unique and consistent for the lifetime of a vehicle. 1-indexed
uint16 interface_error        # [-] Interface error counter

float32[14] voltage_cell_v      # [V] [@invalid 0] Battery individual cell voltages
float32 max_cell_voltage_delta  # [V] Max difference between individual cell voltages

bool is_powering_off  # Power off event imminent indication, false if unknown
bool is_required      # Set if the battery is explicitly required before arming

uint8 warning                # [@enum WARNING STATE] Current battery warning
uint8 WARNING_NONE = 0       # No battery low voltage warning active
uint8 WARNING_LOW = 1        # Low voltage warning
uint8 WARNING_CRITICAL = 2   # Critical voltage, return / abort immediately
uint8 WARNING_EMERGENCY = 3  # Immediate landing required
uint8 WARNING_FAILED = 4     # Battery has failed completely
uint8 STATE_UNHEALTHY = 6    # Battery is diagnosed to be defective or an error occurred, usage is discouraged / prohibited. Possible causes (faults) are listed in faults field
uint8 STATE_CHARGING = 7     # Battery is charging

uint16 faults                          # [@enum FAULT] Smart battery supply status/fault flags (bitmask) for health indication
uint8 FAULT_DEEP_DISCHARGE = 0         # Battery has deep discharged
uint8 FAULT_SPIKES = 1                 # Voltage spikes
uint8 FAULT_CELL_FAIL= 2               # One or more cells have failed
uint8 FAULT_OVER_CURRENT = 3           # Over-current
uint8 FAULT_OVER_TEMPERATURE = 4       # Over-temperature
uint8 FAULT_UNDER_TEMPERATURE = 5      # Under-temperature fault
uint8 FAULT_INCOMPATIBLE_VOLTAGE = 6   # Vehicle voltage is not compatible with this battery (batteries on same power rail should have similar voltage)
uint8 FAULT_INCOMPATIBLE_FIRMWARE = 7  # Battery firmware is not compatible with current autopilot firmware
uint8 FAULT_INCOMPATIBLE_MODEL = 8     # Battery model is not supported by the system
uint8 FAULT_HARDWARE_FAILURE = 9       # Hardware problem
uint8 FAULT_FAILED_TO_ARM = 10         # Battery had a problem while arming
uint8 FAULT_COUNT = 11                 # Counter. Keep this as last element

float32 full_charge_capacity_wh  # [Wh] Compensated battery capacity
float32 remaining_capacity_wh    # [Wh] Compensated battery capacity remaining
uint16 over_discharge_count      # [-] Number of battery overdischarge
float32 nominal_voltage          # [V] Nominal voltage of the battery pack

float32 internal_resistance_estimate  # [Ohm] Internal resistance per cell estimate
float32 ocv_estimate                  # [V] Open circuit voltage estimate
float32 ocv_estimate_filtered         # [V] Filtered open circuit voltage estimate
float32 volt_based_soc_estimate       # [-] [@range 0, 1] Normalized volt based state of charge estimate
float32 voltage_prediction            # [V] Predicted voltage
float32 prediction_error              # [V] Prediction error
float32 estimation_covariance_norm    # [-] Norm of the covariance matrix

[sfuhrer]

Great stuff, that will make the topic descriptions much more readable. Would be cool to include the format check in a CI run as well.

Couple of comments:

• to make the Name more readable I would not put it in the same fields as type
• SOURCE {#SOURCE} - what's the hashtag part for?
• are the fields of the enums gonna be links?

[MaEtUgR]

same fields as type

If it's already a table I'd also put the type in a separate column.

SOURCE {#SOURCE} - what's the hashtag part for?

I think that's a rendering issue. The # should not be shown in the final rendering but is an anchor on the page such that if you click on the "SOURCE" link of a field you are sent to the table. Same as in the MAVLink docs.

range: 1 to 100

For the ranges I'd simply output 1 - 100 (or [1, 100]) but that's just personal preference.

I quickly checked what ROS generally does to document messages and it seems I either don't find it or it just shows the raw message definition... https://docs.ros.org/en/iron/p/control_msgs/interfaces/msg/JointJog.html Your suggestion is definitely nicer.

[hamishwillee]

@sfuhrer @MaEtUgR

1. I've made those changes. Its a bit ugly because right-side crops.

   

2. Do you think this alternative is better?

   

3. Yes, the # disappears when rendered.

4. I have modified so that the enums/constants don't show as links. They are anchors though.

[sfuhrer]

@sfuhrer @MaEtUgR

1. I've made those changes. Its a bit ugly because right-side crops.

I like that first option the best.

[MaEtUgR]

I'd also vote for option 1. It's very clear. btw very similar to the MAVLink message definition rendering.

[hamishwillee]

@MaEtUgR @mrpollo I plan to merge this tomorrow. It needs a little more love, but can be iterated from here. Produces much better docs. Comments welcome either before or after that.

NOTE, since you saw this last it improves rendering by making the pages for uorb topics wide. It also does basic rendering for vehicle command. More work to be done there in error reporting, but already better than current rendering.