Skip to content

ropod-project/black-box-tools

Repository files navigation

Build Status

Summary

A collection of tools for offline processing of data from a robotic black box (as implemented in https://github.com/ropod-project/black-box).

The toolbox primarily uses Jupyter notebooks for data plotting and analysis with the hope that this simplifies the sharing, presentation, and above all, processing of data.

Dependencies

  • Data processing relies on the black_box_utils package that is included here as a standalone Python library
  • pymongo
  • numpy
  • matplotlib
  • scipy
  • rospy
  • rospy_message_converter
  • yaml
  • termcolor

notebooks

The following notebooks are provided with this toolbox:

  • sw_data_plot: Plots data from the smart wheels of a platform
  • cmd_vel_plot: Plots velocity commands
  • cmd_vel_plot-multi_db:
  • pivot_encoder_plot-multi_db:

sw_data_plot

Plots various smart wheel measurements, in particular:

  • wheel currents
  • wheel voltage measurements
  • wheel velocities
  • IMU acceleration measurements per wheel
  • IMU gyroscope measurements per wheel

cmd_vel_plot

Plots planar base velocity commands, namely:

  • linear velocity along x
  • linear velocity along y
  • angular velocity

black_box_utils

A Python package implementing various utilities for working with black box data. There are three scripts that are part of this package: data_utils, db_utils, and plot_utils.

db_utils

Defines a DBUtils class with the following static methods:

  • restore_db: Restores a MongoDB database dumped in a given directory
  • get_all_docs: Returns all documents contained in a specified collection of a given database
  • restore_subdbs: Restores multiple dumps into a single database.
  • dump_db: Dumps a MongoDB database in the specified directory.
  • get_data_collection_names: Returns the names of all black box data collections in the specified database.
  • clear_db: Drop all collections in the given database.
  • drop_db: Drops the given database.
  • get_subdb_metadata: Returns a list of dictionaries containing metadata about any sub-databases stored in the given database.
  • get_subdb_docs: Returns a list of dictionaries in which the keys are sub-database names and the values are lists of documents in the given collection corresponding to the respective sub-databases.
  • get_docs: Returns all documents contained in the specific collection of the given database within given time duration.
  • get_docs_of_last_n_secs: Return documents from given collection name of last n seconds from given db name.
  • get_doc_cursor: Returns a cursor for all documents in the specified collection of the given database which have the given 'timestamp' value in the given range.
  • get_collection_metadata: Returns the entry of the 'black_box_metadata' collection for the specified collection.
  • get_oldest_doc: Returns the oldest document in the given collection name.
  • get_newest_doc: Returns the newest document in the given collection name.
  • get_last_doc_before: Returns the last document in the collection with the given name that is before given timestamp.
  • get_db_oldest_timestamp: Gets the oldest record in the mongo db and returns the corresponding timestamp.
  • get_db_newest_timestamp: Gets the newest record in the mongo db and returns the corresponding timestamp.
  • get_db_client: Returns a MongoDB client at :.
  • get_db_host_and_port: Returns a (host, port) tuple which is ("localhost", 27017) by default, but the values can be overridden by setting the environment variables "DB_HOST" and "DB_PORT" respectively.

data_utils

Defines a DataUtils class with the following static methods:

  • get_all_measurements: Returns all measurements of a single variables (that potentially has multiple instances - e.g. one instance per wheel)
  • filter_data: Filters data using a given data filter and returns the filtered array.
  • find_correlated_variables: Returns a list of variable name pairs where each pair denotes that two measurements in a given measurement matrix are correlated (based on the Pearson correlation coefficient).
  • get_windowed_correlations: Returns a list of variable name pairs and a 2D list of windowed pairwise correlations, given a list of variable names and a data array.
  • get_var_value: Returns the value of a given variable in the given dictionary;
  • get_variable_list: Returns a list of the names of all variables stored in a given collection.
  • get_flattened_variable_names: Recursive method that returns a flattened list of the variable names in a given dictionary
  • expand_var_names: Generates a new list of variable names from a given list of such that the * character in each entry of the original list is replaced by a zero-based index
  • get_bb_query_msg_template: 'Returns a dictionary which represents a template for a black box query message.
  • get_bb_query_msg: Returns a black box data query message.
  • get_bb_latest_data_query_msg: Returns a black box latest data query message.
  • parse_bb_variable_msg: Returns a nested dictionary that reconstructs the structure of the data represented by the variables in the list of a given black box variable message dictionary.
  • parse_bb_data_msg: Returns a tuple (variables, data), where variables is a list of variables that were queried and data a list of variable values.
  • parse_bb_latest_data_msg: Returns a tuple (variables, data), where variables is a list of variables that were queried and data a list of the latest variable values.
  • split_into_windows: Given a one-dimensional list of elements "data", returns a 2D numpy array of sliding windows of size "window_size".
  • safe_literal_eval: Uses ast.literal_eval to parse a given string.

plot_utils

Defines a PlotUtils class with the following static methods:

  • subplot_data: Plots a single time series on a subplot (and potentially annotates any events of interest)
  • subplot_data_lists: Plots multiple time series on a single subplot (and potentially annotates any events of interest)
  • plot_position_velocity: Plots the position of robot with dots and its velocity at each position with a line. Colors represent the intensity of velocity.

About

Python library for interacting with a robotic black box

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published