Split converters (ros2#70)

* ros2GH-134 Split converter interface into Serializer and Deserializer

- Allow plugins which can only read or write
- Most important example: plugin for old rosbags

* ros2GH-134 Switch to using serializer and deserializer in factory

* ros2GH-134 Add test for serializer plugin

* ros2GH-134 Try to load Serializer and Deserializer

- When loading a serializer, try to load both serializer and converter
- Similar for deserializers

* ros2GH-134 Fix e2e test after improving error message for missing converters

* ros2GH-134 Remove duplicate code in converter factory

* ros2GH-134 Change namespace of converter interfaces

- adapt namespaces to folder structure
- folder structure similar to rosbag2_storage

* ros2GH-134 Hide pluginlib import via pimpl

- We want to use template functions that require the pluginlib import
- The pluginlib import should not be exported (this creates issues with
  downstream packages)
- Similar to the storage factory, use a pimpl

* ros2GH-134 Adapt documentation

* Minor documentation updates

Co-Authored-By: Martin-Idel-SI <external.Martin.Idel@bosch-si.com>

* ros2GH-134 Rename converter interface to drop "interface"

- already visible from namespace

Author: Martin-Idel-SI

docs/converter_plugin_development.md

@@ -8,9 +8,10 @@ To simplify conversion between all formats each plugin provides functions to con
 
 Rosbag2 is shipped with a default converter plugin to convert between ROS 2 messages and serialized messages in the CDR format (this is the general serialization format specified by DDS).
 
-## Writing converter plugins
+## Writing converter plugins for both serialization and deserialization
 
-To write a plugin `MyConverter`, implement the interface `rosbag2::SerializationFormatConverterInterface`.
+Most converter plugins are used to convert between a custom serialization format in both directions, i.e. serialization and deserialization.
+To write a plugin `MyConverter` supporting conversion to and from a custom serialization format, implement the interface `rosbag2::converter_interfaces::SerializationFormatConverter`.
 
 The plugin interface provides two functions: 
 
@@ -22,7 +23,7 @@ Add the following lines in the `my_converter.cpp`:
 
 ```
 #include "pluginlib/class_list_macros.hpp"
-PLUGINLIB_EXPORT_CLASS(MyConverter, rosbag2::SerializationFormatConverterInterface)
+PLUGINLIB_EXPORT_CLASS(MyConverter, rosbag2::converter_interfaces::SerializationFormatConverter)
 ```
 
 Furthermore, we need some meta-information in the form of a `plugin_description.xml` file.
@@ -32,7 +33,7 @@ In the case of `MyConverter` this would look like:
   <class
     name="my_storage_format_converter"
     type="MyConverter"
-    base_class_type="rosbag2::SerializationFormatConverterInterface"
+    base_class_type="rosbag2::converter_interfaces::SerializationFormatConverter"
   >
     <description>This is a converter plugin for my storage format.</description>
   </class>
@@ -48,6 +49,16 @@ pluginlib_export_plugin_description_file(rosbag2 plugin_description.xml)
 
 The first argument `rosbag2` denotes the ament index key we add our plugin to (this will always be `rosbag2` for converter plugins), while the second argument is the path to the plugin description file.
 
+## Writing converter plugins for only serialization or deserialization
+
+It is possible to write a plugin which only supports converting to OR from a custom serialization format.
+For example, the default plugin for legacy ROS1 rosbags  supports only conversion from a ROS 1 to a ROS 2 message format (deserialization), since rosbag2 is not intended to be used to write rosbags in the old format.
+When a similar use case applies, it is possible to provide a plugin which supports only one direction (serialization or deserialization).
+
+In order to write a plugin `MyDeserializer` supporting conversion from a custom serialization format, implement the interface `rosbag2::converter_interfaces::SerializationFormatDeserializer.
+This interface only provides the `deserialize` method mentioned above.
+Similarly, there exists a `rosbag2::converter_interfaces::SerializationFormatSerializer` for serialization plugins.
+
 ## How to choose conversion at runtime
 
 The conversion will be chosen automatically according to the storage format specified in the bagfile or specified by the user.

rosbag2/CMakeLists.txt

@@ -82,12 +82,13 @@ if(BUILD_TESTING)
   ament_lint_auto_find_test_dependencies()
 
   add_library(
-    converter_test_plugin
+    converter_test_plugins
     SHARED
+    test/rosbag2/serializer_test_plugin.cpp
     test/rosbag2/converter_test_plugin.cpp)
-  target_link_libraries(converter_test_plugin rosbag2)
+  target_link_libraries(converter_test_plugins rosbag2)
   install(
-    TARGETS converter_test_plugin
+    TARGETS converter_test_plugins
     ARCHIVE DESTINATION lib
     LIBRARY DESTINATION lib
     RUNTIME DESTINATION bin)

rosbag2/include/rosbag2/converter.hpp

@@ -23,7 +23,7 @@
 #include "rosbag2/converter_options.hpp"
 #include "rosbag2/serialization_format_converter_factory.hpp"
 #include "rosbag2/serialization_format_converter_factory_interface.hpp"
-#include "rosbag2/serialization_format_converter_interface.hpp"
+#include "rosbag2/converter_interfaces/serialization_format_converter.hpp"
 #include "rosbag2/types.hpp"
 #include "rosbag2/visibility_control.hpp"
 
@@ -78,8 +78,8 @@ class ROSBAG2_PUBLIC Converter
 
 private:
   std::shared_ptr<SerializationFormatConverterFactoryInterface> converter_factory_;
-  std::unique_ptr<SerializationFormatConverterInterface> input_converter_;
-  std::unique_ptr<SerializationFormatConverterInterface> output_converter_;
+  std::unique_ptr<converter_interfaces::SerializationFormatDeserializer> input_converter_;
+  std::unique_ptr<converter_interfaces::SerializationFormatSerializer> output_converter_;
   std::unordered_map<std::string, ConverterTypeSupport> topics_and_types_;
 };
 

rosbag2/include/rosbag2/converter_interfaces/serialization_format_converter.hpp

@@ -0,0 +1,46 @@
+// Copyright 2018, Bosch Software Innovations GmbH.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_
+#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_
+
+#include <memory>
+#include <string>
+
+#include "rosbag2/types/introspection_message.hpp"
+#include "rosbag2/types.hpp"
+#include "rcutils/types.h"
+#include "rosbag2_storage/serialized_bag_message.hpp"
+#include "rosidl_typesupport_cpp/message_type_support.hpp"
+#include "rosbag2/converter_interfaces/serialization_format_serializer.hpp"
+#include "rosbag2/converter_interfaces/serialization_format_deserializer.hpp"
+
+/**
+ * This is a convenience class for plugin developers. When developing a plugin to both write and
+ * read a specified serialization format, inherit from this class
+ */
+namespace rosbag2
+{
+
+namespace converter_interfaces
+{
+
+class SerializationFormatConverter
+  : public SerializationFormatSerializer, public SerializationFormatDeserializer
+{};
+
+}  // namespace converter_interfaces
+}  // namespace rosbag2
+
+#endif  // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_

…alization_format_converter_interface.hpp …es/serialization_format_deserializer.hpprosbag2/include/rosbag2/serialization_format_converter_interface.hpp renamed to rosbag2/include/rosbag2/converter_interfaces/serialization_format_deserializer.hpp rosbag2/include/rosbag2/serialization_format_converter_interface.hpp renamed to rosbag2/include/rosbag2/converter_interfaces/serialization_format_deserializer.hpp

@@ -12,37 +12,33 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-#ifndef ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
-#define ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
+#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_
+#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_
 
 #include <memory>
-#include <string>
 
 #include "rosbag2/types/introspection_message.hpp"
 #include "rosbag2/types.hpp"
-#include "rcutils/types.h"
-#include "rosbag2_storage/serialized_bag_message.hpp"
-#include "rosidl_typesupport_cpp/message_type_support.hpp"
 
 namespace rosbag2
 {
 
-class SerializationFormatConverterInterface
+namespace converter_interfaces
+{
+
+class SerializationFormatDeserializer
 {
 public:
-  virtual ~SerializationFormatConverterInterface() = default;
+  virtual ~SerializationFormatDeserializer() = default;
 
   virtual void deserialize(
     std::shared_ptr<const rosbag2::SerializedBagMessage> serialized_message,
     const rosidl_message_type_support_t * type_support,
     std::shared_ptr<rosbag2_introspection_message_t> ros_message) = 0;
-
-  virtual void serialize(
-    std::shared_ptr<const rosbag2_introspection_message_t> ros_message,
-    const rosidl_message_type_support_t * type_support,
-    std::shared_ptr<rosbag2::SerializedBagMessage> serialized_message) = 0;
 };
 
+
+}  // namespace converter_interfaces
 }  // namespace rosbag2
 
-#endif  // ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
+#endif  // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_

rosbag2/include/rosbag2/converter_interfaces/serialization_format_serializer.hpp

@@ -0,0 +1,43 @@
+// Copyright 2018, Bosch Software Innovations GmbH.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_
+#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_
+
+#include <memory>
+
+#include "rosbag2/types/introspection_message.hpp"
+#include "rosbag2/types.hpp"
+
+namespace rosbag2
+{
+
+namespace converter_interfaces
+{
+
+class SerializationFormatSerializer
+{
+public:
+  virtual ~SerializationFormatSerializer() = default;
+
+  virtual void serialize(
+    std::shared_ptr<const rosbag2_introspection_message_t> ros_message,
+    const rosidl_message_type_support_t * type_support,
+    std::shared_ptr<rosbag2::SerializedBagMessage> serialized_message) = 0;
+};
+
+}  // namespace converter_interfaces
+}  // namespace rosbag2
+
+#endif  // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_

rosbag2/include/rosbag2/serialization_format_converter_factory.hpp

@@ -19,8 +19,8 @@
 
 #include <memory>
 #include <string>
+#include <vector>
 
-#include "rosbag2/serialization_format_converter_interface.hpp"
 #include "rosbag2/visibility_control.hpp"
 
 // This is necessary because of using stl types here. It is completely safe, because
@@ -31,17 +31,11 @@
 # pragma warning(disable:4251)
 #endif
 
-namespace pluginlib
-{
-
-template<class T>
-class ClassLoader;
-
-}  // namespace pluginlib
-
 namespace rosbag2
 {
 
+class SerializationFormatConverterFactoryImpl;
+
 class ROSBAG2_PUBLIC SerializationFormatConverterFactory
   : public SerializationFormatConverterFactoryInterface
 {
@@ -50,11 +44,14 @@ class ROSBAG2_PUBLIC SerializationFormatConverterFactory
 
   ~SerializationFormatConverterFactory() override;
 
-  std::unique_ptr<SerializationFormatConverterInterface>
-  load_converter(const std::string & format) override;
+  std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
+  load_deserializer(const std::string & format) override;
+
+  std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
+  load_serializer(const std::string & format) override;
 
 private:
-  std::unique_ptr<pluginlib::ClassLoader<SerializationFormatConverterInterface>> class_loader_;
+  std::unique_ptr<SerializationFormatConverterFactoryImpl> impl_;
 };
 
 }  // namespace rosbag2

rosbag2/include/rosbag2/serialization_format_converter_factory_interface.hpp

@@ -18,7 +18,7 @@
 #include <memory>
 #include <string>
 
-#include "rosbag2/serialization_format_converter_interface.hpp"
+#include "rosbag2/converter_interfaces/serialization_format_converter.hpp"
 #include "rosbag2/visibility_control.hpp"
 
 namespace rosbag2
@@ -29,8 +29,11 @@ class ROSBAG2_PUBLIC SerializationFormatConverterFactoryInterface
 public:
   virtual ~SerializationFormatConverterFactoryInterface() = default;
 
-  virtual std::unique_ptr<SerializationFormatConverterInterface>
-  load_converter(const std::string & format) = 0;
+  virtual std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
+  load_deserializer(const std::string & format) = 0;
+
+  virtual std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
+  load_serializer(const std::string & format) = 0;
 };
 
 }  // namespace rosbag2

rosbag2/src/rosbag2/converter.cpp

@@ -39,9 +39,9 @@ Converter::Converter(
   const rosbag2::ConverterOptions & converter_options,
   std::shared_ptr<rosbag2::SerializationFormatConverterFactoryInterface> converter_factory)
 : converter_factory_(converter_factory),
-  input_converter_(converter_factory_->load_converter(
+  input_converter_(converter_factory_->load_deserializer(
       converter_options.input_serialization_format)),
-  output_converter_(converter_factory_->load_converter(
+  output_converter_(converter_factory_->load_serializer(
       converter_options.output_serialization_format))
 {
   if (!input_converter_) {

rosbag2/src/rosbag2/serialization_format_converter_factory.cpp

@@ -16,45 +16,29 @@
 
 #include <memory>
 #include <string>
+#include <vector>
 
-#include "pluginlib/class_loader.hpp"
-#include "rosbag2/logging.hpp"
+#include "./serialization_format_converter_factory_impl.hpp"
 
 namespace rosbag2
 {
 
 SerializationFormatConverterFactory::SerializationFormatConverterFactory()
-{
-  try {
-    class_loader_ = std::make_unique<pluginlib::ClassLoader<SerializationFormatConverterInterface>>(
-      "rosbag2", "rosbag2::SerializationFormatConverterInterface");
-  } catch (const std::exception & e) {
-    ROSBAG2_LOG_ERROR_STREAM("Unable to create class loader instance: " << e.what());
-    throw e;
-  }
-}
+: impl_(std::make_unique<SerializationFormatConverterFactoryImpl>())
+{}
 
 SerializationFormatConverterFactory::~SerializationFormatConverterFactory() = default;
 
-std::unique_ptr<SerializationFormatConverterInterface>
-SerializationFormatConverterFactory::load_converter(const std::string & format)
+std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
+SerializationFormatConverterFactory::load_deserializer(const std::string & format)
+{
+  return impl_->load_deserializer(format);
+}
+
+std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
+SerializationFormatConverterFactory::load_serializer(const std::string & format)
 {
-  auto converter_id = format + "_converter";
-
-  const auto & registered_classes = class_loader_->getDeclaredClasses();
-  auto class_exists = std::find(registered_classes.begin(), registered_classes.end(), converter_id);
-  if (class_exists == registered_classes.end()) {
-    ROSBAG2_LOG_ERROR_STREAM("Requested converter id '" << converter_id << "' does not exist");
-    return nullptr;
-  }
-
-  try {
-    return std::unique_ptr<SerializationFormatConverterInterface>(
-      class_loader_->createUnmanagedInstance(converter_id));
-  } catch (const std::runtime_error & ex) {
-    ROSBAG2_LOG_ERROR_STREAM("Unable to load instance of converter interface: " << ex.what());
-    return nullptr;
-  }
+  return impl_->load_serializer(format);
 }
 
 }  // namespace rosbag2