Add message utilities module

[jacobperron]

Contains functions for setting the values of a ROS message and converting ROS messages to other formats (e.g. YAML or CSV).

These functions have been copied from the package ros2topic so they can be used by other packages.

Specifically, these functions can be reused by ros2service and ros2action CLI tools. I'm proposing moving them to rclpy since they might be used in other tools (e.g. rosbag).

---

Here's the diff of the changes I made to the functions while moving them (excluding the docstrings):

diff --git a/ros2topic/ros2topic/api/__init__.py b/ros2topic/ros2topic/api/__init__.py
index 0156c43..548c62e 100644
--- a/ros2topic/ros2topic/api/__init__.py
+++ b/ros2topic/ros2topic/api/__init__.py
@@ -98,7 +98,7 @@ class SetFieldError(Exception):
         self.exception = exception
 
 
-def set_msg_fields(msg, values):
+def set_msg_fields(msg: Any, values: Dict[str, str]) -> None:
     for field_name, field_value in values.items():
         field_type = type(getattr(msg, field_name))
         try:

diff --git a/ros2topic/ros2topic/verb/echo.py b/ros2topic/ros2topic/verb/echo.py
index 95db572..afe71db 100644
--- a/ros2topic/ros2topic/verb/echo.py
+++ b/ros2topic/ros2topic/verb/echo.py
@@ -90,7 +90,7 @@ def register_yaml_representer():
 # an OrderedDict.
 # Inspired by:
 # http://stackoverflow.com/a/16782282/7169408
-def represent_ordereddict(dumper, data):
+def __represent_ordereddict(dumper, data):
     items = []
     for k, v in data.items():
         items.append((dumper.represent_data(k), dumper.represent_data(v)))
@@ -137,12 +137,15 @@ def subscriber_cb(args):
     return cb
 
 
-def msg_to_yaml(args, msg):
-    return yaml.dump(
-        msg_to_ordereddict(
-            msg,
-            truncate_length=args.truncate_length if not args.full_length else None
-        ), width=sys.maxsize)
+def msg_to_yaml(msg: Any, truncate_length: int = None) -> str:
+    global __yaml_representer_registered
+
+    # Register our custom representer for YAML output
+    if not __yaml_representer_registered:
+        yaml.add_representer(OrderedDict, __represent_ordereddict)
+        __yaml_representer_registered = True
+
+    return yaml.dump(msg_to_ordereddict(msg, truncate_length=truncate_length), width=sys.maxsize)
 
 
 def subscriber_cb_csv(args):
@@ -152,29 +155,29 @@ def subscriber_cb_csv(args):
     return cb
 
 
-def msg_to_csv(args, msg):
+def msg_to_csv(msg: Any, truncate_length: int = None) -> str:
     def to_string(val):
-        nonlocal args
+        nonlocal truncate_length
         r = ''
         if any(isinstance(val, t) for t in [list, tuple]):
             for i, v in enumerate(val):
                 if r:
                     r += ','
-                if not args.full_length and i >= args.truncate_length:
+                if truncate_legnth is not None and i >= truncate_length:
                     r += '...'
                     break
                 r += to_string(v)
         elif any(isinstance(val, t) for t in [bool, bytes, float, int, str]):
             if any(isinstance(val, t) for t in [bytes, str]):
-                if not args.full_length and len(val) > args.truncate_length:
-                    val = val[:args.truncate_length]
+                if truncate_length is not None and len(val) > truncate_length:
+                    val = val[:truncate_length]
                     if isinstance(val, bytes):
                         val += b'...'
                     else:
                         val += '...'
             r = str(val)
         else:
-            r = msg_to_csv(args, val)
+            r = msg_to_csv(val, truncate_length)
         return r
     result = ''
     # We rely on __slots__ retaining the order of the fields in the .msg file.
@@ -189,18 +192,18 @@ def msg_to_csv(args, msg):
 # Convert a msg to an OrderedDict. We do this instead of implementing a generic
 # __dict__() method in the msg because we want to preserve order of fields from
 # the .msg file(s).
-def msg_to_ordereddict(msg, truncate_length=None):
+def msg_to_ordereddict(msg: Any, truncate_length: int = None) -> OrderedDict:
     d = OrderedDict()
     # We rely on __slots__ retaining the order of the fields in the .msg file.
     for field_name in msg.__slots__:
         value = getattr(msg, field_name, None)
-        value = _convert_value(value, truncate_length=truncate_length)
-        # remove leading underscore from field name
+        value = __convert_value(value, truncate_length=truncate_length)
+        # Remove leading underscore from field name
         d[field_name[1:]] = value
     return d
 
 
-def _convert_value(value, truncate_length=None):
+def __convert_value(value, truncate_length=None):
     if isinstance(value, bytes):
         if truncate_length is not None and len(value) > truncate_length:
             value = ''.join([chr(c) for c in value[:truncate_length]]) + '...'
@@ -214,19 +217,18 @@ def _convert_value(value, truncate_length=None):
             # Truncate the sequence
             value = value[:truncate_length]
             # Truncate every item in the sequence
-            value = type(value)([_convert_value(v, truncate_length) for v in value] + ['...'])
+            value = type(value)([__convert_value(v, truncate_length) for v in value] + ['...'])
         else:
             # Truncate every item in the list
-            value = type(value)([_convert_value(v, truncate_length) for v in value])
+            value = type(value)([__convert_value(v, truncate_length) for v in value])
     elif isinstance(value, dict) or isinstance(value, OrderedDict):
-        # convert each key and value in the mapping
+        # Convert each key and value in the mapping
         new_value = {} if isinstance(value, dict) else OrderedDict()
         for k, v in value.items():
-            # don't truncate keys because that could result in key collisions and data loss
-            new_value[_convert_value(k)] = _convert_value(v, truncate_length=truncate_length)
+            # Don't truncate keys because that could result in key collisions and data loss
+            new_value[__convert_value(k)] = __convert_value(v, truncate_length=truncate_length)
         value = new_value
     elif not any(isinstance(value, t) for t in (bool, float, int)):
-        # assuming value is a message
-        # since it is neither a collection nor a primitive type
+        # Assuming value is a message since it is neither a collection nor a primitive type
         value = msg_to_ordereddict(value, truncate_length=truncate_length)
     return value

[dirk-thomas]

Since neither of these functions is related to the client library I would suggest putting it in a package related to rosidl_generator_py (as described in ros2/ros2#442 for C++).

[jacobperron]

Since neither of these functions is related to the client library I would suggest putting it in a package related to rosidl_generator_py (as described in ros2/ros2#442 for C++).

Sure. I could add a new package, rosidl_pyutils to the repo rosidl_python, or some other name if you have a better suggestion.

[dirk-thomas]

We do have message_generation / message_runtime packages in ROS 1. So maybe each rosidl_generator_X package could have a sibling rosidl_runtime_X. Or as suggested in the referenced ticket rosidl_X_runtime. (All assuming that we think this is important enough for message packages to not depend on the generators.)

[jacobperron]

Do you imagine this to be the package that also contains functions like these:

https://github.com/ros2/ros2cli/blob/9ea0672aa6cc1700b2aa4833eee157355cac4291/ros2msg/ros2msg/api/__init__.py#L39-L44

https://github.com/ros2/ros2cli/blob/9ea0672aa6cc1700b2aa4833eee157355cac4291/ros2msg/ros2msg/api/__init__.py#L52-L54

[dirk-thomas]

These functions don't have anything to do with the generated message classes. So I would rather say no.

[jacobperron]

@dirk-thomas See ros2/rosidl_python#34

[jacobperron]

Replaced by ros2/rosidl_python#34