Dynamic Mapper 4.9.0 (Pre-Release 5.0.0)

[Stefan_Witschel]

Overview

The dynamic mapper is an open-source tool to dynamically map any payload from any source to Cumulocity and vice-versa.

Dynamic Mapper - Map (MQTT) device data in a zero-code approach!

Introduction Since years Cumulocity IoT is providing a MQTT endpoint dedicated to the Cumulocity Domain Model to connect MQTT devices. But what about devices using a pre-defined topic structure and any custom payload which cannot be modified because they are so called “closed”? Since the latest release of Cumulocity there is a new Service in private-preview - the MQTT Service - which is a full-fledged MQTT Broker without the limitation of pre-defined topic structure and payload. Still, it does…

We are continuously improving the tool by adding new functionality. For example in our last release focussed on HTTP Inbound processing and webHooks.

WebHooks and Generic HTTP Processing powered by Dynamic Mapper

Overview Cumulocity provides a REST API that expects data in a specific format, our so-called Domain Model: [image] As usual, not all devices support this domain model out of the box, so some mapping is needed to map the device data to that domain model and vice-versa. For that purpose, we originally implemented the Dynamic Mapper, an open-source project that allows such mappings in a zero-code approach. It was mainly developed for messaging protocols like MQTT and Kafka, which follow a ty…

Today we released 4.9.0 as a Pre-Release for 5.0.0 which also contains amazing new features:

• Code-based mapping
• Advanced filtering

As Pre-release because we wanted to get early feedback before we further develop these new features. There might be still bugs and limitations but to get a first impressions they work very well!

What’s New in 4.9.0

Code-Based Mappings

In past releases, we mainly relied on JSONata as an expression language to define complex expressions and map data. JSONata is a commonly used tool in multiple other stacks, which was the reason we decided to use it in the dynamic mapper.

Recently, we received feedback that working with complex payloads and JSONata is very complicated, especially if you are not familiar with it and have to research all available functions to, for example, iterate over an array and select specific data.

Therefore, we implemented another option to define mappings: Code-Based Mappings.

Here, you can use plain JavaScript to do calculations, conversions, and, of course, mapping. As an input, you get the source message as a JsonObject. As an output, it is an array of Substitutions expected. Each Substitution defines:

• key - The target path in the target template.
• value - The value of the path.
• type - The type of the value, e.g., OBJECT, TEXTUAL, ARRAY, IGNORE, NUMBER.
• repairStrategy - What should happen if an issue is detected with this substitution, e.g., you mapped a single value but an array is provided? DEFAULT, USE_FIRST_VALUE_OF_ARRAY, USE_LAST_VALUE_OF_ARRAY, IGNORE, REMOVE_IF_MISSING_OR_NULL, CREATE_IF_MISSING.

If you want to re-use same functions in mappings we introduced shared code where you can define functions and of course use them in your code-based mappings.

Code-based mappings are working for inbound and outbound messages, limited to JSON payloads only currently.

Additional limitations exist regarding array processing, e.g., sending multiple measurements from one single message with multiple values and code validation in the web IDE but this might be addressed in future releases.

We are looking forward to getting your feedback about this feature!

Advanced Filtering

We re-implemented the way you can filter messages. Within the mapping dialog, we introduced a new input field that lets you define filter conditions for when messages should be processed and when they should be ignored.

The result of the optional filter mapping expression must always be true.

In the example above, we only process messages with a temperature value above 20. Here again, you can use the full power of JSONata to define complex regex or expressions to filter messages or keep it simple and just check if a property exists → $exists(Temperature).

For inbound mappings, this filter is optional. For outbound mappings, it is required as we use it as a key for that mapping.

We are looking forward to your feedback!

Other Improvements

Besides that, we fixed some minor issues, such as inventory updates not being processed correctly in outbound mappings, and other small fixes.

Try it!

You can download the latest Pre-Release 4.9.0 here:

Release Pre-Release v4.9.0 (v5.0.0 pre) ·...

Please Note Be aware that this release might contain breaking changes and it could be necessary to update your service configuration and migrate existing mappings.

Let us know if you need support or running into issues.