Apparently I have lost the ability to do basic arithmetic. Thinking is hard and I somehow managed to confuse myself between msg/min and msg/sec. Obviously 55 msg/second gives you ~18ms per message, not a whole second. I do apologise for lowering the IQ of the whole conversation there
Anyway, thanks for the explanation of what your middleware is doing, that really helps to understand the problem. So, if we assume 55 msg/sec and 1.2s processing time per message, you would need at least 55 * 1.2 = 66 threads to keep up. You will want some headroom so call it at least 80 in practice. That should work, but it seems pretty inefficient, and I think we can do better.
The first thing I would consider doing is to cache the external-to-internal ID mapping so that you don’t have to do a lookup on every message. That immediately halves the number of round-trips you need to make to the server.
Then I would look at using SmartREST to deliver your POST requests to Cumulocity. This would let you stream the requests into Cumulocity and avoid having to wait for a complete network round-trip on every request.
Finally, it’s not quite ready yet but within the next couple of weeks we plan to make the new Cumulocity MQTT Service publicly available on our public cloud instances. Assuming your devices use MQTT, this would let them connect directly to a new MQTT endpoint hosted by Cumulocity and continue to publish the same payloads on the same topics that they do now. Your existing logic for mapping those payloads to the Cumulocity API would move into a microservice hosted by Cumulocity. As I said it’s not publicly available yet but you can see the early-access documentation to learn more about it.
Thanks, and apologies again for my bad maths,
Scott