add message handler #6
Description
The message handler is the "splitter".
The consumer consumes all messages from a "raw" kafka broker which contains all wrp messages from talaria.
At the very least splitter must be able to route message based on
- event type in the destination
- device id
The splitter should optionally be able to filter from multiple criteria for a given output stream: For example, both event type and device id. (caduceus currently has this functionality)
Ideally, the criteria should be generic and match any part of the wrp message for flexibility (but is not a hard and fast requirement - caduceus currently matches on event type and/or device id)
A wrp message MAY match multiple output streams.
The splitter must contain configurable buffering, retries and QOS based on the wrp message.
the wrpkafka library already handles a lot of this. What it's currently missing:
- a message is only sent to the first matching stream, therefore it can only be written to one stream
- it does not currently look like handles filtering on device id (double check this)