Skip to content

Support Batching EventHorizon requests #773

@mhelleborg

Description

@mhelleborg

Description

In order to support higher throughput for EventHorizon, it can be rewritten to support batching.

This requires updating the contract to support it, and leveraging them in the consumer & producer infrastructure in the runtime.

Design requirements

Backwards compatibility

New versions of the runtime should be able to communicate as normal with older ones and vice versa. This can be solved by adding optional fields to the subscription request, which will only be recognized by runtimes running the updated version. Thus it can select the batch mode if the consumer asks for it, and publish the old message types if not.

Resource limits

Since we are making the producer a lot faster with sending batches, the consumer will need to be able to limit work in progress. This can be done by specifying batch sizes & using bounded channels to prevent a slow DB from causing OOM-issues.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions