Skip to content

Commit 2aed48c

Browse files
shivamkm07msfussellhhunter-ms
authored
Azure eventhubs bulk subscribe support (#3756)
* azure eventhubs bulk subscribe support Signed-off-by: Shivam Kumar <shivamkm07@gmail.com> * removing localized link Signed-off-by: Shivam Kumar <shivamkm07@gmail.com> * Apply suggestions from code review Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Shivam Kumar <shivamkm07@gmail.com> * Adding example Signed-off-by: Shivam Kumar <shivamkm07@gmail.com> --------- Signed-off-by: Shivam Kumar <shivamkm07@gmail.com> Co-authored-by: Mark Fussell <markfussell@gmail.com> Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com>
1 parent 42f857e commit 2aed48c

File tree

1 file changed

+48
-2
lines changed

1 file changed

+48
-2
lines changed

daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-eventhubs.md

Lines changed: 48 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,14 +117,60 @@ spec:
117117
value: "myeventhubstoragecontainer"
118118
```
119119

120-
## Sending multiple messages
120+
## Sending and receiving multiple messages
121121

122-
Azure Event Hubs supports sending multiple messages in a single operation. To set the metadata for bulk operations, set the query parameters on the HTTP request or the gRPC metadata as documented [here]({{< ref pubsub_api >}})
122+
Azure Eventhubs supports sending and receiving multiple messages in a single operation using the bulk pub/sub API.
123+
124+
### Configuring bulk publish
125+
126+
To set the metadata for bulk publish operation, set the query parameters on the HTTP request or the gRPC metadata, [as documented in the API reference]({{< ref pubsub_api >}}).
123127

124128
| Metadata | Default |
125129
|----------|---------|
126130
| `metadata.maxBulkPubBytes` | `1000000` |
127131

132+
### Configuring bulk subscribe
133+
134+
When subscribing to a topic, you can configure `bulkSubscribe` options. Refer to [Subscribing messages in bulk]({{< ref "pubsub-bulk#subscribing-messages-in-bulk" >}}) for more details and to learn more about [the bulk subscribe API]({{< ref pubsub-bulk.md >}}).
135+
136+
| Configuration | Default |
137+
|---------------|---------|
138+
| `maxMessagesCount` | `100` |
139+
| `maxAwaitDurationMs` | `10000` |
140+
141+
## Configuring checkpoint frequency
142+
143+
When subscribing to a topic, you can configure the checkpointing frequency in a partition by [setting the metadata in the HTTP or gRPC subscribe request ]({{< ref "pubsub_api.md#http-request-2" >}}). This metadata enables checkpointing after the configured number of events within a partition event sequence. Disable checkpointing by setting the frequency to `0`.
144+
145+
[Learn more about checkpointing](https://learn.microsoft.com/azure/event-hubs/event-hubs-features#checkpointing).
146+
147+
| Metadata | Default |
148+
| -------- | ------- |
149+
| `metadata.checkPointFrequencyPerPartition` | `1` |
150+
151+
Following example shows a sample subscription file for [Declarative subscription]({{< ref "subscription-methods.md#declarative-subscriptions" >}}) using `checkPointFrequencyPerPartition` metadata. Similarly, you can also pass the metadata in [Programmatic subscriptions]({{< ref "subscription-methods.md#programmatic-subscriptions" >}}) as well.
152+
153+
```yaml
154+
apiVersion: dapr.io/v2alpha1
155+
kind: Subscription
156+
metadata:
157+
name: order-pub-sub
158+
spec:
159+
topic: orders
160+
routes:
161+
default: /checkout
162+
pubsubname: order-pub-sub
163+
metadata:
164+
checkPointFrequencyPerPartition: 1
165+
scopes:
166+
- orderprocessing
167+
- checkout
168+
```
169+
170+
{{% alert title="Note" color="primary" %}}
171+
When subscribing to a topic using `BulkSubscribe`, you configure the checkpointing to occur after the specified number of _batches,_ instead of events, where _batch_ means the collection of events received in a single request.
172+
{{% /alert %}}
173+
128174
## Create an Azure Event Hub
129175

130176
Follow the instructions on the [documentation](https://docs.microsoft.com/azure/event-hubs/event-hubs-create) to set up Azure Event Hubs.

0 commit comments

Comments
 (0)