Skip to content

Commit ab548c2

Browse files
committed
adds MongoDB CDC Kafka connector configs
Device data that have matching client ids will be added to the kafka topics. Those topics are suffixed with -alerts to help indicate that they're filtered lists, not general purpose mirroring of changes. BACK-2504
1 parent 5a59b4e commit ab548c2

File tree

1 file changed

+36
-0
lines changed

1 file changed

+36
-0
lines changed
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
{{ if .Values.global.kafka.connect.enabled }}
2+
apiVersion: kafka.strimzi.io/v1beta2
3+
kind: KafkaConnector
4+
metadata:
5+
labels:
6+
app: device-data-source
7+
strimzi.io/cluster: {{ .Release.Namespace }}-{{ .Values.global.kafka.connect.clusterName }}
8+
name: device-data-source
9+
namespace: {{ .Release.Namespace }}
10+
spec:
11+
class: com.mongodb.kafka.connect.MongoSourceConnector
12+
config:
13+
change.stream.full.document: updateLookup
14+
collection: deviceData
15+
connection.uri: {{ .Values.global.kafka.connect.mongoConnectionUri }}
16+
copy.existing: false
17+
database: data
18+
key.converter: org.apache.kafka.connect.json.JsonConverter
19+
key.converter.schemas.enable: false
20+
pipeline: >-
21+
[
22+
{
23+
$match: {
24+
"fullDocument.type": { $in: [ "cbg", "dosingDecision" ] },
25+
"fullDocument.provenance.clientID": "tidepool-loop",
26+
}
27+
}
28+
]
29+
topic.prefix: {{ .Release.Namespace }}
30+
topic.suffix: "alerts"
31+
value.converter: org.apache.kafka.connect.json.JsonConverter
32+
value.converter.schemas.enable: false
33+
heartbeat.interval.ms: {{ .Values.global.kafka.connect.mongoHeartbeatIntervalMs }}
34+
heartbeat.topic.name: {{ .Release.Namespace }}-{{ .Values.global.kafka.connect.clusterName }}-heartbeat
35+
tasksMax: {{ .Values.global.kafka.connect.sourceTasksMax | int }}
36+
{{- end }}

0 commit comments

Comments
 (0)