Skip to content

Conversation

gjchoi
Copy link

@gjchoi gjchoi commented Aug 22, 2022

I'd like to develop Custom CDC Handler for N data insert with 1 sinkData.
But I can't find any base kafka connect feature about this ( kafka connect options, SMT )
And CDCHandler, CDCOperation to make custom CDC Handle below

CdcOperation

WriteModel<BsonDocument> perform(SinkDocument doc);

CdcHandler

  public abstract Optional<WriteModel<BsonDocument>> handle(SinkDocument doc);

only support 1 => 1 data handle.

so I make a PR about this new feature

  • sink connector option added ( change.data.capture.multi.row.handler )
  • CdcMultiRowOperation, CdcMultiRowHanlder added
  • modified sink task logic that return list and append to bulkWrite

Can you review this PR for new feature add?

or

Is there any other way for this I don't know?

- sink connector option added ( change.data.capture.multi.row.handler )
- CdcMultiRowOperation, CdcMultiRowHanlder added
- modified sink task logic that return list<WriteModel> and append to bulkWrite
@skecskes
Copy link

This would be nice to have. Or similarly, I would like to have WriteModelStrategy that can issue bulk list of commands, like spilt the input SinkDocument into multiple WriteModels, not just one. So that it would return List and not just one WriteModel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants