What happened?
sources have a checkpoint_key which was supposed to be used in interactions with checkpoint-store per source but while commit_fn is using the checkpoint_key in many other paths source_id is used instead.
checkpoint trait also refers to that param as source_id.
What was expected?
through reconnects the source (mysql for example) switches between the id and the key leading to wrong resume point.
Steps to reproduce
run the binlog_purge or other chaos tests
Environment
- DeltaForge version:
- Source/Sink/Processor:
- OS:
Logs/Error/Details
What happened?
sources have a checkpoint_key which was supposed to be used in interactions with checkpoint-store per source but while commit_fn is using the checkpoint_key in many other paths source_id is used instead.
checkpoint trait also refers to that param as source_id.
What was expected?
through reconnects the source (mysql for example) switches between the id and the key leading to wrong resume point.
Steps to reproduce
run the binlog_purge or other chaos tests
Environment
Logs/Error/Details