Details
-
Enhancement
-
Resolution: Done
-
Major
-
0.7.2
-
None
Description
When launching a new connector while an existing connector is performing a snapshot, the first connector's snapshot fails. This is because the second connector launch triggers a rebalance in kafka connect. It is a known issue that a rebalance cannot take place during the initial snapshot, however we guarantee this will happen when launching multiple connectors at or around the same time.
One solution to this would be an initial snapshot delay: after launching a connector, delay for N minutes before actually beginning the snapshot. This would allow multiple connectors to be launched at the same time and the triggered rebalancing to be completed before the N minutes.
This could be a configurable delay period so those users with only 1 connector can opt to not have the snapshot delay at all.