Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-67

Recover from MySQL failure by pointing connector to new MySQL server instance (without GTIDs)

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Won't Do
    • Major
    • None
    • 0.2
    • mysql-connector

    Description

      Per DBZ-31 and DBZ-37, the MySQL connector is already able to fully and automatically recover when the MySQL cluster is using GTIDs. However, when MySQL does not use GTIDs, recovering from a failed MySQL server is a bit more involved. If the failed server can be restarted, then the connector can be restarted and it will recover as expected.

      However, if the connector is pointed to a different MySQL server instance, then the configuration must point to the correct binlog coordinates on that new machine, which are not the same as those recorded in the offsets.

      Consider a MySQL cluster that is not using GTIDs, perhaps for backup reasons or scale out, and perhaps even using deeper replication topologies or multi-source replication where a single follower tracks multiple distinct masters and is useful for merging multiple shards, backing up multiple servers to a single server, or to consolidating data from multiple servers. In all these cases, if GTIDs are not used then the connector should be configured to use a single specific instance in that cluster. If that instance goes down, the connector becomes unavailable; to recover, either bring up the target MySQL instance (in which case the connector automatically recovers) or explicitly change the connector configuration to point to a different machine and specify the new binlog coordinates on that machine.

      This will require getting the last offset recorded by the MySQL connector, and then using MySQL tools to determine where that position is in the binlog on the new machine. The connector could then be configured with both sets of coordinates, and then upon restart it would continue using the new machine. Note that if the connector were stopped and restarted after that, the connector's offsets would have progressed beyond the mapping still in the configuration, but the connector would continue as normal by restarting at the binlog position found in the offsets (rather than forcing the binlog coordinates in the configuration).

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rhauch Randall Hauch (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: