Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-31

Support initial MySQL snapshots

    XMLWordPrintable

Details

    Description

      The MySQL connector in Debezium 0.1 requires MySQL to have the complete history of all tables to be monitored in its binlog. This means the connector could not be used if MySQL purges old binlogs (which it normally does), or if its row-level binlog was enabled after the tables had been created.

      A newly-deployed MySQL connector should be able to obtain a consistent snapshot using the current state of the database, and then when that is complete proceed to read the binlog as in 0.1. Although obtaining a consistent snapshot of the schema and data requires a global read lock (preventing writes by other DB clients), the connector should hold this lock for as short a duration as possible.

      (This issue originally included a request to support ad hoc snapshots after the connector has been running, but this is more challenging with Kafka Connect and was pulled out into DBZ-66. See the discussion below for details.)

      Attachments

        Issue Links

          blocks

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-66 Support ad hoc snapshots on MySQL connector

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is blocked by

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-52 Add top-level "standard" layer to events for metadata, before/after, and other info

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          relates to

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-67 Recover from MySQL failure by pointing connector to new MySQL server instance (without GTIDs)

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              rhauch Randall Hauch (Inactive)
              criccomini Chris Riccomini (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: