Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-396

MySQL connector "initial_only" snapshot mode results in CPU spike from ConnectorTask polling

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 0.6.2
    • 0.6
    • mysql-connector
    • None

    Description

      I have been testing out the "initial_only" snapshot mode for the MySQL Connector. This mode is supposed to complete the initial snapshot and then stop. It completed the snapshot successfully, and the binglog reader was disabled as I would expect.

      However, the CPU usage spiked indefinitely, even though it should be doing nothing once complete.
      The only resolution was updating the connector configuration to the default "initial", which enables the binlog reader. The the CPU returned to normal. (Screenshot attached from JConsole)

      I briefly looked at the code for the MySQLConnectorTask, and I believe the problem is with the poll() implementation. Kafka Connect is in an infinite loop poll()-ing for records, but since the binlog reader is not running we are just immediately returning 'null', with no delay or pause. This leads to the unnecessary CPU spike.

      Attachments

        Activity

          People

            pgoransson@cainc.com Peter Goransson (Inactive)
            pgoransson@cainc.com Peter Goransson (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: