Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-396

MySQL connector "initial_only" snapshot mode results in CPU spike from ConnectorTask polling

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 0.6
    • Fix Version/s: 0.6.2
    • Component/s: mysql-connector
    • Labels:
      None
    • Environment:

      Confluent 3.3.0

      Description

      I have been testing out the "initial_only" snapshot mode for the MySQL Connector. This mode is supposed to complete the initial snapshot and then stop. It completed the snapshot successfully, and the binglog reader was disabled as I would expect.

      However, the CPU usage spiked indefinitely, even though it should be doing nothing once complete.
      The only resolution was updating the connector configuration to the default "initial", which enables the binlog reader. The the CPU returned to normal. (Screenshot attached from JConsole)

      I briefly looked at the code for the MySQLConnectorTask, and I believe the problem is with the poll() implementation. Kafka Connect is in an infinite loop poll()-ing for records, but since the binlog reader is not running we are just immediately returning 'null', with no delay or pause. This leads to the unnecessary CPU spike.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                pgoransson Peter Goransson
                Reporter:
                pgoransson Peter Goransson
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: