Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-217

Connector fails and stops when coming across corrupt event

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Done
    • Affects Version/s: 0.5
    • Fix Version/s: 0.6.2
    • Component/s: mysql-connector
    • Labels:
      None
    • Environment:

      Amazon RDS (MariaDB 10.0.24)
      Debezium 0.5.0
      confluent-platform-oss-2.11

      Description

      After doing a snapshot and running for a while, an exception occurs and the connector stops. Restarting Kafka Connect results in the same message

      [2017-04-07 23:08:33,579] ERROR Task pushdb-connector-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:141)
      org.apache.kafka.connect.errors.ConnectException: Failed to deserialize data of EventHeaderV4{timestamp=1491606429000, eventType=WRITE_ROWS, serverId=1632581618, headerLength=19, dataLength=49, nextPosition=36405, flags=0}
              at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
              at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:131)
              at io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:702)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:768)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:472)
              at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:657)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1491606429000, eventType=WRITE_ROWS, serverId=1632581618, headerLength=19, dataLength=49, nextPosition=36405, flags=0}
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:212)
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:180)
              at io.debezium.connector.mysql.BinlogReader$1.nextEvent(BinlogReader.java:116)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:741)
              ... 3 more
      Caused by: java.io.EOFException
              at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:190)
              at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readLong(ByteArrayInputStream.java:57)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeLongLong(AbstractRowsEventDataDeserializer.java:216)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:152)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:132)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserializeRows(WriteRowsEventDataDeserializer.java:64)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:56)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:32)
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:206)
              ... 6 more
      

      Connector setup:

      {"config":{"connector.class":"io.debezium.connector.mysql.MySqlConnector","tasks.max":"1","database.hostname":"pushv2-mysql.xxx.xxx.rds.amazonaws.com","database.port":"3306","database.user":"debezium","database.password":"dbz","database.server.id":"2000","database.server.name":"pushdb","database.whitelist":"pushv2","database.history.kafka.bootstrap.servers":"localhost:9092","database.history.kafka.topic":"pushdb"},"name":"pushdb-connector"}
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                gunnar.morling Gunnar Morling
                Reporter:
                captain.magnus Magnus Boman
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: