Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7666

MySQL connector fails to parse DDL with RETURNING keyword

XMLWordPrintable

    • False
    • None
    • False
    • Important

      Hello,

      I'm using Airbyte as a data ETL tool to transfer data from MySQL to BigQuery.

      Airbyte uses Debezium as a backbone for fetching data from MySQL.

      Lately we faced the issue that Debezium is not able to parse specific binary log event.

      SQL that we executed:

      'ALTER TABLE `order` ADD cancelled TINYINT(1) DEFAULT 0 NOT NULL, ADD delivered TINYINT(1) DEFAULT 0 NOT NULL, ADD returning TINYINT(1) DEFAULT 0 NOT NULL'

      What Debezium connector do you use and what version?

      2.4.0

      What is the connector configuration?

      -

      What is the captured database version and mode of depoyment?

      AWS RDS MySQL v. 8.0.36

      What behaviour do you expect?

      Binary log gets imported

      What behaviour do you see?

      Binary log event is causing a fatal error and replication cannot continue

      Do you see the same behaviour using the latest relesead Debezium version?

      I cannot test this as the version is embedded into Airbyte itself

      Do you have the connector logs, ideally from start till finish?

      Error that we receive is as followed:

      2024-03-19 23:46:41 source > 2024-03-19 23:46:41 ERROR i.d.c.m.MySqlStreamingChangeEventSource(logStreamingSourceState):1161 - Error during binlog processing. Last offset stored = {transaction_id=null, ts_sec=1710846937, file=mysql-bin-changelog.117871, pos=2691603, server_id=69937904, event=1}, binlog reader near position = mysql-bin-changelog.117871/2691603
2024-03-19 23:46:41 source > 2024-03-19 23:46:41 ERROR i.d.p.ErrorHandler(setProducerThrowable):52 - Producer failure
2024-03-19 23:46:41 source > io.debezium.DebeziumException: Error processing binlog event
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:395) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$27(MySqlStreamingChangeEventSource.java:930) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1246) [mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-03-19 23:46:41 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1072) [mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-03-19 23:46:41 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) [mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-03-19 23:46:41 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) [mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-03-19 23:46:41 source >         at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
2024-03-19 23:46:41 source > Caused by: io.debezium.text.ParsingException: DDL statement couldn't be parsed. Please open a Jira issue with the statement 'ALTER TABLE `order` ADD cancelled TINYINT(1) DEFAULT 0 NOT NULL, ADD delivered TINYINT(1) DEFAULT 0 NOT NULL, ADD returning TINYINT(1) DEFAULT 0 NOT NULL'
2024-03-19 23:46:41 source > no viable alternative at input 'ALTER TABLE `order` ADD cancelled TINYINT(1) DEFAULT 0 NOT NULL, ADD delivered TINYINT(1) DEFAULT 0 NOT NULL, ADD returning'
2024-03-19 23:46:41 source >         at io.debezium.antlr.ParsingErrorListener.syntaxError(ParsingErrorListener.java:43) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:543) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:136) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at io.debezium.ddl.parser.mysql.generated.MySqlParser.sqlStatements(MySqlParser.java:1272) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.ddl.parser.mysql.generated.MySqlParser.root(MySqlParser.java:988) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.parseTree(MySqlAntlrDdlParser.java:74) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.parseTree(MySqlAntlrDdlParser.java:46) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:78) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlDatabaseSchema.parseDdl(MySqlDatabaseSchema.java:213) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlDatabaseSchema.parseStreamingDdl(MySqlDatabaseSchema.java:199) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleQueryEvent(MySqlStreamingChangeEventSource.java:593) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$15(MySqlStreamingChangeEventSource.java:901) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:375) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         ... 6 more
2024-03-19 23:46:41 source > Caused by: org.antlr.v4.runtime.NoViableAltException
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:2028) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:467) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:393) ~[antlr4-runtime-4.10.1.jar:4.10.1]
2024-03-19 23:46:41 source >         at io.debezium.ddl.parser.mysql.generated.MySqlParser.sqlStatements(MySqlParser.java:1064) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.ddl.parser.mysql.generated.MySqlParser.root(MySqlParser.java:988) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.parseTree(MySqlAntlrDdlParser.java:74) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.parseTree(MySqlAntlrDdlParser.java:46) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:78) ~[debezium-ddl-parser-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlDatabaseSchema.parseDdl(MySqlDatabaseSchema.java:213) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlDatabaseSchema.parseStreamingDdl(MySqlDatabaseSchema.java:199) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleQueryEvent(MySqlStreamingChangeEventSource.java:593) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$15(MySqlStreamingChangeEventSource.java:901) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:375) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-03-19 23:46:41 source >         ... 6 more
2024-03-19 23:46:41 source > 2024-03-19 23:46:41 INFO i.d.c.m.MySqlStreamingChangeEventSource(handleEvent):400 - Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored.
2024-03-19 23:46:41 source > 2024-03-19 23:46:41 INFO i.d.e.EmbeddedEngine(stopTaskAndCommitOffset):1046 - Stopping the task and engine
2024-03-19 23:46:41 source > 2024-03-19 23:46:41 INFO i.d.c.c.BaseSourceTask(stop):284 - Stopping down connector

      How to reproduce the issue using our tutorial deployment?

      Create a table, then later add specific DDL to the database

       

            Unassigned Unassigned
            gediminas.puk Gediminas Puksmys
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: