Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-910

Can't parse create definition on the mysql connector

    XMLWordPrintable

    Details

    • Steps to Reproduce:
      Hide

      STEP 1: create table:
      run on the mysql:

      ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (‘Sally’, ‘George’, ‘Edward’, ‘Anne2’ ))
      

      CREATE TABLE foo (c1 INTEGER NOT NULL, 
                       c2 VARCHAR(22), 
                       CONSTRAINT c_name CHECK (c2 IN ('A', 'B', 'C')));
      

      EXPECTED RESULT:
      Debezium parses create statement in the proper way.

      ACTUAL RESULT:
      Example for "ALTER", but for "CREATE" is the same.

      ceTask]
      connect_1    | 2018-09-13 10:11:24,908 ERROR  ||  WorkerSourceTask{id=inventory-connector-0} Task threw an uncaught and unrecoverable exception   [org.apache.kafka.connect.runtime.WorkerTask]
      connect_1    | org.apache.kafka.connect.errors.ConnectException: Failed to parse statement 'ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (???Sally???, ???George???, ???Edward???, ???Anne2??? ))'
      connect_1    | 	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:200)
      connect_1    | 	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:178)
      connect_1    | 	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:452)
      connect_1    | 	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1055)
      connect_1    | 	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:913)
      connect_1    | 	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559)
      connect_1    | 	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793)
      connect_1    | 	at java.lang.Thread.run(Thread.java:748)
      connect_1    | Caused by: io.debezium.text.ParsingException: Failed to parse statement 'ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (???Sally???, ???George???, ???Edward???, ???Anne2??? ))'
      connect_1    | 	at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java:225)
      connect_1    | 	at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java:200)
      connect_1    | 	at io.debezium.connector.mysql.MySqlSchema.applyDdl(MySqlSchema.java:298)
      connect_1    | 	at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:637)
      connect_1    | 	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:436)
      connect_1    | 	... 5 more
      connect_1    | Caused by: io.debezium.text.ParsingException: Expecting token type 128 at line 1, column 59 but found 'CHECK': NSTRAINT first_name  ===>> CHECK (status IN (??
      connect_1    | 	at io.debezium.text.TokenStream.consume(TokenStream.java:750)
      connect_1    | 	at io.debezium.relational.ddl.LegacyDdlParser.consumeStatement(LegacyDdlParser.java:462)
      connect_1    | 	at io.debezium.relational.ddl.LegacyDdlParser.parseUnknownStatement(LegacyDdlParser.java:309)
      connect_1    | 	at io.debezium.connector.mysql.MySqlDdlParser.parseNextStatement(MySqlDdlParser.java:191)
      connect_1    | 	at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java:219)
      connect_1    | 	... 9 more
      

      Show
      STEP 1: create table: run on the mysql: ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (‘Sally’, ‘George’, ‘Edward’, ‘Anne2’ )) CREATE TABLE foo (c1 INTEGER NOT NULL , c2 VARCHAR (22), CONSTRAINT c_name CHECK (c2 IN ( 'A' , 'B' , 'C' ))); EXPECTED RESULT: Debezium parses create statement in the proper way. ACTUAL RESULT: Example for "ALTER", but for "CREATE" is the same. ceTask] connect_1 | 2018 - 09 - 13 10 : 11 : 24 , 908 ERROR || WorkerSourceTask{id=inventory-connector- 0 } Task threw an uncaught and unrecoverable exception [org.apache.kafka.connect.runtime.WorkerTask] connect_1 | org.apache.kafka.connect.errors.ConnectException: Failed to parse statement 'ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (???Sally???, ???George???, ???Edward???, ???Anne2??? ))' connect_1 | at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java: 200 ) connect_1 | at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java: 178 ) connect_1 | at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java: 452 ) connect_1 | at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java: 1055 ) connect_1 | at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java: 913 ) connect_1 | at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java: 559 ) connect_1 | at com.github.shyiko.mysql.binlog.BinaryLogClient$ 7 .run(BinaryLogClient.java: 793 ) connect_1 | at java.lang.Thread.run(Thread.java: 748 ) connect_1 | Caused by: io.debezium.text.ParsingException: Failed to parse statement 'ALTER TABLE inventory.customers ADD CONSTRAINT first_name CHECK (status IN (???Sally???, ???George???, ???Edward???, ???Anne2??? ))' connect_1 | at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java: 225 ) connect_1 | at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java: 200 ) connect_1 | at io.debezium.connector.mysql.MySqlSchema.applyDdl(MySqlSchema.java: 298 ) connect_1 | at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java: 637 ) connect_1 | at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java: 436 ) connect_1 | ... 5 more connect_1 | Caused by: io.debezium.text.ParsingException: Expecting token type 128 at line 1 , column 59 but found 'CHECK' : NSTRAINT first_name ===>> CHECK (status IN (?? connect_1 | at io.debezium.text.TokenStream.consume(TokenStream.java: 750 ) connect_1 | at io.debezium.relational.ddl.LegacyDdlParser.consumeStatement(LegacyDdlParser.java: 462 ) connect_1 | at io.debezium.relational.ddl.LegacyDdlParser.parseUnknownStatement(LegacyDdlParser.java: 309 ) connect_1 | at io.debezium.connector.mysql.MySqlDdlParser.parseNextStatement(MySqlDdlParser.java: 191 ) connect_1 | at io.debezium.relational.ddl.LegacyDdlParser.parse(LegacyDdlParser.java: 219 ) connect_1 | ... 9 more

      Description

      Debezium fails when try to add "CREATE" or "ALTER" expression with "CHECK" and "IN" constraint.

      Here is my "workaround" to fix "create" definition. Probably it can make sense to create a new one field with "CHECK" constraint description on the TableEditor entity.

      MySqlDdlParser#parseCreateDefinitionList

          protected void parseCreateDefinitionList(Marker start, TableEditor table) {
              tokens.consume('(');
              parseCreateDefinition(start, table, false);
              while (tokens.canConsume(',')) {
                  if(tokens.canConsume("CONSTRAINT", TokenStream.ANY_VALUE, "CHECK")) {
                      consumeExpression(start);
                  }else {
                      parseCreateDefinition(start, table, false);
                  }
              }
              tokens.consume(')');
          }
      

      MySqlDdlParserTest#shouldParseMultipleStatements is green after changes above.

          @Test
          public void shouldParseMultipleStatements() {
              String ddl = "CREATE TABLE foo ( " + System.lineSeparator()
                      + " c1 INTEGER NOT NULL, " + System.lineSeparator()
                      + " c2 VARCHAR(22), " + System.lineSeparator()
                      + "CONSTRAINT c_name CHECK (c2 IN ('A', 'B', 'C'))); " + System.lineSeparator()
                      + "-- This is a comment" + System.lineSeparator()
                      + "DROP TABLE foo;" + System.lineSeparator();
              parser.parse(ddl, tables);
              assertThat(tables.size()).isEqualTo(0); // table created and dropped
              listener.assertNext().createTableNamed("foo").ddlStartsWith("CREATE TABLE foo (");
              listener.assertNext().dropTableNamed("foo").ddlMatches("DROP TABLE foo");
          }
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                pletinka Александр П
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: