Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1264

CREATE INDEX can fail for non-monitored tables after connector restart

    Details

      Description

      Follow these steps to reproduce the issue

      • Register a connector with store.only.monitored.tables set to true
      • Create a new table
      • Add an index to the table
      • restart the connector

      The CREATE INDEX is stored in the database history but not CREATE TABLE which leads to

      2019-05-03 08:08:03,797 WARN   MySQL|my-table|task  Ignoring unparseable statements '/* ApplicationName=PhpStorm 2019.1.1 */ create unique index migration_test_mgb_no_uindex
          on migration_test (mgb_no)' stored in database history: {}   [io.debezium.relational.history.KafkaDatabaseHistory]
      io.debezium.text.ParsingException: Trying to create index on non existing table mydb.migration_test.Query: create unique index migration_test_mgb_no_uindex
          on migration_test (mgb_no)
          at io.debezium.connector.mysql.antlr.listener.CreateUniqueIndexParserListener.enterCreateIndex(CreateUniqueIndexParserListener.java:45)
          at io.debezium.ddl.parser.mysql.generated.MySqlParser$CreateIndexContext.enterRule(MySqlParser.java:2903)
          at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateEnterRule(ProxyParseTreeListenerUtil.java:46)
          at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.enterEveryRule(MySqlAntlrDdlParserListener.java:89)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:41)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:25)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
          at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
          at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:85)
          at io.debezium.relational.history.AbstractDatabaseHistory.lambda$recover$1(AbstractDatabaseHistory.java:104)
          at io.debezium.relational.history.KafkaDatabaseHistory.recoverRecords(KafkaDatabaseHistory.java:238)
          at io.debezium.relational.history.AbstractDatabaseHistory.recover(AbstractDatabaseHistory.java:73)
          at io.debezium.connector.mysql.MySqlSchema.loadHistory(MySqlSchema.java:251)
          at io.debezium.connector.mysql.MySqlTaskContext.loadHistory(MySqlTaskContext.java:165)
          at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:105)
          at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:47)
          at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:198)
          at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
          at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      

      The exception should be thrown only if all DDLs are stored, otherwise a WARN should be logged.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                jpechanec Jiri Pechanec
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: