Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-872

Antlr parser ignores table whitelist filter

    XMLWordPrintable

    Details

    • Steps to Reproduce:
      Hide

      create job with whitelisted tables + "database.history.store.only.monitored.tables.ddl": "true",
      Alter another table

      Show
      create job with whitelisted tables + "database.history.store.only.monitored.tables.ddl": "true", Alter another table

      Description

      Job configuration has whitelist of tables which does not include "custom_field_options"
      "table.whitelist": "company_123+
      .(activity|people...)",

      Also only store ddl of monitored tables!
      "database.history.store.only.monitored.tables.ddl": "true",

      Expected: tables not in whitelist are not parsed and applied to schema learning.
      Reality: parsed and error thrown as table doesn't exist in debezium memory.

      org.apache.kafka.connect.errors.ConnectException: Trying to alter table company_123.custom_field_options, which does not exist. Query: ALTER TABLE custom_field_options ADD COLUMN `extra_params` varchar(255) DEFAULT NULL
      at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:200)
      at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:178)
      at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:452)
      at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1055)
      at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:913)
      at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559)
      at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: io.debezium.text.ParsingException: Trying to alter table company_123.custom_field_options, which does not exist. Query: ALTER TABLE custom_field_options ADD COLUMN `extra_params` varchar(255) DEFAULT NULL
      at io.debezium.connector.mysql.antlr.listener.AlterTableParserListener.enterAlterTable(AlterTableParserListener.java:54)
      at io.debezium.ddl.parser.mysql.generated.MySqlParser$AlterTableContext.enterRule(MySqlParser.java:11447)
      at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateEnterRule(ProxyParseTreeListenerUtil.java:46)
      at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.enterEveryRule(MySqlAntlrDdlParserListener.java:89)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:41)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:25)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
      at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
      at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:85)
      at io.debezium.connector.mysql.MySqlSchema.applyDdl(MySqlSchema.java:297)
      at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:637)
      at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:436)
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                pimpelsang Eero Koplimets
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: