Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7251

Length value is not removed when changing a column's type

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 2.5.0.CR1
    • 2.5.0.Beta1
    • mysql-connector
    • None
    • False
    • None
    • False

    Description

      When changing a column's type, the old default value is still kept in the meta-model, causing an exception when exporting the schema to Connect. For example:

      CREATE TABLE `test` (
       `id` bigint NOT NULL AUTO_INCREMENT,
        `publish_time` bigint(20) null,
        PRIMARY KEY (`id`)
      );
      
      alter table test modify publish_time datetime NULL DEFAULT '1970-01-01 00:00:00';
      

      This will cause the following exception:

      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. 
      at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:374) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$25(MySqlStreamingChangeEventSource.java:875) 
      at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1125) 
      at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:973) 
      at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599) 
      at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:857) 
      at java.base/java.lang.Thread.run(Thread.java:830) 
      Caused by: io.debezium.DebeziumException: Error processing binlog event ... 7 more 
      Caused by: io.debezium.DebeziumException: java.lang.IllegalArgumentException: Maximum width must be from 1 to 9 inclusive but was 20 at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$handleQueryEvent$2(MySqlStreamingChangeEventSource.java:586) 
      at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:302) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleQueryEvent(MySqlStreamingChangeEventSource.java:581) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$14(MySqlStreamingChangeEventSource.java:847) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:354) ... 6 more 
      Caused by: java.lang.IllegalArgumentException: Maximum width must be from 1 to 9 inclusive but was 20 
      at java.base/java.time.format.DateTimeFormatterBuilder$FractionPrinterParser.<init>(DateTimeFormatterBuilder.java:3087) 
      at java.base/java.time.format.DateTimeFormatterBuilder.appendFraction(DateTimeFormatterBuilder.java:707) 
      at io.debezium.connector.mysql.MySqlDefaultValueConverter.timestampFormat(MySqlDefaultValueConverter.java:350) 
      at io.debezium.connector.mysql.MySqlDefaultValueConverter.convertToLocalDateTime(MySqlDefaultValueConverter.java:226) 
      at io.debezium.connector.mysql.MySqlDefaultValueConverter.convert(MySqlDefaultValueConverter.java:143) 
      at io.debezium.connector.mysql.MySqlDefaultValueConverter.parseDefaultValue(MySqlDefaultValueConverter.java:94) 
      at io.debezium.relational.TableSchemaBuilder.lambda$addField$9(TableSchemaBuilder.java:367) 
      at java.base/java.util.Optional.flatMap(Optional.java:287) 
      at io.debezium.relational.TableSchemaBuilder.addField(TableSchemaBuilder.java:367) 
      at io.debezium.relational.TableSchemaBuilder.lambda$create$2(TableSchemaBuilder.java:143) 
      at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) 
      at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) 
      at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621) 
      at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) 
      at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) 
      at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) 
      at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) 
      at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) 
      at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) 
      at io.debezium.relational.TableSchemaBuilder.create(TableSchemaBuilder.java:141) 
      at io.debezium.relational.RelationalDatabaseSchema.buildAndRegisterSchema(RelationalDatabaseSchema.java:122) 
      at io.debezium.connector.mysql.MySqlDatabaseSchema.lambda$applySchemaChange$2(MySqlDatabaseSchema.java:167) 
      at java.base/java.lang.Iterable.forEach(Iterable.java:75) 
      at io.debezium.connector.mysql.MySqlDatabaseSchema.applySchemaChange(MySqlDatabaseSchema.java:167) 
      at io.debezium.pipeline.EventDispatcher$SchemaChangeEventReceiver.schemaChangeEvent(EventDispatcher.java:540) 
      at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$handleQueryEvent$2(MySqlStreamingChangeEventSource.java:583) ... 10 more
      

      Attachments

        Issue Links

          Activity

            People

              ywyuewei Harvey Yue
              ywyuewei Harvey Yue
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: