Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3893

EMPTY_CLOB() and EMPTY_BLOB() should be treated as empty LOB values

    XMLWordPrintable

Details

    Description

      Developers use the EMPTY_CLOB() function to avoid "not null" restriction . Because of this trick, the connector cannot parse DML.
      Error:

      [2021-08-20 13:12:39,065] ERROR Failed to properly convert data value for 'ABCDNEW.ABCD_SHARD_1_3.D_OOS_FTP_PROTOCOL.XML_DATA' of type CLOB for row [276008292,  , EMPTY_CLOB(), 0373200215521000432, 13827284, TO_TIMESTAMP('2021-08-20 13:11:23.315000'), https://ya.ru/, TO_TIMESTAMP('2021-08-20 13:11:23.315000'), TO_TIMESTAMP('2021-08-20 13:11:23.398000'), 0, 0, PP, TO_TIMESTAMP('2021-08-20 13:01:14.'), TO_TIMESTAMP('2021-08-20 13:01:14.'), TO_TIMESTAMP('2021-08-20 13:10:56.595000'), 2, null, 1, ЭА, 0, 1, null, null, null, null, null, null]: (io.debezium.relational.TableSchemaBuilder:269)
      org.apache.kafka.connect.errors.DataException: Invalid value: null used for required field: "XML_DATA", schema type: STRING
              at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:220)
              at org.apache.kafka.connect.data.Struct.put(Struct.java:216)
              at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$5(TableSchemaBuilder.java:265)
              at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:141)
              at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:69)
              at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:45)
              at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:217)
              at io.debezium.connector.oracle.logminer.TransactionalBuffer.commit(TransactionalBuffer.java:296)
              at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:140)
              at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:184)
              at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:63)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:159)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:122)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      

      DDL:

      create table D_OOS_FTP_PROTOCOL
      (
      	ID NUMBER(10) not null
      		constraint PK_OOS_FTP_PROTOCOL
      			primary key,
      	FILE_NAME VARCHAR2(512) not null,
      	XML_DATA CLOB not null,
      	OOS_REG_NUMBER VARCHAR2(64),
      	PROCEDURE_ENTITY_ID NUMBER(10)
      		constraint FK_OSF_PROCEDURE_ENTITY_ID
      			references D_PROCEDURE_ENTITY,
      	ADDED_AT TIMESTAMP(6) not null,
      	PRINT_FORM_URL VARCHAR2(2048),
      	SCHEDULED_AT TIMESTAMP(6) not null,
      	PROCESSED_AT TIMESTAMP(6),
      	TRY_NUMBER NUMBER(10) not null,
      	VERSION NUMBER(10) not null,
      	PROTOCOL_TYPE VARCHAR2(64) not null,
      	PROTOCOL_DATE TIMESTAMP(6),
      	SIGN_DATE TIMESTAMP(6),
      	PUBLISH_DATE TIMESTAMP(6),
      	PROCESS_ORDER NUMBER(10),
      	ABANDONED_REASON_CODE VARCHAR2(50),
      	RECEIVED_FROM_AC2 NUMBER(1) default 0,
      	METHOD_OF_SUPPLIER VARCHAR2(100),
      	FZ223 NUMBER(1),
      	VALID NUMBER(1),
      	NOT_VALID_REASON VARCHAR2(1000),
      	PROTOCOL_NUMBER VARCHAR2(100 char),
      	GUID_HEADER VARCHAR2(100),
      	GUID_ITEM VARCHAR2(100),
      	GUID_PPDATA VARCHAR2(100),
      	PUBLISH_DATE_ETP TIMESTAMP(6)
      )
      

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            chapeauclaque Dmitriy Rum (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: