Details
-
Bug
-
Resolution: Done
-
Major
-
1.3.1.Final
-
None
-
False
-
False
-
Undefined
Description
UPDATE: This apparently only happens if I have snapshot.statement.overrides with SELECT , which includes columns not in my column.include.list. Perhaps this then doesn't count as a bug? Perhaps we should have some validation that the select override matches the schema or otherwise ignore columns from the output of the select which are not configured? In any case, *OID data type issue is still a problem.
There was a ticket that appeared to fix the oid data type issue but it apparently has not: DBZ-1850.
If you want to my reason for using this data type, it is that we summarize postgres catalog data (some of which uses this type) and we want to send that data elsewhere using debezium...
But furthermore, I have excluded this column for this reason using column.include.list, but I am still encountering error below.
Here is the stack trace - the column in question is of oid data type, and is excluded by virtue of column.include.list:
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. \tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) \tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:150) \tat io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140) \tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113) \tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) \tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) \tat java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {transaction_id=null, lsn_proc=84621589413328, lsn_commit=84621594943712, lsn=84621589413328, txId=614217241, ts_usec=1612365551216999} \tat io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:220) \tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.lambda$processMessages$0(PostgresStreamingChangeEventSource.java:228) \tat io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.decodeInsert(PgOutputMessageDecoder.java:380) \tat io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.processNotEmptyMessage(PgOutputMessageDecoder.java:191) \tat io.debezium.connector.postgresql.connection.AbstractMessageDecoder.processMessage(AbstractMessageDecoder.java:42) \tat io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:480) \tat io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:472) \tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processMessages(PostgresStreamingChangeEventSource.java:185) \tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:147) \t... 7 more Caused by: java.lang.NumberFormatException: For input string: \"3059975258\" \tat java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) \tat java.base/java.lang.Integer.parseInt(Integer.java:652) \tat java.base/java.lang.Integer.valueOf(Integer.java:983) \tat io.debezium.connector.postgresql.connection.pgoutput.PgOutputColumnValue.asInteger(PgOutputColumnValue.java:47) \tat io.debezium.connector.postgresql.connection.ReplicationMessageColumnValueResolver.resolveValue(ReplicationMessageColumnValueResolver.java:75) \tat io.debezium.connector.postgresql.connection.pgoutput.PgOutputReplicationMessage.getValue(PgOutputReplicationMessage.java:96) \tat io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder$1.getValue(PgOutputMessageDecoder.java:579) \tat io.debezium.connector.postgresql.PostgresChangeRecordEmitter.columnValues(PostgresChangeRecordEmitter.java:170) \tat io.debezium.connector.postgresql.PostgresChangeRecordEmitter.getNewColumnValues(PostgresChangeRecordEmitter.java:113) \tat io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:64) \tat io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:45) \tat io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:88) \tat io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:193) \t... 15 more
