Loading...

Details

Type: Bug
Resolution: Done
Priority: Blocker
Fix Version/s: 0.8.0.Beta1
Affects Version/s: 0.7.3
Component/s: postgresql-connector
Labels:
None

Steps to Reproduce:

Hide

1. Have a PostgeSQL instance with wal2Json plugin (I used docker debezium/postgres)
2. Have a PostgreSQL connector stood up, my config:
{
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"plugin.name": "wal2json",
"tasks.max": "1",
"database.hostname": "postgres",
"database.port": "5432",
"database.user": "postgres",
"database.password": "***",
"database.dbname": "postgres",
"database.server.name": "elasticsearch",
"table.whitelist": "public.dbo.testmssqltable",
"decimal.handling.mode": "double"
}
3. create matching table:
CREATE TABLE "dbo.testmssqltable" (
"id" int not NULL,
"int" int,
"numeric" numeric,
"time" time,
PRIMARY KEY("id"));
4. Insert example data, with time with precision:
insert into "dbo.testmssqltable" ("id","int","numeric", "time") VALUES (1, 1, 111.234567, '12:12:12.1');
5. Error is thrown and connectors stops to work, and I am actually unable to continue to work with this connector as long as there is such time example in DB, as it will fail each time when restarted.

Show
1. Have a PostgeSQL instance with wal2Json plugin (I used docker debezium/postgres) 2. Have a PostgreSQL connector stood up, my config: { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "plugin.name": "wal2json", "tasks.max": "1", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "***", "database.dbname": "postgres", "database.server.name": "elasticsearch", "table.whitelist": "public.dbo.testmssqltable", "decimal.handling.mode": "double" } 3. create matching table: CREATE TABLE "dbo.testmssqltable" ( "id" int not NULL, "int" int, "numeric" numeric, "time" time, PRIMARY KEY("id")); 4. Insert example data, with time with precision: insert into "dbo.testmssqltable" ("id","int","numeric", "time") VALUES (1, 1, 111.234567, '12:12:12.1'); 5. Error is thrown and connectors stops to work, and I am actually unable to continue to work with this connector as long as there is such time example in DB, as it will fail each time when restarted.

SFDC Cases Counter:
SFDC Cases Links:

Description

When using PostgresSQL source connector on a table with a column of type Time, connector will fail (and stop working) when there is example data present with precision.

Stack trace:

org.apache.kafka.connect.errors.ConnectException: An exception ocurred in the change event producer. This connector will be stopped.
	at io.debezium.connector.base.ChangeEventQueue.throwProducerFailureIfPresent(ChangeEventQueue.java:168)
	at io.debezium.connector.base.ChangeEventQueue.poll(ChangeEventQueue.java:149)
	at io.debezium.connector.postgresql.PostgresConnectorTask.poll(PostgresConnectorTask.java:151)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:179)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.connect.errors.ConnectException: java.time.format.DateTimeParseException: Text '12:12:12.1' could not be parsed, unparsed text found at index 8
	at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.format(DateTimeFormat.java:117)
	at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.time(DateTimeFormat.java:92)
	at io.debezium.connector.postgresql.connection.wal2json.Wal2JsonReplicationMessage.getValue(Wal2JsonReplicationMessage.java:250)
	at io.debezium.connector.postgresql.connection.wal2json.Wal2JsonReplicationMessage$1.getValue(Wal2JsonReplicationMessage.java:135)
	at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$columnValues$2(RecordsStreamProducer.java:430)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at io.debezium.connector.postgresql.RecordsStreamProducer.columnValues(RecordsStreamProducer.java:425)
	at io.debezium.connector.postgresql.RecordsStreamProducer.process(RecordsStreamProducer.java:239)
	at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$streamChanges$1(RecordsStreamProducer.java:119)
	at io.debezium.connector.postgresql.connection.wal2json.Wal2JsonMessageDecoder.processMessage(Wal2JsonMessageDecoder.java:61)
	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:209)
	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.read(PostgresReplicationConnection.java:194)
	at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:119)
	at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$0(RecordsStreamProducer.java:105)
	... 5 more
Caused by: java.time.format.DateTimeParseException: Text '12:12:12.1' could not be parsed, unparsed text found at index 8
	at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:1952)
	at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1851)
	at java.time.LocalTime.parse(LocalTime.java:441)
	at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.lambda$time$2(DateTimeFormat.java:92)
	at io.debez

PostgreSQL connector doesn't handle TIME(p) columns correctly with wal2json

Details

Description

Attachments

Activity

People

Dates