Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-459

Postgresql connector doesn't work with varchar[] field

    XMLWordPrintable

Details

    • Hide
      • Create table without any varchar[] field --> everything seems good, data was pushed to kafka
      • Create table with varchar[] field --> got above error
      • Create table with varchar[] NOT NULL field --> got above error
      Show
      Create table without any varchar[] field --> everything seems good, data was pushed to kafka Create table with varchar[] field --> got above error Create table with varchar[] NOT NULL field --> got above error

    Description

      I use Postgresql-connect to sync data to kafka, but it got error with the varchar[] type fields, this is error:

      ERROR unexpected exception while streaming logical changes (io.debezium.connector.postgresql.RecordsStreamProducer:125)
java.lang.NullPointerException
        at org.apache.kafka.connect.data.Struct.put(Struct.java:214)
        at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$3(TableSchemaBuilder.java:231)
        at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
        at io.debezium.connector.postgresql.RecordsStreamProducer.generateCreateRecord(RecordsStreamProducer.java:262)
        at io.debezium.connector.postgresql.RecordsStreamProducer.process(RecordsStreamProducer.java:234)
        at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:113)
        at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$1(RecordsStreamProducer.java:98)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
[2017-11-14 15:54:24,122] ERROR Task source-postgres-02-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:148)
org.apache.kafka.connect.errors.ConnectException: java.lang.NullPointerException
        at io.debezium.connector.postgresql.PostgresConnectorTask.poll(PostgresConnectorTask.java:165)
        at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:163)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
        at org.apache.kafka.connect.data.Struct.put(Struct.java:214)
        at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$3(TableSchemaBuilder.java:231)
        at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
        at io.debezium.connector.postgresql.RecordsStreamProducer.generateCreateRecord(RecordsStreamProducer.java:262)
        at io.debezium.connector.postgresql.RecordsStreamProducer.process(RecordsStreamProducer.java:234)
        at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:113)
        at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$1(RecordsStreamProducer.java:98)
        ... 5 more
[2017-11-14 15:54:24,122] ERROR Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:149)

      Attachments

        Issue Links

          is duplicated by

          Feature Request - Feature requests from customers and/or users DBZ-506 Add support for Postgres VARCHAR array columns

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              Unassigned Unassigned
              jindov Jin Nguyen (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: