Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-513

AWS RDS Postgresql 9.6.5 not supporting "include-not-null" = "true" in connector setup

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.7.1
    • Component/s: postgresql-connector
    • Labels:
      None
    • Environment:

      Running AWS RDS with 9.6.5 version
      Docker images with tag 0.7.0

    • Steps to Reproduce:
      Hide

      On RDS enabled logical_replication to 1 and reboot happened.
      After zookeeper, kafka and connect docker started from 0.7.0 tagged images connector configuration pushed with below config:

      curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{
      "name": "postgresql-connector",
      "config":

      { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "database.hostname": "<DB-IP-ADDRESS>", "database.port": "5432", "database.user": "<username>", "database.password": "<password>", "database.dbname": "testdb", "database.sslmode": "require", "plugin.name": "wal2json", "database.server.name": "dbserver1", "table.whitelist": "public.table1,public.table2", "column.blacklist": "public.table1.col1,public.table2.col1" }

      }'

      Getting below error on connect docker console.

      Show
      On RDS enabled logical_replication to 1 and reboot happened. After zookeeper, kafka and connect docker started from 0.7.0 tagged images connector configuration pushed with below config: curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "postgresql-connector", "config": { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "database.hostname": "<DB-IP-ADDRESS>", "database.port": "5432", "database.user": "<username>", "database.password": "<password>", "database.dbname": "testdb", "database.sslmode": "require", "plugin.name": "wal2json", "database.server.name": "dbserver1", "table.whitelist": "public.table1,public.table2", "column.blacklist": "public.table1.col1,public.table2.col1" } }' Getting below error on connect docker console.

      Description

      2017-12-17 09:23:37,084 ERROR Postgres|dbserver1|records-stream-producer unexpected exception while streaming logical changes [io.debezium.connector.postgresql.RecordsStreamProducer]
      org.postgresql.util.PSQLException: ERROR: option "include-not-null" = "true" is unknown
      Where: slot "debezium", output plugin "wal2json", in the startup callback
      at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412)
      at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1043)
      at org.postgresql.core.v3.QueryExecutorImpl.readFromCopy(QueryExecutorImpl.java:962)
      at org.postgresql.core.v3.CopyDualImpl.readFromCopy(CopyDualImpl.java:41)
      at org.postgresql.core.v3.replication.V3PGReplicationStream.receiveNextData(V3PGReplicationStream.java:145)
      at org.postgresql.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:114)
      at org.postgresql.core.v3.replication.V3PGReplicationStream.read(V3PGReplicationStream.java:60)
      at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.read(PostgresReplicationConnection.java:179)
      at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:119)
      at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$1(RecordsStreamProducer.java:105)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      2017-12-17 09:23:37,484 INFO || WorkerSourceTask

      {id=postgresql-connector-0} Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]
      2017-12-17 09:23:37,486 INFO || WorkerSourceTask{id=postgresql-connector-0}

      flushing 0 outstanding messages for offset commit [org.apache.kafka.connect.runtime.WorkerSourceTask]

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  surajsavita Suraj Savita
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: