Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1345

PostgreSQL db with materialized view failing during snapshot

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: 0.9.5.Final
    • Fix Version/s: 0.10.0.Beta2
    • Component/s: postgresql-connector
    • Labels:
      None
    • Environment:

      PostgreSQL 9.6 running in AWS RDS using the "wal2json" plugin.
      Connector running in an AWS EC2 instance with the Amazon Linux 2 ami.

      Description

      When starting Debezium, the snapshot process fails if a materialized view is not blacklisted, is included in the table whitelist, or no blacklist or whitelist properties are used.

      [2019-06-20 20:30:55,371] INFO WorkerSourceTask{id=vault-fail-0} Source task finished initialization and start (org.apache.kafka.connect.runtime.WorkerSourceTask:199)
      [2019-06-20 20:30:55,404] INFO Step 0: disabling autocommit (io.debezium.connector.postgresql.RecordsSnapshotProducer:177)
      [2019-06-20 20:30:55,404] INFO Step 1: starting transaction and refreshing the DB schemas for database 'public' and user 'me' (io.debezium.connector.postgresql.RecordsSnapshotProducer:181)
      [2019-06-20 20:30:55,568] INFO Step 2: locking each of the database tables, waiting a maximum of '10.0' seconds for each lock (io.debezium.connector.postgresql.RecordsSnapshotProducer:192)
      [2019-06-20 20:30:55,573] ERROR Cannot rollback snapshot transaction (io.debezium.connector.postgresql.RecordsSnapshotProducer:315)
      org.postgresql.util.PSQLException: This connection has been closed.
          at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:783)
          at org.postgresql.jdbc.PgConnection.rollback(PgConnection.java:791)
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.rollbackTransaction(RecordsSnapshotProducer.java:311)
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.takeSnapshot(RecordsSnapshotProducer.java:286)
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.lambda$start$0(RecordsSnapshotProducer.java:88)
          at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:705)
          at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:687)
          at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      [2019-06-20 20:30:55,573] ERROR unexpected exception (io.debezium.connector.postgresql.RecordsSnapshotProducer:91)
      org.apache.kafka.connect.errors.ConnectException: org.postgresql.util.PSQLException: ERROR: "my_materialized_view" is not a table
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.takeSnapshot(RecordsSnapshotProducer.java:288)
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.lambda$start$0(RecordsSnapshotProducer.java:88)
          at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:705)
          at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:687)
          at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: org.postgresql.util.PSQLException: ERROR: "my_materialized_view" is not a table
          at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
          at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
          at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
          at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
          at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
          at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
          at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
          at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
          at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
          at io.debezium.jdbc.JdbcConnection.executeWithoutCommitting(JdbcConnection.java:1121)
          at io.debezium.connector.postgresql.RecordsSnapshotProducer.takeSnapshot(RecordsSnapshotProducer.java:202)
          ... 7 more
      [2019-06-20 20:30:55,870] INFO WorkerSourceTask{id=vault-fail-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:397)
      [2019-06-20 20:30:55,870] INFO WorkerSourceTask{id=vault-fail-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:414)
      [2019-06-20 20:30:55,870] ERROR WorkerS
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                ccranfor Chris Cranford
                Reporter:
                grant.cooksey Grant Cooksey
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: