Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-279

Warnings and notifications from PostgreSQL are ignored by the connector

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 0.5.1
    • Fix Version/s: 0.6
    • Component/s: postgresql-connector
    • Labels:
      None
    • Environment:

      Ubuntu 16.04, x86-64

    • Steps to Reproduce:
      Hide

      Hack the decoderbufs plugin to produce lots of warnings, and observe that they aren't output by the connector and that memory usage in the connector only goes up.

      Show
      Hack the decoderbufs plugin to produce lots of warnings, and observe that they aren't output by the connector and that memory usage in the connector only goes up.

      Description

      The cause of DBZ-277 was that the decoderbufs plugin was generating bazillions of postgres warnings using elog with level WARNING. Such warnings are sent to the client, in this case the debezium connector. There they are carefully preserved by the JBDC layer, and, as they were never consumed by the connector, used ever more heap memory until finally the heap was exhausted. This was fixed by changing decoderbufs to not generate such warnings. But it seems wrong that the client just ignores warnings and lets them accumulate without limit. The same applies to notifications as far as I can see.

      I didn't look into how notifications are handled, but I took a quick look at warnings.

      It looks like warnings are being received from PostgreSQL in https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java, for example at line 703.

      This is a child of the QueryExecutor class which has a method getWarnings for retrieving warnings (and clearing the internal list of warnings), defined at https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/core/QueryExecutor.java#L369

      It looks like you can't directly get hold of a QueryExecutor instance, instead PgConnection has its own getWarnings method, see https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/jdbc/PgConnection.java#L669 This returns warnings (clearing the internal warnings list) coming from the QueryExecutor as well as from statements. There's also a clearWarnings method that just discards warnings.

      I got lost trying to check that the debezium streaming connection is actually an instance of PgConnection, but it seems likely.

      Summary: the streaming logic should regularly call getWarnings on the connection.

      I reckon it should log any such warnings at DEBUG level.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                duncan.sands Duncan Sands
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: