Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1181

The Postgres wal2json streaming and non-streaming decoders do not process empty events

    XMLWordPrintable

Details

    • Hide

      1. Turn on debugging so that you can see all events read in from wal2json
      2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs)
      3. Make changes like truncate table, DDL, materialized views, etc.
      4. You will see events come in with a format like:

      DEBUG Message arrived for decoding {
    "xid" : 2753761939,
    "timestamp" : "2019-03-06 21:39:01.095414+00",
    "change" : [ ]
}

      5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

      Show
      1. Turn on debugging so that you can see all events read in from wal2json 2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs) 3. Make changes like truncate table, DDL, materialized views, etc. 4. You will see events come in with a format like: DEBUG Message arrived for decoding { "xid" : 2753761939, "timestamp" : "2019-03-06 21:39:01.095414+00" , "change" : [ ] } 5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

    Description

      According to investigations in this WAL2JSON ticket, the plugin sends empty changes for DDL, truncate table, vacuum, and materialized view actions. Debezium's Postgres WAL2JSON streaming and non-streaming decoders do not process these empty changes. They should still process them so that the heartbeat happens and the WAL moves forward.

      Attachments

        Activity

          People

            Unassigned Unassigned
            trolison Taylor Rolison (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: