Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2338

LSNs in replication slots are not monotonically increasing

    XMLWordPrintable

Details

    Description

      Upon restart PostgreSQL connector takes the LSN stored in offets and filters out all changes coming from database with lower LSN. The reason is that database can re-send events already seen by Debezium.

      The problem is that with parallel transactions it can happen that they are not stored in order of their LSNs. Genrally the tx that starts first has lower LSN but the one that is committed first is first written into the replication slot.

      This can be a problem in connector restart when we can have higher LSN already stored in the offset but a tx with lower LSN can still arrive.

      Attachments

        Issue Links

          is blocked by

          Task - Represents a small unit of work that is not end-user facing. DBZ-2200 Refactor: Add domain type for LSN

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. DBZ-2397 Transaction data loss when process restarted

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Task - Represents a small unit of work that is not end-user facing. DBZ-3663 Document LSN resuming logic

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Open

          Activity

            People

              jpechane Jiri Pechanec
              jpechane Jiri Pechanec
              Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: