Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7534

Multi-threaded snapshot can enqueue changes out of order

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 2.6.0.Beta1
    • 2.5.1.Final, 2.6.0.Alpha2
    • core-library
    • None

    Description

      When setting the snapshot max threads to a value greater than 1, there is the possibility that when the BufferingSnapshotChangeRecordReceiver is provided a change event, the events can be enqueued out of order.

      Given two threads where each thread captures changes from two separate tables, it's expected that each thread will read the JDBC data set as follows:

      Thread0 - TableA - PK 1
      Thread0 - TableA - PK 2
      Thread0 - TableA - PK 3
      Thread1 - TableB - PK 1
      Thread1 - TableB - PK 2
      Thread1 - TableB - PK 3
      

      The expectation is that regardless of how the tables are interleaved in the event stream, it's expected that each table's PK dispatch should be in read order; however, that isn't always the case due to a race condition with the following code:

      queue.enqueue(bufferedEventRef.getAndSet(nextBufferedEvent).dataChangeEvent);
      

      It is possible in this case that the buffered event's getAndSet, while atomic, can be interleaved with the call to enqueue in such a way that the dispatch output is:

      Thread0 - TableA - PK 2
      Thread0 - TableA - PK 1
      Thread1 - TableB - PK 1
      ...
      

      To maintain the order of dispatch, the entire line should be synchronized when using multiple threads to guarantee that both the buffered event's getAndSet and the enqueue happen atomically without interference from another thread.

      This fix addresses part of the issues with the PostgreSQL test shouldGenerateSnapshotAndContinueStreaming failing randomly.

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            ccranfor@redhat.com Chris Cranford
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: