Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5130

The number of applied messages isn't logged during schema recovery

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Minor Minor
    • 2.0.0.Alpha2
    • None
    • core-library
    • None
    • False
    • None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      Any connector of any version up to 2.0.0Alpha1.

      What is the connector configuration?

      Irrelevant.

      What is the captured database version and mode of depoyment?

      Irrelevant.

      What behaviour do you expect?

      If I understand the code correctly, during schema recovery, the log messages like "Database history recovery in progress, recovered {} records" and "Already applied {} database changes" should be logged at most every  2 seconds (PAUSE_BETWEEN_LOG_MESSAGES).

      This way, when a long-running schema recovery is in progress, it could be observed through logs.

      What behaviour do you see?

      These messages aren't logged. They will be only logged if there is a 2-second or longer pause between two individual records consumed or applied which normally never happens. The exceptional cases are during a long GC pause.

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes.

      Do you have the connector logs, ideally from start till finish?

      2022-05-12 16:57:04,825 INFO   MySqlConnectorIT||test  Started database history recovery   [io.debezium.relational.history.DatabaseHistoryMetrics]
      2022-05-12 17:19:31,210 INFO   MySqlConnectorIT||test  Already applied 648580 database changes   [io.debezium.relational.history.DatabaseHistoryMetrics] 

      The first message is logged 20 minutes after the recovery has started. This was about the time when the worker started running out of heap memory.

      How to reproduce the issue using our tutorial deployment?

      That would be difficult since it requires a large history.

            Unassigned Unassigned
            sergeimorozov Sergei Morozov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: