Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-131

MySQL GTID logging too verbose

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Critical
    • 0.3.3, 0.4
    • 0.3.2
    • mysql-connector
    • None

    Description

      When testing MySQL connector 0.3.2, we found that DBZ-116 logging is quite verbose. It was flooding our log files with:

      [2016-10-06 22:23:15,064] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238014, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946837,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726410,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993080,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:15,208] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238206, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946838,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726410,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993080,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=1654615818} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:15,209] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000001, pos=953687765, gtids=4d1a4918-44ba-11e6-bf12-42010af0040b:1-216105:216106-473705:473706-478588:478589-478978:478979-481935:481936-707637:707638-724794:724795-726293:726294-726410,716ec46f-d522-11e5-bb56-0242ac110004:1-11904288:11904289-11907963:11907964-12432354:12432355-12433217:12433218-12454010:12454011-12949707:12949708-12991958:12991959-12993080,f8f9cf64-86c9-11e6-8aab-42010af00004:1-4, row=0, server_id=1654615818} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:15,597] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000001, pos=953687957, gtids=4d1a4918-44ba-11e6-bf12-42010af0040b:1-216105:216106-473705:473706-478588:478589-478978:478979-481935:481936-707637:707638-724794:724795-726293:726294-726411,716ec46f-d522-11e5-bb56-0242ac110004:1-11904288:11904289-11907963:11907964-12432354:12432355-12433217:12433218-12454010:12454011-12949707:12949708-12991958:12991959-12993080,f8f9cf64-86c9-11e6-8aab-42010af00004:1-4, row=0, server_id=712553133} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:15,598] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238398, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946838,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726411,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993080,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=712553133} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:16,066] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238600, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946838,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726411,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993081,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:16,213] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000001, pos=953688149, gtids=4d1a4918-44ba-11e6-bf12-42010af0040b:1-216105:216106-473705:473706-478588:478589-478978:478979-481935:481936-707637:707638-724794:724795-726293:726294-726411,716ec46f-d522-11e5-bb56-0242ac110004:1-11904288:11904289-11907963:11907964-12432354:12432355-12433217:12433218-12454010:12454011-12949707:12949708-12991958:12991959-12993081,f8f9cf64-86c9-11e6-8aab-42010af00004:1-4, row=0, server_id=1654615818} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:16,214] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238792, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946839,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726411,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993081,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=1654615818} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:16,602] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=321238984, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946839,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726412,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993081,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=712553133} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:16,603] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000001, pos=953688341, gtids=4d1a4918-44ba-11e6-bf12-42010af0040b:1-216105:216106-473705:473706-478588:478589-478978:478979-481935:481936-707637:707638-724794:724795-726293:726294-726412,716ec46f-d522-11e5-bb56-0242ac110004:1-11904288:11904289-11907963:11907964-12432354:12432355-12433217:12433218-12454010:12454011-12949707:12949708-12991958:12991959-12993081,f8f9cf64-86c9-11e6-8aab-42010af00004:1-4, row=0, server_id=712553133} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:10,830] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=320863677, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946555,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726406,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993076,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:10,838] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=320864390, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946556,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726406,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993076,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:10,846] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=320865080, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946557,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726406,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993076,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)
[2016-10-06 22:23:10,852] INFO 1 records sent, last offset: {ts_sec=1475792, file=mysql-bin.000015, pos=320865846, gtids=01261278-6ade-11e6-b36a-42010af00790:1-112887518:112887519-118345710:118345711-118362550:118362551-118414995:118414996-118419527:118419528-118428415:118428416-118433356:118433357-118942940:118942941-118946558,4d1a4918-44ba-11e6-bf12-42010af0040b:1-478987:478988-724794:724795-726406,716ec46f-d522-11e5-bb56-0242ac110004:1-12444754:12444755-12941724:12941725-12991958:12991959-12993076,ab33eb9e-89ad-11e6-877b-42010af0002d:1-3, row=0, server_id=2094912597} (io.debezium.connector.mysql.BinlogReader)

      Which made it hard to figure out what was really going on. Given that DEBUG already gives an equivalent message, perhaps a better solution is to just emit the messages less frequently.

      Attachments

        Issue Links

          incorporates

          Bug - A problem that impairs or prevents the functions of the product. DBZ-130 DBZ should print out GTID which it has read and committed to log

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Closed
          is caused by

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-116 MySQL connector should periodically output status when reading binlog

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              rhauch Randall Hauch (Inactive)
              criccomini Chris Riccomini (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: