Details
-
Bug
-
Resolution: Unresolved
-
Major
-
1.9.0.Final
-
False
-
None
-
False
-
Important
Description
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
1.9.0.Final
What is the connector configuration?
mysql connector
What is the captured database version and mode of depoyment?
aws rds databases
What behaviour do you expect?
The connector should be able to start using the offsets available.
What behaviour do you see?
The connector fails to start with binlog not available error.
I was able to dig further into this and have narrowed it down to isBinlogAvailable()
When I compare the log messages for the connectors that are failing vs which ones are not, for the failed ones the below condition evaluates to true.
LOGGER.info("GTIDs known by the server but not processed yet {}, for replication are available only {}", gtidSetToReplicate, nonPurgedGtidSetToReplicate); if (!gtidSetToReplicate.equals(nonPurgedGtidSetToReplicate)) { LOGGER.info("Some of the GTIDs needed to replicate have been already purged"); return false; }
From the logs:
MySQL current GTID set
7388bf7d-2aac-11ed-9237-021082410453:1-351497416,
7d187d03-4f6c-11ed-a5ac-0e393478d1e3:1-1291577186,
820d71d6-eeea-11ec-95a8-0e2ad29b0edb:1-487962549,
9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1-1325725070,
d6853f74-3bdf-11e9-9867-0a635193bf30:1-1074470477,
e6330d3c-cd8d-11ec-bf02-02d395688cf9:1-568778112
does contain the GTID set required by the connector 9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1128180378-1325715924
Server has already purged
7388bf7d-2aac-11ed-9237-021082410453:1-351497416,
7d187d03-4f6c-11ed-a5ac-0e393478d1e3:1-1291577186,
820d71d6-eeea-11ec-95a8-0e2ad29b0edb:1-487962549,
9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1-1178611327,
d6853f74-3bdf-11e9-9867-0a635193bf30:1-1074470477,
e6330d3c-cd8d-11ec-bf02-02d395688cf9:1-568778112 GTIDs
GTIDs known by the server but not processed yet
7388bf7d-2aac-11ed-9237-021082410453:1-351497416,
7d187d03-4f6c-11ed-a5ac-0e393478d1e3:1-1291577186,
820d71d6-eeea-11ec-95a8-0e2ad29b0edb:1-487962549,
9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1-1128180377:1325715925-1325725070,
d6853f74-3bdf-11e9-9867-0a635193bf30:1-1074470477,
e6330d3c-cd8d-11ec-bf02-02d395688cf9:1-568778112,
for replication are available only
9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1325715925-1325725070
Above we see that the 2 GTID sets are not equal. The GtidSetToReplicate is bigger than the nonPurgedGtidSetToReplication. In fact, the nonPurged GTID set (9475ad1f-a911-11ed-ae57-0e3cf2efaa99:1325715925-1325725070) is a subset of the GTIDsetToReplicate.
Debezium is concluding that the required Gtids are purged and throws the error.
Do you see the same behaviour using the latest relesead Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
Yes tried with the 2.0 version
Do you have the connector logs, ideally from start till finish?
YES
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
Another example
GTIDs known by the server but not processed yet
01955d90-8bf6-11ec-8e0b-0e713259aa49:1-624,04a40596-e94c-11ec-bdd1-0206de2cb8e3:1-2548764816,2725d589-28db-11ec-84a6-0a4242b0ae27:1-1,2ad16c76-496f-11ec-88ab-0e764c09427d:1-597765848,3af0f9ce-4cd0-11ec-b6c5-027f9764576f:1-624,50fb245d-4558-11ec-b967-48df37123230:1-3088030391,8363c581-2929-11ec-9152-0ec0bb0db8a9:1-1,853cd23e-4aeb-11ec-89fd-d4f5ef034e70:1-207,872985e8-5016-11ed-8b13-022dd6302efd:1-1,88a27582-27e2-11ec-8dbc-1230b81a5e03:1-2,891f06d3-4273-11ec-a8d4-98f2b3dea070:1-4846,900d13cb-cd41-11ed-b68b-02f3bffd01cb:1-2944225410:2952227683-2952289701,929ca87b-8f1f-11ec-8ebe-02f1a96ba199:1-416,b56b4aaa-9a8a-11eb-9d40-d4f5ef034e70:1-34,bdaa6484-2d0c-11ec-88d1-d4f5ef034e70:1-623025111,cd86cecf-331a-11ec-a675-0aa74a9f49f5:1-46,dd46433c-cb89-11ec-99a5-0eb1d868bd1f:1-998526954,de5b887b-532d-11ec-9966-0a71e49ca38d:1-624,ec0ae149-34a7-11ed-907e-0e045769649f:1-4550076894,ed3862b2-8cc2-11eb-8363-48df37123230:1-604410,f7af72c4-55d8-11ed-b25a-0a2a42252c87:1-5,f8e0ebe5-2fe1-11e9-9b17-98f2b3dea070:1-239341820,
for replication are available only
900d13cb-cd41-11ed-b68b-02f3bffd01cb:2545561302-2944225410:2952227683-2952289701
900d13cb-cd41-11ed-b68b-02f3bffd01cb:2545561302-2944225410:2952227683-2952289701 is a subset of
900d13cb-cd41-11ed-b68b-02f3bffd01cb:1-2944225410:2952227683-2952289701 and the method should not look explicitly for exact match
