-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
-
Important
Bug report
From time to time, the connector fails and all data from the history topic disappears, as if the connector is recreating it. The Kafka server itself should not be clearing/deleting the topic, since the retention time for the topic is set to infinity. Currently, the issue is only resolved through a workaround solution:
- Stop the Debezium connector
- Delete the offset topic
- Restart the Debezium connector
After restarting, it recreates the history topic and populates it with schemas.
However, after a few days, the situation repeats itself.
What Debezium connector do you use and what version?
quay.io/debezium/connect:2.5
What is the connector configuration?
{
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"snapshot.locking.mode": "none",
"database.user": "REPLACEME",
"database.server.id": "123456",
"database.server.name": "REPLACEME",
"database.port": "3306",
"include.schema.changes": "true",
"database.hostname": "REPLACEME",
"database.password": "REPLACEME",
"snapshot.mode": "schema_only",
"name": "mysql-connector",
"topic.prefix": "REPLACEME",
"schema.history.internal.kafka.topic": "debezium_producer_mysql_schema_history",
"schema.history.internal.kafka.bootstrap.servers": "***:9092,***:9092",
"table.include.list": "1",
"include.query": "true",
"database.include.list": "REPLACEME",
"snapshot.lock.timeout.ms": "60000",
"connect.timeout.ms": "60000",
"event.processing.failure.handling.mode": "ignore"
}
What is the captured database version and mode of depoyment?
Managed AWS Aurora MYSQL-compatibility
8.0.mysql_aurora.3.06.0, last version.
What behaviour do you expect?
Operation correctly, with all topics set to infinite retention time.
What behaviour do you see?
Intermittently, the Debezium connector fails, leading to the complete disappearance of all data from the history topic, as if the connector is recreating it. This unexpected behavior occurs despite the topics being configured with infinite retention times.
Do you see the same behaviour using the latest relesead Debezium version?
We have not tested with the latest version of Debezium, but we have observed similar behavior with version 2.4
Do you have the connector logs, ideally from start till finish?
In the attachment.
How to reproduce the issue using our tutorial deployment?
No, this link to tutorial doesn't work.
Additional configs:
AWS MSK config
auto.create.topics.enable = true allow.everyone.if.no.acl.found = true delete.topic.enable = true zookeeper.connection.timeout.ms = 6000 log.roll.ms = 604800000 default.replication.factor=3 min.insync.replicas=1 num.io.threads=8 num.network.threads=5 num.partitions=2 num.replica.fetchers=2 log.cleanup.policy=delete log.retention.hours=336 replica.lag.time.max.ms=30000 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 socket.send.buffer.bytes=102400 unclean.leader.election.enable=true zookeeper.session.timeout.ms=18000
POD environments
envs: BOOTSTRAP_SERVERS: ***:9092,***:9092,***:9092 CONFIG_STORAGE_TOPIC: debezium_producer_mysql_configs OFFSET_STORAGE_TOPIC: debezium_producer_mysql_offsets STATUS_STORAGE_TOPIC: debezium_producer_mysql_statuses DB_HOST: ***.com DB_USER: *** offset.storage.replication.factor: "1" config.storage.replication.factor: "1" status.storage.replication.factor: "1"
Dockerfile
FROM quay.io/debezium/connect:2.5 as runtime COPY --from=build /kafka/connector.json /kafka/connector.json
