Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7363

Cassandra-4: Debezium connector stops producing events after a schema change

XMLWordPrintable

    • False
    • None
    • False
    • Important

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      debezium-connector-cassandra cassandra-4 v2.5.0.Final

      What is the connector configuration?

      commit.log.relocation.dir=/cdc/commit_log_relocation
      commit.log.real.time.processing.enabled=true # also tried with false
      http.port=8001

      cassandra.config=/opt/cassandra/conf/cassandra.yaml
      cassandra.hosts=cassandra0:9042
      cassandra.port=9042

      kafka.producer.bootstrap.servers=kafka-broker:9092
      kafka.producer.retries=3
      kafka.producer.retry.backoff.ms=1000
      topic.prefix=test_connector

      key.converter=org.apache.kafka.connect.json.JsonConverter
      key.converter.schema.registry.url=http://schema-registry:8081
      value.converter=org.apache.kafka.connect.json.JsonConverter
      value.converter.schema.registry.url=http://schema-registry:8081

      offset.backing.store.dir=/cdc/offset

      snapshot.consistency=ONE

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      versions tried 4.0.2, 4.0.9

      test deployment in a single machine through docker compose.

      What behaviour do you expect?

      I expect debezium to continue producing events after schema alteration.

      What behaviour do you see?

      Given that there is a table with cdc enabled, any schema change (ALTER TABLE ...) will cause the debezium connector to stop producing events. A restart of the debezium connector process will fix the issue. There is a high possibility of data loss if the commitlog has been rotated

      Do you see the same behaviour using the latest relesed Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      Yes, did not test with Alpha.

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      <Your answer>

      How to reproduce the issue using our tutorial deployment?

      • Start cassandra and debezium connector. We run queries using cqlsh o the running cassandra container. (docker compose cassandra exec cqlsh -e ...)
      • Create a keyspace test_ks
      CREATE KEYSPACE test_ks WITH replication = {'class': 'SimpleStrategy', 'replication_factor':'1'}; 
      • Create a table with cdc enabled, named table_cdc on keyspace test_ks
      CREATE TABLE test_ks.table_cdc (key text PRIMARY KEY, value text) WITH cdc=true;
      • Add a column to the table
      ALTER TABLE test_ks.table_cdc ADD value2 text;
      • debezium stops producing events, a restart fixes it.

            Unassigned Unassigned
            schampilomatis Stavros Champilomatis
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: