Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-78

Describe configurations options for auto-created change data topics

    XMLWordPrintable

Details

    Description

      Kafka can be run with the auto.create.topics.enable boolean property that dictates whether Kafka will auto-create topics that are used by producers or consumers. The challenge is that when this is enabled, Kafka will create the topics with the default number of partitions and replicas. Most people suggest that in production this auto-create behavior be disabled.

      Although a connector that is starting up might be able to log the topics that are missing (so they could be manually created by an admin), this doesn't really help when new tables can be added to the database at any time.

      The MySQL connector should provide a number of topic creation strategies (e.g., never, with 1 partition and up to n replicas, etc.), so that users can configure the connector and specify the strategy for multiple sets of tables (likely list of regexes of table identifiers).

      Attachments

        Issue Links

          Activity

            People

              rk3rn3r René Kerner
              rhauch Randall Hauch (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: