Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1405

Replace Custom Schema with Pluggable Serializers via KC Schema in Cassandra Connector

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Done
    • Minor
    • 0.10.0.CR2
    • None
    • cassandra-connector
    • None

    Description

      The idea is to make the aspect of serialization configurable, i.e. how the events created by the connector are converted into the binary representation sent to Kafka. Kafka Connect has the notion of converters for this which take a KC Record and convert to/from binary (see the "key.converter" and "value.converter" of KC). Currently, as the Cassandra connector is not based on KC, it's exclusively using Avro (via GenericRecord). Ideally, the existing Converter contract from KC (and thus all the existing implementations such as Avro and JSON or custom ones) would be supported. For that, it'd have to change how messages are produced: instead of assembling Avro's GenericRecord, this would have to be done using KC's SourceRecord. That'd establish add a dependency to the KC API which might seem surprising at first, but I think it's advantageous eventually, as it allows for re-use of all the existing converters and makes for a consistent configuration of this matter with the other, actually KC-based connectors.

      Attachments

        Activity

          People

            zhou.bing@husky.neu.edu Bingqin Zhou (Inactive)
            jgao54 Joy Gao (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: