Details
-
Task
-
Resolution: Done
-
Major
-
None
-
None
Description
The unwrap-smt example should be expanded to cover Elasticsearch as a sink, too. Also a blog post could fall out of this.
These are the JARs needed for the sink connector in the local Docker ifle:
# Deploy Confluent Elasticsearch sink connector ENV KAFKA_CONNECT_ES_DIR=$KAFKA_CONNECT_PLUGINS_DIR/kafka-connect-elasticsearch RUN mkdir $KAFKA_CONNECT_ES_DIR && cd $KAFKA_CONNECT_ES_DIR &&\ curl -sO http://packages.confluent.io/maven/io/confluent/kafka-connect-jdbc/3.3.0/kafka-connect-jdbc-3.3.0.jar &&\ curl -sO http://packages.confluent.io/maven/io/confluent/kafka-connect-elasticsearch/3.3.0/kafka-connect-elasticsearch-3.3.0.jar && \ curl -sO http://central.maven.org/maven2/io/searchbox/jest/2.0.0/jest-2.0.0.jar && \ curl -sO http://central.maven.org/maven2/org/apache/httpcomponents/httpcore-nio/4.4.4/httpcore-nio-4.4.4.jar && \ curl -sO http://central.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar && \ curl -sO http://central.maven.org/maven2/org/apache/httpcomponents/httpasyncclient/4.1.1/httpasyncclient-4.1.1.jar && \ curl -sO http://central.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar && \ curl -sO http://central.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar && \ curl -sO http://central.maven.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.jar && \ curl -sO http://central.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar && \ curl -sO http://central.maven.org/maven2/io/searchbox/jest-common/2.0.0/jest-common-2.0.0.jar && \ curl -sO http://central.maven.org/maven2/com/google/code/gson/gson/2.4/gson-2.4.jar
And that's the connector config:
{ "name": "es-sink", "config": { "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", "tasks.max": "1", "topics": "customers", "topic.index.map" : "logs:logs_index", "connection.url" : "http://localhost:9200", "type.name" : "log", "key.ignore" : "true", "schema.ignore" : "true", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope" } }