How to use kafka-connect-opensearch with Opensearch

How can I use the kafka-connect-OpenSearch connector to directly send data from Kafka topic to OpenSearch? Earlier I was using FluentD but now the OpenSearch connector is just launched, so I want to leverage that connector to eliminate the need for fluentD(I was using that for just connecting Kafka and OpenSearch).
I followed this page from the official doc.

I did the following things till now,

  1. Configured Kafka and zookeeper using docker
  2. Schema registry to use connectors
    docker run -d --net=host --name=schema-registry
    -e SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=localhost:2181
    -e SCHEMA_REGISTRY_HOST_NAME=localhost
    -e SCHEMA_REGISTRY_LISTENERS=http://localhost:8081 confluentinc/cp-schema-registry:5.0.0
  3. Created kafka topic to store config, status, and offsets of the connectors in Kafka topics

everything is up and running without errors
now I want to use OpenSearch connector - GitHub - dmathieu/kafka-connect-opensearch: Kafka Connect plugin for sending events to OpenSearch
What will be the next steps to use this particular connector?

hey @spzala!

Looks like that connector is pretty new (11 days!) so Iā€™m not sure how working/stable it is. You might want to touch base with the author on the github repo.

1 Like