Migrating from the Elasticsearch Sink Connector to the HTTP Sink Connector for OpenSearch

Target Audience

This guidance applies to Confluent Platform users who meet the following criteria:

• You are operating a self-managed Elasticsearch Sink connector version below 14.1.0 as outlined in the official Confluent documentation.
• You are using this connector to send data streams to an OpenSearch cluster.

Overview

This guide provides step-by-step instructions for migrating from unsupported versions of the Elasticsearch Sink Connector to the HTTP Sink Connector when streaming data to OpenSearch. It applies exclusively to self-managed connectors using Confluent Platform and does not apply to Confluent Cloud connectors.

In accordance with the updated support policy for self-managed connectors, certain older Connector versions are no longer supported or available for download. To ensure continued support and compatibility, Confluent recommends users transition to one of the following options based on their deployment model:

Recommended Migration Paths

Background: Support Policy Update

Effective July 6, 2025, Confluent has updated its support policy for self-managed connector versions in Confluent Platform. Under this policy:

• Elasticsearch Sink Connector versions older than 14.1.0 are no longer supported by Confluent, as per the Supported Connector Versions documentation.
• Action Required: Starting September 22, 2025, all unsupported versions (i.e., versions below 14.1.0) will be removed from Confluent Hub and will no longer be available for download.
• To help clarify version compatibility and support status, refer to the matrix below:

This change impacts users who rely on older Elasticsearch Sink Connector versions to send data to OpenSearch clusters.

Migration Instructions for Self-Managed Deployments

To ensure the continued and supported operation of your data pipeline to OpenSearch, you must perform a migration from the Elasticsearch Sink connector to the HTTP Sink connector. The HTTP Sink connector is fully supported for this use case.

Step 1: Install the HTTP Sink Connector

If the HTTP Sink connector is not already installed in your Confluent Platform environment, you must download and install the latest version from Confluent Hub and follow the standard installation procedure for your environment

• Action: Download the HTTP Sink Connector from Confluent Hub and follow the standard installation procedure for your environment.

Step 2: Configure the Connector

Use the following configuration as a template for your new HTTP Sink connector. This configuration is designed to forward single messages to a specific OpenSearch index.

{
     "topics": "<your-kafka-topic>",
     "tasks.max": "1",
     "connector.class": "io.confluent.connect.http.HttpSinkConnector",
     "key.converter": "org.apache.kafka.connect.storage.StringConverter",
     "value.converter": "io.confluent.connect.avro.AvroConverter",
     "value.converter.schema.registry.url": "http://schema-registry:8081",
     "confluent.topic.bootstrap.servers": "broker:9092",
     "confluent.topic.replication.factor": "1",
     "reporter.bootstrap.servers": "broker:9092",
     "reporter.error.topic.name": "error-responses",
     "reporter.error.topic.replication.factor": 1,
     "reporter.result.topic.name": "success-responses",
     "reporter.result.topic.replication.factor": 1,
     "reporter.result.topic.value.format": "string",
     "http.api.url": "<Opensearch-endpoint>",
     "auth.type": "BASIC",
     "connection.user": "<username>",
     "connection.password": "<password>",
     "request.body.format" : "json",
     "headers": "Content-Type: application/json",
     "https.ssl.protocol":"TLSv1.2",
     "batch.json.as.array":"false",
     "ssl.enabled":"false"
}

Important: Before deploying, you must replace the placeholder values in the configuration with your specific environment details:

• <your-kafka-topic>: The source Kafka topic containing data for OpenSearch.
• <Opensearch-endpoint>: The full API endpoint URL for your OpenSearch cluster.
• <username>: The username for basic authentication.
• <password>: The password for basic authentication.

You must also update your Kafka and Schema Registry URL to match your environment. 

Step 3: Verify the Migration

After deploying the newly configured connector, verify its operation by performing the following checks:

1. Checking the Connector Status: Confirm that the connector's status is RUNNING in the Confluent Control Center or via the Connect REST API.
2. Sending Test Data: Produce a sample record to your source Kafka topic.
3. Querying OpenSearch: Execute a query against the target OpenSearch index to verify that the record was successfully indexed. 

Support

Contact Confluent Support for migration issues or configuration assistance.