Description
When connecting to Confluent Cloud Schema Registry through proxy, client may return Misdirected Request; error code: 421 error:
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException:
Misdirected Request; error code: 421
at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:353)
at io.confluent.kafka.schemaregistry.client.rest.RestService.lambda$httpRequest$0(RestService.java:426)
Applies To
Confluent Cloud
Schema Registry
Cause
This issue can occur if schema registry client is configured to connect to Confluent Cloud Schema Registry through a Proxy where Proxy Hostname differs from Confluent Cloud Schema Registry endpoint FQDN. Confluent Cloud Schema Registry has an extra validation in place where it checks the requests hostname originated for the Confluent Cloud Schema Registry's endpoint or not, among other checks.
When a Proxy is used for CCloud Schema Registry, example below, the schema registry will be pointing to proxy's hostname instead
schema.registry.url=http://localhost:9000
The proxy http://localhost:9000 will be redirecting the request to Confluent Cloud and due to server side validation the request is returned with 421 error.
Resolution
Regardless of that extra validation, we strongly recommend configuring the Schema Registry endpoint with an FQDN as a best practice.
Please note that using localhost or a hostname not matching Confluent Cloud SR hostname in Schema Registry Client is not recommended and Schema Registry Full FQDN should be configured, example:
schema.registry.url=https://<psrc-abc…>.xxx.confluent.cloud
This is also documented in Confluent's public documentation here What-is-the-best-way-to-set-up-a-custom-managed-application-on-Confluent-Cloud-Schema-Registry-with-a-proxy.
Note: For connectors the schema registry client config would look like below:
"value.converter.schema.registry.url":"https://<psrc-abc…>.xxx.confluent.cloud",
"key.converter.schema.registry.url":"https://<psrc-abc…>.xxx.confluent.cloud",