feat(DOC-2012): document Kafka 4.x client compatibility and KIP-890

modules/develop/pages/kafka-clients.adoc

@@ -3,15 +3,24 @@
 :page-categories: Clients, Development, Kafka Compatibility
 :pp: {plus}{plus}
 :description: Kafka clients, version 0.11 or later, are compatible with Redpanda. Validations and exceptions are listed.
+:page-topic-type: reference
+:personas: developer
+:learning-objective-1: Identify which Kafka clients are validated with Redpanda
+:learning-objective-2: Identify unsupported Kafka features when integrating with Redpanda
 // tag::single-source[]
 
-Redpanda is compatible with Apache Kafka versions 0.11 and later, with specific exceptions noted on this page.
+Redpanda is compatible with Apache Kafka® versions 0.11 and later, with specific exceptions noted on this page.
+
+Use this reference to:
+
+* [ ] {learning-objective-1}
+* [ ] {learning-objective-2}
 
 == Kafka client compatibility
 
-Clients developed for Kafka versions 0.11 or later are compatible with Redpanda. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers.
+Clients developed for Kafka versions 0.11 or later are compatible with Redpanda. The Apache Kafka Java client at version 4.x is validated against Redpanda using the ducktape and chaos test suites; other clients in the table below have not been broadly re-validated at 4.x but remain compatible with the Kafka protocol versions they have always supported. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers.
 
-TIP: Redpanda Data recommends always using the latest supported version of a client.
+TIP: Always use the latest supported version of a Kafka client.
 
 The following clients have been validated with Redpanda.
 
@@ -34,30 +43,29 @@ The following clients have been validated with Redpanda.
 | https://github.com/kafka-rust/kafka-rust[kafka-rust^]
 
 | Node.js
-a| 
-* https://kafka.js.org[KafkaJS^] 
+a|
+* https://kafka.js.org[KafkaJS^]
 * https://github.com/confluentinc/confluent-kafka-javascript[confluent-kafka-javascript^]
 
 |===
 
-Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations below (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python).
+Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations in the next section (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python).
 
-If you find a client that is not
-supported, reach out to the Redpanda team in the community https://redpanda.com/slack[Slack^].
+If you find a client that does not work with Redpanda, reach out in the https://redpanda.com/slack[Redpanda community Slack^].
 
 == Unsupported Kafka features
 
-Redpanda does not currently support the following Apache Kafka features:
+Redpanda does not support the following Kafka features:
 
-* Multiple SCRAM mechanisms simultaneously for SASL users; for example, a user having both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user, either `SCRAM-SHA-256` or `SCRAM-SHA-512`.
+* Multiple SCRAM mechanisms simultaneously for SASL users. For example, a user cannot have both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user: either `SCRAM-SHA-256` or `SCRAM-SHA-512`.
 +
 ifndef::env-cloud[]
-See the xref:manage:security/authentication.adoc#sasl[Configure Authentication] guide for details.
+For details, see xref:manage:security/authentication.adoc#sasl[Configure SASL authentication].
 endif::[]
 ifdef::env-cloud[]
-See the xref:security:cloud-authentication.adoc[Authentication] guide for details.
+For details, see xref:security:cloud-authentication.adoc[Authentication].
 endif::[]
-* HTTP Proxy (pandaproxy): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions.
+* HTTP Proxy (`pandaproxy`): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions.
 ifdef::env-cloud[]
 +
 * The `delete.retention.ms` topic configuration in Kafka is not supported for Tiered Storage topics. Cloud Topics and local storage topics support Tombstone marker deletion using `delete.retention.ms`, but in Tiered Storage topics, Tombstone markers are only removed in accordance with normal topic retention, and only if the cleanup policy is `delete` or `compact, delete`.
@@ -66,7 +74,9 @@ ifndef::env-cloud[]
 +
 * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported.
 endif::[]
++
+* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890, which addresses transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this class of errors. When connecting to Redpanda, Kafka 4.x clients detect that Transactions V2 is unsupported and fall back to the original transaction protocol (per-transaction epoch bumping is part of V2 and does not apply).
 
-If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^].
+If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team.
 
 // end::single-source[]