[FLINK-38515][flink-avro-confluent-registry] Implement OAuth2 mechanism that supports token inheritance for the Schema Registry authentication

[sonyaparau]

What is the purpose of the change

The purpose of the change is to enable OAuth2 authentication for connecting to a remote Schema Registry when validating Avro schemas.

By adding new Avro-Confluent properties, the Schema Registry can securely inherit the bearer token from the Kafka Bootstrap Server, allowing it to authenticate using the same OAuth2 credentials. This improves security and simplifies authentication management between Kafka and the Schema Registry.

Brief change log

• Extend the avro and debezium-avro-confluent format options with 2 new properties to allow OAuth2 authentication with token inheritance
• Extend the avro and debezium-avro-confluent format options with 1 new optional property to configure the logical cluster of the Schema Registry

Verifying this change

This change is already covered by existing tests, such as RegistryAvroFormatFactoryTest.testDeserializationSchemaWithOptionalProperties()
RegistryAvroFormatFactoryTest.testSerializationSchemaWithOptionalProperties()
DebeziumAvroFormatFactoryTest.testSeDeSchemaWithSchemaOption().

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes - the class AvroConfluentFormatOptions.java
• The serializers: don't know
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: don't know

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? not applicable

[flinkbot]

CI report:

• aaf422f Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build