[SPARK-49147][CORE] Mark KryoRegistrator with DeveloperApi interface

[robreeves]

What changes were proposed in this pull request?

Trait org.apache.spark.serializer.KryoRegistrator is a public interface because it is exposed via config spark.kryo.registrator, but it is not annotated to describe the stability level. This adds the DeveloperApi annotation to formalize the contract.

Why are the changes needed?

This will help users understand the compatibility guarantees when using the trait. It is also helpful for scripts that need to identify classes with public APIs. We recently missed this trait in a script used to analyze which namespaces can be shaded.

Does this PR introduce any user-facing change?

Yes, it adds the DeveloperApi interface to the trait.

How was this patch tested?

N/A. Annotation change only.

Was this patch authored or co-authored using generative AI tooling?

No

[robreeves]

@dongjoon-hyun @mridulm please take a look

[dongjoon-hyun]

In addition, this cannot be Stable because JavaSerializer is DeveloperApi.

spark/core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala

Lines 161 to 162 in 435a01a

	@DeveloperApi
	class JavaSerializer(conf: SparkConf) extends Serializer with Externalizable {

If we want to annotate, this should have DeveloperApi in the similar way.

[dongjoon-hyun]

Since it's a documented for a long time, I agree with the proposal.
https://spark.apache.org/docs/latest/api/scala/org/apache/spark/serializer/KryoRegistrator.html

+1, LGTM. Thank you, @robreeves .

[mridulm]

Do we want to make both of these @Stable @dongjoon-hyun ? These interfaces have been around since 0.x days :-)

In general, we have rarely promoted any of our DeveloperApi api to Stable - something to look into for 4.0 ?

[dongjoon-hyun]

If you want, we can do, of course, @mridulm . :)

Do we want to make both of these @Stable

[yaooqinn]

May it be broken if we migrate Kryo from 4.x to 5.x？

[dongjoon-hyun]

May it be broken if we migrate Kryo from 4.x to 5.x？

I didn't see any breaking change in public API of Kryo.java. Do you mean that you know something intrusive recently, @yaooqinn ?

• https://github.com/EsotericSoftware/kryo/blame/master/src/com/esotericsoftware/kryo/Kryo.java

And, we depend on twitter-chill which has no change since 2021. I thought we don't have any plan to move out from this.

• https://github.com/twitter/chill/tags

[yaooqinn]

Hi @dongjoon-hyun the question is just for curiosity to make it Stable or DeveloperAPI

Thank you for the explanation.

[pan3793]

twitter-chill made some progress in migrating to Kryo5 twitter/chill#747

[mridulm]

Thinking more, let us decouple DeveloperApi -> Stable from this PR.
IMO we should revisit our public api and reclassify stable developer API's as stable - but that is a more cross cutting effort (and this specific one can be revisited in that context)

[dongjoon-hyun]

Sounds good to me. +1 for merging this PR in the AS-IS status.

BTW, could you re-trigger the CI, @robreeves ?

[yaooqinn]

The docker test failure is irrelevant. Merged to master.

Thank you @robreeves @dongjoon-hyun @mridulm @pan3793