[SPARK-51418][SQL] Fix DataSource PARTITON TABLE w/ Hive type incompatible partition columns

[yaooqinn]

What changes were proposed in this pull request?

25/03/06 08:25:17 WARN HiveExternalCatalog: Hive incompatible types found: timestamp_ntz. Persisting data source table `spark_catalog`.`default`.`c` into Hive metastore in Spark SQL specific format, which is NOT compatible with Hive.
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: InvalidObjectException(message:Invalid partition column type: timestamp_ntz)

The partition columns are duplicated and stored both in the HMS column meta and the table properties. If they contain incompatible data types, the HMS Meta API will fail the process.

We can rely on the table properties to read/write

Why are the changes needed?

bugfix, otherwise, newly added spark data types are not able to be used as partition columns

Does this PR introduce any user-facing change?

Yes, More type cases are supported for partitioned datasouce tables stored in HMS

How was this patch tested?

new tests

Was this patch authored or co-authored using generative AI tooling?

no