Skip to content

Commit c730365

Browse files
panbingkunHeartSaVioR
authored andcommitted
[SPARK-45588][SPARK-45640][SQL][TESTS][3.4] Fix flaky ProtobufCatalystDataConversionSuite
### What changes were proposed in this pull request? The pr aims to fix flaky ProtobufCatalystDataConversionSuite, include: - Fix the type check (when the random value was empty array, we didn't skip it. Original intention is to skip default values for types.) [SPARK-45588] - When data.get(0) is null, data.get(0).asInstanceOf[Array[Byte]].isEmpty will be thrown java.lang.NullPointerException. [SPARK-45640] Backport above to branch 3.4. Master branch pr: apache#43424 & apache#43493 ### Why are the changes needed? Fix flaky ProtobufCatalystDataConversionSuite. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - Pass GA - Manually test ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#43520 from panbingkun/branch-3.4_SPARK-45640. Authored-by: panbingkun <[email protected]> Signed-off-by: Jungtaek Lim <[email protected]>
1 parent 521c2e0 commit c730365

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,8 @@ class ProtobufCatalystDataConversionSuite
136136
while (
137137
data != null &&
138138
(data.get(0) == defaultValue ||
139-
(dt == BinaryType &&
139+
(dt.fields(0).dataType == BinaryType &&
140+
data.get(0) != null &&
140141
data.get(0).asInstanceOf[Array[Byte]].isEmpty)))
141142
data = generator().asInstanceOf[Row]
142143

0 commit comments

Comments
 (0)