Commit f4493fd
committed
[MINOR][SQL][TESTS] Ignore messages below error level from
### What changes were proposed in this pull request?
This pr ignore messages below error level from `HadoopFSUtils` for sql module tests.
### Why are the changes needed?
From the test logs of Github Actions, there are a large number of logs from `HadoopFSUtils`. For example, in the pipeline named `sql-other tests`, there are approximately 40,000 similar logs like the following:
- https://github.com/apache/spark/actions/runs/13915102741/job/38936641387
```
04:22:51.192 WARN org.apache.spark.util.HadoopFSUtils: The directory /home/runner/work/spark/spark/target/tmp/spark-e3d8a446-2dd8-42fd-a64f-b96504d8cd69/deleted was not found. Was it deleted very recently?
04:22:51.193 WARN org.apache.spark.util.HadoopFSUtils: The directory /home/runner/work/spark/spark/target/tmp/spark-e3d8a446-2dd8-42fd-a64f-b96504d8cd69/deleted was not found. Was it deleted very recently?
[info] - InMemoryFileIndex: root folders that don't exist don't throw exceptions (6 milliseconds)
04:22:51.244 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.271 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.280 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.281 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.307 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.332 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.334 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
04:22:51.334 WARN org.apache.spark.util.HadoopFSUtils: The directory mockFs:/rootDir/subDir was not found. Was it deleted very recently?
```
Similar logs are more like noise, so the current pr makes `HadoopFSUtils` only print error-level logs during SQL module testing.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass Github Actions
- Manual inspection shows that the new tests in the SQL module no longer print similar logs as mentioned above.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #50316 from LuciferYang/sql-log-level.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>HadoopFSUtils for sql module tests1 parent 35e0e08 commit f4493fd
1 file changed
+4
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
0 commit comments