[SPARK-42204][CORE] Add option to disable redundant logging of TaskMe… #10497

Job	Run time
Run / Check changes	32s
Run / Base image build	45s
Run / Run TPC-DS queries with SF=1	0s
Run / Protobuf breaking change detection and Python CodeGen check	0s
Run / Build modules: core, unsafe, kvstore, avro, utils, network-common, network-shuffle, repl, launcher, examples, sketch, variant	1h 15m 28s
Run / Build modules: api, catalyst, hive-thriftserver	48m 10s
Run / Run Docker integration tests	0s
Run / Build modules: mllib-local, mllib, graphx	34m 30s
Run / Run Spark on Kubernetes Integration test	0s
Run / Build modules: streaming, sql-kafka-0-10, streaming-kafka-0-10, streaming-kinesis-asl, kubernetes, hadoop-cloud, spark-ganglia-lgpl, protobuf, connect	1h 18m 41s
Run / Build modules: hive - slow tests	45m 2s
Run / Run Spark UI tests	0s
Run / Build modules: hive - other tests	1h 10m 49s
Run / Build modules: sql - extended tests	1h 1m 3s
Run / Build modules: sql - slow tests	52m 50s
Run / Build modules: sql - other tests	1h 10m 56s
Run / Linters, licenses, and dependencies	30m 28s
Run / Build modules: pyspark-sql, pyspark-resource, pyspark-testing	53m 24s
Run / Build modules: pyspark-core, pyspark-errors, pyspark-streaming	34m 56s
Run / Build modules: pyspark-mllib, pyspark-ml, pyspark-ml-connect	1h 14m 22s
Run / Build modules: sparkr	0s
Run / Documentation generation	0s
Run / Build modules: pyspark-connect	55m 5s
	13h 7m 1s

Provide feedback