[WIP] parallel processing for column-based TRowSet generation

[bowenliang123]

🔍 Description

Issue References 🔗

This pull request fixes #

Describe Your Solution 🔧

• Improvement comparison:

• Support parallel processing for column-based TRowSet generation, within a fork-join pool on the engine side

• The order of columns in TRowSet is still guaranteed by sorting the column index, which is a very light cost operation

• Add a config to enable/disable this feature

Types of changes 🔖

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Behavior With This Pull Request 🎉

Related Unit Tests

---

Checklist 📝

• This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 33 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (96c5632) to head (74106c7).

Files with missing lines	Patch %	Lines
...apache/kyuubi/engine/result/TRowSetGenerator.scala	0.00%	14 Missing ⚠️
...ain/scala/org/apache/kyuubi/util/ThreadUtils.scala	0.00%	9 Missing ⚠️
...in/scala/org/apache/kyuubi/config/KyuubiConf.scala	0.00%	5 Missing ⚠️
...kyuubi/engine/spark/operation/SparkOperation.scala	0.00%	4 Missing ⚠️
...gine/spark/schema/SparkArrowTRowSetGenerator.scala	0.00%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #6667   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         683     683           
  Lines       42213   42236   +23     
  Branches     5755    5756    +1     
======================================
- Misses      42213   42236   +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.