Description
#8209 introduced GlutenAutoAdjustStageResourceProfile for only Exchange plan, and leave a TODO to support final stage.
A final stage is not necessarily a lightweight result-collection step. It can contain substantial computation after reading shuffle output, for example a final aggregate, sort, window, join processing, projection, or row/columnar conversion. These operators may partially or wholly fall back to Spark.
Gluten version
main branch
Description
#8209 introduced
GlutenAutoAdjustStageResourceProfilefor onlyExchangeplan, and leave a TODO to support final stage.A final stage is not necessarily a lightweight result-collection step. It can contain substantial computation after reading shuffle output, for example a final aggregate, sort, window, join processing, projection, or row/columnar conversion. These operators may partially or wholly fall back to Spark.
Gluten version
main branch