[#1750] feat(remote merge): Support MapReduce.

[zhengchenyu]

What changes were proposed in this pull request?

MapReduce support remote merge.

Why are the changes needed?

Fix: #1750

Does this PR introduce any user-facing change?

Yes, I will refine documentation in other PR.

How was this patch tested?

unit test, integration test, real job in cluster.

[github-actions]

Test Results

 2 890 files  + 8   2 890 suites  +8   5h 55m 42s ⏱️ - 1m 0s
 1 027 tests + 8   1 025 ✅ + 7   2 💤 + 1  0 ❌ ±0 
12 897 runs  +29  12 867 ✅ +14  30 💤 +15  0 ❌ ±0 

Results for commit 0840490. ± Comparison against base commit d170004.

This pull request removes 2 and adds 10 tests. Note that renamed tests count towards both.

org.apache.uniffle.test.AccessClusterTest ‑ testUsingCustomExtraProperties
org.apache.uniffle.test.AccessClusterTest ‑ test{File}

org.apache.hadoop.mapred.SortWriteBufferManagerTest ‑ testWriteNormalWithRemoteMerge
org.apache.hadoop.mapred.SortWriteBufferManagerTest ‑ testWriteNormalWithRemoteMergeAndCombine
org.apache.hadoop.mapreduce.task.reduce.RMRssShuffleTest ‑ testReadShuffleWithCombine
org.apache.hadoop.mapreduce.task.reduce.RMRssShuffleTest ‑ testReadShuffleWithoutCombine
org.apache.uniffle.client.shuffle.MRCombinerTest ‑ testMRCombiner
org.apache.uniffle.client.shuffle.RecordCollectorTest ‑ testRecordCollector
org.apache.uniffle.server.buffer.ShuffleBufferWithLinkedListTest ‑ appendRepeatBlockTest
org.apache.uniffle.server.buffer.ShuffleBufferWithSkipListTest ‑ appendRepeatBlockTest
org.apache.uniffle.test.AccessClusterTest ‑ org.apache.uniffle.test.AccessClusterTest
org.apache.uniffle.test.RMWordCountTest ‑ wordCountTest

♻️ This comment has been updated with latest results.

[zuston]

Is this tested in the production enviroment?

[zhengchenyu]

Is this tested in the production enviroment?

Yes, I have test production job in production cluster.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.72%. Comparing base (d170004) to head (c1116da).
Report is 13 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2109      +/-   ##
============================================
+ Coverage     51.72%   52.72%   +1.00%     
+ Complexity     3275     3115     -160     
============================================
  Files           524      482      -42     
  Lines         29833    25589    -4244     
  Branches       2556     2412     -144     
============================================
- Hits          15431    13492    -1939     
+ Misses        13400    11207    -2193     
+ Partials       1002      890     -112     

Flag	Coverage Δ
	52.72% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.