v0.10.0a1

This is the release notes of v0.10.0a1. See here for the complete list of solved issues and merged PRs.

New Features

• Oscar
  • Stop importing main module when starting Mars local cluster (#3110)
• Tensor
  • Integrate special error functions (#3060)
  • Integrate part of scipy elliptic functions and integrals (#3111)
• DataFrame
  • Support sort=True for Groupby (#2959, thanks @sak2002!)

Enhancements

• Disable bloom filter in merge for now (#2967)
• [Ray] Implement ray task executor progress (#3008)
• Dump remote tracebacks to make local ones more friendly (#3028)
• Use tell when remove mapper data after execution (#3027)
• Optimize import speed for Mars package (#3022)
• Do not aggressively choose tree method in tile of groupby for distributed setting (#3032)
• [Ray] Implements get_chunks_result for Ray execution context (#3023)
• Refine ThreadedServiceContext.get_chunks_meta usage (#3037)
• Shuffle both sides at the same time for md.merge (#3041)
• Assign reducer ops in task assigner to make them more balanced across cluster (#3048)
• [Ray] Destroy Ray executor when the task finish (#3049)
• [Ray] Implements get_chunks_meta for Ray execution context (#3052)
• [Ray] Support basic subtask retry and lineage reconstruction (#2969)
• Combine tree and shuffle methods in DataFrameGroupBy.agg tile (#3051)
• [Ray] Implements get_total_n_cpu for Ray execution context (#3059)
• [Ray] Implement cancel method on Ray task executor (#3044)
• Use OS-designated ports instead of random ports to create sub pools (#3053)
• Unify DataFrameGroupByAgg's tile logic for auto method (#3084)
• Simplify router clean up when pools or clusters ends (#3086)
• Call immutable web API only once when previous call blocks (#3085)
• [Ray] Create RayTaskState actor as needed by default (#3081)
• [Ray] Implement gc for ray task executor context (#3061)
• Simplify argument passing in actor batch calls (#3098)
• Optimize performance of transfer (#3091)
• Add n_reducers and reducer_ordinal to shuffle operands (#3055)
• Optimize serializable memory (#3120)

Bug fixes

• Fix errors when deleting mapper data (#3018)
• Fix recursive_tile that it may cause duplicated tile for one tileable (#3021)
• Fix error message when sparse data format not supported (#3046)
• Patch pandas to make pickle compatible between 1.2 and 1.3 (#3047)
• Fix chunk index error in auto_merge_chunks (#3057)
• [Ray] Fix ray worker failover (#3080)
• [Metric] Fix prometheus metric backend (#3124)
• Fix mt.{cumsum, cumprod} when the first chunk is empty (#3134)

Tests

• Check initialization of serializables on CI (#3007)
• Use @pytest_asyncio.fixture instead of @pytest.fixture for async fixtures (#3025)
• Change code owners to Mars PMC maintainers (#3031)
• [Ray] Fix ray executor progress test (#3033)
• [Ray] Optimize Ray CI execution time and stability (#3102)
• Make test_session_set_progress more stable under Ray tests (#3103)
• Update pytest imports for test_special.py (#3129)
• [Ray] Fix flaky test test_optional_supervisor_node (#3133)

Others

• Build web code before CIBW when deploying to PyPI (#3014)
• Make PyPI user name configurable (#3130)