This repository was archived by the owner on Dec 1, 2025. It is now read-only.

Description
map_partitions is an opaque layer for utilizing dask-expr's optimization features. We frequently use map_partitions so it may be of limited use, but it would be useful to get in front of map_partitions with column pre-filtering whenever we can. reduce is a straightforward first case.
The advantage being that we can potentially avoid loading columns we don't use, though it will absolutely be dependent on the user workflow and where it invokes other map_partitions calls