Combining partition, unzip, filter_map

[SuperFluffy]

Related to my issue last week, #801, I have a new use case that I cannot do with the tools currently available in rayon. I currently do this:

let ts: Vec<T> = rs
    .par_iter()
    .filter_map(|r| f_yielding_some_t(r))
    .collect();

let us: Vec<U> = rs
    .par_iter()
    .filter_map(|r| f_yielding_some_u(r))
    .collect();

In order to only perform the traversal once (rs is very big and I am actually doing rs.tuple_combinations().par_bridge(), with the IterTools::tuple_combinations adapter), I want to combine these loops:

let (some_ts, some_us): (Vec<Some<T>>, Vec<Some<U>> = rs
    .par_iter()
    .map(|r| (f_yielding_some_t(r), f_yielding_some_u(r))
    .unzip();

But this would require a second pass over the vectors and demand a second allocation.

I am reading #802 to understand it and also the implementation of unzip.

[cuviper]

One way is to write a wrapper that does further filtering on the unzipped items, like:

struct OnlySome<T>(Vec<T>);

impl<T: Send> ParallelExtend<Option<T>> for OnlySome<T> {
    fn par_extend<I>(&mut self, par_iter: I)
    where
        I: IntoParallelIterator<Item = Option<T>>,
    {
        let some = par_iter.into_par_iter().filter_map(identity);
        self.0.par_extend(some);
    }
}

Here's a full example:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5abffae1c70df13fc496fd82b9e77ede

[SuperFluffy]

Thank you very much. I just implemented this and it helped me go from around 800ms for the construction of 2 hashbrown::HashMaps in successive to about 500ms doing it in one go.

Does it make sense to add something like this to rayon proper?

[cuviper]

Well, I think are a lot of possible variations along these lines, which we can't hope to enumerate. I'm more inclined to present that as an example of rayon composition tricks, but maybe we can have a module of such things provided in their own right.