ReduceToPlaneMF

[WeiqunZhang]

Add a new Reduce to plane function that returns the results in a pair of MultiFabs. The first MultiFab is distributed the same as the original data MultiFab with each Box flatten to 1 cell. The Fabs of the first MultiFab contain box-local reduction results. The second MultiFab has only a single cell in the reduction direction, and it contains global reduction results distributed in the other two directions.

[WeiqunZhang]

Test code

#include <AMReX.H>
#include <AMReX_ParmParse.H>
#include <AMReX_Print.H>
#include <AMReX_MultiFabUtil.H>

using namespace amrex;

int main(int argc, char* argv[])
{   
    amrex::Initialize(argc, argv);
    {   
        int ncomp = 1;
        int n_cell = 64;
        int max_grid_size = 32;

        Box domain(IntVect(0), IntVect(n_cell-1));
        BoxArray ba(domain);
        ba.maxSize(max_grid_size);

        MultiFab mf(ba, DistributionMapping{ba}, ncomp, 0);
        mf.setVal(1);
        auto const& ma = mf.const_arrays();

        auto [mf2, mf2_unique] = ReduceToPlaneMF<MultiFab,ReduceOpSum,Real>
            (2, domain, mf, [=] AMREX_GPU_DEVICE (int b, int i, int j, int k)
        {   
            return ma[b](i,j,k);
        });

        printCell(mf2, IntVect(3,4,0));
        printCell(mf2_unique, IntVect(3,4,0));

        // auto phi = poisson_solver(mf2_unique);
        // mf2.ParallelCopy(phi);
    }
    amrex::Finalize();
}

[WeiqunZhang]

This is only correct if it's reduce sum. We don't have a way for min and max yet. So maybe we can add a static assert there.