Inclusion of redundant software

[espenhgn]

Just a discussion point - the recipes present here incorporates a whole bunch of redundant software which is not needed to compile and run NEST. Why is that the case?

Using buildpack-deps:focal as base image, only a handful of additional installs are needed (taken from https://github.com/LFPy/hybridLFPy_EBRAINS/blob/main/Dockerfile):

FROM buildpack-deps:focal

RUN apt-get update && \
    apt-get install -y \
    cmake \
    libmpich-dev \  # or libopenmpi-dev
    mpich \  # or openmpi-bin
    doxygen \
    libboost-dev \
    libgsl-dev \
    cython3 \
    python3-dev \
    python3-pip

RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10 && \
    update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 10

RUN pip install mpi4py  # or apt-get install -y python3-mpi4py if openmpi is used.

# ----- Install NEST -----
RUN git clone https://github.com/nest/nest-simulator.git && \
    cd nest-simulator && \
    # git checkout master && \
    # git checkout 24de43dc21c568e017839eeb335253c2bc2d487d && \
    cd .. && \
    mkdir nest-build && \
    ls -l && \
    cd  nest-build && \
    cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/nest/ \
        -Dwith-ltdl=ON \
        -Dwith-gsl=ON \
        -Dwith-readline=ON \
        -Dwith-python=ON \
        -Dwith-mpi=ON \
        -Dwith-openmp=ON \
        ../nest-simulator && \
    make && \
    make install && \
    cd /

RUN rm -r nest-simulator
RUN rm -r nest-build

# Add NEST binary folder to PATH
ENV PATH /opt/nest/bin:${PATH}

# Add pyNEST to PYTHONPATH
ENV PYTHONPATH /opt/nest/lib/python3.8/site-packages:${PYTHONPATH}

If all examples and tests are to be run, and make things a bit more user friendly, a few additional dependencies may be needed, e.g.,:

# ---- additional requirements
RUN apt-get install -y \
    python3-numpy \
    python3-scipy \
    python3-matplotlib \
    python3-pandas \
    ipython3 \
    jupyter

RUN update-alternatives --install /usr/bin/ipython ipython /usr/bin/ipython3 10

I know MUSIC etc. may require some additional dependencies, but it can't possibly be that many, no?

[terhorstd]

This has different reasons, (a) some requirements are required to build secondary targets like the user level documentation and (b) the image serves a versatile usage, e.g for the use in a classroom setting.
I think it would make sense to have separate images for different purposes. The overall number should remain low though. Images for laptops do not need the different MPI implementations base, as the HPC ones would, for example.

[espenhgn]

Hi, I think a bit more "granularity" in terms of features included in the images would be preferable - at least having one somewhat lightweight runtime option (nest, pynest, gsl, openmp, MPI, boost) would be nice. But I agree it shouldn't be too many. For me it would make sense to base the hybridLFPy image on such an official NEST image in that case.
As I quite often use MPI locally for testing and debugging, I wouldn't differentiate between HPC images and laptop images. But MPICH and openMPI options may be needed. JUSUF requires MPICH it seems, but other clusters may need openMPI for all that I know.