Disable external downloads in runtime images

[grig03]

Hi everyone!

Is there a way to disable bootstrapper and requirements downloading as the first step of any kfp pipeline?

Downloading https://raw.githubusercontent.com/elyra-ai/elyra/v3.11.0/elyra/kfp/bootstrapper.py
Downloading https://raw.githubusercontent.com/elyra-ai/elyra/v3.11.0/etc/generic/requirements-elyra.txt
Downloading https://raw.githubusercontent.com/elyra-ai/elyra/v3.11.0/etc/generic/requirements-elyra-py37.txt

The goal is to remove any external requests from the pipeline run.
Tried installing all the reqs and setting environment variables ELYRA_BOOTSTRAP_SCRIPT_URL, ELYRA_REQUIREMENTS_URL and ELYRA_REQUIREMENTS_URL_PY37 in the custom runtime image Dockerfile, but this didn't affect the process.

Can this behavior be modified from elyra UI?

[ptitzler]

Currently it is not possible to prevent those downloads without editing the Elyra source code. We have an open work item (@akchinSTC) that will remove the need for these downloads by making dependencies available through other means. What you can do, if you can make those files available within your network, is to set those environment variables to the location of your files in the environment where Jupyterlab/Elyra is running. When the pipeline editor processes the pipeline (for submission or export) it evaluates those variables and embeds the custom values in the generated DAG. When the DAG is executed in KFP or Airflow the files will be downloaded from the location you've specified.

Reference: https://elyra.readthedocs.io/en/latest/recipes/running-elyra-in-air-gapped-environment.html