"fast-initialize" Docker entrypoint for faster setup and testing

[ashiklom]

Currently, the BETY initialization entrypoint takes several minutes to run due to the large volume of data it has to import. That is useful for normal initialization, but it would be nice if there was a minimal version of BETY that could be loaded much more quickly to facilitate testing pull requests.

Looking at the tables included in the initialization, many of the largest are also the ones least necessary for quick tests (e.g. likelihoods, dbfiles, runs).

I think this should be just a matter of creating a new entry in entrypoint.sh that is identical to the current initialize, but which points to a different, minimal BETY dump. E.g. Something like:

    "fast-initialize" )
        echo "Create new database, initialized from minimal data."
        psql -h postgres -p 5432 -U postgres -c "CREATE ROLE bety WITH LOGIN CREATEDB NOSUPERUSER NOCREATEROLE PASSWORD 'bety'"
        psql -h postgres -p 5432 -U postgres -c "CREATE DATABASE bety WITH OWNER bety"
        ./script/load.bety.sh -a "postgres" -d "bety" -p "-h postgres -p 5432" -o bety -c -u -g -m ${LOCAL_SERVER} -r 0 -w https://ebi-forecast.igb.illinois.edu/pecan/dump/FAST/bety.tar.gz
;;

[robkooper]

The other options is to have a special image that is the latest dump of the database that can be copied to the postgis image volume. There is no need at that point to run the code.

[dlebauer]

Should always be storing and passing around all of these records?

…

_______________________ David LeBauer Director of Data Sciences Arizona Experiment Station THE UNIVERSITY OF ARIZONA Bioscience Research Labs, 207 1230 N Cherry Ave | Tucson, AZ 85721 Office: 520-621-4381 [email protected] (sent from my phone - please pardon brevity and typos)

________________________________ From: Alexey Shiklomanov <[email protected]> Sent: Monday, February 18, 2019 9:47 AM To: PecanProject/bety Cc: Subscribed Subject: [PecanProject/bety] "fast-initialize" Docker entrypoint for faster setup and testing (#639) Currently, the BETY initialization entrypoint takes several minutes to run due to the large volume of data it has to import. That is useful for normal initialization, but it would be nice if there was a minimal version of BETY that could be loaded much more quickly to facilitate testing pull requests. Looking at the tables included in the initialization, many of the largest are also the ones least necessary for quick tests (e.g. likelihoods, dbfiles, runs). I think this should be just a matter of creating a new entry in entrypoint.sh<https://github.com/PecanProject/bety/blob/develop/docker/entrypoint.sh#L5-L10> that is identical to the current initialize, but which points to a different, minimal BETY dump. E.g. Something like: "fast-initialize" ) echo "Create new database, initialized from minimal data." psql -h postgres -p 5432 -U postgres -c "CREATE ROLE bety WITH LOGIN CREATEDB NOSUPERUSER NOCREATEROLE PASSWORD 'bety'" psql -h postgres -p 5432 -U postgres -c "CREATE DATABASE bety WITH OWNER bety" ./script/load.bety.sh -a "postgres" -d "bety" -p "-h postgres -p 5432" -o bety -c -u -g -m ${LOCAL_SERVER} -r 0 -w https://ebi-forecast.igb.illinois.edu/pecan/dump/FAST/bety.tar.gz ;; — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#639>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAcX51cAFXMua-mzHBQ5ZyYhHdcPJbK9ks5vOssNgaJpZM4bBJSB>.