Skip to content
This repository was archived by the owner on Nov 5, 2018. It is now read-only.

GPII-3250: Pre-warm the data loaded by dataloader #7

Merged
seg10 merged 3 commits into from
Sep 6, 2018
Merged

GPII-3250: Pre-warm the data loaded by dataloader #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
be6ec65
GPII-3250: Pre-warm the data loaded by dataloader
Sep 4, 2018
4468a3e
Decide what to pre-warm based on querying available views
Sep 5, 2018
52387ec
Install jq as part of the docker image
Sep 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM node:8-alpine

WORKDIR /home/node

RUN apk add --no-cache curl git && \
RUN apk add --no-cache curl git jq && \
git clone https://github.com/GPII/universal.git && \
cd universal && \
rm -f package-lock.json && \
Expand Down
13 changes: 13 additions & 0 deletions loadData.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@ log() {
echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}

warm_indices(){
log "Warming indices..."

for view in $(curl -s $COUCHDB_URL/_design/views/ | jq -r '.views | keys[]'); do
curl -fsS $COUCHDB_URL/_design/views/_view/$view >/dev/null
done

log "Finished warming indices..."
}

loadData() {
log "Loading data from $1"

Expand Down Expand Up @@ -47,3 +57,6 @@ fi
# Submit data
loadData $STATIC_DATA_DIR
loadData $BUILD_DATA_DIR

# Warm Data
warm_indices