crontab-example

# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
15 7 * * 1-5 /Sites/cold/sync_caltech_directory.bash
30 7 * * 1-5 /Sites/cold/rpt_people_csv.bash

# CaltechTHESIS harvest
# Incremental: every 15 min, Mon-Fri, 7am-7pm (requires full harvest to have run first)
*/15 7-19 * * 1-5 /Sites/cold/remote_harvest_caltechthesis_incremental.bash /Sites/cold/caltechthesis.env
# Full refresh: nightly at 2am every day (catches deletions and status changes)
0 2 * * * /Sites/cold/remote_harvest_caltechthesis_full.bash /Sites/cold/caltechthesis.env

# CaltechAUTHORS RDM harvest into rdm_requests.ds
#
# Incremental: every 15 min, Mon-Fri, 7am-7pm. Runs in about 15 seconds. Picks
# up new submissions, state changes, edits to published records, and newly
# published versions, then prunes anything that has left the states of interest.
*/15 7-19 * * 1-5 /Sites/cold/remote_harvest_rdm_requests_incremental.bash /Sites/cold/caltechauthors.env
#
# Full refresh: nightly at 3am every day. Takes about 26 minutes -- it detoasts
# roughly 1.5 GB of jsonb -- so it is deliberately away from the CaltechTHESIS
# full harvest at 2am. It upserts and then sweeps rows it did not touch, so an
# interrupted run leaves the collection stale but whole rather than empty.
0 3 * * * /Sites/cold/remote_harvest_rdm_requests_full.bash /Sites/cold/caltechauthors.env
#
# These two do not collide on this schedule -- the full harvest finishes around
# 3:30am, hours before the first incremental. A lock enforces one-harvest-at-a-
# time anyway, which covers the cases the schedule does not: a harvest run by
# hand during the working day, and a full harvest that runs long. An incremental
# that finds one in progress skips quietly and exits 0, since the watermark is
# untouched and the next run covers the same window.
#
# The full harvest must run once before the first incremental, to create the
# collection and write rdm_requests_lastmod.txt.