Skip to content

Commit dd960dc

Browse files
conloosm3nu
andauthored
Restructure role, add Systemd timer option. By @conloos (borgbase#112)
* add full path * Update Readme.me: reorder optional Arguments, update cron -> systemd timer * remove ssh_key_file; change cron to timer * Removed cronie from package installation because systemd timer is used * docker.sh - Stops all or selected containers to save the persistent data intact. The containers are started in reverse order * Created arguments_specs.yml * Role restructured: - if needed creation of a service user incl. creation of the ssh-key, - add the ssh key to authorized_keys, - auto init of the repos, - creation and start of systemd timer and services and - installation of the Docker helperscript. * restructure role add import logic * cleanup: user backup_user * - "borg_source_directories" is not longer a required Argument - add "borg_keys_directory" to load key from Service user during starting borgmatic by sudo * Add borgmatic_initialization_repo (bool) as option to disable init of repo * cleanup * fix ansible-lint errors and warnings * fix letter turner * add option: borgmatic_timer * add: - borgmatic_timer_systemd: true readd: - borgmatic_cron_name: "borgmatic" * - renamed borgmatic_cron_name to borgmatic_timer_cron_name to be more convergent. - Change recommendations implemented by m3nu so that creation of a timer (systemd or cron) is optional and can be selected via borgmatic_timer. * Add description to borgmatic_timer_cron_name and borgmatic_timer * Add variable borg_cron_package to install the cron-packages in case of using timer: cron * reworked timer install logic * reworked timer install logic * Add comments for running backup with service account * add new parameters for tests * Switch created to perform the backup as root or service account. If a service account is to be used, it will be created. * Refactored: Check for ssh-key if not present, genereate them. * Refactored * Refactored * renamed tasks/03_configure.yml to tasks/04_create_links_to_borg_and_borgmatic.yml * Refactored * Refactored * add example for service account * Update Python version for testing * No auto init * Add description to install_backup * Add description to install_backup * set coverage back to: m3nu.ansible_role_borgbackup * The initialization of the repository must be activated and does not take place automatically. * The initialization of the repository must be activated and does not take place automatically. * Removed install_backup as var (bool) to prevent that this role run * Rename backup_ssh_command to borg_ssh_command, tis was a double definition * Renamed backup_repository to borg_repository and add better explanations * remove copy ssh-keys and cert parts * Add comments to borg_ssh_key_file and borg_ssh_key_type * Set allways the borg_ssh_key_file and borg_ssh_command to load the right ssh-key. Add borg_ssh_key_type to select the key type by user * Add borg_ssh_key_type * renamed id_rsa to backup * generate ssh-keys (backup and backup.pub) and add better explanation * Print out key if borgmatic_initialization_repo is false * Remove 'su - {{ borgbackup_user }} -c' to execute the borgmatic by the right user * Add Check frequency, therefore, we no longer need to distinguish between normal and large repos * Add link to Article * renamed backup_ssh_command and backup_ssh_key_file to borg_ssh_command and borg_ssh_key_file * Removed: borgmatic_initialization_repo * Removed: borgmatic_initialization_repo * Removed: borgmatic_initialization_repo * revert changes * Add Full Automation * polishing * rename backup.timer and bakup.service to borgmatic.timer and borgmatic.service * remove debug * Try to find services in ansible_facts * Forgot to install Cron * change borg_ssh_key_type to ed25519 * remove conditional checks * - add hint to using a service user - renamed: borg_ssh_key_file to borg_ssh_key_file_path - removed advanced example * add borg_ssh_key_name, renamed borg_ssh_key_file to borg_ssh_key_file_path * removed static pointing to ~/.ssh/backup SSH private key * Add README-Advanced-Examples.md for storing more examples * Fix test idempotence * Dont symlink when using distro packages * Remove old test targets, consistent wording, remove tag * Remove helper scripts, fix absolute path * Fix cron job, add assert to prevent duplicate timers * nit-pick * Create bin links as root, no borg_ssh_command by default. * Add breaking changes note to README --------- Co-authored-by: Manu <[email protected]>
1 parent 981d4f9 commit dd960dc

31 files changed

+820
-229
lines changed

.github/workflows/main.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ jobs:
88
steps:
99
- uses: actions/checkout@v2
1010
- name: Set up Python
11-
uses: actions/setup-python@v1
11+
uses: actions/setup-python@v4
1212
with:
13-
python-version: 3.8
13+
python-version: "3.10"
1414
- name: Install Molecule
1515
run: |
1616
pip install -U pip setuptools wheel

EXAMPLES.md

+129
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# Additional Examples
2+
3+
## Custom SSH key for backups only
4+
5+
```
6+
- hosts: webservers
7+
roles:
8+
- role: m3nu.ansible_role_borgbackup
9+
borg_encryption_passphrase: CHANGEME
10+
borg_repository: ssh://[email protected]/./repo
11+
borgmatic_timer: systemd
12+
borg_ssh_key_name: id_backup
13+
borg_ssh_command: "ssh -i {{ borg_ssh_key_file_path }} -o StrictHostKeyChecking=accept-new"
14+
borg_user: backupuser
15+
borg_group: backupuser
16+
```
17+
18+
## Use service user and copy SSH key to target server
19+
20+
Installs and configures the Borgmatic client and also initializes the repo on the
21+
remote backup server. (not tested)
22+
23+
```
24+
- name: Configure backup
25+
hosts: test.lab
26+
pre_tasks:
27+
- name: Get home of {{ borg_user }}
28+
ansible.builtin.user:
29+
name: "{{ borg_user }}"
30+
state: present
31+
register: user_info
32+
changed_when: false
33+
check_mode: true # Important, otherwise user will be created
34+
35+
- name: Save the user_info, we need them for the home_dir
36+
ansible.builtin.set_fact:
37+
backup_user_info: "{{ user_info }}"
38+
vars_files: []
39+
vars:
40+
borg_encryption_passphrase: "CHANGEME"
41+
borg_repository: "USER@TARGET_SERVER:/PATH/TO/BACKUP"
42+
borg_user: "srv_backup"
43+
borg_group: "srv_backup"
44+
borg_ssh_key_name: id_backup
45+
borg_ssh_command: "ssh -i {{ borg_ssh_key_file_path }} -o StrictHostKeyChecking=accept-new"
46+
borgmatic_timer: systemd
47+
borg_source_directories:
48+
- /srv/www
49+
- /var/lib/automysqlbackup
50+
borg_exclude_patterns:
51+
- /srv/www/old-sites
52+
borg_retention_policy:
53+
keep_hourly: 3
54+
keep_daily: 7
55+
keep_weekly: 4
56+
keep_monthly: 6
57+
borgmatic_hooks:
58+
before_backup:
59+
- echo "`date` - Starting backup."
60+
tasks:
61+
- name: Configure Borg Backup and Backupmatic
62+
tags:
63+
- always
64+
- install_backup
65+
ansible.builtin.include_role:
66+
name: ansible_role_borgbackup
67+
apply:
68+
tags:
69+
- always
70+
71+
72+
- name: Copy SSH-Key to Target {{ borg_repository }} and Init Repo
73+
tags:
74+
- never
75+
- backup_init_repo
76+
block:
77+
- name: Read ssh key
78+
ansible.builtin.slurp:
79+
src: "{{ borg_ssh_key_file_path }}.pub"
80+
register: backup_local_ssh_key
81+
82+
- name: Set authorized key taken from file
83+
ansible.posix.authorized_key:
84+
# example:
85+
# borg_repository: [email protected]:repo
86+
# have three parts: "username"@"FQDN":"path/to/store/backup", specific:
87+
# a) user: m5vz9gp4
88+
# b) fqdn: m5vz9gp4.repo.borgbase.co
89+
# c) dir: repo
90+
user: "{{ borg_repository | regex_search('(.*)@', '\\1') | first }}" # part a)
91+
state: present
92+
key: "{{ backup_local_ssh_key['content'] | b64decode }}"
93+
delegate_to: "{{ borg_repository | regex_search('@(.*):', '\\1') | first }}" # part b)
94+
95+
- name: Init repository
96+
ansible.builtin.command:
97+
cmd: "su - {{ borg_user }} -c '/usr/local/bin/borgmatic rcreate --encryption keyfile --append-only'"
98+
99+
- name: Activate systemd service and timer
100+
when:
101+
- borgmatic_timer is defined and borgmatic_timer == "systemd"
102+
tags:
103+
- never
104+
- backup_init_repo
105+
block:
106+
- name: Populate service facts
107+
ansible.builtin.service_facts:
108+
109+
- name: Start borgmatic services
110+
ansible.builtin.systemd:
111+
name: "{{ item }}"
112+
state: started
113+
enabled: true
114+
masked: false
115+
daemon_reload: true
116+
when: "item in services"
117+
with_items:
118+
- borgmatic.service
119+
120+
# bug: Need own section without masked else the timer are skipped
121+
- name: Start borgmatic timers
122+
ansible.builtin.systemd:
123+
name: "{{ item }}"
124+
state: started
125+
enabled: true
126+
daemon_reload: true
127+
with_items:
128+
- "borgmatic.timer"
129+
```

README.md

+61-34
Original file line numberDiff line numberDiff line change
@@ -6,39 +6,58 @@ Set up encrypted, compressed and deduplicated backups using [BorgBackup](https:/
66

77
Works great with [BorgBase.com](https://www.borgbase.com) - Simple and Secure Hosting for your Borg Repositories. To manage BorgBase repos via Ansible, also see Andy Hawkins' [BorgBase Collection](https://galaxy.ansible.com/adhawkins/borgbase).
88

9-
Main features:
10-
- Set up Borg and Borgmatic
11-
- Add cron job at random time
12-
- Provision new remote [BorgBase.com](https://www.borgbase.com) repo for storing backups (optional)
9+
**Main features**
10+
- Install Borg and Borgmatic from PyPi or distro packages
11+
- Set up Borgmatic config
12+
- Schedule regular backups using Cron or Systemd timer
1313

14+
## Breaking changes
15+
- Older versions of this role set up a separate Cron job for creating and checking
16+
backups. With recent Borgmatic version, this feature is now managed in Borgmatic.
17+
As a result the extra Cron job will be removed by this role.
18+
- Older versions of this role only supported Cron for scheduling. If you use
19+
Systemd timers, be sure to remove the Cron job in `/etc/cron.d/borgmatic` first.
20+
The role will also alert you when trying to use both timers.
1421

15-
## Example Playbook
22+
## Example playbook with root as backup user and Cron timer
1623

1724
```
18-
- hosts: webservers
25+
- hosts: all
1926
roles:
2027
- role: m3nu.ansible_role_borgbackup
2128
borg_encryption_passphrase: CHANGEME
22-
borg_repository: m5vz9gp4@m5vz9gp4.repo.borgbase.com:repo
29+
borg_repository: ssh://xxxxxx@xxxxxx.repo.borgbase.com/./repo
2330
borg_source_directories:
24-
- /srv/www
25-
- /var/lib/automysqlbackup
26-
borg_exclude_patterns:
27-
- /srv/www/old-sites
28-
borg_retention_policy:
29-
keep_hourly: 3
30-
keep_daily: 7
31-
keep_weekly: 4
32-
keep_monthly: 6
31+
- /var/www
3332
borgmatic_hooks:
3433
before_backup:
3534
- echo "`date` - Starting backup."
3635
postgresql_databases:
3736
- name: users
3837
hostname: database1.example.org
3938
port: 5433
39+
```
40+
41+
## Example playbook with service user and Systemd timer
4042

4143
```
44+
- hosts: all
45+
roles:
46+
- role: m3nu.ansible_role_borgbackup
47+
borg_encryption_passphrase: CHANGEME
48+
borg_repository: ssh://[email protected]/./repo
49+
borgmatic_timer: systemd
50+
borg_user: "backupuser"
51+
borg_group: "backupuser"
52+
borg_source_directories:
53+
- /var/www
54+
borg_retention_policy:
55+
keep_hourly: 3
56+
keep_daily: 7
57+
keep_weekly: 4
58+
keep_monthly: 6
59+
```
60+
4261

4362

4463
## Installation
@@ -56,39 +75,47 @@ $ git clone https://github.com/borgbase/ansible-role-borgbackup.git roles/ansibl
5675

5776
## Role Variables
5877

59-
### Required Arguments
60-
- `borg_repository`: Full path to repository. Your own server or [BorgBase.com](https://www.borgbase.com) repo. Not required when using auto creation of repositories. Can be a list if you want to backup to multiple repositories.
61-
- `borg_source_directories`: List of local folders to back up.
78+
### Required Variables
79+
- `borg_repository`: Full path to repository. Your own server or [BorgBase.com](https://www.borgbase.com) repo.
80+
Can be a list if you want to backup to multiple repositories.
6281

63-
### Optional Arguments
82+
### Optional Variables
83+
- `borg_dep_packages`: Dependency Packages to install `borg(backup)` and `borgmatic`.
84+
- `borg_distro_packages`: contains the names of distributions packages for `borg(backup)` and `borgmatic`, only used if `borg_install_method` is set to `package`.
6485
- `borg_encryption_passcommand`: The standard output of this command is used to unlock the encryption key.
6586
- `borg_encryption_passphrase`: Password to use for repokey or keyfile. Empty if repo is unencrypted.
6687
- `borg_exclude_from`: Read exclude patterns from one or more separate named files, one pattern per line.
6788
- `borg_exclude_patterns`: Paths or patterns to exclude from backup. See [official documentation](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-patterns) for more.
89+
- `borg_install_method`: By default `pip` is used to install borgmatic. To install via your distributions package manager set this to `package` and (if needed) overwrite the `borg_distro_packages` variable to contain your distributions package names required to install borgmatic. Note that many distributions ship outdated versions of borgbackup and borgmatic; use at your own risk.
6890
- `borg_lock_wait_time`: Config maximum seconds to wait for acquiring a repository/cache lock. Defaults to 5 seconds.
6991
- `borg_one_file_system`: Don't cross file-system boundaries. Defaults to `true`
92+
- `borg_pip_packages`: Dependancy Packages (pip) to install `borg(backup)` and `borgmatic`.
7093
- `borg_remote_path`: Path to the borg executable on the remote. It will default to `borg`.
7194
- `borg_remote_rate_limit`: Remote network upload rate limit in kiBytes/second.
7295
- `borg_retention_policy`: Retention policy for how many backups to keep in each category (daily, weekly, monthly, etc).
73-
- `borg_ssh_command`: Command to use instead of just "ssh". This can be used to specify ssh options.
96+
- `borg_source_directories`: List of local folders to back up. Default is `/etc/hostname` to prevent an empty backup.
97+
- `borg_ssh_key_name`: Name of the SSH public and pivate key. Default `id_ed25519`
98+
- `borg_ssh_key_file_path`: SSH-key to be used. Default `~/.ssh/{{ borg_ssh_key_name }}`
99+
- `borg_ssh_key_type`: The algorithm used to generate the SSH private key. Choose: `rsa`, `dsa`, `rsa1`, `ecdsa`, `ed25519`. Default: `ed25519`
100+
- `borg_ssh_command`: Command to use instead of just "ssh". This can be used to specify SSH options.
101+
- `borg_version`: Force a specific borg version to be installed
102+
- `borg_venv_path`: Path to store the venv for `borg(backup)` and `borgmatic`
103+
74104
- `borgmatic_check_last`: Number of archives to check. Defaults to `3`
75-
- `borgmatic_checks`: List of consistency checks. Defaults to `['repository']`
76-
- `borgmatic_config_name`: Name to use for the borgmatic config file. Defaults to `config.yaml`
77-
- `borgmatic_cron_checks_day`: Day when cron job for infrequent checks will run. Defaults to `{{ 28 | random }}`
78-
- `borgmatic_cron_checks_hour`: Hour when cron job for infrequent checks will run. Defaults to `{{ range(7, 24) | random }}`
79-
- `borgmatic_cron_checks_minute`: Minute when cron job for infrequent checks will run. Defaults to `{{ 59 | random }}`
80-
- `borgmatic_cron_hour`: Hour when regular create and prune cron job will run. Defaults to `{{ 6 | random }}`
81-
- `borgmatic_cron_minute`: Minute when regular create and prune cron job will run. Defaults to `{{ 59 | random }}`
105+
- `borgmatic_checks`: List of consistency checks. Defaults to monthly checks. See [docs](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups/#check-frequency) for all options.
106+
- `borgmatic_config_name`: Name to use for the Borgmatic config file. Defaults to `config.yaml`
107+
- `borgmatic_timer_hour`: Hour when regular create and prune cron/systemd-timer job will run. Defaults to `{{ 6 | random }}`
108+
- `borgmatic_timer_minute`: Minute when regular create and prune cron/systemd-timer job will run. Defaults to `{{ 59 | random }}`
82109
- `borgmatic_hooks`: Hooks to monitor your backups e.g. with [Healthchecks](https://healthchecks.io/). See [official documentation](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) for more.
83-
- `borgmatic_large_repo`: Less frequent, monthly repo checking. Defaults to `true`
110+
- `borgmatic_timer`: If the variable is set, a timer is installed. A choice must be made between `cron` and `systemd`.
84111
- `borgmatic_relocated_repo_access_is_ok`: Bypass Borg error about a repository that has been moved. Defaults to `false`
85112
- `borgmatic_store_atime`: Store atime into archive. Defaults to `true`
86113
- `borgmatic_store_ctime`: Store ctime into archive. Defaults to `true`
87-
- `ssh_key_file`: Path to a private ssh key file (default is `.ssh/id_ed25519`). It generates a ed25519 key if the file doesn't exist yet.
88-
- `borg_version`: Force a specific borg version to be installed
89114
- `borgmatic_version`: Force a specific borgmatic version to be installed
90-
- `borg_install_method`: By default `pip` is used to install borgmatic. To install via your distributions package manager set this to `package` and (if needed) overwrite the `borg_distro_packages` variable to contain your distributions package names required to install borgmatic. Note that many distributions ship outdated versions of borgbackup and borgmatic; use at your own risk.
91-
- `borg_distro_packages`: contains the names of distributions packages for `borg(backup)` and `borgmatic`, only used if `borg_install_method` is set to `package`.
115+
116+
- `borg_user`: Name of the User to create Backups (service account)
117+
- `borg_group`: Name of the Group to create Backups (service account)
118+
92119

93120
## Contributing
94121

@@ -105,4 +132,4 @@ MIT/BSD
105132

106133
## Author
107134

108-
© 2018-2020 Manuel Riel and contributors.
135+
© 2018-2023 Manuel Riel and contributors.

defaults/main.yml

+31-23
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,13 @@
11
---
22
borg_encryption_passphrase: ''
33
borg_exclude_patterns: []
4-
borgmatic_config_name: config.yaml
5-
borgmatic_large_repo: true
6-
borgmatic_hooks:
7-
on_error:
8-
- echo "`date` - Error while creating a backup."
9-
before_backup:
10-
- echo "`date` - Starting backup."
11-
after_backup:
12-
- echo "`date` - Finished backup."
13-
borgmatic_checks:
14-
- repository
15-
borgmatic_check_last: 3
16-
borgmatic_store_atime: true
17-
borgmatic_store_ctime: true
18-
borgmatic_relocated_repo_access_is_ok: false
194
borg_one_file_system: true
205
borg_exclude_from: []
216
borg_encryption_passcommand: false
227
borg_lock_wait_time: 5
8+
borg_ssh_key_type: "ed25519"
9+
borg_ssh_key_name: "id_{{ borg_ssh_key_type }}"
10+
borg_ssh_key_file_path: "{{ backup_user_info.home }}/.ssh/{{ borg_ssh_key_name }}"
2311
borg_ssh_command: false
2412
borg_remote_path: false
2513
borg_remote_rate_limit: 0
@@ -28,13 +16,33 @@ borg_retention_policy:
2816
keep_daily: 7
2917
keep_weekly: 4
3018
keep_monthly: 6
31-
ssh_key_file: .ssh/id_ed25519
32-
borgmatic_cron_name: borgmatic
33-
borgmatic_cron_hour: "{{ 6 | random(seed=inventory_hostname) }}"
34-
borgmatic_cron_minute: "{{ 59 | random(seed=inventory_hostname) }}"
35-
borgmatic_cron_checks_day: "{{ range(1, 28) | random(seed=inventory_hostname) }}"
36-
borgmatic_cron_checks_hour: "{{ range(9, 24) | random(seed=inventory_hostname) }}"
37-
borgmatic_cron_checks_minute: "{{ 59 | random(seed=inventory_hostname) }}"
3819
borg_version: false
20+
borgmatic_timer_cron_name: "borgmatic"
21+
borgmatic_timer: cron
22+
borgmatic_timer_hour: "{{ range(0, 5) | random(seed=inventory_hostname) }}"
23+
borgmatic_timer_minute: "{{ range(0, 59) | random(seed=inventory_hostname) }}"
24+
borg_install_method: "pip"
25+
26+
borgmatic_config_name: config.yaml
27+
borgmatic_hooks:
28+
on_error:
29+
- echo "`date` - Error while creating a backup."
30+
before_backup:
31+
- echo "`date` - Starting backup."
32+
after_backup:
33+
- echo "`date` - Finished backup."
34+
borgmatic_checks:
35+
- name: repository
36+
frequency: "4 weeks"
37+
- name: archives
38+
frequency: "6 weeks"
39+
borgmatic_check_last: 3
40+
borgmatic_store_atime: true
41+
borgmatic_store_ctime: true
42+
borgmatic_relocated_repo_access_is_ok: false
3943
borgmatic_version: false
40-
borg_install_method: pip
44+
45+
borg_venv_path: "/opt/borgmatic"
46+
borg_user: "root"
47+
borg_group: "root"
48+
...

0 commit comments

Comments
 (0)