Lost calendar, address, tasks after server crash

[charleslcso]

So happen that our Proxmox LXC containers all shutdown unexpectedly. Bad stuff.

Then realise all users' Radicale data is gone: addresses, calendar events, tasks/reminders.

I'll need to backup Radicale regularly. Also, how should this kind of event be handled?

[pbiering]

non-transient data should never stored inside containers but outside on stable and backup-ed file system...

usually, all files in the collections-root main folder should be backed-up, simple full backup e.g.

tar czf /path/to/radicale-collection-root-backup-$(date +'%Y%m%d-%H%M%S').tar.gz /var/lib/radicale/collections/collection-root/

one can think about excluding the cache directories,

frequency strongly depending on the change rate

also one can use a more sophisticated tools like amanda or duplicity to run incremental backups and able to have a history in case a particular file only need to be restored

[charleslcso]

Hello @pbiering , thanks for your reply.

This seems very interesting and workable for me. Do you have any comment for v3?

[pbiering]

this is related to v3, storage layout is file based and documented here https://github.com/Kozea/Radicale/wiki/Collection-Storage

[charleslcso]

Hi Peter,

When I come and think of it, I had a container shutdown uncleanly.

All user's data is gone. At least the user sees no data at all.

Saving the directory content in another stable device shouldn't help in this case, I believe, because after the reboot:

1. all files are corrupted in the container or
2. all files vanish because radicale didn't stop properly

Stable drive/devices shouldn't have any benefit in both situations.

Please correct me if I'm wrong.

[pbiering]

this strongly depends how far the (hopefully journaling) file system was abstracted.

per se a container has no persistent file system at all internally, only in-memory

if a container requires persistent file system, this has to be mounted from outside into the container

if the "outside" crashes itself it's strongly depending of which kind it was...

• a virtual machine based on a disk image
  or
• a physical system having disks (RAID)

-> as long as the (virtual) disk is not damaged, the file system should be able to be recover

And radicale is not vanishing files on stop, can it be that you run radicale in a container without persistent storage, e.g. using only tempfs in memory instead of a persistent file system mounted into the container (from the host or via NFS, iSCSI or something else)?

[charleslcso]

Hi @pbiering

The container running radicale is using ext4 fs, from host. Yes, there are tmpfs in the container, but they are not used by radicale.

Using the default radicale parameters, collections are stored in /root/.var/lib/radicale/collections/collection-root//, which means /root is not in tmpfs.

[pbiering]

radicale is not deleting files on startup and ext4 filesystem usually has a journal -> the files should be even after a container or host crash visible on the host file system, except some cleanup magic by used infrastructure either on crash/shutdown or (re)start wipes such mapped directory. Backup should be also done on host level and pushed to or pulled from another host.