Skip to content

Conversation

@BenjaminPelletier
Copy link
Member

This PR adds a few tools to help clients of backup_manager, and in particular, a cloud backup server I'm spinning up. Suggestions for better ways to do things welcome :)

@hcooper
Copy link
Contributor

hcooper commented May 2, 2025

  • I think this might work better as its own repo. The server doesn't need this code, and a backup client doesn't need all the server code.

  • I much prefer using arguments or envvars to set the options.

  • It might suit a lightweight docker container, that way dependencies, crontabs, logs, etc can all be setup without dirtying the underlying host. If not, I suggest a systemd timer unit rather than cron job (they can stopped, started, logs collected etc).

  • Add set -euo pipefail in bash scripts to make it actually fail on errors you'd expect it to.

Just for reference, in case you haven't seen them, this is how I've been collecting & pruning backups for a while now:

Client

#!/bin/bash
set -euo pipefail
# backup container changes key every restart, so StrictHostKeyChecking=no
SSH="ssh -o StrictHostKeyChecking=no -i ~admin/.ssh/id_hcooper_ropewiki_ed25519_230509 -p 22001"
DST="/volume3/b2_backblaze_backups/ropewiki"
rsync --partial --progress -e "$SSH" -avz backupreader@app.ropewiki.com:~/backups/* $DST/db/
$SSH backupreader@app.ropewiki.com "touch ~/backups/last_offsite_copy_db_coops"
rsync --partial --progress -avz -L -e "$SSH" --exclude 'BALLAST_DELETE_IF_OUT_OF_SPACE' --exclude 'lost+found' backupreader@app.ropewiki.com:~/images/* $DST/images/
$SSH backupreader@app.ropewiki.com "touch ~/backups/last_offsite_copy_images_coops"

Server (/rw/mount/sqlbackup/prune_backups.sh)

#!/bin/bash
# This looks at the modification date of `last_offsite_copy_db_coops`, and deletes any backups X days older than that.
# `last_offsite_copy_db_coops` is updated by Coops' offsite rsync script.

find /rw/mount/sqlbackup -type f -name "*.sql.zst" ! -newermt "@$(($(stat -c %Y /rw/mount/sqlbackup/last_offsite_copy_db_coops) - (3 * 24 * 60 * 60) ))" -exec rm {} \;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants