Borg backup configuration

You don't need this you the backups will be place on a location you have direct access. But to place the backups remotely like rsync.net you need to create a passwordless pair of key so that you can automate the backup.

ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_ed25519 -C "$(hostname)"

If using rsync.net check this page on how to copy the public key.

Install borg and borgmatic

Borg

To install the most recente version of Borg its better to download the binary available on github. Check the current version on github

wget -q --show-progress https://github.com/borgbackup/borg/releases/download/<current version>/borg-linux64
sudo mv borg-linux64 /usr/local/bin/borg
sudo chown root:root /usr/local/bin/borg
sudo chmod 755 /usr/local/bin/borg

Borgmatic

If needed, Install pip3

apt install python3-pip

Then install borgmatic

sudo pip3 install --user --upgrade borgmatic

This will install borgmatic executables in /root/.local/bin/ so you need to add to the PATH. Edit /root/.bashrc file and add export PATH=$PATH:/root/.local/bin at the end of the file

Apply the changes by running source /root/.bashrc or logout and login with root.

Run borgmatic --version to check if it's running OK.

Configuration

To create the configuration file for borgmatic, execute generate-borgmatic-config This will create th file /etc/borgmatic/config.yaml

This file has all the configuration options with comments. Since the file is big, just make a backup of it and add the following configuration Change as necessary, remove the comments and copy to /etc/borgmatic/config.yaml

cp /etc/borgmatic/config.yaml /etc/borgmatic/config.yaml.back

location:
    # add the paths you want to backup
    source_directories:
        - /etc
        - /other/paths

    # Check bitwarden for the user name of rsync.net and replace <repo_name> with the name of the repo
    repositories:
        - user@user.rsync.net:borg/linode/<repo_name>

    # add path the you want to exclude from the backup
    exclude_patterns:
        - /var/www/*/storage/app/backup-temp/*
        - /var/www/*/storage/app/temp-files/*
        - /var/www/html

    remote_path: borg12

storage:
    encryption_passcommand: cat /root/.borg-passphrase
    compression: zstd

retention:
    keep_daily: 30
    keep_weekly: 12
    keep_monthly: 6

consistency:
    checks:
        - repository
        - archives
    check_last: 3

output:
    color: false

hooks:
    # if needed add the mysql databases to backup
    # you can leave all to backup all databases ou replace with the database name
    # set the mysql user and password
    mysql_databases:
        - name: all
          username: <user>
          password: <password>
          options: --no-tablespaces --single-transaction

    # check bitwarden of pagerduty integration key 
    # and healthchecks.io url (need one for each backup configuration)
    healthchecks: https://hc-ping.com/4d0d73f2-1eda-4cb6-aa0c-163e0cbbe06f
    pagerduty: <pager duty integration key>

Create a random password using bitwarden, store it in /root/.borg-passphrase file and protect it:

chmod 400 /root/.borg-passphrase

Create BORG_PASSPHRASE environment variable so you don't have to type the password everytime, when configuring the backups. This will not be needed for borgmatic since we have configured the encryption_passcommand option to read the password from the file /root/.borg-passphrase

export BORG_PASSPHRASE=$(cat /root/.borg-passphrase)

Next we need to create a borg repository for ours backups.

borg init --encryption=repokey-blake2 zh1637@zh1637.rsync.net:borg/linode/<repository_name>

Check the repo by running

borg info zh1637@zh1637.rsync.net:borg/linode/<repository_name>

Finally create a folder to store the logs

mkdir -p /backups/borg/logs

Running the backup

Crontab configuration

• Change the run time to a more suitable one for the server
• Be sure to check borgmatic and borg paths.
• Configure healthchecks.io URL

0 1 * * * PATH=$PATH:/usr/bin:/usr/local/bin /root/.local/bin/borgmatic --verbosity 1 --stats --files > /backups/borg/logs/`date +\%Y\%m\%d_\%H\%M\%S`-`hostname`.log 2>&1 && curl -fsS -m 10 --retry 5 -o /dev/null https://hc-ping.com/<replace_with_check_id>

For the first time, it's recomended that you run borgmatic manually so that you can catch any error or problem.

Running backup manually

borgmatic  --verbosity 1  --stats --files > /backups/borg/logs/`date +\%Y\%m\%d_\%H\%M\%S`-`hostname`.log 2>&1

You can also simulate a problem to see if pagerduty is configured correctly, just change the name of the repo in /etc/borgmatic/config.yaml file to something that doesen't exists, and run the backup command again.

You can check if the archive was create by running

 borg list zh1637@zh1637.rsync.net:borg/linode/<repo_name>