Tobias Knöppler - Blog

Awesome Backups 2 - A solid backup process

2025-09-27T00:00:00+00:00

Our backup approach</h1>
In the previous post</a>, I presented my criteria for good backups and outlined some implementation approaches. In this post I'll provide a step-by-step description for setting up a backup process for a Nextcloud Server that will fulfill all those criteria.</p>
Feel free to chose and pick and adjust my approach to your own system.</p>
You can also skip directly to the final code</a> if you're fluent in bash and don't care for the reasoning behind the individual steps.</p>

Step 1: reliable zero downtime snapshots</h2>
A typical Nextcloud backup will likely look like this:</p>

Turn maintenance mode on</li>
dump/backup the database</li>
copy the user files and the nextcloud installation directory to the backup target location</li>
Turn maintenance mode back off</li> </ol>
Enabling maintenance mode serves to ensure, that all the things we are including in our backup are in sync and thus represent a consistent state of our service at a specific point in time. Otherwise, we might run into issues when restoring the backup - for instance files being present in the filesystem but not known to the database (and thus not visible to Nextcloud).</p>
Unfortunately, we have ruled out using maintenance mode, because we don't want any downtime during our backup process, so we need another solution.</p> </sl-details>
To accomplish consistent zero downtime backups, we will create synchronized snapshots</span> </sl-tooltip> of all the things that need to be included in a typical Nextcloud backup: the users' files, the installation directory and the database. For this purpose, we will rely on the snapshot feature of our filesystem and on so-called single-transaction copies for our database. This means, that we must choose a database and filesystems for our server, to include those features.</p>
Popular modern filesystem types like BTRFS, ZFS, XFS, LVM and many VM storage solutions provide snapshot capabilities. Filesystem snapshots are read-only copies of a directory or volume that are usually created nearly instantly and take little to no additional storage space. This guide will give examples for BTRFS and LVM (more might be added in the future).</p>
Most modern transactional database should provide options to do single-transaction copies of the database. I will give examples on how to achieve this for mariadb/mysql and postgresql.</p>
Is any of this applicable to containerized deployments (e.g. Nextcloud All-in-One which is based on Docker)?</p>
Yes, you just need to make sure that the volumes which contain your Nextcloud installation directory and your user files are backed by compatible storage - e.g. mounting a btrfs subvolume under the respective paths.</p>
You will also have to prepend some of the commands in this guide with whatever is necessary to run a command in you container. For example for docker, you would need: `docker exec <name-of-your-container> <original-command></code>. Other commands, especially filesystem operations, always need to be executed on the host system.</p> </sl-details>`
`Taking filesystem snapshots</h3> Let's walk us through the process of creating filesystem snapshots for our user files directory and Nextcloud installation directory.</p> BTRFS</sl-tab> LVM</sl-tab> This requires the nextcloud installation directory (assumed to be at /var/www/nextcloud</code>) or respectively the user files directory (assumed to be at /mnt/data</code>) to be a BTRFS subvolume (created with btrfs subvolume create <path></code>).</p>`

# Path to the btrfs subvolume containing your NC files directory </span>USER_FILES_VOLUME</span>=</span>/mnt/data/nc_files </span># Path where the (temporary) btrfs snapshot for the NC files directory should be created </span>USER_FILES_SNAPSHOT_PATH</span>="</span>/mnt/data/.nc_files_snapshot</span>" </span> </span># Path to the btrfs subvolume containing your NC installation directory </span>NC_INSTALL_VOLUME</span>=</span>/var/www/nextcloud </span># Path where the (temporary) btrfs snapshot for the NC installation directory should be created </span>NC_INSTALL_SNAPSHOT_PATH</span>="</span>/var/www/.nc_snapshot</span>" </span></code></pre>
</sl-icon> </sl-button>
btrfs subvolume snapshot </span>\ </span> </span>"$</span>USER_FILES_VOLUME</span>" "$</span>USER_FILES_SNAPSHOT_PATH</span>" </span></code></pre> </div> </sl-icon> </sl-button> btrfs subvolume snapshot </span>\ </span> </span>"$</span>NC_INSTALL_VOLUME</span>" "$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> This requires the nextcloud installation directory or respectively the user files directory to be a LVM logical volume.</p> # Adjust to match the LVM volume group containing your nc installation directory </span>USER_FILES_VG</span>="</span>lvm-1</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span>USER_FILES_LV</span>="</span>nc-files</span>" </span># Arbitrary name for the (temporary) lvm snapshot </span>USER_FILES_SNAPSHOT_NAME</span>="</span>nc-backup-files</span>" </span> </span># Adjust to match the LVM volume group containing your nc installation directory </span>NC_INSTALL_VG</span>="</span>lvm-1</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span>NC_INSTALL_VOLUME</span>="</span>nextcloud</span>" </span># Arbitrary name for the (temporary) lvm snapshot </span>NC_INSTALL_SNAPSHOT_NAME</span>="</span>nc-backup-nextcloud</span>" </span></code></pre> </sl-icon> </sl-button> lvcreate</span> -</span>s </span>\ </span> --</span>name </span>"${</span>USER_FILES_SNAPSHOT_NAME</span>}" \ </span> </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_LV</span>}" </span></code></pre> </div> </sl-icon> </sl-button> lvcreate</span> -</span>s </span>\ </span> --</span>name </span>"$</span>NC_INSTALL_SNAPSHOT_NAME</span>" \ </span> </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_VOLUME</span>}" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-tab-group>Taking a database snapshot</h3> For creating our database snapshot, we will keep things simple and use the trusty sql dump utilities available for our reference databases mariadb and postgres. An sql dump is basically a set of SQL instructions that can be used to recreate the database in its current state.</p> Following you will find code snippets for creating those dumps:</p> PostgreSQL</sl-tab> MariaDB / MySQL</sl-tab> SQL_DUMP_TARGET_DIR</span>=</span>/tmp/nc_backup/ </span> </span>DB_USER</span>="</span>root</span>" </span>DB_NAME</span>="</span>nextcloud</span>" </span>DB_PASSWORD</span>="</span>password</span>" </span>DB_HOST</span>="</span>localhost</span>" </span>DB_PORT</span>=</span>5432 </span></code></pre> </sl-icon> </sl-button> # pg_dump will be supplied with the connection options by writing them to ~/.pgpass </span>install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "$</span>HOME</span>/.pgpass</span>" </span>pg_dump </span>> "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> SQL_DUMP_TARGET_DIR</span>=</span>/tmp/nc_backup/ </span> </span>DB_USER</span>="</span>root</span>" </span>DB_NAME</span>="</span>nextcloud</span>" </span>DB_PASSWORD</span>="</span>password</span>" </span>DB_HOST</span>="</span>localhost</span>" </span>DB_PORT</span>=</span>5432 </span></code></pre> </sl-icon> </sl-button> # mysqldump will be supplied with the db credentials by writing them to ~/.my.cnf </span>install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "$</span>HOME</span>/.my.cnf</span>" </span>mysqldump </span>\ </span> --</span>host</span>="${</span>DB_HOST</span>}" \ </span> --</span>port</span>="${</span>DB_PORT</span>}" \ </span> --</span>single-transaction </span>\ </span> --</span>skip-lock-tables </span>\ </span> </span>"${</span>DB_NAME</span>}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></code></pre> </div> </div> </div> Note:</strong> We need the parameter --single-transaction</code> to ensure a "real" snapshot (a single transaction copy to be precise) that is unaffected by ongoing writes to the DB. We add --skip-lock-tables</code> as well, to improve performance (not locking tables is okay here, because we're just reading from the DB).</p> </sl-tab-panel> </sl-tab-group> Now that we can create both, filesystem snapshots and an sql dump, we are capable of creating a consistent point-in-time snapshot of our Nextcloud instance with zero downtime (well, actually 3 synchronized snapshots, but we'll get to that). Just make sure, to either trigger all three operations at the same time or perform the filesystem snapshots first (since they are super fast, that's basically the same thing). It's important that all 3 snapshots refer to the same point in time.</p> Merging the snapshots into one directory</h3> Right now, we have 3 individual snapshots lying around. They might even be located on separate disks. For the next steps we want one aggregated view on them, which we will achieve using bind mounts</strong> for btrfs (or other filesystems) and normal mounts for LVM. Bind mounts are a Linux feature which allows us to mount one directory to another path in our filesystem.</p> So, using the same variables as above, the code for this would look like follows:</p> BTRFS</sl-tab> LVM</sl-tab> </sl-icon> </sl-button> mount</span> --</span>bind</span> -</span>o</span> ro </span>\ </span> </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" \ </span> </span>"$</span>SQL_DUMP_TARGET_DIR</span>/nextcloud</span>" </span></code></pre> </div> </sl-icon> </sl-button> mount</span> --</span>bind</span> -</span>o</span> ro </span>\ </span> </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" \ </span> </span>"$</span>SQL_DUMP_TARGET_DIR</span>/nc_files</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-icon> </sl-button> # Activate the LVM snapshot so we can mount it </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span># Mount the snapshot read-only inside $SQL_DUNMP_TARGET_DIR with ownership forced to the user that owns Nextcloud runs as </span>mount</span> -</span>o </span>"</span>ro,uid=33,gid=33</span>" \ </span> </span>"</span>/dev/</span>${</span>USER_FIELS_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" \ </span> </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nextcloud</span>" </span></code></pre> </div> </sl-icon> </sl-button> # Activate the LVM snapshot so we can mount it </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span># Mount the snapshot read-only inside $SQL_DUNMP_TARGET_DIR with ownership forced to the user that owns Nextcloud runs as </span>mount</span> -</span>o </span>"</span>ro,uid=33,gid=33</span>" \ </span> </span>"</span>/dev/</span>${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" \ </span> </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nextcloud</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-tab-group> The result is a directory (/tmp/nc_backup</code>) which looks like it contains all filesystem snapshots and the sql dump:</p> /tmp/nc_backup </span>├── nextcloud # nextcloud installation directory </span>├── nc_files # user files directory </span>└── db.sql # sql dump of the database </span></code></pre> Step 2: Providing Backup Storage</h2> Choosing our remote storage location</h3> As we have established, we need our backups to not be affected by a number of local hazards, including hardware failure or fires. Therefore, we need to store them somewhere else. Because we also want ransomware protection, we need a storage type with one of two options:</p> Either</p> object locking (preferred) or</li> restricted access permissions, that allow creating but not modifying/deleting backups</li> </ul> These features (which I will explain in more detail later) will allow us to prevent attackers (e.g. ransomware) from destroying our backups.</p> In this guide, we will focus on object locking to ensure ransomware protection. If you want to use restricted access permissions, please refer to the Kopia documentation</a>.</p> Because we need those features, we can't go with many of the storage options offered by Kopia and are left with something called "object storage" - basically a special kind of storage often found in cloud computing. Apart from big cloud providers, this kind of storage is so ubiquitous today, that you will find it offered by numerous providers and you can even self-host it. Because we will encrypt the backups you don't need to have a high level of trust in your provider, though (apart from trust in their capability of not losing</em> your data, of course). Here are some options:</p> Google Cloud Storage</li> Azure Cloud Storage</li> Amazon AWS S3 S3 compatible</sl-badge></li> Backblaze B2 S3 compatible</sl-badge> includcode,kbding object locks and separate access permissions</li> quite cheap compared to competitors</li> </ul> </li> Hetzner S3 Storage S3 compatible</sl-badge> Hosting in EU (Germany)</li> </ul> </li> Min-IO S3 compatible</sl-badge> Self-Hosting option</li> No official</em> UI for admin operations (managing users, access keys, buckets, etc.)</li> licensed under AGPL</li> </ul> </li> Ceph with RadosGW S3 compatible</sl-badge> Self-Hosting option</li> licensed under LGPL 2.1 and 3.0</li> </ul> </li> </ul> Note: All options marked with S3 compatible</sl-badge> will share the same code snippets under the name "S3 compatible" going forward.</p> Setting up one of the self-hosting options is out of scope for this guide, however, I will provide instructions for AWS S3, Google Cloud Storage and Min-IO going forward (maybe I'll add Ceph at a later point).</p> For reference though: My personal storage server is formed by a RaspberryPi 5 8GB running Min-IO and coupled with a JBOD</span> </sl-tooltip> system that contains 4 8TB disks.</p> Configuring a bucket</h3> So now, that you have chosen (or become!) a storage provider, you need to create a bucket (which is comparable to the concept of directories in traditional file systems, although they can't be nested). If you need object locking than you need to make sure it is supported by your provider and enabled for your bucket.</p> Now you need to create credentials for your bucket. Depending on your storage provider that can look differently.</p> S3 compatible</sl-tab> Google Cloud Storage</sl-tab> For S3 compatible providers you need to create user and a policy and potentially an access key.</p> Assuming your bucket is named kopia</code>, here is what a policy could look like:</p> </sl-icon> </sl-button> { </span> </span>"</span>Version</span>": "</span>2012-10-17</span>", </span> </span>"</span>Statement</span>": [ </span> </span>{ </span> </span>"</span>Effect</span>": "</span>Allow</span>", </span> </span>"</span>Action</span>": [ </span> </span>"</span>s3:</span>" </span> </span>], </span> </span>"</span>Resource</span>": [ </span> </span>"</span>arn:aws:s3:::kopia</span>", </span> </span>"</span>arn:aws:s3:::kopia/</span>" </span> </span>] </span> </span>}, </span> </span>{ </span> </span>"</span>Effect</span>": "</span>Deny</span>", </span> </span>"</span>Action</span>": [ </span> </span>"</span>s3:DeleteBucket</span>", </span> </span>"</span>s3:DeleteBucketPolicy</span>" </span> </span>], </span> </span>"</span>Resource</span>": [ </span> </span>"</span>arn:aws:s3:::kopia</span>", </span> </span>"</span>arn:aws:s3:::kopia/</span>" </span> </span>] </span> </span>} </span> </span>] </span>} </span></code></pre> To apply (using the Min-IO cli (mcli)</a>):</p> Save the policy as kopiawriter.json</code></li> Edit it to match your Abucket name</li> Now execute the following commands:</li> </ol> </sl-icon> </sl-button> # Configure your S3 storage </span>mc alias set backupserver </span><</span>HOSTNAME</span>> <</span>ACCESS_KEY</span>> <</span>SECRET_KEY</span>> </span></code></pre> </div> </sl-icon> </sl-button> # Create a new user </span>mc admin user add backupserver kopia </span><</span>kopia-login-secret</span>> </span></code></pre> </div> </sl-icon> </sl-button> # Add the policy to your S3 server </span>mcli admin policy add backupserver kopiawriter kopiawriter.json </span></code></pre> </div> </sl-icon> </sl-button> # Attach the policy to the kopia user </span>mcli admin policy set backupserver kopiawriter user=kopia </span></code></pre> </div> </div> </div> Now you can access the bucket with accesskey=kopia</code> and secretaccesskey=<kopia-login-secret></code>.</p> </div> </sl-tab-panel> If you want to use Google Cloud Storage, I recommend setting up a service account</a> that you will then grant access</a> to your google storage bucket. Finally, you will need to create a service account key</a> for your service account that you will later use to connect the storage.</p> For this guide, I won't go into more detail on those steps. Please follow the linked official documentation instead.</p> </sl-tab-panel> </sl-tab-group> Step 3: Connecting our Storage</h2> By now, we have a snapshot of our instance and we have some remote storage for our backups. Now let's address getting our backup there.</p> Introducing Kopia</h3> To turn our instance snapshot into a backup that ticks all the boxes from our backup criteria, we will need a specialized backup tool. There are a number of options available, notably Restic, Borg and Kopia, which should all cover most of our needs. Because I have the most experience with it and use it for my own backups, we're going with Kopia in this guide (feel free to supply information on how to do the same with another tool and I'll consider adding it).</p> Why do we need a specialized tool?</strong> While we could implement some of its features ourselves, this would be more error-prone and very complex to achieve. Some features, like error correction and ransomware protection are especially infeasible to solve by hand.</p> For which features do we use Kopia?</strong> Notable features of Kopia that we rely on are:</p> Deduplication</li> Compression</li> Error correction</li> Support for many remote storage options</li> Repository synchronization</li> Backup validation</li> Object locking (for ransomware protection)</li> Retention and automatic cleanup of backups</li> </ul> </sl-details> Before proceeding, please make sure you have Kopia installed on your server as per the installation instructions</a>.</p> Writing the backup</h3> Now, that we have our backup tool ready, we're going take our instance snapshot and write it to our object storage.</p> Let's first setup a Kopia Repository</span> </sl-tooltip>.</p> S3 compatible</sl-tab> Google CLoud Storage</sl-tab> Use the following commands to initialize a Kopia repository. Adjust the variables ENDPOINT</code>,BUCKET_NAME</code>, ACCESS_KEY</code>, SECRET_ACCESS_KEY</code>, PREFIX</code> according to your S3 storage.</p> For all available paremeters, refer to the kopia documentation</a>.</p> </sl-icon> </sl-button> ENDPOINT</span>="</span>s3.amazonaws.com</span>" </span># The endpoint where to find your S3 compatible server (including port, excluding protocol) </span>BUCKET_NAME</span>="</span>my-backup-bucket</span>" </span># The name of your S3 bucket </span>ACCESS_KEY</span>="</span>my-s3-user</span>" </span># Your access key could be your user name or an ID attached to a specific credential (depending on your provider and configuration) </span>SECRET_ACCESS_KEY</span>="</span>my-secret-s3-password</span>" </span># The secret part of your access key - could be your user password or a secret attached to a specific credential (depending on your provider and configuration </span>PREFIX</span>="</span>/</span>" </span># Path prefix for all objects written by Kopia </span></code></pre> </div> </sl-icon> </sl-button> kopia repository create s3 </span>\ </span> --</span>endpoint </span>"$</span>ENDPOINT</span>" \ </span> --</span>bucket </span>"$</span>BUCKET_NAME</span>" \ </span> --</span>access-key </span>"$</span>ACCESS_KEY</span>" \ </span> --</span>secret-access-key </span>"$</span>SECRET_ACCESS_KEY</span>" \ </span> --</span>prefix </span>"$</span>PREFIX</span>" \ </span></code></pre> </div> </div> </div> </sl-tab-panel> Use the following commands to initialize a Kopia repository. Adjust the variables BUCKET_NAME</code>, SERVICE_ACCOUNT_CREDENTIALS_FILES</code>, PREFIX</code> according to your setup.</p> For all available paremeters, refer to the kopia documentation</a>.</p> </sl-icon> </sl-button> BUCKET_NAME</span>="</span>my-backup-bucket</span>" </span># The name of your Google bucket </span>SERVICE_ACCOUNT_CREDENTIALS_FILE</span>="</span>/path/to/credentials.json</span>" </span># You can skip the --credentials-file parameter to use your default google credentials </span>PREFIX</span>="</span>/</span>" </span># Path prefix for all objects written by Kopia </span></code></pre> </div> </sl-icon> </sl-button> kopia repository create gcs </span>\ </span> --</span>bucket </span>"$</span>BUCKET_NAME</span>" \ </span> --</span>credentials-file </span>"${</span>SERVICE_ACCOUNT_CREDENTIALS_FILE</span>}" \ </span> --</span>prefix </span>"$</span>PREFIX</span>" \ </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-tab-group> Now we need to actually enable the use of object locks in Kopia:</p> </sl-icon> </sl-button> kopia maintenance set</span> --</span>extend-object-locks</span> true </span> </span></code></pre> </div> And adjust our Kopia global policy (we could also do this per repository, but since we only have one, this is fine). Here we setup how many backups to retain and the type of compression to use. You can find the full list of parameters in the Kopia documentation</a>.</p> </sl-icon> </sl-button> kopia policy set </span>\ </span> --</span>compression</span> zstd </span>\ </span> --</span>compression-min-size</span> 100K </span>\ </span> --</span>keep-annual</span> 2 </span>\ </span> --</span>keep-monthly</span> 24 </span>\ </span> --</span>keep-weekly</span> 8 </span>\ </span> --</span>keep-daily</span> 14 </span>\ </span> --</span>keep-hourly</span> 48 </span></code></pre> </div> All set! We're finally ready to actually transfer the snapshot to the backup storage:</p> </sl-icon> </sl-button> kopia snapshot create </span>\ </span> --</span>parallel</span>=</span>4 </span>\ </span> --</span>tags </span>"</span>job:nc-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" \ </span> </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" </span></code></pre> </div> Kopia does a lot of things for us. Apart from the things that we have configured explicitly (like retention, encryption, compression) we get very efficient deduplication, meaning we only need space for data that is actually new. That's also, why we don't compress our database dump - because that way we can make use of the more efficient compression and deduplication Kopia provides.</p> One particularly nice thing about Kopia backups is, that the progress of a partially transferred backup that got aborted due to e.g. network issues will be picked up during the next try, because Kopia still stores all the data blocks that were transferred and never transfers the same data block again (because of deduplication). In other words: If we need 10 attempts to create our initial backup, because we have a large amount of data, we will still make progress during every attempt and end up with a complete backup at the end.</p> </sl-details>Step 4: Cleanup</h2> Lastly, we can cleanup the snapshots, we don't need it anymore. Here's a script snippet to do that depending on the filesystem used:</p> BTRFS</sl-tab> LVM</sl-tab> </sl-icon> </sl-button> # Unmount our --bind mounts </span>umount </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nextcloud</span>" </span>umount </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nc_files</span>" </span> </span># Remove the snapshot directory (including the db.sql) </span>rm</span> -</span>rf </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" </span></code></pre> </div> </sl-icon> </sl-button> # Remove the filesystem snapshots </span>btrfs subvolume delete </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" </span>btrfs subvolume delete </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-icon> </sl-button> # Unmount our snapshot mounts </span>umount </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nextcloud</span>" </span>umount </span>"${</span>SQL_DUMP_TARGET_DIR</span>}</span>/nc_files</span>" </span> </span># Remove the snapshot directory (including the db.sql) </span>rm</span> -</span>rf </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" </span></code></pre> </div> </sl-icon> </sl-button> # Remove the filesystem snapshots </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_INSTALL_VOLGROUP</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT</span>}" </span>lvremove </span>"</span>/dev/</span>${</span>NC_INSTALL_VOLGROUP</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT</span>}" </span> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_FILES_VOLGROUP</span>}</span>/</span>${</span>NC_FILES_SNAPSHOT</span>}" </span>lvremove </span>"</span>/dev/</span>${</span>NC_FILES_VOLGROUP</span>}</span>/</span>${</span>NC_FILES_SNAPSHOT</span>}" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-tab-group>Putting it all together</h1> Now that we know how to create a backup, let's automate the process and run it on a schedule. For your convenience, here is a script, that you can drop in /etc/cron.hourly/</code> or use in a systemd service unit with corresponding systemd timer</a>.</p> Setup Kopia Repository</h2> If you haven't done it already, execute the following snippet (as root) to configure your Kopia Repository. Select the type of remote repository that matches your setup.</p> S3 compatible</sl-tab> Google Cloud Storage</sl-tab> After configuring the options to match your backup storage setup, run this script snippet as root to setup your Kopia repository. </p> </sl-input> </sl-input> </sl-input> </form> </sl-icon> </sl-button> # You will be asked for your access key and access key secret. </span># Your access key could be your user name or an ID attached to a specific credential (depending on your provider and configuration) </span># The secret part of your access key - could be your user password or a secret attached to a specific credential (depending on your provider and configuration </span>read -rsp </span>'</span>Enter S3 Access Key ID: </span>'</span> ACCESS_KEY </span>\ </span> </span>&& </span>read -rsp </span>$'\n''</span>Enter S3 Secret Access Key: </span>'</span> SECRET_ACCESS_KEY </span>\ </span> </span>&& </span>kopia repository create s3 </span>\ </span> --</span>endpoint </span>"</span>[[ENDPOINT]]</span>" \ </span> --</span>bucket </span>"</span>[[BUCKET_NAME]]</span>" \ </span> --</span>access-key </span>"${</span>ACCESS_KEY</span>?}" \ </span> --</span>secret-access-key </span>"${</span>SECRET_ACCESS_KEY</span>?}" \ </span> --</span>prefix </span>"</span>[[PREFIX]]</span>" \ </span> --</span>retention-mode</span> governance </span>\ </span> </span>&& </span>kopia maintenance set</span> --</span>extend-object-locks</span> true </span></code></pre> </div> </div> </context-condition> </parameterized-script> </sl-card> </sl-tab-panel> After configuring the options to match your backup storage setup, run this script snippet as root to setup your Kopia repository. </p> Google Default Credentials</sl-option> Google Service Account key file</sl-option> </sl-select> </sl-input> </context-condition> </sl-input> </sl-input> </form> </sl-icon> </sl-button> kopia repository create gcs </span>\ </span> --</span>bucket </span>"</span>[[BUCKET_NAME]]</span>" \ </span> --</span>prefix </span>"</span>[[PREFIX]]</span>" \ </span> --</span>retention-mode</span> governance </span>\ </span> </span>&& </span>kopia maintenance set</span> --</span>extend-object-locks</span> true </span></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> kopia repository create gcs </span>\ </span> --</span>bucket </span>"</span>[[BUCKET_NAME]]</span>" \ </span> --</span>credentials </span>"</span>[[GOOGLE_CREDENTIALS_FILE]]</span>" \ </span> --</span>prefix </span>"</span>[[PREFIX]]</span>" \ </span> --</span>retention-mode</span> governance </span>\ </span> </span>&& </span>kopia maintenance set</span> --</span>extend-object-locks</span> true </span></code></pre> </div> </div> </context-condition> </parameterized-script> </sl-card> </sl-tab-panel> </sl-tab-group>Setup the backup script</h2> Backup Script</h3> Following, configure the options to match your setup and then copy or download the backup script. Afterward, adjust the configuration section at the top of the script (mostly paths to your Nextcloud setup) and run it as root. The backup script will use your default Kopia repository (which you have initialized in the previous step).</p> Note: Make sure to save the script with permissions</strong> 0700</code> or</strong> 0500</code> to avoid other users from reading the credentials inside! (Or move them outside the script, e.g. inside systemd credentials</a>).</strong></p> S3 or S3 compatible</sl-option> Google cloud Storage</sl-option> </sl-select> BTRFS</sl-option> LVM</sl-option> </sl-select> BTRFS</sl-option> LVM</sl-option> </sl-select> MariaDB</sl-option> PostgreSQL</sl-option> </sl-select> </form> </hl> nextcloud-backup.sh Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> Download </sl-button> </context-condition> </div> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> # Path to the btrfs subvolume containing your NC installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>/var/www/nextcloud</span>" </span></td></tr> 19</td> # Path where the (temporary) btrfs snapshot for the NC installation directory should be created </span></td></tr> 20</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/var/www/.nc_snapshot</span>" </span></td></tr> 21</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 22</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 23</td> </span></td></tr> 24</td> ## Nextcloud files directory config </span></td></tr> 25</td> # Path to the btrfs subvolume containing your NC files directory </span></td></tr> 26</td> USER_FILES_VOLUME</span>="</span>/mnt/data/ncdata</span>" </span></td></tr> 27</td> # Path where the (temporary) btrfs snapshot for the NC files directory should be created </span></td></tr> 28</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/mnt/data/.nc_files_snapshot</span>" </span></td></tr> 29</td> # DB dump config </span></td></tr> 30</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 31</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 32</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 33</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 34</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 35</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 36</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 37</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 38</td> </span></td></tr> 39</td> # Configure the connection options for your database: </span></td></tr> 40</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 41</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 42</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 43</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 44</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 45</td> </span></td></tr> 46</td> # mysqldump will be supplied with the db credentials by writing them to ~/.my.cnf </span></td></tr> 47</td> MYCNF_PATH</span>="$</span>HOME</span>/.my.cnf</span>" </span></td></tr> 48</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "${</span>MYCNF_PATH</span>}" </span></td></tr> 49</td> SQLDUMP_CMD</span>=(</span>mysqldump --host=</span>"${</span>DB_HOST</span>}"</span> --port=</span>"${</span>DB_PORT</span>}"</span> --single-transaction --skip-lock-tables </span>"${</span>DB_NAME</span>}") </span></td></tr> 50</td> </span></td></tr> 51</td> </span></td></tr> 52</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 53</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 54</td> </span></td></tr> 55</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 56</td> NC_INSTALL_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 57</td> </span></td></tr> 58</td> USER_FILES_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 59</td> </span></td></tr> 60</td> </span></td></tr> 61</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 62</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 63</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 64</td> </span></td></tr> 65</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 66</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 67</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 68</td> </span></td></tr> 69</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 70</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 71</td> </span></td></tr> 72</td> </span></td></tr> 73</td> ### You usually should not need to touch the following variables </span></td></tr> 74</td> </span></td></tr> 75</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 76</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 77</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 78</td> </span></td></tr> 79</td> # file for keeping track of running backup operations </span></td></tr> 80</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 81</td> </span></td></tr> 82</td> </span></td></tr> 83</td> # Utility function for logging to both the log file and stdout </span></td></tr> 84</td> log</span>() { </span></td></tr> 85</td> </span>local </span>msg message </span></td></tr> 86</td> </span></td></tr> 87</td> fwd</span>=(</span>echo</span>) </span></td></tr> 88</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 89</td> </span>then </span></td></tr> 90</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 91</td> </span>fi </span></td></tr> 92</td> </span></td></tr> 93</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 94</td> </span>then </span></td></tr> 95</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 96</td> </span>do </span></td></tr> 97</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 98</td> </span>done </span></td></tr> 99</td> </span>return </span></td></tr> 100</td> </span>fi </span></td></tr> 101</td> </span></td></tr> 102</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 103</td> } </span></td></tr> 104</td> </span></td></tr> 105</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 106</td> cleanup</span>() { </span></td></tr> 107</td> </span>set </span>+</span>eu </span></td></tr> 108</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 109</td> </span></td></tr> 110</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 111</td> </span></td></tr> 112</td> </span></td></tr> 113</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 114</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 116</td> </span></td></tr> 117</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 119</td> </span></td></tr> 120</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 121</td> </span># Cleanup NC install dir snapshot </span></td></tr> 122</td> </span></td></tr> 123</td> </span>btrfs subvolume delete </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></td></tr> 124</td> </span></td></tr> 125</td> </span># Cleanup NC files snapshot </span></td></tr> 126</td> </span></td></tr> 127</td> </span>btrfs subvolume delete </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" </span></td></tr> 128</td> </span></td></tr> 129</td> </span></td></tr> 130</td> </span># Remove PID file </span></td></tr> 131</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 132</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 133</td> } </span></td></tr> 134</td> </span></td></tr> 135</td> </span></td></tr> 136</td> </span></td></tr> 137</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 138</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 139</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 140</td> then </span></td></tr> 141</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 142</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span>* </span>3600</span>)) </span>]] </span></td></tr> 143</td> </span>then </span></td></tr> 144</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 145</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 146</td> </span>else </span></td></tr> 147</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 148</td> </span>exit 0 </span></td></tr> 149</td> </span>fi </span></td></tr> 150</td> fi </span></td></tr> 151</td> </span></td></tr> 152</td> # cleanup leftovers from last run if any </span></td></tr> 153</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 154</td> </span></td></tr> 155</td> # Save current process id to pid file </span></td></tr> 156</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 157</td> </span></td></tr> 158</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 159</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 160</td> </span></td></tr> 161</td> # Make sure directories exist </span></td></tr> 162</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 163</td> </span></td></tr> 164</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 165</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 166</td> </span></td></tr> 167</td> # Snapshot disks </span></td></tr> 168</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 169</td> </span></td></tr> 170</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>NC_INSTALL_VOLUME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 171</td> </span></td></tr> 172</td> </span></td></tr> 173</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>USER_FILES_VOLUME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 174</td> </span></td></tr> 175</td> log </span>"</span>Done.</span>" </span></td></tr> 176</td> </span></td></tr> 177</td> # Dump DB </span></td></tr> 178</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 179</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 180</td> log </span>"</span>Done.</span>" </span></td></tr> 181</td> </span></td></tr> 182</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 183</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 184</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 186</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 187</td> log </span>"</span>Done.</span>" </span></td></tr> 188</td> </span></td></tr> 189</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 190</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 191</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 192</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 193</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 194</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 195</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 196</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> # Path to the btrfs subvolume containing your NC installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>/var/www/nextcloud</span>" </span></td></tr> 19</td> # Path where the (temporary) btrfs snapshot for the NC installation directory should be created </span></td></tr> 20</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/var/www/.nc_snapshot</span>" </span></td></tr> 21</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 22</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 23</td> </span></td></tr> 24</td> ## Nextcloud files directory config </span></td></tr> 25</td> # Path to the btrfs subvolume containing your NC files directory </span></td></tr> 26</td> USER_FILES_VOLUME</span>="</span>/mnt/data/ncdata</span>" </span></td></tr> 27</td> # Path where the (temporary) btrfs snapshot for the NC files directory should be created </span></td></tr> 28</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/mnt/data/.nc_files_snapshot</span>" </span></td></tr> 29</td> # DB dump config </span></td></tr> 30</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 31</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 32</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 33</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 34</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 35</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 36</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 37</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 38</td> </span></td></tr> 39</td> # Configure the connection options for your database: </span></td></tr> 40</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 41</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 42</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 43</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 44</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 45</td> </span></td></tr> 46</td> </span></td></tr> 47</td> # pg_dump will be supplied with the connection options by writing them to ~/.pgpass </span></td></tr> 48</td> PGPASS_PATH</span>="${</span>HOME</span>}</span>/.pgpass</span>" </span></td></tr> 49</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "${</span>PGPASS_PATH</span>}" </span></td></tr> 50</td> SQLDUMP_CMD</span>=(</span>pg_dump </span>"${</span>DB_NAME</span>}") </span></td></tr> 51</td> </span></td></tr> 52</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 53</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 54</td> </span></td></tr> 55</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 56</td> NC_INSTALL_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 57</td> </span></td></tr> 58</td> USER_FILES_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 59</td> </span></td></tr> 60</td> </span></td></tr> 61</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 62</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 63</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 64</td> </span></td></tr> 65</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 66</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 67</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 68</td> </span></td></tr> 69</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 70</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 71</td> </span></td></tr> 72</td> </span></td></tr> 73</td> ### You usually should not need to touch the following variables </span></td></tr> 74</td> </span></td></tr> 75</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 76</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 77</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 78</td> </span></td></tr> 79</td> # file for keeping track of running backup operations </span></td></tr> 80</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 81</td> </span></td></tr> 82</td> </span></td></tr> 83</td> # Utility function for logging to both the log file and stdout </span></td></tr> 84</td> log</span>() { </span></td></tr> 85</td> </span>local </span>msg message </span></td></tr> 86</td> </span></td></tr> 87</td> fwd</span>=(</span>echo</span>) </span></td></tr> 88</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 89</td> </span>then </span></td></tr> 90</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 91</td> </span>fi </span></td></tr> 92</td> </span></td></tr> 93</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 94</td> </span>then </span></td></tr> 95</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 96</td> </span>do </span></td></tr> 97</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 98</td> </span>done </span></td></tr> 99</td> </span>return </span></td></tr> 100</td> </span>fi </span></td></tr> 101</td> </span></td></tr> 102</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 103</td> } </span></td></tr> 104</td> </span></td></tr> 105</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 106</td> cleanup</span>() { </span></td></tr> 107</td> </span>set </span>+</span>eu </span></td></tr> 108</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 109</td> </span></td></tr> 110</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 111</td> </span></td></tr> 112</td> </span></td></tr> 113</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 114</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 116</td> </span></td></tr> 117</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 119</td> </span></td></tr> 120</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 121</td> </span># Cleanup NC install dir snapshot </span></td></tr> 122</td> </span></td></tr> 123</td> </span>btrfs subvolume delete </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></td></tr> 124</td> </span></td></tr> 125</td> </span># Cleanup NC files snapshot </span></td></tr> 126</td> </span></td></tr> 127</td> </span>btrfs subvolume delete </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" </span></td></tr> 128</td> </span></td></tr> 129</td> </span></td></tr> 130</td> </span># Remove PID file </span></td></tr> 131</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 132</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 133</td> } </span></td></tr> 134</td> </span></td></tr> 135</td> </span></td></tr> 136</td> </span></td></tr> 137</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 138</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 139</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 140</td> then </span></td></tr> 141</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 142</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 143</td> </span>then </span></td></tr> 144</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 145</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 146</td> </span>else </span></td></tr> 147</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 148</td> </span>exit 0 </span></td></tr> 149</td> </span>fi </span></td></tr> 150</td> fi </span></td></tr> 151</td> </span></td></tr> 152</td> # cleanup leftovers from last run if any </span></td></tr> 153</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 154</td> </span></td></tr> 155</td> # Save current process id to pid file </span></td></tr> 156</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 157</td> </span></td></tr> 158</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 159</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 160</td> </span></td></tr> 161</td> # Make sure directories exist </span></td></tr> 162</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 163</td> </span></td></tr> 164</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 165</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 166</td> </span></td></tr> 167</td> # Snapshot disks </span></td></tr> 168</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 169</td> </span></td></tr> 170</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>NC_INSTALL_VOLUME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 171</td> </span></td></tr> 172</td> </span></td></tr> 173</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>USER_FILES_VOLUME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 174</td> </span></td></tr> 175</td> log </span>"</span>Done.</span>" </span></td></tr> 176</td> </span></td></tr> 177</td> # Dump DB </span></td></tr> 178</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 179</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 180</td> log </span>"</span>Done.</span>" </span></td></tr> 181</td> </span></td></tr> 182</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 183</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 184</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 186</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 187</td> log </span>"</span>Done.</span>" </span></td></tr> 188</td> </span></td></tr> 189</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 190</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 191</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 192</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 193</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 194</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 195</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 196</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> # Path to the btrfs subvolume containing your NC installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>/var/www/nextcloud</span>" </span></td></tr> 19</td> # Path where the (temporary) btrfs snapshot for the NC installation directory should be created </span></td></tr> 20</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/var/www/.nc_snapshot</span>" </span></td></tr> 21</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 22</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 23</td> </span></td></tr> 24</td> ## Nextcloud files directory config </span></td></tr> 25</td> USER_FILES_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 26</td> USER_FILES_LV</span>="</span>nc-files</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 27</td> # DB dump config </span></td></tr> 28</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 29</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 30</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 31</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 32</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 33</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 34</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 35</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 36</td> </span></td></tr> 37</td> # Configure the connection options for your database: </span></td></tr> 38</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 39</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 40</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 41</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 42</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 43</td> </span></td></tr> 44</td> # mysqldump will be supplied with the db credentials by writing them to ~/.my.cnf </span></td></tr> 45</td> MYCNF_PATH</span>="$</span>HOME</span>/.my.cnf</span>" </span></td></tr> 46</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "${</span>MYCNF_PATH</span>}" </span></td></tr> 47</td> SQLDUMP_CMD</span>=(</span>mysqldump --host=</span>"${</span>DB_HOST</span>}"</span> --port=</span>"${</span>DB_PORT</span>}"</span> --single-transaction --skip-lock-tables </span>"${</span>DB_NAME</span>}") </span></td></tr> 48</td> </span></td></tr> 49</td> </span></td></tr> 50</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 51</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 52</td> </span></td></tr> 53</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 54</td> NC_INSTALL_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 55</td> </span></td></tr> 56</td> USER_FILES_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 57</td> # NC files LVM snapshot name </span></td></tr> 58</td> USER_FILES_SNAPSHOT_NAME</span>="</span>nc-backup-nc-files</span>" </span></td></tr> 59</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nc-files</span>" </span></td></tr> 60</td> </span></td></tr> 61</td> </span></td></tr> 62</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 63</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 64</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 65</td> </span></td></tr> 66</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 67</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 68</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 69</td> </span></td></tr> 70</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 71</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 72</td> </span></td></tr> 73</td> </span></td></tr> 74</td> ### You usually should not need to touch the following variables </span></td></tr> 75</td> </span></td></tr> 76</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 77</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 78</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 79</td> </span></td></tr> 80</td> # file for keeping track of running backup operations </span></td></tr> 81</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 82</td> </span></td></tr> 83</td> </span></td></tr> 84</td> # Utility function for logging to both the log file and stdout </span></td></tr> 85</td> log</span>() { </span></td></tr> 86</td> </span>local </span>msg message </span></td></tr> 87</td> </span></td></tr> 88</td> fwd</span>=(</span>echo</span>) </span></td></tr> 89</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 90</td> </span>then </span></td></tr> 91</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 92</td> </span>fi </span></td></tr> 93</td> </span></td></tr> 94</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 95</td> </span>then </span></td></tr> 96</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 97</td> </span>do </span></td></tr> 98</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 99</td> </span>done </span></td></tr> 100</td> </span>return </span></td></tr> 101</td> </span>fi </span></td></tr> 102</td> </span></td></tr> 103</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 104</td> } </span></td></tr> 105</td> </span></td></tr> 106</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 107</td> cleanup</span>() { </span></td></tr> 108</td> </span>set </span>+</span>eu </span></td></tr> 109</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 110</td> </span></td></tr> 111</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 112</td> </span></td></tr> 113</td> </span></td></tr> 114</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 117</td> </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 120</td> </span></td></tr> 121</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 122</td> </span># Cleanup NC install dir snapshot </span></td></tr> 123</td> </span></td></tr> 124</td> </span>btrfs subvolume delete </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></td></tr> 125</td> </span></td></tr> 126</td> </span># Cleanup NC files snapshot </span></td></tr> 127</td> </span></td></tr> 128</td> </span>umount </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 129</td> </span>rm</span> -</span>rf </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 130</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 131</td> </span>lvremove </span>"</span>/dev/</span>${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 132</td> </span></td></tr> 133</td> </span></td></tr> 134</td> </span># Remove PID file </span></td></tr> 135</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 136</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 137</td> } </span></td></tr> 138</td> </span></td></tr> 139</td> </span></td></tr> 140</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 141</td> mount_snapshot_lvm</span>() { </span></td></tr> 142</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 143</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 144</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 145</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 146</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 147</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 148</td> } </span></td></tr> 149</td> </span></td></tr> 150</td> </span></td></tr> 151</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 152</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 153</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 154</td> then </span></td></tr> 155</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 156</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 157</td> </span>then </span></td></tr> 158</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 159</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 160</td> </span>else </span></td></tr> 161</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 162</td> </span>exit 0 </span></td></tr> 163</td> </span>fi </span></td></tr> 164</td> fi </span></td></tr> 165</td> </span></td></tr> 166</td> # cleanup leftovers from last run if any </span></td></tr> 167</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 168</td> </span></td></tr> 169</td> # Save current process id to pid file </span></td></tr> 170</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 171</td> </span></td></tr> 172</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 173</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 174</td> </span></td></tr> 175</td> # Make sure directories exist </span></td></tr> 176</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 177</td> </span></td></tr> 178</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 179</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 180</td> </span></td></tr> 181</td> # Snapshot disks </span></td></tr> 182</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 183</td> </span></td></tr> 184</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>NC_INSTALL_VOLUME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> </span></td></tr> 186</td> </span></td></tr> 187</td> lvcreate</span> -</span>s</span> --</span>name </span>"${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_LV</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 188</td> mount_snapshot_lvm </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> </span></td></tr> 190</td> log </span>"</span>Done.</span>" </span></td></tr> 191</td> </span></td></tr> 192</td> # Dump DB </span></td></tr> 193</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 194</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 198</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 199</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 200</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 201</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 202</td> log </span>"</span>Done.</span>" </span></td></tr> 203</td> </span></td></tr> 204</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 205</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 206</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 207</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 208</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 209</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 210</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 211</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> # Path to the btrfs subvolume containing your NC installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>/var/www/nextcloud</span>" </span></td></tr> 19</td> # Path where the (temporary) btrfs snapshot for the NC installation directory should be created </span></td></tr> 20</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/var/www/.nc_snapshot</span>" </span></td></tr> 21</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 22</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 23</td> </span></td></tr> 24</td> ## Nextcloud files directory config </span></td></tr> 25</td> USER_FILES_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 26</td> USER_FILES_LV</span>="</span>nc-files</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 27</td> # DB dump config </span></td></tr> 28</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 29</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 30</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 31</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 32</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 33</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 34</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 35</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 36</td> </span></td></tr> 37</td> # Configure the connection options for your database: </span></td></tr> 38</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 39</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 40</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 41</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 42</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 43</td> </span></td></tr> 44</td> </span></td></tr> 45</td> # pg_dump will be supplied with the connection options by writing them to ~/.pgpass </span></td></tr> 46</td> PGPASS_PATH</span>="${</span>HOME</span>}</span>/.pgpass</span>" </span></td></tr> 47</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "${</span>PGPASS_PATH</span>}" </span></td></tr> 48</td> SQLDUMP_CMD</span>=(</span>pg_dump </span>"${</span>DB_NAME</span>}") </span></td></tr> 49</td> </span></td></tr> 50</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 51</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 52</td> </span></td></tr> 53</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 54</td> NC_INSTALL_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 55</td> </span></td></tr> 56</td> USER_FILES_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 57</td> # NC files LVM snapshot name </span></td></tr> 58</td> USER_FILES_SNAPSHOT_NAME</span>="</span>nc-backup-nc-files</span>" </span></td></tr> 59</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nc-files</span>" </span></td></tr> 60</td> </span></td></tr> 61</td> </span></td></tr> 62</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 63</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 64</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 65</td> </span></td></tr> 66</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 67</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 68</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 69</td> </span></td></tr> 70</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 71</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 72</td> </span></td></tr> 73</td> </span></td></tr> 74</td> ### You usually should not need to touch the following variables </span></td></tr> 75</td> </span></td></tr> 76</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 77</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 78</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 79</td> </span></td></tr> 80</td> # file for keeping track of running backup operations </span></td></tr> 81</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 82</td> </span></td></tr> 83</td> </span></td></tr> 84</td> # Utility function for logging to both the log file and stdout </span></td></tr> 85</td> log</span>() { </span></td></tr> 86</td> </span>local </span>msg message </span></td></tr> 87</td> </span></td></tr> 88</td> fwd</span>=(</span>echo</span>) </span></td></tr> 89</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 90</td> </span>then </span></td></tr> 91</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 92</td> </span>fi </span></td></tr> 93</td> </span></td></tr> 94</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 95</td> </span>then </span></td></tr> 96</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 97</td> </span>do </span></td></tr> 98</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 99</td> </span>done </span></td></tr> 100</td> </span>return </span></td></tr> 101</td> </span>fi </span></td></tr> 102</td> </span></td></tr> 103</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 104</td> } </span></td></tr> 105</td> </span></td></tr> 106</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 107</td> cleanup</span>() { </span></td></tr> 108</td> </span>set </span>+</span>eu </span></td></tr> 109</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 110</td> </span></td></tr> 111</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 112</td> </span></td></tr> 113</td> </span></td></tr> 114</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 117</td> </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 120</td> </span></td></tr> 121</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 122</td> </span># Cleanup NC install dir snapshot </span></td></tr> 123</td> </span></td></tr> 124</td> </span>btrfs subvolume delete </span>"$</span>NC_INSTALL_SNAPSHOT_PATH</span>" </span></td></tr> 125</td> </span></td></tr> 126</td> </span># Cleanup NC files snapshot </span></td></tr> 127</td> </span></td></tr> 128</td> </span>umount </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 129</td> </span>rm</span> -</span>rf </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 130</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 131</td> </span>lvremove </span>"</span>/dev/</span>${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 132</td> </span></td></tr> 133</td> </span></td></tr> 134</td> </span># Remove PID file </span></td></tr> 135</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 136</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 137</td> } </span></td></tr> 138</td> </span></td></tr> 139</td> </span></td></tr> 140</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 141</td> mount_snapshot_lvm</span>() { </span></td></tr> 142</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 143</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 144</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 145</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 146</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 147</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 148</td> } </span></td></tr> 149</td> </span></td></tr> 150</td> </span></td></tr> 151</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 152</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 153</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 154</td> then </span></td></tr> 155</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 156</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 157</td> </span>then </span></td></tr> 158</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 159</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 160</td> </span>else </span></td></tr> 161</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 162</td> </span>exit 0 </span></td></tr> 163</td> </span>fi </span></td></tr> 164</td> fi </span></td></tr> 165</td> </span></td></tr> 166</td> # cleanup leftovers from last run if any </span></td></tr> 167</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 168</td> </span></td></tr> 169</td> # Save current process id to pid file </span></td></tr> 170</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 171</td> </span></td></tr> 172</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 173</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 174</td> </span></td></tr> 175</td> # Make sure directories exist </span></td></tr> 176</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 177</td> </span></td></tr> 178</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 179</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 180</td> </span></td></tr> 181</td> # Snapshot disks </span></td></tr> 182</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 183</td> </span></td></tr> 184</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>NC_INSTALL_VOLUME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> </span></td></tr> 186</td> </span></td></tr> 187</td> lvcreate</span> -</span>s</span> --</span>name </span>"${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_LV</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 188</td> mount_snapshot_lvm </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> </span></td></tr> 190</td> log </span>"</span>Done.</span>" </span></td></tr> 191</td> </span></td></tr> 192</td> # Dump DB </span></td></tr> 193</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 194</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 198</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 199</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 200</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 201</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 202</td> log </span>"</span>Done.</span>" </span></td></tr> 203</td> </span></td></tr> 204</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 205</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 206</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 207</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 208</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 209</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 210</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 211</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> NC_INSTALL_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>nextcloud</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 19</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 20</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 21</td> </span></td></tr> 22</td> ## Nextcloud files directory config </span></td></tr> 23</td> # Path to the btrfs subvolume containing your NC files directory </span></td></tr> 24</td> USER_FILES_VOLUME</span>="</span>/mnt/data/ncdata</span>" </span></td></tr> 25</td> # Path where the (temporary) btrfs snapshot for the NC files directory should be created </span></td></tr> 26</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/mnt/data/.nc_files_snapshot</span>" </span></td></tr> 27</td> # DB dump config </span></td></tr> 28</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 29</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 30</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 31</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 32</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 33</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 34</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 35</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 36</td> </span></td></tr> 37</td> # Configure the connection options for your database: </span></td></tr> 38</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 39</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 40</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 41</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 42</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 43</td> </span></td></tr> 44</td> # mysqldump will be supplied with the db credentials by writing them to ~/.my.cnf </span></td></tr> 45</td> MYCNF_PATH</span>="$</span>HOME</span>/.my.cnf</span>" </span></td></tr> 46</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "${</span>MYCNF_PATH</span>}" </span></td></tr> 47</td> SQLDUMP_CMD</span>=(</span>mysqldump --host=</span>"${</span>DB_HOST</span>}"</span> --port=</span>"${</span>DB_PORT</span>}"</span> --single-transaction --skip-lock-tables </span>"${</span>DB_NAME</span>}") </span></td></tr> 48</td> </span></td></tr> 49</td> </span></td></tr> 50</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 51</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 52</td> </span></td></tr> 53</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 54</td> NC_INSTALL_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 55</td> # NC install LVM snapshot name </span></td></tr> 56</td> NC_INSTALL_SNAPSHOT_NAME</span>="</span>nc-backup-nextcloud</span>" </span></td></tr> 57</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nextcloud</span>" </span></td></tr> 58</td> </span></td></tr> 59</td> USER_FILES_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 60</td> </span></td></tr> 61</td> </span></td></tr> 62</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 63</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 64</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 65</td> </span></td></tr> 66</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 67</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 68</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 69</td> </span></td></tr> 70</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 71</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 72</td> </span></td></tr> 73</td> </span></td></tr> 74</td> ### You usually should not need to touch the following variables </span></td></tr> 75</td> </span></td></tr> 76</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 77</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 78</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 79</td> </span></td></tr> 80</td> # file for keeping track of running backup operations </span></td></tr> 81</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 82</td> </span></td></tr> 83</td> </span></td></tr> 84</td> # Utility function for logging to both the log file and stdout </span></td></tr> 85</td> log</span>() { </span></td></tr> 86</td> </span>local </span>msg message </span></td></tr> 87</td> </span></td></tr> 88</td> fwd</span>=(</span>echo</span>) </span></td></tr> 89</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 90</td> </span>then </span></td></tr> 91</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 92</td> </span>fi </span></td></tr> 93</td> </span></td></tr> 94</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 95</td> </span>then </span></td></tr> 96</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 97</td> </span>do </span></td></tr> 98</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 99</td> </span>done </span></td></tr> 100</td> </span>return </span></td></tr> 101</td> </span>fi </span></td></tr> 102</td> </span></td></tr> 103</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 104</td> } </span></td></tr> 105</td> </span></td></tr> 106</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 107</td> cleanup</span>() { </span></td></tr> 108</td> </span>set </span>+</span>eu </span></td></tr> 109</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 110</td> </span></td></tr> 111</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 112</td> </span></td></tr> 113</td> </span></td></tr> 114</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 117</td> </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 120</td> </span></td></tr> 121</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 122</td> </span># Cleanup NC install dir snapshot </span></td></tr> 123</td> </span></td></tr> 124</td> </span>umount </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 125</td> </span>rm</span> -</span>rf </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 126</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 127</td> </span>lvremove </span>"</span>/dev/</span>${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 128</td> </span></td></tr> 129</td> </span># Cleanup NC files snapshot </span></td></tr> 130</td> </span></td></tr> 131</td> </span>btrfs subvolume delete </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" </span></td></tr> 132</td> </span></td></tr> 133</td> </span></td></tr> 134</td> </span># Remove PID file </span></td></tr> 135</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 136</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 137</td> } </span></td></tr> 138</td> </span></td></tr> 139</td> </span></td></tr> 140</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 141</td> mount_snapshot_lvm</span>() { </span></td></tr> 142</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 143</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 144</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 145</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 146</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 147</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 148</td> } </span></td></tr> 149</td> </span></td></tr> 150</td> </span></td></tr> 151</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 152</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 153</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 154</td> then </span></td></tr> 155</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 156</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 157</td> </span>then </span></td></tr> 158</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 159</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 160</td> </span>else </span></td></tr> 161</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 162</td> </span>exit 0 </span></td></tr> 163</td> </span>fi </span></td></tr> 164</td> fi </span></td></tr> 165</td> </span></td></tr> 166</td> # cleanup leftovers from last run if any </span></td></tr> 167</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 168</td> </span></td></tr> 169</td> # Save current process id to pid file </span></td></tr> 170</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 171</td> </span></td></tr> 172</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 173</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 174</td> </span></td></tr> 175</td> # Make sure directories exist </span></td></tr> 176</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 177</td> </span></td></tr> 178</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 179</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 180</td> </span></td></tr> 181</td> # Snapshot disks </span></td></tr> 182</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 183</td> </span></td></tr> 184</td> lvcreate</span> -</span>s</span> --</span>name </span>"$</span>NC_INSTALL_SNAPSHOT_NAME</span>" "${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_VOLUME</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> mount_snapshot_lvm </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 186</td> </span></td></tr> 187</td> </span></td></tr> 188</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>USER_FILES_VOLUME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> </span></td></tr> 190</td> log </span>"</span>Done.</span>" </span></td></tr> 191</td> </span></td></tr> 192</td> # Dump DB </span></td></tr> 193</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 194</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 198</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 199</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 200</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 201</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 202</td> log </span>"</span>Done.</span>" </span></td></tr> 203</td> </span></td></tr> 204</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 205</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 206</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 207</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 208</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 209</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 210</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 211</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> NC_INSTALL_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>nextcloud</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 19</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 20</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 21</td> </span></td></tr> 22</td> ## Nextcloud files directory config </span></td></tr> 23</td> # Path to the btrfs subvolume containing your NC files directory </span></td></tr> 24</td> USER_FILES_VOLUME</span>="</span>/mnt/data/ncdata</span>" </span></td></tr> 25</td> # Path where the (temporary) btrfs snapshot for the NC files directory should be created </span></td></tr> 26</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/mnt/data/.nc_files_snapshot</span>" </span></td></tr> 27</td> # DB dump config </span></td></tr> 28</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 29</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 30</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 31</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 32</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 33</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 34</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 35</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 36</td> </span></td></tr> 37</td> # Configure the connection options for your database: </span></td></tr> 38</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 39</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 40</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 41</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 42</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 43</td> </span></td></tr> 44</td> </span></td></tr> 45</td> # pg_dump will be supplied with the connection options by writing them to ~/.pgpass </span></td></tr> 46</td> PGPASS_PATH</span>="${</span>HOME</span>}</span>/.pgpass</span>" </span></td></tr> 47</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "${</span>PGPASS_PATH</span>}" </span></td></tr> 48</td> SQLDUMP_CMD</span>=(</span>pg_dump </span>"${</span>DB_NAME</span>}") </span></td></tr> 49</td> </span></td></tr> 50</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 51</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 52</td> </span></td></tr> 53</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 54</td> NC_INSTALL_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 55</td> # NC install LVM snapshot name </span></td></tr> 56</td> NC_INSTALL_SNAPSHOT_NAME</span>="</span>nc-backup-nextcloud</span>" </span></td></tr> 57</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nextcloud</span>" </span></td></tr> 58</td> </span></td></tr> 59</td> USER_FILES_FSTYPE</span>="</span>btrfs</span>" </span></td></tr> 60</td> </span></td></tr> 61</td> </span></td></tr> 62</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 63</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 64</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 65</td> </span></td></tr> 66</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 67</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 68</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 69</td> </span></td></tr> 70</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 71</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 72</td> </span></td></tr> 73</td> </span></td></tr> 74</td> ### You usually should not need to touch the following variables </span></td></tr> 75</td> </span></td></tr> 76</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 77</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 78</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 79</td> </span></td></tr> 80</td> # file for keeping track of running backup operations </span></td></tr> 81</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 82</td> </span></td></tr> 83</td> </span></td></tr> 84</td> # Utility function for logging to both the log file and stdout </span></td></tr> 85</td> log</span>() { </span></td></tr> 86</td> </span>local </span>msg message </span></td></tr> 87</td> </span></td></tr> 88</td> fwd</span>=(</span>echo</span>) </span></td></tr> 89</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 90</td> </span>then </span></td></tr> 91</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 92</td> </span>fi </span></td></tr> 93</td> </span></td></tr> 94</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 95</td> </span>then </span></td></tr> 96</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 97</td> </span>do </span></td></tr> 98</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 99</td> </span>done </span></td></tr> 100</td> </span>return </span></td></tr> 101</td> </span>fi </span></td></tr> 102</td> </span></td></tr> 103</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 104</td> } </span></td></tr> 105</td> </span></td></tr> 106</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 107</td> cleanup</span>() { </span></td></tr> 108</td> </span>set </span>+</span>eu </span></td></tr> 109</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 110</td> </span></td></tr> 111</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 112</td> </span></td></tr> 113</td> </span></td></tr> 114</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 115</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 117</td> </span></td></tr> 118</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 120</td> </span></td></tr> 121</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 122</td> </span># Cleanup NC install dir snapshot </span></td></tr> 123</td> </span></td></tr> 124</td> </span>umount </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 125</td> </span>rm</span> -</span>rf </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 126</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 127</td> </span>lvremove </span>"</span>/dev/</span>${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 128</td> </span></td></tr> 129</td> </span># Cleanup NC files snapshot </span></td></tr> 130</td> </span></td></tr> 131</td> </span>btrfs subvolume delete </span>"$</span>USER_FILES_SNAPSHOT_PATH</span>" </span></td></tr> 132</td> </span></td></tr> 133</td> </span></td></tr> 134</td> </span># Remove PID file </span></td></tr> 135</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 136</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 137</td> } </span></td></tr> 138</td> </span></td></tr> 139</td> </span></td></tr> 140</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 141</td> mount_snapshot_lvm</span>() { </span></td></tr> 142</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 143</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 144</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 145</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 146</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 147</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 148</td> } </span></td></tr> 149</td> </span></td></tr> 150</td> </span></td></tr> 151</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 152</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 153</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 154</td> then </span></td></tr> 155</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 156</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 157</td> </span>then </span></td></tr> 158</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 159</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 160</td> </span>else </span></td></tr> 161</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 162</td> </span>exit 0 </span></td></tr> 163</td> </span>fi </span></td></tr> 164</td> fi </span></td></tr> 165</td> </span></td></tr> 166</td> # cleanup leftovers from last run if any </span></td></tr> 167</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 168</td> </span></td></tr> 169</td> # Save current process id to pid file </span></td></tr> 170</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 171</td> </span></td></tr> 172</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 173</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 174</td> </span></td></tr> 175</td> # Make sure directories exist </span></td></tr> 176</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 177</td> </span></td></tr> 178</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 179</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 180</td> </span></td></tr> 181</td> # Snapshot disks </span></td></tr> 182</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 183</td> </span></td></tr> 184</td> lvcreate</span> -</span>s</span> --</span>name </span>"$</span>NC_INSTALL_SNAPSHOT_NAME</span>" "${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_VOLUME</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 185</td> mount_snapshot_lvm </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 186</td> </span></td></tr> 187</td> </span></td></tr> 188</td> btrfs subvolume snapshot</span> -</span>r </span>"${</span>USER_FILES_VOLUME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> </span></td></tr> 190</td> log </span>"</span>Done.</span>" </span></td></tr> 191</td> </span></td></tr> 192</td> # Dump DB </span></td></tr> 193</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 194</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 198</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 199</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 200</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 201</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 202</td> log </span>"</span>Done.</span>" </span></td></tr> 203</td> </span></td></tr> 204</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 205</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 206</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 207</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 208</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 209</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 210</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 211</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> NC_INSTALL_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>nextcloud</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 19</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 20</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 21</td> </span></td></tr> 22</td> ## Nextcloud files directory config </span></td></tr> 23</td> USER_FILES_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 24</td> USER_FILES_LV</span>="</span>nc-files</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 25</td> # DB dump config </span></td></tr> 26</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 27</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 28</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 29</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 30</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 31</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 32</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 33</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 34</td> </span></td></tr> 35</td> # Configure the connection options for your database: </span></td></tr> 36</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 37</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 38</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 39</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 40</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 41</td> </span></td></tr> 42</td> # mysqldump will be supplied with the db credentials by writing them to ~/.my.cnf </span></td></tr> 43</td> MYCNF_PATH</span>="$</span>HOME</span>/.my.cnf</span>" </span></td></tr> 44</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "${</span>MYCNF_PATH</span>}" </span></td></tr> 45</td> SQLDUMP_CMD</span>=(</span>mysqldump --host=</span>"${</span>DB_HOST</span>}"</span> --port=</span>"${</span>DB_PORT</span>}"</span> --single-transaction --skip-lock-tables </span>"${</span>DB_NAME</span>}") </span></td></tr> 46</td> </span></td></tr> 47</td> </span></td></tr> 48</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 49</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 50</td> </span></td></tr> 51</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 52</td> NC_INSTALL_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 53</td> # NC install LVM snapshot name </span></td></tr> 54</td> NC_INSTALL_SNAPSHOT_NAME</span>="</span>nc-backup-nextcloud</span>" </span></td></tr> 55</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nextcloud</span>" </span></td></tr> 56</td> </span></td></tr> 57</td> USER_FILES_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 58</td> # NC files LVM snapshot name </span></td></tr> 59</td> USER_FILES_SNAPSHOT_NAME</span>="</span>nc-backup-nc-files</span>" </span></td></tr> 60</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nc-files</span>" </span></td></tr> 61</td> </span></td></tr> 62</td> </span></td></tr> 63</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 64</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 65</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 66</td> </span></td></tr> 67</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 68</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 69</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 70</td> </span></td></tr> 71</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 72</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 73</td> </span></td></tr> 74</td> </span></td></tr> 75</td> ### You usually should not need to touch the following variables </span></td></tr> 76</td> </span></td></tr> 77</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 78</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 79</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 80</td> </span></td></tr> 81</td> # file for keeping track of running backup operations </span></td></tr> 82</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 83</td> </span></td></tr> 84</td> </span></td></tr> 85</td> # Utility function for logging to both the log file and stdout </span></td></tr> 86</td> log</span>() { </span></td></tr> 87</td> </span>local </span>msg message </span></td></tr> 88</td> </span></td></tr> 89</td> fwd</span>=(</span>echo</span>) </span></td></tr> 90</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 91</td> </span>then </span></td></tr> 92</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 93</td> </span>fi </span></td></tr> 94</td> </span></td></tr> 95</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 96</td> </span>then </span></td></tr> 97</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 98</td> </span>do </span></td></tr> 99</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 100</td> </span>done </span></td></tr> 101</td> </span>return </span></td></tr> 102</td> </span>fi </span></td></tr> 103</td> </span></td></tr> 104</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 105</td> } </span></td></tr> 106</td> </span></td></tr> 107</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 108</td> cleanup</span>() { </span></td></tr> 109</td> </span>set </span>+</span>eu </span></td></tr> 110</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 111</td> </span></td></tr> 112</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 113</td> </span></td></tr> 114</td> </span></td></tr> 115</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 117</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 118</td> </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 120</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 121</td> </span></td></tr> 122</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 123</td> </span># Cleanup NC install dir snapshot </span></td></tr> 124</td> </span></td></tr> 125</td> </span>umount </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 126</td> </span>rm</span> -</span>rf </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 127</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 128</td> </span>lvremove </span>"</span>/dev/</span>${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 129</td> </span></td></tr> 130</td> </span># Cleanup NC files snapshot </span></td></tr> 131</td> </span></td></tr> 132</td> </span>umount </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 133</td> </span>rm</span> -</span>rf </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 134</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 135</td> </span>lvremove </span>"</span>/dev/</span>${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 136</td> </span></td></tr> 137</td> </span></td></tr> 138</td> </span># Remove PID file </span></td></tr> 139</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 140</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 141</td> } </span></td></tr> 142</td> </span></td></tr> 143</td> </span></td></tr> 144</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 145</td> mount_snapshot_lvm</span>() { </span></td></tr> 146</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 147</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 148</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 149</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 150</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 151</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 152</td> } </span></td></tr> 153</td> </span></td></tr> 154</td> </span></td></tr> 155</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 156</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 157</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 158</td> then </span></td></tr> 159</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 160</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 161</td> </span>then </span></td></tr> 162</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 163</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 164</td> </span>else </span></td></tr> 165</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 166</td> </span>exit 0 </span></td></tr> 167</td> </span>fi </span></td></tr> 168</td> fi </span></td></tr> 169</td> </span></td></tr> 170</td> # cleanup leftovers from last run if any </span></td></tr> 171</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 172</td> </span></td></tr> 173</td> # Save current process id to pid file </span></td></tr> 174</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 175</td> </span></td></tr> 176</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 177</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 178</td> </span></td></tr> 179</td> # Make sure directories exist </span></td></tr> 180</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 181</td> </span></td></tr> 182</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 183</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 184</td> </span></td></tr> 185</td> # Snapshot disks </span></td></tr> 186</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 187</td> </span></td></tr> 188</td> lvcreate</span> -</span>s</span> --</span>name </span>"$</span>NC_INSTALL_SNAPSHOT_NAME</span>" "${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_VOLUME</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> mount_snapshot_lvm </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 190</td> </span></td></tr> 191</td> </span></td></tr> 192</td> lvcreate</span> -</span>s</span> --</span>name </span>"${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_LV</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 193</td> mount_snapshot_lvm </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 194</td> </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Dump DB </span></td></tr> 198</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 199</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 200</td> log </span>"</span>Done.</span>" </span></td></tr> 201</td> </span></td></tr> 202</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 203</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 204</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 205</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 206</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 207</td> log </span>"</span>Done.</span>" </span></td></tr> 208</td> </span></td></tr> 209</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 210</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 211</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 212</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 213</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 214</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 215</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 216</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-icon> </sl-button> 1</td> #!/bin/bash </span></td></tr> 2</td> </span></td></tr> 3</td> set -euo pipefail </span></td></tr> 4</td> </span></td></tr> 5</td> ### ========== ### </span></td></tr> 6</td> # Procedure </span></td></tr> 7</td> # - The script will create snapshots of the lvm root volume and the btrfs subvolume backing Nextcloud's user files. Then it will start an sqldump of the database </span></td></tr> 8</td> # - Afterwards, all 3 snapshots will be mounted/copied in the same parent directory and uploaded to a kopia repository </span></td></tr> 9</td> ### ========== ### </span></td></tr> 10</td> </span></td></tr> 11</td> ### CONFIGURATION </span></td></tr> 12</td> </span></td></tr> 13</td> # file for writing logs. Can be left empty to only log to stdout </span></td></tr> 14</td> LOG_FILE</span>="</span>/var/log/nc-backup.log</span>" </span></td></tr> 15</td> </span></td></tr> 16</td> ## Nextcloud installation directory config </span></td></tr> 17</td> NC_INSTALL_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 18</td> NC_INSTALL_VOLUME</span>="</span>nextcloud</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 19</td> # Adjust this if the nc installation directory is not in the root of the backing filesystem volume </span></td></tr> 20</td> NC_INSTALL_RELATIVE_PATH</span>="</span>/</span>" </span></td></tr> 21</td> </span></td></tr> 22</td> ## Nextcloud files directory config </span></td></tr> 23</td> USER_FILES_VG</span>="</span>lvm-1</span>" </span># Adjust to match the LVM volume group containing your nc installation directory </span></td></tr> 24</td> USER_FILES_LV</span>="</span>nc-files</span>" </span># Adjust to match the logical volume name containing your nc installation directory </span></td></tr> 25</td> # DB dump config </span></td></tr> 26</td> # Path where to temporarily store database dump (should have enough free space) </span></td></tr> 27</td> SQL_DUMP_TARGET_DIR</span>="</span>/var/nc-backup-db</span>" </span></td></tr> 28</td> # Whatever command can be used to create an sqldump (including parameters like user, host, dbname, password cfg ...) </span></td></tr> 29</td> # - If your DB is run inside docker, just prepend the command with whatever is needed to run a command inside the container, </span></td></tr> 30</td> # for example: SQLDUMP_CMD=(docker exec <containername> pg_dump -U postgres -d nc) </span></td></tr> 31</td> # - For MariaDB (with the InnoDB engine), make sure to use --single-transaction (and optionally --skip-lock-tables to avoid slow downs for your system) to avoid side effects. </span></td></tr> 32</td> # See https://mysqldump.guru/mysqldump-single-transaction-flag.html for more info. </span></td></tr> 33</td> # - For postgresql, no adjustemnts are needed </span></td></tr> 34</td> </span></td></tr> 35</td> # Configure the connection options for your database: </span></td></tr> 36</td> DB_USER</span>="</span>nextcloud</span>" </span></td></tr> 37</td> DB_NAME</span>="</span>nextcloud</span>" </span></td></tr> 38</td> DB_PASSWORD</span>="</span>password</span>" </span></td></tr> 39</td> DB_HOST</span>="</span>127.0.0.1</span>" </span></td></tr> 40</td> DB_PORT</span>="</span>5432</span>" </span></td></tr> 41</td> </span></td></tr> 42</td> </span></td></tr> 43</td> # pg_dump will be supplied with the connection options by writing them to ~/.pgpass </span></td></tr> 44</td> PGPASS_PATH</span>="${</span>HOME</span>}</span>/.pgpass</span>" </span></td></tr> 45</td> install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "${</span>PGPASS_PATH</span>}" </span></td></tr> 46</td> SQLDUMP_CMD</span>=(</span>pg_dump </span>"${</span>DB_NAME</span>}") </span></td></tr> 47</td> </span></td></tr> 48</td> # Setup any tags here that you want to be added to your your kopia snapshot (comma separated in the format label:value) </span></td></tr> 49</td> KOPIA_TAGS</span>="</span>job:nextcloud-full-backup,nc-files:yes,nc-db:yes,nc-nextcloud:yes</span>" </span></td></tr> 50</td> </span></td></tr> 51</td> ### ADVANCED/INTERNAL CONFIG - only adjust if you are sure that you know what you're doing ### </span></td></tr> 52</td> NC_INSTALL_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 53</td> # NC install LVM snapshot name </span></td></tr> 54</td> NC_INSTALL_SNAPSHOT_NAME</span>="</span>nc-backup-nextcloud</span>" </span></td></tr> 55</td> NC_INSTALL_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nextcloud</span>" </span></td></tr> 56</td> </span></td></tr> 57</td> USER_FILES_FSTYPE</span>="</span>lvm</span>" </span></td></tr> 58</td> # NC files LVM snapshot name </span></td></tr> 59</td> USER_FILES_SNAPSHOT_NAME</span>="</span>nc-backup-nc-files</span>" </span></td></tr> 60</td> USER_FILES_SNAPSHOT_PATH</span>="</span>/run/mnt/nc-backup-lvm-nc-files</span>" </span></td></tr> 61</td> </span></td></tr> 62</td> </span></td></tr> 63</td> # FORCE UID and GID that all files from the nextcloud install directory should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 64</td> NC_OWNER_UID</span>=</span>33 </span></td></tr> 65</td> NC_OWNER_GID</span>=</span>33 </span></td></tr> 66</td> </span></td></tr> 67</td> # FORCE UID and GID that all user files should have in the backup (e.g. that of your www-data user and group) </span></td></tr> 68</td> USER_FILES_OWNER_UID</span>=</span>33 </span></td></tr> 69</td> USER_FILES_OWNER_GID</span>=</span>33 </span></td></tr> 70</td> </span></td></tr> 71</td> # Configure path where all parts of the backup will be mapped to before transferring with Kopia </span></td></tr> 72</td> BACKUP_ROOT</span>="</span>/run/nc-backup</span>" </span></td></tr> 73</td> </span></td></tr> 74</td> </span></td></tr> 75</td> ### You usually should not need to touch the following variables </span></td></tr> 76</td> </span></td></tr> 77</td> _USER_FILES_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nc_files</span>" </span></td></tr> 78</td> _NC_INSTALL_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/nextcloud</span>" </span></td></tr> 79</td> _DB_BIND</span>="${</span>BACKUP_ROOT</span>}</span>/db</span>" </span></td></tr> 80</td> </span></td></tr> 81</td> # file for keeping track of running backup operations </span></td></tr> 82</td> _PID_FILE</span>="</span>/var/run/nc-backup.pid</span>" </span></td></tr> 83</td> </span></td></tr> 84</td> </span></td></tr> 85</td> # Utility function for logging to both the log file and stdout </span></td></tr> 86</td> log</span>() { </span></td></tr> 87</td> </span>local </span>msg message </span></td></tr> 88</td> </span></td></tr> 89</td> fwd</span>=(</span>echo</span>) </span></td></tr> 90</td> </span>if </span>[[ </span>-</span>n </span>"$</span>LOG_FILE</span>" </span>]] </span></td></tr> 91</td> </span>then </span></td></tr> 92</td> fwd</span>=(</span>tee -a </span>"$</span>LOG_FILE</span>") </span></td></tr> 93</td> </span>fi </span></td></tr> 94</td> </span></td></tr> 95</td> </span>if </span>[[ </span>-</span>z </span>"${</span>1</span>:-}" </span>]] </span></td></tr> 96</td> </span>then </span></td></tr> 97</td> </span>while </span>read </span>-</span>r</span> message </span></td></tr> 98</td> </span>do </span></td></tr> 99</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>message</span>[</span>@</span>]}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 100</td> </span>done </span></td></tr> 101</td> </span>return </span></td></tr> 102</td> </span>fi </span></td></tr> 103</td> </span></td></tr> 104</td> </span>echo </span>"\$$</span>$</span>></span>$(</span>date</span> -</span>Is</span>)</span>=></span>" "${</span>@</span>}" </span>2</span>>&</span>1 </span>| "${</span>fwd</span>[</span>@</span>]}" </span></td></tr> 105</td> } </span></td></tr> 106</td> </span></td></tr> 107</td> # Utility function for cleaning up snapshots, pid file, etc. </span></td></tr> 108</td> cleanup</span>() { </span></td></tr> 109</td> </span>set </span>+</span>eu </span></td></tr> 110</td> </span>echo </span>"</span>Cleanup...</span>" </span></td></tr> 111</td> </span></td></tr> 112</td> </span>rm</span> -</span>rf </span>"${</span>PGPASS_PATH</span>}" </span></td></tr> 113</td> </span></td></tr> 114</td> </span></td></tr> 115</td> </span>umount </span>"$</span>_DB_BIND</span>" </span></td></tr> 116</td> </span>umount </span>"$</span>_NC_INSTALL_BIND</span>" </span></td></tr> 117</td> </span>umount </span>"$</span>_USER_FILES_BIND</span>" </span></td></tr> 118</td> </span></td></tr> 119</td> </span>[[ </span>-</span>z </span>"${</span>USER_FILES_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>USER_FILES_MOUNT</span>" </span></td></tr> 120</td> </span>[[ </span>-</span>z </span>"${</span>NC_INSTALL_MOUNT</span>}" </span>]] </span>|| </span>umount </span>"$</span>NC_INSTALL_MOUNT</span>" </span></td></tr> 121</td> </span></td></tr> 122</td> </span>rm</span> -</span>rf </span>"$</span>SQL_DUMP_TARGET_DIR</span>"; </span></td></tr> 123</td> </span># Cleanup NC install dir snapshot </span></td></tr> 124</td> </span></td></tr> 125</td> </span>umount </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 126</td> </span>rm</span> -</span>rf </span>"${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span></td></tr> 127</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 128</td> </span>lvremove </span>"</span>/dev/</span>${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" </span></td></tr> 129</td> </span></td></tr> 130</td> </span># Cleanup NC files snapshot </span></td></tr> 131</td> </span></td></tr> 132</td> </span>umount </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 133</td> </span>rm</span> -</span>rf </span>"${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span></td></tr> 134</td> </span>lvchange</span> -</span>an</span> -</span>K </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 135</td> </span>lvremove </span>"</span>/dev/</span>${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" </span></td></tr> 136</td> </span></td></tr> 137</td> </span></td></tr> 138</td> </span># Remove PID file </span></td></tr> 139</td> </span>rm </span>"$</span>_PID_FILE</span>" </span></td></tr> 140</td> </span>echo </span>"</span>Done.</span>" </span></td></tr> 141</td> } </span></td></tr> 142</td> </span></td></tr> 143</td> </span></td></tr> 144</td> # Utility function for mounting lvm snapshot to a mountpoint </span></td></tr> 145</td> mount_snapshot_lvm</span>() { </span></td></tr> 146</td> </span>local </span>snapshot mountpoint ownership </span></td></tr> 147</td> snapshot</span>="${</span>1</span>?}" </span></td></tr> 148</td> mountpoint</span>="${</span>2</span>?}" </span></td></tr> 149</td> </span>mkdir</span> -</span>p </span>"${</span>mountpoint</span>}" </span></td></tr> 150</td> </span>lvchange</span> -</span>ay</span> -</span>K </span>"${</span>snapshot</span>}" </span></td></tr> 151</td> </span>mount</span> -</span>o </span>"</span>ro</span>" "</span>/dev/</span>${</span>snapshot</span>}" "${</span>mountpoint</span>}" </span></td></tr> 152</td> } </span></td></tr> 153</td> </span></td></tr> 154</td> </span></td></tr> 155</td> ## Detect whether a backup is still running (and abort in this case) </span></td></tr> 156</td> # If the backup has been running for more than 10 hours, it is assumed to be stuck and its process killed </span></td></tr> 157</td> if </span>[[ </span>-</span>f </span>"$</span>_PID_FILE</span>" </span>]] </span>&& </span>ps</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" ></span> /dev/null </span></td></tr> 158</td> then </span></td></tr> 159</td> elapsed</span>="$(</span>ps</span> -</span>o</span> etimes=</span> -</span>p </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")")" </span></td></tr> 160</td> </span>if </span>[[ </span>"${</span>elapsed</span>/</span>/ </span>/}" -</span>gt </span>$((</span>10 </span> </span>3600</span>)) </span>]] </span></td></tr> 161</td> </span>then </span></td></tr> 162</td> </span>log </span>"</span>Detected backup process going for over 10 hours, killing it...</span>" </span></td></tr> 163</td> </span>kill -9 </span>"$(</span>cat </span>"$</span>_PID_FILE</span>")" </span></td></tr> 164</td> </span>else </span></td></tr> 165</td> </span>log </span>"</span>Ongoing backup process detected. Aborting...</span>" </span></td></tr> 166</td> </span>exit 0 </span></td></tr> 167</td> </span>fi </span></td></tr> 168</td> fi </span></td></tr> 169</td> </span></td></tr> 170</td> # cleanup leftovers from last run if any </span></td></tr> 171</td> cleanup </span>></span> /dev/null </span>2</span>>&</span>1 </span>||</span>: </span></td></tr> 172</td> </span></td></tr> 173</td> # Save current process id to pid file </span></td></tr> 174</td> echo </span>"$</span>$</span>" > "$</span>_PID_FILE</span>" </span></td></tr> 175</td> </span></td></tr> 176</td> # cleanup will be run at the end of the script whether it fails or succeeds </span></td></tr> 177</td> trap </span>'</span>cleanup 2>&1 | log</span>'</span> EXIT </span></td></tr> 178</td> </span></td></tr> 179</td> # Make sure directories exist </span></td></tr> 180</td> mkdir</span> -</span>p </span>"$</span>_USER_FILES_BIND</span>" "$</span>_NC_INSTALL_BIND</span>" "$</span>_DB_BIND</span>" "$</span>USER_FILES_MOUNT</span>" "$</span>NC_INSTALL_MOUNT</span>" "$</span>SQL_DUMP_TARGET_DIR</span>" </span></td></tr> 181</td> </span></td></tr> 182</td> timestamp</span>="$(</span>date </span>"</span>+%Y-%m-%d %H:%M:%S %Z</span>")" </span></td></tr> 183</td> log </span>"</span>Starting Nextcloud Snapshot at </span>$</span>timestamp</span>" </span></td></tr> 184</td> </span></td></tr> 185</td> # Snapshot disks </span></td></tr> 186</td> log </span>"</span>Create snapshots...</span>" </span></td></tr> 187</td> </span></td></tr> 188</td> lvcreate</span> -</span>s</span> --</span>name </span>"$</span>NC_INSTALL_SNAPSHOT_NAME</span>" "${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_VOLUME</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 189</td> mount_snapshot_lvm </span>"${</span>NC_INSTALL_VG</span>}</span>/</span>${</span>NC_INSTALL_SNAPSHOT_NAME</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 190</td> </span></td></tr> 191</td> </span></td></tr> 192</td> lvcreate</span> -</span>s</span> --</span>name </span>"${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_LV</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 193</td> mount_snapshot_lvm </span>"${</span>USER_FILES_VG</span>}</span>/</span>${</span>USER_FILES_SNAPSHOT_NAME</span>}" "${</span>USER_FILES_SNAPSHOT_PATH</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 194</td> </span></td></tr> 195</td> log </span>"</span>Done.</span>" </span></td></tr> 196</td> </span></td></tr> 197</td> # Dump DB </span></td></tr> 198</td> log </span>"</span>Create DB dump...</span>" </span></td></tr> 199</td> "${</span>SQLDUMP_CMD</span>[</span>@</span>]}" > "${</span>SQL_DUMP_TARGET_DIR</span>}</span>/db.sql</span>" </span></td></tr> 200</td> log </span>"</span>Done.</span>" </span></td></tr> 201</td> </span></td></tr> 202</td> # Bind mount all individual snapshots into a common parent directory </span></td></tr> 203</td> log </span>"</span>Create bind mounts...</span>" </span></td></tr> 204</td> mount</span> --</span>bind</span> -</span>o </span>"</span>ro,uid=</span>${</span>NC_OWNER_UID</span>}</span>,gid=</span>${</span>NC_OWNER_GID</span>}" "${</span>NC_INSTALL_SNAPSHOT_PATH</span>}</span>/</span>${</span>NC_INSTALL_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_NC_INSTALL_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 205</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=</span>${</span>USER_FILES_OWNER_UID</span>}</span>,gid=</span>${</span>USER_FILES_OWNER_GID</span>} "${</span>USER_FILES_SNAPSHOT_PATH</span>}</span>/</span>${</span>USER_FILES_RELATIVE_PATH</span>#</span>/</span>}" "${</span>_USER_FILES_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 206</td> mount</span> --</span>bind</span> -</span>o</span> ro,uid=root,gid=root </span>"${</span>SQL_DUMP_TARGET_DIR</span>}" "${</span>_DB_BIND</span>}" </span>2</span>>&</span>1 </span>| </span>log </span></td></tr> 207</td> log </span>"</span>Done.</span>" </span></td></tr> 208</td> </span></td></tr> 209</td> # Create the final snapshot with kopia using the default kopia repository config (if needed, you can change the config file with the --config-file parameter) </span></td></tr> 210</td> log </span>"</span>Create kopia snapshot...</span>" </span></td></tr> 211</td> # Temporary log file as a workaround to pass kopia logs to our log function </span></td></tr> 212</td> kopia_logfile</span>="$(</span>mktemp</span> --</span>tmpdir</span> kopia-snapshot.XXXXX.log</span>)" </span></td></tr> 213</td> tail</span> -</span>f </span>"$</span>kopia_logfile</span>" & </span></td></tr> 214</td> kopia snapshot create</span> --</span>parallel</span>=</span>4</span> --</span>tags </span>"${</span>KOPIA_TAGS</span>}" --</span>start-time</span>="$</span>timestamp</span>" --</span>log-file</span>="$</span>kopia_logfile</span>" --</span>file-log-level</span>="</span>info</span>" "${</span>BACKUP_ROOT</span>}" </span></td></tr> 215</td> cat </span>"${</span>kopia_logfile</span>%</span>.log</span>}"</span>.log </span>| </span>log </span>></span> /dev/null </span></td></tr> 216</td> log </span>"</span>Done.</span>" </span></td></tr></tbody></table></code></pre> </div> </div> </context-condition> </sl-details> </parameterized-script> Running regular backups</h3> Now set up the script as cron job. If you want hourly backups (that's how I'm running it), you need to do nothing more than drop it into /etc/cron.hourly/</code>.</p> If your want a custom schedule, save it in a separate location instead (e.g. /usr/sbin/</code>) and create an entry in /etc/crontab</code>, e.g.:</p> </sl-icon> </sl-button> # minute hour day month weekday command </span> 30 /4 /2 * bash /usr/sbin/nextcloud-backup.sh </span></code></pre> </div> which will run the script every 4 hours at every 2nd day at the half hour mark.</p> Make sure to adjust the script permissions so it is executable and not readable by other users:</p> </sl-icon> </sl-button> chmod 0500 /usr/sbin/nextcloud-backup.sh </span></code></pre> </div> Restoring a backup</h2> Now, let's have a look at restoring a backup:</p> Reuse or setup a Nextcloud server</h3> In order to restore a snapshot, you need a Nextcloud server, that is working fine (apart from missing your data). So either set one up or reuse your existing server, if you're confident that only the backed up data (i.e. Nextcloud installation directory, user files directory or database) was damaged. This server needs to match your previous one (at the time of the backup) as closely as possible (e.g. the database and PHP version should be the same or at least compatible).</p> Restore a snapshot</h3> The first thing to do is, to have a look at the list of snapshots and restore one of them. Alternatively to using the given commands, we can also use the Kopia GUI</a>.</p> </sl-icon> </sl-button> # List the snapshots and find one that you want to restore </span>kopia snapshots list</span> --</span>all </span>| </span>less </span></code></pre> </div> </sl-icon> </sl-button> # If we're not on a fresh server, remove the existing directories that will be restored </span>rm</span> -</span>rf</span> /var/www/nextcloud/</span></span> /var/www/nextcloud/.</span>[</span>!</span>.</span>]</span> </span>rm</span> -</span>rf</span> /mnt/data/ncdata/</span></span> /mnt/data/ncdata/.</span>[</span>!</span>.</span>]</span> </span></code></pre> </div> </sl-icon> </sl-button> # Restore the snapshot to the original locations (see https://kopia.io/docs/reference/command-line/common/snapshot-restore/) </span># Let's assume the id of the snapshot we want to restore is "kffbb7c28ea6c34d6cbe555d1cf80faa9" </span>kopia snapshot restore </span>"</span>kffbb7c28ea6c34d6cbe555d1cf80faa9/nextcloud</span>"</span> /var/www/nextcloud </span>kopia snapshot restore </span>"</span>kffbb7c28ea6c34d6cbe555d1cf80faa9/nc_files</span>"</span> /mnt/data/ncdata </span>kopia snapshot restore </span>"</span>kffbb7c28ea6c34d6cbe555d1cf80faa9/db.sql</span>"</span> /run/nc-restore-db.sql </span></code></pre> </div> </sl-icon> </sl-button> # Finally, let's ensure, the restored directories have correct ownership. </span># Assuming your nextcloud webserver is running under uid=33, do: </span>chown</span> -</span>R</span> 33: /var/www/nextcloud </span>chown</span> -</span>R</span> 33: /mnt/data/ncdata </span></code></pre> </div> </div> </div> Now we need to restore the database from the restored /run/nc-restore-db.sql</code>:</p> PostgreSQL</sl-tab> MariaDB / MySQL</sl-tab> DB_USER</span>="</span>root</span>" </span>DB_NAME</span>="</span>nextcloud</span>" </span>DB_PASSWORD</span>="</span>password</span>" </span>DB_HOST</span>="</span>localhost</span>" </span>DB_PORT</span>=</span>5432 </span></code></pre> </sl-icon> </sl-button> # psql will be supplied with the connection options by writing them to ~/.pgpass </span>install</span> -</span>m</span> 0600 </span><(</span>echo </span>"${</span>DB_HOST</span>}</span>:</span>${</span>DB_PORT</span>}</span>:</span>${</span>DB_NAME</span>}</span>:</span>${</span>DB_USER</span>}</span>:</span>${</span>DB_PASSWORD</span>}") "$</span>HOME</span>/.pgpass</span>" </span>psql</span> -</span>d </span>"${</span>DB_NAME</span>}" < "</span>/run/nc-restore-db.sql</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> DB_USER</span>="</span>root</span>" </span>DB_NAME</span>="</span>nextcloud</span>" </span>DB_PASSWORD</span>="</span>password</span>" </span>DB_HOST</span>="</span>localhost</span>" </span>DB_PORT</span>=</span>5432 </span></code></pre> </sl-icon> </sl-button> # mysql will be supplied with the db credentials by writing them to ~/.my.cnf </span>install</span> -</span>m</span> 0600 </span><(</span>echo </span>-</span>e </span>"</span>[mysqldump]\nuser=</span>${</span>DB_USER</span>}</span>\npassword=</span>${</span>DB_PASSWORD</span>}") "$</span>HOME</span>/.my.cnf</span>" </span>mysql </span>\ </span> --</span>host</span>="${</span>DB_HOST</span>}" \ </span> --</span>port</span>="${</span>DB_PORT</span>}" \ </span> </span>"${</span>DB_NAME</span>}" < "</span>/run/nc-restore-db.sql</span>" </span></code></pre> </div> </div> </div> </sl-tab-panel> </sl-tab-group> That's it! If things in your setup changed, you might still have to adjust /var/www/nextcloud/config/config.php</code> ( e.g. db credentials, redis adress ... ). But other than that, your backup should be restored successfully at this point.</p> Conclusion</h1> As becomes clear, producing great backups can be a bit involved. But in my opinion, they are more than worth it, because once set, they provide a quality of sleep that's otherwise hard to achieve as sysadmin. :D Jokes aside, having robust backups means a significant reduction in the "worst case" that we could endure with our service (in regard to data loss) and are therefore crucial to have.</p> Revisiting our criteria</h2> Let's check how well we fared in terms of our criteria from part 1 of this blog series</a>:</p> protection from data loss </i> by accidental deletion solved by, well having backups</em>, I guess</li> </ul> </li> </i> from hardware failure solved by having offsite backups</li> </ul> </li> </i> from natural disaster solved by having offsite backups</li> </ul> </li> </i> caused by malicious actors solved by object locking</li> </ul> </li> </ul> </li> </i> reliable recovery process I'd say the recovery process is fairly straight forward</li> </ul> </li> </i> regular backups + long backup history we have setup a backup schedule and Kopia's retention policy according to our needs</li> </ul> </li> </i> minimal storage costs Thanks to Kopia's deduplication and compression features, our backups are very storage efficient</li> </ul> </li> </i> zero downtime backups By combining filesyste + database snapshots we did get away without downtimes</li> </ul> </li> </i> no increase in risk of data leaks Thanks to Kopia's client side encryption, we could even store our backups in otherwise uncompliant storage</li> </ul> </li> confidence in the presence and integrity of our backups Uh oh - looks like this is still unsolved</li> </ul> </li> </ul> As you can see - nearly all boxes are ticked! The remaining box will be addressed in the upcoming 3rd part of this series where I talk about backup monitoring. So stay tuned and consider following me on mastodon</a> or subscribing your feed reader to the RSS feed</a> to make sure, you won't miss it. :)</p> Awesome Backups 1 - What actually makes a good backup? 2025-09-25T00:00:00+00:00 I have been hosting Nextcloud for more than 8 years (and previously Owncloud), both for myself and for customers. During that time, backups have saved me often enouogh and I have continuously optimized my approach to them. In this series, I'll share what I learned and what results in a solution for Nextcloud backups which, at least for me, leaves nothing to be desired.</p> This first part covers the considerations behind my backup design. In the next article</a> I will demonstrate how to implement a backup process that fulfills the requirements discussed here.</p> What actually makes a great backup?</h1> Before jumping into setting up our backup process, let's take a second to think about what backups are actually supposed to achieve.</p> The main purpose of backups</h2> Backups are our last line of defense against data loss. Therefore, in order to figure out whether they serve their purpose, we have to think about are causes of data loss that we need to take into account.</p> Data loss by accident</h3> 8 years of hosting experience definitely means one thing: I have fucked up more than once. Be it a corrupted copy during update, a misconfigured Nextcloud client or a lost encryption key. It might be embarrassing, but pretending that those things don't happen would just be plain dangerous. It might not even be us - users have the capability to accidentally delete their own important data as well. Luckily, if we play our cards right and have strong backups we can use those to get away in such situations with a black eye.</p> </sl-details> Having any</em> sort of backup will safe us here. We just have to make sure that we backup frequently enough for our needs.</p> </sl-details> </div>Data loss from hardware failure</h3> A manufacturing issue, a power surge or just normal disk degradation - hardware failures are just a matter of time when running a server. And once they affect our disks, we are at risk of data loss. While we can (and definitely should!) protect ourselves from some cases of disk failure by disk health monitoring and RAID setups, we can't rule them out entirely. Especially if you have an issue with your power supply, that can cause hardware failures in multiple disks at once. So we really want backups that are not at risk of being affected by hardware failures in our server.</p> </sl-details> The key here is to decouple our backups from the hardware our service is running on. A separate backup disk is better than nothing, a separate server is better. And of course, we need to take care to use reliable storage in the first place, with builtin redundancy (e.g. RAID 5) and error correction.</p> </sl-details> </div>Data loss from natural disaster or theft</h3> Wherever our server is located - there's probably a nonzero chance of "natural" disasters like fire, flood, or similar. And then, depending on how much we're able to protect our server physically, there's also the chance for vandalism or theft. Mine is sitting in my flat, for example, so that's definitely a scenario I need to prepare for. Backups are part of the solution here (of course accompanied by strong disk encryption and information security).</p> </sl-details> To accommodate this, we need to go one step further: Our backup storage should be in a different physical location than our application server. Common recommendations are a minimum distance of 200km, but if you are running a private cloud, having a backup at a friend's place (ideally in a different city) is probably fine.</p> </sl-details> </div>Data loss caused by malicious actors</h3> So, now we're talking about the other kind of malicious actor, not the kind that breaks your front door and leaves with your server under their arm. Instead, this is about our server being compromised by attackers who are using security vulnerabilities in our setup. As demonstrated by the xz utils backdoor</a>, the log4shell</a> vulnerability in a popular java logging library or the regreSSHion</a> vulnerability in SSH, it's not possible to reliably rule out that our server will at any point be vulnerable. A compromised server can be abused by attackers in many ways, using harvested credentials to impersonate users, send spam mail, collect or leak valuable data and all of those are bad. However, there's one specific attack that caused an estimated 40-50 billion dollars of cost in 2024 and can be prevented by backups. I am, of course talking about ransomware, where attackers encrypt our data and try to blackmail us for a ransom to get it back. If we set up our backups with this scenario in mind, however, we can just purge our server, set it up from scratch and restore a backup, which will be annoying but by far better than paying the ransom. For completeness’s sake, though, I want to mention that ransomware attacks often also include risk of leaking sensitive data, which backups can't protect us from.</p> </sl-details> The key here is to setup our backup storage in a way that allows our server to write</em> backups, but not to delete or modify them (at least not before a specified retention time). One way to achieve this is by using advanced storage servers like object storage (e.g. Amazon AWS S3, Google Cloud Storage or open source alternatives like Ceph or Min-IO). Alternatively you could automate a privileged process on the backup location that changes a backups permissions so the writing client can't modify it anymore or move it to another location it does not have access to. Or you trigger the entire backup transfer from the backup server in the first place.</p> As you can see, this can be a bit tricky to setup. In the next part of this series I will give a detailed example for solving this with object storage.</p> </sl-details> </div> Reliable Recovery Process</h2> Of course, any backup is just as good as our capability of restoring it. So knowing exactly, how to do it when establishing a backup process is mandatory, as well as ensuring we will still have that knowledge when we need it.</p> </sl-details> We will be really happy to have good documentation on the restore process when we need our backups. Even better (if applicable) would be a script that can restore a backup in an automated fashion. Don't forget to test it, though! :)</p> </sl-details> </div>Regular backups + retention</h2> We want to achieve a satisfactory frequency of backup creation as well as duration to keep them. The former defines the maximum time period we can lose (time since last backup) and the latter defines the time we have to realize that there's an issue with our data before losing the chance to correct it. Both taken together define the absolute number of backups we want to keep. In an naive approach, where disk spaces scales linearly with the number of backups, this tends to quickly be a tradeoff between expensive storage costs and recoverability. However, I will present options to keep many backups with minimal storage costs.</p> </sl-details> This one is very straight forward: "Just do it". The caveat here is that regular and long retained backups will result in a large number of total backups and therefore potentially rising storage costs. So this is about striking a good compromise between storage cost and backup count - or about very storage efficient backups.</p> </sl-details> </div>Low Storage costs</h2> As just mentioned, we ideally want to keep many backups, so we need to keep an eye on our storage costs.</p> </sl-details> Our two best approaches to reducing storage requirements are deduplication (or incremental backups) and compression. This can be solved at the storage level or by your backup utility/process.</p> </sl-details> </div>Zero downtime backups</h2> As we want regular backups, we need backups without service downtime (If we back up more than once per day we might have a hard time explaining to our users, why our service is interrupted that often).</p> </sl-details> I'm afraid, there's no one size fits all here. How and if zero downtime backups are feasible largely depends on the service's infrastructure, specifically how it's state is managed. If the application state is mostly in traditional relational databases, you will probably be able to use single transaction copies or snapshot features to perform zero downtime backups, if it's mostly files, you can use filesystem snapshot features and in other cases you might want to rely on virtual machine/hypervisor snapshots.</p> In the next part of this series I will give a detailed example for solving this for a Nextcloud server.</p> </sl-details> </div>Functional backups</h2> When we need our backups, they should be present, complete and working, so we should find ways to ensure that. This might sound trivial, right after data loss is an unconvenient time to notice that your backup process has had an issue, but it will be the time we do notice it</em> if we don't take precautions.</p> </sl-details> The key to ensuring the integrity and functionality of our backups is robust backup monitoring. There are a number of things to take into consideration here:</p> What to monitor: Depending on the checks we perform on our backups we get vastly different guarantees: presence monitoring: We know there actually is</em> a backup, but we don't know if it works</li> integrity monitoring: We know the backup is still the same as we wrote it. This can be helpful to detect bit rot or disk failures (if we didn't detect them by other means)</li> restore tests: The gold standard is, of course, to attempt restoring the backup and see if everything is there and working. This may not be feasible though, depending on the complexity of our service and the size of our backup.</li> </ul> </li> Where the is monitoring running: Ideally, we would run it separately from both our application and our backup server. Alternatively we need some way to get notified if our monitoring itself is down (e.g. by a mobile app polling its status endpoint).</li> How we will notice if something isn't right: Monitoring is only half as valuable without good alerting. Our monitoring solution should have a way to notify us if our backups don't look right.</li> </ol> There's a lot to talk about here, which is why I will dedicate the third and final part of this series to backup monitoring.</p> </sl-details> </div> Don't leave the porch open</h2> Having backups should not increase our risk of data leaks. Therefore, we need to make sure that the backups are at least as secure as the service data itself.</p> </sl-details> We have basically two options here: Either we use backup storage that we trust (both in terms of security and compliance) or we securely encrypt our backups when writing them (in both cases we still need to trust in the capability and willingness of our provider to provide storage integrity and availability).</p> </sl-details> </div> Summary</h1> Alright, let's wrap up our goals:</p> protection from data loss </i> by accidental deletion</li> </i> from hardware failure</li> </i> from natural disaster</li> </i> caused by malicious actors</li> </ul> </li> </i> reliable recovery process</li> </i> regular backups + long backup history</li> </i> minimal storage costs</li> </i> zero downtime backups</li> </i> confidence in the presence and integrity of our backups</li> </i> no increase in risk of data leaks</li> </ul> That's certainly a lot of boxes to tick. A detailed walk through my approach to accomplishing this will be the topic of the next part</a> in this series.</p> You are welcome to follow me on mastodon</a> or subscribe your feed reader to the RSS feed</a> to make sure, you won't miss it. :)</p>