OPENTECHLY
OPENTECHLY

Welcome to the Opentechly Blog! You can find personal experience, trends, news and tips in the open source world, in a way that is engaging and easy to understand. Thanks for stopping by!

Habnai
Author

Habnai web developer, IT-tech, and tech junkie! I love to learn all there is to know about tech and programming, working to expand my knowledge every day.Feel free to contact me habnai@opentechly.com

Share


Tags


OPENTECHLY

Rdiff-backup: incremental backup...

Rdiff-backup is an easy backup tool ideal for hard disk, cross-platform backups, which allows you to perform incremental backup of files, directories or an entire tree to another directory that may be on the local machine or on a remote machine.for that i want share about this great tools that's i use for my personal backup.

Rdiff-backup backs up a single directory on a network using SSH ,indicating that the data transfer is encrypted.The exact copy of the target directory is removed from the target directory (on a remote system), however, additional reverse diffs are stored in a special sub-directory in the target directory, which makes it possible to recover previously lost files.

Install RDF backup in Linux

Important : If you're working on a network, you'll need to install both rdiff-back systems, preferably the exact version of both rdiff-backup installations.

The script already exists in the official repositories of the mainstream Linux distribution, just run the following command to install rdiff-backup and its dependencies.

In older Linux distributions the rdiff-backup versions are of the 1.x series, which is not recommended for new installs anymore. Follow the instructions below to install the latest 2.x release of rdiff-backup.

Ubuntu Focal or Debian Bullseye or newer (has 2.0)

sudo apt install rdiff-backup

Ubuntu backports for older versions (needs a backported 2.0)

sudo apt install software-properties-common
sudo add-apt-repository ppa:rdiff-backup/rdiff-backup-backports
sudo apt update
sudo apt install rdiff-backup

CentOS and RHEL 7 (From COPR)

sudo yum install yum-plugin-copr epel-release
sudo yum copr enable frankcrawford/rdiff-backup
sudo yum install rdiff-backup
sudo yum install py3libacl pyxattr
NOTE: the last line is optional to get ACLs and EAs support.

CentOS and RHEL 8 (From COPR)

sudo dnf install dnf-plugins-core epel-release
sudo dnf copr enable frankcrawford/rdiff-backup
sudo dnf --enablerepo=PowerTools install rdiff-backup
NOTE: you can add the option --setopt=install_weak_deps=False to the last line if you don't need ACLs and EAs support. You can install python3-pylibacl and python3-pyxattr also separately. Under RHEL, the repo to enable is codeready-builder-for-rhel-8-x86_64-rpms in order to get access to pyxattr, instead of PowerTools.

Fedora 32+

sudo dnf install rdiff-backup
NOTE: for earlier versions, see the COPR instructions below.

Debian and derivatives, Raspbian, etc. (from PyPi)

sudo apt install python3-pip python3-setuptools python3-pylibacl python3-pyxattr
sudo pip3 install rdiff-backup
NOTE: If your platform is not i386 or amd64, e.g. ARM/MIPS/etc, you may need the build dependencies build-essentials, librsync-dev.

CentOS and RHEL 6 (from PyPi)

sudo yum install centos-release-scl
sudo yum install rh-python36 gcc libacl-devel
scl enable rh-python36 bash
sudo pip install rdiff-backup pyxattr pylibacl
echo 'exec scl enable rh-python36 -- rdiff-backup "$@"' | sudo tee /usr/bin/rdiff-backup
sudo chmod +x /usr/bin/rdiff-backup

Fedora and derivatives (from PyPI)

sudo dnf install python3-pip python3-setuptools py3libacl python3-pyxattr
sudo pip3 install rdiff-backup
NOTE: If your platform is not i386 or amd64, e.g. ARM/MIPS/PowerPC/etc, you may need the build dependencies python3-devel, librsync-dev.

Other Linux (From PyPi)

You need to make sure that the following requirements are met:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
sudo pip3 install rdiff-backup

How to use RDF backup in Linux

As I mentioned earlier, rdiff -back uses SSH to connect to your network's remote machines, and SSH has a default authentication name / password procedure, which requiring common human interaction.‌‌However, in order to automate tasks such as automatic backups with scripts and beyond, you will need to set up SSH password-less logins using SSH keys, because SSH Keys increase trust between two Linux servers for easy file synchronization or transfer.‌‌Once you have set up an SSH Password-less Login you can start using the script with the following examples.

Backup Files to Different Partition

The example below will backup the /etc directory in a Backup directory on another partition:

sudo rdiff-backup /etc /media/aaronkilik/Data/Backup/mint_etc.backup

To exclude a particular directory as well as it’s subdirectories, you can use the --exclude option as follows:

sudo rdiff-backup --exclude /etc/cockpit --exclude /etc/bluetooth /media/aaronkilik/Data/Backup/mint_etc.backup

We can include all device files, fifo files, socket files, and symbolic links with the --include-special-files option as below:

sudo rdiff-backup --include-special-files --exclude /etc/cockpit /media/aaronkilik/Data/Backup/mint_etc.backup

There are two other important flags we can set for file selection; --max-file-size size which excludes files that are larger than the given size in bytes and --min-file-size size which excludes files that are smaller than the given size in bytes:

sudo rdiff-backup --max-file-size 5M --include-special-files --exclude /etc/cockpit /media/aaronkilik/Data/Backup/mint_etc.backup

Backup Remote Files on Local Linux Server

For the purpose of this section, we’ll use:

Remote Server 	        : 192.168.56.102 
Local Backup Server (backup) 	: 192.168.56.10

As we stated before, you must install the same version of rdiff-backup on both machines, now try to check the version on both machines as follows:

rdiff-backup -V
Backup Files to Different Partition
Backup Files to Different Partition

On the backup server, create a directory which will store the backup files like so:

mkdir -p /backups

Now from the backup server, run the following commands to make a backup of directories /var/log/ and /root from remote Linux server 192.168.56.102 in /backups:

# rdiff-backup root@192.168.56.102::/var/log/ /backups/192.168.56.102_logs.backup
# rdiff-backup root@192.168.56.102::/root/ /backups/192.168.56.102_rootfiles.backup

The screenshot below shows the root file on remote server 192.168.56.102 and the backed up files on the back server 192.168.56.10:

Check rdiff Version on Servers
Check rdiff Version on Servers

Take note of the rdiff-backup-data directory created in the backup directory as seen in the screenshot, it contains vital data concerning the backup process and incremental files.

Backup Remote Directory on Local Server
Backup Remote Directory on Local Server

Now, on the server 192.168.56.102, additional files have been added to the root directory as shown below:

Verify Backup Directory
Verify Backup Directory
rdiff-backup - Backup Process Files
rdiff-backup – Backup Process Files

Let’s run the backup command once more time to get the changed data, we can use the -v[0-9] (where the number specifies the verbosity level, default is 3 which is silent) option to set the verbosity feature:

# rdiff-backup -v4 root@192.168.56.102::/root/ /backups/192.168.56.102_rootfiles.backup
Incremental Backup with Summary
Incremental Backup with Summary

And to list the number and date of partial incremental backups contained in the /backups/192.168.56.102_rootfiles.backup directory, we can run:

# rdiff-backup -l /backups/192.168.56.102_rootfiles.backup/

Automating rdiff-back Backup Using Cron

We can print summary statistics after a successful backup with the --print-statistics. However, if we don’t set this option, the info will still be available from the session statistics file. Read more concerning this option in the STATISTICS section of the man page.

And the –remote-schema flag enables us to specify an alternative method of connecting to a remote computer.

Now, let’s start by creating a backup.sh script on the backup server 192.168.56.10 as follows:

# cd ~/bin
# vi backup.sh

Add the following lines to the script file.

#!/bin/bash

#This is a rdiff-backup utility backup script

#Backup command
rdiff-backup --print-statistics --remote-schema 'ssh -C %s "sudo /usr/bin/rdiff-backup --server --restrict-read-only  /"'  root@192.168.56.102::/var/logs  /backups/192.168.56.102_logs.back

#Checking rdiff-backup command success/error
status=$?
if [ $status != 0 ]; then
        #append error message in ~/backup.log file
        echo "rdiff-backup exit Code: $status - Command Unsuccessful" >>~/backup.log;
        exit 1;
fi

#Remove incremental backup files older than one month
rdiff-backup --force --remove-older-than 1M /backups/192.168.56.102_logs.back

Save the file and exit, then run the following command to add the script to the crontab on the backup server 192.168.56.10:

# crontab -e

Add this line to run your backup script daily at midnight:

0   0  *  *  * /root/bin/backup.sh > /dev/null 2>&1

Save the crontab and close it, now we’ve successful automated the backup process. Ensure that it is working as expected.

Read through the rdiff-backup man page for additional info, exhaustive usage options and examples:

# man rdiff-backup

credits:

rdiff-backup : http://www.nongnu.org/rdiff-backup/

Tecmint : https://www.tecmint.com/

Habnai
Author

Habnai

Habnai web developer, IT-tech, and tech junkie! I love to learn all there is to know about tech and programming, working to expand my knowledge every day.Feel free to contact me habnai@opentechly.com

View Comments