previous article: The Perfect Backup Solution, part 1: CYGWIN

This is the second part of the "Perfect Backup" set of articles where I discuss the backup solution for Windows and Linux OS (local and remote) that works perfect for me. Sharing it here in hope that it might be helpful to someone..

The topic of this article is:  offline or local or disk backup for your computer.

To me, perfect backup means this:

It works when I tell it to work: in my case, I really DON'T like constant backup. Having a software program running constantly in the background makes my computer work slow. I want to click and run the backup, or leave the computer switched on and let the backup start automatically at nights.

Makes a perfect copy of my data: I DON'T like compressed directories as a backup option. So many times in the past I needed for one reason or another to access my backup data only to find out that it's compressed. Also, sometimes I want a few versions of older files to be kept in a separate folder.

Is fast and stable:  I have 1TB of data to be checked/backed up; also, I need to know that the software performs the same task in the same way every time I click a button (or enter a command). Believe me, I used to have backing software that would suddenly backup in a different way after a few months.

Is Linux and Windows OS compatible: I started my computing life with Windows, switched to Linux later. But at my work I still use Windows - so the software needs to be multi-platform.

After trying numerous software solutions, both for Windows and Linux I finally settled with RSYNC.

RSYNC does everything required above and much, much more. It's an amazingly sophisticated yet simple to use and understand piece of software.

RSYNC does not have windows, or GUI (Graphical User Interface). It's a command-line program, meaning it runs from a terminal. It comes pre-installed in Linux and Apple OS, and in Windows it can be installed and used easily via CYGWIN. CYGWIN is a Linux-like terminal app for Windows that comes with pre-configured Linux apps ready to be installed and used on Windows OS.

But exactly this lack of GUI is what makes RSYNC so fast. There's no processing power used to draw nice pictures on your computer - instead, all power goes to the execution of the command. And computing on a terminal is one of the most exiting things to do, anyway:-).

(Still, if not convinced, there's a program called GRSYNC that adds a GUI to RSYNC. Available for Linux and Windows. Scroll to the bottom of this page...)

Installation

In Perfect Backup Part 1 I discuss the installation of CYGWIN on Windows (and enabling RSYNC) and Linux (in case it isn't installed). Windows users, please read the last few paragraphs of that article where Windows (C:\folder\) versus Linux (/c/cygdrive/folder/) directory paths are explained in more detail.

Usage

Rsync can be used to backup to a local as well as a remote destination. But for online backup I found that LFTP does a great job and so I'll limit my discussion here to local backup.

So in the terminal we need to input a sentence that executes/awakes RSYNC and lets it know what exactly we want it to do and how (options), what to copy/backup (source) and where to (destination).

Thus, the command sequence will follow the following template:
rsync -options --option2 --option3  /source/folder/ /destination/folder/

The rsync at the beginning of the sequence is to tell the OS that the following sequence needs to be executed by RSYNC

Basic Options

All options are preceded with one or two dashes. One letter abbreviations use one dash (-a), while words use two dashes (--archive). We can have a few letters grouped together using only one dash (ex: -avvhi)

-a The main option we use for backup is -a. The letter "a" stands for archive, and in itself it's an abbreviation (syntactic sugar) for a few functions consisting of:

• preserve owner
• preserve permissions
• preserve time

and a few other useful backup options. This article provides a nice detailed analysis and description of the -a (--archive) function in RSYNC.

-v (-vv) stands for "verbose". RSYNC can be quite dull. If you input the command:
rsync -a /source/ /destination/
RSYNC will do its job perfectly, but silently, without any signs that it's working. I personally don't like staring in an empty terminal waiting for a program to finish. It's no fun. The "verbose" option will let RSYNC communicate with us as to what's going on at the moment. The double v (-vv) will let it speak a bit more.

So, the most used command for backup on RSYNC is:

rsync -av /source/folder/ /destination/folder/
or
rsync -avv /source/folder/ /destination/folder/
for a bit more detail about RSYNC's whereabouts.

This command is good when making the first backup. When repeating the operation, the destination folder needs to be an exact copy of the source folder (and its subfolders, of course).

This is why the option --delete needs to be added. It tells RSYNC  to delete all files and folders that are not present in the source folder.

rsync -avv --delete /source/folder/ /destination/folder/

So, the above is the most common command for backup on RSYNC.

Source (and destination)

The important part here is the last slash of the source folder path. Depending of whether you put it or not, you will have two slightly different outcomes:

• WITHOUT the slash: ( /source/folder ) will copy the actual source folder (and all its files) into the destination folder.
• WITH the slash: ( /source/folder/ ) will copy all the files and folder from the source folder into the destination folder

The last slash of the destination folder can be added or left out - it makes no difference.

If I am on a Linux machine and I want want to backup my system files (or other files where I'm not the owner), then I either need to become a root user or (in *buntu systems) precede the command with sudo:

sudo rsync -avv --delete /source/folder/ /destination/folder/

Interesting Options

Here are some options that are not so useful, but make computing more interesting

• --progress  (show progress during transfer, interesting for large files)
• -h --human-readable (file sizes; you’ll get 100mb instead of 102400k)
• --stats (give some file-transfer stats at the end of the transfer)
• -i --itemize-changes output a change-summary for all updates

So, my personal RSYNC sequence (on Linux) looks as follows:
sudo rsync -avvhi --delete --progress --stats /source/folder/ /destination/folder/

Final Tips and Tricks

You can set RSYNC not to delete all changed/old versions of your files and folders, but to put them in a separate folder using the following option:
-b --backup-dir=/backup/folder/
(the /backup-folder destination must be outside the source or destination paths)

rsync -avvhib  --backup-dir=/backup/folder/ --delete --progress --stats /source/folder/ /destination/folder/

For Linux users, there is an RSYNC version with a GUI: GRSYNC
(sudo apt-get install grsync)

GRSYNC is now ported on Windows as well.

CWRSYNC:  a yet another packaging of Rsync and Cygwin

RSYNC documentation: as I said earlier, these are only basic options. RSYNC  is a fine and sofisticated app that has many interesting options. Read the documentation and discover a world of possibilities hidden behind the word "rsync" typed on a terminal. In Linux, simply type rsync -man to get the manual printed on the terminal.

You can execute rsync with a script. This same article again gives a nice info on how to do it.

Have fun computing.