Filter what to backup with rsync

August 11th, 2007 Felix Leong Posted in Misc, rsync | Hits: 85871 | 5 Comments »

It’d be easy to rsync if you have everything in one folder with simple folder hierarchy, but when it comes to things like backing up your /home folder, you’d might want to be selective to what to backup. Rsync provides this function via the –include=* and –exclude=* command line options, however, if you want to maintain a list of what to filter over time (say, you want to set up an automated backup script), it’d make more sense keeping your filter rules in a separate file and have rsync read it by using the –include-from=* and –exclude-from=* command line option.

So now let us just define a simple text file to store your filter rules:

+ task/
+ *.py
+ *.odt
- *

A quick explanation on the format: Each line defines a rule, which the rule have the format of RULE(space)FILENAME. So for this example, the RULE being used is + (include) and – (exclude). The FILENAME portion can then be a directory or file name, which directory names required to be denoted by appending the / (backslash) character.

FILENAME can contain wildcard characters so it’d extremely useful if you want filter your backup by file type. Notice that we used two wildcards here: one being *, the other being **. The wildcard * would mean it’ll match all non-path components (i.e. it’ll stop at the / character). For instance, the + *.odt rule will include “document.odt” for backup but not “directory/document.odt”. To make rsync recursively find all files with the particular file type, you’d have to use the ** wildcard. For example **/*.cppwill match all “test.cpp”, “level1/test.cpp” and “level2/level3/test.cpp”.

So decoding our filter rule, it’d read: 1. only search in the . (current directory, defaulted) and task directories for backup files; 2. Only backup *.py and *.odt files from these two directories; 3. Exclude everything else

Once you have this rules file (assume we saved it as “filelist.txt”), we can execute our rsync command to use this set of filters like this “rsync –exclude-from=filelist.txt /home/felix felix@192.168.1.280:backup/”. Now, having /home/felix being the base directory which will backup from, the filter rules will be executed for files in /home/felix.

If you are interested to learn more complex filter rules, do read the FILTER RULES and INCLUDE/EXCLUDE PATTERN RULES in rsync’s man page.

5 Responses to “Filter what to backup with rsync”

  1. Phil Marshall Says:

    >> denoted by appending the / (backslash) character

    This character, “/”, is the “slash” character.
    This character, “\”, is the “backslash” character.

    Sorry, but this is one of my pet peeves…

  2. good stuff. Thanks.

  3. Thanks, this helped me wrap my mind around the –exclude-from option.

  4. Anyway I’m adding this RSS to my e-mail and can look out for a lot more of your respective fascinating content. Make sure you update this again soon.. Get the Drive to Accomplish What You Want Now http://archjpres.com/post/test-post#comment-105178

Leave a Reply