Backup on Linux: storeBackup / storeBackupRecover

You made a good start into 2010? What about new years pledges?
Here one thing, which you can add to your list:

• BACKUP / RESTORE for your private data
  (your company should already do backups ;-)
  

I am using storeBackup which has some really nice features:

• The first measure to decrease the necessary hard drive storage space would be the compression of data - if that makes sense. storeBackup allows the use of any compression algorithm as an external program. The default is bzip2.
• Within storeBackup, a hard link is used for referencing. With this trick of adding hard links, which were already created in existing backup files, each file is present in each backup although it exists physically on the hard drive only once. Copying and renaming of files or directories takes only the storage space of the hard links - nearly nothing.
• Unlike with traditional backups, there is no need to consider if an incremental backup is depending on previous backups. The options permit the deletion or saving of backups on specific workdays, first or last existing backup of the week/month or year.

So what do you have to do?

storeBackup -s /home/schroff -t /home/backup/truecrypt/schroff --progressReport 1000

and everything inside /home/schroff is backuped to /home/backup/truecrypt/schroff (if you like to exclude some directories use -e .wine for example). Here the output of the first run:

STATISTIC 2009.10.18 07:56:11 20965 [sec] | user| system
STATISTIC 2009.10.18 07:56:11 20965 -------+----------+----------
STATISTIC 2009.10.18 07:56:11 20965 process| 31.84| 59.91
STATISTIC 2009.10.18 07:56:11 20965 childs | 16.49| 0.84
STATISTIC 2009.10.18 07:56:11 20965 -------+----------+----------
STATISTIC 2009.10.18 07:56:11 20965 sum | 48.33| 60.75 => 109.08 (1m49s)
STATISTIC 2009.10.18 07:56:11 20965 directories = 8923
STATISTIC 2009.10.18 07:56:11 20965 files = 53726
STATISTIC 2009.10.18 07:56:11 20965 symbolic links = 417
STATISTIC 2009.10.18 07:56:11 20965 named pipes = 0
STATISTIC 2009.10.18 07:56:11 20965 new internal linked files = 0
STATISTIC 2009.10.18 07:56:11 20965 old linked files = 16
STATISTIC 2009.10.18 07:56:11 20965 unchanged files = 53260
STATISTIC 2009.10.18 07:56:11 20965 copied files = 10
STATISTIC 2009.10.18 07:56:11 20965 compressed files = 23
STATISTIC 2009.10.18 07:56:11 20965 excluded files because pattern = 0
STATISTIC 2009.10.18 07:56:11 20965 included files because pattern = 0
STATISTIC 2009.10.18 07:56:11 20965 max size of copy queue = 3
STATISTIC 2009.10.18 07:56:11 20965 max size of compression queue = 4
STATISTIC 2009.10.18 07:56:11 20965 calced md5 sums = 49
STATISTIC 2009.10.18 07:56:11 20965 forks total = 46
STATISTIC 2009.10.18 07:56:11 20965 forks md5 = 23
STATISTIC 2009.10.18 07:56:11 20965 forks copy = 0
STATISTIC 2009.10.18 07:56:11 20965 forks bzip2 = 23
STATISTIC 2009.10.18 07:56:11 20965 sum of source = 4.6G (4943663018)
STATISTIC 2009.10.18 07:56:11 20965 sum of target all = 3.8G (4131620957)
STATISTIC 2009.10.18 07:56:11 20965 sum of target all = 83.57%
STATISTIC 2009.10.18 07:56:11 20965 sum of target new = 10M (10961305)
STATISTIC 2009.10.18 07:56:11 20965 sum of target new = 0.22%
STATISTIC 2009.10.18 07:56:11 20965 sum of md5ed files = 14M (14712199)
STATISTIC 2009.10.18 07:56:11 20965 sum of md5ed files = 0.30%
STATISTIC 2009.10.18 07:56:11 20965 sum internal linked (copy) = 0.0 (0)
STATISTIC 2009.10.18 07:56:11 20965 sum internal linked (compr) = 0.0 (0)
STATISTIC 2009.10.18 07:56:11 20965 sum old linked (copy) = 405 (405)
STATISTIC 2009.10.18 07:56:11 20965 sum old linked (compr) = 411k (420508)
STATISTIC 2009.10.18 07:56:11 20965 sum unchanged (copy) = 2.4G (2545886135)
STATISTIC 2009.10.18 07:56:11 20965 sum unchanged (compr) = 1.5G (1574352604)
STATISTIC 2009.10.18 07:56:11 20965 sum new (copy) = 19k (19126)
STATISTIC 2009.10.18 07:56:11 20965 sum new (compr) = 10M (10942179)
STATISTIC 2009.10.18 07:56:11 20965 sum new (compr), orig size = 14M (14272160)
STATISTIC 2009.10.18 07:56:11 20965 sum new / orig = 76.70%
STATISTIC 2009.10.18 07:56:11 20965 size of md5CheckSum file = 1.7M (1810889)
STATISTIC 2009.10.18 07:56:11 20965 size of temporary db files = 7.6M (7921664)
STATISTIC 2009.10.18 07:56:11 20965 deleted old backups = 0
STATISTIC 2009.10.18 07:56:11 20965 deleted directories = 0
STATISTIC 2009.10.18 07:56:11 20965 deleted files = 0
STATISTIC 2009.10.18 07:56:11 20965 (only) removed links = 0
STATISTIC 2009.10.18 07:56:11 20965 freed space in old directories = 0.0 (0)
STATISTIC 2009.10.18 07:56:11 20965 add. used space in files = 12M (12772194)
STATISTIC 2009.10.18 07:56:11 20965 backup duration = 3m12s
STATISTIC 2009.10.18 07:56:11 20965 over all files/sec (real time) = 279.82
STATISTIC 2009.10.18 07:56:11 20965 over all files/sec (CPU time) = 492.54
STATISTIC 2009.10.18 07:56:11 20965 CPU usage = 56.81%
END 2009.10.18 07:56:11 20965 backing up directory </home/schroff> to </home/backup/2009.10.18_07.52.59>

The first execution ran for several hours because all files had to be compressed. A run few days later just finished within several minutes and consumed only few space more...
And recovery? Just use storeBackupRecover.

This program recovers files saved with storeBackup.pl.
usage:
storeBackupRecover -r restore [-b root] -t targetDir [--flat]
[-o] [--tmpdir] [--noHardLinks] [-p number] [-v] [-n]
--restoreTree -r file or (part of) the tree to restore
when restoring a file, the file name in the backup has
to be used (eg. with compression suffix)
--backupRoot -b root of storeBackup tree, normally not needed
--targetDir -t directory for unpacking
--flat do not create subdirectories
--overwrite -o overwrite existing files
--tmpdir -T directory for temporary file, default is </tmp>
--noHardLinks do not reconstruct hard links in restore tree
--noRestoreParallel -p max no of paralell programs to unpack, default is 12
--verbose -v print verbose messages
--noRestored -n print number of restored dirs, hardlinks, symlinks, files
Copyright (c) 2002-2004 by Heinz-Josef Claes
Published under the GNU General Public License