Full Circle Magazine FR

Way back in 1984, I was preparing an important sales proposal using an early PC that used notoriously unreliable floppy diskettes for both software and data. When I tried to open and print the 8-page proposal, the diskette could not be read. I tried the diskette on a different PC; no luck. Cleaning attempts were futile. The customer came to pick up the proposal a few hours later, but he could not wait until the next day when the proposal could be re-created. The loss of that $6,000 sale taught me how vital backups can be. Over the next three decades, I made backups on diskettes, digital audio tape (DAT), CDs, DVDs, flash drives, and external hard drives. These backups used a variety of backup software under DOS, OS/2, Windows, and Linux. Those experiences taught valuable lessons in what “thou shalt” and “thou shalt not” do. I gladly share my lessons learned to save you the “tuition” in the College of Hard Knocks or the University of Adversity.

Il y a très longtemps, en 1984, je préparais une proposition de vente importante sur un PC d'époque qui utilisait des disquettes, connues pour leur manque de fiabilité, pour, à la fois, les logiciels et les données. Quand j'ai essayé d'ouvrir et d'imprimer la proposition de 8 pages, la disquette était illisible. J'ai essayé la disquette sur un autre PC, sans succès. Mes tentatives de nettoyage furent vaines. Le client est venu chercher la proposition quelques heures plus tard, but ne pouvait pas attendre le lendemain quand la proposition pouvait être recréée. La perte de cette vente de 6 000 $ m'a appris à quel point il est vital de faire des sauvegardes.

Pendant les trois décennies suivantes, j'ai fait des sauvegardes sur disquette, sur Digital Audio Tape (DAT - support d'enregistrement et lecture numérique sur bande magnétique mm conçu par Sony, voir fr.wikipedia.org), sur CD, DVD, clés USB et disques durs externes. Ces sauvegardes étaient faites avec toute une panoplie de logiciels de sauvegarde sous DOS, OS/2, Windows et Linux. Ces expériences-là m'ont enseigné des leçons essentielles sur ce que je devais faire et ce qu'il ne fallait point que je fasse. Je suis heureux de partager ces leçons pour vous éviter les « frais d'inscription » à l'Université de l'Adversité.

1. Have more than one backup. Murphy’s Law is more reliable than any backup. When you need them most, your backup file(s) may be unreadable… damaged medium, interrupted backup, infected file, etc. Grandfather or archive your backups to provide additional protection; more on grandfathering later. 2. Keep your backups somewhere other than on the top of your PC tower. A burglar, fire, or natural disaster (flood, tropical storm, earthquake, etc.) may condemn your backups to the same fate as your PC. Have at least one or two backups off-site so you will have something to retrieve after almost any disaster. Off-site locations include your office, attorney’s office, a friend’s or employee’s home, or a bank safe-deposit box. 3. Do not assume that if the backup worked fine, you will be able to restore without any problem. Test the restore before it is needed. I prefer to avoid backup utilities to reduce possible points of failure. A good alternative is to use rsync (Remote Sync), which is included in Linux distros. Rsync can copy and synchronizes files and directories both locally and remotely. When rsync is used to make backups, there is no need to keep a copy of a restore program with your backups.

4. Match the backup medium to the size and frequency of backups. Factors to consider when choosing what medium to use for storing your backups include capacity, speed, durability, reusability and cost. For backups of only a few GB of data, DVD-R discs are a great option. If the backups will contain large databases or media files (many GB), then an external USB hard drive may be a better option. 5. Base both the type and frequency of backups on the number of file additions/modifications and on how difficult/expensive it would be to recreate lost files if they are not available from a backup. People use computers in many different ways. To illustrate a solid backup strategy, let’s use the files on my PC as an example. The files on my PC can be classified in three groups: A. System files and Apps – This includes Ubuntu, Firefox, LibreOffice, Thunderbird and other software (24 GB). These files change only when applications are installed or removed, when software updates are applied, or when some configuration changes are made. B. Photos, audio files, video files – These are usually large files that are added or edited only rarely – typically a few times a month (88 GB). These files are in subdirectories under the /home directory).

C. Non-media data files – Scanned files, downloads, word processing, spreadsheet, and database files, financial records and correspondence (19 GB). These files are in the /home directory and its subdirectories). Typically, a few hours each day may be spent adding or updating files in this group. To keep life simple, no differential or incremental backups are used. All backups are “full,” but filters are used to determine which files will be backed up. There are three types of backups: Daily – Group C only Weekly – Group B and Group C Monthly – Groups A, B and C Despite their names, any one of those backups can be created whenever desired. I use 2 TB external hard drives for these backups, regardless of backup type; more about backup drives later. This is based on my situation. Your style of PC usage will likely be different and you may make adjustments to these guidelines to better suit your needs.

6. When looking for a particular backup, the search is made easier when the backup name indicates whether it is a daily, weekly, or monthly backup and when the backup name includes a datestamp. 7. Separate scripts are used to create each type of backup. Give each script a descriptive name to indicate what type of backup it will produce: e.g. BackUp-Monly.sh (here, Monly means ‘Monthly’). Scripts Let’s look at what the scripts do, then the actual code will follow. The first line of each script sets a variable named BUDATE for use in naming the backup directory. The second line makes a directory on an external hard drive. My external hard drive is named SUNbackup—Ext4 The directory, for a backup at 2:15 AM on Dec. 5 2016 would use BUDATE to create a file name similar to this: Monly BU of 2016.12.05 02:15:00 The third line provides a three-second pause to allow completing the creation of the backup directory.

The fourth (very long) line uses rsync to copy the files. This fourth line should be entered in the script as one line without wrapping. Here are the five actions that the rsync line performs (a – e below): a) Since this script requires super-user (su) privileges to perform the backup, echo is used to send the su password to avoid the need to enter a password manually when performing backups. Replace the supasswordhere with a valid su password. Likewise, in this example script, replace all occurrences of SomeUserName with the username of the directory (under /home) to be backed up. Finally, replace SUNbackup—Ext4 with the name of the drive that will hold the backup files. b) Some of the rsync options in these three bash backup scripts are obvious, but to help you understand the options that may not be obvious, below is a list of the options used in these scripts. For a comprehensive list of rsync options and a more complete understanding of how rsync works, in a Terminal window, type man rsync <Enter> and scroll down to the heading OPTIONS SUMMARY.

c) A list of all the files backed up, and some backup summary data, are written to a log file named /home/SomeUserName/Documents/BU-log.txt The entries in this plain-text log file are cumulative from multiple backups. So periodically you should delete the BU-log.txt file (it will be created again during the next backup), block mark and delete the old entries in that file that you no longer need, or simply delete the rsync –log option from this script to cancel logging. d) Notice that the scripts below use the –exclude option to filter out several file types that you would not want to restore (such as caches, trash, lost+found, and others). The –exclude command is used in the Daily script to exclude media files (Group B above) from the daily backups. e) Before running/executing the scripts, in their File Properties dialog boxes, on the Permissions tab, set the Access permission to Execute for anyone permitted to make backups – Owner, Group or Others. The scripts can be executed manually or they can also be executed automatically at desired times by using a utility such as Cron or Gnome-schedule.

The Backup Drives The number of backup drives and the frequency of various backup types may be customized for each system being backed up to provide adequate protection from various risks. Costs and time to perform backups are also factors to consider when choosing the medium for backups. Here is a real-life plan that is used to back up the files described under item 5 above. This plan may be modified to suit your needs. For example, a five-day-per-week PC user would make five, not six or seven, backups each week. Or you may decide to use your existing flash drives rather than external hard drives. Start with three fast (USB3) external 2-TB hard disks if your PC supports USB3. Each drive costs about £65-UK or $90-US. If your PC does not support USB3, choose a hard disk type or other medium that your PC does support.

On day 1, back up files to disk 1 On day 2, back up files to disk 2 On day 3, back up files to disk 3 On day 4, back up files to disk 1 On day 5, back up files to disk 2 On day 6, back up files to disk 3 Continue this process day-by-day (or whatever interval you prefer), remembering to switch from Daily backups to a Weekly backup at the end of each week and a monthly backup at the end of each month. When backing up files like those described under item 5 above: • my Daily backups of Group C would be about 20 GB • the Weekly backups of Groups B and C, about 90 GB • the Monthly backups of Groups A, B and C, about 115 GB. A recent backup hard disk should be stored off-site in case an event occurs that destroys the backup hard disks along with the PC. When one of the three disks is stored off-site, the above rotation will involve only two hard disks instead of three.

After about six or seven weeks of daily backups, a backup hard disk will be nearly filled. When that happens, the disk should be reformatted as you would a new disk and then returned to use until it is filled again. Grandfathering Backups Grandfathering backups provides even greater protection, especially against zero-day threats, ransomware, remote access Trojans, and files damaged or deleted in error but not discovered until months afterwards. To implement grandfathering, one additional external hard disk is required. On a regular basis (such as quarterly or semiannually) make a Monthly backup to this additional drive. A 2TB hard drive will hold about 16 such backups (four years of quarterly backups). Sometime in year four, delete the first three quarterly backups from the first year, leaving the fourth quarterly backup. That fourth quarterly backup becomes the sole (annual) backup for the first year.

When the available space on the external hard disk will allow adding only one more quarterly backup, delete the first three quarterly backups from the second year, leaving the fourth quarterly backup. That fourth quarterly backup becomes the sole (annual) backup for the second year. Repeat this process each year until the disk is nearly full. Then you can either add another hard drive or develop a new backup strategy. By then, the technology and options will be very different than they are today. Restoring from a Backup Restoring is simple. Use cp (copy) from the command-line interface or use your favorite file manager to drag-and-drop from the backup directory to the destination directory. Remember, some files or directories may have restricted permissions that require super-user privileges to restore by copying. Happy, dependable backups and peaceful dreams! The follleowing are comments by Michael Kennedy, and not part of Randy's original article.

Additional observations by Michael Kennedy: For backups of Databases, Financial records, etc, note that new versions of the software may be unable to process the backup files from older versions of the software. In that case, either: • Retain the old software versions in a usable state, or • Ensure you can convert the old files, should you need to restore them, or • If you’re happy to do so, you can treat the old backups are useless, and discard them. For Grandfathering, another rotation system could be adopted (I call it a Binary system but perhaps the “professionals” have a better title!). This approach might not be suitable in Randy’s situation - which has three separate file Groups - but it might be very relevant if a backup script simply backs up ALL important changed files - on every run. It’s more relevant where full backup ets are being held, rather than incremental or differential backups. Basically: • Disk-1 is used for every SECOND backup run. • Disk-2 is used for every FOURTH run. • Disk-3 is used for every EIGHTH run. • And so on. Eg: on successive backups, use Disk #1, #2, #1, #3, #1, #2, #1, #4, #1, #2, #1, #3, #1, #2, #1, etc.

As Randy says, some backups should be held off-site. There are some advantages… The exact same procedure is always used, for all backups. Extra backups can be run at any time (eg, working weekends), or skipped (eg, national holidays). If backups are usually run daily, then about 8 disks will reach back about 6 months, 9 disks will cover about 1 year, 10 disks about 2 years, etc. The main downside is that someone has to initially PLAN the overall approach, eg: the exact rotation, who-does-what-when logs, how to verify backups, how to initialise all disks initially, how to replace a faulty/lost disk, how to introduce another disk into the rotation, the off-site procedures, etc. Also, some media are used very frequently, and some very rarely, and this aspect should be considered - eg, wear-and-tear on tapes.