Volume Number: 24 (2008)
Issue Number: 08
Column Tag: Backups

Creating a Backup Policy

Design and document an effective backup system for your business

by Joe Kissell

The president of a company I once worked for was fond of bragging that even if the headquarters building burned down overnight, all the organization's data would be safe and business could go on more or less as usual tomorrow. He slept soundly knowing that the company's computers were backed up daily and that a current set of data tapes was kept in a secure offsite location. Fortunately, the company never experienced a devastating data loss that put this system to the test, but at least someone had carefully thought through the matter of backups and prepared a disaster recovery plan.

I had to wonder, though, if all the company's data was truly as safe as my boss thought. No one ever told me, an ordinary employee, how the backup system worked. Were the Mac and PC on my desk among the computers that were backed up regularly? If so, how-and how often? (I was unaware of any backup software running on them.) What would I have had to do if I needed to restore a lost file? Did the company's IT people ever check to make sure the data on those tapes was really recoverable?

These are just a few of the things that should be spelled out in a backup policy-a brief document that describes, concisely and in plain English, how your organization deals with backups. Although it may be left to the IT staff to sort out the technical details and implement the policy, everyone in the organization who creates or uses digital data should know what the policy says or at least where to find it.

Of course, if you're talking about backing up a handful of Macs in your home, you don't need a policy at all; you only need a plan, which could be as simple as "Buy Time Capsule. Plug in. Activate on all Macs." However, what works well for individuals, families, and small offices doesn't necessarily scale to larger businesses.

In this article I discuss many of the considerations involved in creating a backup policy for your organization. The point of developing a formal policy is not merely to spell out what you already do, but to think through a number of important issues that are often overlooked. If your organization has a backup system in place already, working through a policy statement can reveal holes and shortcomings that should be addressed. If you have no backups, a solid policy can serve as a guide to set up the right sort of system, saving time, money, and aggravation later on.

Let me be blunt: an awful lot of people-not only technologically clueless executives but even otherwise savvy IT guys-take a completely backward approach to backups. They learn about some nifty piece of hardware or software, get it in their heads that it's the answer to all their backup needs, and then try to design a plan around that toy. This can be seriously bad news; your company can get stuck with completely inadequate backups despite using what appears to be the latest and greatest (or, at the other extreme, tried and true) technology. The much wiser approach is to decide first exactly what you want to accomplish, and then go find the tools that enable you to get that done.

Needless to say, we don't live in a perfect world, and any number of situations might force you to make do with less-than-ideal hardware or software. Even if you know your hands are tied with, for example, a legacy hardware system and a tiny budget, I still recommend crafting your policy initially as though those limitations did not exist. By the time you're done, you may discover a workaround. At the very least, you'll be able to demonstrate a gap between what your company needs and what you can provide. If you can show the Powers That Be how effective and crucial your ideal policy is and then point out the gaps between that and what you currently have, you never know-someone just might be able to authorize an infrastructure change or find more money to spend on solving the problem.

With that said, let's look at the three major categories of information your backup policy should contain: backup procedure, media management, and data restoration. Although I go into some detail here about each aspect of your policy, keep in mind that we're looking for a concise end result-a brief outline or maybe a dozen bullet points that should fit on a single sheet of paper.

Your backup procedure encompasses several main points: what data you back up, using what method(s), how often, and to what media. Each one of these matters involves a number of decisions.

Without a doubt, one of the most important things your backup policy should state is precisely which data is backed up. Some people would immediately say "all of it," but that's not necessarily a smart thing to do. As the amount of data you back up increases, so does the time it takes for each backup to run, as well as the amount of storage space needed-not to mention network congestion. All of this may not be a big deal when you have half a dozen computers, but it's a nightmare waiting to happen when you're talking about hundreds or thousands.

You probably do not need to back up the operating system and preinstalled applications for every single computer on your network. For one thing, this could result in massive duplication of files (though admittedly, some backup software is smart enough not to store multiple identical copies of a given file). For another, it's probably unnecessary in that your IT department likely has preconfigured startup drives, or even spare computers, that could be used in the event of a serious disk failure. In most cases, it should be sufficient to back up individual user data-as in the user's home folder on Mac OS X or Windows, or even a subset of that folder, depending on where and how your users store their data. If your network is set up in such a way that home folders are stored on a server, you may not need to back up individual workstations at all.

Users may have some files that should not be backed up for one reason or another-for example, Photoshop scratch files, caches, or personal files that just happen to live on a company machine. It's helpful, therefore, to state explicitly in your policy not only what's included in a backup, but also what's excluded. Your policy might state that all files put in a designated "Do Not Back Up" folder will be omitted from backups, or that MP3 files and photographs, regardless of their location, won't be backed up (if they're deemed to be irrelevant to your business).

The situation changes, though, when talking about servers rather than individual users' workstations. You very likely do want every single file on your servers to be backed up, assuming that the servers are crucial to the operation of your business and that only business-related files reside there. You may already have mirrored RAIDs in place to reduce the risk of downtime due to disk failure, but a RAID by itself is not the same thing as a backup. If you have a multi-drive mirror and you periodically swap out drives, then you do have some additional insurance against accidental file changes or deletions. You may also want to create separate bootable duplicates of your servers' drives on a regular schedule, or simply include the server's drives in your regular archiving scheme. In any case, spell this all out in your backup policy.

Be aware that, depending on the size, structure, nature, and location of your business, you may be required by law to maintain copies of certain kinds of data for a specified period of time. Before setting your backup policy in stone, consult with a legal expert to determine which data you must be certain to include in your backups.

When I say "method" here, I'm referring to full versus incremental or differential backups. (Although "incremental" and "differential" mean different things to different people, one common usage is for a differential backup to contain all the data that changed since the last full backup, while an incremental backup contains only the data that changed since the last update. Incremental backups run faster because they contain less data, but differential backups may be easier and quicker to restore, especially if you use tape drives.) Your policy should state under which circumstances one method or another is used. For example, you might specify that a full backup occurs once a month with incremental backups twice a day and differential backups once a week. If the method is different for servers than it is for workstations, say what those differences are.

Your backup policy should state how frequently backups run, whether that's several times a day, a couple of times per week, or whatever. Backups are often scheduled to run after hours, when they'll have the least impact on users' work. Although you can't precisely state how long a backup will take, you can say something like "Backups begin every morning at 2:00 a.m." or "Backups run between midnight and 6:00 every weekday morning."

Increasingly, backup software that runs only on a fixed schedule is becoming "old school." Some backup software constantly watches for changes and copies new or altered data after a short delay (say, 15 minutes). Other software simply checks to make sure any given computer was backed up at least once within, for example, the last 24 hours-providing flexibility for users of laptops or other computers that aren't always available on the network.

One way or another, be sure your policy (and the software choices that you make as a result of it) can accommodate mobile users. If your policy states that backups of networked computers always and only occur at three in the morning, laptop users might never get backed up. Explicitly state whether or not network backups will occur while users are outside the office-and if so, whether remote users have to do something special to make that possible (such as connecting to a VPN).

In most large organizations, high-capacity tape drives of one sort or another are taken for granted as a backup medium, often with an automated loading and retrieval system. That may indeed be the best choice for your business, but it's not the only option. In particular, given the rapidly rising capacities and falling prices of hard drives, you may find that some sort of hard drive array is just as economical, while providing much faster performance, especially for restoring files (since the data can be read nonlinearly). Your written backup policy can perhaps be worded in a generic way to accommodate potential changes in the media you use, but think through the implications carefully. For example, the schedule and means by which sets of backup media are rotated offsite might be very different for a hard drive array than for a stack of tapes.

The next section of your backup policy should discuss how your physical backup media is handled. This includes rotating among multiple sets of media; recycling, replacing, or destroying old media; storing backups offsite; and keeping your backup media encrypted.

All backup media is subject to failure-for any number of reasons. A smart backup policy assumes that a certain percentage of media will fail much sooner than it should. The usual way to deal with this unfortunate fact of life is with redundancy: have two, three, or more copies of each backup and rotate them on a regular basis. For example, if you're backing up to tapes, you might have three complete sets, each of which contains a complete backup of all your systems. Use Set A on Monday, and then switch to Set B on Tuesday, Set C on Wednesday, back to Set A on Thursday, and so on. When a set is not in active use, keep it stored offsite in a safe location (see "Offsite Storage," ahead).

Your backup policy, therefore, should specify not only how many sets of media you have but also how they're rotated. Do you switch sets daily, weekly, or at some other interval? At what time do the rotations occur? Who's in charge of moving the media around?

This may be a good time to mention media labeling as well. Because any number of people may (either now or in the future) have to deal with your backups, you should develop a scheme for labeling physical media that makes it crystal clear what's on it. This should include, at minimum, a designation for the set to which it belongs, a number identifying its sequence within the set, and the date on which it was first used. For instance, "Set Alpha, Tape 3, 6/1/2008." Depending on how complex your backup system is and the sort of media you use, you may need more information too, but the key is to make it obvious to anyone who might be using the system what data is on what media.

Sooner or later, you'll fill up whatever media you use for backups. If you're backing up the computers for an entire university onto tapes, maybe this happens numerous times each day; if you're backing up a few computers from a small office onto a high-capacity RAID, it might not happen for months or years. (Sure, there are ways of selectively erasing old data from your backups, leaving only more-recent copies of your files, but even so, the volume of backup data is bound to exceed your media's capacity eventually.) In any case, your backup policy should specify exactly what happens when a storage device fills up. You have a few main choices:

• Recycle: Erase the media and record over it.

• Store: Hang onto the full media.

• Destroy: Ditch the old media in a way that prevents anyone else from reading your backups.

If you choose "store" or "destroy," you'll start over with new, blank media. Regardless of your choice, list the details. If you recycle media, how many times will you do that before storing or destroying it? Do you replace an entire set of media all at once (generally a good idea) or by the individual piece? How long will you store old media, and where? If and when you destroy old media, how will you do so securely? (And, just as a reminder: you may be legally obligated to maintain certain kinds of data for a number of years-be sure to check before destroying anything.)

In addition, because media reliability decreases over time, you should plan for any media to "expire" after a certain point-full or not. I wouldn't trust a ten-year-old hard drive to store critical data, for example. How long you choose to trust your media will depend on your personal experience, the manufacturer's MTBF (mean time between failures) ratings, educated guesses, your budget, and so on. However you determine the schedule, do plan to replace your media at regular intervals.

Remember my boss, the guy who wasn't worried about the building burning down? You, the person designing your business's backup policy, should definitely worry about that! It's all well and good to keep backup media in a fireproof safe or other secure location onsite, but you must also have at least one copy (and preferably more than one) stored in another building. As unlikely as it may be, something could happen-theft, espionage, earthquake, terrorist attack, whatever-that destroys all your backups if they're kept in a single location. Don't take any chances. You already specified that you have more than one set of media in rotation, so designate a safe offsite location to store media that's not actively in use, as well as a system for shuttling media back and forth. Needless to say, you don't need to share the precise location of your offsite backups with every employee in your company, but you should make sure that several trustworthy people know it, in case something happens to you.

Let's take for granted that your backups contain valuable, confidential data about your business that should not fall into anyone else's hands. It could be that your server room is highly secure, but as soon as you take backup media offsite, it becomes vulnerable-especially while in transit. And even if you employ armed guards and armored cars for transport, there's always the chance that a nosy or disgruntled employee with access to your backup equipment could poke around in your backup data. So I strongly recommend specifying in your backup policy that all backups will be encrypted, and I leave it to you to figure out what method of encryption meets your organization's needs the best.

Encrypted data is useless unless someone knows the pass phrase. As with any important password, you have to strike a balance in security. If the system administrator is the only one who knows the pass phrase and gets hit by a bus, your business will be in trouble. So carefully choose a small number of people who'll be trusted to know how to decrypt your backups if the need arises.

Backups are of no use whatsoever if you can't restore your data when you need it. Unfortunately, restoration is usually the part of a backup policy that gets the least attention. Give careful thought to all the different scenarios for restoring data that might arise, and make sure your policy spells out the following information.

In most business situations, only an IT person can restore backed-up files, since they reside on a secure server and since it would be all too easy for a user to mistakenly overwrite good files with backups. The downside, of course, is that restoration can be inconvenient and time-consuming for the user as well as for the IT staff. You could choose to implement a backup system in which individual users can directly restore files, if you're aware of the trade-offs and willing to live with them.

If restoring backups is the responsibility of a network administrator, your backup policy should state exactly who has this capability, and how to reach the person or department in charge. It should further specify the hours and days during which it's possible to restore backups and what to do in case of an off-hours emergency.

Assuming your users have to go to the IT staff to get files restored, exactly what is the process-make a phone call? Fill out a form on the intranet? Send an email? What if the user doesn't know the exact file name, date, or location? Is there a different procedure if a whole drive or user folder has to be restored? Spell out, in simple end-user terms, what someone has to do to get back data that's in the backup archives somewhere. And, if you have a system that lets users restore their own files, point them to a step-by-step guide on a Web page somewhere for how to do this.

You, as an enlightened techie, will understand that it's not always quick or easy to retrieve data from backups. In some cases, it might require fetching an offsite data set, waiting until there's a free tape drive, and laboriously searching through archives. But lots of people will assume that restoration should be immediate. Therefore, it doesn't hurt to put estimated time ranges for restoring files in your backup policy so that your users can set their expectations appropriately.

You can't assume that your backups are perfectly and indefinitely intact, just because your backup software didn't report any errors. Stuff happens. Make it an explicit part of your backup policy to test backups on a regular basis. When I say "test," again, I don't merely mean run through a verification procedure with your backup software. I mean actually restore files. Ideally, you should at least spot-check a few random files on each piece of physical media once every month or two. Less frequently, it wouldn't hurt to attempt much larger restorations. If you catch bad media or unreported software errors early, you'll be able to make adjustments before someone actually needs a file.

Beyond the three key categories of backup procedure, media management, and data restoration, you may want to think through a few more miscellaneous details and include them in your backup policy.

Because your backup policy is a document intended not just for computer geeks but for executives and run-of-the-mill employees, make sure you spell out exactly what everything means in nontechnical language. It might be helpful to have a brief "definitions" section toward the beginning where you state just what you mean by words like "backup," "incremental," and "restore."

Even though you've already specified where people go when they need to have files restored, your policy should also indicate other responsible parties. Who has physical access to the backup media? Who knows the pass phrase for encrypted backups? Who makes the policy decisions? These might be titles or positions rather than individual names, but either way, make it clear.

The hardware that backs up your computers-the servers, tape drives, hard drives, and so on-needs routine cleaning and other maintenance. Some components may need to be replaced or upgraded from time to time. Backup software will require updates. State in your backup policy who's responsible for maintaining the system, and (at least in broad terms) what sort of maintenance schedule is followed.

I've been advocating the creation of a backup policy that every single computer user in your organization will read. But some facts aren't really for public consumption-things like encryption pass phrases and the exact location of offsite storage. Also, some of the implementation details are too technical for the average user to be concerned with. So consider creating two different versions of your policy statement. One will be a short, snappy, one-page overview for the common folk, and the other will be a thorough guide for the technical people who will implement the system.

Your finished backup policy, which should be a great deal shorter than this article, serves two important purposes. First, it should guide the creation (or rehabilitation) of your backup system, ensuring that all the important factors are weighed before reaching specific hardware, software, or implementation decisions. And second, it tells everyone who uses your network exactly what to expect-from the technology and from you. Will their data, and their company, be safe in the event of a major catastrophe or a minor user error? Making sure that's true is the function of your backup system, and conveying a sense of confidence to your users is the function of your backup policy.

Joe Kissell is Senior Editor of TidBITS, a frequent contributor to Macworld, and author of numerous books and ebooks about Mac OS X, including Take Control of Mac OS X Backups, Take Control of Easy Backups in Leopard, and Real World Mac Maintenance and Backups. You can reach him at jwk@mac.com.