Data Integrity: Ensuring Your Critical Business Information Is Reliable and Recoverable

I am a big coffee drinker, so I naturally found it interesting last week when I talked to someone who told me about a persistent problem he had at a coffee shop where he used to work. He said the shop had a nice atmosphere and friendly staff, but their approach to training was very casual, with the result that new baristas would learn to make drinks the wrong way—and then go on to teach those incorrect methods to the people hired after them. The upshot was, the staff all had different ideas on the right way to do things, and the customers never knew what they were going to get.

As a coffee lover, I found this story unnerving. But as a business continuity consultant, I found it all too familiar.

The reason isn’t because I do a lot of work with neighborhood coffee shops. It’s because I see the same problem over and over again in company training programs that have a bearing on business continuity.

And nowhere is this more true than in the area of data integrity.

Data integrity, of course, refers to the accuracy and consistency of data over the course of its lifecycle. It involves making sure the data is good from the time it’s born until the time it is removed—and it’s critical to the effectiveness and reliability of any system that stores and processes information.

Over and over again in this area I see the same problem of poor training—and the metastasizing of incorrect methods and information—that my friend described as happening at his old coffee shop.

However, this is only one of many problems with data integrity that I encounter at the companies I work with and hear about as I travel the country working on business-continuity projects.

In thinking about this issue, it occurred to me that it might be worthwhile to do a whole post on this topic, so without further ado, here are a few of my thoughts on the subject of how to ensure data integrity.

I’ll cap the post off with a list of five specific issues that I think might be worthwhile for you to think about, if you’re a BC professional committed to making sure your organization can recover its data in the event of a disruption.

Data integrity is critical for the successful recovery of any data-dependent organization that has been hit by an outage.

For you as a business-continuity professional, the critical questions to ask yourself as you assess the level of data integrity at your organization are:

• Do I have the right controls around my data to ensure that it is accurate and complete?
• Do only the right people have access to my data?
• Do I have the right procedures in place to ensure that any changes made to the data are done correctly, to prevent the introduction of inaccuracies?
• Have I established and thoroughly trained my staff in the proper way to extract and report the data, to make sure data integrity is maintained?

In a nutshell, you want to make sure that the right people have access to the data, that they know what to do with it, and that they know how to report it.

If you don’t have the right controls in place, sooner or later you will have problems with the wrong people accessing the data or with people processing it wrong.

What kind of problems am I talking about? Here are a few of them:

• Correct fields being changed for no reason, making them incorrect.
• Incorrect fields which should be changed not being changed.
• Incorrect fields which should be changed being changed incorrectly, to another wrong value.
• Employees with a need to access the data not knowing how to access it.
• People believing the data is wrong when it isn’t.
• Management being provided with incorrect data and making decisions based on that incorrect data.

When it comes to business continuity, the big issue with data integrity is, if you have an event, once IT recovers from your backup, how do you know where you stand in terms of the accuracy of that data? Can you be confident the restored data is good? Do you have a clear understanding of how up-to-date the restored data is?

The better your data controls, the more confident you can be in the quality of your restored data.

When we talk about how up-to-date the restored data is, obviously the restoration will only go up to the time of your most recent backup. That’s why everyone on your team should know when IT regularly runs the backup.

With the exception of a few types of data-heavy and highly sensitive environments such as hospitals and casinos, most companies run a backup once a day, in the middle of the night.

So if your organization does a backup once every 24 hours at, say, 2 a.m., and you have an outage one day at 3 p.m., obviously all the work people did from 2:01 a.m. until the outage at 3 p.m. will be lost and will have to be reconstructed.

If and when this happens to you, do you have a process in place to help you reconstruct that lost data?

Here are five issues you should particularly think about as you assess your organization’s data integrity and work to strengthen it:

1. Training. Is your organization giving people the training they need to enable them to properly and safely manage and massage your data? I see so many disasters happen because people lack adequate training in data entry. Just like at the coffee shop, there’s bad onboarding for new people, and once a few people get trained wrong, it spreads throughout the organization because those people incorrectly train those who come after them.
2. Accountability. A big part of accountability is information security: making sure the right people have access to the information. Then it’s all about, are they doing the right things with the data? Are they processing it properly?
3. Data cleaning and maintenance. This is huge. Think about what happens when you have a file cabinet and you don’t clean it out at the end of the year. Over time, chaos develops. You will have data in there from this year, last year, two years ago. You can’t find what you need when you need it. Most companies do not have a regular cleaning process, but the fact is you’re only as good as the most current information you have. I recommend that you establish and follow a rational cleaning and maintenance procedure. There is a real cost to your recoverability if your database is full of useless information. The bigger the database, the longer it takes to restore, and the longer your company is offline.
4. Data validation rules. It’s very important that your tool incorporates rules to minimize the opportunity for people to make mistakes. A field for phone numbers should not accept any input that does not match the format of a phone number. A field for tax ID numbers should match the format of those identifiers. You might have to keep after your programmers on this one because I have noticed that incorporating data validation rules is not one of their favorite things to do. Make sure your validation rules fit the culture and your business operations. Technology can do a lot to prevent well-intentioned people from making mistakes.
5. Data modeling and graphing. This one comes down to big data. Data modeling and graphing is potentially a very powerful tool, but its effectiveness is reduced when the data it is built on lacks integrity. Garbage in, garbage out. You can’t do good modeling if your data is riddled with errors. The smartest people in the world cannot make good decisions if they’re unknowingly basing them on invalid data.

So is there any way to tackle the issue of ensuring good data integrity at the root? There is, and it’s through the BIA.

The Business Impact Analysis is the cornerstone of your program. Among other things, it makes sure the right impacts, processes, dependencies, and records are entered into your system. The BIA requires a high level of data integrity because this drives everything else, including recovery strategies and recovery plans.

CONSIDER BCMMETRICS^TM BUSINESS CONTINUITY MANAGEMENT TOOLS

Numerous third-party tools exist that can help you conduct a rigorous business impact analysis. One you might consider is BCMMETRICS’ tool BIA On-Demand (BIAOD), a cloud-based, secure tool that gives you everything you need to conduct a complete Business Impact Analysis at the Company, Division or Department level. BIA On-Demand was built with a deep awareness of the importance of data integrity and can help you ensure that your data is accurate, up-to-date, and recoverable if and when your system is hit with an outage.