Imagine you are developing a system that will be used by millions of people. In a situation like this, a system has to be very well-tested for any type of error that can cause the system to break while in production. But what’s the best way to test a system for any possible system failure because of bugs? This is where test data management comes in.

In this post, I will explain why test data management is critical in software delivery. To develop high quality software products, you have to continuously test the system as it’s being developed. Let’s dive in straight to understanding how this problem can be solved by using test data management.

What Is Test Data Management?

Well, in simple terms, test data management is the creation of data sets that are similar to the actual data in the organization’s production environment. Software engineers and testers then utilize this data to test and validate the quality of systems under development.

Now, you might be wondering why you need to create new data. Why not just use the existing production data? Well, data is essential to your organization, so you should protect it at all costs. That means developers and testers shouldn’t have access to it. This has nothing to do with the issue of trust but security. Data should be highly regarded, or else there can be a data breach. And as you know, data breaches can cause loss to an organization.

How Can You Create Test Data?

So, now that we know why we need test data that is separate from our production data, how can we create it?

The first thing you must do is understand the type of the business you are dealing with. More specifically, you need to know how your software product will work and the type of end users that will use the software. By doing so, it will be easier to prepare test data. Keep in mind that test data has to be as realistic as the actual data in the production environment.

You can use automated tools to generate test data. Another way of creating test data is by copying or masking production data that your actual end users will use. Here you have to be creative as well and create different types of test data sets. You can’t rely only on the masked data from production data for testing.

Benefits of Test Data Management in Software Delivery

Test data management has many benefits in software delivery. Here are some of the benefits of test data management in software delivery in any software development environment.

High Quality Software Delivery

When you apply test data management to software delivery, it will give software developers and testers to test the systems and make solid validations of the software. This enhances the security of the system and can prevent possible failures of the system in the production environment. Testing systems with test data gives assurance that the system will perform as expected in the production environment without defects or bugs.

Faster Production and Delivery of Software Products to the Market

Imagine that, after some months of hard work of developing a software application, you’ve just released a software application on the market, only for it to fail at the market level. That’s not only a loss of resources, but it’s also a pain.

A system that’s well-tested using test data will have a shorter production time and excel at the production level. That’s because it’s much more likely to perform the way it was intended to. If the system fails to perform in production because it was not tested well, then the system has to be redone. This wastes time and resources for the organization.

Money Needs Speed

Test data management is critical when it comes to software delivery speed. Having data that’s of good quality and is similar to production data makes development easier and faster. System efficiency is cardinal for any organization, and test data management assures that a system will be efficient when released in production. Therefore, you start generating revenue as soon as you deploy the system.

Imagine having to redo a system after release because users discover some bugs. That can waste a lot of time and resources, and you may also lose the market for that product.

Testing With the Correct Test Data

Testing with good quality test data will help in making sure that the tests you run in the development phase will not change the behavior of the application in the production phase. For example, you might test that the system is accepting supported data by entering a username and password in the text box with all types of data that a user can possibly input into the system.

No matter how many times you test the software, if the test data is not correct, you should expect the software to fail in the production phase. This is why it is always important to ensure that test data is of great quality and resembles your actual production data.

Bug and Logical Fixes

How can you know that the text box is accepting invalid input such as unsupported characters or blank fields from users? Well, you find out by validating the system through testing.

The whole point of having test data in software delivery is to make sure that the software performs as expected. Additionally, you need to make sure that the same tests will pass in production and have no loopholes that could damage the organization’s reputation. Therefore, test data becomes a critical part of software delivery life cycle, as it helps to identify errors and logical problems in the system. Thanks to this, you can make fixes before releasing the software.

For example, imagine a loaning system that makes incorrect calculations by increasing the interest rate by a certain percentage. That can be unfair to the borrowers and can backfire for the lending company.

Earning Trust

Trust is earned, and if you want to earn it from the end users or management, you have to deliver a software product that’s bug-free and works as expected. In fact, every software development and testing team should utilize test data management. Test data management enables teams to deliver software products that stand out and earn trust from management. After all, you can’t ship an error-prone system to the market and expect happy users.

Why Test Data Management Matters

Test data management is essential for ensuring that software applications will function as expected in a production environment. By testing with realistic data, organizations can gain assurance that their software will not fail in production, strengthening their relationship with clients and reducing the chances of fixing bugs in production and rollbacks. Test data management also speeds up the software development life cycle, reducing costs and improving the speed of software delivery. This helps organizations stay competitive in a rapidly changing market by detecting errors at an early stage and fixing them before release.

Additionally, test data management helps reduce compliance and security risks, provides Product Owners and their Steering Committees with assurance that the software they are releasing is of high quality, reduces the risk of data breaches by ensuring only valid and secure data is used in testing, and helps them make informed decisions about product features by evaluating the impact of changes on performance, scalability, and usability.

Summary

In simple terms, test data is simply the data used to test a software application that’s under the software testing life cycle. Test data management, on the other hand, is the actual process of administering data that’s necessary for use in the software development test life cycle.

You can’t deny that test data management is an essential part of testing and developing software. It plays a crucial role in helping you produce high quality software that’s bug-free and works as expected.

You should take test data management seriously and apply it when delivering software. If you do so, your organization will gain more revenue because you’ll deliver higher quality software products. Higher quality products make the customers happy instead of giving them a reason to complain about some bug.

This post was written by Mathews Musukuma. Mathews is a software engineer with experience in web and application development. Some of his skills include Python/Django, JavaScript, and Ionic Framework. Over time, Mathews has also developed interest in technical content writing.

You lean back in your chair with a satisfied grin. You did it. It wasn’t easy, but you did it. You diagnosed and fixed the bug that kept defying your team. And you have the unit tests to prove it.

The grin slowly fades from your face as you realize that you still need your code to pass the integration tests. And you need to get data to use in them. Not your favorite activity.

You can put that grin back on your face because there is another way: using a gold copy.

Read on to learn what a gold copy is and why you want to use one. You will also find out how it can help you work on an application with low test coverage. You know, the dreaded legacy systems.

What Is a Gold Copy

In essence, a gold copy is a set of test data. Nothing more, nothing less. What sets it apart from other sets of test data is the way you use and guard it.

• You only change a gold copy when you need to add or remove test cases.
• You use a gold copy to set up the initial state of a test environment.
• All automated and manual tests work on copies of the gold copy.

A gold copy also functions as the gold standard for all your tests and for everybody testing your application. It contains the data for all the test cases that you need to cover all the features of your product. It may not start out as comprehensive, but that’s the goal.

Building a comprehensive gold copy isn’t easy or quick. But it’s definitely worth it, and it trumps using production data almost every time.

Why You Don’t Want to Test in Production

Continuous delivery adepts rave about testing in production. And yes, that has enormous benefits. However:

• It requires the use of feature toggles to restrict access to new features and changed functionality.
• Running the automated tests in your builds against a production environment is not going to make you any friends.
• The sheer volume of production data usually is prohibitive for a timely feedback loop.
• Giving developers access to production data can violate privacy and other data regulations.

There’s more:

• Production data changes all the time, and its values are unpredictable, which makes it unsuitable as a base for automated testing.
• Finding appropriate test data in production is a challenge. Testing requires edge cases, when users and thus their data tend to be much more alike than they would like to know.
• To comply with privacy and other data regulations, extracts need to be anonymized and masked.

Contrived Test Data Isn’t Half as Bad as It Sounds

Contrived examples usually mean that you wouldn’t encounter the example in the real world. However, when it comes to testing, contrived is what you want. A contrived set of test data:

• has only one purpose—verifying that your application works as intended and expected and that code changes do not cause regressions
• contains a limited amount of data, enabling a faster feedback loop even for end-to-end tests
• can be made to be self-identifying and self-descriptive to help understand what specific data is meant to test
• contains edge cases that willtrip you up in the real world but are generally absent from production data by their very definition
• can be built into a comprehensive, optimized, targeted set of data that fully exercises your application

Of course, production data can be manipulated to achieve the same. But extracting it stresses production, and manipulating it takes time and effort. And you really don’t want to be doing that again and again and again.

That’s why you combine contrived data and gold copies. You start your gold copy with an extract from production data that is of course anonymized and otherwise made to conform to privacy and data regulations. Over time, you manipulate it into that optimized, targeted set of data. But using that initial set of test data as a gold copy will bring you benefits immediately.

Benefits of Gold Copies

In addition to the benefits of contrived data, using a gold copy gets you these benefits:

• You can easily set up a test environment with a comprehensive set of test data
• You can easily revert the data in a test environment to its original state
• The ability to automate spinning up test environments
• Automated regression testing for legacy systems

Everyone working on your application will appreciate it. They no longer have to hunt for good data to use in their test cases. And they no longer have to create test data themselves. A good thing, because creating test data and tests that produce false positives (i.e., tests that succeed when they should fail) is incredibly easy. You only have to use the same values a tad too often.

The ability to automate spinning up a test environment is what makes using a gold copy so invaluable for large development shops and shops that need to support many different platforms. Just imagine how much time and effort can be saved when providing teams and individuals with comprehensive, standard test data that can be automated. For example, using containers and a test data management tool like Enov8’s.

Finally, gold copies can help reduce the headaches and anxiety of working with legacy code. Here’s how.

Slaying the Dreaded Legacy Monster

Any system that does not have enough automated unit and integration tests guarding it against regressions is a legacy system. They are hard to change without worrying.

The lack of tests, especially the lack of unit tests, allowed coding practices that now make it hard to bring a legacy system under test. Because bringing it under test requires refactoring the code. And you can’t refactor with any confidence if you have no tests to tell you if you broke something.

Fortunately, a gold copy can bail you out of this one. It allows you to add automated regression testing by using the golden master technique. That technique takes advantage of the fact that any application with value to its users produces all kinds of output.

Steps in the Golden Master Technique

How you implement the golden master technique depends on your environment. But it always follows the same pattern, and it always starts with a gold copy.

1. Use your current code against the gold copy to generate the output you want to guard against regressions. For example, a CSV export of an order, a PDF print of that order, or even a screenshot of it.
2. Save that output. It’s your golden master.
3. Make your changes.
4. Use your new code against the gold copy to generate the “output under test” again.
5. Compare the output you just generated to your golden master.
6. Look for and explain any differences.

If you were refactoring, which by definition means there were no functional changes, the comparison should show that there are no differences.

If you were fixing a bug, the comparison should show a difference. The golden master would have the incorrect value, while the output from the fixed code would have the correct value. No other differences should be found.

If you were changing functionality, you can expect a lot of differences. All of them should be explicable by the change in functionality. Any differences that cannot be explained that way are regressions.

Explaining the differences requires manual assessment by a human. It’s known as the “Guru Checks Output” anti-pattern. And it needs to be done every test run if you want to stay on top of things. Marking differences as expected can help. Especially when you can customize the comparison so it won’t report them as differences.

Go Get Yourself Some Gold

Now that you know what a gold copy is and how you can use it to your advantage, it’s time for action. It’s time to start building toward the goal of a comprehensive set of test data and use it as a gold copy.

Your first step is simple: save the data from the test environment you set up for the issue or feature you’re working on now. That is going to be your gold copy. If your application uses any kind of SQL database, you could use that to generate a DML-SQL script that you can add to a repository.

Use your gold copy to set up the test environment for your next issue. Make sure you don’t (inadvertently) change your gold copy while you’re working on that issue. When you’re finished, and if you needed to add test data for the test cases of this issue, update your gold copy.

Rinse and repeat, and soon enough you’ll be well on your way to a truly useful comprehensive set of test data.

This post was written by Marjan Venema. Marjan’s specialty is writing engaging copy that takes the terror out of tech: making complicated and complex topics easy to understand and consume. You’ll find samples on her portfolio. Her content is optimized for search engines, attracting more organic traffic for small businesses and independent professionals in IT and other Tech industries, that she’ll also help with content audits and strategy.

Setting up a great test data management strategy is a crucial step for taking your test automation process to its fullest potential. However, many software professionals are still not familiar with the concept of test data management (TDM). Even those that are familiar with TDM might have a hard time putting it in practice. Why is that?

When it comes to test data management, the “what” is relatively straightforward, but we can’t say the same about the “how.” As it turns out, there are several competing methods of managing test data. Which one should you choose? As you’ll see in this post, this isn’t a one-approach-fits-all kind of situation. Each method has its unique strengths and weaknesses and might be more or less appropriate for your use case.

Today’s post will cover some of the existing test data management approaches, listing the advantages and disadvantages of each one. Let’s get started.

Replicating Data From Production

The first approach we’re going to cover in this post is perhaps the most popular one, at least for beginners. And that makes perfect sense if you think about it. When you first encounter the challenge of coming up with data to feed your testing processes, it isn’t too far-fetched to think you should just copy data from production and be done with it. It’s the easiest way to obtain data that is as realistic as possible. You just can’t get more real than production.

Not everything is a bed of roses when it comes to production data replication. Quite the opposite, actually. The easy access to data is pretty much the only advantage this method has. And what about the disadvantages? These, sadly, abound.

Here Be Dragons: Some Downsides of the Approach

Here’s the first problem: replicating data from production continues to be mostly a manual process. Sure, you can come up with scripts and automated jobs to do most of the heavy lifting for you. But keep in mind that generating the data isn’t the whole job of a TDM management solution. “Availability” is an integral part of the package. That means that the TDM tool is responsible for making sure the data is available where it’s needed, at the right time. A naive approach based on scripts might not be sufficient to manage the demands of a complex testing process, forcing you to rely on a manual process to do so.

Secondly, production replication doesn’t lend itself well to negative test cases. It’d be out of the scope of this post to give a lengthy explanation of negative testing. In a nutshell, negative test cases are tests that validate the system against invalid data. Basically, you throw faulty data at your application to check how well it can handle it. Since production data would (hopefully) be in good shape, this approach isn’t well suited to this type of testing.

Production data replication also doesn’t work…if there is not data replication for you to replicate in the first place! What should you do when you need to test an application that is still in the alpha stage of development or even a prototype? Since no one is actually using the application, there would be no production data for you to copy. That’s a severe downside of this approach since every new application will face this problem.

Here Be Dragons (For Real): Legal Implications

Finally, we have the most serious downside of this approach—data sensitivity. Data compliance is a crucial part of the modern IT landscape since companies are responsible for the data they store and manipulate. It’s up to them to protect their client’s data, ensuring it’s not abused. When replicating data from production, software organizations run the risk of failing to comply with privacy acts, such as GDPR. And that can bring catastrophic consequences, legal, financial, and reputation-wise.

Data Masking

In order to solve the downsides of production data replication (a.k.a the naive approach), test data management tools have come up with more sophisticated methods. One of the
most popular of these approaches is test data masking. As its name implies, tools that adopt this approach enable its users to apply masks to production data. Such masks will remove personally identifiable information (PII) from the data.

Data masking is an improvement over naive production data replication, for sure. But the approach is not without its downsides.

First, consider the “time” variable. Data masking doesn’t reduce the time spent generating (or rather, copying) the data for testing. On the contrary, it increases it because now you have a new added in the process. You could argue—and I’d gladly agree—that it’s time well spent, but it’s more time nonetheless.

Then, you also have to keep in mind that data masking isn’t a standalone approach on its own. Instead, it complements the previous approach by solving one of its more serious issues. The problem is data masking can’t fix every problem that the production replication approach has. For instance, if you intend to test an application still in development, for which there is no production data at all, data masking is powerless to help you.

Synthetic Data Generation

Synthetic data generation is yet another method of test data management. As its name suggests, this approach consists of generating “fake”—or synthetic—data from a data model. Tools that implement this approach are able to preserve the format of the data. The values themselves, though, are completely disconnected from any original data. What does that imply?

The implication of this is that synthetic data generation’s greatest asset is simultaneously its most significant downside. By populating the database with entirely “made-up” values, the approach dramatically reduces (virtually eliminates) the risk of exposing sensitive data. On the other hand, depending on the tool’s sophistication—or lack of—you might end up with data that feels “fake-y.” One of the goals of an excellent TDM strategy is to provide data that is as production-like as possible.

To wrap-up, let’s talk about the biggest advantage of synthetic data generation, namely: speed. Once you have a model in place, you can quickly generate data from it, effectively eliminating the time delays that plague other approaches.

Test Data Management Is More Than Test Data Generation

In this post, we’ve covered some of the most used approaches to generate test data. The list is definitely not exhaustive; there are many more methods that we didn’t cover. However, many of them are variations or combinations of the approaches we did talk about.

Another thing to keep in mind is that test data management is much more than just generating test data. TDM is responsible for ensuring the quality of the test data, its availability, and also its security. In other words: the data must be good, and it must be available at the right place, at the right time. And bad actors shouldn’t be allowed to expose it or misuse it in any way. That’s why, depending on the needs of your organization, you should consider adopting a full-fledged data compliance solution, which can not only supply your data generation needs but also make sure your data adhere to the compliance requirements you must follow.

This post was written by Carlos Schults. Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.