Ep. 55 The value of Synthetic Data for the Software Development Lifecycle

Leaders at NASA know the value of simulation.  Before the Rover was sent to Mars, designers ran it though many scenarios on earth. Of course, they couldn’t reproduce the climactic conditions on the Red Planet, but they knew they had to try to simulate the environment before it left for a 103.3-million-mile journey.

Their pioneer work probably was developed out of training simulations for pilots.  Why crash a $200 million plane if you can rehearse a “life-like” cockpit from a training facility?

Software developers are presented with a similar problem, but the variables are different. Instead of a pilot not being able to land a plane, they may design a system that exposes private information or a system that does not have the interoperability that the designers planned for.

The solution is to artificially generate data that is very similar to “live” information.  That way, they can run simulations to learn about unexpected events when systems collide. 

Thomas George from Vidoori is a data scientist from Vidoori who explains many of the concepts behind synthetic data and its application.  For example, a simulation can be run with synthetic data that shows what the expected value of a financial transaction should be.  If a set of data is very close to real data, one can tell if there have been any waste, fraud, or abuse possibilities.

From a data management perspective, one can take a large data set and “pressure test” the workflow to see if there is latency in the architecture.

Thomas George observes applying artificial data should be an essential part of large organizations that have sensitive data that needs to be protected.