The Importance of Sequential Testing

3min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Merritt Ajo started working on sequential testing. Matt Palicastro did a presentation to the test and learn community where he talked about sequential testing but part of that was simulated like what would your false positive rate be affected if you peaked early without changing some constraints. Nick Potter on the measure slack posted this and it is 100.datavizproject.com vis with a Z which sounds crazy but they were just kind of like how many different ways could we visualize it? They came up with a hundred distinct visualization so it's kind of a thought experiment taken to a to an extreme that's it's worth checking out, Tim.

When it comes to simulation, we're all really asking the same question: are we living in one? Alas! We did not tackle that on this episode. Instead, with Julie Hoyer as a guest co-host while Moe is on leave, we were joined by Frances Sneddon, the CTO of Simul8, to dig into some of the nuts and bolts of simulation as a tool for improving processes. It turns out that effectively putting simulations to use means focusing on some of the same foundational aspects of effectively using analytics, data science, or experimentation: clearly defining the problem, tapping into the domain experts to actually understand the process or scenario of focus, and applying some level of "art" to complement the science of the work! For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page.