Data does not just magically spring into existence. Someone, somewhere, has to decide what data gets created and the rules for its creation. We would claim that this often starts as a pretty simple exercise, and then, over time, that simplicity balloons to be pretty complex! What if, for instance, you decided to listen to every #1 song on the Billboard Hot 100 going back to its inception in 1958? You may start by just capturing the song name, the artist, and the week(s) it was the #1 song. But, before you know it, you may find that you're adding in artist details…and songwriter details…and producer details…and genre details…and instrumentation details, and your dataset has 105 columns! But, oh, the questions that dataset could answer! And that's exactly the dataset that our guest for this episode, Chris Dalla Riva, created. He uses it (with a range of supplemental datasets) for his pieces in his Substack, Can't Get Much Higher, as well as the underlying raw material for his upcoming book, Uncharted Territory: What Numbers Tell Us about the Biggest Hit Songs and Ourselves. While the underlying material was music, the parallels to more staid business data were many when it comes to the underlying processes and challenges for doing that work!
This episode's Measurement Bite from show sponsor Recast is an explanation of the miracle of randomization when it comes to addressing unobserved confounders from Michael Kaminsky!
For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page.
