Misconceptions and Challenges in Data Analysis

There is no shortage of books on the growing impact of data collection and analysis on our societies, our cultures, and our everyday lives. David Hand's new book Dark Data: Why What You Don't Know Matters (Princeton University Press, 2020) is unique in this genre for its focus on those data that aren't collected or don't get analyzed. More than an introduction to missingness and how to account for it, this book proposes that the whole of data analysis can benefit from a "dark data" perspective—that is, careful consideration of not only what is seen but what is unseen. David assembles wide-ranging examples, from the histories of science and finance to his own research and consultancy, to show how this perspective can shed new light on concepts as classical as random sampling and survey design and as cutting-edge as machine learning and the measurement of honesty. I expect the book to inspire the same enjoyment and reflection in general readers as it is sure to in statisticians and other data analysts.

Suggested companion work: Caroline Criado Perez, Invisible Women: Data Bias in a World Designed for Men.

Cory Brunson (he/him) is a Research Assistant Professor at the Laboratory for Systems Medicine at the University of Florida.