In this episode, we discuss Quantifauxcation, described by statistician Philip Stark as “situations in which a number is, in effect, made up, and then is given credence merely because it is quantitative.” We give examples of quantifauxcation in psychology, including errors of the third kind. We spend the second half of the podcast discussing how to develop quantitative measures that are meaningful and bridge the divide between qualitative and quantitative observations.

Shownotes

Statistics textbook by Philip Stark.

Stark, P. B. (2022). Pay No attention to the model behind the curtain. Pure and Applied Geophysics, 179(11), 4121–4145. https://doi.org/10.1007/s00024-022-03137-2

Burgess, E. W. (1927). Statistics and case studies as methods of sociological research, Vol 12(3), 103-120. (Thanks to Andy Grieve!)

Nick Brown's role in pointing out flaws in the positivity ratio.

Retraction notice of the positivity ratio paper.

Blog by Tania Lombrozo on nonsensical formulas in abstracts.

Kimball, A. W. (1957). Errors of the third kind in statistical consulting. Journal of the American Statistical Association, 52(278), 133–142. https://doi.org/10.1080/01621459.1957.10501374

Type III errors: Philip Stark’s post of Deborah Mayo’s blog

Brower, D. (1949). The problem of quantification in psychological science. Psychological Review, 56(6), 325–333. https://doi.org/10.1037/h0061802

Guttman scales

Wilson, M. (2023). Constructing measures: An item response modeling approach. Taylor & Francis.

Wilson, M., Bathia, S., Morell, L., Gochyyev, P., Koo, B. W., & Smith, R. (2022). Seeking a better balance between efficiency and interpretability: Comparing the likert response format with the Guttman response format. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000462

Bhatti, H.A., Mehta, S., McNeil, R., Wilson, M. (2023). A scientific approach to assessment: Rasch measurement and the four building blocks. In X. Liu & W. Boone (Eds.), Advances in Applications of Rash Measurement in Science Education. Springer Nature.

099 - Neither Vertical nor Horizontal, Part 4