Although the two sample proportion test is well known within the marketing experimentation community, there are a host of categorical data analysis techniques that are amenable to the analysis of A\B test data. This presentation provides a quick overview of Chi-square analysis and a discussion of its use cases in marketing experimentation. Specifically, we discuss how to 1) test a\b\n experiments without inflating Type I error after multiple comparisons, 2) analyze tests with more than dichotomous outcome variables and 3) the process for calculating goodness-of-fit and tests-of-independence. Simple guidelines will be provided regarding test assumptions and use case scenarios.
Experiment Nation: The Podcast