#144 - The Craft of Sample Sizes with Lauren Stern of WHOOP
Mar 25, 2024
auto_awesome
Lauren Stern, a research sample size expert, discusses tailoring sample sizes to research objectives, qualitative confidence, diverse methodologies in quantitative research, and engaging stakeholders. She provides insights on recruiting participants, analysis approaches, international samples, drop-off rates, and participant compensation strategies. The conversation highlights the importance of advocating for sample sizes and provides resources for making informed decisions.
Engage stakeholders early to discuss sample size for confident research outcomes.
Balance qualitative and quantitative approaches for impactful research storytelling.
Empower research teams to openly discuss sample size challenges and strategies for better outcomes.
Deep dives
The Importance of Discussing Sample Size
It is crucial for researchers to engage in discussions about sample size early in the research process. Junior researchers often feel apprehensive about addressing sample size concerns but having these conversations can provide confidence and clarity. Understanding the available resources for determining sample size, such as personal experience, team input, literature, and calculators, is essential for conducting effective research studies. By proactively discussing and determining appropriate sample sizes, researchers can engage stakeholders effectively and make informed decisions.
Balancing Qualitative and Quantitative Approaches
When conducting research studies that involve both qualitative and quantitative aspects, it is important to maintain a balance between the two approaches. Strategies like using ratings questions in interviews can provide quantitative data for more convincing insights without compromising the qualitative depth of the research. This hybrid approach allows researchers to tell cohesive stories backed by objective data while catering to different stakeholder preferences for numbers versus narratives.
Considering Impact on Stakeholders
Researchers should consider the impact of their sample size decisions on stakeholders and the narratives they intend to convey. Striking a balance between the truth of the research findings and the storytelling necessary to engage stakeholders effectively is key. By framing discussions around sample sizes as tools to enhance the impact of research insights and cater to stakeholder needs, researchers can build confidence and credibility in their work, fostering better decision-making and understanding among project stakeholders.
Navigating Mixed Methods Studies
In mixed methods studies that blend qualitative and quantitative approaches, researchers face the challenge of harmonizing diverse data types for meaningful analysis. By leveraging tools like Qualtrics' confidence calculator and navigating between numerical insights and deeper qualitative understanding, researchers can navigate the complexities of mixed methods studies effectively. Understanding the nuances between qualitative and quantitative goals within a study is essential for achieving comprehensive research outcomes.
Embracing Conversations on Sample Size
Encouraging open conversations about sample size within research teams can foster a culture of collaboration and continuous improvement. Junior researchers should feel empowered to seek guidance, utilize available resources, and engage in constructive discussions about sample size considerations. By promoting a supportive environment for discussing sample size challenges and strategies, research teams can enhance their decision-making processes and research outcomes for greater impact.
In this episode, Carol and Erin dive into all things research sample sizes with Lauren Stern. No matter your experience with user research, you'll need to recruit folks, making this an evergreen topic. Lauren has coached both new-to-research and junior UXRs on this critical topic and she shares some of her best advice.
Lauren shares importance considerations and nuances around different types of studies and even analysis approaches. She also unpacks her approach for international samples, the impact of drop-off rates, and participant compensation strategies.
Going beyond sample sizes, we conclude with a discussion of how to better engage with stakeholders when advocating sample sizes, making these conversations about "how many to recruit" more informed. She also shares resources to use when making the case to stakeholders.
Episode Highlights
05:41 - Crafting research goals and parameters: a collaborative journey
13:48 - Flexible research design: navigating sample sizes and methodologies
19:57 - Tailoring sample sizes to research objectives: finding the right fit
26:30 - Qualitative confidence and stakeholder expectations
34:11 - Diverse methodologies in quantitative research: beyond surveys
43:05 - Departing research wisdom on sample sizes generally
About Our Guest Lauren Stern is a mixed-methods research leader focused on creating the most human-centered technology possible. Over the last ten years her work has explored how perception and social cognition shape our experiences with automated systems from military zones to living rooms. Whether exploring individual experiences in the field or looking at large-scale data collections, she loves the puzzle of study design and coaching new researchers through the process.