S3E12: 'How Intentional Experimentation in A/B Testing Supports Privacy' with Matt Gershoff (Conductrics)
Jun 4, 2024
auto_awesome
Matt Gershoff, Co-founder of Conductrics, discusses A/B testing and data collection. He emphasizes intentional data collection to support privacy and shares insights on the value of experimentation. Topics include minimizing privacy risks, data collection processes, and the importance of attending privacy conferences.
Intentional A/B testing aids in supporting high-level privacy by minimizing unnecessary data accumulation and emphasizing purposeful data collection.
Implementing privacy by design principles facilitates data minimization, ensuring default behaviors align with privacy principles and emphasizing beneficial trade-offs.
Deep dives
Improved Decision Making Through Intention and Principled Action
Being intentional and explicit in decision-making processes is crucial as it facilitates a principled approach to learning about problems and taking necessary actions. Guidance and clarity in decision-making bring substantial value alongside technological advancements.
Privacy by Design Implementation in Technical Solutions
Implementing privacy by design principles in software solutions fosters data minimization by default, prioritizing minimal collection of personally identifiable information. This approach ensures the default behavior aligns with data minimization and emphasizes constraint but with beneficial trade-offs.
Efficient Experimentation Through Data Minimization and Intent Focus
Conducting experiments with a focus on intentionality and data minimization streamlines the data collection process, resulting in efficient regression testing and multivariate testing capabilities. The aggregate data storage method supports privacy by default principles and facilitates easy auditing of data structures.
Enhancing Privacy and Computational Efficiency Through K-Anonymity
Implementing K-anonymity principles in data storage not only ensures anonymity and data protection but also improves computational efficiency in processing data. By limiting unique identifiers and focusing on aggregate data, a more privacy-conscious and efficient approach is established to manage data effectively.
Today, I'm joined by Matt Gershoff, Co-founder and CEO of Conductrics, a software company specializing in A/B testing, multi-armed bandit techniques, and customer research and survey software. With a strong background in resource economics and artificial intelligence, Matt brings a unique perspective to the conversation, emphasizing simplicity and intentionality in decision-making and data collection.
In this episode, Matt dives into Conductrics' background, the role of A/B testing and experimentation in privacy, data collection at a specific and granular level, and the details of Conductrics' processes. He emphasizes the importance of intentionally collecting data with a clear purpose to avoid unnecessary data accumulation and touches on the value of experimentation in conjunction with data minimization strategies. Matt also discusses his upcoming talk at the PEPR Conference and shares his hopes for what privacy engineers will learn from the event.
Topics Covered:
Matt’s background and how he started A/B testing and experimentation at Conductrics
The major challenges that arise when companies run experiments and how Conductrics works to solve them
Breaking down A/B testing
How being intentional about A/B testing and experimentation supports high level privacy
The process of the data collection, testing, and experimentation
Collecting the data while minimizing privacy risks
The value of attending the USENIX Conference on Privacy Engineering Practice & Respect (PEPR24) and what to expect from Matt’s talk