How Experimentation Helps You Build Better Travel Digital Products [Booking.com Case]
Mar 18, 2021
auto_awesome
Lukas Vermeer, Director of Experimentation at Booking.com, discusses how Booking.com uses experimentation to build travel digital products. Topics include organization structure, types of experiments to run, integrating experimentation and user research, metrics to use, and the value of purposeful friction in user experience design.
Experimentation is crucial in building great travel digital products and can be integrated with user research for better outcomes.
The choice of an organizational model for experimentation depends on the organization's maturity and context, and a center of excellence model can provide guidance and quality assurance for experiments.
Deep dives
The Value of Experimentation and Building Great Digital Products
In this podcast episode, the host interviews Lucas Ramere, director of experimentation at booking.com, to discuss the importance of experimentation, research, and UX in building great digital products. They highlight the significance of these elements in the travel industry and how they can be used to enhance the customer experience. Lucas emphasizes the need for a center of excellence model to ensure effective communication and collaboration between decentralized teams. He also explains the role of experimentation platforms in providing technical enablement, data collection, and support to other parts of the organization. The conversation emphasizes the importance of making data-driven decisions and constantly challenging assumptions about customer behavior to achieve long-term success.
Finding the Right Organizational Model for Experimentation
Lucas shares insights into the different organizational models for embedding experimentation within an organization. He explains how booking.com initially adopted a fully decentralized model, where he acted as an evangelist and coach for experimentation. However, they later realized the limitations of this model and transitioned to a center of excellence model. Lucas discusses the benefits of having a centralized organization focused on experimentation and providing guidance and quality assurance for experiments. He also acknowledges that the choice of the organizational model depends on the maturity and context of the organization, emphasizing the importance of finding the right approach based on the organization's journey.
The Role of Experimentation in Product Development
The conversation delves into the role of experimentation in product development. Lucas highlights that any change made to a product is essentially an experiment, as it involves testing a hypothesis about the expected impact of the change. He emphasizes the importance of running controlled experiments that measure the short-term effects of changes to understand their impact on customer behavior. Lucas also discusses the need to distinguish between micro-optimization and addressing larger assumptions and challenges in the product space. He emphasizes the value of running experiments on the bigger aspects of a product to gain insights into customer behavior and inform long-term strategic direction.
Addressing Challenges in Measuring the Right Metrics
The podcast explores the challenge of measuring the right metrics in experimentation. Lucas discusses the difficulty of measuring long-term effects, such as loyalty, and how short-term metrics can serve as proxies for those long-term outcomes. He emphasizes the importance of both short-term and long-term impacts and how they can be used to make informed decisions. The conversation touches on the need for collaboration between experimentation, analytics, and user research to ensure insights from each field are integrated and complement each other. Lucas also acknowledges the ongoing challenge of determining the right metrics and measuring the impact of changes in product development.
