Data Engineering Podcast Exploring The Evolution And Adoption of Customer Data Platforms and Reverse ETL
Nov 5, 2021
01:02:07
Summary
The precursor to widespread adoption of cloud data warehouses was the creation of customer data platforms. Acting as a centralized repository of information about how your customers interact with your organization they drove a wave of analytics about how to improve products based on actual usage data. A natural outgrowth of that capability is the more recent growth of reverse ETL systems that use those analytics to feed back into the operational systems used to engage with the customer. In this episode Tejas Manohar and Rachel Bradley-Haas share the story of their own careers and experiences coinciding with these trends. They also discuss the current state of the market for these technological patterns and how to take advantage of them in your own work.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Struggling with broken pipelines? Stale dashboards? Missing data? If this resonates with you, you’re not alone. Data engineers struggling with unreliable data need look no further than Monte Carlo, the world’s first end-to-end, fully automated Data Observability Platform! In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Monte Carlo monitors and alerts for data issues across your data warehouses, data lakes, ETL, and business intelligence, reducing time to detection and resolution from weeks or days to just minutes. Go to dataengineeringpodcast.com/montecarlo and start trusting your data with Monte Carlo today!
- Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? Hightouch is the easiest way to sync data into the platforms that your business teams rely on. The data you’re looking for is already in your data warehouse and BI tools. Connect your warehouse to Hightouch, paste a SQL query, and use their visual mapper to specify how data should appear in your SaaS systems. No more scripts, just SQL. Supercharge your business teams with customer data using Hightouch for Reverse ETL today. Get started for free at dataengineeringpodcast.com/hightouch.
- Your host is Tobias Macey and today I’m interviewing Rachel Bradley-Haas and Tejas Manohar about the combination of operational analytics and the customer data platform
Interview
- Introduction
- How did you get involved in the area of data management?
- Can we start by discussing what it means to have a "customer data platform"?
- What are the challenges that organizations face in establishing a unified view of their customer interactions?
- How do the presence of multiple product lines impact the ability to understand the relationship with the customer?
- We have been building data warehouses and business intelligence systems for decades. How does the idea of a CDP differ from the approaches of those previous generations?
- A recent outgrowth of the focus on creating a CDP is the introduction of "operational analytics", which was initially termed "reverse ETL". What are your opinions on the semantics and importance of these names?
- What is the relationship between a CDP and operational analytics? (can you have one without the other?)
- How have the capabilities of operational analytics systems changed or evolved in the past couple of years?
- What new use cases or capabilities have been unlocked as a result of these changes?
- What are the opportunities over the medium to long term for operational analytics and customer data platforms?
- What are the most interesting, innovative, or unexpected ways that you have seen operational analytics and CDPs used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on operational analytics?
- When is a CDP the wrong choice?
- What other industry trends are you keeping an eye on? What do you anticipate will be the next breakout product category?
Contact Info
- Rachel
- Tejas
- @tejasmanohar on Twitter
- tejasmanohar on GitHub
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
- Big-Time Data
- Hightouch
- Segment
- Customer Data Platform
- Treasure Data
- Rudderstack
- Airflow
- DBT Cloud
- Fivetran
- Stitch
- PLG == Product Led Growth
- ABM == Account Based Marketing
- Materialize
- Transform
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA