Data Engineering Podcast Using Your Data Warehouse As The Source Of Truth For Customer Data With Hightouch
Jan 19, 2021
59:34
Summary
The data warehouse has become the central component of the modern data stack. Building on this pattern, the team at Hightouch have created a platform that synchronizes information about your customers out to third party systems for use by marketing and sales teams. In this episode Tejas Manohar explains the benefits of sourcing customer data from one location for all of your organization to use, the technical challenges of synchronizing the data to external systems with varying APIs, and the workflow for enabling self-service access to your customer data by your marketing teams. This is an interesting conversation about the importance of the data warehouse and how it can be used beyond just internal analytics.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days. Datafold helps Data teams gain visibility and confidence in the quality of their analytical data through data profiling, column-level lineage and intelligent anomaly detection. Datafold also helps automate regression testing of ETL code with its Data Diff feature that instantly shows how a change in ETL or BI code affects the produced data, both on a statistical level and down to individual rows and values. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Go to dataengineeringpodcast.com/datafold today to start a 30-day trial of Datafold. Once you sign up and create an alert in Datafold for your company data, they will send you a cool water flask.
- This episode of Data Engineering Podcast is sponsored by Datadog, a unified monitoring and analytics platform built for developers, IT operations teams, and businesses in the cloud age. Datadog provides customizable dashboards, log management, and machine-learning-based alerts in one fully-integrated platform so you can seamlessly navigate, pinpoint, and resolve performance issues in context. Monitor all your databases, cloud services, containers, and serverless functions in one place with Datadog’s 400+ vendor-backed integrations. If an outage occurs, Datadog provides seamless navigation between your logs, infrastructure metrics, and application traces in just a few clicks to minimize downtime. Try it yourself today by starting a free 14-day trial and receive a Datadog t-shirt after installing the agent. Go to dataengineeringpodcast.com/datadog today to see how you can enhance visibility into your stack with Datadog.
- Your host is Tobias Macey and today I’m interviewing Tejas Manohar about Hightouch, a data platform that helps you sync your customer data from your data warehouse to your CRM, marketing, and support tools
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you start by giving an overview of what you are building at Hightouch and your motivation for creating it?
- What are the main points of friction for teams who are trying to make use of customer data?
- Where is Hightouch positioned in the ecosystem of customer data tools such as Segment, Mixpanel, Amplitude, etc.?
- Who are the target users of Hightouch?
- How has that influenced the design of the platform?
- What are the baseline attributes necessary for Hightouch to populate downstream systems?
- What are the data modeling considerations that users need to be aware of when sending data to other platforms?
- Can you describe how Hightouch is architected?
- How has the design of the platform evolved since you first began working on it?
- What goals or assumptions did you have when you first began building Hightouch that have been modified or invalidated once you began working with customers?
- Can you talk through the workflow of using Hightouch to propagate data to other platforms?
- How do you keep data up to date between the warehouse and downstream systems?
- What are the upstream systems that users need to have in place to make Hightouch a viable and effective tool?
- What are the benefits of using the data warehouse as the source of truth for downstream services?
- What are the trends in data warehousing that you are keeping a close eye on?
- What are you most excited for?
- Are there any that you find worrisome?
- What are some of the most interesting, unexpected, or innovative ways that you have seen Hightouch used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while building Hightouch?
- When is Hightouch the wrong choice?
- What do you have planned for the future of the platform?
Contact Info
- @tejasmanohar on Twitter
- tejasmanoharon GitHub
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Links
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA