AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
DataTalks.Club
Data Engineering for Fraud Prevention - Angela Ramirez
Oct 6, 2023
Angela Ramirez, a data engineer with experience in fraud prevention, talks about her career journey, the usefulness of knowing ML as a data engineer, best practices for system design and data engineering, working with different types of databases including document and network-based databases, and selecting the appropriate database type to work with. She also discusses the importance of software engineering knowledge in data engineering, data quality check tooling, debugging failed jobs, and working with external data sources.
54:14
Episode guests
AI Summary
Highlights
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Working with external data sources requires establishing data contracts, understanding data reliability, and considering batch versus real-time data.
- Dealing with failed jobs and debugging requires identifying root causes, relying on experience, documentation, and runbooks.
Deep dives
Challenges of Working with External Data Sources
Working with external data sources can present challenges in terms of identifying the right teams to work with, obtaining proper documentation, and ensuring the data remains consistent and accessible. Data engineers must establish data contracts and understand the frequency and reliability of the data received. Additionally, considerations such as batch versus real-time data and potential changes to the data source must be taken into account.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode