As AI takes over the world, data is more than ever “the new oil”, and data engineering is the discipline that makes data usable behind the scenes. In this episode, we dive deep into the present and future of data engineering with Ben Rogojan, also known as the Seattle Data Guy. A seasoned data engineering consultant, Ben has built a big brand and reputation in the field with over 100k followers on platforms like YouTube and Substack.
We started the conversation with a deep dive into data engineering as a profession: what do data engineers actually do? What is the career path, and what should aspiring data engineers learn?
We then explored some of the biggest stories of 2024 (including the rise of Iceberg) and went into some predictions for 2025, as a way to discuss some key topics everyone should be familiar with in data engineering, including the integration of AI in data workflows, the potential for automation, and why SQL isn't going anywhere. Discover how companies are navigating the complexities of data infrastructure, the rise of open table formats like Iceberg, and the ongoing battle between data giants like Snowflake and Databricks.
Ben Rogojan
Website - https://www.theseattledataguy.com
Newsletter - https://seattledataguy.substack.com
LinkedIn - https://www.linkedin.com/company/seattle-data-guy
X/Twitter - https://x.com/seattledataguy
FIRSTMARK
Website - https://firstmark.com
X/Twitter - https://twitter.com/FirstMarkCap
Matt Turck (Managing Director)
LinkedIn - https://www.linkedin.com/in/turck/
X/Twitter - https://twitter.com/mattturck
(00:00) Intro
(01:20) Why 2025 will be huge for data engineering
(02:55) The story of the Seattle Data Guy
(06:51) What exactly is data engineering?
(07:41) Data, AI, and ML: where do they overlap?
(09:23) Data analyst vs. data engineer vs. data scientist: what’s the difference?
(11:20) A day in the life of a data engineer
(12:58) Data engineering: Silicon Valley vs. everywhere else
(15:27) How to become an AI engineer
(28:46) Will AI replace AI engineers?
(33:42) Why is the data world so complex?
(36:53) The functional consolidation of the data world
(38:34) Big data stories from 2024
(39:28) Why Iceberg is a game-changer
(46:02) How startups manage data in their early days
(48:44) Seattle Data Guy’s favorite tools
(50:09) Bold predictions for 2025
