Monday Morning Data Chat cover image

Monday Morning Data Chat

#158 - Joe Reis & Matt Housley - Ask Us Anything

Jan 16, 2024
01:06:37
Snipd AI
The hosts discuss the impact of Jack Welch's management style, the importance of building a quality product, the role of DevOps in data engineering, the use of co-pilot and ChatGPT in workflows, recommended tools for data contract work, using open source software in data engineering, the three tiers of data modeling, and tracking personal health data
Read more

Podcast summary created with Snipd AI

Quick takeaways

  • Implement a caching layer between reporting tools and data warehouse for improved performance and scalability while minimizing costs.
  • Reduce query burden on the central data warehouse by employing query optimization techniques, data governance strategies, and providing self-service analytics tools.

Deep dives

Architecting the Reporting Layer for Power BI, Tableau, etc.

When architecting the reporting layer for tools like Power BI and Tableau to read data from a central data warehouse, it is important to consider scalability and performance. One challenge is the bottleneck in querying the data warehouse, which may be difficult to scale. Making copies of data is not an ideal solution due to potential data inconsistency and increased storage costs. One approach to address this is to implement a caching layer that sits between the reporting tools and the data warehouse. This caching layer can store frequently accessed data and respond to queries more efficiently, reducing the load on the data warehouse. Another consideration is using query optimization techniques and data modeling best practices to ensure efficient and optimized queries. By carefully architecting the reporting layer and leveraging caching and query optimization, you can strike a balance between scalability and performance while minimizing costs.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode