265: Mailbag Hot Takes

Nov 7, 2023

The podcast covers topics such as optimizing data warehouses and reports, using Power BI reports server for cloud-averse companies, and developing an organizational policy for Power BI workspace creation. They also discuss new functions for table creation in a lake house, challenges of bringing custom code and libraries into Microsoft fabric, optimizing partition strategy and file size in Microsoft Databricks, and moving data to the cloud with Power BI.

Ask episode

Chapters

Transcript

Episode notes

Introduction

00:00 • 4min

New Function for Table Creation in a Lake House and Optimizing Tables

04:01 • 3min

Discussion on Microsoft fabric and bringing your own library

06:55 • 3min

Optimizing Partition Strategy and File Size in Microsoft Databricks

09:59 • 13min

Moving Data to the Cloud with Power BI

23:21 • 25min

Exploring New Tools and Challenges in Data Engineering

48:50 • 10min

We had a ton of mailbags we needed to get off our chests! Quick Shots on the following:

Can Data Warehouses and Reports Only Pull New Data Instead of Full Refreshes? Thanks for your show. I have been enjoying it for a few months now. I'd like to say I'm new to Data work, but I have been working as an analyst for a few years now.

I have observed that my data warehouse goes out and grabs a massive chunk of data from source databases, and brings it in to the DW and overwrites 99% of the exact same data that is already in the warehouse. Then my on-prem power bi reports do the same thing. Bring in a mass of data from the DW, 99% of it was brought in yesterday. This feels wrong. Is there anyway to just grab the *new* data? Is this the way it has to be, or is it a bad *data smell*? Being relatively new, I don't know how, but think there must be a better way.

Request for Tips and Tricks on Power BI Reports Server for Cloud-Averse Companies

Thanks for your great efforts, I have been listening to your podcast for more than 9 months now.

I wish if you could cover support for PBI Reports Server (Tweeks, tricks, hints, ... etc).

My company is hesitant to load its data to the cloud, we installed PBI RS and currently use it for our internal use.

This is a data culture thing

Developing an Organizational Policy for Power BI Workspace Creation to Support Self-Service BI

Topic: Power BI (Now Fabric) Workspace creation policy for self service Power BI

What should be the organisational policy for creating Power BI workspaces in order to support Self Service Power BI within the organisation?

The factors that affect this are

1) To prevent the explosion of number of workspaces

1.1) Is it a problem if number of workspaces increase exponentially? Why is it a problem? We don’t worry about the explosion of number of child folders in a organisational shared drive, so why is this a problem in Power BI workspaces?

2) Only 1 App can be published in a workspace, therefore, even a single team will need multiple workspaces contributing to the explosion of number of workspaces

3) Who should be allowed to create workspaces?

3.1) Does IT create workspaces when requested by end user teams?

4) Should there be a naming standard/pattern for workspaces?

4.1) Why?

4.2) If so, can you please suggest some naming strategies?

Thank you for your podcast. I enjoy listening to it.

Get in touch:

Send in your questions or topics you want us to discuss by tweeting to @PowerBITips with the hashtag #empMailbag or submit on the PowerBI.tips Podcast Page.

Visit PowerBI.tips: https://powerbi.tips/

Watch the episodes live every Tuesday and Thursday morning at 730am CST on YouTube: https://www.youtube.com/powerbitips

Subscribe on Spotify: https://open.spotify.com/show/230fp78XmHHRXTiYICRLVv

Subscribe on Apple: https://podcasts.apple.com/us/podcast/explicit-measures-podcast/id1568944083‎

Check Out Community Jam: https://jam.powerbi.tips

Follow Mike: https://www.linkedin.com/in/michaelcarlo/

Follow Seth: https://www.linkedin.com/in/seth-bauer/

Follow Tommy: https://www.linkedin.com/in/tommypuglia/