Data Engineering Podcast
Business Intelligence Beyond The Dashboard With ClicData
Podcast summary created with Snipd AI
Quick takeaways
- Business intelligence extends beyond dashboards to data exploration functionalities for effective decision-making.
- Data management tools like ClicData prioritize essential metrics daily and offer new insights module for users.
- Ensuring data security and adaptability across regions are crucial for platforms like ClicData to address customer needs.
Deep dives
Atlan: A Collaborative Workspace for Data Teams
Atlan is a collaborative workspace designed for data-driven teams, akin to GitHub for engineering or Figma for design teams. It serves as a central hub for various data assets, from tables and dashboards to SQL snippets and code, enabling teams to establish a unified source of truth for their data assets. With integrations with tools like Snowflake, Slack, and Looker, Atlan facilitates collaboration across the modern data stack.
Click Data: Revolutionizing Business Intelligence
Click Data is a business intelligence and data management platform that aims to empower businesses of all sizes to consolidate their data into a single data warehouse, create visualizations, and share insights with a wider audience. The platform's founder, Tulmo Silva, was motivated to create Click Data after facing challenges with traditional business intelligence tools, realizing the need for a more effective way to leverage data for decision-making.
Challenges and Innovations in Business Intelligence
The field of business intelligence still faces challenges despite technological advancements. Traditional players like Tableau and Looker continue to dominate the market, emphasizing the importance of connecting data sources, cleansing and matching data, intelligent visualization, and automation. Innovations in embedding analytics and data management within dashboards are crucial for enhancing decision-making processes and driving meaningful actions based on data insights.
Evolution of Business Reporting in Data Management
The podcast episode highlights how data management tools like Clek Data have evolved from being purely business reporting tools to incorporating data exploration functionalities. The speaker emphasizes the shift towards prioritizing support for businesses with essential metrics daily but also the need for users to have a clear understanding of the metrics they are looking for. Clek Data introduces a new module called Insights to aid users in this aspect. Furthermore, the discussion delves into the importance of building dashboards without reliance on intricate data modeling to avoid performance issues.
Challenges and Innovations in Data Security and Expansion
The episode delves into the challenges and innovations in data security faced by companies like Clek Data, especially when dealing with storing and managing diverse data sets across multiple regions. The emphasis is on ensuring high levels of physical and digital security through encryption protocols like HTTPS and geo-replication. The podcast also touches on the adaptability of the platform to cater to different customer needs regarding data residency and encryption key management. Additionally, it explores the continuous evolution of the platform to address scalability and cost-effectiveness challenges in providing global data services.
Summary
Business intelligence is often equated with a collection of dashboards that show various charts and graphs representing data for an organization. What is overlooked in that characterization is the level of complexity and effort that are required to collect and present that information, and the opportunities for providing those insights in other contexts. In this episode Telmo Silva explains how he co-founded ClicData to bring full featured business intelligence and reporting to every organization without having to build and maintain that capability on their own. This is a great conversation about the technical and organizational operations involved in building a comprehensive business intelligence system and the current state of the market.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days. Datafold helps Data teams gain visibility and confidence in the quality of their analytical data through data profiling, column-level lineage and intelligent anomaly detection. Datafold also helps automate regression testing of ETL code with its Data Diff feature that instantly shows how a change in ETL or BI code affects the produced data, both on a statistical level and down to individual rows and values. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Go to dataengineeringpodcast.com/datafold today to start a 30-day trial of Datafold.
- Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. By acting as a virtual hub for data assets ranging from tables and dashboards to SQL snippets & code, Atlan enables teams to create a single source of truth for all their data assets, and collaborate across the modern data stack through deep integrations with tools like Snowflake, Slack, Looker and more. Go to dataengineeringpodcast.com/atlan today and sign up for a free trial. If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription
- Your host is Tobias Macey and today I’m interviewing Telmo Silva about ClicData,
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you describe what ClicData is and the story behind it?
- How would you characterize the current state of the market for business intelligence?
- What are the systems/capabilities that are required to run a full-featured BI system?
- What are the challenges that businesses face in developing in-house capacity for business intelligence?
- Can you describe how the ClicData platform is architected?
- How has it changed or evolved since you first began working on it?
- How are you approaching schema design and evolution in the storage layer?
- How do you handle questions of data security/privacy/regulations given that you are storing the information on behalf of the business?
- In your work with clients what are some of the challenges that businesses are facing when attempting to answer questions and gain insights from their data in a repeatable fashion?
- What are some strategies that you have found useful for structuring schemas or dashboards to make iterative exploration of data effective?
- What are the most interesting, innovative, or unexpected ways that you have seen ClicData used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on ClicData?
- When is ClicData the wrong choice?
- What do you have planned for the future of ClicData?
Contact Info
- @telmo_clicdata on Twitter
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
- ClicData
- Tableau
- Superset
- Pentaho
- D3.js
- Informatica
- Talend
- TIBCO Spotfire
- Looker
- Bullet Chart
- PostgreSQL
- Azure
- Crystal Reports
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Remember Everything You Learn from Podcasts
