Evolving Data Storage Solutions

This chapter explores the challenges of traditional data storage methods and the need for more efficient solutions. It highlights the advantages of using Apache Iceberg over ClickHouse for managing large datasets, emphasizing its role in modern cloud storage and analytics.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

The ClickHouse® project is a rising star in observability and analytics, challenging performance conventions with its breakneck speed. This open source OLAP column store, originally developed at Yandex to power their web analytics platform at massive scale, has quickly evolved into one of the hottest open source observability data stores around. Its published performance benchmarks have been the topic of conversation, outperforming many legacy databases and setting a new bar for fast queries over large volumes of data.

Our guest for this episode is Robert Hodges, CEO of Altinity — the second largest contributor to the ClickHouse project. With over 30 years of experience in databases, Robert brings deep insights into how ClickHouse is challenging legacy databases at scale. We’ll also explore Altinity’s just-launched groundbreaking open source project—Project Antalya—which extends ClickHouse with Apache Iceberg shared storage, unlocking dramatic improvements in both performance and cost efficiency. Think 90% reductions in storage costs and 10 to 100x faster queries, all without requiring any changes to your existing applications.

The episode was live-streamed on 20 May 2025 and the video is available at https://www.youtube.com/watch?v=VeyTL2JlWp0

You can read the recap post: https://medium.com/p/2004160b2f5e/

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

⁠⁠https://www.youtube.com/@openobservabilitytalks⁠

https://www.twitch.tv/openobservability⁠

Show Notes:

00:00 - Intro

01:38 - ClickHouse elevator pitch

02:46 - guest intro

04:48 - ClickHouse under the hood

08:15 - SQL and the database evolution path

11:20 - the return of SQL

16:13 - design for speed

17:14 - use cases for ClickHouse

19:18 - ClickHouse ecosystem

22:22 - ClickHouse on Kubernetes

31:45 - know how ClickHouse works inside to get the most out of it

38:59 - ClickHouse for Observability

46:58 - Project Antalya

55:03 - Kubernetes 1.33 release

55:32 - OpenSearch 3.0 release

56:01 - New Permissive License for ML Models Announced by the Linux Foundation

57:08 - Outro

Resources:

ClickHouse on GitHub: https://github.com/ClickHouse/ClickHouse
Shopify’s Journey to Planet-Scale Observability: https://medium.com/p/9c0b299a04dd
Project Antalya: https://altinity.com/blog/getting-started-with-altinitys-project-antalya
https://cmtops.dev/posts/building-observability-with-clickhouse/
Kubernetes 1.33 release highlights: https://www.linkedin.com/feed/update/urn:li:activity:7321054742174924800/
New Permissive License for Machine Learning Models Announced by the Linux Foundation: https://www.linkedin.com/feed/update/urn:li:share:7331046183244611584