The Joe Reis Show cover image

Jordan Tigani - Why Small Data is Awesome, DuckDB, and More

The Joe Reis Show

NOTE

Small Data Can Be Mighty

Measuring database performance based on user experience emphasizes the importance of query response times rather than absolute speed differences between systems. Users prioritize responsiveness, such as the number of queries executing in under a second, over minute variations in performance. A study of half a billion queries on Redshift reveals that many users do not work with massive datasets; even those with large datasets often query only small portions. This indicates that advanced systems may not be necessary for typical user tasks. With the modern separation of storage and compute, effective data handling can occur without extensive computational resources for small data inquiries. The emerging interest in small data demonstrates its significance in current data practices, leading to events that celebrate its value.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner