Talking Postgres with Claire Giordano

No one likes benchmarking. But it can be one of the highest impact things you do. Jelte Fennema-Nio and Marco Slot joined Claire Giordano and Pino de Candia on this episode of Path To Citus Con* podcast for developers who love Postgres—to discuss their journeys into performance benchmarking. And how it can change the course of your career. Do you need to find bottlenecks in your Postgres? Do you want to build skills with database benchmarks? There are many lovely benchmarking tools in the Postgres world: HammerDB, pgbench, YCSB, BenchBase, perf, & more. And in addition to running benchmarks themselves—asking the right questions, introspection, and profiling matter just as much. *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode:Podcast: Path To Citus Con Ep01: Working in public on open source with Simon Willison & Marco Slot: https://talkingpostgres.com/episodes/working-in-public-on-open-sourceStreetlight effect: https://en.wikipedia.org/wiki/Streetlight_effectBenchBase: https://db.cs.cmu.edu/projects/benchbase/HammerDB: https://www.hammerdb.com/Slides: Intro to benchmarking with pgbench at PGConf NYC 2023 by Melanie Plageman: https://speakerdeck.com/melanieplageman/intro-to-benchmarking-with-pgbenchLocust: https://locust.io/Blog post: How to benchmark performance of Citus and Postgres with HammerDB on Azure by Jelte Fennema-Nio: https://www.citusdata.com/blog/2022/03/12/how-to-benchmark-performance-of-citus-and-postgres-with-hammerdb/Profiling with perf: https://wiki.postgresql.org/wiki/Profiling_with_perfFlame Graphs: https://www.brendangregg.com/flamegraphs.htmlBrendan Gregg’s Website, a super-valuable resource for performance engineering: https://www.brendangregg.com/overview.htmlVideo: Analyzing Postgres performance problems using perf and eBPF by Andres Freund: https://www.youtube.com/watch?v=HghP4D72NocVideo: Explanatory talk about compiler optimization and memory & caches by Matt Godbolt: https://www.youtube.com/watch?v=4_smHyqgDTU&t=52sCompiler Explorer is fantastic, especially if you want to know how different compilers will optimize your code: https://godbolt.org/Mark Callaghan Twitter account @MarkCallaghanDB: https://twitter.com/MarkCallaghanDBPGConf.dev CFP is open until Mon Jan 15 2024 at 11:59pm PST: https://2024.pgconf.dev/cfp/3rd party performance benchmark in 2023 by GigaOM on Transaction Processing & Price-Performance Testing of Distributed SQL Databases:  https://gigaom.com/report/transaction-processing-price-performance-testing/Blog post: "Query from any node" feature for Citus, by Marco Slot: https://www.citusdata.com/blog/2022/06/17/citus-11-goes-fully-open-source/

Do you monitor your Postgres error logs for gold? Lukas Fittl and Rob Treat join Claire Giordano and Pino de Candia on the Path To Citus Con* podcast for developers who love Postgres—to discuss their respective journeys into Postgres monitoring. Have you ever asked yourself: “Why is my query so slow?” Or had to figure out which query is slowing things down? Or why your database server is at 90% CPU? There are so many ways to monitor Postgres: pganalyze, pgMustard, pgBadger, pgDash, your cloud provider’s Query Performance Insights, pg_stat_statements, pg_stat_io, & more. If you’re running Postgres on a managed service, what kinds of things do you need to monitor & optimize for (vs. what will your cloud service provider do)? There’s also a segue on monitoring vs. observability: what’s the difference? *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode:OpenTelemetry: https://opentelemetry.io/pganalyze: https://pganalyze.com/pgDash: https://pgdash.io/pgMustard: https://www.pgmustard.com/pg_stat_statements docs: https://www.postgresql.org/docs/current/pgstatstatements.htmlpg_hint_plan: https://github.com/ossc-db/pg_hint_plan pg_hint_plan hint list: https://github.com/ossc-db/pg_hint_plan/blob/master/docs/hint_list.mdExample for PostgreSQL with pg_hint_plan: https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-optimizer_hints5mins of Postgres by pganalyze: https://www.youtube.com/playlist?list=PLhqxwIAgz78HZhWyu3UyKrCWNk7VWjVpjMonitoring page on PostgreSQL wiki: https://wiki.postgresql.org/wiki/MonitoringPgHero GitHub repo: https://github.com/ankane/pgheroInsights on pgBadger: A PGSQL Phriday #010 Recap: https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/community-insights-on-pgbadger-a-pgsql-phriday-010-recap/ba-p/3880911Get PostgreSQL Logs Into Honeycomb: https://docs.honeycomb.io/getting-data-in/logs/postgresql/Blog post by Lukas Fittl about pg_stat_io by Lukas: https://pganalyze.com/blog/pg-stat-ioBlog post by Andrew Atkinson about pg_stat_io: https://andyatkinson.com/blog/2023/11/01/PostgreSQL-IO-Visibility-wehack-pg_stat_ioBPFtrace by iovisor GitHub repo: https://github.com/iovisor/bpftraceTrace PostgreSQL locks with pg_lock_tracer: https://jnidzwetzki.github.io/2023/01/11/trace-postgresql-locks-with-pg-lock-tracer.htmlsysdig by draios GitHub repo: https://github.com/draios/sysdigUsing BPFtrace to trace PostgreSQL vacuum operations: https://www.timescale.com/blog/using-bpftrace-to-trace-postgresql-vacuum-operations/PostgreSQL Mailing Lists: https://www.postgresql.org/list/psql — PostgreSQL interactive terminal: https://www.postgresql.org/docs/current/app-psql.htmlOngoing discussion thread about pg_stat_statements: https://commitfest.postgresql.org/46/2837/Reconnoiter project referenced by Rob: https://github.com/circonus-labs/reconnoiter/tree/master/sqlFunny tweet about PostgreSQL pronunciation: https://twitter.com/as_w/status/1648373353214885892O11ycast EP63 with Lukas Fittl: https://www.heavybit.com/library/podcasts/o11ycast/ep-63-observability-in-the-database-with-lukas-fittl-of-pganalyzeOxide and Friends podcast: https://oxide-and-friends.transistor.fm/

Is being lazy a good reason to learn SQL? Dimitri Fontaine and Vik Fearing join Claire Giordano and Pino de Candia on the Path To Citus Con* podcast for developers who love Postgres—to discuss whether every data problem can be (or should be) solved in SQL. Have you tried to solve all the Advent of Code puzzles with SQL? Or written a book for application developers about The Art of PostgreSQL? Or tried to solve a murder mystery by running SQL queries? Regardless of whether you pronounce SQL as “sequel” or as “ess-cue-ell”, getting skilled at SQL is like going to the gym for exercise. It’s ideal to do it every day to build up your strength. Also, this episode includes an explanation of what a “declarative” language like SQL is—plus a fun segue into time zones.*[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode, in the order they were covered:Dimitri Fontaine’s blog: https://tapoueh.org/ Advent of Code: https://adventofcode.com/Dimitri’s book, The Art of PostgreSQL: https://theartofpostgresql.com/ Blog post about What’s new in SQL:2023: https://peter.eisentraut.org/blog/2023/04/04/sql-2023-is-finished-here-is-whats-newPostgreSQL Exercises at pgexercises.com: https://pgexercises.com/SQL Murder Mystery for learning SQL: https://mystery.knightlab.com/Pgvector extension for Postgres and AI embeddings: https://github.com/pgvector/pgvectorVik’s Advent of Code puzzle solutions in SQL on GitHub: https://github.com/xocolatl/advent-of-codeStack Overflow data in Postgres, from pgtreats GitHub repo: https://github.com/pgtreats/stackoverflow_in_pgOpenStreetMap runs on Postgres: https://www.openstreetmap.org/#map=4/38.01/-95.84Uber data set: https://github.com/fivethirtyeight/uber-tlc-foil-responseIdeas for fun, open data sets: https://data.world/data-society?entryTypeLabel=dataset&tab=resources“Don’t Do This” Timestamp learnings on PostgreSQL wiki: https://wiki.postgresql.org/wiki/Don't_Do_This#Don.27t_use_timestamp_.28without_time_zone.29

Lots of stories of how folks got started as developers! Andres Freund and Heikki Linnakangas join Claire Giordano and Pino de Candia to explore more paths for getting into Postgres on Path To Citus Con*, the podcast for developers who love Postgres. How do you do development: with a cup of coffee, with music in the background, maybe at 3am? How do you approach mentoring other developers? Why did you stick with Postgres and make it a career? Lots of lively discussion about building not only code, but relationships in the community, in the open. Also, stories about Heikki’s and Andres’s first Postgres patch submissions, and working via the hackers mailing list. Finally, what advice would you give to your younger self starting in the development world? *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode, in the order they were covered: Neon: https://neon.tech/ Rob Conery and Scott Hanselman's book: The Imposter's Handbook (https://twitter.com/shanselman/status/1610805353255677953) Path To Citus Con Ep04: https://talkingpostgres.com/episodes/how-i-got-started-as-a-developer-in-postgres Andres’ first patch to Postgres: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c43feefa806c81d68115ed03a7f723720cefad31  PGConf NYC 2023: https://2023.pgconf.nyc/ Flow book: https://www.google.com/books/edition/Flow/QVjPsd1UukEC  Archives of Postgres hackers mailing list: https://www.postgresql.org/list/pgsql-hackers/ List of Postgres Contributors: https://www.postgresql.org/community/contributors/ Description of Postgres Core Team: https://www.postgresql.org/developer/core/ Postgres Weekly newsletter: https://postgresweekly.com/  

The geospatial world of Postgres is so much more than mapping. Paul Ramsey and Regina Obe join Claire Giordano and Pino de Candia to explore the "where" on Path To Citus Con*, the podcast for developers who love Postgres. What are some of the unexpected use cases for PostGIS, one of the most popular extensions to Postgres? How have Large Language Models helped in the geospatial world? Can you really model almost anything with pgRouting? “Where” is the universal foreign key. They talk about communities and governments using geospatial data and how it's very difficult to build a database that does not have some sort of spatial component to it. Why do people care about PostGIS? Find out more about OpenStreetMap and its place in the open source geospatial world. Finally, Paul and Regina share the origin story for the PostGIS extension to Postgres. *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode, in the order they were covered:PostGIS: https://postgis.net/  FOSS4G NA: https://foss4gna.org/ Ushahidi: https://www.ushahidi.com/  Humanitarian Open Street Map: https://www.hotosm.org/  OpenStreetMap: https://www.openstreetmap.org/ pgRouting: https://pgrouting.org/ Regina Obe’s books: https://locatepress.com/book/pgr  Regina’s book “PostGIS In Action”: https://www.manning.com/books/postgis-in-action-third-edition?experiment=B MobilityDB: https://github.com/MobilityDB/MobilityDB Blog: Analyzing GPS trajectories at scale with Postgres, PostGIS, MobilityDB, & Citus: https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/analyzing-gps-trajectories-at-scale-with-postgres-mobilitydb-amp/ba-p/1859278  OSGeo: https://www.osgeo.org/  Simon Willison’s presentation on "The weird world of LLMs": https://simonwillison.net/2023/Aug/3/weird-world-of-llms/ QGIS: https://qgis.org/en/site/ QGIS “Gentle Introduction” documentation: https://docs.qgis.org/3.28/en/docs/gentle_gis_introduction/ PostGIS Workshops: https://postgis.net/documentation/training/#workshop Locate Press: https://locatepress.com/ FedGeoDay 2023: https://www.fedgeo.us/about-2023 Schedule of FOSS4G NA 2023: https://foss4gna.org/schedule.html#schedule FOSS4G Brazil, December 2024: https://www.osgeo.org/foundation-news/foss4g-2024-has-been-awarded-to-belem-brazil/  Paul's keynote talk at PGConfEU in Lisbon in 2018, titled "Put some "where" in your WHERE clause": https://docs.google.com/presentation/d/1xyXA4-0wmNX7WfiLeH9h10bIkZxrej278-mMaClagys/edit?usp=sharing 

Drop the fear, not the tables. Chelsea Dole and Floor Drees join Claire Giordano and Pino de Candia to explore the app developer perspective on Path To Citus Con*, the podcast for developers who love Postgres. If you’re an app developer, you’re probably already using Postgres. Now what? What do you need to know? Are databases your best friend or your worst enemy? They talk about the steps to becoming more Postgres-savvy. Should you go depth-first or breadth-first in order to learn more about the underlying database? What are Postgres extensions and how do you go about adopting them? Find out more about the strength of what Floor calls “boring technology.” Finally, both guests tell stories of their non-traditional entries into Postgres that led to their deep work with databases today.*[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comLinks mentioned in this episode: Fintech startup where Chelsea works, Brex: https://www.brex.com/  Open source data platform where Floor works, Aiven: https://aiven.io/  “Mission-Critical PostgreSQL Databases on Kubernetes" by Karen Jex at KubeCon Europe 2023: https://www.youtube.com/watch?v=_NBQ9JmOMko The Imposters Handbook by Rob Conery: https://bigmachine.io/products/the-imposters-handbook/ Designing Data-Intensive Applications by Martin Kleppmann: https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/  Devopsdays Amsterdam: https://devopsdays.org/events/2023-amsterdam/welcome/  Building Community in Open Source with Floor Drees on the Last Week in AWS podcast: https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/building-community-in-open-source-with-floor-drees/ pg_stat_statements: https://www.postgresql.org/docs/current/pgstatstatements.html PostGIS: https://postgis.net/  “Postgres tips for optimizing Django & Python performance, from my PyCon workshop” by Louise Grandjonc: https://www.citusdata.com/blog/2020/05/20/postgres-tips-for-django-and-python/  Video of Louise’s PyCon talk, Optimize Django & Python performance with Postgres superpowers: https://youtu.be/dyBLGjCQJHsGrafana: https://grafana.com/pganalyze: https://pganalyze.com/ auto_explain: https://www.postgresql.org/docs/current/auto-explain.html EXPLAIN ANALYZE in PostgreSQL: https://www.postgresql.org/docs/current/sql-explain.html  psql: https://www.postgresguide.com/utilities/psql/ Path To Citus Con Episode 05: My favorite ways to learn more about PostgreSQL with Grant Fritchey and Ryan Booz: https://talkingpostgres.com/episodes/my-favorite-ways-to-learn-more-about-postgresql-with-grant-fritchey-and-ryan-booz Coffee Meets Bagel (dating app): https://coffeemeetsbagel.com/ 

Everyone learns differently. Grant Fritchey and Ryan Booz, database advocates at Redgate focusing on PostgreSQL, talk with Path To Citus Con* co-hosts Claire Giordano and Pino de Candia to explore the learning resources available to developers and users in all the corners of the PostgreSQL world. What drives you to learn: need or curiosity? What can podcasts teach us while we bike to work? Are conference talks good for growing skills, or are they better for networking? What about books? And do older books still have much to offer? It turns out, most people need much more than one approach to build their knowledge. *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comSome of the (many) links shared in the order they were mentioned: Talk: Ryan’s talk Point-in-time query tuning and observability with pg_stat_statements at Citus Con: An Event for Postgres 2022Blog: Learning PostgreSQL with Grant, a series for SQL Server devs learning about Postgres  Podcast: postgres.fm, a weekly podcast about all things Postgres Podcast: Path To Citus Con Episode 01: Working in public on open sourceBlog aggregator: Planet PostgreSQL Email Newsletters: Cooperpress, including the Postgres Weekly email Podcast: Scaling PostgreSQL with Creston Jamison User Groups: PostgreSQL Community User Groups Videos: pganalyze "5 minutes of Postgres," by Lukas Fittl Book: The Art of PostgreSQL, by Dimitri Fontaine Book: PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries, by Henrietta Dombrovskaya Book: SQL Performance Explained, by Markus Winand Blog: Modern SQL, by Markus WinandBlog: Use The Index, Luke, by Markus WinandBook: Database Administration, by Craig Mullins Book: A Curious Moon, by Rob Conery Book: The Little SQL Book, by Rob Conery, “Learn SQL While Watching Football This Weekend - Free!” Event: PGDay Chicago Blog: Redgate – Simple Talk Videos: CMU Database Group’s Talks on YouTube: Quarantine (2020), First Dose (2021), Second Dose (2021),  Booster (2022) Crunchy Data’s Postgres Playground Blog: CYBERTEC Blog: Citus Open Source Blog Talk: How To Make Your Postgres Blog Posts Reach A Ton More People, by Claire GiordanoConference: PGCon 2023, super useful to watch recorded talks after the fact Conference: PGConf.EU, 2022, good example of an in-person event with lots of opportunities for learning Conference: Citus Con: An Event for Postgres 2023 Conference: PGConf NYC 2023 Blog Series: PGSQL Phriday, created by Ryan BoozBlog Series: PostgreSQL Person of the Week, by Andreas Scherbaum Blog: Robert Haas' blogBlog: select * from depesz;Book: PostgreSQL 14 internals, by Egor Rogov

In this episode of Path To Citus Con*, Melanie Plageman, a PostgreSQL hacker working at Microsoft, and Thomas Munro, PostgreSQL developer and committer also at Microsoft talk with co-hosts Claire Giordano and Pino de Candia. They talk through all the different ways they got started as developers. Does making your first patch to Postgres get you hooked for a lifetime? Do you have to be a tinkerer to be a good software engineer? What is the “toothbrush test”—and how do you make your avocation be your vocation? We hear stories about dropping out of school or dropped out of career fields before they found their true passions in development and Postgres. *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comSome of the links mentioned in the order they were said: Parallelism in PostgreSQL 15: Thomas’ Citus Con talk Additional IO Observability in Postgres with pg_stat_io: Melanie’s Citus Con talk Visualizing PostgreSQL I/O Performance for Development: Melanie’s talk at PGCon 2023Add pg_stat_io view, providing more detailed IO statistics, committed by Melanie Plageman in PG 16  Neil deGrasse Tyson’s podcast StarTalk From Nand to Tetris by Noam Nisan and Shimon Schocken Sinclair ZX81 All Things Open conference PostgreSQL BuildFarm Queues in PostgreSQL: Thomas’ 2022 talk 

Álvaro Herrera, and Boriss Mejías, both longtime members of the Postgres developer community, explore the value of giving conference talks—as well as the work involved, the time it takes, and the many different types of conference talks, including presentations about about failure and things that have gone wrong.  In this episode of Path To Citus Con*, Claire and Pino guide the conversation on questions like: Should you add humor to your talks? How does your personality—introvert or extrovert—affect your conference presentations? Is it OK to give the same conference talk at different events?  *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comSome of the links mentioned in the order they appeared: Postgres Storytelling: Support in the Darkest Hour: Boriss’ Citus Con talk A Curious Moon book by Rob Conery Tomas Vondra's talks on YouTubepgDay Paris 2022FOSDEM PostgreSQL devroomFOSDEM PGDAY 2023Nordic PGDay 2022PGConf.EU 2022LFMF: How a CREATE INDEX CONCURRENTLY led to a 6 hour downtime by Gunnar "Nick" Bluth

Join Claire and Pino as they talk with Citus and Postgres open source team members to explore how to get Postgres ready for the next 100 million users. What will future Postgres users look like? How will the Postgres development process evolve with more users? What are the common challenges faced by Postgres users?  Citus open source team members Abdullah Ustuner and Burak Yucesoy are joined by Postgres open source teammates Melanie Plageman and Samay Sharma—and co-hosts Claire Giordano and Pino de Candia—in this episode of Path To Citus Con*. Listen to the deep dive on what it means to scale the code and the community far beyond the Postgres world of today. *[Update: July 2024] Path To Citus Con has been renamed to Talking Postgres. All of the past podcast episodes from Path To Citus Con—now called Talking Postgres with Claire Giordano—can be found here: https://talkingpostgres.comSome links from the show in the order they were mentioned: Additional IO Observability in Postgres: Melanie's talk at Citus Con 2023Optimizing Postgres for write heavy workloads ft. Checkpoint and WAL configs: Samay's talk at Citus Con 2023The Design of Postgres, by Michael Stonebraker and Lawrence A. Rowe, 1986 HyperLogLog PostGIS timescale/pgspot