Hasty Treat - What is the n+1 problem?

Mar 29, 2021

The hosts dive into the n+1 problem, a common and pesky issue in web development, particularly within GraphQL. They explain how it can lead to inefficient database queries due to complex data structures. Listeners learn effective solutions, like batching queries, to streamline performance. The discussion also highlights practical examples from the podcasting world while emphasizing the importance of managing relationships in data. Plus, discover tools like Data Loader and eager loading that help tackle this challenge head-on.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

N+1 Problem Explained

The N+1 problem occurs when fetching related data, like podcast hosts, requires an additional database query for each item.
This leads to exponentially increasing queries, slowing down requests, especially in GraphQL.

ANECDOTE

Level Up Tutorials N+1 Issue

Scott Tolinsky shares an example of the N+1 problem on Level Up Tutorials.
Fetching tutorial and playlist data caused excessive database calls due to API-layer relationships.

ADVICE

Solving N+1 Problem

Wes Bos advises first checking if the N+1 problem significantly impacts performance before addressing it.
If it does, batch queries by collecting IDs and fetching related data in a single request.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this Hasty Treat, Scott and Wes talk about a common problem you’ll encounter in your development career — the n+1 problem.

Hasura - Sponsor

With Hasura, you can get a fully managed, production-ready GraphQL API as a service to help you build modern apps faster. You can get started for free in 30 seconds, or if you want to try out the Standard tier for zero cost, use the code “TryHasura” at this link: hasura.info. We’ve also got an amazing selection of GraphQL tutorials at hasura.io/learn.

Sentry - Sponsor

If you want to know what’s happening with your code, track errors and monitor performance with Sentry. Sentry’s Application Monitoring platform helps developers see performance issues, fix errors faster, and optimize their code health. Cut your time on error resolution from hours to minutes. It works with any language and integrates with dozens of other services. Syntax listeners new to Sentry can get two months for free by visiting Sentry.io and using the coupon code TASTYTREAT during sign up.

Show Notes

05:00 - What is the n+1 problem?

The N+1 problem can happen in any language but is especially prevalent in GraphQL because it’s so easy to query relationships.

09:33 - The solution

The solution to the n+1 problem is to batch the queries. As you loop over each podcast, keep an array of host IDs to lookup. Once you have looped over the podcasts, make a single query to the database with your large array of podcast host Ids.

11:11 - Should you care?

Sometimes no: its often fine to do multiple DB Calls
Facebook DataLoader
Mercurious
Many ORMs take care of this for you
- These then break it down into my appropriate SQL
- MongoDB Ruby: eager loading
- Laravel ORM does it
- Aggregation pipelines
Prisma N+1: https://www.youtube.com/watch?v=7oMfBGEdwsc&vl=en
Mongoose Populate
Apollo Studio

Tweet us your tasty treats!

Scott’s Instagram
LevelUpTutorials Instagram
Wes’ Instagram
Wes’ Twitter
Wes’ Facebook
Scott’s Twitter
Make sure to include @SyntaxFM in your tweets