Engineering Enablement by Abi Noda cover image

Intercom’s approach to a great on-call experience | Brian Scanlan (Intercom)

Engineering Enablement by Abi Noda

How to Fine-Tune Your on-Call Experience

3min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

I'm curious at a high level are there a set of metrics or signals that you're relying on to fine-tune this balance at all times for example. We review these things constantly in terms of our availability metrics and our we survey our on-call engineers around their experiences as well. It's not as simple as like a single SLO that drives these kind of decisions and it's a number of inputs with the experience of our engineers being a large factor in there as well but lots of work we do look at the customer experience, support resolution satisfaction scores from customers versus internal developer burnout sentiment. I didn't expect to learn so much when I asked that question but that was really

In this deep-dive episode, Brian Scanlan, Principal Systems Engineer at Intercom, describes how the company’s on-call process works. He explains how the process started and key changes they’ve made over the years, including a new volunteer model, changes to compensation, and more.

Discussion points:

(1:28) How on-call started at Intercom
(10:11) Brian’s background and interest in being on-call
(14:06) Getting engineers motivated to be on-call
(16:37) Challenges Intercom saw with on-call as it grew
(19:53) Having too many people on-call
(23:20) Having alarms that aren’t useful
(26:03) Recognizing uneven workload with compensation
(27:22) Initiating changes to the on-call process
(30:08) Creating a volunteer model
(33:02) Addressing concerns that volunteers wouldn’t take action on alarms
(34:40) Equitability in a volunteer model
(36:36) Expectations of expertise for being on-call
(40:56) How volunteers sign up
(44:15) The Incident Commander role
(46:19) Using code review for changes to alarms
(50:02) On-call compensation
(52:50) Other approaches to compensating on-call
(55:08) Whether other companies should compensate on-call
(57:32) How Intercom’s on-call process compares to other companies
(1:00:46) Recent changes to the on-call process
(1:04:13) Balancing responsiveness and burnout
(1:07:12) Signals for evaluating the on-call process

‍

Mentions and links:

Follow Brian on LinkedIn or Twitter
Brian’s article: How we fixed our on call process to avoid engineer burnout
Gergely Orosz’s On-Call Compensation

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

App store banner

Play store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode