245. The Avocado Challenge: Tell The Future

Nov 26, 2020

Ask episode

Chapters

Transcript

Episode notes

It’s hard to predict the future, but you can be better at predicting the future.

All you need is a few delicious avocados.

Even the “experts” are bad at predicting the future

Wharton professor Phillip Tetlock wanted to make the future easier to predict. So he held “forecasting tournaments,” in which experts from a variety of fields made millions of predictions about global events.

Tetlock found that experts are no better at predicting the future than dart-throwing chimps. In fact, the more high-profile experts – the ones who get invited onto news shows – were the worst at making predictions.

But, Tetlock found that some people are really great at telling the future. He calls them “Superforecasters”, and regardless of their area of expertise, they consistently beat the field with their predictions.

Tetlock also found that with a little training, people can improve their forecasting skills. The superforecasters in Tetlock’s Good Judgement Project – people from all backgrounds working with publicly-available information – make forecasts 30% better than intelligence officers with access to classified information.

Creative work is uncertain. Does it have to be?

As someone working in the “Extremistan” world of creative work, I’m always trying to improve my forecasting skills. If I publish a tweet, how many likes will it get? If I write a book, how many copies will it sell?

The chances of getting any of these predictions exactly right are so slim, it doesn’t feel worth it to try to predict these things. But that doesn’t mean I can’t rate my predictions and make those predictions better.

Introducing the Avocado Challenge

If you would like to be better at predicting the future, I have a challenge for you. I call it the Avocado Challenge.

Elon Musk recently asked on Twitter “What can’t we predict?” I answered “whether or not an avocado is ready to open.” 12 likes. People agree with me.

https://twitter.com/kadavy/status/1309643017599569920

Here’s how the Avocado Challenge works.

The next time you’re about to open an avocado, make a prediction: How confident are you the avocado is ripe?

Choose a percentage of confidence, such as 50% or 20% – or if you’re feeling lucky, 100%. To make it simple, you can rate your confidence on a scale of 0 to 10. State your prediction out loud or write it down.

Now, open the avocado. Is it ripe? Yes or no?

Scoring your avocado predictions

You now have two variables: Your prediction as stated in percentage confidence, and the outcome of avocado ripeness.

With these two variables, you can calculate what’s called a Brier score. This tells you just how good your forecast was. The Brier score is what Phillip Tetlock uses to score his forecasting tournaments.

Two variables: confidence and outcome

It works like this: Translate your percentage confidence into a decimal between 0 and 1. So 50% would be 0.5, 20% would be 0.2, and 100% would just be 1.

Now, translate the avocado ripeness outcome into a binary number. If the avocado was not ripe, your outcome value is “0.” If the avocado was ripe, your outcome value is “1.” (You may wonder: How do I determine whether or not an avocado is ripe? I’ll get to that in a minute. Let’s pretend for a second it’s easy.)

Calculating your Brier score

Once you have those two variables, there are two steps to follow to find out your Brier score:

Subtract the outcome value from your confidence value. If I was 50% confident the avocado would be ripe that confidence value is 0.5. If the avocado was in fact ripe I subtract the outcome value of 1 from 0.5 to get -0.5.
Square that number, or multiply it by itself. -0.5² = 0.25.

Our Brier score is 0.25. Is that good or bad?

The lower your Brier score, the better your prediction was. If you were 100% confident the avocado would be ripe and it was not, your Brier score would be 1 – the worst score possible. If you were 100% confident the avocado would be ripe and it was ripe, your Brier score would be 0 – the best score possible. So, 0.25 is pretty solid.

Predict your next 30 avocados

This is a fun exercise to try one time, but it doesn’t tell you a whole lot about your forecasting skills overall, and it doesn’t help you improve your forecasting skills. Where it gets interesting and useful is when you make a habit of the Avocado Challenge.

After you’ve tried the Avocado Challenge a couple times, make a habit out of it. For 30 consecutive avocados, tally your results. Calculate your Brier score, and find the average of your 30 predictions. If you regularly open avocados with a roommate or partner, make a competition out of it.

My partner and I predicted the ripeness of, then opened, 36 avocados over the course of several weeks. We recorded our predictions and outcomes on a notepad on the fridge – then tallied our results in a spreadsheet.

Our findings: 28% of avocados were ripe. Her Brier score was 0.22 – mine was 0.19. (I win!)

The Avocado Challenge teaches you to define your predictions

Most of us don’t make predictions according to our percentage confidence. We say, “I think so and so is going to win the election,” or “I think it might rain.”

Phillip Tetlock even found this with political pundits – the ones who get lots of airtime on news shows. They’ll say things like “there’s a distinct possibility.” That’s not a forecast.

If so and so wins the election, you can say, “ha! I knew it!” If it didn’t rain, you can remind your friend you said you thought it might rain. And what does a “distinct possibility” mean? You can be “right” either way.

And when it comes to getting airtime on news shows, the news show doesn’t care if the political pundit gets their prediction right. All that matters is they can be exciting on camera, speak in sound bites, argue a clear point, and hold the viewer’s attention a little longer so it can be sold to advertisers during the commercial break.

We normally don’t make our predictions with a percentage confidence, because we aren’t used to it. The Avocado Challenge gets you in the habit of rating the confidence of your predictions.

The Avocado Challenge helps you define reality

The Avocado Challenge also helps you define reality. This is something we’re also bad at.

If you’re on a walk with your friend and you say you think it’s going to rain, how much rain equals rain? By what time is it going to rain? You’re traveling on foot – is it going to rain where the walk started, or the place you’ll be a half hour from now?

To rate your predictions and become a better forecaster, you need to make falsifiable claims.

It’s hard to tell if an avocado is ripe before you open the avocado, but it’s also hard to tell if an avocado is ripe after you open the avocado. You’ll have to come up with criteria for determining whether or not an avocado should be defined as “ripe.”

When we did the Avocado Challenge, we defined a “ripe” avocado as a “perfect” avocado: uniform green color, with the meat of the avocado sticking to no more than 5% of the pit.

The Avocado Challenge can improve your real-life predictions

A few weeks after we did the Avocado Challenge, my partner and I were at her family’s finca – a rustic cabin in the Colombian countryside.

It was Sunday afternoon, and we were getting ready to head back to Medellín. I was eager to get home and get ready for my week. I asked my partner what time we would leave. She said about 3 p.m.

As I mentioned on episode 235, the Colombian sense of time takes some getting used to for me as an American. Even though my partner is a very prompt person, I’m also aware of “the planning fallacy.” I know the Sydney Opera House opened ten years late and cost 15 times the projected budget to build.

So when I looked at the Mitsubishi Montero parked in the grass, and thought about how long it might take to pack in eight people, three dogs, and a little white rabbit, the chances of us leaving right at 3 p.m. seemed slim.

Fortunately, we had done the Avocado Challenge. I asked my partner, in Spanish, “what’s your percentage confidence we’ll leave before an hour after 3 p.m. – 4 p.m.?” She shifted into Avocado mode, thought a bit, and said sesenta por ciento. She was 60% sure we’d leave before 4 p.m.

That didn’t seem super confident, so I asked for another forecast. I asked what her percentage confidence was that we’d leave before 5 p.m. – two hours after the target time. She said cien por ciento. She was 100% sure we’d leave before 5 p.m.

Now, instead of choosing between expecting to leave at exactly 3 p.m. – or leaving “whenever” – I now had a range. It was a range I could trust from someone with experience with similar situations – and training in forecasting.

The time we did leave: 3:30 p.m. My partner’s Brier score for that first prediction: 0.16. Average Brier score for the two predictions: 0.08. Not bad.

Mind Management, Not Time Management now available!

After nearly a decade of work, Mind Management, Not Time Management is now available! This book will show you how to manage your mental energy to be productive when creativity matters. Buy it now!

My Weekly Newsletter: Love Mondays

Start off each week with a dose of inspiration to help you make it as a creative. Sign up at: kadavy.net/mondays.

Listener Showcase

Abby Stoddard makes the Dunnit app – the "have-done list." It’s a minimalist tool designed to motivate action and build healthy habits.

About Your Host, David Kadavy

David Kadavy is author of Mind Management, Not Time Management, The Heart to Start and Design for Hackers. Through the Love Your Work podcast, his Love Mondays newsletter, and self-publishing coaching David helps you make it as a creative.

Follow David on:

Subscribe to Love Your Work

Support the show on Patreon

Put your money where your mind is. Patreon lets you support independent creators like me. Support now on Patreon »

Show notes: http://kadavy.net/blog/posts/avocado-challenge