102. When AI goes wrong - Zillow case study

Jun 8, 2022

Ask episode

Chapters

Transcript

Episode notes

Real estate marketplace Zillow took $500 million in write downs and fired 25% of its workforce in late 2021, largely because a pricing algorithm made a mistake. Learn what went wrong and how to avoid it.

Learning notes from this episode:

Zillow started life as an online marketplace for real-estate in 2005, and monetised via advertising. The company decided to diversity and get into the business of flipping houses. Zillow used an algorithm to find properties it believed were undervalued and bought them.
When the housing market turned, Zillow was left with massive losses, but its competitors were not.
Behind every algorithm is a set of assumptions made by humans. For example, factors like crime rates and commuting distances affect real estate prices and would go into a property pricing algorithm.
Machine learning models often assume that the past equals the future, but that is generally not the case in the real world. When the economy changed, the Zillow algorithm did not adjust.
Data scientist Prof Datta from Carnegie Mellon says that tools can be built to monitor whether an algorithm is accurate or not. In essence: create a tool to monitor another tool.
Non-technical leaders can also participate: ask your tech colleagues how the model is being updated when circumstances change. Simply asking: "the economy is changing. Have we adjusted our pricing algorithm for these changes? How have we done it?" is a good start.

Resources mentioned in this episode:

Tech for Non-Techies podcast: Intro to Algorithms and Big Data
Inside Big Data: The $500mm+ Debacle at Zillow Offers – What Went Wrong with the AI Models?

-----

If you like learning about how tech products and profits get made, you'll like our newsletter.

It's funny too. Sign up here.

-----

There are 2 ways to apply this work to your goals:

For individuals, APPLY FOR A CONSULTATION CALL for Tech For Non-Techies membership.

For companies: If you want to increase productivity, innovation and diversity, then your non-technical teams need to learn how to collaborate with the techies.

BOOK A CALL to discuss bespoke training & consulting.

We love hearing from our readers and listeners. So if you have questions about the content or working with us, just get in touch on info@techfornontechies.co

Say hi to Sophia on Twitter and follow her on LinkedIn.

Following us on Facebook, Instagram and TikTok will make you smarter.