Why Labelling Is Just Superhard

There's tons of these stories around in industry, tons of themye we talked about how basically there may be a lot of fraud happening. And it's like if a tree falls in the woods, doesn't make a sound. So this is also yet another reason why labelling is just superhard, because there's bias coming at you from all sorts of directions. It's very much a human problem. And we humans are not perfect.

Play episode from 26:14

chevron_right

Transcript

chevron_right

Transcript

Episode notes

MLOps Coffee Sessions #83 with Vincent Warmerdam, Better Use Cases for Text Embeddings.

Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠⁠⁠

// Abstract
Text embeddings are very popular, but there are plenty of reasons to be concerned about their applications. There's algorithmic fairness, compute requirements, as well as issues with the datasets that they're typically trained on.
In this session, Vincent gives an overview of some of these properties while also talking about an underappreciated use-case for the embeddings: labeling!

// Bio
Vincent D. Warmerdam is a senior data professional who has worked as an engineer, researcher, team lead, and educator in the past. He's especially interested in understanding algorithmic systems so that one can prevent failure. As such, he has a preference for simpler solutions that scale, as opposed to the latest and greatest from the hype cycle. He currently works as a Research Advocate at Rasa, where he collaborates with the research team to explain and understand conversational systems better.

Outside of Rasa, Vincent is also well known for his open-source projects (scikit-lego, human-learn, doubtlab, and more), collaborations with open source projects like spaCy, his blog over at koaning.io, and his calm code educational project.

--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletter, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with Vincent on LinkedIn: https://www.linkedin.com/in/vincentwarmerdam/

Timestamps:

[00:00] Takeaways

[04:10] Favorite purchases this pandemic

[05:05] What drives Vincent to understand how ML can fail?

[08:33] How and why to make systems simpler?

[11:37] Techniques shared by Vincent in his talks

[15:51] ML as a UI problem

[17:02] Figuring out rules in your data

[20:01] Detecting bad labels

[23:53] Labeling isn't necessarily easy

[25:48] Fraud use case