Specialization in Data Science

Expecting data scientists to have expertise in multiple areas such as data pipeline, CI/CD, Kubernetes, monitoring, and tools like Datadog is unrealistic. The concept of T-shape specialization, where one understands the whole data lifecycle but focuses deeply on one area, is more practical. Data scientists are naturally inclined to learn new things, which leads them to acquire diverse skills once in the field.

Play episode from 38:30

chevron_right

Transcript

chevron_right

Transcript

Episode notes

Catherine Nelson is a freelance data scientist and writer. She is currently working on the forthcoming O’Reilly book "Software Engineering for Data Scientists”.

Why All Data Scientists Should Learn Software Engineering Principles // MLOps podcast #245 with Catherine Nelson, a freelance Data Scientist.

A big thank you to LatticeFlow AI for sponsoring this episode! LatticeFlow AI - https://latticeflow.ai/

// Abstract

Data scientists have a reputation for writing bad code. This quote from Reddit sums up how many people feel: “It's honestly unbelievable and frustrating how many Data Scientists suck at writing good code.” But as data science projects grow, and because the job now often includes deploying ML models, it's increasingly important for DSs to learn fundamental SWE principles such as keeping your code modular, making sure your code is readable by other people, and so on. The exploratory nature of DS projects means that you can't be sure where you will end up at the start of a project, but there's still a lot you can do to standardize the code you write.

// Bio

Catherine Nelson is the author of "Software Engineering for Data Scientists", a guide for data scientists who want to level up their coding skills, published by O'Reilly in May 2024. She is currently consulting for GenAI startups and providing mentorship and career coaching to data scientists. Previously, she was a Principal Data Scientist at SAP Concur. She has extensive experience deploying NLP models to production and evaluating ML systems, and she is also a co-author of the book "Building Machine Learning Pipelines", published by O'Reilly in 2020. In her previous career as a geophysicist, she studied ancient volcanoes and explored for oil in Greenland. Catherine has a PhD in geophysics from Durham University and a Master's of Earth Sciences from Oxford University.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksSoftware Engineering for Data Scientists book by Catherine Nelson:https://learning.oreilly.com/library/view/software-engineering-for/9781098136192/https://www.amazon.com/Software-Engineering-Data-Scientists-Notebooks/dp/1098136209

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Catherine on LinkedIn: https://www.linkedin.com/in/catherinenelson1/

Timestamps:

[00:00] Catherine's preferred coffee