#203 - Building Effective and Thriving Machine Learning Teams - David Tan & Dave Colls

22 snips

Jan 27, 2025

David Tan, a lead ML engineer passionate about effective engineering, and Dave Colls, a digital transformation expert, delve into building successful machine learning teams. They discuss the critical differences between ML and traditional software engineering. Learn why many ML projects struggle to deliver value and discover strategies for overcoming these challenges. The duo highlights the importance of team composition and product thinking, sharing essential best practices like automated testing and continuous delivery to ensure robust ML solutions.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

ANECDOTE

Career Change

David Tan's career began in government before a bout of hand, foot, and mouth disease led him to explore data analytics.
After a coding bootcamp and joining ThoughtWorks, he transitioned into ML engineering.

ANECDOTE

Career Shifts

Dave Colls's career path shifted from technical data roles to leadership and organizational transformation.
A key turning point was joining ThoughtWorks as a project manager after initially interviewing as a developer.

ANECDOTE

Book's Origin

The book's origin lies in a blog post about refactoring data science codebases, which gained significant traction.
This highlighted the desire within the data science and ML community for more robust engineering practices.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Is your ML project stuck in POC hell and failing to deliver real value?

In this episode, I sit down with David Tan and Dave Colls, co-authors of the book “Effective Machine Learning Teams” to discuss the complexities of building and deploying ML models, building and managing effective ML teams, and ensuring successful ML projects.

Key topics discussed:
- Learn the key differences between ML engineering and traditional software engineering.
- Why so many ML projects fail to reach production or deliver real business value and how to overcome common challenges.
- Discover the ideal ML team composition and how to apply concepts like team topologies for effective collaboration.
- The importance of product thinking in ML projects and best practices for designing robust, valuable ML products.
- Essential engineering best practices for successful ML projects, including automated testing, continuous delivery, and MLOps practices.

Listen out for:
(02:35) Career Turning Points
(04:54) Writing "Effective Machine Learning Teams"
(08:29) ML Engineering vs Other Types of Engineering
(12:24) ML and LLM
(14:59) Why Many ML Projects Fail
(19:53) ML Success Modes
(23:32) Ideal ML Engineering Team Composition
(31:39) Building the Right ML Product
(39:23) ML Engineering Best Practices
(49:14) MLOps
(52:44) Make Good Easy
(53:56) 3 Tech Lead Wisdom

Tune in to learn how to build and deploy ML products that truly make a difference!

David Tan’s Bio

David Tan is a lead ML engineer with more than six years of experience in practicing Lean engineering in the field of data and AI across various sectors such as real estate, government services and retail. David is passionate about engineering effectiveness and knowledge sharing, and has also spoken at several conferences on how teams can adopt Lean and continuous delivery practices to effectively and responsibly deliver AI-powered products across diverse industries.

Follow David Tan:

- LinkedIn – linkedin.com/in/davified

- X – @davified

- GitHub – https://github.com/davified

Dave Colls’ Bio

Dave loves building new things and helping others build too – products and services, people and teams – using technology, Data & AI.

His work building the Thoughtworks Data & AI practice shows how these elements combine: creating a new business, consulting to leaders and making a home for data people to grow. Currently, Dave is working at Nextdata, pushing the frontiers of how organisations work with data.

David thrives on navigating complex challenges and evolving environments. His extensive experience in large-scale digital transformations and deep tech R&D has honed his ability to align efforts and drive impactful results.

A dedicated mentor, David is passionate about guiding data professionals in their career growth. He actively shares his expertise as a speaker, blogger, and author of the book “Effective Machine Learning Teams.”

Follow Dave Colls:
- LinkedIn – linkedin.com/in/davidcolls

Links:

- 🎧 #198 - Better Software Faster: Measure & Improve Developer Productivity with DX Core 4 - Laura Tacho – https://techleadjournal.dev/episodes/198/

- 📚 Effective Machine Learning Teams – https://www.oreilly.com/library/view/effective-machine-learning/9781098144623/

- Free preview and overview – https://www.thoughtworks.com/content/dam/thoughtworks/documents/e-book/EffectiveMLTeams_preface.pdf

_____

Our Sponsors

Enjoy an exceptional developer experience with JetBrains. Whatever programming language and technology you use, JetBrains IDEs provide the tools you need to go beyond simple code editing and excel as a developer.
Check out FREE coding software options and special offers on jetbrains.com/store/#discounts.
Make it happen. With code.

Manning Publications is a premier publisher of technical books on computer and software development topics for both experienced developers and new learners alike. Manning prides itself on being independently owned and operated, and for paving the way for innovative initiatives, such as early access book content and protection-free PDF formats that are now industry standard.
Get a 40% discount for Tech Lead Journal listeners by using the code techlead24 for all products in all formats.

Like this episode?
Show notes & transcript: techleadjournal.dev/episodes/203.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Buy me a coffee or become a patron.