Susan Shu Chang on Bridging Foundational Machine Learning and Generative AI
Jan 6, 2025
auto_awesome
Susan Shu Chang, Principal Data Scientist at Elastic, delves into the synergy between foundational machine learning and generative AI. She emphasizes the critical role of collaboration between machine learning practitioners and software engineers to ensure effective model deployment. Chang discusses the challenges of harnessing large language models for summarizing log data and explores practical tools for developing generative AI applications. Additionally, she shares strategies for navigating machine learning job interviews, highlighting the importance of a well-rounded skill set.
The significance of foundational machine learning techniques remains critical in real-world applications, despite the growing prominence of generative AI.
Effective deployment of machine learning models necessitates understanding user interface integration and the importance of relevant skill preparation for job candidates.
Deep dives
Foundational Insights in Machine Learning
The discussion highlights the significance of foundational machine learning techniques that remain highly relevant in the current landscape, despite the rising hype around generative AI. Personalized recommendations on platforms like YouTube, Spotify, and Netflix are examples of how traditional machine learning, such as recommender systems and reinforcement learning, fundamentally drives user experiences. The track emphasized the challenges of deploying machine learning models in production, emphasizing that having a robust algorithm is not enough if it cannot be integrated effectively to deliver value to users. This serves to remind software engineers about the importance of pragmatism in applying machine learning techniques to real-world applications.
Tools and Techniques for Machine Learning Practitioners
For those beginning their journey in machine learning, popular frameworks like PyTorch and TensorFlow are recommended for building foundational models. Additionally, connecting machine learning algorithms to user interfaces is crucial, as mere models lack functionality without the proper front-end integration. Streamlit, a user-friendly tool, allows engineers to quickly prototype web applications that showcase machine learning functionalities, facilitating better communication with stakeholders. This highlights the need for practitioners to not only develop algorithms but also understand how to deploy them in practical, user-centered applications.
Navigating Machine Learning Interviews
The conversation underscores the complexity of machine learning job interviews, where candidates must tailor their preparation to the specific responsibilities outlined in job descriptions. Candidates often mistakenly believe they need to master a wide range of topics, but focusing on the relevant lifecycle phases—such as data processing, model training, or MLOps—is crucial. This approach also extends to interviewers, who should define the skills and roles they require, ensuring a good fit for both the candidate and the organization. Ultimately, understanding the dynamics of what companies seek in candidates can significantly influence the interview outcomes in this evolving field.
Live from the QCon San Francisco Conference, we are talking with Susan Shu Chang, Principal Data Scientist at Elastic. Chang shares insights on bridging foundational machine learning with generative AI, emphasizing the importance of deploying ML models effectively, leveraging collaborative tools for prototyping, and aligning team roles with the ML life cycle to create scalable AI solutions.
Read a transcript of this interview: https://bit.ly/4gzllet
Subscribe to the Software Architects’ Newsletter for your monthly guide to the essential news and experience from industry peers on emerging patterns and technologies:
https://www.infoq.com/software-architects-newsletter
Upcoming Events:
QCon London (April 7-9, 2025)
Discover new ideas and insights from senior practitioners driving change and innovation in software development.
https://qconlondon.com/
InfoQ Dev Summit Boston (June 9-10, 2025)
Actionable insights on today’s critical dev priorities.
devsummit.infoq.com/conference/boston2025
InfoQ Dev Summit Munich (Save the date - October 2025)
QCon San Francisco 2025 (17-21, 2025)
Get practical inspiration and best practices on emerging software trends directly from senior software developers at early adopter companies.
https://qconsf.com/
InfoQ Dev Summit New York (Save the date - December 2025)
The InfoQ Podcasts:
Weekly inspiration to drive innovation and build great teams from senior software leaders. Listen to all our podcasts and read interview transcripts:
- The InfoQ Podcast https://www.infoq.com/podcasts/
- Engineering Culture Podcast by InfoQ https://www.infoq.com/podcasts/#engineering_culture
- Generally AI: https://www.infoq.com/generally-ai-podcast/
Follow InfoQ:
- Mastodon: https://techhub.social/@infoq
- Twitter: twitter.com/InfoQ
- LinkedIn: www.linkedin.com/company/infoq
- Facebook: bit.ly/2jmlyG8
- Instagram: @infoqdotcom
- Youtube: www.youtube.com/infoq
Write for InfoQ:Learn and share the changes and innovations in professional software development.
- Join a community of ex
perts.
- Increase your visibility.
- Grow your career.
https://www.infoq.com/write-for-infoq
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode