Sachin Abeywardana, Deep Learning Engineer at Canva AI, shares insights on incorporating AI and ML capabilities, challenges of building CLIP and other language models, balancing work and family life, limitations of transformers in NLP, and trade-offs in deploying features.
Read more
AI Summary
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Graph ML offers potential for incorporating diverse data types and solving complex problems.
Transformers are effective in NLP tasks and continuously improve, especially in text-related problems.
Consider the trade-off between model complexity and speed and use existing tools for faster results.
Align ML projects with product goals and engage with product teams for better focus.
Regularly assess project progress and be open to alternate approaches.
Balance parental responsibilities with professional growth and time management.
Deep dives
Use of Graph ML in Machine Learning
Graph ML offers the potential to incorporate different data types, such as images, text, and more, into machine learning models. It allows for inference on graph structures, enabling the inclusion of diverse data sources. While graph ML is still an emerging area, it holds promise for solving complex problems that cannot be easily tackled with traditional models.
The Benefits of Using Transformers
Transformers have proven to be effective in natural language processing tasks and deliver impressive results. While there are limitations to their computational complexity, workarounds such as flash attention have been developed. Transformers are continually being improved and provide valuable solutions for various tasks, especially in text-related problems.
Balancing Complexity and Speed in Machine Learning Models
While there is room for complex and sophisticated models, it is crucial to consider the trade-off between complexity and speed. Investing three months in a model should be the upper limit, as it allows for rapid development and progress. Using existing tools like Hugging Face can help simplify the process and deliver faster results.
Aligning Machine Learning Projects with Product Goals
A common mistake is not aligning machine learning projects with product goals. It is important to understand the product and engage with the product team to ensure the right focus. ML engineers should prioritize understanding product metrics and deliver solutions that meet customer needs and drive business value.
Recognizing the Need to Let Go of Projects
ML engineers should be mindful of when to let go of projects and move on. It is essential to assess the progress and impact of a project regularly. Strive for faster iterations and be willing to consider alternate approaches if a project is not showing promising results.
Parenting and Pursuing Machine Learning
Balancing parental responsibilities with keeping up with the fast-paced world of machine learning can be challenging. Finding ways to manage time effectively and prioritize tasks is crucial for success. It is also valuable to explore how other ML engineer parents handle their responsibilities and maintain their professional growth.
Value of Incorporating Machine Learning Solutions
Machine learning can provide valuable solutions across different areas of a company or product. Investing in ML talent within the organization allows for quicker turnaround times and in-house expertise. There is also significance in identifying the right problems to solve with machine learning and delivering products that bring tangible value to customers.
Importance of Understanding Product Metrics
To ensure success, ML engineers should focus on product metrics rather than just technical metrics. This includes understanding and aligning with the key performance indicators that drive the business forward. Building models that optimize these metrics can lead to more effective and impactful solutions.
Learning from Mistakes and Iterating
Recognizing and learning from mistakes is vital for growth. ML engineers should strive for continuous improvement, iterate on their models, and seek feedback from users and stakeholders. Being open to new approaches and methodologies can lead to better outcomes and enhanced problem-solving abilities.
Effectively Balancing Machine Learning and Product Knowledge
Finding the right balance between deep knowledge of machine learning concepts and product understanding is crucial. ML engineers should actively engage with product teams, acquire product knowledge, and cultivate a mindset that prioritizes solving real-world problems. This balance can drive innovation and create valuable products.
Consideration of Graph ML in Machine Learning Solutions
Graph ML holds immense potential for incorporating diverse data types and solving complex problems. The ability to analyze relationships between nodes and leverage the power of graphs can lead to more advanced and comprehensive machine learning solutions. It is an area worth exploring for ML engineers.
Value of Speed to Market in Machine Learning
Speed to market is a critical factor in machine learning projects. Delivering results quickly allows for early testing, feedback, and validation. Utilizing tools like Hugging Face and pre-trained models can expedite the development process and lead to faster iterations.
Key Role of Product Metrics in Machine Learning
Understanding and aligning with product metrics is essential for successful machine learning projects. By focusing on metrics that demonstrate value to the business and customers, ML engineers can develop models that directly impact the desired outcomes. This alignment ensures the work is aligned with company goals and contributes to overall success.
MLOps podcast #180 with Sachin Abeywardana Deep Learning Engineer at Canva AI, Adventures in Building CLIP and Other (Largeish) Language Models
sponsored by Prem AI.
// Abstract
Sachin takes us on an adventure, sharing insights on the pitfalls of not understanding the broader product and the importance of incorporating AI and machine learning capabilities. From the use of AI models to grammar correction and code generation to the fascinating Clip model and the challenges of balancing work and family life, this episode promises to be both informative and thought-provoking.
// Bio
Sachin is the father of two beautiful children. He completed his PhD in Bayesian Machine Learning at University of Sydney in 2015. In 2016 he discovered Deep Learning and hasn't looked back. He currently works as a Senior Machine Learning Engineer at Canva and is mainly focusing on NLP problems.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Sachin Blogs: https://sachinruk.github.io/blog.html
https://sachinruk.github.io/blog/
Graph ML link: http://web.stanford.edu/class/cs224w/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Sachin on LinkedIn: https://www.linkedin.com/in/sachinabeywardana/
Timestamps:
[00:00] Sachin's preferred beverage
[00:26] Takeaways
[02:30] Chat GPT user
[05:58] Understanding on reliable Agents
[08:10] Sachin's background
[12:45] Staying at Deep Learning
[16:17] Recommendation or Lead Scoring
[17:36] Vector database
[19:00] Sachin's blogs
[23:26] The cap people
[26:10] Pursuing business case
[27:33] Canva
[31:16] Incorporating AI and Machine Learning
[32:17] Sponsor Ad
[38:22] Eliminating unnecessary steps
[39:00] Interacting with the product team
[43:04] Criticisms on the current architecture limitations
[45:58] Insufficient exploration of Transformers
[47:42] Explaining GraphML
[52:35] Fine-tuning ChatGPT2
[57:54] Leading ML Engineers and teams
[59:40] Being practical with Math
[1:05:52] Wrap up
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
