Alex Milowski is a researcher, developer, entrepreneur, mathematician, and computer scientist.
Evolving Workflow Orchestration // MLOps Podcast #291 with Alex Milowski, Entrepreneur and Computer Scientist.
// Abstract
There seems to be a shift from workflow languages to code, mostly annotation Python - happening and getting us. It is a symptom of how complex workflow orchestration has gotten. Is it a dominant trend or will we cycle back to “DAG specifications”? At Stitchfix, we had our own DSL that “compiled” into airflow DAGs, and at MicroByre, we used an external workflow language. Both had a batch task executor on K8S, but at MicroByre, we had human and robot in the loop workflows.
// Bio
Dr. Milowski is a serial entrepreneur and computer scientist with experience in a variety of data and machine learning technologies. He holds a PhD in Informatics (Computer Science) from the University of Edinburgh, where he researched large-scale computation over scientific data. Over the years, he's spent many years working on various aspects of workflow orchestration in industry, standardization, and research.
// Related Links
Website: https://www.milowski.com/
--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
// MLOps Swag/Merch
https://shop.mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Alex on LinkedIn: https://www.linkedin.com/in/alexmilowski/
Timestamps:
[00:00] Alex's preferred coffee
[00:20] Takeaways
[01:27] Workflow Systems Overview
[05:54] Everything as a DAG
[09:25] Workflows, Layering, and Ownership
[12:46] Workflow Awareness for Success
[17:31] Process Engineering Challenges
[21:20] ML Workflow Trends
[27:57] Data Engineering and ML Tools
[33:18] Infrastructure as Code Pros and Cons
[36:20] DSLs in Organizations
[43:04] Agents vs DAGs
[47:48] Agent Workflow Selection
[50:31] RPA Business Automation Tools
[55:00] LLM Integration in BPA
[1:01:14] Diversity in Decision Workflows
[1:03:09] Shadow AI Governance Risks
[1:08:34] Centralized ML Workflow Standardization
[1:12:50] Wrap up
