MLOps Coffee Sessions #158 with Nils Reimer, MLOps Build or Buy, Large Language Model at Scale co-hosted by Abi Aryan. // Abstract Large Language Models with billions of parameters have the possibility to change how we work with textual data. However, running them on scale at potentially hundred millions of texts a day is a massive challenge. Nils talks about finding the right model size for respective tasks, model distillation, and promising new ways on transferring knowledge from large to smaller models. // Bio Nils Reimers is highly recognized throughout the AI community for creating and maintaining the now-famous Sentence Transformers library (www.SBERT.net) used to develop, train, and use state-of-the-art LLMs. The project has 900+ stars on GitHub and 30M+ installations. Nils is currently the Director of Machine Learning at Cohere where he leads the team that develops and trains Large Language Models (LLM) with billions of parameters. Prior to Cohere, Nils created and led the science team for Neural Search at HuggingFace. Nils holds a Ph.D. in Computer Science from UKP in Darmstadt. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links (www.SBERT.net) https://www.nils-reimers.de/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Nils on LinkedIn: https://www.linkedin.com/in/reimersnils/ Timestamps: [00:00] Nils' preferred coffee [00:45] Nils' background [01:30] Takeaways [06:47] Subscribe to our Newsletters and IRL Meetups, and leave your reviews! [07:32] Nils' history [10:39] From IT Security to Machine Learning [13:22] Tangibility of IT and Security [14:46] NLP transition [15:55] Bad augmentation to new capabilities of LLMs [16:59] Nils' concern during his PH.D. [19:55] Making Money from Machine Learning [22:06] Catastrophic forgetting [26:34] Updating solutions [28:42] Neural search space and building adaptive models [31:23] Filtering models [32:32] Latency issues [36:53] Models running in parallel [37:54] Generative models problems [38:43] Nils' role at Cohere [41:41] To build or not to build API [43:00] Search models [45:38] Large use cases [46:43] Open source discussion within Cohere [50:48] Competitive Edge [55:27] Future world of API [58:14] LLMs in Production Conference Part 2 announcement! [1:00:17] Hopeful direction of Cohere's future [1:02:33] Data silos [1:04:34] Where to update the database and code [1:05:24] Nils' focus [1:08:49] Make money or save money [1:10:30] Cohere's upcoming project [1:12:37] Time spent red teaming the models [1:14:05] Wrap up