Open sourcing a checkpoint in the multi-lingual space

The speakers discuss the idea of open sourcing a checkpoint and creating a business around it in the multi-lingual space. They highlight the lack of language support in voice and text models, and mention the role of tokenizers and verbosity in language models.

Play episode from 41:02

chevron_right

Transcript

chevron_right

Transcript

Episode notes

Aleksa Gordić is an ex-Google DeepMind / Microsoft ML engineer currently working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project and YugoGPT.

MLOps podcast #203 with Aleksa Gordić, Founder of OrtusAI, Pioneering AI Models for Regional Languages.

// Abstract

Dive deep into Aleksa's work with the YugoGPT, a language model serving Serbian, Croatian, Bosnian, and Montenegrin dialects - emphasizing the need for multilingual AI developments. Explore the unique language dynamics in the Balkans and Eastern Europe, the potential business opportunities around multilingual models, and the challenges in deploying large language models. Aleksa shares his experience with vision and image models, his collaborations with key tech players, and his use of advanced technologies. Hear about Aleksa Gordić's journey of being active and visible in the tech community and his insights into the world of machine learning and AI. Prepare to have your thinking challenged and horizons widened as we converse about the intriguing and complex world of MLOps.

// Bio

Working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project. Worked at DeepMind on the Flamingo project as a research engineer. Worked at Microsoft on the HoloLens 2 project & next-gen mixed reality glasses.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://gordicaleksa.com/https://github.com/gordicaleksa

- I build stuff :) https://discord.com/invite/peBrCpheKE

- active AI Discord server (~6000) I bring the best AI researchers in the world to give talks (James Betker, DALL-E 3 author, Tri Dao (Flash Attention), etc.)

https://gordicaleksa.medium.com/how-i-got-a-job-at-deepmind-as-a-research-engineer-without-a-machine-learning-degree-1a45f2a781de

- how I landed a job at DeepMind (and a couple more potentially interesting writings)Aleksa Gordić The AI Epiphany Youtube Channel: https://www.youtube.com/channel/UCj8shE7aIn4Yawwbo2FceCQ/videos

W&B AI Academy: http://wandb.me/mlops_com_llm_course

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Aleksa on LinkedIn: https://www.linkedin.com/in/aleksagordic/

Timestamps:

[00:00] Aleksa's preferred coffee

[00:17] Takeaways

[02:51] Humming the GPU's

[06:23] Built a Chrome extension for communicating with videos

[08:04] Rig Doubles Throughput Time