🔥 ThursdAI Sep 14 - Phi 1.5, Open XTTS 🗣️, Baichuan2 13B, Stable Audio 🎶, Nougat OCR and a personal life update from Alex

This is a free preview of a paid episode. To hear more, visit sub.thursdai.news

Hey, welcome to yet another ThursdAI 🫡

This episode is special for several reasons, one of which, I shared a personal life update (got to listen to the episode to hear 😉) but also, this is the first time I took the mountainous challenge of fixing, editing and “video-fying” (is that a word?) our whole live recording! All 3 hours of it, were condensed, sliced, sound improved (x audio quality is really dogshit) and uploaded for your convenience. Please let me know what you think!

Premium folks get access to the full podcast in audiogram format, and a full transcription with timestamps and speakers, here’s a sneak preview of how that looks, why not subscribe? 😮

TL;DR of all topics covered

* Open Source LLM

* Microsoft Phi 1.5 - a tiny model that beats other 7B models (with a twist?) (Paper, Model)

* Baichuan 7B / 13B - a bilingual (cn/en) model with highly crafted approach to training (Paper, Github)

* Big Co LLMs + API updates

* Nothing major this week

* Voice & Audio

* Stable Audio 🎶 - A new music generation model from Stability AI. (Website)

* Coqui XTTS - an open source multilingual text to speech for training and generating a cloned voice (Github, HuggingFace)

* AI Art & Diffusion

* Würstchen v2 - A new super quick 1024 diffusion model (Announcement, Demo, Github)

* DiffBIR - Towards Blind Image Restoration with Generative Diffusion Prior (Annoucement, Demo, Github)

* Tools

* Nougat from Meta - open-source OCR model that accurately scans books with heavy math/scientific notations (Announcement, Github, Paper)

* GPT4All Vulkan from Nomic - Run LLMs on ANY consumer GPUs, not just NVIDIA (Announcement)

* Nisten’s AI ISO disk - Announcement

And here are timestamps and chapter/discussion topics for your convenience:

[00:05:56] Phi 1.5 - 1.3B parameter model that closely matches Falcon & LLaMa 7B

[00:09:08] Potential Data Contamination with Phi 1.5

[00:10:11] Data Contamination unconfirmed

[00:12:59] Tiny models are all the rage lately

[00:16:23] Synthetic Dataset for Phi

[00:18:37] Are we going to run out of training data?

[00:20:31] Breaking News - Nougat - OCR from Meta

[00:23:12] Nisten - AI ISO disk

[00:29:08] Baichuan 7B - an immaculate Chinese model

[00:36:16] Unique Loss Terms

[00:38:37] Baichuan ByLingual and MultiLingual dataset

[00:39:30] Finetunes of Baichuan

[00:42:28] Philosophical questions in the dataset

[00:45:21] Let's think step by step

[00:48:17] Is breath related text in the original dataset?

[00:50:27] Counterintuitive prompting for models with no breath

[00:55:36] Idea spaces

[00:59:59] Alex - Life update about ThursdAI

[01:04:30] Stable Audio from Stability AI

[01:17:23] GPT4ALL Vulkan

[01:19:37] Coqui.ai releases XTTS - an open source TTS - interview With Josh Meyer

[01:30:40] Summary

Here’s a full video of the pod, and a full transcription, and as always, 🧡 thank you for bring a paid subscriber, this really gives me the energy to keep going, get better guests, release dope podcast content, and have 3 hours spaces and then spend 7 hours editing 🔥