The Future of AI Inference

Note for Latent Space Community members: we have now soft-launched meetups in Singapore, as well as two new virtual paper club/meetups for AI in Action and LLM Paper Club. We’re also running Latent Space: Final Frontiers, our second annual demo day hackathon from last year.

Edit from March 2024: We did a followup on the Four Wars on the AI Breakdown.

For the first time, we are doing an audio version of monthly AI Engineering recap that we publish on Latent Space! This month it’s “The Four Wars of the AI Stack”; you can find the full recap with all the show notes here: https://latent.space/p/dec-2023

* [00:00:00] Intro

* [00:01:42] The Four Wars of the AI stack: Data quality, GPU rich vs poor, Multimodality, and Rag/Ops war

* [00:03:17] Selection process for the four wars and notable mentions

* [00:06:58] The end of low background tokens and the impact on data engineering

* [00:08:36] The Quality Data Wars (UGC, licensing, synthetic data, and more)

* [00:14:51] Synthetic Data

* [00:17:49] The GPU Rich/Poors War

* [00:18:21] Anyscale benchmark drama

* [00:22:00] The math behind Mixtral inference costs

* [00:28:48] Transformer alternatives and why they matter

* [00:34:40] The Multimodality Wars

* [00:38:10] Multiverse vs Metaverse

* [00:45:00] The RAG/Ops Wars

* [00:50:00] Will frameworks expand up, or will cloud providers expand down?

* [00:54:32] Syntax to Semantics

* [00:56:41] Outer Loop vs Inner Loop

* [00:59:54] Highlight of the month