LLMs like ChatGPT are not just fascinating, they're becoming increasing useful in our working lives. They've graduated from novelty to valuable tool. But building those tools is still in the hands of huge companies. Or is it?
In this week's episode of Developer Voices, we're learning how you can run LLMs on your own laptop, and how you can customize the system to make a tailored research assistant, a better documentation-searcher, and much more. All you need is a guide on which pieces you need, and how they fit together, and that's exactly what this week's guest—Tobi Fankhänel—is here to take us through.
A leaked memo from Google recently outlined how the Big Company Advantage has almost completely eroded, and how the next wave of LLM development is going to come from the open source community. So hackers rise up - the open source AI revolution begins now!
--
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Tobias on LinkedIn: https://www.linkedin.com/in/tobias-fankh%C3%A4nel-749712180/
Tobias’ blog: https://blog.exxample.eu
LangChain: https://python.langchain.com/docs/get_started/introduction.html
Embeddings: https://weaviate.io/blog/vector-embeddings-explained
Vector Databases: https://en.wikipedia.org/wiki/Vector_database
"We have no moat" – Google Employee on Open-source LLMs: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
“Attention is all you need” - https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
Timeline since Meta open-sourced their first-gen models: https://www.semianalysis.com/i/119223672/the-timeline
Run LLMs on CPU only or, since May, mix CPU and GPU usage: https://github.com/abetlen/llama-cpp-python
Samantha: https://erichartford.com/meet-samantha
Embedding model leaderboards: https://huggingface.co/spaces/mteb/leaderboard
Open-source LLMs: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
LLaMA: https://ai.facebook.com/blog/large-language-model-llama-meta-ai/
Blog post: Design-pattern ‘In-context learning’ https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/#section--2
Tobi's GitHub branch ‘In-context learning with LangChain’ https://github.com/aviav/turmbauten/blob/spaghetti-code/CHANGELOG.md
Prompt Syntax Cheat Sheet: https://github.com/oobabooga/text-generation-webui/tree/main/characters/instruction-following
Google Workspace Labs Sign-Up: https://workspace.google.com/labs-sign-up/
GMail Workspace Labs Demo Video, click ‘See it in action’: https://workspace.google.com/solutions/ai/#m10
Prediction trading on open-source LLMs vs GPT-4: https://manifold.markets/PeterWildeford/will-i-peter-wildeford-think-that-t-c95ff3c1b385
Developer Voices