AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
June marked the release of Mosaic's MPT, a foundational model with a 64,000 context window and a commercial license. This was a significant boost in capabilities and steerability for GPT models. OpenAI's Voyager paper also showcased the blueprint for building incredible agents with memory and tool augmentation. Additionally, vector databases and embedding models saw increased investment and exploration, with June being a significant month for developments in context windows and memory for AI.
May saw the release of GPT 3.5 Turbo, which came with the functions API, allowing developers to interact with the model using specific JSON responses and providing the concept of tool use. This was significant as it opened new possibilities for developers to build AI-powered interfaces. Llama 2, an open-source model from LMC, was also released, showcasing significant improvements in performance and context window size, with 100,000 tokens now possible.
April saw the release of AutoGPT, an AI model capable of writing code and improving itself. This opened new doors for autonomous AI agents and showcased the power of self-improvement in AI models. Open Source agents also gained significant attention, with the release of Llama CPP and the rise of multi-modality as a concept. With the ability to combine text and images in AI models, significant progress was made towards building more versatile and capable agents.
March was marked by the release of GPT 4, which brought significant improvements in capabilities, including a larger context window and the ability to generate code for tool use. Additionally, OpenAI introduced the ChatGPT API, allowing developers to build their own ChatGPT-powered applications. This marked a shift towards more interactive and conversational AI experiences and set the stage for further advancements in natural language understanding.
February saw the release of Llama 1, an open-source language model that approached the capabilities of closed-source models. This sparked an explosion in open-source language models and served as the starting point for the development of Llama 2 and other subsequent models. The month also saw the emergence of META AI's LLM, which further fueled the open-source movement and showcased the potential of large language models.
In January, Microsoft announced an additional $10 billion investment in OpenAI, showing the increasing interest and support for AI research and development. The podcast episode also touched on the advancements and updates in AI over the previous year, setting the stage for the further progress and innovation to come.
Over the past year, there have been significant advancements in AI capabilities and open source models. One of the key highlights is the release of GPT-4 Turbo, which is a faster and cheaper model with a larger context window. OpenAI also introduced the combined mode, which allows for multimodality input and output and the ability to generate text based on uploaded images. In addition, OpenAI's Dev Day brought several updates, including the Assistance API with stateful memory and retrieval capabilities. The marketplace for shareable GPTs was announced, providing developers with the ability to create and share their own models. Additionally, there were notable releases of open source models like Mistral and Grok from XAI, showcasing the potential of open source AI models. Furthermore, advancements in multimodality were demonstrated with DALI 3, enabling chat-to-image generation. These developments highlight the rapid progress in AI capabilities and the increasing accessibility of advanced models through open source initiatives.
Microsoft made a significant investment of $13 billion in OpenAI and announced the integration of AI-powered tools into its suite of office applications. Co-pilot Everywhere, powered by Chechai PT, was introduced, allowing for AI assistance in tasks like coding, email, and collaboration. This move showcases Microsoft's commitment to leveraging AI technologies and integrating them across their platforms. The partnership between Microsoft and OpenAI positions them as major players in the AI landscape, with the potential to shape the future of AI-powered productivity tools for a wider user base.
🎶 Happy birthday to you, happy birthday to you, happy birthday chat GPT-eeeeeeee, happy birthday to you.
Hey everyone, welcome to this special edition of ThursdAI where you're probably gonna have two emails and two podcast episodes today and you can choose which one you want to but we actually recorded both of them live it just they went a little long.
ThursdAI - The only podcast that brings you yearly recaps since chatGPT was released (😂)
This one is the more celebratory one, today is one year from the release of chat GPT and we (and by we I mean I, Alex) decided to celebrate it by recapping not just the last week in AI but the last year (full timeline posted at the bottom of this newsletter)
Going month by month with a swoosh sound in the editing and covering the most important thing that happened in LLM and open source LLMs since chatGPT was released and imagination unlocked the capability for everyone!
We also covered Meta stepping in with Lama and then everything that happened since then in the multi modality and vector databases and agents and everything everything everything, it was a one hell of an hour and a half, we had almost 1K audience members! and so I recommend you listen to this one first and then the week updates later because there were some incredible releases this week as well! (as there are every week)
I think it's important to do like a Spotify wrapped type thing for AI, for something like a one year for chat GPT and I think we'll be doing this every year so hopefully in the year we'll see you here on November 30th covering the next year in AI.
And hopefully the next year in AI system will actually help me summarize all this because it's a lot of work but with that I will just leave you with the timeline and no notes and you should listen to everything because we talked about everything live!
I hope you enjoy this special birthday celebration! (OpenAI sure did, check out this incredibly cute little celebration video they just posted)
Here’s the full timeline with everything important that happened month by month that we’ve covered:
* December 2022 - ChatGPT becomes the fastest growing product in history
* GPT3.5 with 4K context window, instruction finetuning and conversational RLHF
* January
* Microsoft invests additional $10B into OpenAI (Jan 23, Blog)
* February
* LLaMa 1 - Biggest Open Source LLM (February 24 - Blog)
* No commercial license
* 30% MMLU
* No instruction fine-tuninig (RL;HF)
* ChatGPT unofficial APIs exist
* March (the month of LLM superpowers)
* ChatGPT API (March 1, announcement)
* Developers can now build chatGPT powered apps
* All clones so far were completion based and not conversation based
* LLama.cpp from ggerganov + Quantization (March 10, Blog)
* Stanford - Alpaca 7B - Finetune on self-instruct GPT3.5 dataset (March 13, Blog)
* GPT4 release + chatGPT upgrade (March 14 - GPT-4 demo)
* 67.0% HumanEval | 86.4% MMLU
* 8K (and 32K) context windows
* Anthropic announces Claude + Claude instant (March 14 - Blog)
* 56.0% HumanEval
* Folks previously form OAI leave to open Anthropic as research, then pivot from research into commercial
* LMSYS Vicuna 13B - Finetuned based on shareg.pt exports (March 30, Blog)
* April (Embedings & Agents)
* AutoGPT becomes the fastest github starred project + writes it's own code (April 1, Blog)
* Agents start to pop up like mushrooms after the rain
* LLaVa - Multimodality open source begins (April 18, Blog)
* CLIP + Vicuna smushed together to get LLMs eyes
* Bard improvements
* May (Context windows)
* Mosaic MPT-7B with 64K context, 1T parameters, commercial license (May 5, Blog)
* Anthropic updates Claude with 100K context window (May 11, Blog)
* LLongBoi summer begins (Context windows are being stretched)
* Nvidia shows Voyager agents that play Minecraft + Memory stored in Vector DB (May 27, Blog)
* June
* GPT-3.5-turbo + functions API (June 6, Blog)
* GPT3.5 and 4 got a boost in capabilities and steer-ability
* Price reduction on models + 75% reduction on ada embeddings model
* LLaMa context window extended to 8K with RoPE scaling
* AI Engineers self determination essay by swyx
* July
* Code Interpreter GA - ChatGPT can code (July 11, Blog)
* Anthropic Claude 2 - (July 11 - Blog)
* 200K context window
* 71% HumanEval
* LLaMa 2 (July 18 - Blog)
* Base & Chat models (RLHF)
* Commercial license
* 29.9% Human Eval | 68.9% MMLU
* August
* Meta releases Code-LlaMa, code finetune models
* September
* DALL-E 3 - Adds multi-modality on output and chat to image gen (Sep 20, Blog)
* Mistral 7B top performing open source LLM via torrent link (Sep 27, Blog)
* GPT4-V (vision & voice) - Adds multimodality on input (Sep 27, Blog)
* October
* OpenHermes - Mistral 7B finetune that tops the charts from Teknium / Nous Research (Oct 16, Announcement)
* Inflection PI gets connected to the web + supportPi mode (Oct 16, Blog)
* Adept releases multimodal FuYu 8B (Oct 19, blog)
* November
* Grok from Xai - with realtime access to all of X content
* OpenAI dev day
* Combined mode for MMIO (multi modal on input and output)
* GPT-4 Turbo with 128K context, 3x cheaper than GPT-4
* Assistants API with retrieval capabilities
* Share-able GPTs - custom versions of GPT with retrieval, DALL-E, Code Interpreter and vision
* Chatbots with real business use-cases, for example WandBot (that we just launched today! Blog)
* Has vector storage memory
* Available via Discord/Slack
* And custom GPT!
* Microsoft has copilot everywhere in office
Aaaand now we’re here!
What an incredible year, can’t imagine what the next year holds for all of us, but 1 thing is for sure, ThursdAI will be here to keep you all up to date!
P.S - If you scrolled all the way to here, DM me the 🎊 emoji so I know you celebrated with us! It really helps me to know that there is at least a few folks out of the thousands that get this newsletter that scrolls all the way through!
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode