Jonathan Frankle, Chief Scientist at MosaicML, and Abhinav Venigalla, Research Scientist at MosaicML, dive into the groundbreaking MPT-7B model. They discuss its unprecedented 84,000-token context length and how it was trained on 1 trillion tokens, achieving industry-leading performance for a fraction of the cost. The duo also navigates the complexities of AI model training, ethical considerations in creative generation, and the balance between open research and business interests, providing fascinating insights into the future of AI technologies.
