AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Fine-Tuning Models and Hyperparameters
The speakers discuss the process of fine-tuning models and offer advice on hyperparameter selection. They recommend training for more tokens and epochs if overfitting is not an issue and mention Axolotl as a useful library. Various techniques such as instruction tuning, model merging, and reward model stuff are also discussed.