Local LLMs: Speed, Context, and Hardware Constraints

AI in the cloud dominates, but what can you run locally? Carl and Richard speak with Joe Finney about his work in setting up local machine learning models. Joe discusses the non-LLM aspects of machine learning, including the vast array of models available at sites like Hugging Face. These models can help with image recognition, OCR, classifiers, and much more. Local LLMs are also a possibility, but the hardware requirements become more significant - a balance must be found between cost, security, and productivity!