Balancing Data and Engineering in Training LLMs

Understanding the trade-off between requiring more data or more engineering efforts for training Large Language Models (LLMs) is crucial. More data is needed for abstract tasks, while more engineering is essential for tasks needing control. LLMs lack comprehension of 'why' behind actions, as they focus solely on syntax and lack semantic and pragmatic understanding. Incorporating pragmatics is crucial to enhance LLMs' comprehension. The combination of LLMs and knowledge graphs has shown promising results in reducing hallucinations and improving accuracy in predictions.