I've Been Thinking About Language Models a Lot.

i've been thinking about language models a fair bit, but the learning of things from large, unlabelled corpora that's interesting to me. It doesn't seem like data curation is ever going to get u thad scales. One idea i was playing with a year or two ago was around making a knowledge base that grew itself instead of us putting in all the knowledge ourselves. Is there a way that you can make a system that's boots trapped, that almost let it go and do google searches every time it's confused? Oh, i'm not sure. Let me go look that up.