NLP, Data Set Creators and Data Set Engineers

Some people estimate that maybe 80 or 90% of all AI computation is done on inference, not on training. And there have been multiple approaches that probably, as you said, the distillation is one of the most studied approach. Another project that's, from what I understand, works very well in industry set up is quantization where you use lower precision of your folding point instead of using 32 big series.