The Quest for Explainability in AI

I think this is a super interesting question and there's been a big tension kind of in the AI community. How do we bump up the accuracy as much as possible? And then how do we actually understand how all these algorithms are working? A lot of people point to, oh well, people can explain their thinking. But I think it's really only true to first order and that no human could actually tell you what they're doing.