We recently stumbled across the first ever Hacker News thread discussing Attention Is All You Need, the landmark paper that describes the transformer architecture. The work had come out in the summer of 2017, but it took a while for the wider AI community to begin to grasp its significance and build on it.
Most commenters at the time expressed interest or admiration. Some didn’t understand it. Others thought that the paper would probably be out of date in a few months. And one person asked what it could do for capsule networks.
But whatever happened to capsule networks?