AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is There a Distributed Distribution Over Words?
'It's kind of ridiculous that we're doing a distribution over every single possible word for language and it works. It feels like that's a really dumb way to do it.' 'We don't know how to represent properly distributions over combinatorial sequences of symbols, because the number grows exponentially with the length of the symbols,' he says. "There is a huge independence hypothesis in this process, which is not actually true"