AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
5min chapter
[22] Graham Neubig - Unsupervised Learning of Lexical Information
The Thesis Review
The Probabilistic Alignment Grammar
The model that I devised here was based on a PhD thesis by Karl DeMarkin at MIT in 1996 which was far before I started researching. The idea is that you do binary merges of characters based largely on frequency until you find words and if you're familiar with byte pairing coding BPE it's basically the same idea. And my contribution on top of that was largely to apply it to the multilingual setting where instead of doing binaryMerges of characters in one language you jointly do binary merging of characters in two languages. It also incorporated the kind of non-parametric Bayesian statistics that I talked about before. There's an ON to the six sampling algorithm included in learning this
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode