Exploring Model Compression Techniques in Neural Machine Translation

Today we’re joined by Arul Menezes, a Distinguished Engineer at Microsoft. 

Arul, a 30 year veteran of Microsoft, manages the machine translation research and products in the Azure Cognitive Services group. In our conversation, we explore the historical evolution of machine translation like breakthroughs in seq2seq and the emergence of transformer models. 

We also discuss how they’re using multilingual transfer learning and combining what they’ve learned in translation with pre-trained language models like BERT. Finally, we explore what they’re doing to experience domain-specific improvements in their models, and what excites Arul about the translation architecture going forward. 

The complete show notes for this series can be found at twimlai.com/go/458.