Jakob Uszkoreit

Jakob Uszkoreit, CEO and Co-founder of Inceptive, is a deep learning pioneer who previously co-authored the game-changing paper 'Attention is All You Need' at Google. In this conversation, he delves into the future of biological software and its pivotal role in optimizing mRNA therapies and vaccine production. He highlights the innovative intersection of deep learning and biology, discussing challenges in drug discovery and the promise of simpler model architectures driven by hardware advancements. Collaborations at Inceptive showcase the exciting frontier of biotechnological research.

Ashish Vaswani and Jakob Uszkoreit, co-authors of the "Attention Is All You Need" paper, discuss the motivation behind replacing RNNs and CNNs with a self-attention mechanism in the Transformer model. They delve into topics such as the positional encoding mechanism, multi-headed attention, replacing encoders in other models, and what self-attention actually learns. They highlight how lower layers learn n-grams and higher layers learn coreference, showcasing the power of the self-attention mechanism.