Research focuses on neural language generation from structured knowledge bases.
Interest in multilingual and low-resource NLP and controlled generation.
Exploration of distributional control and energy-based models in language models.
Deep dives
Research Focus on Controlling Language Models
Heidi Elsahar's research focuses on neural language generation under constrained and controlled conditions, particularly exploring interactions between natural language and structured knowledge bases. His PhD work delved into data-to-text generation and relation extraction, leading to a keen interest in multilingual and low-resource natural language processing (NLP) as well as controlled generation. The discussion delves into controlling language models, particularly in prompting, distributional control, and energy-based models.
Career Journey and Academic Development
Heidi Elsahar reflects on his unconventional path into research, tracing it back to a Cairo upbringing where academia was not a prevalent environment. His research journey started with a question-answering system during his bachelor's, sparking his interest in NLP and knowledge graphs. Transitioning to an internship at Microsoft Research Labs and collaborating on recommendation systems further fueled his passion for NLP and knowledge bases.
Research Environment and Opportunities in Egypt
Heidi Elsahar discusses the evolving research landscape in Egypt, highlighting increased access to information due to the internet and the emergence of research startups and machine learning applications across various domains. He notes the challenges of brain drain but also highlights the growing research communities that facilitate collaboration and remote work, enabling individuals with similar interests to connect and engage in research.
Challenges and Innovations in Multilingual NLP Research
Heidi Elsahar's work addresses the challenges and opportunities in multilingual natural language processing (NLP), emphasizing the importance of tackling underrepresented languages and low-resource settings. He envisions these research endeavors paving the way for more efficient approaches in few-shot learning and advocating for the exploration of new methods to enhance language model efficiency and generalization in diverse linguistic contexts.
Distributional Control in Text Generation
Heidi Elsahar's research explores distributional control in text generation, aiming to match predefined constraints while minimizing deviations from the original language model. By fine-tuning models with minimal deviation and dynamically defining constraints per user, his work delves into transformative ideas that balance control, efficiency, and adaptability in language model training and updates.
Hady Elsahar is a Research Scientist at Naver Labs Europe. His research focuses on Neural Language Generation under constrained and controlled conditions.
Hady's PhD was on interactions between Natural Language and Structured Knowledge bases for Data2Text Generation and Relation Extraction & Discovery, which he completed in 2019 at the Université de Lyon.
We talk about his phd work and how it led to interests in multilingual and low-resource in NLP, as well as controlled generation. We dive deeper in controlling language models, including his interesting work on distributional control and energy-based models.
- Episode notes: www.wellecks.com/thesisreview/episode44.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about the show at www.wellecks.com/thesisreview
- Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
