The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Robust Visual Reasoning with Adriana Kovashka - #463
Mar 11, 2021
Adriana Kovashka, an Assistant Professor at the University of Pittsburgh, dives into her research on visual commonsense and robust visual reasoning. She discusses the interplay between media studies and machine learning, using examples like public service announcements to highlight the complexity of interpretation. Adriana elaborates on the pitfalls in visual question answering datasets and the innovations in weakly supervised object detection. She also shares insights into practical AI applications, emphasizing the need for common sense in assistive technologies.
AI Snips
Chapters
Transcript
Episode notes
Complex Ad Example
- Adriana Kovashka recalls a complex anti-secondhand smoke ad.
- It depicted a child seemingly smoking, but with mismatched arm sizes, revealing an unseen smoker.
Visual Reasoning Definition
- Visual reasoning involves seeing relationships between concepts, relying on unencoded knowledge, and selective attention.
- Defining "reasoning" is difficult, as it goes beyond data and involves priors and inductive biases.
VCR Dataset Example
- Kovashka uses the Visual Common Sense Reasoning dataset, which includes movie frames and questions.
- An example question asks why a person points at another, requiring reasoning about restaurant norms.