Navigating Dataset Complexities in Visual Question Answering

Today we’re joined by Adriana Kovashka, an Assistant Professor at the University of Pittsburgh.

In our conversation with Adriana, we explore her visual commonsense research, and how it intersects with her background in media studies. We discuss the idea of shortcuts, or faults in visual question answering data sets that appear in many SOTA results, as well as the concept of masking, a technique developed to assist in context prediction. Adriana then describes how these techniques fit into her broader goal of trying to understand the rhetoric of visual advertisements. 

Finally, Adriana shares a bit about her work on robust visual reasoning, the parallels between this research and other work happening around explainability, and the vision for her work going forward. 

The complete show notes for this episode can be found at twimlai.com/go/463.