The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Visual Generative AI Ecosystem Challenges with Richard Zhang - #656

Nov 20, 2023

In this discussion, Richard Zhang, a Senior Research Scientist at Adobe Research specializing in visual generative AI, tackles significant challenges in the AI ecosystem. He dives into the creation of effective perceptual metrics for AI, emphasizing the role of LPIPS in aligning human and machine evaluations. Zhang also addresses the pressing need for detection tools to combat fake visuals and the complexities of data attribution in generative art. His insights emphasize the delicate balance between creator autonomy and consumer trust in this rapidly evolving field.

40:40

Creator website

Episode guests

Richard Zhang

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Richard Zhang's work on perceptual metrics like LPIPS improves human-computer alignment in generative AI.

Addressing the challenges of detecting fake visual content is crucial for maintaining trust in generative AI.

Deep dives

Training image-based generative AI models

Richard Zhang discusses his work on image-based generative AI, specifically focusing on the use of deep networks for image generation. He highlights the challenges faced in predicting high-dimensional signals and the limitations of existing loss functions. Zhang shares his early experiences in generative tasks, such as image colorization, and the shift towards adding controllability to generative AI systems.

The Point of Diminishing Returns in Deep Neural Networks

01:21

The Challenge of Retraining Pre-trained Models with Removed Data

01:20

Intro

2min

Evolving Visual Generative AI: Balancing Quality and Community

3min

Creating Effective Perceptual Metrics for Generative AI

3min

Neural Networks and Human Perception

5min

Enhancing Control in Generative AI

4min

Challenges of Visual Generative AI Detection

11min

Understanding Contributions in AI Image Synthesis

12min

Today we’re joined by Richard Zhang, senior research scientist at Adobe Research. In our conversation with Richard, we explore the research challenges that arise when regarding visual generative AI from an ecosystem perspective, considering the disparate needs of creators, consumers, and contributors. We start with his work on perceptual metrics and the LPIPS paper, which allow us to better align human perception and computer vision and which remain used in contemporary generative AI applications such as stable diffusion, GANs, and latent diffusion. We look at his work creating detection tools for fake visual content, highlighting the importance of generalization of these detection methods to new, unseen models. Lastly, we dig into his work on data attribution and concept ablation, which aim to address the challenging open problem of allowing artists and others to manage their contributions to generative AI training data sets.

The complete show notes for this episode can be found at twimlai.com/go/656.

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Visual Generative AI Ecosystem Challenges with Richard Zhang - #656

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

Training image-based generative AI models

Improving perceptual metrics and loss functions

Visual generative AI as an ecosystem

Detection and attribution in generative AI

The Point of Diminishing Returns in Deep Neural Networks

The Challenge of Retraining Pre-trained Models with Removed Data

Remember Everything You Learn from Podcasts