The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Visual Generative AI Ecosystem Challenges with Richard Zhang - #656
15 snips
Nov 20, 2023 In this discussion, Richard Zhang, a Senior Research Scientist at Adobe Research specializing in visual generative AI, tackles significant challenges in the AI ecosystem. He dives into the creation of effective perceptual metrics for AI, emphasizing the role of LPIPS in aligning human and machine evaluations. Zhang also addresses the pressing need for detection tools to combat fake visuals and the complexities of data attribution in generative art. His insights emphasize the delicate balance between creator autonomy and consumer trust in this rapidly evolving field.
AI Snips
Chapters
Transcript
Episode notes
Colorization Challenges
- Richard Zhang's early work on image colorization revealed the challenge of creating loss functions.
- Early colorization attempts yielded dull, blurry results due to inadequate loss functions, not aligned with human perception.
Perceptual Metrics
- Defining a mathematical function that captures the nuances of human visual perception is difficult.
- A simple L2 loss function, comparing images pixel by pixel, doesn't accurately reflect human perception.
Data-Driven Perceptual Metric
- Richard Zhang developed LPIPS, a perceptual metric, using a data-driven approach.
- This involved collecting human judgments on distorted image patches to train a model aligned with human perception.