The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Expressive Deep Learning with Magenta DDSP w/ Jesse Engel - #452
Feb 1, 2021
Jesse Engel, a Staff Research Scientist at Google focused on the Magenta Project, dives into the intersection of creativity and AI. He shares insights on how Differentiable Digital Signal Processing (DDSP) marries classical sound design with deep learning. Jesse discusses training music generation models that efficiently capture polyphonic relationships while navigating the challenges of traditional audio techniques. He also highlights collaborative projects enhancing music, language, and visual art, promoting a future of democratized musical creativity.
AI Snips
Chapters
Transcript
Episode notes
AI-Augmented Creativity
- Magenta aims to enhance creative agency through ML, not replace humans.
- They focus on assistive technologies for creativity, exploring ML's role in the creative process.
Creative AI Landscape
- Current creative AI focuses on realistic outputs, but Magenta prioritizes expressivity, interactivity, and adaptability.
- They explore how to make models useful and empower creative control.
DDSP Explained
- DDSP combines interpretable classical DSP elements (filters, oscillators) with deep learning's expressivity.
- It generates raw waveforms by modulating simple components with a neural network, allowing detailed manipulation.