arxiv preprint - Self-correcting LLM-controlled Diffusion Models

In this episode, we discuss Self-correcting LLM-controlled Diffusion Models by Tsung-Han Wu, Long Lian, Joseph E. Gonzalez, Boyi Li, Trevor Darrell. The paper introduces Self-correcting LLM-controlled Diffusion (SLD), a novel approach to improve text-to-image generation by incorporating a loop where an image is generated, evaluated, and corrected iteratively based on a given text prompt using a Language Model (LLM). SLD can be applied to existing diffusion models and has shown proficiency in generating more accurate images, particularly in aspects requiring understanding of numbers, attributes, and spatial relations. The authors also highlight SLD's capability for image editing through prompt modification and announce their intention to make the code publicly available to foster further research.