arxiv preprint - InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes

In this episode, we discuss InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes by Mohamad Shahbazi, Liesbeth Claessens, Michael Niemeyer, Edo Collins, Alessio Tonioni, Luc Van Gool, Federico Tombari. InseRF is a new approach for inserting generated objects into 3D scene reconstructions using NeRF, based on textual descriptions and 2D reference images. This method overcomes the limitations of existing scene editing techniques, which struggle with the generation of new objects, by performing a 2D insertion in a reference view and extrapolating it to 3D with the help of single-view reconstruction and monocular depth estimation priors. Extensive evaluations show that InseRF achieves controllable and 3D-consistent object insertions, outperforming current methods, and it does so without needing explicit 3D models as input.