Enhancing Filmmaking with Gaussian Splats in VFX

Filmmaker Gareth Edwards and co-cinematographers Greig Fraser and Oren Soffer embraced the unconventional while making the new science fiction movie The Creator. From the camera used, to how it was shot, to the visual effects, the team brought together film techniques both new and old. It's rare to have two cinematographers working on the same film, but Greig Fraser had a set date to begin prepping for Dune and could not be on location in Thailand for shooting The Creator. Gareth was the co-writer, director and camera operator on the film, and Greig knew Gareth needed support prepping the camera and lighting each location. Greig enjoyed the close collaboration with another cinematographer while shooting the series The Mandalorian and he knew having a second DP would be the ideal situation for shooting The Creator. Cinematographer Oren Soffer was brought in, and Oren, Gareth and Greig all prepped the film together, discussing in detail how Gareth wanted to tell the story. Once shooting began, Greig was tasked with managing the LUT and screening the dailies in a Los Angeles theater, while Gareth and Oren managed the day to day on set. Oren and Greig would talk every day about lighting setups, and they both appreciated having another DP around for feedback and ideas. With a collaborator, they both felt like working on the film was less stressful and it led to better creativity. As Greig told us in his interview with The Cinematography Podcast in 2022, The Creator was shot on a Sony FX3. The FX3 is a very affordable, small, lightweight camera that Gareth was familiar with. It was easier for him to move around handheld, explore his shots, and have the freedom to interact with his actors. Gareth's approach to The Creator was documentary-style, much like his first film, Monsters, but it was important to him that it still looked composed like a film. The FX3 could deliver a quality image at the level they needed for color grading and for visual effects company Industrial Light & Magic to add VFX. Oren points out that if a camera can deliver an image quality that looks like what you want, and fits the technical specifications you need, then any camera the director or DP chooses is the right tool. The images shot on the FX3 did have a lot of digital noise at higher ISOs, but this was a look they embraced for its similarity to film grain. The tools a cinematographer uses will continue to evolve and unlock more creativity. With advances in post production and lighting technology, how the image is made matters a lot less. The most important thing to consider is how does the audience respond to the film? Is the cinematographer doing their job as the storyteller? For his part, Greig likes to know about all the tools available to tell the story, and he wants to have enough knowledge about what's possible to pass on to a director when he's asked. While shooting The Creator, Gareth would let the crew know the general story beats they needed for the day, but he would not share the shot list- it was a reference he kept for himself, so that he could shoot on the fly in an improvisational manner. As the operator, he didn't need to spend a lot of time explaining the shots he needed to get, or rely on storyboards. Since the visual effects were designed after the footage was shot, the storyboards only acted as a reference. Gareth wanted all of the pieces, including the action, to have the energy of spontaneity. Oren was able to “set up the sandbox for him and the actors to play in. It meant lighting more broadly, but we would know which direction he'd be shooting, and augmenting it on a shot by shot basis with small LED lights or a helios tube on a boom pole. It was like growing a film in a pot of dirt in your backyard.” For the visual effects on The Creator, Gareth chose to be very sparing in his use of 3D special effects., spending the budget only when it was needed to render detailed objects like the robots.