Exploration of Screen Replacement in Spatial Computing for Immersive VR Experiences

YouTuber Brad Lynch (aka SadlyItsBradley) has completely replaced all of his computer screens with an Apple Vision Pro, even going as far as getting a headless Mac Book Pro that does not even have a screen. He's been using the Apple Vision Pro for around 8 hours a day since launch, and I wanted to get a sense of how he's been using it. It turns out the he is mostly streaming his gaming PC via Moonlight and using social VR apps like VRChat via ALVR to hang out with friends and ambient hang out virtual spaces. He's also using SteamVR overlays to augment his virtual reality experience with Steam apps like XSOverlay, VRHandsFrame, and OVR Advanced Settings. It's feels like a very niche use case of a hardcore VR enthusiast, but one that mixes and mashes realities in a way that might be a sign of things to come. Some of the most compelling apps for Lynch are open source that enable experiences that are being driven by his high-end Windows machine. https://twitter.com/SadlyItsBradley/status/1783406980192702842 Lynch has been closely following a lot on VR hardware developments over the last number of years, but the Apple Vision Pro has satisfied most of his desires in what he wants within a high-end spatial computing device. The resolution is high enough and the quality good enough so that he can spend more time exploring different screen replacement use cases, augmented VR experiences via overlays, and productivity use cases of XR. Most PCVR enthusiasts are Windows users, and so Lynch's audience has traditionally focused more on the gaming use cases of VR. As a result they have not been as interested in the Apple Vision Pro due to the lack of high-fidelity input controls. But for Lynch, the basic locomotion gestures made available in ALVR are good enough for him to get around within VRChat without needed to hook up or use any external controllers. Because of the perceived or actual gaps between his ideal spatial computing use cases and his VR gaming audience, then Lynch actually scrapped his formal review video and is considering releasing clips or falling back to Q&A livestreams to field many questions about the trajectory of hardware in the XR industry. Lynch also has been enjoying the mashing up of spatial contexts in XR, mostly via the SteamVR overlays and windows but mentions some experiments of bringing in fully spatial objects. It reminds me of the interview that I did with the PlutoVR founders in 2020 when they were experimenting a lot with the idea of multi-app spatial computing paradigms with WebXR and apps like Aardvark by Joe Ludwig. Apple is slowly building out more and more spatial primitives across all of their operating systems, and are slowly becoming more and more game engine-like as new APIs were announced as a part of their WWDC, where visionOS 2.0 was announced as coming out later this Fall. We talk about some of the quality of life features, but also the role of an integrated ecosystem, and what BigScreen Beyond, Valve, and Meta might do to keep up with how Apple is pushing forward these ideas of multi-app integrations within spatial computing. At the moment Lynch's 8 hours of daily usage is likely an extreme outlier, but the types of ways that he's blending realities together feels like there's something deeper that we'll continue to see moving forward. VR typically involves a complete context shift, while AR tends to bring in modular elements of other contexts to shift your existing context. Lynch is on the bleeding edge of fully immersing himself within these virtual contexts, but modulating his experience with these SteamVR overlays in what could best be described as a sort of AR within VR use case. The visionOS 2.0 Beta release (coming this Fall) allows users to overlay their Mac Virtual Display over immersive environments, but SteamVR Overlays already enable this on PCVR experiences and so Lynch's experiences with overlays in VRChat could definitely be a sign of how Apple might e...