Tropic is a company that focuses on X-Risk and AI safety broadly. It's trying to trade off the need to build very powerful systems, but also to make them safe. A Vox article has an almost socratic dialogue between Evan Hubinger and Jared Kaplan over at Tropic.