With a custodial wallet, it's centralized and your funds are basically shared with joint custody between you and the exchange. Whereas with the non costodial wallet, those funds are completely yours. You have access to the private key, so if anything were to go wrong with the exchange, you can always withdraw your funds.