How should regulators view AI risk and opportunity?

Patrick McKenzie (@patio11) shares his remarks to the Bank of England on critical vulnerabilities in financial infrastructure. Drawing from the July 2024 CrowdStrike outage which brought down teller systems at major US banks, Patrick discusses how regulatory guidance inadvertently created dangerous software monocultures. He also examines the stablecoin market, its impressive growth, and the elephant tethered to the room. He also delivers a message from Silicon Valley to other centers of power on the urgent necessity of waking up regarding AI, which almost the entire world currently far underrates.
–

Full transcript available here: www.complexsystemspodcast.com/talking-to-the-bank-of-england/

–

Sponsor: Mercury
This episode is brought to you by Mercury, the fintech trusted by 200K+ companies — from first milestones to running complex systems. Mercury offers banking that truly understands startups and scales with them. Start today at Mercury.com
Mercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group, Column N.A., and Evolve Bank & Trust; Members FDIC.

–

Links:

• The Bank of England: https://www.bankofengland.co.uk/ 
• Bits about Money, Why the CrowdStrike bug hit banks hard: https://www.bitsaboutmoney.com/archive/crowdstrike-bug-hit-banks-hard/ 
• Scaling Laws for Neural Language Models" by Kaplan et al: https://arxiv.org/pdf/2001.08361 
• Stripe Annual Letter 2024: https://stripe.com/annual-updates/2024 

–

Timestamps:

(00:00) Intro
(01:48) The importance of implementation-level understanding
(03:00) Single points of failure
(04:25) Can a 22-year-old engineer close all the banks?
(05:18) The CrowdStrike incident: A case study
(08:34) The culture of "shut up and shuffle"
(09:54) Blameless postmortems
(12:25) What actually happened during CrowdStrike
(18:01) Five whys: Root cause analysis
(19:03) How software monocultures are created
(22:54) Understanding endpoint monitoring software
(25:25) Distributed systems and the nature of CrowdStrike
(31:22) The economics of software monocultures
(33:29) Why wasn't there defense in depth?
(37:05) Why was recovery so difficult?
(40:32) The domino effect across financial institutions
(43:36) What went right: Electronic systems remained up
(45:10) This was a near miss
(49:29) Potential policy responses
(54:03) Switching gears: Stablecoins
(01:01:37) The elephant in the room: Tether
(01:15:32) Who loses if Tether implodes?
(01:16:59) AI and the future of trading
(01:26:47) AI risks in the trading space
(01:30:41) Closing