Going Serverless in Financial Services with Brian McNamara
Jan 7, 2025
auto_awesome
Brian McNamara, a Distinguished Engineer at Capital One, dives into serverless computing in financial services. He discusses how Capital One transitioned to this model, focusing on unique challenges like governance and cost management. Brian highlights the importance of collaboration among teams and the strategic shift in engineering roles. He explores the benefits of re-architecting applications for the cloud and the necessity of effective notification systems for developers. Listeners gain insights into the future trends of serverless technology in the banking sector.
Capital One's shift to serverless computing significantly improved developer experience by reducing operational burdens and fostering innovation through minimized costs.
The adoption of serverless technologies in regulated financial services requires rigorous governance and compliance, integrating controls seamlessly into development pipelines for security.
Deep dives
Transition to Serverless Computing
Capital One made a strategic shift to serverless computing to enhance the developer experience while reducing operational burdens. This transition, initiated in 2021, aimed to establish Capital One as a 'serverless-first' company, encouraging teams to consider serverless technologies like AWS Lambda and Fargate for new applications. The underlying motivation was to minimize not just cloud infrastructure costs, but also engineering and maintenance costs, allowing developers to focus on innovation. Leadership recognized that serverless technologies offer built-in resiliency and flexibility, enabling applications to scale based on real-time business demand.
Challenges in Implementation
Adopting serverless computing at Capital One encountered several challenges, largely rooted in fear, uncertainty, and doubt (FUD) about the technology's capabilities. Many developers initially viewed serverless options like AWS Lambda as unsuitable for business-critical applications, primarily due to misconceptions about scalability and performance limits. To overcome these biases, the company invested in thorough education around the proper architecture and efficient utilization of serverless technologies, encouraging teams to assess their workloads to determine appropriate solutions. This proactive approach facilitated an understanding of how to optimize serverless functions, particularly regarding processing duration and resource allocation.
Governance and Compliance in Cloud Adoption
Operating in the heavily regulated financial services industry, Capital One emphasizes rigorous governance and compliance when adopting cloud technologies. The company employs tools like Open Policy Agent to ensure that all deployed services conform to internal controls and regulatory standards. Developers are educated on compliance requirements to maintain the privacy and security of customer data while enhancing agile practices. This balance is achieved by integrating governance seamlessly into development pipelines, enabling teams to effectively create compliant applications while remaining productive.
Monitoring and Observability in Serverless Environments
In serverless environments, effective monitoring and observability become critical, as traditional methods for managing performance do not apply. Developers must rely on AWS-provided metrics and custom instrumentation to track application health and performance. Tools like AWS Lambda Power Tuning assist in optimizing resource allocation by aiding developers in determining the right function size based on usage patterns. This disciplined approach to observability ensures that the development teams remain agile, capable of swiftly adapting applications while maintaining oversight of their performance metrics.
Serverless computing is a cloud-native model where developers build and run applications without managing server infrastructure. It has largely become the standard approach to achieve scalability, often with reduced operational overhead. However, in banking and financial services, adopting a serverless model can present unique challenges.
Brian McNamara is a Distinguished Engineer at Capital One where he works in serverless integration and development. Brian joins the show with Sean Falconer to talk about why Capital One shifted to a serverless approach, how to think about cloud costs, establishing governance controls, tools to stay well-managed, and much more.
Full Disclosure: This episode is sponsored by Capital One.
Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from AI to quantum computing. Currently, Sean is an AI Entrepreneur in Residence at Confluent where he works on AI strategy and thought leadership. You can connect with Sean on LinkedIn.