Understanding Data Lineage in Model Training and Output

MLOps podcast #190 with Ads Dawson, Senior Security Engineer at Cohere, Guarding LLM and NLP APIs: A Trailblazing Odyssey for Enhanced Security. // Abstract Ads Dawson, a seasoned security engineer at Cohere, explores the challenges and solutions in securing large language models (LLMs) and natural language programming APIs. Drawing on his extensive experience, Ads discusses approaches to threat modeling LLM applications, preventing data breaches, defending against attacks, and bolstering the security of these critical technologies. The presentation also delves into the success of the "OWASP Top 10 for Large Language Model Applications" project, co-founded by Ads, which identifies key vulnerabilities in the industry. Notably, Ads owns three of the top 10 vulnerabilities, including Training Data Poisoning, Sensitive Information Disclosure, and Model Theft. This OWASP Top 10 serves as a foundational resource for stakeholders in AI, offering guidance on using, developing, and securing LLM applications. Additionally, the session covers insider news from the AI Village's 'Hack the Future' | LLM Red Teaming event at Defcon31, providing insights into the inaugural Generative AI Red Teaming showdown and its significance in addressing security and privacy concerns amid the widespread adoption of AI. // Bio A mainly self-taught, driven, and motivated proficient application, network infrastructure & cyber security professional holding over eleven years experience from start-up to large-size enterprises leading the incident response process and specializing in extensive LLM/AI Security, Web Application Security and DevSecOps protecting REST API endpoints, large-scale microservice architectures in hybrid cloud environments, application source code as well as EDR, threat hunting, reverse engineering, and forensics. Ads have a passion for all things blue and red teams, be that offensive & API security, automation of detection & remediation (SOAR), or deep packet inspection for example. Ads is also a networking veteran and love a good PCAP to delve into. One of my favorite things at Defcon is hunting for PWNs at the "Wall of Sheep" village and inspecting malicious payloads and binaries. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://github.com/GangGreenTemperTatum OWASP Top 10 for Large Language Model Applications Core Team Member and Founder - https://owasp.org/www-project-top-10-for-large-language-model-applications/CoreTeam Fork for OWASP Top 10 for Large Language Model Applications - https://github.com/GangGreenTemperTatum/www-project-top-10-for-large-language-model-applications Security project: llmtop10.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ads on LinkedIn: https://www.linkedin.com/in/adamdawson0/ Timestamps: [00:00] Ads' preferred coffee [00:46] Takeaways [02:52] Please like, share, and subscribe to our MLOps channels! [03:11] Security and vulnerabilities [05:24] Work at Cohere and OWASP [08:11] Previous work vs LLMs Companies [09:46] LLM vulnerabilities [10:38] Good qualities to combat prompt injection problems [13:26] Data lineage [16:03] Red teaming [19:39] Freakiest LLM vulnerabilities [22:17] Severe Autonomy Concerns [25:13] Hallucinations [27:59] Prompt injection [29:15] Vector attacks to be recognized [32:02] LLMs being customed [33:18] Security changes due to maturity [38:17] OWASP Top 10 for Large Language Model Applications [44:31] Gandalf game [46:06] Prompt injection attack [49:46] Overlapping security [53:26] Data poisoning [56:57] Toxic data for LLMs [58:50] Wrap up