The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) CTIBench: Evaluating LLMs in Cyber Threat Intelligence with Nidhi Rastogi - #729
86 snips
Apr 30, 2025 
In this engaging discussion, Nidhi Rastogi, an assistant professor at the Rochester Institute of Technology specializing in Cyber Threat Intelligence, dives into her project CTIBench. She explores the evolution of AI in cybersecurity, emphasizing how large language models (LLMs) enhance threat detection and defense. Nidhi discusses the challenges of outdated information and the advantages of Retrieval-Augmented Generation for real-time responses. She also highlights how benchmarks can expose model limitations and the vital role of understanding emerging threats in cybersecurity.
AI Snips
Chapters
Transcript
Episode notes
LLMs Add Context to Cybersecurity
- Large language models (LLMs) enable cybersecurity by adding context to detection decisions and speeding analysis.
- Before LLMs, decisions relied purely on patterns without understandable context, limiting comprehension.
RAG Keeps Models Updated
- Retrieval-Augmented Generation (RAG) methods update LLMs with recent threat info beyond training cutoff.
- This approach keeps models current despite static training data limited to past cybersecurity events.
Understanding Cyber Threat Intelligence
- Cyber Threat Intelligence (CTI) aggregates diverse data like logs, reports, and indicators to detect and defend threats.
- CTI helps predict threat patterns in networks by correlating internal activity with external intelligence.