Ep121: Ethical Hackers and AI Agents: The Future of Vulnerability Management with HackerOne

HackerOne uses ethical hackers and AI to find vulnerabilities before criminals

White hat hackers stress test systems to identify security weaknesses proactively

Generative AI plays a huge role in HackerOne's security operations

Security teams struggle with constant toil of finding and fixing vulnerabilities

AI helps minimize toil through natural language interfaces and automation

Both good and bad actors have access to generative AI tools

Success requires measuring individual task inputs and outputs, not just aggregates

Breaking down workflows into granular tasks reveals measurable AI improvements

HackerOne deployed "Hive," their AI security agent to reduce customer toil

Initial focus was on tasks where AI clearly outperformed humans

Started with low-hanging fruit before tackling more complex strategic workflows

Accuracy is the primary success metric, not just efficiency or speed

Security requires precision; wrong fixes create bigger problems than inefficiency

Customer acceptance and reduced time to remediation are north star metrics

Humans remain the source of truth for validation and feedback loops

Break down human jobs into granular AI tasks using systems thinking

Build specific agents for individual tasks rather than entire job roles

Keep humans accountable for end-to-end outcomes to maintain customer trust

AWS Bedrock chosen for security, confidentiality, and data separation requirements

Moving from efficiency improvements to entirely new AI-enabled capabilities