Paul's Security Weekly (Audio) AI Is Oversharing and Leaking Data - Sounil Yu - PSW #865
11 snips
Mar 13, 2025 Sounil Yu, CTO and co-founder of Gnostic, addresses pressing AI data leakage issues. He highlights how tools like Microsoft's Copilot lack adequate access controls, risking exposure of sensitive files. Gnostic’s innovative automation tackles these challenges, ensuring security without hindering innovation. The conversation dives into the complex balance of non-human identities and the dual risks of oversharing and undersharing in data management. Additionally, the impact of AI on job applications is examined, raising questions about the future role of human expertise in an increasingly automated world.
AI Snips
Chapters
Books
Transcript
Episode notes
Codify Need-To-Know For LLMs
- Need-to-know must be codified for LLMs so models respect role-based knowledge boundaries.
- Treat knowledge segmentation as a policy layer that balances undersharing and oversharing.
Need-To-Know Lives Mostly In Heads
- Sunil observes need-to-know often lives only 'in our heads' and isn't captured in formal guides.
- That informal discretion makes access reviews hard because business context is missing.
Vendors Gate AI Features When Governance Is Weak
- Vendors may restrict AI features (e.g., Gemini on Drive) until they ensure governance due to poor access controls in customer environments.
- Product rollouts sometimes reflect an expectation of weak enterprise governance.
