AI Is Oversharing and Leaking Data - Sounil Yu - PSW #865
Mar 13, 2025
auto_awesome
Sounil Yu, CTO and co-founder of Gnostic, addresses pressing AI data leakage issues. He highlights how tools like Microsoft's Copilot lack adequate access controls, risking exposure of sensitive files. Gnostic’s innovative automation tackles these challenges, ensuring security without hindering innovation. The conversation dives into the complex balance of non-human identities and the dual risks of oversharing and undersharing in data management. Additionally, the impact of AI on job applications is examined, raising questions about the future role of human expertise in an increasingly automated world.
Enterprise AI search tools, like Microsoft's Copilot, often lack necessary access controls, increasing the risk of data leaks.
Knostic's solution enables organizations to implement protective access controls, safeguarding sensitive data without stifling innovation.
Recent appointments in national cybersecurity roles have raised concerns due to a perceived lack of relevant technical expertise in leadership.
The vulnerability of poorly secured digital wallets was highlighted by one of history's largest crypto heists, prompting calls for enhanced security measures.
Deep dives
AI Defenses Against Oversharing
Artificial Intelligence (AI) tools are increasingly utilized within organizations, but their potential for oversharing and data leaks poses significant risks. Tools like Microsoft's Copilot and Glean often lack the necessary access controls to ensure sensitive information is protected from unauthorized exposure. Gnostic is introduced as a solution to this problem, enabling enterprises to implement proper access controls through the establishment of a 'need-to-know' policy. This policy ensures that sensitive information is only accessible to individuals with a legitimate requirement, thereby helping organizations mitigate the risks of data leaks.
The appointment of a new National Cyber Director and the head of CISA has sparked controversy due to the individuals' lack of cybersecurity experience. Critics argue that effective leadership in cybersecurity requires a deep understanding of the intricate challenges involved in protecting digital infrastructures. While the intent of such appointments might be to bring fresh perspectives, the absence of technical expertise may hinder the ability of these leaders to effectively address the complexities of the cybersecurity landscape. This has raised concerns about the future of national cybersecurity efforts amid growing threats.
Dark Storm Takes Down Social Media
A recent attack dubbed 'Dark Storm' managed to take down the social media platform X, also known as Twitter. Attribution for the attack has been controversial, as claims of responsibility pointed to various sources. The motivations behind the attack seem intertwined with geopolitical contexts, although concrete evidence regarding the origins remains elusive. This incident reflects ongoing challenges in ensuring the security and stability of popular social media platforms amidst increasing pressures from malicious actors.
Use Cases for Laura Technology
Laura technology has emerged as a significant tool for secure long-range communications between devices. Recent discussions highlighted its effectiveness in creating covert communication channels, particularly in cybersecurity applications. By leveraging Laura technology, hackers can maintain low-power, stealthy communications that can evade detection. These compelling use cases underscore Laura's potential in various domains, especially in environments that require secure, long-distance transmissions.
The Biggest Crypto Heist
One of the largest crypto heists in history has brought to light vulnerabilities associated with poorly secured digital wallets. The perpetrators exploited flaws in a hardware wallet system, which, despite its intended security features, lacked essential safeguards. Their ability to bypass standard transaction protocols highlighted significant gaps in both security practices and technology solutions within the cryptocurrency space. This incident stresses the necessity for enhanced security measures as the adoption of cryptocurrency continues to rise.
JTAG and Flipper Zero Integration
The integration of JTAG capabilities into devices like the Flipper Zero has drawn attention for its potential applications in hardware hacking and security testing. The article discusses how this technology can be used to access and manipulate hardware at a low level, providing insights into potential vulnerabilities. By using a Flipper Zero for JTAG operations, users can leverage its features to probe devices more effectively than traditional methods. These developments reflect broader trends in the hardware hacking community, emphasizing accessibility and innovation in security research.
Exploiting EDR by Manipulating Hardware
Innovative techniques have emerged that allow attackers to bypass endpoint detection and response (EDR) systems through low-level hardware manipulation. By leveraging devices that lack comprehensive security measures, attackers can execute ransomware and other malicious software without triggering alerts. This highlights a significant gap in security protocols and the need for organizations to enhance their defenses against sophisticated threats that target the underlying hardware. The discussion emphasizes the importance of continuous monitoring and improvement of security measures to keep pace with evolving attack techniques.
The Funkalicious Vulnerability Disclosure
A recent vulnerability disclosure humorously dubbed the 'Funkalicious exploit' exemplifies the creative spirit in cybersecurity discussions. The exploit details a weakness within a PHP-based building management system, illustrating how vulnerabilities can persist in seemingly mundane technology. This disclosure not only serves as a reminder of the importance of rigorous security assessments but also revives a playful approach to describing cybersecurity flaws. Such colorful terminology in vulnerability disclosures can often help make technical discussions more engaging and accessible.
Sounil Yu joins us to kick things off with AI defenses: Enterprise AI search tools like Copilot for Microsoft 365 lack the in-depth access controls required to ensure that query responses align with the user’s need-to-know boundaries. Without proper controls, these tools accelerate the discovery of improperly secured sensitive files within the organization. Knostic’s solution ensures that enterprise data is safeguarded without slowing down innovation. By automating the detection and remediation of LLM data exposure, Knostic helps organizations mitigate the security, privacy, and compliance risks associated with AI chatbots and enterprise search tools.
In the security news: The controversial pick for National Cyber Director, the not-so-controversial pick to lead CISA, complete with funding cuts, the controversial ESP32 backdoor that is not a backdoor but hidden features, Dark Storm takes down X, interesting use cases for LoRa, using AI to get your dream job, details on the biggest crypto heist in history, an EDR bypass and a 404 error, slipping through the cracks in CVSS, old school vulnerability disclosure in 2025, Rayhunter, a pen test that should not have been, JTAG and your Flipper Zero, a Linux webcam was used for what now?, and "Spatial-Domain Wireless Jamming with Reconfigurable Intelligent Surfaces"!
