#95 – Dawn Song: Adversarial Machine Learning and Computer Security
May 12, 2020
auto_awesome
Dawn Song, a UC Berkeley professor specializing in security and machine learning, discusses crucial topics like the vulnerabilities in software and the risks posed by human error. She delves into adversarial machine learning, revealing its implications for autonomous vehicles and the need for enhanced defenses. Privacy concerns and data ownership dynamics are highlighted, alongside emerging strategies like differential privacy. The conversation also touches on program synthesis and the journey from physics to computer science, emphasizing the beauty of both fields.
Software vulnerabilities are prevalent due to various attacks like memory safety issues and buffer overflows.
Formal verification techniques provide probable guarantees of program security by analyzing for memory safety vulnerabilities.
Human vulnerabilities are exploited by social engineering attacks, highlighting the need for AI defenses.
Adversarial machine learning poses challenges to autonomous systems like Tesla's Autopilot, requiring defense strategies and constant vigilance.
Deep dives
Security Vulnerabilities in Software Systems
It is challenging to write completely bug-free code without vulnerabilities, considering the broad definition of vulnerabilities, including various types of attacks like memory safety vulnerabilities. The dynamic nature of attacks, such as buffer overflows, can lead to unintended changes in program states, allowing attackers to take control. There are different attack forms, like side channels, where attackers can exploit program behaviors. Form verification techniques aim to provide probable guarantees of a program's security properties by analyzing code for memory safety vulnerabilities.
Developing Formally Verified Systems
The development of formally verified systems has shown progress in creating secure systems by conducting static verification of program properties. Dedicated teams have worked on program verification techniques to prove code integrity, resulting in verified systems like microkernels, compilers, and crypto libraries. While advancements in verified systems are significant, the evolving nature of attacks necessitates continual progress. Program verification techniques are primarily conducted statically without executing the code to analyze program behaviors.
Security Vulnerabilities in Humans
Attacks are shifting focus from system vulnerabilities to targeting humans due to humans often being recognized as the weakest link in security. Social engineering attacks manipulate humans to breach systems, emphasizing the need for AI and machine learning defenses to assist humans. Phishing attacks and deep fake manipulations are becoming severe issues targeting human vulnerabilities. AI chatbot technologies can aid in detecting social engineering attacks by observing and engaging with suspicious correspondences to protect users.
Adversarial Machine Learning and Real-World Attacks
Adversarial machine learning attacks pose significant challenges to real-world systems, such as autonomous vehicles, by demonstrating the feasibility of attacks like perturbations on stop signs affecting visual systems like Tesla's Autopilot. While Elon Musk downplays concerns, demonstrated attacks and research underline the vulnerability of systems to misclassifications and targeted attacks. Multimodal sensor integration and consistent checks are essential defense strategies to safeguard autonomous vehicles against sensory-based attacks in vision and other sensory inputs, contributing to a more robust learning system.
The Significance of Differentially Private Language Models
Training differentially private language models can enhance privacy protection while maintaining utility. By adding noise during the training process in a differentially private manner, the models can resist attacks and offer increased privacy. This approach ensures data privacy through perturbations in the training process, creating a differentially private learned model.
Ownership of Data and Monetization Potential
The discussion covers the idea of ownership of personal data and its value in targeted advertisements. Companies leverage personal data for effective ad targeting due to the free services users enjoy online. The potential future involves individuals understanding their data's value, monetizing it explicitly, and gaining control over its usage.
Differential Privacy Mechanisms and Data Security
Differential privacy mechanisms add noise during data processing to ensure individuals' presence in datasets remains undisclosed. The perturbation in training processes ensures a differentially private learned model, thus enhancing privacy protection. Integrating noise and perturbation in computations strengthens data security and privacy measures.
Balancing Privacy and Utility in Data Economy
The dialogue emphasizes the need for nuanced discussions in balancing utility and privacy, particularly crucial in data economy. Establishing ownership of data is a foundational step in determining data usage. Resolving complex issues necessitates understanding ownership rights to regulate data utilization responsibly.
Dawn Song is a professor of computer science at UC Berkeley with research interests in security, most recently with a focus on the intersection between computer security and machine learning.
Support this podcast by signing up with these sponsors:
– Cash App – use code “LexPodcast” and download:
– Cash App (App Store): https://apple.co/2sPrUHe
– Cash App (Google Play): https://bit.ly/2MlvP5w
This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon.
Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time.
OUTLINE:
00:00 – Introduction
01:53 – Will software always have security vulnerabilities?
09:06 – Human are the weakest link in security
16:50 – Adversarial machine learning
51:27 – Adversarial attacks on Tesla Autopilot and self-driving cars
57:33 – Privacy attacks
1:05:47 – Ownership of data
1:22:13 – Blockchain and cryptocurrency
1:32:13 – Program synthesis
1:44:57 – A journey from physics to computer science
1:56:03 – US and China
1:58:19 – Transformative moment
2:00:02 – Meaning of life
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
