The podcast discusses the disastrous launch of the Humane AI Pin, deepfake nudify apps advertised on Instagram, and AI-generated product placements on Reddit. They also touch on tech failures, AI investments, and the importance of accountability in the tech industry.
Facebook profits from deepfake ads, lacking proactive monitoring measures.
Deep dives
AI-generated NFT apps advertised on Instagram and their reconciliation with regulations
Instagram has been featuring ads for AI tools that can create non-consensual deepfake images, which poses ethical concerns. The apps include tools like 'Neutify' that undress people in photos and models that create nude images of individuals. These apps were originally found on the dark web but have now surfaced openly on platforms like Instagram, leading to widespread exposure. While Instagram has taken action to remove these ads upon notification, their sporadic monitoring raises questions about their ability to proactively moderate harmful content.
Issues with monitoring and regulation of deepfake advertisement on Instagram
The process of monitoring and regulating deepfake ads on Instagram highlights larger issues in the tech space. Despite the challenges of content moderation at scale, the discovery and removal of harmful deepfake ads from Instagram reveal gaps in proactive moderation. The lack of systematic review and the reactive responses to flagged content indicate a need for more stringent oversight and preventive measures to curb the dissemination of harmful deepfake material.
Facebook's profiting from deepfake ads and the call for improved oversight
Facebook, the parent company of Instagram, is accepting money for running deepfake app ads that promote unethical practices. While the ads are targeted and may not reach all users, Facebook's passive approach to monitoring and responding to such content raises concerns. Facebook has the resources to institute dedicated trust and safety teams for ad moderation but appears to lack proactive measures in scrutinizing advertisements promoting non-consensual deepfake imagery. The financial gains from these ads underscore the need for Facebook to improve its oversight and identify and address harmful content more efficiently.
The ease and necessity of content moderation in addressing non-consensual deepfake ad proliferation
The prevalence of non-consensual deepfake ads, especially those promoting tools like Neutify, underscores the urgency for effective content moderation mechanisms. While platforms like Facebook and Instagram face challenges in moderating massive volumes of content, the rapid identification and removal of harmful ads point to the feasibility of efficient moderation strategies. The exposure of deepfake ad incidents calls for enhanced monitoring practices and investigative efforts to prevent the proliferation of unethical and damaging content.
Exploring the implications and responses to deepfake ad dissemination on social media
The dissemination of deepfake ads on social media platforms like Instagram sheds light on the broader implications of unregulated ad content. The incident raises questions about the accountability of tech companies in controlling the spread of harmful deepfake material. While efforts to remove offending ads upon notification are commendable, more proactive measures are essential to combat the growing trend of non-consensual deepfake ad campaigns. Robust trust and safety teams, coupled with stringent monitoring protocols, can help mitigate the negative impact of such ads on social media platforms.
This is a long one! First Joseph previews his upcoming book, DARK WIRE. Go to the link below and enter "DW20" for 20% off! Also make sure to redeem your preorder behind-the-scenes bonus content. As for this week's stories, we start off with Jason explaining why we can, we must, dunk on the Humane Ai Pin, a piece of hardware that is AI's Juicero moment. After the break, Emanuel shows that Instagram is advertising deepfake nudify apps, the sort that are being abused right now in schools. In the subscribers-only section, we talk a bunch about Reddit, and especially how a tool has been made to sprinkle AI-generated product placements into popular Reddit threads.
