This Week In Digital Trust

**CONTENT WARNING** This episode contains discussion of suicide and mental health themes. Please contact Lifeline on 13 11 14 or Beyond Blue on 1300 224 636. This week we discuss the rise of AI companions.In the midst of a continued AI acceleration push in Australia, we discuss the emergence of several disturbing stories in which AI chatbots have apparently lead young people and adults alike to self-harm.In the face of known harms, many of these AI chatbots continue to be designed primarily to increase engagement and - as some have argued - addiction and dependency. Links:Article about AI companions impact on young Australians (ABC News) https://www.abc.net.au/news/2025-08-12/how-young-australians-being-impacted-by-ai/105630108Podcast including discussion of case of Adam Raine https://podcasts.apple.com/us/podcast/how-openais-chatgpt-guided-a-teen-to-his-death/id1460030305?i…Discussion about case of Sewel Setzer III (Tech Policy Press) https://www.techpolicy.press/ai-companions-and-the-law/Research into impacts of AI companions (Tech Policy Press) https://www.techpolicy.press/new-research-sheds-light-on-ai-companions/Report on AI companion market size https://www.businessresearchinsights.com/market-reports/ai-companion-market-117494?ref=theoverview.…Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we unpack the national conversation about AI's role in solving Australia's productivity challenges.The Productivity Commission recently published a report arguing for a cautious approach to AI regulation, on the basis that it could prevent Australia from "unlocking a surge of productivity".Business and technology advocates have also doubled down, using the moment to call for various legal exemptions and a more laissez-faire approach to restrictions on companies building and deploying AI models.We explore these ideas and the specific implications for areas like privacy and copyright.Links:Productivity Commission report https://www.pc.gov.au/inquiries/current/data-digital/interimArticle about Productivity summit (ABC News) https://www.abc.net.au/news/2025-08-07/artificial-intelligence-jim-chalmers-economics-reform-roundtable/105618958Report on AI benefits (METR) https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/Article on how most AI experimentation is wasted effort (HBR) https://hbr.org/2025/08/beware-the-ai-experimentation-trapArticle about CBA call center AI bungle and stats on unsuccessful AI adoption (SMH) https://www.smh.com.au/business/banking-and-finance/bring-back-the-humans-cba-s-embarrassing-ai-jobs-bungle-a-salutary-lesson-20250821-p5moqk.htmlPrivacy Commissioner's response to Productivity Commission report https://www.oaic.gov.au/news/blog/getting-ai-right-benefits-businesses,-productivity-and-the-communityArticle about artists response to Productivity Commission report  (ABC News) https://www.abc.net.au/news/2025-08-13/productivity-commission-ai-report-copyright-law-authors-respond/105646086Interview with Scott Farquhar (ABC) https://www.abc.net.au/news/2025-08-12/could-australia-benefit-from-the-revolution-in-ai/105645406 Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we take a deeper look at recent high-profile cyber attacks, which have breached organisations like Qantas and Marks & Spencer.For all the talk of technical sophistication, especially in the time of AI, hackers like Scattered Spider are relying less on fancy code and more on their ability to exploit gaps in our systems of trust.We explore how the biggest vulnerabilities in our systems are sometimes found not in software flaws but in how we vouch for ourselves online, and the way we govern relationships between companies and their suppliers. Links:Article about Scattered Spider (Axios) https://www.axios.com/newsletters/axios-future-of-cybersecurity-fab0c0d0-5760-11f0-8c09-e72f7ef7d797?utm_source=chatgpt.comArticle about the psychological edge behind Qantas hack (AFR) https://www.afr.com/companies/transport/qantas-says-6-million-aussies-caught-up-in-cyberattack-20250702-p5mbupAustralian Government advisory on Scattered Spider https://www.cyber.gov.au/about-us/view-all-content/alerts-and-advisories/scattered-spider?utm_source=linkedin&utm_campaign=scatter-spider&utm_medium=social&utm_content=advisoryDarknet Diaries episode about profit driven hacking communities ⁠https://darknetdiaries.com/episode/112⁠Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we explore why so many organisations fall short in how they respond publicly to a cyber incident or data breach.Even for organisations that have a well-exercised "crisis communications" muscle, too often a data breach results in a response that infuriates and erodes trust in equal measure.We explore why standard crisis comms principles don't always apply in a data breach, and what good looks like.Links:Article with analysis of Qantas comms response (Mumbrella) https://mumbrella.com.au/just-trust-us-were-qantas-880124Article with analysis of Qantas comms response (Security Brief) https://securitybrief.com.au/story/experts-give-qantas-mixed-reviews-on-cyber-breach-responseOped from James Turner on Qantas data breach (AFR) https://www.afr.com/technology/why-the-qantas-hack-should-send-chills-around-corporate-australia-20250710-p5me2rArticle about Qantas data breach (Guardian) https://www.theguardian.com/business/2025/jul/02/qantas-confirms-cyber-attack-exposes-records-of-up-to-6-million-customersArticle about PageUp data breach (Guardian) https://www.theguardian.com/technology/2018/jun/07/thousands-of-job-seekers-details-potentially-exposed-in-hackelevenM blog on cyber crisis communications (elevenM) https://elevenm.com.au/blog/the-four-pitfalls-of-cyber-crisis-communications/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

This week we revisit the conversation around banning under 16s from social media, and the range of technologies being considered to verify the ages of would-be social media users.We examine the preliminary findings of a government-commissioned trial of age assurance technologies, as well as some independent media reporting on technologies in the trial.The preliminary results have received mixed reviews, with particular concerns over the accuracy of facial recognition-based age estimation. Links:Preliminary findings of age assurance trial https://ageassurance.com.au/wp-content/uploads/2025/06/News-Release-Preliminary-Findings-for-publication-20250620.pdfArticle about preliminary findings of age assurance trial (ABC News) https://www.abc.net.au/news/2025-06-20/age-assurance-trial-preliminary-report/105440288Article highlighting issues with facial age estimation technologies (ABC News) https://www.abc.net.au/news/2025-06-19/teen-social-media-ban-technology-concerns/105430458Article about expert resigning from assurance trial advisory board (Crikey) https://www.crikey.com.au/2025/06/23/expert-resigns-teen-social-media-ban-tech-trial-advisory-board/TWIDT #79 The long and winding road to age verification https://elevenm.com.au/podcast/episode/the-long-and-winding-road-to-age-verification/TWIDT #122 Big ban theory – Why Australia’s social media ban for kids is raising eyebrows https://elevenm.com.au/podcast/episode/122-big-ban-theory-why-australias-social-media-ban-for-kids-is-raising-eyebrows/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we explore growing tension between the importance of safeguarding user privacy online, and the public and social value that comes from researchers having access to growing volumes of data on social media platforms.In recent years, researchers have used platform data to shed useful light on a range of issues including mental health, social and cultural dynamics, the evolution of democracy online ... but also, importantly, how the platforms themselves are managing domains like user privacy, content algorithms and content moderation.In some cases, the inability to access the data safely has either thwarted research or led to unsavoury data access practices. We look at ways researchers could be granted safe and ethical data access, including legislated transparency requirements. Links:Article about researchers accessing Discord user data (404) https://www.404media.co/researchers-scrape-2-billion-discord-messages-and-publish-them-online/Article about Twitter clamping down researcher access to its API (TechCrunch) https://techcrunch.com/2023/02/14/twitters-restrictive-api-may-leave-researchers-out-in-the-cold/Article about Meta disabling researcher access to data (The Verge) https://www.theverge.com/2021/8/6/22613525/facebook-nyu-research-ban-cambridge-analytica-platformerStatement: Research Cannot Be the Justification for Compromising People’s Privacy (Meta) https://about.fb.com/news/2021/08/research-cannot-be-the-justification-for-compromising-peoples-privacy/Article about DSA transparency requirements (Tech Policy Press) https://www.techpolicy.press/researcher-access-to-platform-data-and-the-dsa-one-step-forward-three-steps-back/Report on achieving digital platform transparency in Australia (Reset.Tech Australia) https://au.reset.tech/news/achieving-digital-platform-public-transparency-in-australia/Mozilla research into platforms data access initiatives (Mozilla) https://www.mozillafoundation.org/en/blog/new-research-tech-platforms-data-access-initiatives-vary-widely/Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we discuss the increasing prevalence of AI generated content online (aka 'AI slop'), on social media and even in books, journalism and academic journals. We explore how AI tools are enabling the generation of slop at scale, unleashing a flood of low quality content that is degrading our information ecosystem, making it harder to access high quality, verified information, and even getting in the way of our ability to connect with other humans online.Links:Max AI slop feature article (New York Magazine - paywalled) https://nymag.com/intelligencer/article/ai-generated-content-internet-online-slop-spam.htmlMax Read discussing his article in New York Magazine - linked above (Read Max newsletter - not paywalled) https://maxread.substack.com/p/were-in-our-slop-eraArticle on ecosystem behind AI slop content (The Conversation) https://theconversation.com/side-job-self-employed-high-paid-behind-the-ai-slop-flooding-tiktok-and-facebook-237638Article about Meta using AI to generate comments (TechCrunch) https://techcrunch.com/2025/03/21/meta-spotted-testing-ai-generated-comments-on-instagram/Article about Trump's use of slop (New Yorker)  https://www.newyorker.com/culture/critics-notebook/trump-is-the-emperor-of-ai-slopArticle about LinkedIn posts being AI slop (WIRED) https://www.wired.com/story/linkedin-ai-generated-influencers/Article: AI slop is killing the internet (Guardian) https://www.theguardian.com/global/commentisfree/2025/jan/08/ai-generated-slop-slowly-killing-internet-nobody-trying-to-stop-it404 Media coverage of AI slop (some paywalled) https://www.404media.co/tag/ai-slop/Article on the closure of Wordfreq (404 Media) https://www.404media.co/project-analyzing-human-language-usage-shuts-down-because-generative-ai-has-polluted-the-data/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we discuss the observable global shift in mood away from AI regulation and safety towards acceleration and innovation.It's a trend we observed several episodes ago, but appears to be gathering further speed with announcements by global political and business leaders of the urgency of re-doubling efforts to enhance AI capability. With Trump setting the tone at a federal level in the US, lawmakers have even gone as far as proposing a 10-year moratorium on ant state-based AI laws.We explore the implications of this mood shift for Australia, as a re-elected Albanese Government and its new Industry minister plot the way forward for AI in this country. Links:Article about 10-year moratorium (Tech Policy Press) https://www.techpolicy.press/us-house-committee-advances-10-year-moratorium-on-state-ai-regulation/Article about Trump rescinding Biden exec order (Cybersecurity dive) https://www.cybersecuritydive.com/news/trump-repeals-biden-ai-executive-order/738114/Article about OpenAI CEO reversing AI regulation position (The Information) https://www.theinformation.com/briefings/apparent-reversal-openai-ceo-advocates-less-ai-regulationArticle about tech execs pushing back on regulation (Seattle Times) https://www.seattletimes.com/business/microsofts-brad-smith-other-tech-execs-give-congress-ai-wish-list/Article about new Industry Minister Tim Ayres (InnovationAus) https://www.innovationaus.com/tim-ayres-named-industry-minister-in-new-look-cabinet/Article about need for Australia to pursue AI sovereignty (InnovationAus) https://www.innovationaus.com/owning-the-algorithm-australias-path-to-ai-sovereignty/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we reflect on one of the most visible and annoying features of the recently concluded federal election campaign - the incessant spam from political parties.We break down the various exemptions and carve-outs that make it legal for political parties to collect information about voters and spam us without consent.While the premise of the exemptions is to preserve the freedom of political communication in the name of democracy, we explore how the practice might be subverting our democratic ideals.Links:Article explaining how political parties can legally spam voters (ABC News) https://www.abc.net.au/news/2025-04-29/political-parties-can-send-you-texts/105226350Article breaking down Trumpet of Patriots spam campaign (SBS News) https://www.sbs.com.au/news/article/trumpet-of-patriots-texts/vsqj4l75xArticle about "H Fong" the man behind Trumpet of Patriots spam campaign (SBS News) https://www.theguardian.com/australia-news/2025/may/01/who-is-h-fong-harry-trumpet-of-patriots-sms-text-messages-federal-election-ntwnfbCrikey for PM campaign (Crikey - Paywalled) https://www.crikey.com.au/2025/04/28/spam-texts-election-clive-palmer-trumpet-of-patriots-monique-ryan-privacy-act-crikey-for-pm/Article about political parties' postal vote application texts (ABC News) https://www.abc.net.au/news/2025-03-31/aec-warns-of-unsolicited-postal-vote-texts/105116220Credits:Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

This week we're in conversation with Rys Farthing, who is Research Director at Reset.Tech Australia, a Not-for-profit that aims to develop evidence around digital issues so that policy makers can be more informed.We explore some of the challenges for effective digital policy making in Australia and around the world, and discuss our hopes and expectations for the Children's Online Privacy Code.If you're a regular listener, you'll know that Reset's work comes up somewhat regularly (both episode #120 about real time bidding and #93 about data brokers were based on research reports published by Reset, for example). We believe that there's a real need for the kind of work that Reset.Tech Australia does, to build an Australian evidence base around tech policy issues and solutions, and how they play out in the Australian context. Links:The OAIC's announcement about phasing consultation is here: https://www.oaic.gov.au/news/blog/sunshine-and-double-rainbows-building-a-better-online-environment-for-children-and-young-people The law that give rise to the Children's Online Privacy Code is here: https://www.aph.gov.au/Parliamentary_Business/Bills_LEGislation/Bills_Search_Results/Result?bId=r7249About grooming on Facebook. The leaked files that 75% of grooming came from Facebook's People You May Know (PYM) feature were reproduced in a US court, on page 12 here: https://socialmediavictims.org/wp-content/uploads/2022/06/Spence-Complaint-6_6_22.pdf Also Zoe Daniels asked if PYMK was still turned on for Australian kids in Sept 2024, and the answer was yes: https://www.theaustralian.com.au/business/media/metas-disregard-for-the-public-interest-is-galling-says-independent-mp-zoe-daniel/news-story/7783e5551f8669f665c7599d4e1dc47cThe document that notes that industry was aware of the issues of setting the bar of protection at 16 not 18. In response to a number of orgs raising this in their consultation process they responded: "in response to feedback the Code provisions concerning privacy settings on children's accounts have been amended to apply to children under 16." https://onlinesafety.org.au/wp-content/uploads/2022/11/221118_Submissions-log-responses_FINAL.pdfFinally, we also mention the Centre for automated decision making & society's Australian ad observatory project - a fantastic participatory research project providing much needed visibility into how ads are targeted to people in Australia: https://www.admscentre.org.au/australian-ad-observatory-investigating-mobile-and-dynamic-advertising-via-computational-and-participatory-approaches/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com