This Week In Digital Trust

AI chatbots are becoming increasingly intimate spaces, with people using them for companionship, therapy and as a helpful personal assistant.While our interactions with chatbots like ChatGPT and Copilot get more personal, we're also seeing logs and transcripts of these interactions increasingly find their way to the public - via FOI requests, court orders and security and privacy breaches.We explore the convergence of these opposing trends. Links:Article about national security chief using Copilot (Crikey - PAYWALL) https://www.crikey.com.au/2025/11/12/australia-national-security-chief-ai-speech-writing/Article about UK tech minister using ChatGPT for policy advice (New Scientist) https://www.newscientist.com/article/2472068-revealed-how-the-uk-tech-secretary-uses-chatgpt-for-po…Article about California court order ChatGPT logs discoverable (AI Buzz) https://www.ai-buzz.com/court-rules-chatgpt-history-is-discoverable-evidence-in-lawsuitsArticle about Grok chats exposed in Google search results (BBC) https://www.bbc.com/news/articles/cdrkmk00jy0oArticle about Meta leaking chatbot prompts (Toms Guide) https://www.tomsguide.com/computing/online-security/meta-ai-was-leaking-chatbot-prompts-and-answers…OpenAI v New York Times https://openai.com/index/fighting-nyt-user-privacy-invasion/NSW Government advisory on AI and recordkeeping https://www.nsw.gov.au/nsw-government/recordkeeping/create-and-capture/ai-and-recordkeeping Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we analyse the implications of the first ever civil penalties ordered under the Privacy Act.In October, the Federal Court ordered that Australian Clinical Labs pay $5.8 million in civil penalties in relation to a data breach by its Medlab Pathology business in February 2022 that led tothe exposure of the personal information of over 223,000 people.We discuss the key insights for cyber, privacy and risk practitioners emerging from the judgement. Links:OAIC statement https://www.oaic.gov.au/news/media-centre/australian-clinical-labs-ordered-to-pay-penalties-in-relation-to-medlab-pathology-data-breach-in-first-for-privacy-actFederal Court judgement https://www.judgments.fedcourt.gov.au/judgments/Judgments/fca/single/2025/2025fca1224OAIC guidance on APP 11 "reasonable steps" (OAIC) https://www.oaic.gov.au/privacy/australian-privacy-principles/australian-privacy-principles-guidelines/chapter-11-app-11-security-of-personal-informationCredits:Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

This week Arj is joined by elevenM colleague Brett Watson to breakdown the issues surrounding the rollout of facial recognition technology (FRT) across retail settings, stadiums and other venues in Australia.The Privacy Commissioner recently ruled that the use of FRT by Australian retailers Bunnings and Kmart breached Australians' privacy. Bunnings is appealing the ruling, while Kmart is also considering an appeal - both arguing that their use of FRT is a proportionate response to issues like violence and theft in their stores.With that as the backdrop, Arj and Brett debate the merits of privacy advocates' arguments against FRT. Links:OAIC ruling against Bunnings https://www.oaic.gov.au/news/media-centre/bunnings-breached-australians-privacy-with-facial-recogni…OAIC ruling against Kmart https://www.oaic.gov.au/news/media-centre/18-kmarts-use-of-facial-recognition-to-tackle-refund-frau…Article about OAIC ruling against Kmart (ABC News) https://www.abc.net.au/news/2025-09-18/kmart-facial-recognition-technology-privacy-commissioner/105…Article about Bunnings arguments in favour of FRT (AFR - paywall) https://www.afr.com/technology/bunnings-boss-wants-new-laws-to-allow-facial-recognition-in-stores-2…Article about Wesfarmers citing rise in retail theft (Skynews) https://www.skynews.com.au/business/finance/wesfarmers-boss-rob-scott-laments-uptick-in-retail-thef…NZ Privacy Commissioner inquiry into Foodstuffs use of FRT https://www.privacy.org.nz/focus-areas/frt-inquiry-report/Article about woman misidentified by Foodstuffs FRT (RNZ) https://www.rnz.co.nz/news/national/535871/woman-wrongly-kicked-out-over-foodstuffs-facial-recognit… Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we unpack the concept of sovereign AI, which is increasingly shaping both Australian and global conversations about national AI approaches.We explore what sovereign AI means, the problem it seeks to solve, and the variety of options being proposed. We also debate how feasible it is for Australia to can achieve a truly sovereign AI capability.Links:Article urging Australia to build sovereign AI (InnovationAus) https://www.innovationaus.com/build-sovereign-ai-now-or-face-lock-in-australia-warned/Article about Australian startups building sovereign AI (AFR) https://www.afr.com/technology/we-can-do-it-for-under-100m-start-up-joins-race-to-build-local-chatgpt-20250908-p5mt5oArticle about Productivity Commission's views on sovereign AI (AFR) https://www.innovationaus.com/productivity-tsar-hoses-down-sovereign-ai-support-hopes/Oped #1 about Australia's limitations in achieving sovereign AI (AFR) https://www.afr.com/technology/australia-is-seeking-its-place-in-the-ai-world-order-20250910-p5mu38?utm_source=substack&utm_medium=emailOped #2 about Australia's limitations in achieving sovereign AI (AFR) https://www.afr.com/policy/economy/labor-must-resist-the-temptation-to-build-sovereign-ai-20250903-p5ms97Oped by Vic senator Michelle Ananda-Rajah (AFR)https://www.afr.com/technology/our-ai-future-is-being-built-overseas-we-can-t-afford-that-20250813-p5mmlqBlog about South-East Asia's approach to sovereign AI (ASPI) https://aspicts.substack.com/p/the-monthly-roundup-angela-suriyaseneeCredits:Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

This week we discuss the growing use of workplace surveillance technologies, triggered by a legal case involving an Australian compliance training company.The case highlights the use of monitoring software that records an employee's screen activity and even their conversations detected by laptop microphones.With employees increasingly working from home, the case speaks to broader implications for worker privacy and employee-employer trust relationships. Links:Article about Safetrac case (AFR) https://www.afr.com/work-and-careers/workplace/company-turned-laptops-into-covert-recording-devices-to-monitor-wfh-20250822-p5mp0zArticle about the trend of workplace surveillance (Guardian) https://www.theguardian.com/technology/2022/apr/27/remote-work-software-home-surveillance-computer-monitoring-pandemicOAIC summary of workplace monitoring and surveillance laws https://www.oaic.gov.au/privacy/your-privacy-rights/surveillance-and-monitoring/workplace-monitoring-and-surveillanceVic Gov inquiry into workplace surveillance https://www.parliament.vic.gov.au/49a1f1/contentassets/5232a88edcc34487aa0223bcbb2c8974/laeic-60-02-inquiry-into-workplace-surveillance2.pdfCredits: Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

**CONTENT WARNING** This episode contains discussion of suicide and mental health themes. Please contact Lifeline on 13 11 14 or Beyond Blue on 1300 224 636. This week we discuss the rise of AI companions.In the midst of a continued AI acceleration push in Australia, we discuss the emergence of several disturbing stories in which AI chatbots have apparently lead young people and adults alike to self-harm.In the face of known harms, many of these AI chatbots continue to be designed primarily to increase engagement and - as some have argued - addiction and dependency. Links:Article about AI companions impact on young Australians (ABC News) https://www.abc.net.au/news/2025-08-12/how-young-australians-being-impacted-by-ai/105630108Podcast including discussion of case of Adam Raine https://podcasts.apple.com/us/podcast/how-openais-chatgpt-guided-a-teen-to-his-death/id1460030305?i…Discussion about case of Sewel Setzer III (Tech Policy Press) https://www.techpolicy.press/ai-companions-and-the-law/Research into impacts of AI companions (Tech Policy Press) https://www.techpolicy.press/new-research-sheds-light-on-ai-companions/Report on AI companion market size https://www.businessresearchinsights.com/market-reports/ai-companion-market-117494?ref=theoverview.…Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we unpack the national conversation about AI's role in solving Australia's productivity challenges.The Productivity Commission recently published a report arguing for a cautious approach to AI regulation, on the basis that it could prevent Australia from "unlocking a surge of productivity".Business and technology advocates have also doubled down, using the moment to call for various legal exemptions and a more laissez-faire approach to restrictions on companies building and deploying AI models.We explore these ideas and the specific implications for areas like privacy and copyright.Links:Productivity Commission report https://www.pc.gov.au/inquiries/current/data-digital/interimArticle about Productivity summit (ABC News) https://www.abc.net.au/news/2025-08-07/artificial-intelligence-jim-chalmers-economics-reform-roundtable/105618958Report on AI benefits (METR) https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/Article on how most AI experimentation is wasted effort (HBR) https://hbr.org/2025/08/beware-the-ai-experimentation-trapArticle about CBA call center AI bungle and stats on unsuccessful AI adoption (SMH) https://www.smh.com.au/business/banking-and-finance/bring-back-the-humans-cba-s-embarrassing-ai-jobs-bungle-a-salutary-lesson-20250821-p5moqk.htmlPrivacy Commissioner's response to Productivity Commission report https://www.oaic.gov.au/news/blog/getting-ai-right-benefits-businesses,-productivity-and-the-communityArticle about artists response to Productivity Commission report  (ABC News) https://www.abc.net.au/news/2025-08-13/productivity-commission-ai-report-copyright-law-authors-respond/105646086Interview with Scott Farquhar (ABC) https://www.abc.net.au/news/2025-08-12/could-australia-benefit-from-the-revolution-in-ai/105645406 Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we take a deeper look at recent high-profile cyber attacks, which have breached organisations like Qantas and Marks & Spencer.For all the talk of technical sophistication, especially in the time of AI, hackers like Scattered Spider are relying less on fancy code and more on their ability to exploit gaps in our systems of trust.We explore how the biggest vulnerabilities in our systems are sometimes found not in software flaws but in how we vouch for ourselves online, and the way we govern relationships between companies and their suppliers. Links:Article about Scattered Spider (Axios) https://www.axios.com/newsletters/axios-future-of-cybersecurity-fab0c0d0-5760-11f0-8c09-e72f7ef7d797?utm_source=chatgpt.comArticle about the psychological edge behind Qantas hack (AFR) https://www.afr.com/companies/transport/qantas-says-6-million-aussies-caught-up-in-cyberattack-20250702-p5mbupAustralian Government advisory on Scattered Spider https://www.cyber.gov.au/about-us/view-all-content/alerts-and-advisories/scattered-spider?utm_source=linkedin&utm_campaign=scatter-spider&utm_medium=social&utm_content=advisoryDarknet Diaries episode about profit driven hacking communities ⁠https://darknetdiaries.com/episode/112⁠Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com

This week we explore why so many organisations fall short in how they respond publicly to a cyber incident or data breach.Even for organisations that have a well-exercised "crisis communications" muscle, too often a data breach results in a response that infuriates and erodes trust in equal measure.We explore why standard crisis comms principles don't always apply in a data breach, and what good looks like.Links:Article with analysis of Qantas comms response (Mumbrella) https://mumbrella.com.au/just-trust-us-were-qantas-880124Article with analysis of Qantas comms response (Security Brief) https://securitybrief.com.au/story/experts-give-qantas-mixed-reviews-on-cyber-breach-responseOped from James Turner on Qantas data breach (AFR) https://www.afr.com/technology/why-the-qantas-hack-should-send-chills-around-corporate-australia-20250710-p5me2rArticle about Qantas data breach (Guardian) https://www.theguardian.com/business/2025/jul/02/qantas-confirms-cyber-attack-exposes-records-of-up-to-6-million-customersArticle about PageUp data breach (Guardian) https://www.theguardian.com/technology/2018/jun/07/thousands-of-job-seekers-details-potentially-exposed-in-hackelevenM blog on cyber crisis communications (elevenM) https://elevenm.com.au/blog/the-four-pitfalls-of-cyber-crisis-communications/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) ⁠www.eastcoaststudio.com.au⁠Music by Bensound.com

This week we revisit the conversation around banning under 16s from social media, and the range of technologies being considered to verify the ages of would-be social media users.We examine the preliminary findings of a government-commissioned trial of age assurance technologies, as well as some independent media reporting on technologies in the trial.The preliminary results have received mixed reviews, with particular concerns over the accuracy of facial recognition-based age estimation. Links:Preliminary findings of age assurance trial https://ageassurance.com.au/wp-content/uploads/2025/06/News-Release-Preliminary-Findings-for-publication-20250620.pdfArticle about preliminary findings of age assurance trial (ABC News) https://www.abc.net.au/news/2025-06-20/age-assurance-trial-preliminary-report/105440288Article highlighting issues with facial age estimation technologies (ABC News) https://www.abc.net.au/news/2025-06-19/teen-social-media-ban-technology-concerns/105430458Article about expert resigning from assurance trial advisory board (Crikey) https://www.crikey.com.au/2025/06/23/expert-resigns-teen-social-media-ban-tech-trial-advisory-board/TWIDT #79 The long and winding road to age verification https://elevenm.com.au/podcast/episode/the-long-and-winding-road-to-age-verification/TWIDT #122 Big ban theory – Why Australia’s social media ban for kids is raising eyebrows https://elevenm.com.au/podcast/episode/122-big-ban-theory-why-australias-social-media-ban-for-kids-is-raising-eyebrows/ Credits:Editing and post-production by Martin Franklin (East Coast Studio) www.eastcoaststudio.com.auMusic by Bensound.com