AI Education Podcast

This week we cover a range of AI in Education related news and academic research papers. Here's all the links you need to read more about each of the topics we dicussed: News Australia's Voluntary AI Safety Standard In September the Australian Government published: Voluntary AI Safety Standard (68pg): https://www.industry.gov.au/publications/voluntary-ai-safety-standard Proposals paper for introducing mandatory guardrails for AI in high-risk settings: https://consult.industry.gov.au/ai-mandatory-guardrails Stanford's STORM https://storm.genie.stanford.edu/ Students give English HSC exam an F over use of image with ‘hallmarks’ of AI https://www.theguardian.com/australia-news/2024/oct/16/nsw-hsc-english-exams-2024-ai-image-paper-1 Investigation into the use of ChatGPT by a Child Protection worker - Office of the Victorian Information Commissioner https://ovic.vic.gov.au/regulatory-action/investigation-into-the-use-of-chatgpt-by-a-child-protection-worker/   Research Papers To what extent is ChatGPT useful for language teacher lesson plan creation? https://arxiv.org/abs/2407.09974 Have We Reached AGI? Comparing ChatGPT, Claude, and Gemini to Human Literacy and Education Benchmarks https://arxiv.org/abs/2407.09573 The Life Cycle of Large Language Models: A Review of Biases in Education https://arxiv.org/abs/2407.11203 The Future of Learning: Large Language Models through the Lens of Students https://arxiv.org/abs/2407.12723 How to Mitigate the Dependencies of ChatGPT-4o in Engineering Education https://arxiv.org/abs/2407.12693 StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions https://arxiv.org/abs/2407.12423 Apostles, Agnostics and Atheists: Engagement with Generative AI by Australian University Staff https://eprints.qut.edu.au/252079/  

Welcome to the first Episode of Series 10, where we are going to take some time to review some of the fundamental changes we're going to be seeing as AI becomes more pervasive.  In this Episode we're taking a different approach, with Dan and Ray using Google's NotebookLM to create AI-delivered mini-podcasts about each other, which leads to a discussion about the backgrounds of both hosts, and a discussion about how this new AI might be used in education. Obviously, both Dan and Ray conclude that AI won't replace them...yet! For your own experiments, you can find NotebookLM at https://notebooklm.google.com/ The two mini podcasts in this episode were produced by simply feeding NoteBookLM with the hosts LinkedIn profiles (in PDF format - available as a download from any individual's LinkedIn page). We'd love to hear how you're using NotebookLM, and other AI tech, in your work. The easiest way to get hold of us is via our LinkedIn page https://www.linkedin.com/company/ai-in-education-podcast/ And if you're enjoying the podcast, would now be a good time to leave a review in your podcast app - whether you're using Apple Podcasts, Google Podcasts or Spotify, we'd love it if you left a rating and review.

This week we're joined by the brains behind South Australia's EdChat chatbot. It was the first state in Australia to start pilot a custom built generative AI chatbot for their schools, and we spent some time with the team to understand the success story. In the podcast, we heard from Simon Chapman (Director of Digital Architecture and Operations) and Cody Little (AI Technical specialist), both from the Department for Education South Australia Find out more about the South Australia Department AI approach: Artificial intelligence (AI) in schools – information for parents and carers (education.sa.gov.au) and details on their trial - Nation-leading trial in SA schools to focus on the safe use of AI (education.sa.gov.au) Microsoft have written a case study on their News Centre here: Learning in the AI era: How South Australia’s Department for Education is empowering students and teachers with AI in the classroom - Microsoft Australia News Centre Video of their amazing work: Department for Education South Australia | Reimagine Education 2024 9 News report : Artificial intelligence set to shake-up South Australian high schools (9news.com.au) AI Safety - AI content safety Towards the end of the episode we also talked about Career chat: SA Career Chat

Unbelievably, it's our 5th Anniversary (AI-nniversary?) and to mark the milestone of an AI podcast that's had more pivots than a robot's legs, and more chat than a chatbot, we've pulled together some of the top episodes from the 108 episodes of the podcast so far, and got the band back together (regretfully, we were unable to persuade Beth Worral, who co-hosted in 2022-2023, to come back for our party podcast - she'd reached escape velocity), so you'll hear Dan, Ray and Lee reminiscing like old men in a shed. Great Guests & Top Episodes Our very first guest - Microsoft's Troy Waller in AI for Accessibility in 2019 Our most listened episode from 2019 and very first external guest - the brilliant Dr David Kellerman in Using AI to personalise learning at UNSW Sydney Our top 2020 episode was A Brief History of AI, which looked backin to the human obsession with creating artificial replicas of themselves. When did it start, who started it, what is the AI winter and how did it accelerate so quickly in the last 10 years. (And even faster in the last 4 years since?) In 2021, another great guest topped our episode charts, with Feel the force: Emerge to the future with Sly Lee as Sly talked about his work around VR and AR and the mergeing of physical and virtual worlds For 2022, it was the Christmas edition that hit the heights, with all four co-hosts - Beth, Lee, Dan & Ray - together for the quizzically named Christmas, Infinite Monkeys and everything By 2023, we were back on solid pedagogy again, with the episode AI - The fuel that drives insight in K12 with Travis Smith (the kind of title a marketing manager would love?) And, although we've not yet reached the end of 2024 and we've got some great episodes ahead, the top episode this far is University of Sydney and the future of assessment with Danny Liu and Adam Bridgman We also had fond memories of: The AI in Popular Culture episode from 2020 Martin O'Sullivan's tales from Turing House School in From AI Detectors to Heartwarming Help: Two Tales of AI in a school Elle Graham (aka Woodes) in Music, Creativity and Minecraft with Woodes Practical lessons from chatbot land with The Search for the Education Holy Grail in NSW, with the stories shared by Dan Hart and Michelle Michael 

News This week's news we rushed through includes "Study Buddy or Influencer?", the Australian Parliamentary Inquiry into the use of Gen AI in the Australia Education System  https://www.aph.gov.au/Parliamentary_Business/Committees/House/Employment_Education_and_Training/AIineducation/Report NSW EduChat has been extended for all teachers in all New South Wales Schools - https://www.innovationaus.com/nsw-teachers-to-get-time-saving-genai-tool/ Learn more about the NSW EduChat chatbot in the first episode of Series 9 Microsoft announced Copilot Wave 2 https://www.microsoft.com/en-us/microsoft-365/blog/2024/09/16/microsoft-365-copilot-wave-2-pages-python-in-excel-and-agents/ OpenAI's new model o1-preview came out - and it's much, much better and logical reasoning and maths Find out what people think by looking at what Ethan Mollick is saying about it (and, as I suggest on the podcast, keenly watch Dan Hart's posts as he's benchmarking it now in the context of EduChat) Google released a podcast-generator as part of NotebookLM, which turns any kind of paper/book etc into a short podcast. To find out what it's like I gave it the top research paper from the podcast last time and you can compare it versus me! There's a quick demo review by Ethan here and you can login and have a play directly here: https://notebooklm.google.com/ (it has two main uses, because on top of this autopodcast, it's original purpose was to allow you give it a pile of documents and it can work with all of them at the same time for q&a, summaries etc)   Research Here's the links to all nine papers we discussed in this week's episode: Generative AI in Real-World Workplaces https://www.microsoft.com/en-us/research/uploads/prodnew/2024/07/Generative-AI-in-Real-World-Workplaces.pdf  Can Large Language Models Make the Grade? https://dl.acm.org/doi/10.1145/3657604.3664693 PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection https://arxiv.org/abs/2406.16288  The global landscape of academic guidelines for generative AI and Large Language Models https://arxiv.org/abs/2406.18842  "Is ChatGPT a Better Explainer than My Professor?": Evaluating the Explanation Capabilities of LLMs in Conversation Compared to a Human Baseline  https://arxiv.org/abs/2406.18512 Perceived Impact of Generative AI on Assessments: Comparing Educator and Student Perspectives in Australia, Cyprus, and the United States https://www.sciencedirect.com/science/article/pii/S2666920X24000729 Jill Watson: Scaling and Deploying an AI Conversational Agent in Online Classrooms https://dilab.gatech.edu/publications/jill-watson-scaling-and-deploying-an-ai-conversational-agent-in-online-classrooms/  https://dilab.gatech.edu/test/wp-content/uploads/2024/05/ITS2024_JillWatson_paper.pdf Integrating AI in College Education: Positive yet Mixed Experiences with ChatGPT https://arxiv.org/abs/2407.05810 The AI Companion in Education: Analyzing the Pedagogical Potential of ChatGPT in Computer Science and Engineering https://arxiv.org/abs/2407.05205    

This week's guest is Professor Phillip Dawson, who is Co-Director of the Centre for Research in Assessment and Digital Learning at Deakin University in Australia. In addition to Phill's website, we recommend following Phill on LinkedIn, or Twitter, where shares a lot of his work on the future of assessment, and also his passion project - The Peer Revue - where Phill combines his passion for academic research with his work in Improv Comedy. Every month he hosts a research who talks about their professional research, which is then turned into comedy gold by his improv team at The Improv Conspirancy Theatre (highly recommend keeping an eye out for this if you're in Melbourne) You can find Phill's research papers here on Google Scholar, and his LinkedIn feed has his books and his contributions to other advice and consultations in the education sector Phil mentions a number of researchers and their work in the podcast. Here's the links: James Reason's work on the Swiss Cheese model for failure of complex systems "The contribution of latent human failures to the breakdown of complex systems" You can either read the original research paper or this easier to grasp Wikipedia article "The Swiss Cheese Mode" Kiata Rundle's work on then applying this to academic integrity - you can find all of her papers on Google Scholar Alfie Kohn "Punished by Rewards" - https://www.mv.helsinki.fi/home/hotulain/Punished.pdf Phill mentioned Deci & Ryan's work, so here's a good place to start reading on Self-Determination Theory And here's a starting point if you need it for reading about Vygotsky's Zone of Proximal Development Then Phill mentioned the book Thanks for the Feedback, but Stone & Heen, which is on Amazon here  

Series 9, Episode 9 - Homework and Sex   News How do people actually use ChatGPT? How do people use ChatGPT? We analyzed real AI chatbot conversations - The Washington Post   Make AI tools to reduce teacher workloads, tech companies urged https://www.theguardian.com/education/article/2024/aug/28/make-ai-tools-to-reduce-teacher-workloads-tech-companies-urged     New AI in Teams - and it's all free https://www.linkedin.com/posts/miketholfsen_ai-edtech-microsoftteams-activity-7236817645625335809-08OC https://aka.ms/TeamsEDUAIQuickGuide   Research Papers   Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course https://arxiv.org/abs/2407.05216   Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms https://arxiv.org/abs/2406.07571    Evaluating ChatGPT-4 Vision on Brazil's National Undergraduate Computer Science Exam https://arxiv.org/abs/2406.09671    Generative AI Can Harm Learning https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4895486   ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering https://www.tandfonline.com/doi/epdf/10.1080/22054952.2024.2372154   How critically can an AI think? A framework for evaluating the quality of thinking of generative artificial intelligence https://arxiv.org/abs/2406.14769   Analyzing Large Language Models for Classroom Discussion Assessment https://arxiv.org/abs/2406.08680    Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics https://arxiv.org/abs/2406.11858    ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictor https://arxiv.org/abs/2406.14765    70B-parameter large language models in Japanese medical question-answering https://arxiv.org/abs/2406.14882    I don't trust you (anymore)! -- The effect of students' LLM use on Lecturer-Student-Trust in Higher Education https://arxiv.org/abs/2406.14871    Large Language Models in Student Assessment: Comparing ChatGPT and Human Graders https://arxiv.org/abs/2406.16510 

This week we hear from Professor Rose Luckin, from University College London and Educate Ventures Research  In the podcast, Rose mentioned the 1% project in Finland from 2020 - here's some reporting on it  Through Educate Ventures Research there are a range of AI consultancy and training services for schools, including the AI Readiness Online Course for teachers Rose also has a monthy newsletter "The Skinny on AI for Education" which has an extensive reading list every edition, on a number of AI topics, not just what's happening in education.

News   TEQSA's new paper on Academic Integrity & AI The evolving risk to academic integrity posed by generative artificial intelligence: Options for immediate action https://www.teqsa.gov.au/sites/default/files/2024-08/evolving-risk-to-academic-integrity-posed-by-generative-artificial-intelligence.pdf   State of Generative AI in the Enterprise: An Australian Perspective https://www.deloitte.com/au/en/services/consulting/analysis/state-generative-ai-enterprise.html     Research   A review on the use of large language models as virtual tutors https://arxiv.org/abs/2405.11983   Jill Watson: A Virtual Teaching Assistant powered by ChatGPT https://arxiv.org/abs/2405.11070   An empirical study to understand how students use ChatGPT for writing essays and how it affects their ownership https://arxiv.org/abs/2405.13890   Intelligent Tutor: Leveraging ChatGPT and Microsoft Copilot Studio to Deliver a Generative AI Student Support and Feedback System within Teams https://arxiv.org/abs/2405.13024   Large Language Models as Partners in Student Essay Evaluation https://arxiv.org/abs/2405.18632   Grade Like a Human: Rethinking Automated Assessment with Large Language Models https://arxiv.org/abs/2405.19694    Designing Prompt Analytics Dashboards to Analyze Student-ChatGPT Interactions in EFL Writing https://arxiv.org/abs/2405.19691    Experiences from Integrating Large Language Model Chatbots into the Classroom https://arxiv.org/abs/2406.04817    Delving into ChatGPT usage in academic writing through excess vocabulary https://arxiv.org/abs/2406.07016    Understanding Students' Acceptance of ChatGPT as a Translation Tool: A UTAUT Model Analysis https://arxiv.org/abs/2406.06254 

BONUS EPISODE ! We'd love to meet more of our listeners in person, so here's a list of events over the next few weeks when Dan, Ray, or Dan and Ray are speaking about AI in Education. One of the things that makes the podcast special is the amazing stories we get from our guests, and there's more stories than fit into an episode, and we're always on the lookout for more. So come and say hi at any of these events, and per   haps we can share an untold story, or you can tell us yours! August  19 August - Melbourne - Ray Melbourne EdTech Summit Ray's on the panel discussing "The Transformed Learning Landscape Through AI" at this event for edtech companies and universities 21 August - Sydney - Ray - Free event The Future of Human AI - IATD If you're in Sydney, the Institute of Applied Technology has scheduled the perfect event on Wednesday evening (starting at 6PM). Ray's hosting the free learning session "The Future of Human AI", designed for the thousands of small businesses and employees around Western Sydney who want to make sense of the AI hype and start to understand what everybody else is talking about. If you're a teacher, you'll also walk away with some good ideas of how businesses are using AI to help with your discussion of AI in the classroom   23 August - Sydney - Dan AI in Education Conference | Teaching Tomorrow: Harnessing AI Tools Today Organised by friends of the podcast Matt Esterman and Nick Jackson at WSU's Parramatta campus, it'll be a festival of ideas from a great lineup of teachers. Dan's one of the speakers, and will be wearing his Microsoft badge on the day, talking about all the latest great tech announcements from that world.   September 3 September - Online - Dan - Free event Build a Bot in Copilot Studio Doing his day job, Dan's taking part in this online workshop designed to help Microsoft Copilot users build their own bot. Like Build a Bear workshop, but with less bear and more bot. Open to any Microsoft customers in Australia and New Zealand.   4 September - Christchurch, NZ - Ray NZ Tertiary ICT Conference For New Zealand listeners Ray's heading over the ditch to deliver a keynote called "AI, why?" at the annual conference for digital teams of universities and polytechnics. If you're going to be there, please say Hi!   12-13 September - Adelaide - Ray HE FEST 24 Ray's taking part in the "Marketing, Recruitment, Advancement and AI in Higher Education Conference" in Adelaide. Ray's on a panel discussion "The place of AI in university " with Eddie Major, on applications of AI beyond teaching and learning, and then delivering the closing keynote, grandly titled "The future of HE - The future of higher education in a world where gen AI is ubiquitous"   16-19 September - Online - Dan and Ray - Free event Toddle's AI for AussieEd online event  The toddle team have pulled together 20 speakers from across Australia, who will be talking about assessment, LMSs, chatbots, the AI Framework for schools, leadership and data-driven learning. And tacked on right at the end of the run, at 5:15 on the 19th, we're going to have a crack at a live podcast recording