This podcast discusses the latest research in AI education, including topics like remote proctoring debates, chatbot support for international students, using chat GPT in veterinary medicine, and exploring the impact of generative AI on education. It also covers computing students' attitudes towards large language models, ethical considerations of AI in education, and the benefits of AI in enhancing creative learning in children.

This week we continue our series on Assessment and AI. Ray talks with Jason Lodge from The University of Queensland, and who must have the longest business card in Australia, as he's Associate Professor of Educational Psychology in School of Education and Deputy Associate Dean in the Faculty of Humanities, Arts and Social Sciences!   The conversation talks about the challenges of assessment, and the options for rethinking assessment - and then we go deeper into Jason's views on the future of learning and assessment.   Jason's a great guest to share his experiences, as during 2023 he was on the TEQSA group of experts that came together to produce a report on assessment for Australian universities, Assessment reform for the age of artificial intelligence https://www.teqsa.gov.au/guides-resources/resources/corporate-publications/assessment-reform-age-artificial-intelligence   Working on policy and guidance in an area where technology is developing so rapidly - and students are racing ahead of institutions, was interesting and Jason talks about the group dynamic. One of the interesting notes he talks about is the mindset: "The mantra we kept returning to is that we weren't trying to develop a map, but a compass. This is the direction we think we might need to head here."

AI and the Future of Assessment: Transforming Educational Practices Episode Overview: In this episode of the AI Education Podcast, hosts Dan and Ray, alongside guests Adam Bridgman and Danny Liu, dive into the evolving landscape of academic assessment in the age of artificial intelligence. Recorded in the University of Sydney's own studios, this discussion explores the significant shifts in assessment strategies and the integration of AI in educational settings. Guest Introductions: Professor Adam Bridgeman: Pro Vice Chancellor Educational Innovation at the University of Sydney - focused on enhancing teaching quality across the university. [University bio] Professor Danny Liu: Professor of Educational Technologies - dedicated to empowering educators to improve their teaching methods through innovative technologies. [University page - LinkedIn page] Key Topics Discussed: The Persistence of Traditional Assessment Models: Despite the push to digital platforms during the COVID-19 pandemic, traditional assessment methods have largely remained unchanged, continuing the practice of replicating physical exam environments online. AI's Role in Rethinking Assessment: The guests discuss how AI challenges the conventional reasons for assessments, advocating for a paradigm shift towards assessments that truly measure student understanding and application of knowledge. Two-Lane Assessment Approach: Adam introduces a dual-lane strategy for assessment: Lane One: Ensures the rigorous verification of student competencies necessary in professional fields. Lane Two: Uses AI to foster skill development in using technology effectively, moving beyond traditional assessment forms to embrace innovative educational practices. Implementation Challenges and Solutions: The transition to new assessment models is recognised as a gradual process, needing careful planning and support for educators in rethinking their assessment strategies. Inclusivity and Access to Technology: Ensuring equitable access to AI tools for all students is highlighted as a critical aspect of the evolving educational landscape, emphasizing the need to support diverse student backgrounds and technological proficiencies. Future Outlook: The discussion concludes with reflections on the potential long-term impacts of AI on educational practices, the necessity of ongoing adaptation by educational institutions, and the importance of preparing students for a future where AI is seamlessly integrated into professional and everyday contexts. Further Reading: We recommend these three articles from the team, that give more detail on the topics discussed Where are we with generative AI as semester 1 starts? What to do about assessments if we can’t out-design or out-run AI? Embracing the future of assessment at the University of Sydney

It's time to start a new series, so welcome to Series 8! This episode is the warm up into the series that's going to be focused on Assessment. We'll interview some fascinating people about what's happening in school and university assessment, how we might think differently about assessing students, and what you can be thinking about if you're a teacher. There's no shownotes, links or anything else for your homework for this episode - just listen and enjoy! Dan and Ray

The season-ending episode for Series 7, this is the fifteenth in the series that started on 1st November last year with the "Regeneration: Human Centred Educational AI" episode. And it's an unbelievable 87th episode for the podcast (which started in September 2019). When we come back with Series 8 after a short break for Easter, we're going to take a deeper dive into two specific use cases for AI in Education. The first we'll discuss is Assessment, where there's both a threat and opportunity created by AI. And the second topic is AI Tutors, where there's more of a focus on how we can take advantage of the technology to help improve support for learning for students. This episode looks at one key news announcement - the EU AI Act - and a dozen new research papers on AI in education. News EU AI Act https://www.europarl.europa.eu/news/en/press-room/20240308IPR19015/artificial-intelligence-act-meps-adopt-landmark-law The European Parliament approved the AI Act on 13 March and there's some stuff in here that would make good practice guidance. And if you're developing AI solutions for education, and there's a chance that one of your customers or users might be in the EU, then you're going to need to follow these laws (just like GDPR is an EU law, but effectively applies globally if you're actively offering a service to EU residents). The Act bans some uses of AI that threaten citizen's rights - such as social scoring and biometric identification at mass level (things like untargeted facial scanning of CCTV or internet content, emotion recognition in the workplace or schools, and AI built to manipulate human behaviour) - and for the rest it relies on regulation according to categories.  High Risk AI systems have to be assessed before being deployed and throughout their lifecycle. In the High Risk AI category it includes critical infrastructure (like transport and energy), product safety, law enforcement, justice and democratic processes, employment decision making - and Education. So decision making using AI in education needs to do full risk assessments, maintain usage logs, be transparent and accurate - and ensure human oversight. Examples of decision making that would be covered would be things like exam scoring, student recruitment screening, or behaviour management. General generative AI - like chatgpt or co-pilots - will not be classified as high risk, but they'll still have obligations under the Act to do things like clear labelling for AI generated image, audio and video content ; make sure there's it can't generate illegal content, and also disclose what copyright data was used for training. But, although general AI may not be classified as high risk, if you then use that to build a high risk system - like an automated exam marker for end-of-school exams, then this will be covered under the high risk category. All of this is likely to become law by the middle of the year, and by the end of 2024 prohibited AI systems will be banned - and by mid-2025 the rules will start applying for other AI systems. ResearchAnother huge month. I spent the weekend reviewing a list of 350 new papers published in the first two weeks of March, on Large Language Models, ChatGPT etc, to find the ones that are really interesting for the podcast Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges arXiv:2401.08664   A Study on Large Language Models' Limitations in Multiple-Choice Question Answering arXiv:2401.07955   Dissecting Bias of ChatGPT in College Major Recommendations arXiv:2401.11699   Evaluating Large Language Models in Analysing Classroom Dialogue arXiv:2402.02380    The Future of AI in Education: 13 Things We Can Do to Minimize the Damage https://osf.io/preprints/edarxiv/372vr   Scaling the Authoring of AutoTutors with Large Language Models https://arxiv.org/abs/2402.09216   Role-Playing Simulation Games using ChatGPT https://arxiv.org/abs/2402.09161   Economic and Financial Learning with Artificial Intelligence: A Mixed-Methods Study on ChatGPT https://arxiv.org/abs/2402.15278   A Study on the Vulnerability of Test Questions against ChatGPT-based Cheating https://arxiv.org/abs/2402.14881   Incorporating Artificial Intelligence Into Athletic Training Education: Developing Case-Based Scenarios Using ChatGPT https://meridian.allenpress.com/atej/article/19/1/42/498456   Incorporating Artificial Intelligence Into Athletic Training Education: Developing Case-Based Scenarios Using ChatGPT https://meridian.allenpress.com/atej/article/19/1/42/498456   RECIPE4U: Student-ChatGPT Interaction Dataset in EFL Writing Education https://arxiv.org/abs/2403.08272   Comparison of the problem-solving performance of ChatGPT-3.5, ChatGPT-4, Bing Chat, and Bard for the Korean emergency medicine board examination question bank https://journals.lww.com/md-journal/fulltext/2024/03010/comparison_of_the_problem_solving_performance_of.48.aspx?context=latestarticles   Comparing the quality of human and ChatGPT feedback of students’ writing https://www.sciencedirect.com/science/article/pii/S0959475224000215          

This week we talked with Professor Danny Liu and Dr Joanne Hinitt, of The University of Sydney, about the Cogniti AI service that's been created in the university, and how it's being used to support teaching and learning. Danny is a molecular biologist by training, programmer by night, researcher and academic developer by day, and educator at heart. He works at the confluence of educational technology, student engagement, artificial intelligence, learning analytics, pedagogical research, organisational leadership, and professional development. He is currently a Professor in the Educational Innovation team in the DVC (Education) Portfolio at the University of Sydney. Here's Danny's academic profile. If you want to follow Danny's future work you can find him on LinkedIn and Twitter Joanne is a Lecturer in Occupational Therapy, and her primary area of interest is working with children and their families who experience difficulties participating in occupations related to going to school. She has extensive clinical experience working within occupational therapy settings, providing services for children and their families. Her particular interest is working collaboratively with teachers in the school setting and she completed her PhD in this area. Here's Joanne's academic profile Further reading on the topics discussed in the podcast Cogniti's website is at https://cogniti.ai/ Articles about the topics discussed: How Sydney educators are building ‘AI doubles’ of themselves to help their students, Dec 2023 AI as an authentic and engaging teaching tool for occupational therapy students, Oct 2023 Meet ‘Mrs S’: a classroom teacher who helps budding occupational therapists hone their skills, Oct 2023 Recorded talks Using Cogniti to design for Diversity, Feb 2023      

It's a News and Research Episode this week    There has been a lot of AI news and AI research that's related to education since our last Rapid Rundown, so we've had to be honest and drop 'rapid' from the title! Despite talking fast, this episode still clocked in just over 40 minutes, and we really can't out what to do - should we talk less, cover less news and research, or just stop worrying about time, and focus instead on making sure we bring you the key things every episode?     News More than half of UK undergraduates say they use AI to help with essays https://www.theguardian.com/technology/2024/feb/01/more-than-half-uk-undergraduates-ai-essays-artificial-intelligence This was from a Higher Education Policy Institute of 1,000 students, where they found 53% are using AI to generate assignment material. 1 in 4 are using things like ChatGPT and Bard to suggest topics 1 in 8 are using it to create content And 1 in 20 admit to copying and pasting unedited AI-generated text straight into their assignments Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’ https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html An HK-based employee of a multinational firm wired out $25M after attending a video call where all employees were deepfaked, including the CFO. He first got an email which was suspicious but then was reassured on the video call with his “coworkers.”   NSW Department of Education Launch NSW EduChat https://www.theguardian.com/australia-news/2024/feb/12/the-ai-chat-app-being-trialled-in-nsw-schools-which-makes-students-work-for-the-answers NSW are rolling out a trial to 16 public schools of a chatbot built on Open AI technology, but without giving students and staff unfettered access to ChatGPT. Unlike ChatGPT, the app has been designed to only respond to questions that relate to schooling and education, via content-filtering and topic restriction. It does not reveal full answers or write essays, instead aiming to encourage critical thinking via guided questions that prompt the student to respond – much like a teacher.   The Productivity Commission has thoughts on AI and Education https://www.pc.gov.au/research/completed/making-the-most-of-the-ai-opportunity The PC released a set of research papers about "Making the most of the AI opportunity", looking at Productivity, Regulation and Data Access. They do talk about education in two key ways: "Recent improvements in generative AI are expected to present opportunities for innovation in publicly provided services such as healthcare, education, disability and aged care, which not only account for a significant part of the Australian economy but also traditionally exhibit very low productivity growth" "A challenge for tertiary education institutions will be to keep up to date with technological developments and industry needs. As noted previously by the Commission,  short courses and unaccredited training are often preferred by businesses for developing digital and data skills as they can be more relevant and up to date, as well as more flexible"   Yes, AI-Assisted Inventions can be inventions News from the US, that may set a precedent for the rest of the world. Patents can be granted for AI-assisted inventions - including prompts, as long as there's significant contribution from the human named on the patent https://www.federalregister.gov/public-inspection/2024-02623/guidance-inventorship-guidance-on-ai-assisted-inventions   Not news, but Ray mentioned his Very British Chat bot. Sadly, you need the paid version of ChatGPT to access it as it's one of the public GPTs, but if you have that you'll find it here: Very British Chat   Sora was announced https://www.abc.net.au/news/2024-02-16/ai-video-generator-sora-from-openai-latest-tech-launch/103475830 Although it was the same day that Google announced Gemini 1.5, we led with Sora here - just like the rest of the world's media did!  On the podcast, we didn't do it justice with words, so instead here's four threads on X that are worth your time to read\watch to understand what it can do: Taking a video, and changing the style/environment: https://x.com/minchoi/status/1758831659833602434?s=20 Some phenomenally realistic videos: https://x.com/AngryTomtweets/status/1759171749738840215?s=20 (remember, despite how 'real' these videos appear, none of these places exist outside of the mind of Sora!) Bling Zoo: https://x.com/billpeeb/status/1758223674832728242?s=20 This cooking grandmother does not exist: https://x.com/sama/status/1758219575882301608?s=20 (A little bit like her mixing spoon, that appears to exist only for mixing and then doesn't)   Google's Gemini 1.5 is here…almost https://www.oneusefulthing.org/p/google-gemini-advanced-tasting-notes       Research Papers   Google's Gemini 1.5 can translate languages it doesn't know https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf Google also published a 58 page report on what their researchers had found with it, and we found the section on translation fascinating. Sidenote: There's an interesting Oxford Academic research project report from last year that was translating cuneiform tablets from Akkadian into English, which didn't use Large Language Models, but set the thinking going on this aspect of using LLMs   Understanding the Role of Large Language Models in Personalizing and Scaffolding Strategies to Combat Academic Procrastination arXiv:2312.13581   Challenges and Opportunities of Moderating Usage of Large Language Models in Education arXiv:2312.14969   ChatEd: A Chatbot Leveraging ChatGPT for an Enhanced Learning Experience in Higher Education arXiv:2401.00052    AI Content Self-Detection for Transformer-based Large Language Models arXiv:2312.17289   Evaluating the Performance of Large Language Models for Spanish Language in Undergraduate Admissions Exams arXiv:2312.16845   Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education arXiv:2401.00832   Empirical Study of Large Language Models as Automated Essay Scoring Tools in English Composition - Taking TOEFL Independent Writing Task for Example arXiv:2401.03401   Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors arXiv:2401.03238   Future-proofing Education: A Prototype for Simulating Oral Examinations Using Large Language Models arXiv:2401.06160   How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes arXiv:2401.05914   How does generative artificial intelligence impact student creativity? https://www.sciencedirect.com/science/article/pii/S2713374523000316   Large Language Models As MOOCs Graders arXiv:2402.03776    Can generative AI and ChatGPT outperform humans on cognitive-demanding problem-solving tasks in science? arXiv:2401.15081   

The podcast explores the potential of AI in teacher education, including using AI for lesson planning, assessment, and feedback. Leanne Cameron discusses how AI can alleviate administrative burdens and inspire innovative teaching ideas. The hosts reflect on the insights shared and discuss the future of teacher education.

This podcast covers the drama at OpenAI and Microsoft, the efficiency of AI in legal documents, AI outperforming clinicians in medical diagnosis, GPT versions impacting education, generative AI in creating comprehension questions, the impact of tipping information on AI performance, AI's advice superiority over columnists, AI in psychology and mental health, changing job market demand for AI skills, and the ethical considerations of manipulating AI systems.

Vitomir Kovanovic, an Associate Professor of Education Futures at the University of South Australia, discusses the transformative potential of generative AI in education, and its impact on teaching methodologies, organizational structures, and job markets. Tomas Trescak, the Director of Academic Programs in Undergraduate ICT at Western Sydney University, delves into the challenges of online assessments and suggests using AI to enhance personalized and frequent assessments.