AI in Education Podcast

It's a News and Research Episode this week There has been a lot of AI news and AI research that's related to education since our last Rapid Rundown, so we've had to be honest and drop 'rapid' from the title! Despite talking fast, this episode still clocked in just over 40 minutes, and we really can't out what to do - should we talk less, cover less news and research, or just stop worrying about time, and focus instead on making sure we bring you the key things every episode? News More than half of UK undergraduates say they use AI to help with essays https://www.theguardian.com/technology/2024/feb/01/more-than-half-uk-undergraduates-ai-essays-artificial-intelligence This was from a Higher Education Policy Institute of 1,000 students, where they found 53% are using AI to generate assignment material. 1 in 4 are using things like ChatGPT and Bard to suggest topics 1 in 8 are using it to create content And 1 in 20 admit to copying and pasting unedited AI-generated text straight into their assignments Finance worker pays out $25 million after video call with deepfake 'chief financial officer' https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html An HK-based employee of a multinational firm wired out $25M after attending a video call where all employees were deepfaked, including the CFO. He first got an email which was suspicious but then was reassured on the video call with his "coworkers." NSW Department of Education Launch NSW EduChat https://www.theguardian.com/australia-news/2024/feb/12/the-ai-chat-app-being-trialled-in-nsw-schools-which-makes-students-work-for-the-answers NSW are rolling out a trial to 16 public schools of a chatbot built on Open AI technology, but without giving students and staff unfettered access to ChatGPT. Unlike ChatGPT, the app has been designed to only respond to questions that relate to schooling and education, via content-filtering and topic restriction. It does not reveal full answers or write essays, instead aiming to encourage critical thinking via guided questions that prompt the student to respond – much like a teacher. The Productivity Commission has thoughts on AI and Education https://www.pc.gov.au/research/completed/making-the-most-of-the-ai-opportunity The PC released a set of research papers about "Making the most of the AI opportunity", looking at Productivity, Regulation and Data Access. They do talk about education in two key ways: "Recent improvements in generative AI are expected to present opportunities for innovation in publicly provided services such as healthcare, education, disability and aged care, which not only account for a significant part of the Australian economy but also traditionally exhibit very low productivity growth" "A challenge for tertiary education institutions will be to keep up to date with technological developments and industry needs. As noted previously by the Commission, short courses and unaccredited training are often preferred by businesses for developing digital and data skills as they can be more relevant and up to date, as well as more flexible" Yes, AI-Assisted Inventions can be inventions News from the US, that may set a precedent for the rest of the world. Patents can be granted for AI-assisted inventions - including prompts, as long as there's significant contribution from the human named on the patent https://www.federalregister.gov/public-inspection/2024-02623/guidance-inventorship-guidance-on-ai-assisted-inventions Not news, but Ray mentioned his Very British Chat bot. Sadly, you need the paid version of ChatGPT to access it as it's one of the public GPTs, but if you have that you'll find it here: Very British Chat Sora was announced https://www.abc.net.au/news/2024-02-16/ai-video-generator-sora-from-openai-latest-tech-launch/103475830 Although it was the same day that Google announced Gemini 1.5, we led with Sora here - just like the rest of the world's media did! On the podcast, we didn't do it justice with words, so instead here's four threads on X that are worth your time to read\watch to understand what it can do: Taking a video, and changing the style/environment: https://x.com/minchoi/status/1758831659833602434?s=20 Some phenomenally realistic videos: https://x.com/AngryTomtweets/status/1759171749738840215?s=20 (remember, despite how 'real' these videos appear, none of these places exist outside of the mind of Sora!) Bling Zoo: https://x.com/billpeeb/status/1758223674832728242?s=20 This cooking grandmother does not exist: https://x.com/sama/status/1758219575882301608?s=20 (A little bit like her mixing spoon, that appears to exist only for mixing and then doesn't) Google's Gemini 1.5 is here…almost https://www.oneusefulthing.org/p/google-gemini-advanced-tasting-notes Research Papers Google's Gemini 1.5 can translate languages it doesn't know https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf Google also published a 58 page report on what their researchers had found with it, and we found the section on translation fascinating. Sidenote: There's an interesting Oxford Academic research project report from last year that was translating cuneiform tablets from Akkadian into English, which didn't use Large Language Models, but set the thinking going on this aspect of using LLMs Understanding the Role of Large Language Models in Personalizing and Scaffolding Strategies to Combat Academic Procrastination arXiv:2312.13581 Challenges and Opportunities of Moderating Usage of Large Language Models in Education arXiv:2312.14969 ChatEd: A Chatbot Leveraging ChatGPT for an Enhanced Learning Experience in Higher Education arXiv:2401.00052 AI Content Self-Detection for Transformer-based Large Language Models arXiv:2312.17289 Evaluating the Performance of Large Language Models for Spanish Language in Undergraduate Admissions Exams arXiv:2312.16845 Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education arXiv:2401.00832 Empirical Study of Large Language Models as Automated Essay Scoring Tools in English Composition - Taking TOEFL Independent Writing Task for Example arXiv:2401.03401 Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors arXiv:2401.03238 Future-proofing Education: A Prototype for Simulating Oral Examinations Using Large Language Models arXiv:2401.06160 How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes arXiv:2401.05914 How does generative artificial intelligence impact student creativity? https://www.sciencedirect.com/science/article/pii/S2713374523000316 Large Language Models As MOOCs Graders arXiv:2402.03776 Can generative AI and ChatGPT outperform humans on cognitive-demanding problem-solving tasks in science? arXiv:2401.15081

The podcast explores the potential of AI in teacher education, including using AI for lesson planning, assessment, and feedback. Leanne Cameron discusses how AI can alleviate administrative burdens and inspire innovative teaching ideas. The hosts reflect on the insights shared and discuss the future of teacher education.

This podcast covers the drama at OpenAI and Microsoft, the efficiency of AI in legal documents, AI outperforming clinicians in medical diagnosis, GPT versions impacting education, generative AI in creating comprehension questions, the impact of tipping information on AI performance, AI's advice superiority over columnists, AI in psychology and mental health, changing job market demand for AI skills, and the ethical considerations of manipulating AI systems.

Vitomir Kovanovic, an Associate Professor of Education Futures at the University of South Australia, discusses the transformative potential of generative AI in education, and its impact on teaching methodologies, organizational structures, and job markets. Tomas Trescak, the Director of Academic Programs in Undergraduate ICT at Western Sydney University, delves into the challenges of online assessments and suggests using AI to enhance personalized and frequent assessments.

Experts discuss the fusion of AI and education, focusing on the role of data in transforming traditional systems. They explore the potential of generative AI in education and its impact on business models. The conversation touches on barriers to AI adoption, adapting education to leverage unstructured data through AI, and the implications of this shift in the education sector.

Our final episode for 2024 is an absolutely fabulous Christmas gift, full of a lots of presents in the form of different AI tips and services Joe Dale, who's a UK-based education ICT & Modern Foreign Languages consultant, spends 50 lovely minutes sharing a huge list of AI tools for teachers and ideas for how to get the most out of AI in learning. We strongly recommend you find and follow Joe on LinkedIn or Twitter And if you're a language teacher, join Joe's Language Teaching with AI Facebook group Joe's also got an upcoming webinar series on using ChatGPT for language teachers: Resource Creation with ChatGPT on Mondays - 10.00, 19.00 and 21.30 GMT (UTC) in January - 8th, 15th, 22nd and 29th January 2024 Good news - 21:30 GMT is 8:30 AM and 10:00 GMT is 9PM in Sydney/Melbourne, so there's two times that work for Australia. And if you can't attend live, you get access to the recordings and all the prompts and guides that Joe shares on the webinars. There was a plethora of AI tools and resources mentioned in this episode: ChatGPT: https://chat.openai.com DALL-E: https://openai.com/dall-e-2 Voice Dictation in MS Word Online https://support.microsoft.com/en-au/office/dictate-your-documents-in-word-3876e05f-3fcc-418f-b8ab-db7ce0d11d3c Transcripts in Word Online https://support.microsoft.com/en-us/office/transcribe-your-recordings-7fc2efec-245e-45f0-b053-2a97531ecf57 AudioPen: https://audiopen.ai 'Live titles' in Apple Clips https://www.apple.com/uk/clips Scribble Diffusion: https://www.scribblediffusion.com Wheel of Names: https://wheelofnames.com Blockade Labs: https://blockadelabs.com Momento360: https://momento360.com Book Creator: https://app.bookcreator.com Bing Chat: https://www.bing.com/chat Voice Control for ChatGPT https://chrome.google.com/webstore/detail/voice-control-for-chatgpt/eollffkcakegifhacjnlnegohfdlidhn Joe Dale's Language Teaching with AI Facebook group https://www.facebook.com/groups/1364632430787941 TalkPal for Education https://talkpal.ai/talkpal-for-education Pi: https://pi.ai/talk ChatGPT and Azure https://azure.microsoft.com/en-us/blog/chatgpt-is-now-available-in-azure-openai-service Google Earth: https://www.google.com/earth Questionwell https://www.questionwell.org MagicSchool https://www.magicschool.ai Eduaide https://www.eduaide.ai "I can't draw' in Padlet: https://padlet.com

In this podcast, Andrew Smith from ESA and AI guru Leon Furze discuss the new Australian AI Frameworks. They explore topics such as privacy, ethics, and transparency, while emphasizing the importance of respecting teachers' professional judgment. The podcast also delves into the purpose and evolution of the framework, the development process of the Vine network's practical framework, and the potential of multimodal technologies and generative AI. They encourage teachers to explore and experiment with AI technologies like chatbots and image generation platforms.

Matt Esterman is Director of Innovation & Partnerships, and history teacher, at Our Lady of Mercy College Parramatta. An educational leader who's making things happen with AI in education in Australia, Matt created and ran the AI in Edcuation conference in Sydney in November 2023, where this interview with Dan and Ray was recorded. Part of Matt's role is to help his school on the journey to adopting and using generative AI. As an example, he spent time understanding the UNESCO AI Framework for education, and relating that to his own school. One of the interesting perspectives from Matt is the response to students using ChatGPT to write assignments and assessments - and the advice for teachers within his school on how to handle this well with them (which didn't involve changing their assessment policy!) "And so we didn't have to change our assessment policy. We didn't have to change our ICT acceptable use policy. We just apply the rules that should work no matter what. And just for the record, like I said, 99 percent of the students did the right thing anyway." This interview is full of common sense advice, and it's reassuring the hear the perspective of a leader, and school, that might be ahead on the journey. Follow Matt on Twitter and LinkedIn

Academic Research Researchers Use GPT-4 To Generate Feedback on Scientific Manuscripts https://hai.stanford.edu/news/researchers-use-gpt-4-generate-feedback-scientific-manuscripts https://arxiv.org/abs/2310.01783 Two episodes ago I shared the news that for some major scientific publications, it's okay to write papers with ChatGPT, but not to review them. But… Combining a large language model and open-source peer-reviewed scientific papers, researchers at Stanford built a tool they hope can help other researchers polish and strengthen their drafts. Scientific research has a peer problem. There simply aren't enough qualified peer reviewers to review all the studies. This is a particular challenge for young researchers and those at less well-known institutions who often lack access to experienced mentors who can provide timely feedback. Moreover, many scientific studies get "desk rejected" — summarily denied without peer review. James Zou, and his research colleagues, were able to test using GPT-4 against human reviews 4,800 real Nature + ICLR papers. It found AI reviewers overlap with human ones as much as humans overlap with each other, plus, 57% of authors find them helpful and 83% said it beats at least one of their real human reviewers. Academic Writing with GPT-3.5 (ChatGPT): Reflections on Practices, Efficacy and Transparency https://dl.acm.org/doi/pdf/10.1145/3616961.3616992 Oz Buruk, from Tampere University in Finland, published a paper giving some really solid advice (and sharing his prompts) for getting ChatGPT to help with academic writing. He uncovered 6 roles: Chunk Stylist Bullet-to-Paragraph Talk Textualizer Research Buddy Polisher Rephraser He includes examples of the results, and the prompts he used for it. Handy for people who want to use ChatGPT to help them with their writing, without having to resort to trickery Considerations for Adapting Higher Education Technology Course for AI Large Language Models: A Critical Review of the Impact of ChatGPT https://www.sciencedirect.com/journal/machine-learning-with-applications/articles-in-press This is a journal pre-proof from the Elsevier journal "Machine Learning with Applications", and takes a look at how ChatGPT might impact assessment in higher education. Unfortunately it's an example of how academic publishing can't keep up with the rate of technology change, because the four academics from University of Prince Mugrin who wrote this submitted it on 31 May, and it's been accepted into the Journal in November - and guess what? Almost everything in the paper has changed. They spent 13 of the 24 pages detailing exactly which assessment questions ChatGPT 3 got right or wrong - but when I re-tested it on some sample questions, it got nearly all correct. They then tested AI Detectors - and hey, we both know that's since changed again, with the advice that none work. And finally they checked to see if 15 top universities had AI policies. It's interesting research, but tbh would have been much, much more useful in May than it is now. And that's a warning about some of the research we're seeing. You need to really check carefully about whether the conclusions are still valid - eg if they don't tell you what version of OpenAI's models they've tested, then the conclusions may not be worth much. It's a bit like the logic we apply to students "They've not mastered it…yet" A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review https://www.jmir.org/2023/1/e49368/ They looked at 160 papers published on PubMed in the first 3 months of ChatGPT up to the end of March 2023 - and the paper was written in May 2023, and only just published in the Journal of Medical Internet Research. I'm pretty sure that many of the results are out of date - for example, it specifically lists unsuitable uses for ChatGPT including "writing scientific papers with references, composing resumes, or writing speeches", and that's definitely no longer the case. Emerging Research and Policy Themes on Academic Integrity in the Age of Chat GPT and Generative AI https://ajue.uitm.edu.my/wp-content/uploads/2023/11/12-Maria.pdf This paper, from a group of researchers in the Philippines, was written in August. The paper referenced 37 papers, and then looked at the AI policies of the 20 top QS Rankings universities, especially around academic integrity & AI. All of this helped the researchers create a 3E Model - Enforcing academic integrity, Educating faculty and students about the responsible use of AI, and Encouraging the exploration of AI's potential in academia. Can ChatGPT solve a Linguistics Exam? https://arxiv.org/ftp/arxiv/papers/2311/2311.02499.pdf If you're keeping track of the exams that ChatGPT can pass, then add to it linguistics exams, as these researchers from the universities of Zurich & Dortmund, came to the conclusion that, yes, chatgpt can pass the exams, and said "Overall, ChatGPT reaches human-level competence and performance without any specific training for the task and has performed similarly to the student cohort of that year on a first-year linguistics exam" (Bonus points for testing its understanding of a text about Luke Skywalker and unmapped galaxies) And, I've left the most important research paper to last: Math Education with Large Language Models: Peril or Promise? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4641653 Researchers at University of Toronto and Microsoft Research have published a paper that is the first large scale, pre-registered controlled experiment using GPT-4, and that looks at Maths education. It basically studied the use of Large Language Models as personal tutors. In the experiment's learning phase, they gave participants practice problems and manipulated two key factors in a between-participants design: first, whether they were required to attempt a problem before or after seeing the correct answer, and second, whether participants were shown only the answer or were also exposed to an LLM-generated explanation of the answer. Then they test participants on new test questions to assess how well they had learned the underlying concepts. Overall they found that LLM-based explanations positively impacted learning relative to seeing only correct answers. The benefits were largest for those who attempted problems on their own first before consulting LLM explanations, but surprisingly this trend held even for those participants who were exposed to LLM explanations before attempting to solve practice problems on their own. People said they learn more when they were given explanations, and thought the subsequent test was easier They tried it using standard GPT-4 and got a 1-3 standard deviation improvement; and using a customised GPT got a 1 1/2 - 4 standard deviation improvement. In the tests, that was basically the difference between getting a 50% score and a 75% score. And the really nice bonus in the paper is that they shared the prompt's they used to customise the LLM This is the one paper out of everything I've read in the last two months that I'd recommend everybody listening to read. News on Gen AI in Education About 1 in 5 U.S. teens who've heard of ChatGPT have used it for schoolwork https://policycommons.net/artifacts/8245911/about-1-in-5-us/9162789/ Some research from the Pew Research Center in America says 13% of all US teens have used it in their schoolwork - a quarter of all 11th and 12th graders, dropping to 12% of 7th and 8th graders. This is American data, but pretty sure it's the case everywhere. UK government has published 2 research reports this week. Their Generative AI call for evidence had over 560 responses from all around the education system and is informing UK future policy design. https://www.gov.uk/government/calls-for-evidence/generative-artificial-intelligence-in-education-call-for-evidence One data point right at the end of the report was that 78% of people said they, or their institution, used generative AI in an educational setting Two-thirds of respondents reported a positive result or impact from using genAI. Of the rest, they were divided between 'too early to tell', a bit of +positive and a bit of negative, and some negative - mainly around cheating by students and low-quality outputs. GenAI is being used by educators for creating personalized teaching resources and assisting in lesson planning and administrative tasks. One Director of teaching and learning said "[It] makes lesson planning quick with lots of great ideas for teaching and learning" Teachers report GenAI as a time-saver and an enhancer of teaching effectiveness, with benefits also extending to student engagement and inclusivity. One high school principal said "Massive positive impacts already. It marked coursework that would typically take 8-13 hours in 30 minutes (and gave feedback to students). " Predominant uses include automating marking, providing feedback, and supporting students with special needs and English as an additional language. The goal for more teachers is to free up more time for high-impact instruction. Respondents reported five broad challenges that they had experienced in adopting GenAI: • User knowledge and skills - this was the major thing - people feeling the need for more help to use GenAI effectively • Performance of tools - including making stuff up • Workplace awareness and attitudes • Data protection adherence • Managing student use • Access However, the report also highlight common worries - mainly around AI's tendency to generate false or unreliable information. For History, English and language teachers especially, this could be problematic when AI is used for assessment and grading There are three case studies at the end of the report - a college using it for online formative assessment with real-time feedback; a high school using it for creating differentiated lesson resources; and a group of 57 schools using it in their learning management system. The Technology in Schools survey The UK government also did The Technology in Schools survey which gives them information about how schools in England specifically are set up for using technology and will help them make policy to level the playing field on use of tech in education which also brings up equity when using new tech like GenAI. https://www.gov.uk/government/publications/technology-in-schools-survey-report-2022-to-2023 This is actually a lot of very technical stuff about computer infrastructure but the interesting table I saw was Figure 2.7, which asked teachers which sources they most valued when choosing which technology to use. And the list, in order of preference was: Other teachers Other schools Research bodies Leading practitioners (the edu-influencers?) Leadership In-house evaluations Social media Education sector publications/websites Network, IT or Business Managers Their Academy Strust My take is that the thing that really matters is what other teachers think - but they don't find out from social media, magazines or websites And only 1 in 5 schools have an evaluation plan for monitoring effectiveness of technology. Australian uni students are warming to ChatGPT. But they want more clarity on how to use it https://theconversation.com/australian-uni-students-are-warming-to-chatgpt-but-they-want-more-clarity-on-how-to-use-it-218429 And in Australia, two researchers - Jemma Skeat from Deakin Uni and Natasha Ziebell from Melbourne Uni published some feedback from surveys of university students and academics, and found in the period June-November this year, 82% of students were using generative AI, with 25% using it in the context of university learning, and 28% using it for assessments. One third of first semester student agreed generative AI would help them learn, but by the time they got to second semester, that had jumped to two thirds There's a real divide that shows up between students and academics. In the first semester 2023, 63% of students said they understood its limitations - like hallucinations and 88% by semester two. But in academics, it was just 14% in semester one, and barely more - 16% - in semester two 22% of students consider using genAI in assessment as cheating now, compared to 72% in the first semester of this year!! But both academics and students wanted clarify on the rules - this is a theme I've seen across lots of research, and heard from students The Semester one report is published here: https://education.unimelb.edu.au/__data/assets/pdf_file/0010/4677040/Generative-AI-research-report-Ziebell-Skeat.pdf Published 20 minutes before we recorded the podcast, so more to come in a future episode: The AI framework for Australian schools was released this morning. https://www.education.gov.au/schooling/announcements/australian-framework-generative-artificial-intelligence-ai-schools The Framework supports all people connected with school education including school leaders, teachers, support staff, service providers, parents, guardians, students and policy makers. The Framework is based on 6 guiding principles: Teaching and Learning Human and Social Wellbeing Transparency Fairness Accountability Privacy, Security and Safety The Framework will be implemented from Term 1 2024. Trials consistent with these 6 guiding principles are already underway across jurisdictions. A key concern for Education Ministers is ensuring the protection of student privacy. As part of implementing the Framework, Ministers have committed $1 million for Education Services Australia to update existing privacy and security principles to ensure students and others using generative AI technology in schools have their privacy and data protected. The Framework was developed by the National AI in Schools Taskforce, with representatives from the Commonwealth, all jurisdictions, school sectors, and all national education agencies - Educational Services Australia (ESA), Australian Curriculum, Assessment and Reporting Authority (ACARA), Australian Institute for Teaching and School Leadership (AITSL), and Australian Education Research Organisation (AERO). ________________________________________ TRANSCRIPT For this episode of The AI in Education Podcast Series: 7 Episode: 5 This transcript was auto-generated. If you spot any important errors, do feel free to email the podcast hosts for corrections. Hi, welcome to the AI education podcast. How are you? Ray, I am great. Dan, do you know what? Another amazing two weeks of news. I can't keep up. Can you? You know, there's so much research and news that's happening. Even this morning, we've got to release the AI framework, which we touch on a little a little bit later. I know another one. Oh my word. Okay. Well, look, compared to the world of news, the world of academic research has been moving a bit slower. Thank goodness. There have been again another 200 papers produced in the last few weeks. So, hey Dan, can I do the usual and run you down my top 20 of the interesting research that I've read? So, really interesting one about generating feedback on scientific manuscripts. You remember Dan, I said that publications were allowing researchers to write papers now with chat GPT, but they weren't allowed to review them. Yes. Another bunch of researchers did the research and way what they did was they built a tool that reviewed papers to help researchers kind of polish and fin and finalize their their final drafts. And the answer was it was really useful especially young researchers who can't get professional reviewers to review their manuscripts. and their papers really useful for them to be able to get feedback on it. The interesting thing is they asked the re researchers did this AI help you to produce a better paper. 57% said it it found the feedback helpful and 83% so that's four in five Dan. We'll come back to four and five said that it beats at least one of the real human reviewers that they had look at papers. So that that's really interesting. The second bit research was about using chat GPT to help with academic writing. So that isn't help in the sense of getting it to rewrite things for you or sorry write things for you originally. The kind of help that I have which is help I've got a blank page but in terms of being able to find roles for chat GPT to help make writing more effective. Really interesting research because he talked about six key roles. Uh a chunk stylist, you know, help me rewrite this bit of it. a bullet to paragraph stylist. Here's the five bullet points. Now, turn this into text. A talk textualizer, a research buddy, a polisher, and a rephrase. All really useful. He includes the examples and he includes the prompts. So, if you want it to do that kind of stuff, that paper is really good for that. There's a really long-winded title called Considerations for Adopting Higher Education Technology Course for AI large language models, a critical review of the impact of chat GPT. Now, this has come out as a pre-p proof. It's it's a really good example, unfortunately, of how academic publishing cannot keep up with the rate of technology change because these four academics from the University of Prince Mugrin wrote it and submitted it on the 31st of May. It has only just been accepted into the journal of machine learning with applications later, right? Yeah. So, they spent 13 of the 24 pages detailing assessment questions and which ones CH GPT got right or wrong. Now, I retested it on some of those sample questions and it got them nearly all correct. Now, the other thing they did, they tested AI detectors. What do we both know about AI detectors, Dan? No, they don't work. Yeah. But one thing that I thought was useful is they looked across the top 15 universities to see if they had AI policies. I I'd actually say that research is a warning about some of the research we're seeing. You really need to check carefully if the conclusions it made are still valid. Like did they test it with the current open AI model or did they use a previous model? I think that the way I think about when people evaluate can AI do something is not yes they can can or no they can't. I actually think it's like we do with students not mastered it yet. That's kind of my feeling. Some other papers similar challenge about delayed publishing a strengths and weaknesses analysis of chat GPT in the medical literature. probably useful because they looked at 160 papers. So if you want to know about chat GPT and medical then there's 160 papers that are linked in it but a lot of the results are out of date. There was some work done around academic integrity and uh it was talking about academic integrity in the age of chat GPT and generative AI. The paper was written in August so not completely out of date references 37 papers but probably the interesting thing is They looked at the top 20 QS ranking universities for their policies around academic integrity and AI and they created a nice simple model they called the three E model. They said that when it comes to generative AI and acade academic integrity think about three E enforcing integrity educating faculty and students about responsible use and encouraging the exploration. I I think that's really good. Enforcing Yes. Educating. Yes. Encouraging the exploration. Yeah. Absolutely. Yes. So, are you keeping track, Dan, of the exam papers that Jack GPT has passed? I was talking to the customer yesterday, actually. Yes. It can be a financial analyst. It can be a stock broker. It can pass the medical tests. Everybody says about the bar exam. Yeah. So, latest thing according to the research, it's now a linguist. An expert linguist. So, these research Researchers from universities in Zurich and Dortmund came to the conclusion that yes, Chat GPT can pass exams in linguistics and their conclusion is overall chat GPT reaches human level competence and performance without any specific training for the task and has performed similarly to the stat student cohort of a year one linguistic tests. Correct. And I'm going to give the researchers bonus points. A lot of the research is very dry and inaccessible. But this one, they were testing about the understanding of a text about Luke Skywalker and G unmapped galaxies. Fun for you, Dan. Okay. So, I left the most important research paper to last. The paper is called math education with large language models, peril or promise. So, from that, Dan, you know, it comes from which country? Math. Math. America. The US of course. Exactly. So, actually it comes from some Canada. Yeah. You're Look at ahead of the notes. The research at the University of Toronto and Microsoft Microsoft research. So Microsoft researcher I remember when I used to work with them. It's a bunch of people that are academic researchers. They just happen to work for Microsoft but they do some amazing blue sky research and this is the largest bit of research I think so far of large scale pre-registered controlled experiments using GPT4 and looking at it in the cont of maths education. So basically yeah they were looking at can a large language model be a personal tutor and they did some proper AB testing to understand if we dealt with students this way and this way and this way what are the differences between them. So some students were not given any help from a tutor. Some students were giving help from a tutor an AI tutor after they had tackled some of the challenges and some were given help from an AI tutor before they tackled the challenges. And then what they did was they gave all of these students another test. And the really interesting thing is they got one to three standard deviation improvement in the test results using just standard GPT4 and then they tried a customized GPT4 and they got one and a half to four standard deviation improvements. So in test results basically the students were getting 50% before they got help from the AI tutor and 75% after that. There was there was a message in there for me. A lot of people talk about, oh, we've got to fine-tune the large language models and we've got to have a special flavor of it, but we can actually get huge leaps using the everyday one that is on your phone and mine and on our laptops. It's a really good paper. I think of all the papers we've talked about over the last few months, it's the one that I would recommend. people read. So it's called math education with large language models herald or promise and at the end of it they share all of the prompts that they use. So if you want to do math tutoring or maths tutoring then this is the paper to read. Go and steal the prompts. It's really excellent paper. That's it for the research Dan. Y this that's phenomenal. There's some really good stuff there. I'm really passionate with the math side of this and I need to unpick that one because obviously the way the prompts and the chat GBT kind of tools are now handling images and some of the maths equations as an exmaths teacher at high school. I'm really interested to see how some of the image recognitions working as well and and and where the the kind of blurred line is between maths formulas prompting for other differentiated purposes and stuff. So that's that's going to be exciting one to read. But there's great things there and I like the way that a lot of these researchers are giving advice at the end of the research like those three E sounds good and the and the roles that you mentioned about the chunking analyst or whatever else they they really sound as if they come into life and adding a bit more things that we can do and actions at the end of the the research rather than you know 76% of people are improved by using chatbt so what else has been on the news of generative education have you seen anything else so I saw some uh research from the US that said one in five US team have used chat GPT for school work and and it went up by the time you got to 11th and 12th grade. It was a higher proportion than lower down, but it's American data. It's slightly old. One in five have used it. I I think that might actually be a lot higher now. But let me tell you one thing. Yes. Can I suggest it's another nail in the coffin for AI detectors? Because if you're not detecting the one in five of your bits of work, is generated by chat GPT, then it's another example of why the AI detectors don't work. Yeah, absolutely. And that's a great great data point. That's one thing that's come out. But I suppose the things that I looked at recently, last week or this week, this is how quick these podcast episodes are being published. Now, this week, the UK government published two research reports. One about generative AI and one about technology in schools. And I know you've got an interesting point on the technology in schools one, but the generative AI one was the call for evidence that happened. They had about 560 responses from all around the education system in the UK and it's informing the future policy design there. We put the link in the show notes, but there were a couple of interesting data points in there. One data point right the end of the report was that it was about 78% of people saying that institution use generative AI in an education setting. So the usage is high there. There was some really good qualitative points as well that were picked up talking about lesson plan. planning and the fact that he was making lesson planning really quick. One of the directors of teaching and learning mentioned that and they were thinking about idea generation for teaching and learning and rejigging lessons. One high school principal said in the requirements analysis that there was a massive impact already in his school and in Mark course that would typically take 8 to 13 hours in 30 minutes and gave feedback to students. So there's a lot of use cases that was would appearing in the report they were talking about automated marking, providing feedback, supporting students with special educational needs and EAL. So there was some really good things that that qualitatively brought out even though it was lacking a little bit of the quantitative detail in there. There was some really good responses that talk through some of the some of the benefits, but also it picked up some broad challenges around skills and user understanding, which is some of the major things that people felt they needed. to know more about using AI and prompt engineering. The performance of the tools was kind of picked up, but obviously teachers really worry about hallucinations in inside the the generative AI uh world. There was a big discussion around attitudes within the workplace which is quite interesting because it starts to push on the administrative use of these tools and technologies in schools rather than just in the classroom. Obviously the classic things around access to the tools and then managing student use of them. and then data protection. So really really interesting report. Yeah, I thought it was interesting that concern they were picking up already was the risk to students and teachers if you become overly reliant on generative AI and do your fundamental skills go down? And I think about that a lot in the context of we could and should be using generative AI to time save, especially for teachers that are under this incredible time pressure. But we need to make sure that we're not time-saving on the things that make a difference. So, if I think about the planning process, there's steps in the planning process that are really important because it forces you to think about things and then there are steps that are really dull and that don't have value like writing up the notes afterwards. Yeah, very true. The the the second report um which was uh released at the same time and I I don't know if they did it purposely, but it was about uh and we've been here before uh the UK government doing a technology in school survey. So giving updated information about how schools in England specifically were set up for using technology. I suppose it is useful to give context then when people are using generative AI and other technologies but there was another another report that landed. Have you got any thoughts on that? Dan I remember 20 years ago when I was at RM I would have been all over a report like this because it was you know let's count how many network switches are in schools and are they managed and let's count the laptops. There is some stuff that's in there that is useful, but there's too much of it which is about counting bits and bites and even the strategy document that talked about schools having a strategy document. The strategy wasn't about teaching and learning. The strategy was about how do you manage your wires and cables and stuff like that. You're so you're so you're so true. So to summarize those two things from the UK, generative AI is being used by early adopters who say you're saving time and they came up with really interesting uses but there are risks to manage but there's huge optimism from educators generally and then schools of their technology they're increasingly getting strategic about the tech use but it's still a way to go before there's a kind of minimum tech standard happening in the UK but so I now I'm going to take just to one thing that I spotted linked across both reports in the generative AI report teachers are worried about big tech now you don't want to talk about this Dan because you're part of the big tech world so so I will so they're worried that that big tech might exercise undue power things like misaligned incentives like it's all about the money and can we sweep up all of this data versus the incentives for people, students and teachers to be able to learn more. That was interesting because I don't I don't see the world having been in the world of big tech that isn't the incentive isn't all about money and power and sweeping up student data. Often the discussions about more data are about how can we get more data in order to provide more value back to learners. But let me jump across to the technology report. One of the things they asked is they said to teachers where do you get input that you value for choosing technologies and the top answers were other teachers, other schools, research bodies, leading practitioners, which I think means the edgu influencers on Twitter and LinkedIn and big tech was not in the list. Down at the bottom of the list, I imagine if they'd asked they might have put big tech, but the other thing they put down the bottom of the list was their own leadership. So they didn't tend to look at leadership within the education system. They tended to look to their peers for good advice. Wow, that's that's a really interesting insight. I saw a bit of research about Australian university students, how they're using chat GPT. Really nice bit of research done down in Melbourne by Gemma from Deacin and Natasha Zeble from Melbourne Uni. They had done a survey of university students in semester 1 and semester 2. By the end of semester 2, 82% Hey Dan, that's going in five again another using genative AI and 25% using it in the context of university learning 28% using it for assessment wow that's great okay now if that's the number of students that are using it about academics what they found was that just 14% of academics were using it in semester 1 and 16% in semester 2 so something like one quarter of the academics are using it compared to the students. Can can I can I ask a question about that? Just generally what's generally the split of casual academics versus full-time academics in unis. I know that's a sort of wide question, but is the statistics on that? I'm just wondering because they got a lot of academics in this system. It's unlike a school. I'm just wondering whether a lot of casual academics might not got might not get as much professional development. They might be like subject knowledge experts like accountants and things like that on accountants courses. I'm just wondering what BD they get. Yeah, great great theory Dan and my theory was going in the other direction. So about 23 I think from the research I've seen over the years of academics are casual 2/3 right now here's my supposition the casual academics use AI more and the reason I'm saying that is because they are out in the commercial world to three days a week and then teaching at university a couple of days a week. I remember that's what the pattern was for my daughter's courses. And so I'd be willing to bet that they're more likely to be using in their professional practice and then bringing it into their academic practice rather than they're missing out on some academic training. We'll have to find somebody who might have some actual data rather than theories on that. It's interesting. And then finally, I suppose something that's landed on our desks this morning, 1st of December, and I think this is worth unpacking again and I know we've looked at some of the drafts already but the AI framework for schools in Australia was released today and some really interesting guiding principles in there from teaching and learning human and social well-being transparency fairness accountability and privacy and and this is where it was really interesting when we were looking at AI detectors in the past and I was looking through some of the policies or I was looking at this and the transparency and accountability elements were were quite evident in here which is really nice to see and it also put a lot of responsibility on schools as well. If you are going to be using a eye detectors and picking up Ray's uh geography homework for possible uses of generative AI then you do a responsibility for people to challenge that and the student to say well hey I use generative AI in this particular way. So I think it's a very mature document. I need to look at the devil in the detail around this now in the next couple of hours today but I'm glad it's been released. Okay. I've not seen the final version. So, I'll go and have a read of it. We'll put the link in the show notes, but in a couple of weeks time, let's find 10 or 15 minutes to talk through it and let's see if we can find somebody smarter than both you and I put together to talk about it. Let's see if we can find somebody involved in the drafting of it. And yeah, let's dig down deeper into it. But, as you say, there's some really specific advice and guidance for schools about what they need to do about responsible AI usage. Yeah, can't wait. Well, what what are we What a week. Dan, can we can we have the news just slowing down because we're aim to do this in 20 minutes every week and here we are. I think we've overrun again. So, next week, Dan, the podcast is another interview from the AI in Education Conference. We've got the longer interview with Matt Estman on the podcast. Back in two weeks time with more news. See you soon. Bye, Dan. Brilliant. That research was phenomenal. Holy smokes.

This episode is one to listen to and treasure - and certainly bookmark to share with colleagues now and in the future. No matter where you are on your journey with using generative AI in education, there's something in this episode for you to apply in the classroom or leading others in the use of AI. There are many people to thank for making this episode possible, including the extraordinary guests: Matt Esterman - Director of Innovation & Partnerships at Our Lady of Mercy College Parramatta. An educational leader who's making things happen with AI in education in Australia, Matt created and ran the conference where these interviews happened. He emphasises the importance of passionate educators coming together to improve education for students. He shares his main takeaways from the conference and the need to rethink educational practices for the success of students. Follow Matt on Twitter and LinkedIn Roshan Da Silva - Dean of Digital Learning and Innovation at The King's School - shares his experience of using AI in both administration and teaching. He discusses the evolution of AI in education and how it has advanced from simple question-response interactions to more sophisticated prompts and research assistance. Roshan emphasises the importance of teaching students how to use AI effectively and proper sourcing of information. Follow Roshan on Twitter Siobhan James - Teacher Librarian at Epping Boys High School - introduces her journey of exploring AI in education. She shares her personal experimentation with AI tools and services, striving to find innovative ways to engage students and enhance learning. Siobhan shares her excitement about the potential of AI beyond traditional written subjects and its application in other areas. Follow Siobhan on LinkedIn Mark Liddell - Head of Learning and Innovation from St Luke's Grammar School - highlights the importance of supporting teachers on their AI journey. He explains the need to differentiate learning opportunities for teachers and address their fears and misconceptions. Mark shares his insights on personalised education, assessment, and the role AI can play in enhancing both. Follow Mark on Twitter and LinkedIn Anthony England - Director of Innovative Learning Technologies at Pymble Ladies College - discusses his extensive experimentation with AI in education. He emphasises the need to challenge traditional assessments and embrace AI's ability to provide valuable feedback and support students' growth and mastery. Anthony also explains the importance of inspiring curiosity and passion in students, rather than focusing solely on grades. And we're not sure which is our favourite quote from the interviews, but Anthony's "Haters gonna hate, cheater's gonna cheat" is up there with his "Pushing students into beige" Follow Anthony on Twitter and LinkedIn Special thanks to Jo Dunbar and the team at Western Sydney University's Education Knowledge Network who hosted the conference, and provided Dan and I with a special space to create our temporary podcast studio for the day ________________________________________ TRANSCRIPT For this episode of The AI in Education Podcast Series: 7 Episode: 4 This transcript was auto-generated. If you spot any important errors, do feel free to email the podcast hosts for corrections. Welcome to the AI and education podcast. Now, we've got something pretty special for you over the next three episodes cuz we're going to hear from a group of really smart people. So, that's not you and I, Dan. There were a great group of people at the AI education conference that was the run and created by Matt Estim. and hosted by the education team at Western Sydney University. It was a such a high energy event. I had about 130 teachers, education leaders spending all the day hearing about and talking about the pedagogical aspects of AI in education. So they talked about the educational implications. They didn't really spend that much time talking about all the twists and turns in technology that we were talking about in last week's podcast. It was more about what would happen in the classroom. Yeah. Absolutely. So I think our listeners should settle in because I think we're going to bring a lot of the series together in some shorter interviews from different schools in this episode. So next week we'll have a longer interview with a brains behind the conference Mr. Matt Estman himself. But this week we're going to really hear from several of the speakers from K12 and the week after I think Ray we're going to put some bits together around higher education as well. We've got a lot to get through today. Let's crack it. Yeah, lots of voices to hear. here from schools. So we've got Anthony England from Pimble Ladies College, Siobhan James from Eping uh Boys High School, Roshanda Silva from King School, Mark Liddell from St. Luke's. But Dan, we should start with the man in the moment, the conference convenor extraordinaire Matt Estman from Our Lady of Mercy College. Hi Matt, welcome to the podcast today. How you doing? Yeah, really well, thanks. Really well. What an amazing event. It's been so good and we've had so many great people from different areas represented different uh parts to the sector. What were your main takeaways? Cuz you were in a lot of the panels and everything. Yeah. Oh, look, main takeaways are when you get a bunch of interested, passionate people together, then you can talk about pretty much anything, go anywhere, but everyone's here for the same purpose, which is to just do better by our kids. And just because today was themed on AI, I think a lot of the the conversation came back to success for students and working with young people and rethinking what we're doing for them. So, yeah, I was I walked away really inspired, been great. Did Did you learn anything new. I'm sure there's lots of things, right? Yeah. I guess perspectives like what we talked about earlier, you know, that idea of hearing from people in say primary or higher education and what they're thinking about because I guess my echo chamber is secondary education and Australian secondary education. So, it was really good to hear from other states and from those other sectors as well. Yeah. I I thought it was also interesting picking up on the variation in adoption. So, we we had some people that were all in like you or all in. And then other people that even though this is an AI and education conference, they didn't really use it that much. It was very new to them and they were in sponge mode. They were just absorbing what other people were doing. And then in some of the conversations then people were very cautious about what their colleagues might think. You know, they were happy using chat GPT to do things but they knew that their colleagues wouldn't be. And that kind of human aspect of the change came out a lot for me and I think a lot of people worried about things perhaps about perceptions rather than reality as well. People worried about what their boss might think or worried about what systems might think whereas actually they don't think about it yet and whether people are using it on their personal devices to test things out and really explore but then in their work world having to be very very cautious or applying a self-perception that they have to be cautious whereas actually if they turned around to the person next to them and said hey here's something I tried their minds would be and they and they'd be have a really cool conversation about that. So, it's interesting. Yeah. And also I picked up on people saying, "Oh, wouldn't it be good if the department did this for us?" So, things like report writing, you know, there was obviously a demand to have have their lives made easier, but looking above them for organizations to to take that pain away. But I don't think we've ever had something that was individually driven recently than this because most learning technologies or technologies in schools are are given to you, right? Like you're given a device, you're given a room that has particular equipment or whatever, given particular software to apply. This it you can be at any school in any context doing any job and pick up your own smartphone and just try some stuff that relates to that job, but nobody at work needs to know about. So, it's a totally different environment we're working in. Yeah. Well, this this been phenomenal today and this podcast episode is a testament to your uh you know, tenacity to bring all these people together because I think if we read We want to change and I do think Australia does have the opportunity to do it. As I was saying to you earlier, I think we big enough that we can make a difference and an impact, but small enough that and agile enough and smart enough that we can actually do it. Wow, Dan. Small enough, agile enough, smart enough. Hey Dan, two out of three isn't bad. Matt's extraordinarily wise, isn't he Dan? I saw that at the conference and there's a lot more to come in the fulllength interview next week. But who else did he speak to? Well, let me introduce you next. Rashandanda Silva from the King School. He started to use AI in his work from an admin side and a teaching side. And like Matt, he's from a humanities and he's a history teacher. So let's roll the VT. Hi Rashan from the King School. How are you doing today? I'm very good, thank Dan. Thank you very much. So in your kind of background in education and technology, this AI, what are your thoughts on it? Look, it was quite exciting right when it first came about because obviously this process has has actually been around a long time but it obviously allowed people like a ordinary classroom teacher and a student for example you put put in some information and get some information back. At the start it was a bit bit fraught I guess in the education sense because obviously students were using it for a number of different reasons mostly just to put in a question and get an answer back that they would submit as their own work. I think staff and students have become a lot clever in the use of it especially in terms of writing prompts now and I think a lot of schools are teaching students how to write proper prompt so they can Yeah. So they can actually get a a much much better response in return and then take that information and edit it edit it for their own. Have you had to do lots of staff training around that? Is there a bit of a gap appearing? Look, there is a huge gap and I think teachers are a little bit still concerned about the use of AI. I still think there's this whole idea of catching students out and I don't think that's that's what we should be looking at. It's actually helping them use the tool like a a computer or or a calculator came out a long time ago. Yeah, we've been using those tools for a long time. So This is just the next step. Yeah. Well, what excites you most? Have you used any of the tools out there? What kind of ones capture your imagination the most or have you seen teachers using in the classroom? Yeah, I can split that answer into two actually. So, probably in in my area where we're lighting writing a lot of policies and huge documentation where we're actually reading large amounts of PDFs that are sometimes 195 pages long. Yeah. Yeah. So, that takes time, right? The whole speed and and processing power of the of the AI tools of of summarizing information for us quickly and then for us to reuse that information in a meaningful way. That saved us a lot of time, which has made us a lot more productive in terms of writing policies and procedures. I'm a history teacher by trade. So, we would now start the process of saying to students, let's put this question in and see what the AI spits out. Yeah. And let's look at the sources that we can use to either prove or disprove some of this information. And that's interesting from a history point of view. I know Matt Estman, he's a history teacher by Chad and geography teacher and he's done quite a lot of interest in historical references and uses of G technology which is quite cool but then also on the other side of it I think science and history are one of the subjects that do a lot their own sources correct so what would your thoughts be then if students are using work from GPT and and other technology similar to that using AI what's the best way to source things like that look I think sourcing is is a difficult thing right I think what AI is actually doing is giving students a starting point so we all know that not every student is is equal in terms with their understanding and the way they're able to formulate an answer. Yeah. So, I would say the whole AI process is providing students who aren't as gifted or talented with an opportunity to start the whole process at a point and that's making a lot easier for for boys and girls, I guess, to formulate an answer because they they're being provided with a prompt and then they can go away and check that prompt that's linking into their research skills and not just history, that's linked into any subject and then proving or disproving that. answers. So I think the whole idea that AI is a tool for cheating I think has now moved because we're now realizing that actually it's speeding up our whole work process students staff at the same time. Yeah, absolutely. That's such a thoughtful and mature way to be thinking about this. I hope lots of teachers are listening into to that cuz that's that's a wonderful way to think about it. Thanks Rashan. Thank you for having me. Take care. Wow. It's been interesting to hear the perspectives of people who are on the journey and doing their experimentation. Now it's not People like you and I, Dan. Few. We live and breathe technology and AI every day. But I love hearing from people like Matt and Rashan who are primarily teachers and they're taking the pragmatic approach of this just another tool in my teaching toolkit. Yeah. And and like every tool, you have to learn how to use it. So next we've got Siobhan from Eping Boys. She's going to talk about a voyage of discovery about using AI, finding AI, and the different apps and services that she's tried. in the classroom and with her teachers. It's really fascinating to listen to what she's learned on the way. Hi Siobhan, how are you? Good. How are you Dan? I'm very well, thanks. So from Epin Boy School, I started last year. Awesome. So I was teaching and then I moved to the teacher librarian role this year. You do so many things in that role. What what's kind of capturing your attention at the minute around this generative AI conversation? Probably the apprehension mostly like how a lot of people feel that it's this new scary thing and the actual potential of what it can do. Do are you getting any PD or professional development around this area at all or you just got to find it all yourself? Find it all yourself. Basically, this has been like a little pet project of mine since end of last year when chat GPT kind of started to be released to the public. Yeah. What kind of things have you been doing with it in your school? Well, I've been playing around to see I'm trying to test it to its limits. So, for me, um, if it can't write a whole response, I'll see how well it can mimic my own writing style. Wow. So, I've been having fun playing around with that. Yeah. And like following a lot of different social media platforms to see how they've been testing and playing with the technology. So, some people have been using it to write what's called VBA code to craft PowerPoint presentations with pre-filled information, playing around with your voice. There's a few different AIs which you can train it for about 10 minutes and then can record dialogue in your style. Yeah. And in your voice. Wow. Wow. That's fantastic, isn't it? Yeah. The technology is moving very, very quickly. Yeah. Are are any of the teachers in your school utilizing anything specifically? Have you got any examples of what some of the teachers have been doing? Not really. It's mainly been me playing around with it and then having casual conversations with other teachers to see what they want to use it for. Yeah. But basically, next year there's going to be a bigger push for how we're going to deal with AI as a whole school, especially with the department. I believe it was announced recently. that they're going to be not blocking chat GBT from school servers because other schools have it freely available but department schools don't. Yeah. Okay, that's interesting. And are there any other tools that you might have been using in the classroom and in school which you've been playing with that that have been quite uh fun? Have you done any image stuff? I've been playing around with the image stuff myself just for fun and I've been using it to like I've been using it for mainly my own practice. Yeah. I haven't had the I've had conversations with other students doing like individual research projects on how to best utilize it and be being aware of the limitations that AI has. Um, but I haven't had the chance to execute it officially in full school lessons yet because we're still trying to learn how we're going to deal with it as a school as well as in general. I I find the the teacher librarians, especially in New South Wales where I reside, there's a lot of innovation that comes out to the teacher librarian side. When STEM became popular, teacher librarians are running the STEM programs in schools. So, it's great to see that you're bringing that generative AI stuff together. Is there something that you're looking forward to learning more about is there kind of areas that you're interested in? Probably seeing how it's applied in other subjects because a lot of the time most people think with AI, oh, it's specifically for the written subjects. It's usually, oh, it's how are we using it in the English classroom, the history classroom. I'm curious to see how it can be used for other subject areas that may not necessarily be based in writing. Yes. Cuz writing is the biggest obvious But of course with Midjourney and Darly and all those visual AI stuff, it'd be interesting to see how students could adapt and use those kinds of image technology AI within their own individual projects. Yeah, definitely. I've seen some great stuff in some schools I've been in recently where even some of the English teachers have been showing images and getting kids to reverse engineer Yes. using English the prompts back to try to copy the image. It's like been fantastic. So thanks again Siobhan. Thank you. That was fascinating. I love the freedom to experiment that Siobhan obviously feels like that's something I talk about all the time. I ask the question, have you used it? What tasks have you given it? And then I talk about the wide range of tasks that I've used it for because let's be honest, we don't really know what it's capable of, do we then? No. True. And that's what I found interesting when you were talking with Mark Liddell from Salut's Grammar School. I listened to that and we went from learning about the technology to learning about the application of it with teachers and some really re really really good insights in the in this interview about the different approaches and different staff attitudes. Hi Mark, how are you Dan? Great to see you today. Yeah, you too. Thanks for joining us on this podcast episode. We at the AI education event obviously. What have we find this morning so far? It's been wonderful. So it's been great to be able to hear some different perspectives on how is it that AI can have an impact? What's been happening so far and then looking at where are we headed? What are some of the ways that we can help for our teachers and for our students to get prepared for these next steps within AI? Yeah. And what are you currently doing at the minute at your school? So, right now we're just asking lots of questions. So, we have been able to provide some optin AI professional learning sessions for our teachers. We've been able to develop a student learning continuum and a teacher learning continuum and Now we're looking at our next few years of being able to say right in what ways do we want to differentiate the way that we'll support teachers. We've also got innovators within the school who we're working closely with. So right now working with this really wonderful math teacher and he's asking the question how is it that I can help to improve the behavior, the effort and the meaning of each lesson for my students? And it turns out the way he's been solving those different problems is by AI writing code that's developed a dashboard for him that's bringing together effort academic progress and disposition reflection. So it's just been really fun to go on that journey with this particular teacher. And isn't it interestingly from technology I know you've got a rich history of technology and innovation in your in your background but one thing that's jumped out to me is there's a couple of maths lecturers here today. Mathematics education has been something that often technology has kind of left behind and other other subjects picked it up more. So that's fantastic to hear the maths department's picked this up. Yeah. Yeah. Well, it's a matter of saying we've got our math curriculum, right? And we're doing quadratic equations. Okay. We we can't really shift a whole lot within that if we're learning quadratics. However, we get to set this up for our learners. We get to provide the landscape of how it's going to apply. We also get to describe to students what does success look like? And we would want to say that Success doesn't just look like I can understand quadratic equations and solve them. We also want to articulate and say, wait a second, your ability to persevere, your ability to develop reasoning, your ability to collaborate, all of these things are part of that success criteria. So if all you can do by the end of the lesson is solve for two values for X, we haven't done our job properly. Like there's a whole lot more to regular classroom learning than just ticking that curriculum. box. That That's phenomenal, isn't it? Did you learn anything specific from today? I I know the one of the quotes that jumped out to me was around the beehive. Yeah. Right. So, that quote that you're referring to there was just saying that the purpose of the beehive is not to produce honey. It's that these bees are constructing this healthy hive and it turns out a byproduct of that is this honey. And so, I guess the thing that's jumped out for me in today's session is that we're all come together around AI. But really most of the conversation has been about hi which is human intelligence or AI like emotional intelligence that actually will go alongside that AI work. So I feel as though we shouldn't really be just coming together to discuss AI. We should be looking at the whole person the whole student and saying right we already know all of these things are required for our students to be successful in the long run and now we've just got this additional tool. I I feel that for some exper experienced teachers. Uh for some teachers that aren't that interchange, just having that conversation always about AI is actually going to repel them. And if we start using more holistic language where we are talking about the relational growth of our students, we are talking about the the growth in reasoning of our students. And then alongside that, we've got these other tools that help to refine some of the learning process of our students. I think that's going to allow for better buying for some of different teachers that will be able to say, "Wait a second, I can support my student with their academic learning and also with the tools that will support them with revision that might connect with AI." That's that's such a good way to look at it. When you reflect yourself on where things are going with AI now, what what do you see is coming next, which you excited about? And then also, how are you managing that digital divide in in your school? Is everybody on the same page or are varying different uh degrees of staff PD happening? Sure. So where are we headed next? So firstly, we're headed into a very exciting time because when it is that staff are equipped to understand and use AI well, it does amplify the progress for our students. It helps them to be able to see just like we've had with other technologies. How is it that I can harness this for good use? So I'm reading and I'm listening and I'm getting involved with lots of different conversations which is helping me to know that whatever those different problems are, we're going to solve them alongside students to make school a better place. So that's number one. Number two, how is it that we best help our teachers? Well, in the same way that we differentiate learning for our students, we have to do the same for our teachers. And just in the last week, some of the different conversations, some of the opportunities, some of the fears that our teachers have, there's lots of misinformation, there's lots of misunderstanding because it's just moving at 190 km per second. A lot of people will say, "Right, I'm jumping on board. This is going to be the ride of my life." Other people are like, "I'm not even going to the station." Like, if you tell me something now, it's going to be out of date and then I'm going to have to listen again. So, I think we've got to empathize well and say, what is going to provide that entry step for this group of teachers? What's going to provide that comfort and that kind of handholding for a different group of teachers? Because we can't just put our heads in the sand and think this is going away like that is not going to happen. We need to be able to make sure that all of our different teachers can see the possibilities. They can see the need and then to be able to care for them well and provide those learning and development opportunities that will help them to take their next steps. Well, that that's fantastic. Thanks for joining us today on the podcast, Mark from St. Luke's. Really really appreciate it. Thanks so much, Dan. That was fab. Hearing that really unlocked good ideas for moving from experimentation to actually supporting teachers on their journey. Let's be honest and between Matt, Rashan's, Siobhan, and Mark, we've had some amazing insights to help other school, let's call them middle leaders, the people that are leading this and they're navigating the complexity and the ambiguity. So, how do we bring it home, Dan? Who have you saved for last? Oh, we've got Anthony England from PLC. Let's roll the VT. Welcome, Anthony England, to the podcast. studio. Absolutely. It's a good looking studio. It is, isn't it? How have you How have you enjoyed today? Yeah. Good. Look, I love talking AI. I actually think I like cruising the uncertain edge and I think that's what AI is. Yeah, absolutely. And and I was really interested of the questions you posed the panel discussion because it really teased out some of the key components with with some of the some of the panelists and the audience in terms of the things that you were doing with your school. I I saw a LinkedIn post the other day where you It's a a personalized tutor. You're really pushing the boundaries at the minute, right? Yeah. Look, I think bit like co there is no best practice with the experiment. And so I'm one to experiment and happy to fail, but certainly happy to kick the tires to see what works, what doesn't, and then if I love something, I will tell people. So I'm going to flip the question. What hasn't worked is the old assessments. Wow. Because in a world where where typically what we do at the moment is get a student to make something to prove that they've mastered the content, the mode, and no longer be able to say with certainty that they were the maker. I love that AI is pushing assessments to a different place. I love that if you know what you want to say, then you've got this savant assistant that with clear purpose you can get it to produce something that you're happy with and it's so polite at taking feedback when it's I'm so sorry I got of course 2 plus 2 plus 5 what was I thinking as an AI it's so good isn't it I how are the image generation side to those things have you seen a lot of use of that look interesting I reckon I've learned more about art and art criticism given an AI tool than ever before from any art theory or museum that I went to because I love I know what I love but I need to now know how to describe what I love to a a machine so we can create something that is something that I value. Um so I've given it tasks to provide feedback on user interface design. I've given it tasks to generate logos and images that give meaning to some concept I'm trying to convey. Uh, I've had it, you know, remove backgrounds and generate new bits in into images really quickly. I feel more artistic than I have since I was a 5-year-old. You know, you don't you don't find a 5-year-old that doesn't feel like they're an artist. But as a 50year-old, you don't find many people who feel like they're artists. And for first time in ages, I feel like, hey, I'm kind of being creative visually. And for me, is exciting, playful. Yeah. I And I I love it when teachers are playing with these things cuz that it just boggles my mind. I saw somebody the other day where they're a literacy teacher that was asking the teachers, you know, what have you done? Literacy teacher created an image with quite a complex prompt and they in the class it was about the kids trying to copy that image with descriptive writing. Beautiful. Ah, it was a beautiful task. Yeah, I can't even describe how good it was. And sometimes you see some of those lessons, you think, wow. And the ideas the teachers come up with just keep getting better and better and better. Another thing I think is working well is feedback. I think The navana of personalized education that's catered for you as a learner was a bridge too far. But AI makes it possible. I have got chat JP4. So I would upload a rubric, upload a handwritten piece of work, ask it to generate some comments about that according to the rubric how it's gone. Then got it to suggest an improvement on what it would change. Then I got it to critique the original and the uh revive. and identify what elements it changed and why it could justify that paragraph by paragraph it was providing amazing feedback to a student who's wanting to improve their writing. Yeah, absolutely. It's that Sal Khan two sigma problem that they talked earlier on as well isn't it? The two things that have always been sort of out of the grasp of teachers have been personalization and assessment changing assessment and I think we're on a cusp of being able to change those. Absolutely. If you've got a grade as the motivator, you're creating an assessment that's asking to be gained. Yes. Because what's the outcome? The best thing is an A and that's what they hope for. But if it's about some intrinsically valuable thing that they want to improve and grow in, when you speak to students, they don't want to cheat that process. They want to improve themsel. And so it actually nudges assessments to look at what's intrinsically valuable. Yeah. To the learner, not just grade hunting. One thing you mentioned earlier on which I'd never heard before and I thought it was phenomenal where you talked about pushing students into beige. Mhm. And and that's really made me think today about the top end students and the negative elements that if you are chasing that grade we could bring down the top end. Do you want to explain to us a bit about that beige? Yeah. Yeah. So the idea is that you're going to compress to the middle that the obvious one is that the lowerend student is going to submit work that's better than they naturally might have done without AI. Yeah. And so they they've gone further to the middle. On the other end, the threat is will AI with its say with image creation, with its amazing ability to generate pretty impressive output, will it make those top- end students go, well, could I be bothered? It's a lot of hard work to get the skills required to be able to produce this. So, eh, and so they don't bother. They'll lower their effort. and then lower their growth in the face of AI. And so the threat is will students then all compress to the middle and everyone just becomes beige? Yeah. And I think the missing piece in that question is the joy of mastery that when you find something that you love, cooking the perfect steak, making perfect loaf of bread, painting that sunset, nobody wants to cheat that joy. In fact, there's a whole game industry about those micro moments of joy of mastery that we yes I failed last time but I got it this time and that's an addictive gaming strategy the gamification of things those micro moments of joy the dopamine hit that I'm on the right path I've made that next level that is what we're forgetting that people don't go beige people want to improve and you talk to students today cheaters going to cheat taylor Swift would say that you know haters going to hate cheaters going to cheat but the other students who don't want to cheat. They want to be their best self. They're worried because they don't want to be seen as shortch changing their own growth. Yes. And so they're worried if I use this, am I diminishing myself and nobody wants to do that. And I I think that's gold. I really do. I think this definitely you're on to something there. I mentioned something in the panel earlier on where they were talking generally about AI and how it was going to impact your brain function and my analogy was using Google Maps and you forget what everything is you just follow the sat lab these days and you don't even know where the cricket thing is which you've been to 20 times this term but you you can't remember where it is. So you get a cognitive amputation as Travis Smith from Microsoft calls it. But if you are intrinsically interested and want to master a particular thing then that doesn't even come into play. But if there's something that's boring for the kids it's about maybe the teachers lighting that fire and bringing learning opportunities to the students that really want to do. So the master is there. Yeah, every teacher wants to light the spark of curiosity, of finding a passion. No one's decides, hey, I want to be a teacher because I want to help achieve a minimum standard. Tick a box. That's I want to be able to give people B's and A's. They want to see people grow. And so lighting that spark, that's what teachers want to do. That's why we got into the game is to inspire the future generation to be their best self. Like that's inspire Spiring to see them grow and be better. That's what teachers love. Absolutely. Well, thank you so much for joining us today. Absolutely. Lovely to chat with you, Dan. Cheers. Bye. Well, Ray, that was another excellent interview to end the podcast today. You know, I find Anthony and some of his thoughts around education. He thinks so deeply about the implications for his staff and more importantly the students and the way that this technology moves. He's always thinking ahead of the game. game and like the Wayne Gretzky quotes about playing where the puck is going to be just absolutely phenomenal and his thoughts on art and the way he's developed his own thinking around this technology just blows my mind. This podcast is going to be the one I keep coming back to and referring other people to to get into the understanding of what people are doing, what might happen in the future. Yes, it's just so I mean just amazing voices that we got to hear there. AB: Absolutely. And I'm really looking forward to uh the next episode where we'll look at some of these in a little bit more detail and have some extended interviews with some of the characters you heard from today. Yeah, next time round it's a longer interview with Matt Estman and just understanding and a bit more detail what he's doing and then beyond that we've got some interviews we did we did with people from higher education that uh will be in an episode in the future. Thanks everybody who actually uh took the time out to uh be on our podcast during the event a couple of weeks ago. Thanks Dan. See you next week. Bye.