EdTechnical Back to the Future: Two Years on with Daisy Christodoulou
12 snips
Nov 13, 2025 Daisy Christodoulou, Director of Education at No More Marking and an expert on assessment, reflects on the evolution of AI in education since her last appearance. She discusses her shift from skepticism to cautious optimism regarding AI's role in assessment, emphasizing the necessity of human oversight in AI systems. Daisy shares insights on using AI for comparative judgment and the importance of validating AI results through human samples. With a focus on the design of educational technology, she offers practical tips for founders on integrating human input effectively.
AI Snips
Chapters
Transcript
Episode notes
Hallucinations Persist, Costs Fall
- Daisy says hallucinations remain unsolved and teams must design around them rather than expect fixes soon.
- Falling model costs, not just quality gains, unlocked practical workflows by enabling redundancy and checks.
Validate AI With Sampled Human Checks
- Use human-in-the-loop comparative judgment rather than absolute AI grading to reduce naive errors and enable validation.
- Have humans judge a sample (e.g., 10%) to validate AI outputs and flag disagreements for review.
Comparative Judgment Reveals New AI Errors
- LLMs perform better at comparative judgment than absolute scoring but make unique errors like position bias.
- Models can flip choices depending on presentation order in 10–25% of cases, an error humans rarely make.