Data Skeptic Auditing LLMs and Twitter
Jan 29, 2025
In this insightful discussion, Erwan Le Merrer, a collaborator in graph theory and distributed systems, reveals how graph-based techniques can expose patterns in large language models and shadow banning on Twitter. He explains the application of epidemic models to examine shadow banning spread across user networks. The conversation also highlights the use of graph metrics to audit LLM outputs, the challenge of bias detection in AI, and innovative methodologies for understanding algorithmic behavior, shedding light on often-overlooked platform moderation practices.
AI Snips
Chapters
Transcript
Episode notes
Memorizing with Spaghetti
- Gilles Tredan memorized the Karate Club dataset by building it with spaghetti with his kids.
- They even won a network science competition with their spaghetti graph layout.
Graph Hallucinations
- LLMs hallucinate, and graphs reveal specific aspects of these hallucinations.
- Studying these graph hallucinations helps understand the internal workings of LLMs.
Prompting for Graphs
- The researchers prompted LLMs to output known graphs as Python edge lists.
- Some LLMs refused, claiming the graph data was private information.