Data Skeptic

Nieves Montes, Ph.D. student specializing in value-based reasoning and theory of mind, discusses her latest research on combining theory of mind and abductive reasoning in agent-oriented programming. The podcast explores the mechanics and challenges of Hanabi, the relationship between theory of mind and abduction, using predicate logic to represent desire and motivation in an agent, reasoning about other players in Hanabi, and future plans and online presence.

Maximilian Mozes, PhD student at the University College, London, specializing in NLP and adversarial machine learning, discusses the potential malicious uses of Large Language Models (LLMs), challenges of detecting AI-generated harmful content, reinforcement learning with Human Feedback, limitations and safety concerns of LLMs, threats of data poisoning and jailbreaking, and approaches to avoid issues with LLMs.

Machine Learning Engineer, Vid Kocijan, discusses the Winograd Schema Challenge and the advancements in Natural Language Processing. They explore the different schools of thought in NLP, the difficulty and techniques in the challenge, and the resolution of the challenge including alternative metrics.

Petter Törnberg, an Assistant Professor in Computational Social Science, discusses findings from his research papers on the performance of Chat GPT in interpreting political tweets, the ease of using language models in social science research, and the controversy surrounding large language models in social science.

Carlos Hernández Oliván, a Ph.D. student at the University of Zaragoza, discusses building new models for symbolic music generation. He explores whether these models are truly creative and shares situations where AI-generated music can pass the Turing test. He also highlights essential considerations when constructing models for music composition, including the role of creativity and the comparison between language models and music modeling. The podcast also delves into the potential of collaboration between music theorists, composers, and researchers.

Hongyi Wang, a Senior Researcher at Carnegie Mellon University, discusses his research paper on low-rank model training. He addresses the need for optimizing ML model training and the challenges of training large models. He introduces the Cuttlefish model, its use cases, and its superiority over the Low-Rank Adaptation technique. He also offers advice on entering the machine learning field.

On today’s episode, we have Daniel Rock, an Assistant Professor of Operations Information and Decisions at the Wharton School of the University of Pennsylvania. Daniel’s research focuses on the economics of AI and ML, specifically how digital technologies are changing the economy. Daniel discussed how AI has disrupted the job market in the past years. He also explained that it had created more winners than losers. Daniel spoke about the empirical study he and his coauthors did to quantify the threat LLMs pose to professionals. He shared how they used the O-NET dataset and the BLS occupational employment survey to measure the impact of LLMs on different professions. Using the radiology profession as an example, he listed tasks that LLMs could assume. Daniel broadly highlighted professions that are most and least exposed to LLMs proliferation. He also spoke about the risks of LLMs and his thoughts on implementing policies for regulating LLMs.

We are excited to be joined by J.D. Zamfirescu-Pereira, a Ph.D. student at UC Berkeley. He focuses on the intersection of human-computer interaction (HCI) and artificial intelligence (AI). He joins us to share his work in his paper, Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts.  The discussion also explores lessons learned and achievements related to BotDesigner, a tool for creating chat bots.

In this episode, we are joined by Ryan Liu, a Computer Science graduate of Carnegie Mellon University. Ryan will begin his Ph.D. program at Princeton University this fall. His Ph.D. will focus on the intersection of large language models and how humans think. Ryan joins us to discuss his research titled "ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing"

The creators of large language models impose restrictions on some of the types of requests one might make of them.  LLMs commonly refuse to give advice on committing crimes, producting adult content, or respond with any details about a variety of sensitive subjects.  As with any content filtering system, you have false positives and false negatives. Today's interview with Max Reuter and William Schulze discusses their paper "I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black-Box Generative Language Models".  In this work, they explore what types of prompts get refused and build a machine learning classifier adept at predicting if a particular prompt will be refused or not.