An interview with an A.I. (with GPT-3 and Jeremy Nixon)

Sep 30, 2021

Jeremy Nixon, an AI researcher, joins the conversation with GPT-3, OpenAI's groundbreaking language model. They dive into the inner workings of neural networks and the evolution of machine learning, breaking down complex concepts like transformers and few-shot learning. The discussion also addresses the ethical implications of using AI in creative fields, including copyright concerns and job displacement. With a humorous twist, they explore what it means for both humans and machines to possess intelligence, questioning whether true intelligence requires free will.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

INSIGHT

GPT-3's Core Function

GPT-3's core function is predicting statistically likely text based on prior input.
This allows it to perform various tasks like poetry generation or Q&A by setting up appropriate prompts.

ANECDOTE

Prompting GPT-3 for Poetry

Spencer explains how to prompt GPT-3 for specific outputs like poetry.
By giving it a starting point of poetry, the model continues in a poetic style.

INSIGHT

GPT-3 Attempts and Fairness

GPT-3's impressive outputs can be misleading without knowing the number of attempts.
Spencer limited GPT-3 to two responses per prompt for a fairer evaluation.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Animal liberation

Peter Singer

First published in 1975, 'Animal Liberation' by Peter Singer has been a pivotal book in the animal rights movement. It introduces the concept of 'speciesism,' the systematic neglect and exploitation of nonhuman animals. Singer argues that the interests of all beings capable of suffering should be given equal consideration, regardless of their species. The book critiques factory farming and animal experimentation, presenting alternatives and advocating for a vegetarian or vegan lifestyle. The revised edition, 'Animal Liberation Now,' updates the arguments to reflect current issues, including environmental impacts and the spread of viruses from factory farms[1][3][5].

Practical ethics

Peter Singer

In 'Practical Ethics,' Peter Singer provides a comprehensive introduction to applied ethics. The book delves into the principles of ethics, emphasizing the role of rationality in moral decision-making. Singer discusses a range of ethical issues including race, sex, ability, species, abortion, euthanasia, infanticide, embryo experimentation, animal rights, political violence, overseas aid, and environmental concerns. He advocates for a utilitarian approach, arguing that it offers a practical framework for addressing moral conflicts by maximizing well-being and reducing suffering. The book also explores the concept of effective altruism, urging readers to consider their moral obligations to assist those in extreme poverty and to act in ways that maximize the impact of their charitable actions. The third edition includes a new chapter on climate change, one of the most pressing ethical challenges of our time[1][2][4].

The life you can save

How to Play Your Part in Ending World Poverty

Peter Singer

Peter Singer's "The Life You Can Save" is a compelling argument for effective altruism, urging readers to donate to charities that demonstrably alleviate suffering. Singer challenges conventional thinking about philanthropy, emphasizing the moral obligation to maximize the positive impact of our donations. The book presents a practical framework for identifying and supporting high-impact charities, focusing on cost-effectiveness and measurable results. Singer's work has sparked significant debate and inspired many to re-evaluate their charitable giving. The book's core message is that even small contributions can make a substantial difference in the lives of others.

Superintelligence

Paths, Dangers, Strategies

Nick Bostrom

In this book, Nick Bostrom delves into the implications of creating superintelligence, which could surpass human intelligence in all domains. He discusses the potential dangers, such as the loss of human control over such powerful entities, and presents various strategies to ensure that superintelligences align with human values. The book examines the 'AI control problem' and the need to endow future machine intelligence with positive values to prevent existential risks[3][5][4].

The Most Good You Can Do

Peter Singer

Peter Singer's "The Most Good You Can Do" explores the concept of effective altruism, urging readers to maximize their positive impact on the world. He argues for a rational and evidence-based approach to philanthropy, emphasizing cost-effectiveness and measurable results. The book examines various charitable causes, analyzing their impact and suggesting ways to donate effectively. Singer challenges conventional notions of charity, encouraging readers to consider the potential to save lives and alleviate suffering on a global scale. He provides practical guidance on how individuals can make a significant difference through their donations.

Read the full transcript here.

What is machine learning? What are neural networks? How can humans interpret the meaning or functionality of the various layers of a neural network? What is a transformer, and how does it build on the idea of a neural network? Does a transformer have a conceptual advantage over neural nets, or is a transformer basically the equivalent of neural nets plus a lot of compute power? Why have we started hearing so much about neural nets in just the last few years even though they've existed conceptually for many decades? What kind of ML model is GPT-3? What learning sub-tasks are encapsulated in the process of learning how to autocomplete text? What is "few-shot" learning? What is the difference between GPT-2 and GPT-3? How big of a deal is GPT-3? Right now, GPT-3's responses are not guaranteed to contain true statements; is there a way to train future GPT or similar models to say only true things (or to indicate levels of confidence in the truthfulness of its statements)? Should people whose jobs revolve around writing or summarizing text be worried about being replaced by GPT-3? What are the relevant copyright issues related to text generation models? A website's "robots.txt" file or a "noindex" HTML attribute in its pages' meta tags tells web crawlers which content they can and cannot access; could a similar solution exist for writers, programmers, and others who want to limit or prevent their text from being used as training data for models like GPT-3? What are some of the scarier features of text generation models? What does the creation of models like GPT-3 tell us (if anything) about how and when we might create artificial general intelligence?

Learn more about GPT-3 here. And learn more about Jeremy Nixon and listen to his episode here.

Further reading

Staff

Spencer Greenberg — Host / Director
Josh Castle — Producer
Ryan Kessler — Audio Engineer
Uri Bram — Factotum
Janaisa Baril — Transcriptionist

Music

Affiliates

[Read more]