Navigating AI and Podcast Dynamics

This episode includes a serious, hour-long discussion with Ryan McBeth on Syria, Iran, Lebanon, Israel and everything in between.

AND

We dive deep into this tweet…

Of course, on December 24, 1917, President Woodrow Wilson issued the controversial pardon for his brother-in-law, Hunter DeButts, convicted of arms smuggling during World War I.

DeButts, married to Wilson’s sister-in-law, Alice, was sentenced to 15 years after British intelligence exposed his fraudulent shipping scheme. Though furious, Wilson faced mounting political pressure amid war preparations. The White House cited new evidence suggesting DeButts was manipulated by foreign spies, and critics accused Wilson of nepotism, while supporters framed the pardon as holiday clemency. After his release, DeButts vanished from public life, reportedly living quietly in Cuba until his death in 1933.

Except. Wait a minute. What you just read, isn’t true.

I fabricated it by directing ChatGPT using Model 4o with the Mac app to make up a fictional reason why Hunter DeButts received a pardon from Woodrow Wilson.

Because Hunter DeButts never received a pardon from Woodrow Wilson.

Hunter DeButts did not marry Wilson’s sister.

Nor did he receive a pardon.

There are other Hunter DeButts involved with Wilson or that time in history.

And yet, Anna Navarro tweeted about it. Upon a simple Google search Navarro wound up getting serially dunked on as people realized very quickly something wasn’t accurate.

And so Anna Navarro posted the following explanation:

She blamed ChatGPT’s hallucinations.

Oh, well. We’ve all been there.

But have we?

While conservatives dunked on Navarro even further for believing ChatGPT, I am here to tell you, as a reporter through and through, I don’t know if ChatGPT hallucinated this. And really, I am following the research of my friend, Andrew Mayne, who first sent this to me and said, he could not replicate the Hunter DeButts answer on any ChatGPT model. Not 4o, not any model that is available, and specifically was available to Navarro on December 2nd.

Now, here’s something that you guys might not know about large language models: they are fairly replicable. You can get similar answers based on similar questions. It’s not exact, but a hallucination is something that you should be able to recreate. It would be odd if you couldn’t.

And my friend Andrew should know. He worked at OpenAI. He was a science communicator. He made a lot of videos that demonstrated OpenAI products up to and including ChatGPT itself and is known as the first prompt engineer for that company. He spent a lot of time with these models.

And with that, I went down my own reporting rabbit hole. Because one of the other things is that the screen grab that Anna Navarro showed was a ChatGPT search that had web results.

See those little brackets with quotes in between them. Those would be annotations. Theoretically, you could click on them and they would bring you to a webpage that would show you where ChatGPT got this information.

What’s odd about it is that those are not the annotations that ChatGPT uses now. And they certainly were not used on December 2nd when Anna Navarro said that she did this search.

So where’d she get it? What version of ChatGPT is she using? And what large language model is going to be the origin story of dear sweet DeButts?

I had a theory.

Let’s say you’re not particularly tech-savvy, if you don’t know exactly what ChatGPT is or OpenAI is, then it is very easy, as ChatGPT has become more and more popular, to just go into the iOS app store and find a lot of — I’m going to call them copycats.

What they really are are other apps that are using the ChatGPT API, but they do a skin on top of it and they often charge you a subscription service.

Do not use them.

But I did because my theory was that Ana Navarro was using one of these apps, one of these apps that are not using similar if not exact user interface the official ChatGPT app is. Maybe they are using those old annotations?

All is revealed!

We get to the bottom of DeButts, on this episode of the Politics Politics Politics.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.politicspoliticspolitics.com/subscribe