The Future of Language Models

3min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Open a i announced gpt two in february 20 teen, it originally said it wouldn't release its model because of concerns about malicious use. But before that release, university student conner lehe was able to replicate it using a couple of weeks of effort and some cloud computing credits. Lehe now leads an independent group of volunteer research called a luther a i, which is aiming to create a gpt three sized model. The biggest hurdle, he says, is not code or training data, but computation, which a cloud provider called core weave has offered to provide. One future path lies in combining language models with knowledge bases, curated data bases of declarative facts.

In 2020, the artificial intelligence (AI) GPT-3 wowed the world with its ability to write fluent streams of text. Trained on billions of words from books, articles and websites, GPT-3 was the latest in a series of ‘large language model’ AIs that are used by companies around the world to improve search results, answer questions, or propose computer code.

However, these large language model are not without their issues. Their training is based on the statistical relationships between the words and phrases, which can lead to them generating toxic or dangerous outputs.

Preventing responses like these is a huge challenge for researchers, who are attempting to do so by addressing biases in training data, or by instilling these AIs with common-sense and moral judgement.

This is an audio version of our feature: Robo-writers: the rise and risks of language-generating AI

Hosted on Acast. See acast.com/privacy for more information.