Analyzing Language Models In Inferring Author Characteristics

Produced as part of the MATS Winter 2023-4 program, under the mentorship of @Jessica Rumbelow

One-sentence summary: On a dataset of human-written essays, we find that gpt-3.5-turbo can accurately infer demographic information about the authors from just the essay text, and suspect it's inferring much more.


 Introduction.
 
 Every time we sit down in front of an LLM like GPT-4, it starts with a blank slate. It knows nothing[1] about who we are, other than what it knows about users in general. But with every word we type, we reveal more about ourselves -- our beliefs, our personality, our education level, even our gender. Just how clearly does the model see us by the end of the conversation, and why should that worry us?

Like many, we were rather startled when @janus showed that gpt-4-base could identify @gwern by name, with 92% confidence, from a 300-word comment. If [...]

The original text contained 12 footnotes which were omitted from this narration.

---

 First published:
May 17th, 2024

Source:
https://www.lesswrong.com/posts/dLg7CyeTE4pqbbcnp/language-models-model-us

---

Narrated by TYPE III AUDIO.