Mitigating Algorithmic and Human Bias

Transcript

Ross Dawson: Ganna, it is wonderful to have you on the show.

Ganna Pogrebna: Yeah, it’s great to be here. Thanks for inviting me.

Ross Dawson: So you are a behavioral data scientist. Let’s start off by saying, what is a behavioral data scientist? And what does that mean in a world where AI has come along?

Ganna Pogrebna: Yeah, that’s right. That’s a loaded term, I guess—lots of words there. But what that kind of boils down to is, I’m trying to make machines more human, if you will. Basically, making sure that machines and algorithms are built based on our values and things that we are interested in as humans.

So that’s kind of what it is. My background is in decision theory. I’m an economist by training, but in 2013 I got a job in an engineering department, and my professional transformation started from there. I got involved in a lot of engineering projects, and my work became more and more data science-focused.

Now, what I do is called behavioral data science. Back in the day, in 2013, they just asked me, “What do you want to be called?” and I thought, okay, I do behavior and I do data science, so how about behavioral data scientist?

Ross Dawson: Sounds good to me. So unpacking a little bit of what you said before—you’re saying you make machines more like humans, so that means you are using data about human behavior in order to inform how the systems behave. Is that correct?

Ganna Pogrebna: Yeah, that’s correct. I think in any setting—so in a business setting, for example—many people do not realize that practically all data we feed into machines, any algorithm you take, whether it’s image recognition or decision support, it’s all based on human data. Effectively, some humans labeled a dataset, and that normally goes into an algorithm. Of course, an algorithm is a formula, but at the core of it, there is always some human data, and most of the time we don’t understand that.

We kind of think that algorithms just work on their own, but it’s very important to understand that human data is part of the training data for the algorithm, and it carries all the issues that we have with human data. For example, we know that humans are biased in many ways, right? All of these biases actually end up ultimately in the algorithm if you don’t take care of it at the right time.

If you want, I can give you a classic example with the Amazon algorithm—I’m sure you’ve heard of it. Amazon trained an HR algorithm for hiring, specifically for the software engineering department, and every single person in that department was male. So if you sent this algorithm a female CV with something like a “Women in Data” award or a female college, it would significantly disadvantage the candidate based on that. It carried gender discrimination within the algorithm because it was trained on their own human data.

Ross Dawson: Yeah, well, that’s one of the big things, as I’ve been saying since the outset, is that AI is trained on human data, so human biases get reflected in those. The difficult question is, there is no such thing as no bias. I mean, there’s no objective view—at least that’s my view.

Ganna Pogrebna: Absolutely. Yeah.

Ross Dawson: So we talk about bias auditing. All right, so we have an AI system trained with human data, whatever it may be. In this case, with the Amazon recruitment algorithm, you could actually look at it and say, “All right, it’s probably not making the right decisions,” with some degree of explainability. So how do we then debias? Or how do we have an algorithm which is trained on implicitly biased data? Are there ways that we can reduce at least those biases?

Ganna Pogrebna: Yeah, a lot of my work is trying to understand human bias in organizations and trying to offset that with machine decision making, and equally, to understand machine bias and offset it with human decision making. Well, I now have, if you notice, a digital background with books. We’ve done some work with hiring algorithms. If you’re interviewing with a company, a lot of times you have pre-screening done by an algorithm, and in the interview process, you might have some automated interview where you record yourself and send a video. I bet many people have been through this process.

What we found was, we had exactly the same recording of an individual answering questions, but in one case, we put a plain background—everything was shot on green screen—and in another, we put a background with books. The algorithm rated people with books in the background higher on the same questions and answers than the person against the plain background.

So, going back to your point, how do we offset algorithmic problems? First, we need to understand what they are. If we know that an algorithm would rate exactly the same answers differently depending on the background, we should probably tell people to shoot all their answers against a plain background or something like this, to equalize it. So the first thing is understanding where this is coming from. Second is, do you really need an algorithm in the particular case, or can it be done by a simple process?

Finally, you try to understand where the issues are with human decision making and how algorithms can potentially offset them—or is the algorithm making things worse? Because sometimes it does. I think it all comes to boils down to first understanding where the problems are, and then using the two systems—the human systems and algorithmic systems—to offset the issues.

Ross Dawson: Which I think goes back to the point of humans plus AI. Either individually is not necessarily as well designed as a system of both.

Ganna Pogrebna: Yeah, exactly. Oftentimes, organizations don’t have the possibility to implement generative AI or AI systems. If you’re doing all your analytics on an Excel sheet, it’s probably not a great idea to think straight away about implementing AI. But on the other hand, there are some great applications where algorithms can facilitate better, more structured decision making.

I work a lot with executive teams and leaders, and in the majority of cases, they expect precision from algorithmic output. If they put something into ChatGPT or Claude, they expect precise statistics, everything to be impeccably well researched. These tools are completely inappropriate for that. They are good for thinking outside the box.

For example, we were recently hiring people into my team—engineers, software engineers. We had four candidates who came to the interview, and three of them, when we got to the point where we asked, “Do you have any questions for us?”—three people asked exactly the same questions. So what happened is like they went to ChatGPT, asked for questions, memorized them, and gave us exactly what the algorithm told us. The fourth person asked more creative questions. I don’t know whether this fourth person used a different algorithm or just used the algorithm more creatively, but we hired the fourth person because they thought outside the box in terms of what questions to ask us.

You need to be careful, because one of the problems is algorithms can make us all the same. You can tell that by looking at, for example, LinkedIn posts, when they start with, “I’m excited to tell you,” or “I’m so thrilled to inform you.” That’s probably written by ChatGPT, and you know that straight away. But a smart person who understands how algorithms think would structure it differently. They can still use input from the algorithm, but at the same time appear as if the content is unique and nicely positioned.

Ross Dawson: Let’s dig into that. The way I think of it is humans plus AI workflows. There are obviously many sequencings, but one is: you’ve got a human, they’ve got a situation—job interview, decision, whatever—and they use AI to help them. What are the specific capabilities, attitudes, or techniques that people need to use to make sure they’re taking the best of what AI can offer, but also bringing their own unique perspectives, experience, and insights, so that it’s a net positive, as opposed to just echoing what the AI says?

Ganna Pogrebna: I think most of the time people use, at least in my experience, generative AI as a Google search. They just type something, and that’s okay, because that’s how we’ve communicated with technology for many years, since the 90s with Google search. But when you’re talking to generative AI, you need a completely new way of doing that.

You need to first provide the algorithm a lot of context. Tell it, “I’m a founder,” or “I’m a leader in an organization,” or “I’m a CEO, and this is what it’s for—I’m creating a pitch deck,” for example, or “I’m preparing meeting notes.” Give it a lot of context and tell it what it is—”Is it an advisor to you? Is it a coach?”—and only then ask a question. Many people don’t do that. They just ask a question straight away and then say, “Oh, the algorithm gave me this useless answer,” or “It gave me an answer with a lot of false information.”

This is very interesting in terms of hallucinations. First of all, hallucinations are mistakes—they’re not hallucinations. The biggest problem is actually referencing, because a lot of times references do not exist. For example, I do this thing with my executive students: I give half the class fake articles that do not exist, and the other half real articles, and tell them to provide a summary for the next lecture. People come with these summaries, and I can immediately tell who actually verified whether the source existed. The first thing you should do is go to the library, check if the article actually exists, and then try to do the summary. But most people just put the title into ChatGPT, get a summary, and happily submit that summary. That’s a good way to understand that there are limitations.

Ross Dawson: So this takes us to the point of team. We have human teams—a group of people collaborating to create an outcome. Now we have AI in the mix, and this can be thought of in a whole array of ways, including AI as a team member. An AI agent becomes part of the team. Another is the AI can be an assistant. And one of the very interesting things is AI can provide behavioral nudges to the team. So in terms of making a team more effective, more capable, where you have the human members and you’re adding AI, what are the best ways to bring in AI? What are the ways in which we can get better team performance?

Ganna Pogrebna: Yeah, I think the first thing to do is to become better at prompting. You need to understand that when you’re working in a group, you’re not just working with people—you’re working with human-machine teams, because everyone would at least Google stuff before the meeting, I’m assuming. Many people do not realize that when you Google, there are hundreds of algorithms working in the Google search. So what you see is not necessarily chosen by humans; it’s chosen by the algorithm. The output you see at the top of the search is shaped by what algorithms are doing.

In a team setting, that’s particularly important, because different people have different biases and skill sets in terms of coming up with decisions. At board level, for example, I see very little appropriate use of ChatGPT or generative AI tools like Claude. People generally just ask generative AI or an LLM something as if they were talking to Google search, without providing any context, so they’re not using it in the best way.

The best thing to think about is that when we communicate with an algorithm, we normally judge an algorithm on intent, and we judge people on output. For example, if I lied to you, Ross—if I promised something and didn’t do it, like if I said, “We have a recording today,” but I didn’t show up—you would think, “Ganna is probably not a very reliable person.” That would have an immediate effect on my reputation. That’s not how people judge algorithms, because an algorithm can provide you with wrong information and you would still trust it again if I tell you, “Oh, we’ve improved it, it’s a new version, you should try it again.” That’s what OpenAI does all the time, and equally, other developers of generative AI.

But in a group meeting, it’s your reputation at stake. If you come and provide some evidence that doesn’t really exist, people can look it up, and it will have an immediate effect on you as an individual. That’s something to keep in mind. Generally, in terms of how to get better, try to get very proficient with prompting, provide context to the algorithm, tell it what you want as an output, and remember that this is a brainstorming tool. It’s not an advisor or a person who will give you statistics or something very precise. Keep in mind that it doesn’t understand what it’s saying. Many people think it’s another human, but it’s not. These models are trained on lots of data, but they don’t know what they’re saying.

You can see that very well in generating visuals—very often you get people with three hands or diagrams that don’t make sense, with repeated words, and it’s just because it doesn’t know what it is. So just keeping that in mind helps.

Ross Dawson: This actually goes back to what you said at the very beginning, which is using human behavioral data in order to make the AI perform more like machines. But in there, there is a danger—if we make the machines seem very much like humans, as we’re heading at the moment, then that’s often not useful if you’ve got a human plus AI team, where you should be treating the AI and the humans differently. But if the AI is behaving very much like a human, then it’s harder. Shouldn’t we be designing the AI systems so that they are distinctive from humans, as opposed to mimicking human behaviors?

Ganna Pogrebna: Well, they are already distinctively different, because machines do not think in the same way we do. For example, we do a lot of research on developmental learning versus machine learning. To teach a kid what a duck is—well, I just have this on my table because my son is using it—that’s a duck, right? It takes a human to see a duck one time, and then you see something in the shape of a duck and you can make associative connections in your brain. The machine needs to see a duck millions of times to learn that this is a duck. So in terms of thinking, that’s already quite distinct.

The problem is not in how we design machines, but in how we teach humans to understand that this is a machine, and you’re talking to a machine. Here, I’m an optimist. I really think we will figure it out. A few years back, Google released their first chatbot assistant—you would call an assistant in a hairdresser shop, and it would respond to you. People remember that, and people were not able to understand that they were talking to an algorithm. Now, we talk to algorithms a lot. We talk to algorithms when we call a bank, when we call an airline, for example. We talk to algorithms all the time, and we can recognize that’s an algorithm. With experience, we will develop those skills.

I think the competitive advantage of companies will be actually offering real people versus algorithms. So I think the problem is not so much in the development, but in the way we communicate with algorithms. Think about all the influencers we have online that are actually fake, that do not exist. We have digital models on Instagram, AI-generated YouTube videos with people that do not exist. Some people believe that’s real, but others who have more exposure and experience understand, “Oh, this person is sitting, not moving much, not turning their head,” and all that kind of stuff—so that’s probably a deep fake, not a real person. But that only comes with experience, and it’s okay to make mistakes. I don’t think it’s a development problem; it’s really our perception of machines and the way we communicate and collaborate with them.

Ross Dawson: One of the things we’re particularly interested in with humans plus AI is complex decision making and strategic decision making. Your classic example is a board or executive team. What are structures, architectures, approaches, or tools where AI can augment what are, of course, human-first decisions?

Ganna Pogrebna: There is lots of stuff available at the moment, from the usual generative AI inputs that we’ve already discussed. By the way, I can recommend a book—not one I wrote personally, but by a guy called David Boyle, who used to be an executive at the BBC. He has a very nice book called “Prompt.” If you want to understand how to prompt for behavioral segmentation or understanding stakeholders, there are some really good tips in that book on how you go from step one to step twenty-five to get good output.

Apart from generative AI tools, my personal bias is that I’m really excited about simulation tools like digital twinning, particularly because, as you know, we are running out of data. We need more and more data to train algorithms, and you’ve probably noticed in the literature that people say algorithms are becoming dumber, making people dumber as well. The problem is that we just don’t have as much data to feed algorithms to train them better, and a lot of output—we call it “data inbreeding” in scientific literature. We generate a lot of output or content and post it online using generative AI, feed it back into the system, and it gives us worse and worse results. Eventually, we will completely run out of this data if we don’t have humans talking to machines more—labeling more datasets and so on.

Simulation tools are really powerful if you properly collect data. You can simulate, for example, how customers will respond to a product, simulate different outcomes of your decision making, and in your supply chain. My personal bias is using digital twins—I’m really passionate about this. These are powerful tools that allow you to simulate scenarios of what will happen in the future. Many people are not familiar with what they are. Usually, you see just some 3D model of a city and people think that’s a digital twin—it’s not. I want to make a clear distinction: there are digital twins and digital shadows. If we make a holographic replica of me and put it here, that would be a digital shadow, because the data only flows one way—this is real Ganna, and this is digital Ganna. But if we simulate what I would do in different situations, then it becomes a twin, because it gives us different outputs in scenarios that haven’t necessarily happened, but you can simulate them.

Ross Dawson: So if we have a good simulation of what people would do, and there’s increasing data that very well-trained simulations are within 90% of the behavior of the original person used to train it, how specifically do we use that in decision-making contexts? Do we have two chairpersons on a board? Do we simulate our stakeholders? Do we simulate consumers? What are the most useful ways in which we can use these simulations for better decisions and action?

Ganna Pogrebna: Let’s take a specific example with stakeholders. Maybe I’ll give you an example from what I’ve done. We were working with a really large media corporation that was trying to figure out—I’ve worked a lot on movie content, for example, models predicting revenue of films using just the script. A lot of times, you’re trying to figure out what content to produce for what type of stakeholders, how to strategically allocate your portfolio between projects.

We were working with a large corporation trying to figure out how to invest in different types of content, and they were completely missing out on one particular stakeholder group—people between the ages of 20 and 50 who were really fans of fantasy-type stories. We actually found that stakeholder group for content production just by doing simulations, because previously, if you’re familiar with marketing work, most marketing in entertainment and media is done by age group. For example, they would produce a TV show for men aged 20 to 30—that would be the typical way of thinking about it. But very often, they do not look at behavior—what these people like—because you can have demographically exactly the same people, but liking different things.

Instead of looking at demographic characteristics, we looked at six months’ worth of behavior and discovered that there are quite a lot of these fantasy fans. As a result, this company produced a content project, and it was one of the most successful projects they’ve ever done in terms of revenue. That was done purely by feeding customer behavioral data into an algorithm, which would give us the potential output of what features of a product these stakeholders would be interested in. We fed that back into the production teams, and it was a constant loop of testing, simulating, and talking to customers.

A very important thing to remember: I was recently at a panel where someone did some analysis of transport systems and told a huge audience that soon we will not need customer pulses, we will just completely simulate everything. Very bad idea. You really need to talk to real customers somewhere in between, because you always want to know what your customer thinks. Never simulate 100% of your output—always base it on actual behavior. But if you have good data on behavior—not necessarily a lot of data, just high-quality data on what people actually do—then you can create really powerful simulations that will completely change your value chain and deliver really good results.

Ross Dawson: There was a very interesting Stanford study last year where they had some of the best behavioral correlation, based on two-hour interviews with individuals, in order to build an AI simulation of them. But when you say behavioral data, obviously it’s context-specific, depending on what you’re trying to simulate. Let’s say in an organizational context, not so much in a consumer context. I know that some leaders of large organizations have created simulations or digital twins of themselves in order to provide coaching and first call—instead of people calling them first, they call their digital twin first.

Ganna Pogrebna: When I was traveling, for example, when I was an exec director, you get a lot of emails. When I was traveling and knew I was going to be on a plane, you would first get an email—very politically correct, polite—of what I would normally say in the first instance, and then to talk to a person in depth, I would obviously use my PA first, and then I would talk to people. So I completely understand why people do that.

Ross Dawson: So what data do you use then to train them? There’s enough public information on Ganna Pogrebna to be able to say, “Okay, AI, create a simulation of you just based on public information.” That would be one level. What data do you want to get to better, more effective responses?

Ganna Pogrebna: For personal twinning, that would be more a shadow product than a twin, because it doesn’t really simulate my behavior, but just responds to messages and things like that. All you need is actually public information—not necessarily public, but direct speech information. It could be your emails.

A very good example is a guy at Georgia Tech University—I forgot his name, I think his last name is Goyal. You can find the TED talk about him. He was a professor at the university and taught huge classes. I understand the problem myself, because when I was a professor at the University of Sydney, I taught classes of 1,000 or 1,700 people—huge numbers, and everyone emails you, so you don’t have a life because you have to respond quickly. My solution was to have a Snapchat and just send people yes or no answers—”Send me a yes or no question and I will respond.” But what he did was notice that all these queries from students were exactly the same—90% of questions were the same year after year.

So he took all his emails from students and his answers, and trained an algorithm called Jill Watson, using IBM Watson as a basis. And basically Jill Watson responded to student emails when they we’re writing  an email to a professor, and at the end of the first year, Jill Watson was nominated as TA of the year at Georgia Tech University, because the algorithm was really good.

So if you just have a lot of direct speech—emails, maybe—you also need to be very careful to train your model confidentially in a closed environment. Don’t train it on open source if you’re dealing with confidential customer or board data. But any direct speech, like minutes from board meetings, can be fed into an algorithm, and it will provide you with pretty good trained data to train a good algorithm.

Ross Dawson: Fantastic. So where can people find out more about your work?

Ganna Pogrebna: I’m on all social media, so if you can’t find me, I guess it’s your fault, because it’s very easy. I’m probably most active on LinkedIn, so that’s a good place to start. Generally, I think there is a lot of work in the public domain in terms of books and other things. I recently wrote a book on bias, which systematizes 202 human biases. Because I work in human behavior, I tend to work on a wide variety of applications, so it’s quite easy to connect with any part of my work, because it’s relevant to many different areas. But yeah, just Google me—probably don’t ask generative AI, because you might get some fake things that are not true.

Ross Dawson: Well, it’s very high potential work. I think it’s wonderful to be able to bring in this behavioral lens—it’s critically important. Thank you so much for your work and your time and sharing today.

Ganna Pogrebna: No worries. Thanks a lot. Just to finish, maybe I’ll leave you with a thought: many people think about AI systems as terminators because they want to control the machines. But if you embrace the fact that we are already dependent on technology in many ways and try to collaborate with it, you may find lots of benefits for yourself and your business. So I highly encourage you to just try.

Ross Dawson: Thank you.

Ganna Pogrebna: Thanks a lot.