The Dangers Of Relying On Data And Algorithms For Decision Making

With companies making a move towards relying on data and algorithms to make decisions, it is important to remember that there is still a human aspect that has to be considered.

Many organizations are making a move towards relying on data and algorithms for their decision making, but we have to ask if this is always a good thing. The truth is, no matter how calculated and precise these programs may seem, there is still a human programming the algorithms and there is plenty of room for error.

One example of this was included in a book by Cathy O'Neil entitled, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. In her book O'Neil told a story about a school in Washington D.C. that decided to use an algorithm to find the lowest performing teachers in the school. They ended up identifying around 200 teachers who had low performance levels and the school let those teachers go. One of the teachers, Susan, was particularly surprised that she was one of the teachers who was let go as she always had high reviews from students and was well-liked by peers and supervisors.

It turns out that what the algorithm couldn't identify was that the low scores were not Susan's fault. What had happened was the students in Susan's class came from another school where teachers were editing the tests to make it look like the students got all the answers correct. So when those students moved into Susan's class where their answers were not edited, it appeared that their scores were falling drastically.

What we have to understand is we still need a human aspect when it comes to making decisions. Data and algorithms are great, but we still need human input to understand why the data is the way that it is. We cannot simply rely on the data alone.