Psych Tech @ Work Social Media based assessments- Hot or Not? With special guest: Dr. Shawn Bergman
May 8, 2019
33:03
This episode explores the various ways social media information is being used to evaluate job applicants and provides expert opinions on their viability and general acceptance. Special guest Dr. Shawn Bergman’s expertise in the area of social media for hiring, as well as his own research provide a great backdrop for filtering general perceptions about this hot topic. The conclusions drawn provide a useful context for anyone interested in exploring the use of social media data as an assessment tool.
Before looking at the various ways social data is used in hiring, it is important to look at the some of the issues that apply to using social media data to make employment decisions.
Concerns
There are four overall areas of concern when it comes to using social data as an assessment.
1. Problems with source materials- There are a great deal of individual differences in the use of social media. For instance, both Dr. Bergman and Dr. Handler admit that they are not very active on social platforms. It seems logical that a lack of data based on infrequent posting or not using a particular channel- could have a negative impact on an applicant’s evaluation. The potential differences in how much people post and where- represent a challenge when comparing them to others who may post frequently to many channels.
2. Accuracy- The accuracy of social media data in measuring human traits is speculative at best. Without confidence in construct related measures- the value of social data as a predictor of job performance is suspect. Of course it is possible to simply correlate patterns extracted from social data with performance metrics- but this brings us right back to the same issues raised with most AI based evaluation tools. Unfortunately, many companies making social media assessment tools do not involve I/O psychologists - adding additional concerns when it comes to the measurement of actual job related constructs.
3. Privacy- The default assumption with social data in hiring is that the individual being evaluated has not given specific consent for their information to be used. This is probably the most talked about issue when it comes to social media data for hiring, and one that will not go away anytime soon. One of the main problems in the realm of privacy is that it is often impossible for an individual to know their data is being used to evaluate them. While opting in is becoming a standard requirement- it is virtually impossible to police the use of tools that harvest and evaluate social media data. There is also the question of who owns social data. Questions of data ownership can turn the concept of privacy on its ear- requiring legal precedent and legislation to sort out.
4. Bias- There are several ways bias can enter the room when it comes to hiring and social media data. Any process in which profiles are reviewed manually presents a serious snakepit when it comes to bias. Automated tools are also famous for creating systematic biases when making evaluations. While there is some great work being done to train AI/ML to actually reduce bias, the fact remains that social media data helps keep the very real issue of bias alive and well.
Use Cases
Dr. Bergman reports that 70% of organizations use some form of social or public data when evaluating applicants. So how are they using these tools?
Manual review of profiles/posts
While the sexiest and most talked about use case centers around the use of AI based tools to systematically evaluate data from social media accounts and posts, the lowest hanging fruit when it comes to the use of social data for hiring decisions is the simple review of profiles by humans. Social media data is often used for a very low tech and manualized process of reviewing profiles of job candidates to look for inappropriate information, revealing posts, etc. This process is accessible to anyone with a computer, setting up many a disaster when it comes to subjectivity, accusations of immorality, etc.
Human review of profiles for problematic behavior definitely opens up a great deal of concern. It is accessible and there is no accountability for reporting the results of these often ad hoc evaluations. While these evaluations can be outsourced to firms that specialize in the evaluation and return a report, this does not legitimize the method.
The available research in this area shows that there is no relation between these evaluations and performance on the job. Furthermore, this type of easily disadvantages protected classes, and its job relevance is often hard to demonstrate.
AI based tools
The core of all these tools is tech that spiders the web to find profiles and information, scrapes the data, and then interprets and processes it into an output that can be used to support decision making.
The tech can work passively- based on open web searches with no opt ins, or more actively - with applicants opting in to share information that can be used to evaluate the applicant.
Social media data is often used as part of the sourcing process. The most common use case of technology enabled tools is passive sourcing in which social data is used to identify persons who may be a good fit for a particular job so that they can be contacted about an opportunity. This use case presents a number of difficulties because it happens outside of the actual application process. Individuals may not know their data is being used and the tools used to harvest data may do so in a biased manner. These methods are not actually assessments if they aren’t part of the formal hiring process. However, there is accountability in record keeping when it comes to sourcing efforts.
Social media data is also used to create a “super profile” that can be used for hiring. Profiles constructed using social media data most commonly package the data into a personality profile. There are currently many different tools that allow anyone to try creating a personality profile from their social accounts. Both Drs. Handler and Bergman report that when they tried out these tools- the results did not seem accurate.
Research so far has not shown there to be any strong correlation between machine derived personality profiles and those of more traditional methods. These tools have also not shown any real correlations with job performance.
The future is bright
The consensus from the research and experts in the area of social media based assessment is that “we are not there yet”.
As with all advanced technology tools, we can expect to see much improvement in the future. While the technology side of things will definitely advance our ability to make meaningful predictions from social data- the moral and ethical boundaries surrounding the use of these tools will likely remain.
Looking to the future it is important that we understand that technology is a tool, not a solution. When this approach is taken- the tools of the future will be less haphazard and more systematic in nature.
The net of it all is that there are exciting times ahead in the realm of social data and predictive hiring. But getting there is going to require an interdisciplinary approach that recognizes legal and ethical boundaries.
Shawn Bergman is a Professor of Industrial-Organizational Psychology and Human Resource Management at Appalachian State University, focusing on organizational systems, soft-skills training, leadership development, and the application of technology and analytics to solve applied problems.
Dr. Bergman has been inducted into the Appalachian State University Faculty Hall of Fame and received numerous teaching honors and awards, including a Board of Governors Excellence in Teaching Award. He is also the co-founder and Director of Research and Evaluation at the Vela Institute.
Learn more about Shawn and the Vela Institute here: https://velainstitute.org/our-team/shawn-bergman/
See what Appalachian State is building here:
https://iohrm.appstate.edu
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com
Before looking at the various ways social data is used in hiring, it is important to look at the some of the issues that apply to using social media data to make employment decisions.
Concerns
There are four overall areas of concern when it comes to using social data as an assessment.
1. Problems with source materials- There are a great deal of individual differences in the use of social media. For instance, both Dr. Bergman and Dr. Handler admit that they are not very active on social platforms. It seems logical that a lack of data based on infrequent posting or not using a particular channel- could have a negative impact on an applicant’s evaluation. The potential differences in how much people post and where- represent a challenge when comparing them to others who may post frequently to many channels.
2. Accuracy- The accuracy of social media data in measuring human traits is speculative at best. Without confidence in construct related measures- the value of social data as a predictor of job performance is suspect. Of course it is possible to simply correlate patterns extracted from social data with performance metrics- but this brings us right back to the same issues raised with most AI based evaluation tools. Unfortunately, many companies making social media assessment tools do not involve I/O psychologists - adding additional concerns when it comes to the measurement of actual job related constructs.
3. Privacy- The default assumption with social data in hiring is that the individual being evaluated has not given specific consent for their information to be used. This is probably the most talked about issue when it comes to social media data for hiring, and one that will not go away anytime soon. One of the main problems in the realm of privacy is that it is often impossible for an individual to know their data is being used to evaluate them. While opting in is becoming a standard requirement- it is virtually impossible to police the use of tools that harvest and evaluate social media data. There is also the question of who owns social data. Questions of data ownership can turn the concept of privacy on its ear- requiring legal precedent and legislation to sort out.
4. Bias- There are several ways bias can enter the room when it comes to hiring and social media data. Any process in which profiles are reviewed manually presents a serious snakepit when it comes to bias. Automated tools are also famous for creating systematic biases when making evaluations. While there is some great work being done to train AI/ML to actually reduce bias, the fact remains that social media data helps keep the very real issue of bias alive and well.
Use Cases
Dr. Bergman reports that 70% of organizations use some form of social or public data when evaluating applicants. So how are they using these tools?
Manual review of profiles/posts
While the sexiest and most talked about use case centers around the use of AI based tools to systematically evaluate data from social media accounts and posts, the lowest hanging fruit when it comes to the use of social data for hiring decisions is the simple review of profiles by humans. Social media data is often used for a very low tech and manualized process of reviewing profiles of job candidates to look for inappropriate information, revealing posts, etc. This process is accessible to anyone with a computer, setting up many a disaster when it comes to subjectivity, accusations of immorality, etc.
Human review of profiles for problematic behavior definitely opens up a great deal of concern. It is accessible and there is no accountability for reporting the results of these often ad hoc evaluations. While these evaluations can be outsourced to firms that specialize in the evaluation and return a report, this does not legitimize the method.
The available research in this area shows that there is no relation between these evaluations and performance on the job. Furthermore, this type of easily disadvantages protected classes, and its job relevance is often hard to demonstrate.
AI based tools
The core of all these tools is tech that spiders the web to find profiles and information, scrapes the data, and then interprets and processes it into an output that can be used to support decision making.
The tech can work passively- based on open web searches with no opt ins, or more actively - with applicants opting in to share information that can be used to evaluate the applicant.
Social media data is often used as part of the sourcing process. The most common use case of technology enabled tools is passive sourcing in which social data is used to identify persons who may be a good fit for a particular job so that they can be contacted about an opportunity. This use case presents a number of difficulties because it happens outside of the actual application process. Individuals may not know their data is being used and the tools used to harvest data may do so in a biased manner. These methods are not actually assessments if they aren’t part of the formal hiring process. However, there is accountability in record keeping when it comes to sourcing efforts.
Social media data is also used to create a “super profile” that can be used for hiring. Profiles constructed using social media data most commonly package the data into a personality profile. There are currently many different tools that allow anyone to try creating a personality profile from their social accounts. Both Drs. Handler and Bergman report that when they tried out these tools- the results did not seem accurate.
Research so far has not shown there to be any strong correlation between machine derived personality profiles and those of more traditional methods. These tools have also not shown any real correlations with job performance.
The future is bright
The consensus from the research and experts in the area of social media based assessment is that “we are not there yet”.
As with all advanced technology tools, we can expect to see much improvement in the future. While the technology side of things will definitely advance our ability to make meaningful predictions from social data- the moral and ethical boundaries surrounding the use of these tools will likely remain.
Looking to the future it is important that we understand that technology is a tool, not a solution. When this approach is taken- the tools of the future will be less haphazard and more systematic in nature.
The net of it all is that there are exciting times ahead in the realm of social data and predictive hiring. But getting there is going to require an interdisciplinary approach that recognizes legal and ethical boundaries.
Shawn Bergman is a Professor of Industrial-Organizational Psychology and Human Resource Management at Appalachian State University, focusing on organizational systems, soft-skills training, leadership development, and the application of technology and analytics to solve applied problems.
Dr. Bergman has been inducted into the Appalachian State University Faculty Hall of Fame and received numerous teaching honors and awards, including a Board of Governors Excellence in Teaching Award. He is also the co-founder and Director of Research and Evaluation at the Vela Institute.
Learn more about Shawn and the Vela Institute here: https://velainstitute.org/our-team/shawn-bergman/
See what Appalachian State is building here:
https://iohrm.appstate.edu
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com