Psych Tech @ Work Game Based Assessments for Hiring - What you need to know! - S4H003
Feb 5, 2019
39:48
Ben Hawkes is an I/O psychologist who is a thought leader in area of game based pre-employment assessment tools. Ben is currently the co-Founder of BlackHawke Behavior Science and has served as the Selection Assessment Lead for Shell International since 2016.
Ben’s background and experience make him a perfect resource for helping Dr. Handler answer the question,
"Are game-based assessments the future of employment testing?"
The short answer as revealed in this episode is: Don’t believe the hype!
Game based assessments have a ton of value and are are PART of the future, but they have the same limitations as traditional assessments (if not more). Game based assessments will make a positive and essential contribution to the future of assessments because they will help us learn how to make assessments more lifelike and enjoyable.
This episode touches on few key points to help educate and orient those interested in game based assessments
1. Gamified assessments are not true games
It is important to know what you are looking at. It is important to be careful not to believe the hype that a gamified assessment is really a game. Doing so is made harder because it is difficult to say exactly what is a game.
Games have to meet certain requirements such as the presence of rules, an enjoyment factor, suspense, and control by the player. But there are grey areas around the true definition and it is common for things that have game like elements to be called games when they really aren’t.
Gamified assessments- that is assessments with game like elements, are becoming increasingly common and are more likely to be an integral part of the future of assessment.
Gamification has many advantages because they make assessments more enjoyable but allow them to maintain the characteristics of a good measurement tool. This is essential because when all the dressing is stripped away- assessments must be psychometrically sound measurement devices.
Assessments that provide gamified elements such as: feedback, choices, dynamic user interfaces, and realistic environments definitely have a lot going for them. The upgraded user experience provided by gamification means we can expect to see an increase in their use. Just know they are not truly games and that gamification alone is not enough to make an assessment legit.
2. Where does the risk lie?
A good rule of thumb when looking at game based assessments is don’t buy based on the sizzle because the steak may taste like crap.
For those shopping they first need to know that to be usable as an assessment a game must meet the minimum standards of being fair, reliable, and valid. Many tools may look like or be called an assessment but may not fit these minimum requirements.
One type of game that is commonly mistaken for real assessment is the “attraction games”. These are branded experiences that allow the job seeker to interact with a job or organization. They are great for employment branding but typically are not designed to be measurement tools.
Another type of game that must be approached with caution are “non-contextualized” games. These games take place in a simulated environment such as outer space or under the ocean. While these are sold as being attractive to applicants they may actually have the opposite effect.
Candidates actually value assessments that appear job relevant more than they value being entertained by the experience. In the world of candidate experience, fairness is king and candidates value job relevance over fun.
Personality tests alone, be they games or regular measures, are not strong predictors of work performance. So it is not a surprise that personality games have struggled to be effective as selection tools and their ability as strong predictors of applicant performance should be met with a healthy dose of skepticism. Yes, it is possible for games to measure aspects of personality but they have not proven to do so with at the accuracy level of more traditional assessments.
Many games, including those that claim to measure personality, are sold based on the number of data points they generate about an applicant. These data points are called “paradata” and they are the byproduct of all the actions within the gaming experience. The sheer number of data points does not mean an assessment is a good measurement tool. Measuring constructs that underlie work performance requires understanding what you are measuring first and foremost and then creating accurate and reliable measurement tools. Just because you have millions of data points does not mean the patterns that belie personality are found within it.
Finally- games may not meet accessibility standards required of selection tools. It is important to fully evaluate any assessment games for compliance to ADA standards.
3. So what can we trust?
Gamified assessments are great as long as they are held to the same standards as regular assessments. Currently the strongest type of assessment games are cognitive games. These games have many advantages over regular cognitive assessments they are more engaging, work better than traditional assessments on mobile devices and early research shows that they actually work better than regular cognitive assessments while having less adverse impact.
Cognitive games are also very good at measuring multi-tasking which is becoming increasingly important for many jobs.
3a. We can trust that games will be an important part of the future of assessments
In the future games will help measure things that are not easily measured by static assessments. We can expect to see an increase in the use of gamification techniques to make games increasingly more enjoyable. We can also expect to see games become more like simulations that mimic the real workplace. We can also expect to see a modular approach where many mini games that measure specific traits are packaged together in a branded wrapper. We can also expect to see more Virtual and Augmented reality based assessment games.
Are games the future? They are definitely part of it as they offer many attractive elements that will help talent assessments evolve.
Visit www.blackhawke.io to learn more about Ben and his work supporting venture capitalists (VC) and other investors by analyzing the psychological strengths and weaknesses of entrepreneurs and startup teams. BlackHawke equip VCs with insights and guidance to minimize conflict, address the inevitable 'people issues', and accelerate startup growth. And connect with Ben on LinkedIn: https://www.linkedin.com/in/benhawkes/ or Twitter: https://twitter.com/WorkPsy or even email him at Ben(at)yacmo.com
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com
Ben’s background and experience make him a perfect resource for helping Dr. Handler answer the question,
"Are game-based assessments the future of employment testing?"
The short answer as revealed in this episode is: Don’t believe the hype!
Game based assessments have a ton of value and are are PART of the future, but they have the same limitations as traditional assessments (if not more). Game based assessments will make a positive and essential contribution to the future of assessments because they will help us learn how to make assessments more lifelike and enjoyable.
This episode touches on few key points to help educate and orient those interested in game based assessments
1. Gamified assessments are not true games
It is important to know what you are looking at. It is important to be careful not to believe the hype that a gamified assessment is really a game. Doing so is made harder because it is difficult to say exactly what is a game.
Games have to meet certain requirements such as the presence of rules, an enjoyment factor, suspense, and control by the player. But there are grey areas around the true definition and it is common for things that have game like elements to be called games when they really aren’t.
Gamified assessments- that is assessments with game like elements, are becoming increasingly common and are more likely to be an integral part of the future of assessment.
Gamification has many advantages because they make assessments more enjoyable but allow them to maintain the characteristics of a good measurement tool. This is essential because when all the dressing is stripped away- assessments must be psychometrically sound measurement devices.
Assessments that provide gamified elements such as: feedback, choices, dynamic user interfaces, and realistic environments definitely have a lot going for them. The upgraded user experience provided by gamification means we can expect to see an increase in their use. Just know they are not truly games and that gamification alone is not enough to make an assessment legit.
2. Where does the risk lie?
A good rule of thumb when looking at game based assessments is don’t buy based on the sizzle because the steak may taste like crap.
For those shopping they first need to know that to be usable as an assessment a game must meet the minimum standards of being fair, reliable, and valid. Many tools may look like or be called an assessment but may not fit these minimum requirements.
One type of game that is commonly mistaken for real assessment is the “attraction games”. These are branded experiences that allow the job seeker to interact with a job or organization. They are great for employment branding but typically are not designed to be measurement tools.
Another type of game that must be approached with caution are “non-contextualized” games. These games take place in a simulated environment such as outer space or under the ocean. While these are sold as being attractive to applicants they may actually have the opposite effect.
Candidates actually value assessments that appear job relevant more than they value being entertained by the experience. In the world of candidate experience, fairness is king and candidates value job relevance over fun.
Personality tests alone, be they games or regular measures, are not strong predictors of work performance. So it is not a surprise that personality games have struggled to be effective as selection tools and their ability as strong predictors of applicant performance should be met with a healthy dose of skepticism. Yes, it is possible for games to measure aspects of personality but they have not proven to do so with at the accuracy level of more traditional assessments.
Many games, including those that claim to measure personality, are sold based on the number of data points they generate about an applicant. These data points are called “paradata” and they are the byproduct of all the actions within the gaming experience. The sheer number of data points does not mean an assessment is a good measurement tool. Measuring constructs that underlie work performance requires understanding what you are measuring first and foremost and then creating accurate and reliable measurement tools. Just because you have millions of data points does not mean the patterns that belie personality are found within it.
Finally- games may not meet accessibility standards required of selection tools. It is important to fully evaluate any assessment games for compliance to ADA standards.
3. So what can we trust?
Gamified assessments are great as long as they are held to the same standards as regular assessments. Currently the strongest type of assessment games are cognitive games. These games have many advantages over regular cognitive assessments they are more engaging, work better than traditional assessments on mobile devices and early research shows that they actually work better than regular cognitive assessments while having less adverse impact.
Cognitive games are also very good at measuring multi-tasking which is becoming increasingly important for many jobs.
3a. We can trust that games will be an important part of the future of assessments
In the future games will help measure things that are not easily measured by static assessments. We can expect to see an increase in the use of gamification techniques to make games increasingly more enjoyable. We can also expect to see games become more like simulations that mimic the real workplace. We can also expect to see a modular approach where many mini games that measure specific traits are packaged together in a branded wrapper. We can also expect to see more Virtual and Augmented reality based assessment games.
Are games the future? They are definitely part of it as they offer many attractive elements that will help talent assessments evolve.
Visit www.blackhawke.io to learn more about Ben and his work supporting venture capitalists (VC) and other investors by analyzing the psychological strengths and weaknesses of entrepreneurs and startup teams. BlackHawke equip VCs with insights and guidance to minimize conflict, address the inevitable 'people issues', and accelerate startup growth. And connect with Ben on LinkedIn: https://www.linkedin.com/in/benhawkes/ or Twitter: https://twitter.com/WorkPsy or even email him at Ben(at)yacmo.com
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com