Psych Tech @ Work

Think using assessments in a tight labor market is a bad idea? Think again! In this episode of Science 4-Hire, Dr. Charles Handler and Bas van de Haterd discuss how well-designed talent assessments improve hiring and fight bias in a tight labor market. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

Selection scientists and recruiters must work as a team to find the best candidates, but their efforts often conflict. Recruiters often want to bring more people in, while pre-hire testing works to screen them out. But the data paints a compelling case for employee selection — and my guest today has no shortage of data.On today’s episode of Science 4-Hire, I’m speaking with David Futrell, senior director of assessment and selection at Wal-Mart, fellow IO psychologist and, frankly, a selection geek extraordinaire. “The sample sizes are just enormous,” David says, and he’s not kidding — Wal-Mart tests 10,000 to 15,000 people every day, or about 4 million annually. And that’s just for entry-level roles.With all of the data that generates, David and his dedicated team of IO psychologists have been able to run the numbers and come to some fascinating conclusions about assessments and employee selection. Here’s what they’ve learned.It’s not That It Isn’t Fun; It Just Doesn’t Seem RelevantWal-Mart’s selection assessments include game-like elements but aren’t gamified. When Wal-Mart bought a company that used a game-based assessment, David surveyed applicants to find out how they liked it. To get at the truth, he chose his words carefully. He didn’t want to learn whether candidates enjoyed it (most did). He had to discover how they felt about the game determining whether they got the job. And he discovered that most applicants trust a traditional personality test as a better, fairer measure of job-related factors.We’ve Turned Assessment Length Into a BoogeymanThe idea that candidates will bail on a test that lasts more than five minutes is more prevalent than it is true. David’s abundance of data shows that well over 90% of candidates will finish an assessment of any length. He even measured and compared completion rates based on when candidates drop out of the process.How many candidates would get 15 minutes into a test and then drop out, for example? Versus candidates who drop out after 30 minutes? The difference is negligible. But cutting your test to fit an arbitrary amount of time can negatively affect its validity, which does have a significant effect on its usefulness. David’s research has also demonstrated that the applicants who dropped out before taking the assessment would have had significantly lower performance on the job compared to those that chose to participate in the assessment.Selection is the Natural Enemy of Recruiting. Or is It?Leveraging his wealth of data has allowed David and his team to make a compelling case for employment testing’s role in performance. Looking at the mountains of data he has collected from different locations, he’s discovered that stores that rely on assessment to make better, more objective hiring decisions perform significantly better than stores that hire randomly.And that translates into a ridiculous amount of money saved. According to David, improving retention by making better hires saves hundreds of millions of dollars in turnover and lost work each year. That’ll make any executive a believer in assessments!People in This EpisodeCatch David Futrell on LinkedIn This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

In this episode I speak with Linda Frietman- one of the founders of IamProgrez a talent assessment firm located in the Netherlands.Linda’s core mission is to create pathways that allow people to show who they are through the data they create- and her firm is doing this in a most novel and interesting way.While there are many firms out there who offer game based assessments- Linda and her company have turned the traditional model for gaming assessments on its ear by using data from existing entertainment game play to measure soft skills and predict job fit.To help my skeptical mind process the legitimacy of IamProgrez’ approach - much of the episode centers around the origins and the technical aspects of the development and use of their assessment methodology.The origin story---Linda relates that a few years ago - as her team was developing some new assessment tools on a tight deadline- she observed her dev ops team playing an online game during a much needed sanity break. As she observed the gamers interacting within the game environment, Linda clearly saw that the skills needed in-game were very similar to those needed to effectively perform jobs in the real world. From this critical moment forward- Linda has hitched her firm’s destiny to extracting trait based measures from commercially available game play.“Game Dynamics” is the rosetta stone-- IamProgrez focuses on the “Steam” gaming platform which functions somewhat like the Netflix of games, and has become a de facto social network for gamers. Steam has thousands of games that are accessible via one subscription. The part of the steam platform that opens the door to assessments is the collection of game play metrics that are standardized across all the games on the platform (AKA “Game Dynamics). While many game based assessments attempt to extrapolate traits from data such as use metrics such as cursor movement and time between clicks- Game Dynamics data packages game play behaviors into more interpretable metrics that are transportable and transcend any one specific game.Construct validation of soft skills & the Dutch militaryWithin games- players make many choices from what kind of character they want to play, to the extent to which they lead or follow others. All of these choices can be tracked in Game Dynamics. With a staff of ½ psychologists and ½ gamers - IamProgrez has been mapping Game Dynamics to soft skills by giving gamers standard personality and soft skills assessments and examining the relation between these inventories and their game play data. This effort has been successful in demonstrating the construct validity of the game play metrics as soft skills measures. Like many other countries- the Netherlands is facing a skills crisis in that the pace of learning institutions preparing students for the workforce is not keeping up with technological demands required on the job. It has also been hard for more established companies to connect with the emerging workforce. Facing an increasing generation gap and an aging force- the Dutch Military put out a call for assistance in helping them engage the younger side of the talent pool. Linda’s team answered the call and has been deeply engaged with the military in constructing programs that use gaming and game play data as a recruiting tool that also measures important work related skills. While still in its infancy- the program has been a big success in helping the military speak the language of the millions of gamers who are so critical to building their future ranks.Where do we go from here?IamProgrez is now working to define the soft skills that are needed for specific jobs/roles so they can connect these profiles to game play data and build out pathways from game play to prediction of job success. This program is now under development and will eventually allow the concept of using mainstream gaming data as a predictor to go to the next level.We close our conversation by discussing the future plans for this innovative approach, including using it to help level the playing field for those who may not come from traditional backgrounds. This methodology has the potential to open access to millions of young people who may not be enticed, evaluated, or engaged by more traditional methods- but who have all the tools needed to make a meaningful contribution.You can find and follow Linda on LinkedIn:https://www.linkedin.com/in/linda-frietman-7401a5b/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

“Persona” (a documentary recently launched by HBO Max) is a misinformed, slightly schizophrenic film with an identity crisis. The film provides a multifaceted view on personality testing that never fully ties its various threads together coherently enough to make a credible and cogent argument about the subject matter. What the film does manage to do rather well is to paint IO psychologists and employment testing in an extremely negative light that completely misses the essence and prime directive of our profession.This episode of Science 4-Hire uses the film’s negativity as a vehicle to accentuate the positive truth about what IOs do for both individuals and organizations. We begin this mission by providing the IOs featured in the film: Ben Dattner (Dattner consulting), David Scarborough (Western New Mexico University- formerly of Unicru), and Nathan Mondragon (HireVue) with an opportunity to share the parts of their talk tracks that were left on the cutting room floor. The bulk of my guests’ wisdom about the subject matter was trashed by the producers in order to support the film’s direct and deliberate assertion that personality testing is a dangerous tool that was created by corporations to enslave individuals in an effort to maximize profits.From there we shed some light on many of the other negative delusions that the film puts forth, including:The idea that the Meyers-Briggs is a legitimate tool for employment testing The position that the employment tests created by IO psychologists are instruments of eugenics sponsored by heartless corporations The opinion that personality tests unfairly discriminate against persons with disabilities The notion that pre-hire testing is the main source of bias in the hiring process At the end of the day- all four of us relish the opportunity to proudly and confidently share our strong conviction that our field makes the world a better place by helping people find meaning in their lives through their work, while at the same time helping organizations of all shapes and sizes reap the benefits of an engaged and happy workforce. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

In this episode Charles welcomes fellow psychologist and personality expert Dave Winsborough for a fascinating and contemporary discussion about the cutting edge of personality testing - with a focus on critiquing AI based personality measures.Mutual agreement on the value of personality assessment for both life and work provides a foundation for the discussion. The discussion proceeds to the good stuff- the use of “novel signals” (i.e., facebook likes, sociometric badges, voice, text, etc.) to extract personality profiles.Both Dave and Charles agree that we are still not ready to bless these measures as ready for prime time when it comes to personnel selection. While there is expectation that we will get there- at present foundational issues such as reliability- are significant limitations to the use of AI based personality tools for predictive hiring.At present this part of the industry is being fueled mostly by hype and wishful thinking- both of which seem to be working when it comes to convincing companies to adopt these tools. This trend creates some concern for a future where true psychological measurement will become irrelevant.Here in our present day reality- Dave discusses his role as a founder of “Deeper Signals” a company that offers an innovative personality assessment tool designed to support a deep level of self insight. The assessment is available to anyone - and offers a really unique interactive report. For those who wish to follow Dave, look for him on LinkedIn (https://www.linkedin.com/in/davewinsborough/) or through the Deeper Signals website.Until next time, we encourage our listers to check out the exciting things going on at Rocket-Hire:2020/2021 Market Trends Report (to be released in Dec. 2020)- We profiled and categorized over 250 vendors of predictive hiring tools and assessments - collecting data on over 25 research parameters. This data allowed us to unlock the data to identify market trends that will surprise you! Sign up for your free copy of the report here: https://go.rocket-hire.com/assessment-platform-marketLaunchPad test authoring platform- When we struggled to find a test authoring platform that is flexible enough to meet our needs - we created our own. Our platform is infinitely flexible- allowing us to build the platform out to meet your exact needs. We can support global testing programs securely at scale. Check out Launchpad at: https://www.launchpad-testing.com/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

In this episode Charles welcomes Cathy O’Neil- founder of ORCAA, a consultancy that helps companies and organizations manage and audit algorithmic risks. Cathy is a mathematician and former investment banker whose goal is to help ensure algorithms are used responsibly and fairly, advocating for and addressing the concerns of those who are on the receiving end of the algorithm (vs. those who created it). Cathy also works with regulators and lawmakers in the course of developing standards for algorithmic auditing, including translating existing fairness laws into rules for algorithm builders.Besides having a genius title- Cathy’s book- Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy provides an in-depth look at the potential damage that is caused by the irresponsible use of algorithms and is a must read for anyone who wants to understand the big picture when it comes to algorithms gone bad.Charles and Cathy engage in a conversation about the dangers of the science hype factor in AI and the fact that irresponsibility begins with the blind faith that people generally put in the concept that algorithms are created by really smart people who can be trusted to ensure they work as intended and do not bring harm. Charles brings up the fact that the primary focus in the world of employment testing is evaluating the fairness of a specific test without consideration of the impact caused by all of the other decision points in the hiring process. There is agreement that this viewpoint leaves a good deal of opportunity for fairness on the table. Cathy explains that her process goes beyond just looking at bias - and begins with the broader question of why the algorithm was created in the first place and how the creator defines its success. There is discussion of the pending case in which employers were sued by the father of a bipolar teen who was denied employment due to the use of a personality test. The discussion centers on the appropriateness of personality tests for the workplace given the fact that they can be discriminatory for those with personality disorders that may not preclude the applicant from performing the job for which they are applying.Finally- Charles and Cathy agree that when it comes to hiring- sourcing is one of the biggest causes of unfair practices. Specifically- there is great concern around the manner in which automation determines whom to show specific job openings returned from a search or presented via paid advertisements. These “optimization engines” are trained to be biased by their very nature and this bias can exclude minorities or members of protected classes from ever seeing jobs for which they may want to apply. Regulation is badly needed in this area because it is currently a silent killer of hopes and dreams.For those who wish to follow Cathy - here blog is https://mathbabe.org/, and her company’s website is: https://orcaarisk.com/ Check out the exciting things going on at Rocket-Hire:2020/2021 Market Trends Report (to be released in Dec. 2020)- We profiled and categorized over 250 vendors of predictive hiring tools and assessments - collecting data on over 25 research parameters. This data allowed us to unlock the data to identify market trends that will surprise you! Sign up for your free copy of the report here: https://go.rocket-hire.com/assessment-platform-marketLaunchPad test authoring platform- When we struggled to find a test authoring platform that is flexible enough to meet our needs - we created our own. Our platform is infinitely flexible- allowing us to build the platform out to meet your exact needs. We can support global testing programs securely at scale. Check out Launchpad at: https://www.launchpad-testing.com/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

Are we learning yet?  Only your data knows- so you’d better pay attention to it.Data- In this episode Charles welcomes Eric Surface, fellow I/O Psychologist and founder/CEO of ALPS Insights, a training analytics company that provides tools and advice to help companies extract value from their learning programs via actionable insights based on data.Charles and Eric both share a passion for helping their clients achieve great things through data.  Their conversation covers the gamut of issues related to working with clients and their data- focusing on the inevitable challenges that come with collecting and managing client data.  Eric is an expert in learning outcomes and the data that provide them- so the main topic of conversation focuses on various methodologies for helping clients unleash the power of their training data and the key role that software plays in this endeavour.  Being an employee selection specialist, Charles drives the conversation towards an exploration of how training and hiring data can complement one another.For those listeners who wish to follow Eric and his company’s work:CHECK OUT THE ALPS Ibex product at the 2020 HR Technology Conference and Expo virtual booth:https://app.swapcard.com/event/hr-technology-r-conference-and-exposition/exhibitor/RXhoaWJpdG9yXzE5Nzc2Mw%3D%3DContact/Follow Eric:Dr. Eric A. SurfaceFounder, CEO and Principal ScientistALPS Insights, Inc.2409-107 Crabtree BlvdSuite 304Raleigh, NC, 27604919-706-0716, ext. 701esurface@alpsinsights.comwww.alpsinsights.com@easphd @ericatehereCheck out the exciting things going on at Rocket-Hire:2020/2021 Market Trends Report (to be released in Nov. 2020)- We profiled and categorized over 250 vendors of predictive hiring tools and assessments - collecting data on over 25 research parameters.  This data allowed us to unlock the data to identify market trends that will surprise you!  Sign up for your free copy of the report here: https://go.rocket-hire.com/assessment-platform-marketLaunchPad test authoring platform- When we struggled to find a test authoring platform that is flexible enough to meet our needs - we created our own.  Our platform is infinitely flexible- allowing us to build the platform out to meet your exact needs.  We can support global testing programs securely at scale.  Check out Launchpad at:  https://www.launchpad-testing.com/Charles and Eric both share a passion for helping their clients achieve great things through data. Their conversation covers the gamut of issues related to working with clients and their data- focusing on the inevitable challenges that come with collecting and managing client data. Eric is an expert in learning outcomes and the data that provide them- so the main topic of conversation focuses on various methodologies for helping clients unleash the power of their training data and the key role that software plays in this endeavour. Being an employee selection specialist, Charles drives the conversation towards an exploration of how training and hiring data can complement one another.For those listeners who wish to follow Eric and his company’s work:CHECK OUT THE ALPS Ibex product at the 2020 HR Technology Conference and Expo virtual booth:https://app.swapcard.com/event/hr-technology-r-conference-and-exposition/exhibitor/RXhoaWJpdG9yXzE5Nzc2Mw%3D%3DContact/Follow Eric:Dr. Eric A. SurfaceFounder, CEO and Principal ScientistALPS Insights, Inc.2409-107 Crabtree BlvdSuite 304Raleigh, NC, 27604919-706-0716, ext. 701esurface@alpsinsights.comwww.alpsinsights.com@easphd @ericatehere This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

The pandemic is reshaping our entire world- accelerating changes that were already in the works and creating the need for entirely new and unplanned adaptations.Join Charles and his guest, William Tincup as they explore the impact of the pandemic on humans and the organizations for whom they work (or are attempting to work for).Tincup’s shares his experience from the front lines of recruiting and HR technology during the pandemic.The conversation focuses on a range of topics that are defining the new normal for employers and workers- identifying the trends that are here to stay and speculating on their long term impact. Topics include:The labor market during and post COVID Remote work- including the dream of working remotely from BarbadosTechnological innovation driven by the pandemicThe Human experience at workJoin two dyed in the wool optimists as they find a silver lining and look to how the lessons and pressures of the pandemic will be viewed from the rear view mirror.Follow William Tincup!Podcast- https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5idXp6c3Byb3V0LmNvbS85NTg1MzcucnNz?sa=X&ved=2ahUKEwjajrS57vLrAhVaU80KHagDB-kQ4aUDegQIARAC&hl=enLinked in- https://www.linkedin.com/in/tincup/Twitter- @williamtincupWebsite- https://recruitingdaily.com/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

Interest in and the adoption of Artificial Intelligence (AI) in the talent acquisition space has skyrocketed since 2015. During this episode of Science-4 Hire, Dr. Charles Handler seeks to learn more about how AI-based hiring platforms work- with a specific focus on job matching. Eightfold.AI is doing just that, and much more in pursuit of the company’s core mission to find the right career for everyone in the world. Mihir Ghandi, VP of Strategic Marketing for Eightfold.AI, discusses how AI is being used to match often not-so-obvious skills of applicants’ to open positions with companies worldwide. Mihir provides insight into how Eightfold’s technology identifies corollary skills that may not have fit on the traditional, single-page resume. In tandem with skills definition, Eightfold’s AI is creating calibration profiles for hiring managers to use, making it more effective for them to identify information beyond the core skills outlined within job descriptions.While the discussion goes deep into data and the use of it, there is also discussion around the role of the humans involved. How data is adjusted based on hiring manager feedback, changes to job descriptions, the evolution of skills, and candidates’ stories.The evolution of candidate skills along with the changing needs of organizations is of particular interest as a result of the current job market and the impact of COVID-19. The pair dive into the importance of candidates’ fluidity and need to reskill, or as Dr. Handler refers to it “self-skill", in an effort to expand their opportunities. The use of AI to identify the correlation of skills required for certain positions but that are not explicitly outlined could not come at a better time for TAs and job seekers. Gandhi offers the recently released Talent Exchange, a partnership with McKinsey & Company, which provides companies and individuals free access to this powerful technology. Using their tech to help match those most in need ties directly back into their mission, and democratizes the technology.Science4Hire is brought to you by Rocket-Hire - your partner for launching successful talent assessment strategies and solutions. For more information on Rocket-Hire, visit https://rocket-hire.com Interested in learning more about talent acquisition platforms that use AI? Then you’ll want to read Rocket-Hire’s in-depth 2020 Talent Assessment Market Trends Report. Learn more at https://go.rocket-hire.com/assessment-platform-marketRelated Resources Connect with Mihir Ghandi via email at mihir@eightfold.ai Find him on LinkedIn at https://www.linkedin.com/in/mihircgandhiIf you are at a company that is hiring, has recently laid-off employees, have recent graduates to place, or are an individual looking for career opportunities, register at Talent Exchange at https://eightfold.ai/talent-exchange/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com

During this episode of Science-4 Hire, Dr. Charles Handler is joined by Betsy Wills, Founder of YouScience, to discuss what some might call a “pre” pre-hire assessment. For decades Wills has been dedicated to helping people manage their careers forward. This session focuses on YouScience’s origin story and the role of assessment as a core pillar of their value proposition. YouScience is all about using self-discovery as an input into a virtual ecosystem that allows users to empower themselves in their career choices.Wills laments about the brief, even laughable suggested career paths that many of us who took the standardized tests like the ACT or SAT provided can remember. The time has come to reimagine a better, more accurate way to guide those entering college or the workforce into careers they were more suited for than the inaccurate inventories that have been hanging around for decades. Betsy began work with ETS (the makers of the ACT) to create an assessment that would help to assess these new college grads based on highly sought “21st-century skills”. The team quickly realized that there was value to be had by both individuals taking the assessment and the employers seeking skilled labor, and so began their pivot. The search for an ideal assessment to power a career discovery engine landed Wills on the Johnson O’Connor Aptitude Test, a well researched and pedigreed, but with many interesting points of differentiation from traditional work-related aptitude tests.So why assess an individual’s aptitude over their interest?Wills provides a summary of key tenants covered in her TedTalk - notably the “boomerang” nature of traditional interest tests. These tests simply repeat back the rote things that individuals tell the test, often leading to no real revelations. Wills explains the differences, the superiority of aptitude tests such as the Johnson O’Connor test. Such assessments measure core aptitudes across a full range of areas (think musical abilities, psychomotor capabilities, etc.) that provide a foundation for holistic self-awareness. YouScience takes testing leaps and bounds further for all that engage with the platform. Their connection to live employment data and listings, college course catalogs, and their recent merger with Precision Exams, a provider of professional certification tools, makes YouScience’s platform invaluable to job seekers at every stage. Over 40K adult users have engaged with the YouScience platform, creating awareness for the benefits it provides. Throughout the episode, Wills offers listeners insight into how our individual aptitudes play a role in how to best engage coworkers, finding fulfillment outside of the office through the right personal past-times, and understanding what to steer clear of in and outside of the workplace.Science4Hire is brought to you by Rocket-Hire - your partner for launching successful talent assessment strategies and solutions. For more information on Rocket-Hire, visit https://rocket-hire.com Access free resources and reports by Rocket-Hire. Download our most recent research including our Covid-19 Talent Acquisition Benchmarking Report at https://go.rocket-hire.com/covid-19-report. Learn more about our upcoming, in-depth Talent Assessment Market Trends Report at https://go.rocket-hire.com/assessment-platform-marketRelated Resources Connect with Betsy Wills on Instagram at https://www.instagram.com/betsywills/Find her on LinkedIn at https://www.linkedin.com/in/betsywills/Check out her passion project, Art Stormer at https://artstormer.com/author/artstomer/Watch her TedTalk “The Science of Aptitudes” at https://www.youtube.com/watch?v=wp4n2HiU938 Interested in taking your own aptitude assessment?Save 50% by using promo code DISCOVER50 at https://www.YouScience.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com