A Commentary from Previsor on the Risks of Profiling in the use of Pre-Employment Psychometric Assessment Tools in building a Conscious Hiring Process.
Overview
This commentary describes the profiling approach to using pre-employment psychometric assessments, compares it to the predictive approach to using assessments, and
then contrasts these two very different approaches. It then describes
the dramatically different results derived from an analysis in which
the two methods were compared side-by-side in the selection process for
a given organization. At KeenHire, we use both approaches in tandem to deliver the best hiring process to our customers. The Key is in the Role Analysis (referred to as job analysis in this article).
What is Profiling and How Does it Compare to the Predictive Model?
The profiling approach involves assessing a small group of employees
that have been identified by the organization as high-performers, and
using the resulting test scores to determine how to evaluate job
candidates. Candidates whose assessment scores more closely resemble
the scores obtained by the high-performers are judged to be better
“fits” for the role, and are forecasted to be better hires. A
predictive approach, by contrast, involves identifying job-relevant
traits based on an in-depth job analysis, and selecting those
candidates who earn higher scores on measures of those traits.
Seven Problems with the Profiling Approach to Assessment
Most
assessment experts do not advocate using a profiling approach, even
though it would be easy for them to implement, due to seven major
problems with profiling. These are summarized below.
#1: The "Ceiling Effect" Problem
The scores obtained by
the high-performing employees asked to complete the assessment define
the ceiling—the maximum possible score that can be obtained on the
assessment. This means that it is actually impossible to identify a
candidate who is a better fit for the position than current
high-performers. The high performers used to develop the model don’t
just represent “desirable,” they represent “the best imaginable. ”The
profiling model has no answer for the question, “If a candidate came
along who was an even better fit for the position than your current
high performers, how would you know?”
#2: The Not Enough Data Problem
The profiling approach
relies on a large number of conclusions and decisions made on the basis
of a very small amount of data. Profiling assumes that the test scores
obtained from a handful of employees produces information that will be
consistent with and generalizable to job candidates. But the
unfortunate reality is that small data samples are very rarely
conclusive. Models built on the basis of test scores from five people,
or even fifteen people, are very unlikely to be accurate. They may not
even be close, and can even lead to false conclusions. At KeenHire, we measure the actual on the job performance of every participant in the job benchmark study. If the on the job results are not exceptional, we encourage our customers to exclude that participant from the study.
#3: The (Non)Essential Trait Problem
Profiling is more
likely to incorrectly identify unimportant traits as job-relevant, or
overlook traits that are essential to job success. This is because the
profiling approach is based on a few individuals and their traits, not
on how relevant those traits are to the job demands or requirements.
Maybe the handful of assessed employees just happen to share traits
that are not actually job relevant. Maybe they differ on traits that
are job relevant because one or two of them have learned to compensate
for a limitation in one area with particular strengths in others. Maybe
the traits common to the top-performers who were tested are the same
traits possessed by weaker performers—but since weaker performers were
not assessed as part of the approach, there is no way to know. Any of
these possibilities would result in a profiling model that has no value
in identifying future high performers.
#4: The “Over-Cloning” Problem
Profiling emphasizes
similarities without any regard to whether or not they are important
similarities. For example, if the profile is created based on a
ten-scale personality instrument, but only four of those scales are
actually relevant for success in the position, the organization is
needlessly forcing consistency and uniformity in six areas. Good,
high-potential candidates may be passed over because they are not
sufficiently similar to current employees in areas that have no bearing
on their likelihood of success. At KeenHire, our process begins with an Organizational Development approach. We work diligently to build a balanced organization by first digging deep into who the organization is, we seek to understand the current corporate culture, the desired culture, the strategic objectives and goals as well as the barriers to achievement. We then assess and evaluate the Executive team,the mission critical and key contributors. We look for what is working, what is not and what is missing, the presence of which would make a difference. Only after we have a full picture of the current state and the desired future state do we make recommendations. No one with any level of discernment or sanity really wants the "Stepford" culture. Most companies simply do not understand how to build a high performance culture. That is where we come in and make the difference.
Continue reading "Do You Want to Profile Performance or Predict It?" »
