Documente Academic
Documente Profesional
Documente Cultură
HR practitioners often have misperceptions regarding research findings in the area of employee
selection. This article reviews research on what selection tools work, what recruitment strategies
work, how selection-tool use relates to workforce diversity, and what staffing and recruiting
processes lead to positive applicant perceptions. Knowledge and implem-entation gaps in these
areas are discussed, and key research findings are presented. © 2004 Wiley Periodicals, Inc.
Correspondence to: Ann Marie Byan, Michigan State University, Dept. of Psychology, East Lansing, MI
48824-1116, ryanan@msu.edu
Human Resource Management, VVinter 2004, Vol. 43, No. 4, Pp. 305-318
© 2004 Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com).
DOI: 10.1002/hrm.20026
306 HUMAN RESOURCE MANAGEMENT, Winter 2004
examples will encompass both internal and tools for specific types of positions (e.g.,
external staffing situations. managerial, customer service). However,
We contend that a great deal of the re- wide gaps between knowledge and practice
search in the staffing and recruiting area has exist in many organizations. For example,
not heen widely embraced hy the HR practi- Rynes, Colbert, and Brown (2002) reported
We contend tioner. For example, Rynes, Brown, & Colhert that 72% of the HR managers they surveyed
that a great (2002) reported that some of the most com- thought that, on average, conscientiousness
deal of the mon misperceptions of HR practitioners were is a better predictor of employee perfor-
research in the in the area of selection. There are several rea- mance than intelligence, whereas the reverse
staffing and
recruiting area sons for these gaps. First, research is not well is actually true. Similarly, they found that the
has not been disseminated and remains huried in jargon- majority of respondents believed that compa-
widely laden pages of academic journals (Rynes, nies that screen job applicants for values
embraced hy Colhert, & Brown, 2002), rather than trans- have higher employee performance than
theHR lated into "something useful" and made ac- those that screen for intelligence, another
practitioner. practice that is not supported by the re-
cessible to HR executives (hence, the pur-
pose of this special issue). Second, research search. Another example provided by Rynes,
is focused on the situation in the abstract and Brown and Colbert (2002) is the common
seldom takes into account the many contex- misperception that integrity tests do not
tual factors (e.g., budget, time constraints) an work because individuals lie on them, when
HR practitioner must juggle in making deci- in reality these tests can predict job perfor-
sions about what tools and strategies to em- mance despite any tendencies of candidates
ploy. Third, many HR professionals in staffing to misrepresent themselves.
functions and particularly those in tight lahor In our work with organizations, we run
markets face constant pressure to deliver across many hiring managers who make
qualified candidates quickly and lack the time blanket statements that testing is not worth-
to create new recruiting and selection pro- while, despite the fairly substantial body of
grams that take into account current research research that demonstrates that testing can
findings. Fourth, some stakeholders' percep- be very useful. Even when presented with ev-
tions and goals for hiring are not compatible idence of strong relationships hetween test
vvdth the research (Rynes, Colbert, & Brown, scores and joh performance, some managers
2002), making it difficult for the HR profes- are unwilling to concede that structured se-
sional to incorporate the research findings lection programs are not just good ways to se-
into a hiring program. Fifth, many HR pro- lect employees; they are better than other
fessionals find myriad legal requirements less structured alternatives. The power of
confusing. Consequently, they avoid testing "gut instinct" and "chemistry" seems to over-
and other systematic approaches to selection ride the hard data and rational arguments.
despite the research because they believe er- What do we know about which tools are
roneously that testing will create legal prob- most useful? Table I provides a summary of
lems rather than solve them. In this article, the basic characteristics of various tools (i.e.,
we hope to close some of these gaps hy pre- validity, costs, and group differences), based
senting some key, agreed-upon findings re- on the current body of research. The first
garding staffing and discussing the practical column provides a listing and brief descrip-
steps one must go through to effectively im- tion of common measures used in staffing
plement these findings. situations. The second column reports "va-
lidity," which is a statistical measure ranging
Selection Tools from 0 to 1.00 of the relationship between
test scores and a criterion—in this case, joh
After over a century of research on methods performance. Note that these values repre-
of selecting employees, there is a consider- sent likely upper-limit values in that they are
able body of knowledge regarding what works not reduced by factors that would lower val-
well across jobs and organizations and what ues ohtained in operation (e.g., unreliahility
does not, as well as substantial research on in the measurement of job performance, re-
Attracting and Selecting: What Psychological Research Tells Us 307
striction in range of scores). While larger ferences is contained in the fourth column of
numbers generally indicate more accurate Table I. The data are presented in terms of
prediction statistically, even low numbers standardized group differences (i.e., all data
(i.e., less than .20) increase the accuracy of are presented on the same numeric scale).
prediction, particularly when the applicant "BAV: -1.0" indicates that in the research re-
pool is large and the numher selected is rela- ported, blacks scored one standard deviation
tively small. The third column provides over- below the mean of whites.
all estimates of the costs to develop and de- There are some key points to keep in
liver each instrument relative to other mind when applying Table I to a specific or-
selection tools. Information about group dif- ganizational context. First, the table presents
Tool Comparison
Unstructured interviews measure a variety of skills and .31 Low/high BAV: -.32 Huffcutt & Roth,
abilities, particularly noncognitive skills (e.g., HAV:-,71 1998
interpersonal skills, leadership style, etc) using
questions that vary from candidate to candidate and
interviewer to interviewer for the same job. Often,
specific standards for evaluating responses are not used.
Work samples measure job skills (e.g., electronic repair, ,54 High/high BAV: .38 Schmitt, Rogers,
planning and organizing), using the actual performance Chan, Sheppard,
of tasks that are similar to those performed on the job. & Jennings, 1996
Typically, work samples use multiple, trained raters and
detailed rating guides to classify and evaluate behaviors.
Job knowledge tests measure bodies of knowledge (often .48 High/low BAV: ,38 Schmitt et al..
technical) required by a job, often using formats such as 1996
multiple-choice questions or essay-type items.
Conscientiousness measures the personality trait ,31 Low/low BAV: -.06 Hough et al,.
"conscientiousness," typically with multiple-choice or HAV: -,04 2001
true/false formats. AAV: -.08
W/M: .08
Biographical information measures a variety of noncognitive .35 High/low BAV: -,78 Roth & Bobko,
skills and personal characteristics (e.g., conscientiousness, for grades 2000
achievement orientation) through questions about BAV: - ,27 Hough et al,.
education, training, work experience, and interests. biodata 2001
HAV: ,08
biodata
W/M:-.15
biodata (continued)
308 HUMAN RESOURCE MANAGEMENT, Winter 2004
Integrity tests measure attitudes and experiences related .41 Low/low BAV: -.04 Hough et al..
to a person's honesty, dependability, trustworthiness, and HAV: .14 2001
reliability, typically with multiple-choice or true/false AAV: .04
formats. W/M: .16
Assessment centers measure knowledge, skills, and .37 High/high Varies by Goldstein, Yusko,
abilities through a series of work samples/exercises that exercise; & Nicolopoulos,
reflect job content and types of problems faced on the -.02 to -.58 2001
job, cognitive ability tests, personality inventories,
and/or job knowledge tests.
Reference checks provide information about an .26 Low/low •?•>
"Source for validity coefficients: Schmidt & Hunter (1998), except Situational Judgment Test validity, from McDaniel, Morgeson, Finnegan,
Campion, & Braverman (2001). Validity values range from 0 to 1.0, with higher numbers indicating hetter prediction of job performance.
''The labels "high" and "low" are designations relative to other tools rather than based on some specific expense level.
'Values are effect sizes expressed in standard deviation units. Higher numbers indicate a greater difference; negative values mean the first group
scores lower. BAV is black/white difference; HAV is Hispanic/white difference; AAV is Asian/white difference; W/M is female/male difference.
potential values and exact results will depend dicting work outcomes, such as job perfor-
upon the specific situation present. As noted mance, turnover, and absenteeism. However,
earlier, these are not operational values but beyond this key objective, there are a number
estimates without the influence of some fac- of other factors that may influence tool use-
tors that likely will depress values obtained in fulness in practice that need to be consid-
a given setting. Second, any hiring manager ered in making choices. HR managers will be
or researcher will agree that the number-one concerned about issues such as the cost to
criterion for a useful selection device is that develop the tool and use it, the ease of ad-
it provides information on who will be a good ministration, and the likelihood of adverse
employee. In research terms, this translates impact (i.e., disproportionate hiring rates for
into which devices have good validity for pre- different ethnic or gender groups). However,
Attracting and Selecting: What Psychological Research Tells Us • 309
rather than considering validity, costs, and how well tools might work in combination
subgroup differences as tradeoffs, choices (Society for Industrial and Organizational
among valid tools should involve cost and Psychology, 2003). In most cases, measuring
subgroup differences as additional consider- more job-related skills and abilities results in It never makes
ations only when an acceptable level of va- better predictions of overall job perfor- sense to employ
lidity has been established. That is, it never a tool without
mance. Schmidt and Hunter (1998) noted
supporting
makes sense to employ a tool without sup- that combining a general mental ability validity
porting validity evidence—regardless of how measure with a structured interview or with evidence—
cheap, how low the adverse impact, or how a work sample is likely to yield the highest regardless of
easy to administer. A test with low validity composite validity. Determining the optimal how cheap,
will not result in good hiring decisions and number of tools for any given situation will how low the
adverse impact,
will be much more costly in the long run. involve considering how each tool enhances or how easy to
Thus, the validity column in Table I should prediction relative to its additional expense administer.
always be the first factor in tool choice. and time requirements.
To that primary consideration, we add a Finally, the usefulness of a selection tool
number of others. First, specific tools vary in in any given situation will require evaluating
usefulness according to how well developed context-specific factors not presented in the
they are—a poorly developed tool will not table, such as the selection ratio (number of
function at the optimal level suggested by candidates hired relative to the number of
the table. For example, a structured inter- candidates who applied), hiring cycle time,
view may have a validity of. 51 if interviewers costs of a selection error (e.g., cost of re-
are trained, adhere to the standard set of placement, error, lost opportunities), and so
questions, and use the behavioral anchors to on. Higgs, Papper, and Carr (2000) noted
rate candidate responses. Deviations from eight "drivers of selection process success."
these "best practices" for interviews may re- Table I presents information on two of
sult in considerably Iower validity (see these—empirical validity and expense. We
Posthuma, Morgeson, & Campion, 2002 for discuss two others (face validity and candi-
a recent review of research on interviewing). date reactions) in the section on applicant re-
Second, some selection tools are not ap- actions. The other drivers—selection ratio
propriate for a particular situation. For exam- (how many applicants per position), mar-
ple, job knowledge tests may not be appropri- ketability (getting people in the organization
ate for an entry-level position if they tap to use the tools), timeliness (feedback to ap-
knowledge easily acquired on the job and not plicants and hiring managers), and manage-
needed at the time of hire (Schmitt & Chan, ment of the process (selection system admin-
1998). Similarly, some personality measures istrator ability and credibility)—are all
are inappropriate for employee-selection con- context-specific and may be more challenging
texts, and some personality traits do not have for some HR managers than others. Further,
much relation to job performance (see Bar- Tippins (2002) noted that putting a selection
rick, Mount, & Judge, 2001 for a review of system into use involves a host of implemen-
the research evidence on personality testing). tation issues—decisions about the ordering of
An experience with an inappropriately ap- process elements, the ways in which informa-
plied tool can lead to a generalization about tion will be combined, the use of technology
the usefulness of all personality testing that is in delivery of tools, the training of tool users,
flawed. Another reason for the research-prac- policies (e.g., waivers), the database structure
tice gap may be managers making judgments and access, and communications about the
about a category of tools for all positions system—all of which contribute to the suc-
based on negative experiences with a specific cess or failure of a selection system.
tool (e.g., a poorly developed tool or a well- Another contributor to the research-
developed tool used inappropriately) for a practice gap in selection-tool use is the lack
specific position. of information regarding the business rea-
Further, to maximize the effectiveness of sons for tool use. HR managers need infor-
a selection system, one needs to consider mation on selection tools in the language of
310 HUMAN RESOURCE MANAGEMENT, Winter 2004
ducing the validity of a selection program coaching and orientation programs, banding
(Sackett et al., 2001). The extent of adverse scores without having preferential selection,
impact reduction is influenced by a complex removing culturally biased items, altering
set of factors, such as the nature of the ap- formats, or increasing time limits. It is im-
plicant pool, the selection ratio, the manner portant to note that while research does not
in which tools are used, and the degree of re- fully support combining a tool with higher
lationship among the tools used. As Table III adverse impact with one having lower ad-
indicates, research does support considering verse impact to mitigate overall adverse im-
a broader spectrum of desired work out- pact of the selection process, the research
comes as the criteria for success, using does support broadening tbe selection
noncognitive predictors exclusively, actively process so tbat both task performance and
diversifying the applicant pool with qualified so-called "contextual" performance are pre-
individuals, or appropriately orienting and dicted. Thus, an action such as adding a tool
motivating applicants as ways to lower the that measures competencies related to con-
adverse impact of a selection system. The re- textual performance and has less adverse im-
search does not provide strong support for pact may have a mixed effect on overall ad-
other common practices such as providing verse impact.
3J2 HUMAN RESOURCE MANAGEMENT, Winter 2004
Practices Used To Reduce the Adverse Impact of a Selection System and Research Support
Practices Used To Reduce the Adverse Impact of a Selection System and Research Support (continued)
Use portfolios, accomplishment +10 Evidence suggests group differences may not be reduced by realistic
records, and performance assessments, and reliable scoring of these methods may be
assessments (work samples) instead problematic (Sackett et al,, 2001), Well-developed work samples may
of paper-and-pencil measures. have good validity and less adverse impact than cognitive ability tests
(see Table I),
Relax time limits on timed tools, 0 Research indicates that longer time limits do not reduce subgroup dif-
ferences, and may actually increase them (Sackett et al,, 2001),
While many of the rationales for the gap the most desirable applicants. That is, high
between research and practice apply to the performers can be more selective about
specific gap involving adverse impact and di- where they choose to work. A legitimate con-
versity research and practice, one rationale is cern of the HR professional is the extent to
particularly relevant here. The HR profes- which research takes into account how ap-
sional confronts an extremely perplexing plicants feel about the various tools and
dilemma. The two goals of identifying good strategies recommended. Over the last
candidates accurately and building a diverse decade, there have been considerable ad-
workforce are difficult to reconcile. Some of vances in our knowledge of what applicants
the more effective strategies, such as using prefer and what they see as fair and unfair
only noncognitive measures, also reduce the and what can be done to mitigate negative
validity of the selection process. Thus, the reactions of applicants (see Ryan & Ployhart,
HR professional can be faced with a difficult 2000 for a review). After all, a hiring process
choice—maximizing the accuracy of predic- is an evaluative one, and many individuals
tions made on the basis of tests or reducing will walk away without a job. While the re-
adverse impact. In addition, it merits noting search in this area is nascent, there are some
that no one approach works, and the HR basic practical steps that HR managers can
manager must pursue several avenues to consider in designing and implementing se-
achieve these goals. Because the research lection and recruiting practices.
presents no easy solutions, the reluctance of Table IV provides some key suggestions
HR professionals to employ any of these arising from basic psychological research on
ideas is understandable. perceptions of justice, ways to mitigate nega-
tive perceptions of a negative outcome (i.e.,
Applicant Perceptions a rejection letter), and best approaches to ex-
plaining the process and decisions to ensure
HR managers are often concerned that the the most positive perceptions possible.
tools and recruitment approaches best sup- One overall conclusion from this area of
ported by research might serve to "turn off research is that strong generalizations re-
3J4 HUMAN RESOURCE MANAGEMENT, Winter 2004
Audit Questions
garding applicant preferences for tools fully and sensitively—these concerns apply
should not be made. For example, we often to all tools equally.
hear opinions that applicants will react neg- The gaps between research and practice
atively to biodata questions, find integrity in the area of applicant perceptions can be
tests and personality tests invasive, view cog- attributable to many factors, including the
nitive ahility tests as not job-relevant, find as- ones already cited. However, an additional
sessment centers to be taxing and overkill, important reason for the wide gap may be the
and see structured interviews as restrictive. lack of accurate knowledge about the entire
Overall, the research suggests that appli- applicant pool and erroneous assumptions
cants will not react negatively to tools that about applicant preferences. It is often diffi-
are well developed, job-relevant, and used in cult to get information from the entire range
selection processes in which the procedures of applicants. Many HR managers are reluc-
are appropriately applied, decisions are ex- tant to solicit feedback from applicants who
plained, and applicants are treated respect- are not hired and hear only from the dis-
Education:
• What is your highest degree?
• From what institutions are your degrees?
• What kind of education and training in test development, measurement, validation, and statistics do you have?
• Were the courses for credit? From what institution were they granted?
Comment: Ask about a consultant's educational background. A PhD in industrial and organizational psychology or a closely
related field is often a fundamental requirement for competence in testing work. While continuing education courses can
strengthen a person's skills in this area, they rarely provide an adequate foundation. Also, ensure that the institution granting
the degree is a reputable one.
Experience:
• How long have you been doing test development and validation? Can you describe some selection procedures you developed
and the process you used to develop them?
• How are these instruments being used currently?
• How do you evaluate the effectiveness of your work?
• Who are some of your other clients?
• What would I need to know to ensure that a process you developed or sold to me was valid?
• With what kinds of employee populations have you worked?
• With what kinds of industries have you worked?
• What kind of experience do you have defending selection programs?
Comment: Most industrial and organizational psychologists learn how to conduct test validation studies by working with oth-
ers. In general, an employer does not want to be the first "guinea pig," Explore with consultants the kinds of testing experi-
ence they have had as well as the industries and cultures with which they have worked. Also, find out if a potential consultant
has experience defending selection instruments and programs against legal and other challenges. Cood selection programs
may be legally challenged, and a challenged instrument should not be a reason to avoid a consultant. In fact, most organiza-
tions want to use a consultant who fully understands the legal issues regarding selection.
Professional Credentials:
• With which professional organizations are you a member?
• What is your ethics code? Who enforces that ethics code?
Comment: One way to find a testing professional is to look for members of professional organizations such as the Society for
Industrial and Organizational Psychology (www.siop.org) that represent a large group of people who are trained in this area.
Ethics are an important consideration when personal information about an individual's capabilities is collected. Members of
SIOP must subscribe to the American Psychological Association's Code of Ethical Conduct, The American Psychological As-
sociation enforces the Code,
316 HUMAN RESOURCE MANAGEMENT, Winter 2004
gruntled. Furthermore, applicants may be re- find one who can explain research,
luctant to share their true feelings regarding understands your organizational en-
the recruiting and selection process. Conse- vironment, and has experience in
quently, many decisions are based on intu- real-world settings. Table VI con-
ition rather than facts. tains a brief list of questions to aid
the HR professional in, ascertaining
Summary the competence of experts.
Educate yourself and critically
We have very briefly highlighted four areas in evaluate. We do not advocate that
staffing where gaps between research and most HR professionals subscribe to
practice exist, noting both knowledge gaps journals that report research studies,
and implementation gaps. To aid the HR as lack of time and background ex-
manager in evaluating how well a staffing pertise will make it difficult to gain
system fits with current research knowledge. value from these. We do suggest that
Table V presents a list of audit questions. you find ways to stay current on what
Our goal is to assist the reader in not only is taking place in the fields of re-
understanding the present gaps between re- cruitment and selection. That may
search and practice in recruitment and selec- mean reading broadly, taking a class,
tion, but also in developing skills for employing or attending a lecture. Regardless of
research in HR practice. We close with three how you stay up-to-date, we recom-
recommendations for the HR professional. mend that you think critically about
all you find (Edwards, Scott, & Raju,
• Use a professional. The world of 2003). Table VII presents a list of
psychological research in general can questions to assist in the evaluation
be difficult to understand, and selec- of research.
tion research is particularly arcane. Systematically collect data and
In fact, not all psychologists under- evaluate efforts. Many of the mis-
stand all areas of research! It takes conceptions in HR practice are due
years of education and experience for to mistaken beliefs about what is ac-
an industrial and organizational (I/O) tually happening. Accurate informa-
psychologist to master recruiting and tion is a requirement for understand-
selection. Consequently, HR profes- ing what is taking place and making
sionals cannot be expected to achieve correct interpretations of the facts
the same level of mastery quickly. An and careful evaluations. While data
I/O psychologist who is an expert in collection can be a difficult process
recruiting or selection research can in many organizations, new technol-
assist in translating research into ef- ogy can greatly simplify the process
fective practice. and capture much information with
Finding an expert is not difficult; little effort on the part of staffing
finding a good one can be. Be sure to personnel.
NANCY T. TIPPINS is president of the Selection Practice Group of Personnel Research As-
sociates (PRA), where she is responsihie for the development and execution of firm
strategies related to employee selection and assessment. Prior to joining PRA, she spent
over 20 years conducting selection research in husiness and industry. She is active in a
number of professional organizations and a past president of the Society for Industrial
and Organizational Psychology. She is the associate editor of Personnel Psychology.
performance, and selection rates. International ment, credentialing, and higher education:
Journal of Selection and Assessment, 10, Prospects in a post-affirmative action world.
258-270. American Psychologist, 56, 302-318.
Posthuma, R. A., Morgeson, F. P., & Campion, M. A. Schmidt, F. L., & Hunter, J. E. (1998). The validity
(2002). Beyond employment interview validity; and utility of selection methods in personnel
A comprehensive narrative review of recent re- psychology: practical and theoretical implica-
search and trends over time. Personnel Psy- tions of 85 years of research findings. Psycho-
chology, 55, 1-81. logical Bulletin, 124, 262-274.
Roth, P L., & Bobko, P. (2000). College grade point Schmitt, N., & Chan, D. (1998). Personnel selec-
average as a personnel selection device: Ethnic tion: A theoretical approach. Thousand Oaks,
group differences and potential adverse impact. CA: Sage.
Journal of Applied Psychology, 85, 399-406. Schmitt, N., Rogers, W., Chan, D., Sheppard, L., &
Ryan, A. M., & Ployhart, R. E. (2000). Applicants' Jennings, D. (1997). Adverse impact and pre-
perceptions of selection procedures and deei- dictive efficiency of various predictor combina-
sions: A critical review and agenda for the fu- tions. Journal of Applied Psychology, 82,
ture. Journal of Management, 26, 565—606. 719-730.
Rynes, S. L., Brown, K. G., & Colbert, A. E. (2002). Society for Industrial and Organizational Psychol-
Seven common misconceptions about human ogy. (2003). Principles for the validation and
resource praetices: Research findings versus use of personnel selection procedures. Bowling
practitioner beliefs. Academy of Management Green, OH: Society for Industrial and Organi-
Executive, 16(3), 92-102. zational Psychology.
Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). Taylor, M. S., & Collins, C. J. (2000). Organizational
HR professionals' beliefs about effective human recruitment: Enhancing the intersection of re-
resource practices: Correspondence between search and practice. In C. L. Cooper & E. A.
research and practice. Human Resource Man- Locke (Eds.), Industrial and organizational psy-
agement, 41, 149-174. chology: Linking theory with practice (pp.
Sackett, P R., & EUingson, J. E. (1997). The effects 304-330). Oxford, UK: Blackwell.
of forming multi-predictor composites on group Terpstra, D. E., & Rozell, E. J. (1993). The relation-
differences and adverse impact. Personnel Psy- ship of staffing practices to organizational level
chology, 50, 707-721. measures of performance. Personnel Psychol-
Sackett, P. R., & Roth, L. (1991). A Monte Carlo ex- ogy, 46, 27-48.
amination of banding and rank order methods Tippins, N. T. (2002). Issues in implementing large-
of test score use in personnel selection. Human scale selection programs. In J. W. Hedge & E.
Performance, 4, 279-295. D. Pulakos (Eds.), Implementing organization
Sackett, P R., & Roth, L. (1996). Multi-stage selec- interventions: Steps, processes, and best prac-
tion strategies: A Monte Carlo investigation of tices (pp. 232—269). San Francisco: Jossey-
effects on performance and minority hiring. Bass.
Personnel Psychology, 49, 549-562. Whyte, G., & Latham, G. (1997). The futility of util-
Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, ity analysis revisited: When even an expert fails.
M. B. (2001). High stakes testing in employ- Personnel Psychology, 50, 601-610.