Sunteți pe pagina 1din 6

Journal oj Applied Psycholoty 1.,, \ol. 52, No.

4, 325-330

HEURISTIC SIMULATION OF PSYCHOLOGICAL DECISION PROCESSES1


ROBERT D SMITH
Department oj Management, Pennsylvania State University Complex thought processes used by a skilled psychologist dealing with personnel selection and placement are analyzed and programmed in a computer simulation model Heuristic methods used to limit the total possible set of decision branches in the model are defined and illustrated. Research results indicate a strong relationship between human and machine output with a 94% level of agreement between simulated psychological inferences and human decisions with identical ultimate employment recommendations in 22 of 24 test cases. Implications of research findings on psychological instruction, experimentation, information retrieval, test validation, and general decision making are discussed.

Recent developments in the study of man and machine systems have resulted in significant interdisciplinary achievements. The areas of psychology, industrial management, and operations research have been increasingly integrated through quantitative and scientific methods which aid in the definition, explanation, and prediction of human behavior. One of the most useful of these methods is simulation, a research methodology which facilitates the design and testing of models of physical and human systems. Through the use of simulation, the models can be programmed to yield results which closely approximate realworld phenomena. Simulation models are of varying types. Some are applied to the study of physical systems such as oil refineries and railroad switching yards. Others deal with man-machine systems such as the Rand Air Defense Simulation in which military personnel and hardware exhibit integrated reactions to simulated changes in the air environment. Yet another type involves human cognition and can be used to replicate the manner in which man makes decisions in solving complicated problems. The purpose of this article is to present the methodology, results, and, most important, the implications of research dealing with this latter form of simulation.
This article is based on a paper presented at The College on Management Psychology Session of The Institute of Management Science Fourteenth International Meeting, Mexico City, August 26, 1967.
1

Considerable work in the development of cognitive simulations has been conducted by university and privately sponsored research groups. Colby (1964) at Stanford, Maruyama (1966) at Berkeley, Kleinmuntz (1963) at Carnegie-Mellon, Swenson (1962) at the Mayo Clinic, and Finney (1967) at The University of Kentucky have been instrumental in designing computerized simulations which describe individual psychological characteristics based on some type of test scores. The present research represents an extension of work previously accomplished. Specifically, this study investigated in depth and subsequently simulated the thought processes of a psychologist involved in the analysis and interpretation of test results, job requirements, and personal characteristics of potential clerical employees. In this type of decision making, the psychologist is faced with ill-structured problems not amenable to solution by algorithmic techniques in which the researcher is able to express an objective function and constraints in well-defined mathematical terms. Instead, it is necessary to rely upon the construction of simulated decision networks which replicate as closely as possible the psychologist's thinking processes.
METHOD As shown in Figure 1, the experimental model extended the number and types of variables included in previous studies by adding job specifications and personal characteristics of applicants. These additional variables made the task of simulation more difficult since inferences derived from test scores had 325

326
COMPUTE** TRANSFORMS DATA

ROBERT D. SMITH

OUAUTATtVE INTTHWCTATIO TEST SCOftCS *. OOMPISTER TRANSFORMS DATA

CMPIOTMENT DECISION

statements generated by the computer was judged to be 94% by a skilled analyst not involved in the collection of the protocol. Further, statistical tests of hiring recommendations showed that the probability of obtaining such results by chance was so small that the hypothesis was accepted, and it was concluded that the model did simulate very closely the human analyst's psychological inferences. Limitations The preceding paragraphs have very briefly presented the methodology and results of the research.2 It should be noted that no attempt was made to improve upon the decision-making process of the analyst and it would have been preferable to test a larger sample of cases had they been available. It might be true that a more valid and reliable model would have resulted if the combined decisionmaking capabilities of a group of analysts had been used. However, rather than spending time in a discussion of the limitations of the study, more significant issues involve the use of heuristics in development of the model and the implications of results on general psychological decision making. Heuristic Reduction oj the Decision Space In developing the computer model, the matter of relevancy of information became all important. Consider the clerical selection battery used in this experiment. It is comprised of the Otis Mental Ability Test, the Short Employment Tests, the Washburne S-A Inventory, and the Gordon Personality Inventory and Profile. This battery includes 20 different test scores, some of which can range from 1 to 99 such as the Gordon scores while others range from excellent to maladjusted (7-point scale) on the Washburne Test. In all, there are 1,151 possible psychological test scores, in addition to job and personal data, which can be obtained for a single applicant. Thus, it is plain to see that the number of combinations of relationships and elements that can be chosen quickly exceeds manageable proportions. In fact, using the variables
2 For a complete discussion of research design refer to Smith and Greenlaw (1967).

T S T SCORES COMPUTER TRANSFORMS OATA

QUALITATIVE INTERPRETATION

WCHOUMSCAL JOB RCOUREMCN

EMPLOYMENT DECISION

Fic. 1. Evolution of model complexity. to be viewed in light of specific job requirements and personal characteristics of the individual such as her age, experience, and length of time in her present position. It was hypothesized that a simulation model could be developed which would replicate the thought processes of a skilled psychologist as he performed the decision task. The Protocol Method was used to gather data (i e, the analyst's verbalization of thoughts were recorded as he evaluated a series of cases dealing with actual job applicants) The applicants were all female and were applying for various types of clerical positions requiring varying levels of skill. These positions were billing clerk, statistical clerk, clerk typist, receptionist, administrative assistant, and executive secretary. The protocol was tape-recorded, transcribed, and analyzed in depth prior to the development of a computer flow chart defining the decision processes Finally the model was tested by utilizing a set of cases which were analyzed by the human and processed by the computer RESULTS

In all, 24 cases were evaluated by both human and machine methods. In 22 cases the machine selected the same employment recommendation, which was one of the four classifications: hire, reject, hire as a fair risk, or check background further. In one case the machine was not programmed to handle the input data and consequently the human analyst was called in, and, in the other case of nonagreement, the human recommended hiring as a fair risk while the model suggested a further background check of the applicant. Accuracy of the simulated interpretative

SIMULATION OF DECISION PROCESSES

327

in this study, it would require 1070 outcomes to provide a model encompassing all possible events. This figure approaches the estimated number of atoms in the known universe (107C) representing a programming task of impossible magnitude. To avoid this situation, it is necessary to restrict the boundaries of the system and consider only those relationships which are most relevant to the solution of the decision problem. To simplify development of the experimental model, the experience of the decision maker was organized into a graphic representation of thought processes. The principles used in the simplification of the model (i.e., to limit the boundaries of relevance) were actually applied by the psychologist and merely transformed into more quantitative terms by the researcher. These principles are often referred to in the literature as "heuristics" taken from the Greek heveiskein which means to limit search. Heuristics provide genFIG 3 Partial heuristic decision map for intelligence eral decision rules or "rules of thumb" which and ability assist researchers in the development of workable models of systems which are within the boundaries of human and machine capabilities. tain alternatives are excluded from consideraHeuristic solutions to problems are not neces- tion which hopefully are irrelevant but, in sarily the best or optimum solutions since cer- fact, may not be. Clarkson's (1962) Portfolio Selection and Tonge's (1961) Assembly Line Models were both heuristic in nature as the authors turned their approach from a search for the optimum solution to one that, with high probability, would be suitable for their purposes. The same heuristic approach was adopted in the present research. Instead of providing for each possible value of a particular variable, relevant ranges for most variables were defined. This type of system definition involved the use of heuristics very similar to those used by the analyst in his interpretations. An example will help to clarify this point. The sociability score on the Gordon Test could range between 1 and 99. The analyst, however, treated only particular ranges for certain job types. For the statistical clerk, a sociability score of 95 or greater coupled with a low cautiousness score led to a negative response since the individual could conceivably be a "social butterfly" and incapable of maintaining the prolonged concentration required for the statistical work. FIG 2 Partial heuristic decision map for sociability
STMEMEHTS i 9*m*\ Nntrwcy tcnwifrtt at cwiiotu b *t7 fttviitg M tub imttsnv, noi stxHd for | t t > 4 A|lManr apprrart totMCrnintrrrMrt, nrrr j rarme orrt M T M I U I M

328

ROBERT D. SMITH

Figures 2 and 3 provide examples of heuristics as well as the general methodology used in mapping the thought processes. Note that, in Figure 2, the sociability score has been subdivided into three broad ranges at the lower end of its scale, and within one of these ranges (i.e., 6 through 10) it is compared with certain job specifications. This is not the only place in the program where the sociability score is considered, but it is evident here that a low score coupled with work requiring high levels of verbal proficiency and decision making lead to a negative recommendation.3 The psychologist placed his initial and greatest emphasis on scores which appeared at the extremes of the rating scale. That is, he first recognized scores above 90 or below 25 on the Gordon Tests and this information also provided a useful heuristic for the model builder. If certain combinations of variables were relevant but not provided for in the model, the computer was instructed to branch to the next case and leave the exceptional case for human interpretation. In this manner, heuristics were used to reduce the number of combinations of variables within the system to those most relevant for solving the majority of cases. Another type of heuristic was employed which added efficiency to the search process. It was discovered during the analysis of protocol that the psychologist tended to divide his problem into four general components paralleling the four tests used in the battery. He chose mental ability as his first area of investigation and seemed to weigh the Otis scores heavier than other factors for most jobs. Next, he referred to the clerical aptitude results followed by the emotional stability scores, and finally the personality profiles. This does not mean, however, that he followed an unbroken sequence from one test to another. If, for example, he found a girl who performed remarkably well on the Otis Test, he would investigate the Gordon score on "original thinking" as well as the type of job for which she was applying. The intent 'Proficiency levels required in specific jobs were
expressed on a numerical scale ranging from 1 (lowest level) to 3 (highest level)

here was to prevent very intelligent girls from being placed in positions where their abiliu and interest could lead to poor performance on repetitive or otherwise unchallenging jobs The broad approach to sequencing, however, did lead the researcher to design separate computer subroutines for each major area as a starting point in the development of the integrated system. Naturally, there was much subsequent interaction among the subsystems, but the heuristic technique of fractionating the total problem further simplified the work and, more importantly, permitted the simulation to follow a path through the network whereby the most heavily weighted variables were given highest priority. Items which tended to eliminate candidates most often were considered almost immediately, and time was saved since many unacceptable applicant* were discovered relatively early in the network. Implications oj Research Results One of the most important implications of the results of this research is the knowledge that it is possible to program fairly complex psychological decision tasks. This means that time normally spent by highly skilled analysts in decision making which is fairly routine for them can be made available for more significant problem solving and research. Second, the probability that boredom might influence results may be lessened as the psychological evaluator concentrates on those cases of unusual nature which require diverse and flexible skills found only in the human analyst. With computerized processing of routine data comes a marked reduction in the cost per interpretation when, of course, there is sufficient volume to introduce economies of scale Where the psychologist normally took about 20 minutes to give a complete analysis, the machine performed the same type of task in about 3 seconds.4 In this manner, expert analyses may be provided to those organizations which previously could not afford the
4 The decision model designed for the experiment was programmed to branch to the next case whenever an applicant was rejected. The psychologist, however, continued with a complete analysis even though a combination of factors led him to formulate a negative recommendation somewhere during the course of his evaluation.

SIMULATION OF DECISION PROCESSES

329

services of skilled psychologists. This may be particularly true in personnel offices and guidance centers which sometimes offer submarginal services in the field of psychological analysis. Simulation models of this type can be useful in the field of computer-assisted instruction. A laboratory approach in basic psychology courses might be established whereby students would verify their beginning and intermediate attempts at analyses against the results of an expert. With the information storage and retrieval capabilities of today's computers, teaching efforts might be made more efficient through the use of feedback data. Professors could evaluate the most common errors made by their students during the laboratory analyses and thereby concentrate upon these areas of weakness in subsequent lectures. Computerized simulations of psychological decision processes offer other advantages from an information-utilization standpoint. It is now possible to study the effects of policy changes on end results without disrupting the present system. For example, in the personnel model discussed in this paper, one could determine the effect on mental ability standards caused by a labor shortage which required a business to increase the acceptance ratio by 20% (i.e., how much would one have to lower mental ability scores to gain a 20% increase in the number of applicants accepted). In this type of model it would be fairly simple to determine the effects of changing the values of individual parameters on final decisions as well as to determine which factors were responsible for the greatest percentage of rejected applicants. If any of these rejection factors were amenable to correction through training, it would be possible to estimate the trade offs between the costs of additional personnel search versus the costs of training persons not quite acceptable under present standards. From the view of a science of decision making, this experiment lends some credence to the theory that human thinking can be denned as a complex network of simple binary choices. It is difficult to determine whether the analyst's decisions were or were not based on simultaneous consideration of multivariate criteria. But, by sequencing his thought proc-

esses in a series of binary alternatives, it was possible to replicate his final decision even though it was based on relatively complex interrelationships among variables. Research of this type gives deeper insight into the manner in which people resolve problems. The methodology allows the researcher to map an equivalent thought process at a particular point in time and could permit the study of the effects of aging and experience on decision-making capabilities of individuals and groups. Suggestions jor Further Research Work should be done to determine whether there is a significant difference in job performance between those personnel selected by human methods and those chosen through the use of a model. Of course, the problem still exists that a representative portion of the experimental group is lost since profit-oriented organizations usually do not hire applicants who have been rejected by skilled interpreters. The practice of obtaining "objective" evaluations of employee performance still permeates all forms of organizations. Even with the development of modern information systems, managers of all types are faced with the unpleasant task of judging the performance of their workers. Halo and leniency effects predominate. Perhaps it would be possible to design a simulation model whereby the records of each employee could be subjected to a kind of skilled, nonbiased, objective evaluation without fear of political pressure or subjective interpretations. Results of the study indicate that psychological and other factors can be integrated in a workable decision model This fact leads one to believe that computer-assisted psychological clinics for low-income classes might be feasible. Admittedly this idea connotes a world of cold impersonalism but there is the possibility that many people would willingly avail themselves of expert services even though these services were not obtained through direct human contact. Further experimentation along these lines would require the development of a model encompassing psychological test results and environmental factors. The model could then be used to screen large numbers of people, some of whom might

330

ROBERT D. SMITH KtEiNiiUNTZ, B. Profile analysis revisited: A heuristic approach. Journal of Counseling Psychology 1963,4,315-324 MARUYAMA, M. The use of computers as industrial counselors Computers and Automation, July 1966 34-39.
SMITH, R. D., & GREENLAW, P. S. Simulation of a

exhibit symptoms of present or potential mental illness. A computer-assisted screening center would be used to diagnose cases in early stages for referral to a specialist, thus providing minimum-cost psychological services to broader segments of our society. REFERENCES
CLARKSON, G P. Portfolio selection: A simulation of trust investment Englewood Cliffs, N J : Prentice-Hall, 1962
COLBY, K. M , & GILBLRT, J B. Programming a

psychological decision process in personnel selection. Management Science, 1967, 13, 409-419.
SWENSON, W. M (Chm.), ROME, H., MATAYA, P MCCARTHY, C , PEARSON, J., KEATING, F., & HATH-

computer model of neurosis. Journal of Mathematical Psychology, 1964, 1, 405-417 FINNEY, J. C. Methodological problems in programmed composition of psychological test reports Behavioral Science, 1967, 12, 142-152

AWAY, S. Symposium on automation technics m personality assessment Proceedings of the Slaf Meetings of the Mayo Clinic, Vol. 37, No. 3, January 1962, 61-82. TONCE, F . M. A heuristic program for assembly line balancing Englewood Cliffs, N. J.: Prentice-Hall. 1961. (Received September 12, 1967)

(Continued from pape 320) Skimming Lists of Food Ingredients Printed in Different Sizes E C. Poulton* Medical Research Council, Applied Psychology Research Unit, 15 Chaucer Road, Cambridge, England. Knowledge of Score and Goal Level as Determinants of Work Rate- Edwin A. Locke* and Judith I'. Bryan Department of Psychology, University of Maryland, College Park, Maryland 20740 Organizational Factors and Individual Performance: A Longitudinal Study George I" Farns* Massachusetts Institute of Technology, Sloan School of Management, Room 52-590, Cambridge, Massachusetts 02139 Conceptual and Operational Problems in the Measurement of Various Aspects of Job Satisfaction Martin G Evans*: School of Business, University of Toronto, 119 St. George Street, Toronto 5, Canada Relation between Birth Order and Being a Beautician Philip S. Very* and Joseph A. Zannini: Rhode Island College, 600 Mount Pleasant Avenue, Providence, Rhode Island 02908. Psychological Concomitants and Determinants of Vocational Choice Kenneth M Kunert*. Department of Psychology, University of Detroit, Detroit, Michigan 48221. Ratee Relevance in Peer Nominations: Frank T. Passini* and Warren T. Norman: Department of Psychology The University of Michigan, Ann Arbor, Michigan 48104. Punitive Supervision and Productivity: An Experimental Analog David R Schmitt: Department of Sociolog\, University of Washington, Seattle, Washington 98105 Marginal Productivity Procedure for Staff Selection Purnell H Benson*- 123 Milligan Place, South Orange. New Jersey 07079. Interaction of Achievement Cues and Facilitating Anxiety in the Achievement of Women W J. McKeachie* Department of Psychology, University of Michigan, Ann Arbor, Michigan 48104 * Asterisk indicates author for whom the address is supplied.

S-ar putea să vă placă și