Sensory Evaluation Practices

Davis

Preface

In this edition, readers will find that there have been changes in content as well as in authorship. After three editions, my co-author, Joel L. Sidel, has chosen not to continue, and I acknowledge his invaluable contributions. Two new co-authors are introduced. Both have decades of experience in sensory evaluation, both were students of Professor Rose Marie Pangborn, and both bring new perspectives and additional experiences to this edition.

It has been 8 years since the publication of the third edition of this book, and developments continue to be made, offering new approaches to the product evaluation process, to the analysis of the data, and to the applications of sensory resources in a wide range of product-related topics. One of these new topics is involvement in early stage or front-end research focused on techniques such as qualitative research, imagery, and related social psychological methods such as laddering and emotional profiling. These techniques provide insight into how consumers think about a product. In various publications, the senior author has called attention to the importance of sensory professional involvement in these activities as part of any product development effort. Awareness of these activities is important because it enables the sensory staff to decide how best to test a particular product, to become familiar with the language used by consumers, and to create more synergy within the research team.

Such efforts serve as a reminder of the high level of interest in the sensory sciences and specifically sensory testing activities. As noted in the previous edition, much remains to be done to improve the quality of the information obtained. Some concerns expressed previously remain concerns, such as an almost blind reliance on software that enables a wide range of multivariate analyses to be done without regard for whether the original data set yielded significant differences or whether the results make sense (i.e. whether it has face validity). The power of today’s PCs and access to cloud computing enable the sensory professional to provide information with minimal delay anywhere in the world on a 24-hour basis. It is a blessing and a curse; marketing and technology want to be able to respond quickly to reformulate or to proceed with a market test with minimal delay. This process makes sense, but the downside is the pressure on sensory professionals to make decisions without adequate time to determine if the results make sense. As Groopman (2009) noted, Statistical analysis is not a substitute for thinking—a statement that is essential for all sensory professionals to keep in mind when examining test results.

Despite these impediments, the field continues to grow and to attract interest. Whereas two decades ago there were very few texts on the topic, today there are literally dozens available and more in press. New journals devoted to sensory evaluation have appeared, and all are available in hard copy and/or via the Internet. Professional societies continue to grow. In addition, the number of conferences and short courses has increased dramatically. In any one year, there are at least 25 courses and as many as six conferences throughout the world. Perhaps most gratifying has been the increase in academic programs offering course work and degrees in sensory evaluation. This has occurred in both the United States and Europe, and some universities in China have offered courses in sensory evaluation. In addition, the University of California at Davis has been offering a yearlong web-based distance learning class since 2001 attended by more than 400 students throughout the world. These programs are welcome because they will eventually lead to a more scientific approach to the testing process. In this edition, we have reviewed the organizational issues and where necessary we have made changes that we believe will help maintain programs despite the many changes taking place in the consumer products industries. Consideration has also been given to the reviews of earlier editions and comments provided by numerous individuals who have written and/or talked with us about what is missing in the book.

As in previous editions, we provide detailed information about test methods, in part because there continues to be confusion about these methods. We encounter individuals describing a method that bears little resemblance to the method of the same name as described in the literature. Failure to understand the methods and how they were developed leads to their misuse, and recommended practices no longer appear to be practiced as rigorously as in the past. One important example is the use of analytical methods (i.e. discrimination and descriptive) without replication, which is akin to a chemical analysis done only once. Some have advocated the use of large numbers of subjects to avoid qualifying subjects or to compensate for subject variability. On the one hand, there is use of sophisticated statistical procedures, and on the other hand, there is use of data obtained randomly and/or without evidence that the individual was qualified to be in the test. Finally, the search continues for the universal scale and the invariant subject despite evidence that such a search will not be successful.

As stated in the first edition, and restated here, this book is not a review of the literature. We do, however, discuss literature relevant to specific issues and cite what we consider to be pertinent to the applications of sensory resources to provide actionable information. In this edition, we continue to emphasize the importance of planning, and the importance of learning how to manage and communicate product sensory information in a way that is actionable, that is understood by others.

Finally, we acknowledge the important early contributions of our longtime friend and associate, the late Professor Rose Marie Pangborn of the University of California at Davis, who worked tirelessly to educate students, encouraged them to pursue a career in sensory evaluation, and worked with a total commitment for the betterment of the science of sensory evaluation.

Herbert Stone

Rebecca N. Bleibaum

Heather A. Thomas

Chapter 1

Introduction to Sensory Evaluation

Chapter Outline

1.1 Introduction and objective

1.2 Historical background

1.3 Development of sensory evaluation

1.4 Defining sensory evaluation

1.5 A physiological and psychological perspective

1.1 Introduction and objective

Since publication of the third edition in 2004, developments continue to be made in the sensory sciences and their application in the product evaluation process. In today’s highly competitive, global business environment, company executives acknowledge the need for actionable product sensory information to supplement what they already believe they know about consumer behavior. This is especially the case for foods, beverages, and most other consumer products, including the ubiquitous electronic devices that have such a large impact on our daily lives. As noted elsewhere (see, for example, Stone and Sidel, 2007, 2009), brand managers and marketing researchers (also known as consumer insights professionals), like sensory scientists, look for ways to increase their knowledge about consumer responses to products before and after purchase. Of major interest is identifying variables besides those that are typically measured (e.g. preference) to gain a competitive advantage. Product sensory information has long been one of those variables often overlooked or not well understood, but it is now becoming a more integral part of a product’s business strategy. This has led to increased participation of sensory professionals as part of project teams versus being on the outside, waiting for test requests to arrive, if at all, without knowing the background or the basis for a request. Where successful, it has yielded benefits for the sensory professional in the form of improved status, increased compensation, and, for some, a stronger and earlier voice in a company’s product decision-making process. Direct and indirect rewards also were realized, including willingness to support university research, more course offerings, and an increasing demand for newly trained sensory professionals. Another development has been a media connection. In the past, the likelihood of media mention of sensory evaluation was rare; today, there are many books, blogs, and other media outlets describing results from sensory tests or interviews with individuals possessing unique sensory skills. Although one may question some of these claims/curiosities, this exposure has brought more attention to the field and that is a positive effect. Changes within the profession also continue to take place, such as more professional meetings and symposiums discussing research results. Other changes are more focused on the ways in which professionals design, analyze, and report results. There has been a major increase in use of direct data capture and a concomitant reduction in use of paper ballots—a more rapid turnaround from data collection to reporting results. The availability of relatively inexpensive software has enabled professionals to more easily undertake design studies. However, none of this has come without some costs—for example, use of software not appropriate for behavioral data or using data that are not appropriate for a particular analysis. This makes it relatively easy for the inexperienced professional to satisfy a request but significantly increases the likelihood of decisions errors. As Groopman (2009) noted as part of a discussion about the state of clinical trials but equally relevant to sensory tests, Statistical analysis is not a substitute for thinking. Just because one obtains a graphical display or a series of tables with associated statistical significances does not mean it has any meaning or external validity. Considering the major increases in computing power and the trivial cost, one can undertake a wide range of analyses with little or no knowledge as to their relevancy or if the data set is appropriate for that analysis (May, 2004). One needs to appreciate that output will always be obtained regardless of the quality and quantity of the original data. When unexpected product changes are obtained and not understood, it is the responsibility of the sensory staff to explain the result and minimize confusion. We explore these issues in more detail later in this book.

As mentioned previously, using sensory information as a part of a product marketing strategy has given it unprecedented attention. Identifying specific sensory and chemical measures that have a significant effect on preference and purchase intent has important consequences for a company. In those instances in which this information has been used, its significance has been recognized and the product success in the marketplace is appreciated. Clearly, it is a powerful approach to enhancing product preference versus using a simple trial-and-error process. However, this has only been possible with the use of quantitative descriptive analysis to identify the sensory differences and similarities among competitive products and the availability of user-friendly software. The next logical step in this process has been to incorporate imagery into the process. Exploiting this information to the fullest extent possible has enabled companies to grow their market share as well as implement cost savings through better use of technology, etc. (Stone and Sidel, 2007, 2009). All this has been possible as a direct result of using sensory resources effectively, a better understanding of the measurement process, combined with a more systematic approach to the testing. Much of this progress has been achieved within the technical and marketing structures of companies that recognized the unique contributions of sensory evaluation. In the past, such developments were the exception; today, it is a more common occurrence, again reflecting the increased awareness of sensory information. For a summary of these developments, the reader is directed to Schutz (1998). However, much more needs to be done, in part because the links between sensory, marketing, and production are not strong and in part because there is a lack of appreciation for the principles on which the science is based. For some, sensory evaluation is not considered a science capable of providing reliable and valid information. This is not so surprising, given that this perception is fostered in part by the seemingly simplistic notion that anyone can provide a sensory judgment. We are born with our senses and barring some genetic defects, we are all capable of seeing, smelling, tasting, etc. It certainly seems simple enough, so why should a technologist or a brand manager believe results from a test that are inconsistent with their expectations and their own evaluations? As a result, demonstrating that there is a scientific basis to the discipline continues to be a challenge. Further challenges develop when tests are fielded without qualified and sufficient numbers of subjects, again leading to incorrect recommendations. It is no longer a surprise to hear statements such as We don’t have the time or money to do it right, but we will be able to do it over again later. It takes a lot of effort to overcome this kind of thinking. Since the previous editions of this book, advances continue to be made, albeit at a slow pace, not because test procedures are inadequate but, rather, as noted previously, because the science is not readily acknowledged as such. In all fairness, it should be noted that sensory professionals have not been effective spokespeople for their work or for the science. In one company, sensory evaluation will be used successfully, but in another it will be misused or the information will be ignored because it is inconsistent with expectation. Unfortunately, this latter situation has encouraged use of other information sources or the development of competing test capabilities in the hopes of obtaining acceptable information without fully appreciating the consequences.

Throughout the years, numerous efforts have been made and continue to be made to develop a more permanent role for sensory evaluation within a company. Reviewing the technical and trade literature shows that progress in the development and use of sensory resources continues. There has been a noticeable increase, and much of the impetus continues to come from selected sectors of the economy, notably foods and beverages, and their suppliers (Jackson, 2002; Lawless and Heymann, 2010; Meiselman and MacFie, 1996; Piggott, 1988). In their seminal textbook on sensory evaluation published more than 45 years ago, Amerine et al. (1965) correctly called attention to three key issues: the importance of flavor to the acceptance of foods and other products, the use of flavor-related words in advertising, and the extent to which everyday use of the senses was largely unappreciated, at that time. Perhaps a secondary benefit of today’s concerns about food safety has been awareness by consumers of the sensory aspects of the foods they purchase.

Current interest in sensory evaluation reflects a more basic concern than simply being able to claim use of sound sensory practices. A paper published more than three decades ago (Brandt and Arnold, 1977) described the results of a survey on the uses of sensory tests by food product development groups. The survey provided insight into some of the basic issues facing sensory evaluation then and now. Of 62 companies contacted, 56 responded that they were utilizing sensory evaluation. If this question were asked today, one would expect the number to be close to 100%. Since publication of that survey, there have been many similar surveys with additional results reflecting greater interest in and use of sensory evaluation. However, descriptions of tests being used revealed then, as now, that confusion existed about the various methods; for example, it was found that the triangle test (a type of discrimination test) was the most popular, followed by hedonic scaling (a type of acceptance test) and paired comparison (either an acceptance test or a discrimination test). Because these and other methods mentioned in the survey provide different kinds of information, it is not possible to evaluate the listing other than to assume that most companies use a variety of methods.

Not surprisingly, there was confusion about acceptance test methods and the information that each provides. For example, single-sample presentation is not a test method, but 25 of the 56 companies responded that it was one of the test methods in use. It is in fact a serving procedure, and such responses may reflect poorly worded or a misunderstood question in the survey. Single-sample testing would be referred to as a monadic test. Another example of the confusion is scoring, which was said to be in use by only 7 of the 56 companies contacted. However, all sensory tests involve some kind of scoring. Statistical tests included in the survey elicited similarly confusing responses. The failure to clearly define the terms confirms that the consumer packaged goods industry uses sensory evaluation but there is confusion as to what methods are used and for what applications. As previously mentioned, sensory evaluation is still not well understood, methods are not always used in appropriate ways, and results are easily misused. There continues to be a lack of qualified professionals, but more are being trained. Books on sensory evaluation continue to be published; however, the science still has not achieved a status commensurate with its potential. Similarly, its role within individual companies is far from clear. One of the goals of this text is to provide a perspective on all these issues and how one should develop resources and integrate them into the business environment. This is not to imply that the scientific basis of sensory evaluation is overlooked. Rather, it is not as well understood as it should be considering its widespread use and strategic potential.

This edition continues to emphasize the importance of a systematic approach to the organization, development, and operation of a sensory program. Primary emphasis is directed toward the more practical aspects of sensory evaluation and approaches to implementing a respected and credible program, but we do not neglect the fundamental, underlying issues, including but not limited to experimental design, the reliability and validity of results, and related topics. From a sensory science perspective, reliability and validity are essential to developing a credible program and providing actionable recommendations that enable a company to build a strong brand consistent with its brand strategy (Aaker, 1996). From a business perspective, it is these latter issues that loom as most important because they build trust that others will place on these recommendations and the extent to which managers will act on them. Sensory professionals must communicate information clearly to ensure that superiors, peers, and subordinates understand what was done and what the results mean in terms of meeting that test’s specific objective and how the research facilitates the product link with the business strategy.

The importance of the relationship between reliability and validity of results and the credibility assigned to sensory recommendations and, in a global sense, the credibility of a sensory program are at stake. Although it is logical to assume that a test result should speak for itself, and that it will be understood, this is not the usual situation. It takes a conscious effort to gain the confidence of a brand manager and consumer insights advisors so that recommendations will be acted on sufficiently. When it succeeds, it is reassuring, but more often it does not, which raises questions as to why sensory information is not better understood and used more effectively. There is no single or simple answer. However, it is clear that the ease with which one can evaluate a product makes it easy for a product manager to disregard information that does not satisfy expectations and substitute one’s own judgment. That and an unwillingness to appreciate the complexity of the judgmental process leads to decision problems and product failures. However, it is the responsibility of the sensory professional to be sure that tests are organized and fielded using best practices, that replication is an integral part of every sensory analytical test, that subjects are qualified and understand the task, that the scorecard is relevant to the test objective, etc. Product evaluation is a complex multistep process in which a group of individuals respond to stimuli (a set of products) by marking a scorecard (electronically or by paper ballot) according to a specified set of instructions. There are many steps in the process where errors can occur, and not all of them are obvious. Of particular importance is the matter of whether the test objective is understood and the requester can explain how the results will be used; that is, what questions will be answered? Failure to obtain such information is a clear sign of problems when reporting results. Once there is agreement on the objective, a test plan can be established, a design prepared, and actual fielding initiated. As already emphasized, there are many challenges faced by sensory professionals; some are self-induced, whereas others come from requesters specifying the test method, the type of scale, or output that yields a single numerical value. The concept of number biases impacting responses seems to have been lost or not at all understood. Measurement or scaling is another area of confusion. Requests to use only a company’s standard scale so that it is possible to compare results reflect a poor understanding of the measurement process, as is the request to use a universal scale for which there is no evidence. These measurement issues are discussed in detail in Chapter 3.

Subjects have a major impact on a program’s credibility. How they were selected and what kind of training, if any, they received are important considerations, as are their responses, in terms of both their sensitivity and their reliability. To a degree, a panel of subjects can be considered as functioning like a null instrument, recording what is perceived.

Within the sensory community, there is the implication of a human (or group of humans) functioning like an instrument, which has obvious appeal in a technical sense. However, there is an equally large body of scientific evidence that recognizes individual variability in perception based on genetic differences at the receptor level, differences in sensory acuity, and experiential differences that all impact perception. The human instrument position within sensory science is sensitive, particularly to those who mistakenly envision an invariant system yielding the same numerical values time and time again. It has particular appeal among those that lack an understanding of psychology, human behavior, and the perceptual process. The realities of the situation are, of course, quite different. Subjects, no matter what their level of skill or number of years of training and practice, exhibit differences in sensitivity from one another, as well as differences in variability that are unique (to the individual). Some training programs (e.g. see Spectrum analysis in Meilgaard et al., 2006) imply that this sensitivity and accompanying variability can be overcome through training and use of appropriate reference standards that represent absolute intensities. Such training, as much as 10 hours per week and often lasting 4 or more months, has considerable appeal; however, there is no evidence that such an approach has scientific merit. Lawless and Heymann (2010) stated, We are somewhat skeptical of this claim since there are no published data to support it (p. 360). It is inconsistent with our knowledge of human perception and the physiology of the senses. There are no peer-reviewed results reported in the literature, and one should not expect to obtain invariance. Theoretically, if panelists were in complete agreement one would expect the standard deviation for any specific product-attribute combination to be close to zero. However, most Spectrum studies have attributes with non-zero standard deviations indicating that the panel is not absolutely calibrated (Lawless & Heymann, 2010, p. 361). Such an approach is a form of behavior modification rather than a means of capturing responses as a function of a stimulus (whether that stimulus is a purified chemical or a consumer-ready beverage). It ignores the fact that subjects themselves are changing within a trial and across time. The stimulus also changes, which is why one uses replication to capture sufficient data to partition the different sources of variation before concluding that the difference of interest was detected. It is misguided to assume that a human can be invariant, just as it is misguided to assume that an instrument yields the same value every time it is used and also to assume that product is always the same. In each instance, the end result is to overcome what appear to be limitations of the sensory process in the mistaken belief that they will make results more acceptable. Short of directly telling an individual what should be an answer, there will always be variability. Nonetheless, the requestor of a test expecting that test to yield an invariant result (the same response or numerical value every time) is disappointed and concerned about this unique information source. This disappointment is also accompanied by reservations as to any conclusions and recommendations; that is, the issue of credibility arises. Alternatively, when results are not in agreement with information obtained elsewhere (and often not sensory information) and no attempt is made to understand and explain the basis for any disagreement, then further erosion of program credibility will occur. After all, products are not the same despite a documented formulation. The precision of weighing and volumetric devices, flow through pumps, and heating and cooling equipment varies so that the finished product will exhibit this variation. To somehow expect a subject to provide the same judgment each time the product is presented is naive, at best.

Sensory professionals must not ignore fundamental scientific truths in our data collection instrument: the human. If sensory is indeed a science, we must embrace the knowledge of genetic differences at the receptor level, perceptual differences, and behavioral differences and provide scientifically derived quantitative data to measure and understand perception.

The success of a sensory program, and particularly its credibility, begins with having a plan and an organized effort and making sure that the testing process follows accepted procedures and practices—what method was used, who the subjects were and how they were selected, what test design was used, how the data were analyzed, including the evidence of reliability, and so forth. In a business sense, it begins with an explanation of what sensory information is and is not, how results are communicated and whether they are understood, and ends with actionable recommendations being implemented to everyone’s satisfaction. Although these issues are discussed in detail in succeeding chapters, their inclusion here is to provide an initial perspective to the issue of business credibility and direct involvement of sensory evaluation in product decisions. Without an organized product evaluation effort and demonstrated reliable and valid results that are communicated in a way that is readily understood, one is returning to reliance on product experts, the "N of 1," who made product decisions by virtue of their expertness and not because there were data to support their judgments. Decisions derived in this manner are neither new nor unusual; however, they make it very difficult for individuals trying to organize and operate a credible sensory test program. As noted by Eggert (1989) and by Stone and Sidel (1995), sensory evaluation must develop a strategy for success. It must reach out to its customers; it must educate them about the benefits that can be realized from using sensory information. At the same time, it must gain management support through an active program of selling its services and how the company can benefit from those services.

This book is not intended as an introduction to the topic of sensory evaluation. Nonetheless, for some aspects of sensory evaluation, considerable detail is provided in an almost stepwise manner. Readers, however, will probably derive more from this book if they have a basic understanding of sensory evaluation, experimental design and statistics, and especially the perceptual process.

Where appropriate, background information sources are cited in this text and should be considered recommended reading. In addition to its benefit to the sensory professional, this book is intended to serve as a guide for the research and development executive seeking to have a capability in sensory evaluation and to develop a more efficient and cost-effective product development program. It should also be of interest to marketing, market research/consumer insights, and technical managers, all of whom have an interest in their company’s products and their quality as measured by consumer responses and through sales, market share, and profitability.

1.2 Historical background

Of the many sectors of the consumer products industries (food and beverage, cosmetics, personal care products, fabrics and clothing, pharmaceutical, etc.), the food and beverage sectors provided much early support for and interest in sensory evaluation. During the 1940s and through the mid-1950s, sensory evaluation received additional impetus through the U.S. Army Quartermaster Food and Container Institute, which supported research in food acceptance for the armed forces (Peryam et al., 1954). It became apparent to the military that adequate nutrition, as measured by analysis of diets or preparation of elaborate menus, did not guarantee food acceptance by military personnel. The importance of flavor and the degree of acceptability for a particular product were acknowledged. Resources were allocated to studies of the problem of identifying what foods were more or less preferred as well as the more basic issue of the measurement of food acceptance. These problems and their solutions were overlooked or forgotten during the 1960s and early 1970s when the federal government initiated its War on Hunger and Food from the Sea programs. The government’s desire to feed the starving and malnourished met with frustration when product after product was rejected by the recipients primarily because no one bothered to determine whether the sensory properties of these products were acceptable to the targeted groups. This is not to suggest that each country’s ethnic and regional food habits and taboos were not important but, rather, in the context of these government programs, there was scant attention given to the sensory evaluation of the products as they were being developed. Unfortunately, this situation continues to exist because there remains a fundamental lack of appreciation for the importance of the sensory properties of the consumer packaged goods being developed. There is no question that providing nutrients to those in need is the first priority, and in extreme situations that is about all there is. However, large giveaway programs often yield less than satisfactory results that are well below everyone’s expectations, in part because the sensory properties fail to come close to the expectations of a culture or intended consumer group.

The food and beverage industry, possibly taking a cue from the government’s successes and failures in sensory evaluation (Peryam et al., 1954), provided support for this emerging science. Although many industries have since recognized its value in formulating and evaluating products, general appreciation for sensory evaluation as a distinct function within a company remained relatively minimal until approximately the past two decades. In general, there is agreement on the role of sensory evaluation in industry but not necessarily how sensory evaluation should be organized and how it should operate within a company. As with any discipline, divergent opinions and philosophies on sensory evaluation exist both within and outside the field. It is not necessary that we examine all these opinions and philosophies in detail; however, some discussion is appropriate to enable the reader to gain a greater appreciation for the challenges involved in the organization and operation of a sensory program.

The food and beverage industry traditionally viewed sensory evaluation in the context of the company expert (the N of 1) who through years of accumulated experience was able to describe company products and set standards of quality by which raw materials would be purchased and each product manufactured and marketed. Examples of such experts include the perfumer, flavorist, brewmaster, winemaker, and coffee and tea tasters. In the food industry, experts provided the basis for developing the cutting sessions and canning bees (Hinreiner, 1956). In the canning industry, products usually were evaluated on a daily basis and in comparison with the previous day’s production, competitive products, new products, etc. In addition, there were industrywide cutting bees to assess general product quality. These sessions enabled managers and experts to assess product quality at their own plants as well as to maintain a familiarity with all other companies’ products. This process continues today in most companies as well as in trade associations seeking to solve common problems that are usually related to product quality. In recognizing the purpose of the cutting bee and its overall function, Hinreiner described the efforts undertaken to improve the quality of the information derived from one group, the olive industry. The Processed Products Branch of the Fruit and Vegetable Division of the U.S. Department of Agriculture (File Code 131A-31, 1994) has updated its guidelines for approved illumination for cutting rooms, an action that recognizes the importance of providing a more standardized environment for product evaluations. In addition to the evaluation, record-keeping was formalized, making it possible to compare results from one year with those of another and thus provide for a great degree of continuity. It is important to remember that the industry recognized a problem, and with assistance from sensory evaluation, it took some action to improve its product quality information. This activity continues today, especially in industries that rely on basic agricultural products that do not experience substantial value-added processing—for example, the wine industry, fresh fruits and vegetables, and olive oil. The enormous attention given to the Mediterranean diet and emphasis on olive oil has resulted in a major increase in consumption of olive oil, specifically extra virgin olive oil. This in turn has resulted in renewed focus on how olive oil quality is determined, along with conferences about olive oil, its nutritional benefits, and the criteria by which quality is determined. As such, extra virgin olive oil is an interesting case study to illustrate new challenges with combining expert evaluations, U.S. Department of Agriculture (USDA) grading, and sensory sciences. This topic is discussed in more detail later in this book.

Cutting sessions and the work of product experts have continued; however, the impact of both has been reduced but not entirely eliminated. In retrospect, the results accomplished (and failed to be accomplished) by these experts and the cutting sessions were quite remarkable. By and large, experts determined which alternatives—from among many alternative ingredients, products, and so forth—were appropriate for sale to the consumer. Their success reinforced their role for establishing quality standards for particular products, such as canned fruits and vegetables, and these standards in turn received additional support through official USDA standards that referenced these results. As long as the food industry was involved solely in the preserving of a basic agricultural crop (e.g. frozen peas, canned fruits and vegetables, juice, or oil such as extra virgin olive oil), then it was relatively easy (or uncomplicated) for the product expert to understand a particular product category and to make reasonable sound recommendations.

In the early stages of the growth of the food-processing industry and where competition was primarily regional in character, such standards and evaluative procedures by experts were extremely useful. In most industries, experts also created an array of scorecards and unique terminologies to serve as a basis for maintaining records and presenting a more scientific process. Subsequently, numerical values were assigned to the various attributes being measured, as described by Hinreiner (1956). These scores soon became targets or standards; for example, the 100-point butter scorecard, the 10-point oil quality scale, and the 20-point wine scorecard all had specific numbers that connoted levels of product acceptance (equated with quality). All of these and others continue to be used within their respective industries. Certain values became fixed in people’s minds, and they were transposed inappropriately into measures of consumer acceptance, creating a multitude of new problems. That some of these scorecards have survived virtually intact after 50+ years is remarkable, considering their weaknesses. Where they have not survived one can usually find the concept still alive, particularly that of the single number equated with quality and the associated belief of the invariance of the expert. Although it is more common in quality control, the reemergence of experts in sensory evaluation is somewhat surprising and discouraging. Not only does it reflect a basic lack of understanding of human behavior and the perceptual process but also it may reflect a wistful desire of some to reduce behavior to some simplistic level. As noted previously, humans are not invariant and their responses to products are not invariant; they should not be as no two products are the same. Sensory professionals do an injustice to themselves and to the science when they embrace these simplistic notions about human behavior without fully appreciating the consequences. They also do a disservice when they participate as subjects, thereby perpetuating the notion of the expert, the N of 1 who can make these absolute judgments. In one sense, the activities associated with product grading, merit awards in product competitions, obtaining the right to label a product as coming from a particular growing region, and so on are linked with sensory evaluation but not in the mainstream of how sensory professionals define the work that they do. These systems provide a basis for assuring the buyer that the particular product meets the previously established criteria. They also have value when buying and selling goods; however, they do not reflect the marketplace, which is dynamic, whereas grading systems are static. The problem arises when the leap is made to assume that highest grade equates with highest quality and with product best liked by the consumer. There is more to product success than simply a designated emblem of best quality.

With the growth of the economy and competition and the evolution of processed and formulated foods, experts faced increasing difficulty in maintaining an awareness of all developments concerning their own product interests. As a further complication, product lines expanded to the extent that it was virtually impossible for an expert to have detailed knowledge about all products, let alone the impact of different technologies. While the expert was required to continue making finite decisions about product quality, consumer attitudes were changing in ways that were not fully appreciated. With the development of contemporary measurement techniques and their application to sensory evaluation, it became evident that reliance on a few experts was questionable. To deal with this problem, some companies turned to sensory evaluation (which was often referred to as organoleptic analysis in the early literature). In truth, companies did not turn directly to sensory evaluation as a solution to the inadequacies of experts; rather, the marketplace created opportunities. As competition increased and became more national (and eventually international) in scope, the need for more extensive product information became evident. Managers were disappointed with results from some types of consumer tests and/or costs became increasingly difficult to justify to management, and now they were more willing to consider alternative sources of product information. For those companies in which there were sensory resources, opportunities developed, and in some instances considerable success was achieved. To that extent, sensory evaluation represented a new and as yet untried resource. Before discussing this contemporary view, it is necessary to further explore the earlier developments of sensory evaluation.

As noted previously, sensory evaluation was of considerable interest in the late 1940s and into the 1950s, prompted in part by the government’s effort to provide more acceptable food for the military (Peryam et al., 1954), as well as by developments in the private sector. For example, the Arthur D. Little Company introduced the Flavor Profile Method (Caul, 1957), a qualitative form of descriptive analysis that minimized dependence on the technical expert. Although the concept of a technical expert was and continues to be of concern, the Flavor Profile procedure replaced the individual with a group of approximately six experts (whom the technical expert trained) responsible for yielding a consensus decision. This approach provoked controversy among experimental psychologists who were concerned with the concept of a group decision and the potential influence of an individual (in the group) on this consensus decision (Jones, 1958). Nonetheless, at that time and continuing to the present, the method provided a focal point for sensory evaluation, creating new interest in the discipline, which stimulated more research and development into all aspects of the sensory process. This topic is covered in more detail in the discussion on descriptive methods in Chapter 6.

By the late 1950s, the University of California at Davis was offering a series of courses on sensory evaluation, providing one of the few academic sources for training of sensory evaluation professionals. Other universities, including Oregon State University, the University of Massachusetts, and Rutgers University, offered course work in sensory evaluation but not to the extent as offered by the University of California. Subsequently, many other universities initiated independent courses in sensory evaluation, producing more professionals in the discipline. These developments are reflected in the food science literature of this same period, which includes many interesting studies on sensory evaluation by Boggs and Hansen (1949), Giradot et al. (1952), Baker et al. (1954), Harper (1950), Pangborn (1964), and Anonymous (1968). These studies stimulated and facilitated the use of sensory evaluation in the industrial environment. The early research was especially thorough in its development and evaluation of specific test methods. Discrimination test procedures were evaluated by Boggs and Hansen (1949), Giradot et al. (1952), and Peryam et al. (1954). In addition to discrimination testing, other measurement techniques also were used as a means for assessing product acceptance. Although scoring procedures were used as early as the 1940s (Baten, 1946), primary emphasis was given to use of various paired procedures for assessing product differences and preferences. Rank-order procedures and hedonic scales became more common in the mid- to late 1950s. During this time period, various technical and scientific societies, such as Committee E-18 of the American Society for Testing and Materials (now ASTM International), the Food and Agriculture Section of the American Chemical Society, the European Chemoreception Organization, and the Sensory Evaluation Division (now the Sensory and Consumer Sciences Division) of the Institute of Food Technologists, organized activities focusing on sensory evaluation and the measurement of flavor. For a review of the activities of ASTM, see Peryam (1991) or any of the more recent documents on the topic.

1.3 Development of sensory evaluation

It would be difficult to identify any one or two developments that were directly responsible for the emergence of sensory evaluation as a unique discipline and its acceptance (albeit, on a limited basis) in product business decisions. Certainly the international focus on food and agriculture in the mid-1960s and into the 1970s (and that continues today), the energy crisis, food fabrication, the cost of raw materials (Stone, 1972), competition, and the internationalization of the marketplace have, directly or indirectly, created opportunities for sensory evaluation. For example, the search for substitute sweeteners stimulated new interest in the measurement of perceived sweetness along with an interest in time-intensity measures. This, in turn, stimulated development of new measurement techniques (Inglett, 1974) and indirectly stimulated interest in development and use of direct data entry systems as a means for evaluating the sweetness intensity of various ingredients (for more information about the latter topic, see Anonymous, 1984; Gordin, 1987; Guinard et al., 1985; Winn, 1988). Today, this situation has changed with respect to capturing responses and real-time data analysis; other opportunities remain to be satisfied. Whether companies are developing new products, attempting to enter new markets, or attempting to compete more effectively in existing markets, the need for sensory information remains (Stone, 2002). Although much more could be written on and speculated about these opportunities and their antecedents, it is more important that our attention be focused on how this sensory resource should be structured so it can function more effectively in the future.

After a long and somewhat difficult gestation, sensory evaluation has emerged as a distinct, recognized scientific specialty (Sidel et al., 1975, see also Sidel and Stone, 2006; Stone and Sidel, 1995). Although the focus of the former article was on the use of sensory evaluation in the development of fabricated foods, there were implications for sensory evaluation in general. As a unique source of product information, it had important marketing consequences, providing direct, actionable information quickly and at low cost. It was proposed that organizing of sensory evaluation test services along well-defined lines (e.g. formal test requests, selection of a test method based on an objective, and selection of subjects based on sensory skill) would increase the likelihood of such services being accepted as an integral part of the research and development process or other business units within a company. It has become clearer that without an organized approach, a management-approved plan, and an operational strategy, sensory resources are rarely used effectively and are less likely to have a significant, long-term impact.

In a short course given several decades ago, Pangborn (1979) called attention to misadventures that have occurred in sensory evaluation. The article was one of several by this author as part of her continuing efforts to improve the quality of the research being done and indirectly enhance the skill level of sensory professionals in companies. The three issues of particular concern were the lack of test objective, adherence to a test method regardless of application, and improper subject selection procedures. These three issues remain even now. These are not the sole property of the sensory literature (and by default many in teaching roles) but also are quite commonplace in the business environment. It is clear that much more needs to be done to improve the quality of sensory information.

An interesting development for sensory evaluation has been the continued growth in the number of short courses and workshops being offered. When there were few university offerings, such programs served a useful purpose for individuals with responsibility for their company’s sensory program. In the past decade, there has been a quantum increase in the number of courses being offered, including distance learning certificate programs in sensory and consumer sciences, which suggests that university offerings are still insufficient for industry’s needs. Our own experience in offering courses during the past four decades reflects a continued interest in sensory evaluation, especially the more practical issues of developing a program within a business environment. Some of the material presented in this book evolved from workshop material that has proven especially beneficial to participants. Newspapers and other public information sources present articles about sensory evaluation (not regularly, but often enough to be noticeable). These articles usually include some impressive revelations (to attract the reader) about the special tongue or nose of certain individuals who are claimed to have almost mystical powers. These individuals are generally associated with such products as wine, beer, coffee, and fragrance, or they function as wine and food writers. Still other approaches convey an impression that the subconscious mind is being tapped by a new sensory technique, with the end result being the ideal consumer product. A current example is the development of methodology described as emotional profiling about which more can be found in Chapter 6. Although sensory professionals should welcome new methods, such developments need to be examined before their adoption. After all, it is very easy to obtain a response from a consumer/subject; it does not mean that the question was understood or that the responses will yield actionable information and eventually a superior product versus competition. Despite this lack of consistency, continued discussion about sensory evaluation is helpful to the field, if only because it reaches key people who might not otherwise read about it. These changes, and a greater awareness of sensory evaluation, appear to have coincided with a dramatic shift by the consumer packaged goods industry toward a stated consumer-oriented environment and away from the more traditional manufacturing/production-oriented environment. By that we mean a recognition that understanding consumer attitudes and behavior is essential information and ought to be known before one formulates a product rather than manufacturing a product and looking to others (e.g. marketing) to convince the consumer to purchase that product.

Opportunities for sensory evaluation continue to develop primarily as a result of significant changes in the marketplace and to a much greater extent than changes in sensory evaluation methodology. Mergers, leveraged buyouts, and other financial restructuring activities and the internationalization of the marketplace have created even greater challenges in the consumer products industry. There are numerous choices in terms of brands, flavor alternatives, convenience, pricing, new products, and combinations not thought of a decade ago (e.g. yogurt beverages). Many companies have determined that new products at competitive prices are essential for long-term growth and success. However, this has presented its own unique challenges and risks (Meyer, 1984). New product development and the proliferation of choices within a product category rapidly accelerated in the 1980s at a rate neither appreciated nor believed possible in the past. This acceleration was accompanied by considerable financial risk (Anonymous, 1989). In separate 1977 publications on the topic, Crawford and Carlson determined that the failure rate of new products has, at times, been as high as 98% for all new products. In the time between then and now, 30+ years, one can find similar reports in the trade literature; clearly, this situation has not changed very much and certainly not for the better. From a business perspective, this risk severely challenges creative skills and available technical resources and has provided renewed interest in resources such as sensory evaluation (Stone, 2002; Stone and Sidel, 2007) as well as alternative approaches to the development process (Smith, 2007; Smith and Oltmann, 2010). Companies are now more receptive to new approaches and to new ways of anticipating and measuring the potential for a product’s success in the marketplace. Of course, some companies may choose to not introduce new products and thereby minimize much of that risk but to rely on brand and line extensions (Lieb, 1989). Here too, the need for sensory information is essential if such products are to be brought into the market within reasonable time and budgetary considerations. These changes should have accelerated the acceptance of sensory evaluation; however, this has not occurred to any great extent until recently. Companies are now more aware of sensory evaluation; however, the organization and operation of sensory business units with full management support still lag other related activities such as consumer insights (a successor to marketing research). Nonetheless, the fact that some programs are fully operational bodes well for the