Sensory Evaluation Practices

Sensory Evaluation Practices

Fifth Edition

Herbert Stone

Rebecca N. Bleibaum

Heather A. Thomas

Table of Contents

Cover image

Title page

Copyright

Foreword

Preface

Chapter 1. Introduction to sensory evaluation

Abstract

Chapter outline

1.1 Introduction and objective

1.2 Historical background

1.3 Development of sensory evaluation

1.4 Defining sensory evaluation

1.5 A physiological and psychological perspective

Chapter 2. Organizing and operating a sensory science capability

Abstract

Chapter outline

2.1 Introduction

2.2 Organizing sensory resources

2.3 Conclusions

Chapter 3. Measurement

Abstract

Chapter outline

3.1 Introduction

3.2 Components of measurement: scales

3.3 Selected measurement techniques

3.4 Conclusion

Chapter 4. Test strategy and the design of experiments

Abstract

Chapter outline

4.1 Introduction

4.2 Test request and objective

4.3 Product criteria

4.4 Sources of variability

4.5 Psychological errors

4.6 Statistical considerations

4.7 Experimental design considerations

4.8 Selected product designs

4.9 Worked examples

Chapter 5. Discrimination testing

Abstract

Chapter outline

5.1 Introduction

5.2 Methods

5.3 Components of testing

5.4 Special problems

5.5 Summary

Chapter 6. Descriptive analysis

Abstract

Chapter outline

6.1 Introduction

6.2 Test methods

6.3 Applications for descriptive analysis

6.4 Conclusions

Chapter 7. Affective testing

Abstract

Chapter outline

7.1 Introduction

7.2 Methods

7.3 Subjects

7.4 Types and locations of acceptance testing

7.5 Special issues

7.6 Conclusions

Chapter 8. Strategic applications

Abstract

Chapter outline

8.1 Introduction

8.2 Front end of product innovation

8.3 Product improvement and new product development

8.4 Product optimization—creating best in class products

8.5 Sensory, physical, and chemical relationships

8.6 Stability testing—establishing shelf life

8.7 Quality control

8.8 Sensory specifications

8.9 Marketplace audits

8.10 Extended use testing

8.11 Sensory claims for advertising and legal disputes

8.12 Conclusion

Chapter 9. Epilogue

Abstract

Chapter outline

9.1 Introduction

9.2 Educating the sensory professional

9.3 Professional ethics

9.4 The future

Bibliography

Further reading

References

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2021 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

ISBN: 978-0-12-815334-5

For Information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Charlotte Cockle

Acquisitions Editor: Megan Ball

Editorial Project Manager: Barbara Makinster

Production Project Manager: Sruthi Satheesh

Cover Designer: Greg Harris

Typeset by MPS Limited, Chennai, India

Foreword

It is an honor to offer this foreword for Sensory Evaluation Practices, as it has been my trusted guide since 1985. I retain my original copy of the first edition on my shelf at home, where it remains useful and oddly loved. Why loved? This single book had been my go-to on hundreds of occasions during my professional career as a sensory scientist. Early in my career I relied on it to conduct a self-audit of our laboratory, and then later to set up practices in other labs around the world. It was my reliable advisor. My consultant. It gave me confidence and courage. I understood it. It has always been practical and perfectly useful.

The authors of Sensory Evaluation Practices have a collective 120+ years of diverse, hands-on field experience. They have coded many cups, devised thousands of test plans, written novel algorithms, interpreted complex statistics, advised the largest consumer products companies globally, led professional associations, and taught in universities and short courses, all with excellence. They are icons in the field, yet the book addresses the topics in an almost conversational tone, with practical examples and clear graspable discussion.

The field continues to evolve as consumer product companies demand more reliable tools for rapid development of winning products, for assuring the ongoing quality of manufactured products, and for advertising and legal claims. I expect that future editions will address more comprehensive practical methods for measurement of the integrated human response, methods that incorporate artificial intelligence and machine learning, all to better map the human experience of product consumption.

As I write this foreword, our planet is battling COVID 19. For sensory scientists this pandemic has an interesting clinical symptom: the loss of smell and/or taste. This unique effect will provide researchers with fresh insight into how the chemical senses are perceived and processed, and provide the medical community with new clinical methods for practical diagnostic testing. Perhaps COVID 19 will provide useful test methods for the sensory science community, and be included in future editions of this book.

I am pleased that Sensory Evaluation Practices continues its 35-year legacy of providing sound, clear, thought-provoking guidance for the sensory community. This new edition will sit next to my old brown copy on the shelf, with its updated practical knowledge and advice. It will continue to serve thousands of practitioners new to Sensory Science, as well as to the executives responsible for sensory programs worldwide. If you could afford only one book on the subject, Sensory Evaluation Practices is the single reference to stock. It is practical, simple to understand, and thorough its treatment of the methods. You may even come to love it.

Marianne Gillette

Retired, McCormick and Co, Inc.

IFT Past President

Preface

Herbert Stone, Rebecca N. Bleibaum and Heather A. Thomas

This fifth edition, like previous editions, emphasizes the fundamental principles of sensory evaluation as well as identifying newer ways of obtaining sensory information. Probably the biggest impact has been the use of the Internet, providing the sensory scientist with more resources and more ways of being responsive. It has given birth to an array of methods allowing for rapid data capture and an almost limitless array of hardware and software for data collection and analyses on a real-time basis. One reads of a great future as a result of the use of artificial intelligence and machine learning. These serve as a reminder of the high level of interest in sensory information and what the future may have in store for the field.

There has been expansion of sensory ideas and techniques in early stage or front-end research focused on research, imagery, and related social psychological methods such as laddering and emotional profiling. These techniques provide insight into how consumers think about a product. In various publications, the authors have called attention to the importance of sensory professional involvement in these activities as part of any product development effort. It is important because it also enables sensory staff to be aware of potential projects in early stages, determine that resources are available, become familiar with available consumer language, and create more synergy within the research team. However, much still remains to be accomplished if these developments are to contribute in a significant way. Tests continue to be fielded with unqualified subjects or the wrong population of consumers; repeated trials and balanced designs are ignored. There is reliance on software without an appreciation as to whether the design is appropriate; creating multivariate plots based on a few products, to name just a few. The power of today’s PCs and access to cloud computing enables sensory professionals to provide summary results as they are obtained. This is a blessing and a curse. Marketing and consumer insights professionals want information quickly as does technology should they need to reformulate a product. However, the downside is the pressure on sensory professionals to make decisions quickly, without adequate time to determine if the results make sense.

As Groopman (2009) noted, Statistical analysis is not a substitute for thinking—a statement that is essential for all sensory professionals to keep in mind when examining test results.

Despite these impediments, the field continues to grow and to attract interest. Whereas three decades ago there were very few texts on the topic, today there are many available and more in press. New journals devoted to sensory evaluation have appeared, and all are available in hard copy and/or via the Internet. Professional societies continue to grow. In addition, the number of conferences and short courses has increased dramatically. In any one year, there are at least 25 courses and as many as 6 conferences throughout the world. Perhaps most gratifying has been the increase in academic programs offering course work and degrees in sensory evaluation. This has occurred in both the United States and Europe, and more recently Chinese Universities have offered courses in sensory evaluation. In addition, the University of California, Davis has been offering a yearlong web-based distance learning class since 2001 attended by more than 700 students from over 30 countries around the world. These programs are welcome because they will eventually lead to a more scientific approach to the testing process. In this edition, we have reviewed the organizational issues and where necessary we have made changes that we believe will help maintain programs despite the many changes taking place in the consumer products industries. Consideration has also been given to the reviews of earlier editions and comments provided by numerous individuals who have written and/or talked with us about what is missing in the book.

As in previous editions, we provide detailed information about test methods, in part because there continues to be confusion about these methods. We continue to encounter individuals describing a method that bears little resemblance to the method of the same name as described in the literature. Failure to understand the methods and how they were developed leads to their misuse, and recommended practices no longer appear to be practiced as rigorously as in the past. One important example is the use of analytical methods (i.e., discrimination and descriptive) without replication, which is akin to a chemical analysis done only once. Some have advocated the use of large numbers of naïve subjects to avoid qualifying subjects or to compensate for subject variability or worse, to collect the information and then remove those files that are least consistent. Such ideas reflect a lack of knowledge about statistics, human behavior, and the scientific process itself. Increasing the number of subjects does not decrease error but does increase the likelihood of achieving statistical but not practical significance.

The statistical power of a test is best achieved when one uses qualified subjects, a repeated trials design, and good laboratory practices.

Finally, the search continues for the universal scale, the invariant reference standard, and the invariant subject despite evidence that such a search will not be successful.

As stated in the first edition, and restated here, this book is not a review of the literature. We do, however, discuss literature relevant to specific issues and cite what we consider to be pertinent to the applications of sensory resources to provide actionable information. In this edition, we continue to emphasize the importance of planning, and the importance of learning how to manage and communicate product sensory information in a way that is actionable, that is understood by others.

We acknowledge the recent loss of our longtime friend, associate, and mentor, Prof. Howard Schutz of the University of California, Davis. He was an early pioneer in the field and always provided us with great, creative ideas and interesting commentary about human behavior. His presence will be missed.

We also acknowledge our friend, associate, and mentor, the late Prof. Rose Marie Pangborn of the University of California, Davis, who worked tirelessly to educate students, encouraged them to pursue a career in sensory evaluation, and worked with a total commitment for the betterment of the science of sensory evaluation.

Chapter 1

Introduction to sensory evaluation

Abstract

Developments continue to be made in sensory science methodology and their application in the product evaluation process. In today's competitive global business environment, there is an increased need for actionable product sensory information to supplement what is known about consumer behavior. Brand managers and marketing researchers/consumer insights need ways to increase their knowledge about consumer responses to products before and after purchase. Of major interest is identifying variables beyond those that are typically measured (preference) to gain a competitive advantage. Sensory information is becoming an integral part of a product's business strategy. Identifying specific sensory and physical/chemical measures that have a significant effect on preference, consumer behavior, and purchase intent has important benefits for a company. This has only been possible with the use of quantitative descriptive analysis to identify the sensory differences and similarities, along with development of mathematical models to determine relationships between consumer behavior, sensory characteristics, formulation, and market strategy. Sensory science is a powerful approach to enhancing product preferences of consumer goods instead of using a trial and error process

Keywords

Sensory evaluation; sensory science; product testing; sensory history; physiological psychology; experimental psychology; measurement; sensory definition; consumer behavior; variability

Chapter outline

Outline

1.1 Introduction and objective 1

1.2 Historical background 7

1.3 Development of sensory evaluation 12

1.4 Defining sensory evaluation 14

1.5 A physiological and psychological perspective 17

1.1 Introduction and objective

Since publication of the fourth edition, there have been many changes in sensory evaluation. Probably most changes are the direct result of the internet. It has connected sensory scientists with consumers in ways not imagined a few decades ago. It has facilitated recruiting consumers for a test; made surveys much easier to organize and obtain information; enabled researchers to more easily obtain language used by consumers as part of the product development process; enabled descriptive analysis tests to be fielded in typical use situations; enabled data collection independent of location, and analyses on a real-time basis. All these benefits appeal to sensory scientists as well as brand managers looking for ways to speed the data collection process and more quickly make product decisions. New sensory methods have been developed, also facilitated by the internet; for example, check all that apply (CATA) and other similar types of methods seeking consumer language without having to rely on more traditional methods to qualify subjects or to develop a language to describe products. Descriptions of some of these methods are described in a text by Delarue et al. (2015). More information about the methods can also be found in any journal focused on the sensory sciences. Other developments that have had an impact include the availability of low-cost hardware and software enabling sensory scientists to design and collect data and analyze tests on an as needed basis.

Not surprisingly, the marketplace also has changed (Sloan, 2018), reflecting such developments as: consumer initiatives such as using the internet to demand changes in food safety, more clean label/organic foods, venture capital investments in new products and new technologies (Buss, 2018; Davis, 2018), the government’s implementation of new regulations on food safety and food traceability, and the industry’s focus on cost cutting resulting in loss of sales and market share. Consumer concerns about safety, health, ingredients such as salt, sugar, additives, genetically modified organism, and gluten free, and more recently ultra-processing add to an already difficult market environment for traditional food companies. Probably the biggest impact has been the role of venture capital in support of new and alternative food products such as laboratory produced meat (an interesting name at this time). This has made the product development process much more complex and more competitive (Martindale et al., 2019).

All this translates into a significant growth in demand for sensory information. Anticipating this need and having resources available represents an opportunity that cannot be ignored. This book is focused on how this can be achieved.

Company executives acknowledge the need for actionable product sensory information to supplement what they already know about consumer behavior. As previously mentioned (Stone and Sidel, 2009; Stone, 2018), brand managers and marketing researchers (also known as consumer insights professionals) look for ways to better understand the sensory information that influences consumer purchase behavior. Using this information enables technologists to focus on formulation efforts and be more productive. In situations where it has been successful, it has led to greater reliance on sensory scientists in the project planning process and respect for sensory information. Companies have also shown a willingness to build sensory infrastructure. This has led to a growing demand for newly trained sensory scientists and more course offerings at universities. Media attention about sensory evaluation also has developed—what is it and why is it important (e.g., Kuhn, 2016). In the past, the likelihood of media mention of sensory evaluation was rare; today, there are books, blogs, and other media outlets describing results from sensory tests or interviews with individuals possessing unique sensory skills. Although one may question some of these claims/curiosities, this exposure has brought more attention to the field and that is a positive development for the science.

Changes within the profession also are happening, such as more meetings and symposiums discussing research results. Other changes are more focused on the ways in which professionals design, analyze, and report test results. Examples include use of direct data entry and a concomitant reduction in use of paper ballots; a rapid turnaround from data collection to reporting results; low-cost software and hardware. However, none of this has come without some risk; for example, using a design and analysis that is not appropriate for behavioral data. The inexperienced scientist needs to understand the nature of a problem and be sure that the plan, design, and analyses will answer the questions stated in the requests. As Groopman (2009) noted as part of a discussion about the state of clinical trials but equally relevant to sensory tests, Statistical analysis is not a substitute for thinking. Just because one obtains a statistically significant result does not mean it has practical value. One can undertake a wide range of analyses with scant appreciation for whether the results will answer the questions being asked (May, 2004). Regardless of how a test is fielded, one always obtains results and sensory scientists need to decide if they make sense. If results do not make sense to the sensory scientist, how likely will they be to those requesting the test?

Identifying the specific sensory and chemical measures that have a significant effect on consumer preferences and purchase intent has significant benefits for a company and its brands. Still greater benefits are obtained when this information is connected with marketing strategy. Exploiting this information is achieved through better use of sensory information (Schutz, 1998; Stone and Sidel, 2009; Stone, 2018). Despite the evidence that such an approach works, skeptics abound especially with regard to the sensory information. For some, sensory evaluation is not considered a science. This is not so surprising since the process has no elaborate equipment and with just a computer, one can obtain information from the consumer. We are born with our senses and barring some genetic defects, we are all capable of seeing, smelling, tasting, feeling, etc. It certainly seems simple enough, so why should a technologist or a brand manager believe results from a test that are inconsistent with their expectations and their own evaluations? Further challenges develop when tests are fielded without following accepted guidelines leading to actions that subsequently prove to be incorrect. It takes a lot of effort to overcome this.

Since previous editions of this book, advances continue to be made, albeit at a slow pace, not because test procedures are inadequate but, rather, as noted previously, because the science is not readily acknowledged as such. In addition, sensory scientists may not be as effective in communicating results as they could be. Relying on numbers and statistics may not work when speaking with nonscientists. In one company, sensory evaluation will be used successfully, but in another it will be misused or the information will be ignored because it is inconsistent with expectations. Unfortunately, this latter situation has encouraged the use of other information sources or the development of alternative sensory resources in hopes of obtaining information that fits expectations without fully appreciating for having two sets of results that are not in agreement.

Throughout the years, numerous efforts have been made and continue to be made to develop a more permanent role for sensory evaluation within a company. Reviewing the technical and trade literature shows that progress in the development and use of sensory resources continues. There has been a noticeable increase, and much of the impetus continues to come from selected sectors of the economy, notably foods and beverages and their suppliers (Jackson, 2002; Lawless and Heymann, 2010; Meiselman and MacFie, 1996; Piggot, 1988; Prescott, et al., 2014). In their seminal textbook on sensory evaluation published more than 50 years ago, Amerine et al. (1965) correctly called attention to three key issues: the importance of flavor to the acceptance of foods and other products, the use of flavor-related words in advertising, and the extent to which everyday use of the senses was largely unappreciated, at that time. Perhaps a secondary benefit of today’s concerns about food safety has been the awareness by consumers of the sensory aspects of the foods they purchase. In recent surveys of what consumers expect, taste is one of the most frequently mentioned.

Current interest in sensory evaluation reflects a more basic concern than simply being able to claim the use of sound sensory practices. Early surveys indicated that there was a confusion about the use of sensory testing (Brandt and Arnold, 1977). The survey provided insight into some of the basic issues facing sensory evaluation such as confusion about methods and the extent of usage of testing. Unpublished surveys today suggest greater knowledge and use of sensory testing, as would be expected; however, it is evident that some confusion exists about methods possibly because some companies may use combinations of methods without appreciating the consequences.

This fifth edition continues to emphasize the importance of a systematic approach to the organization and the use of sensory resources. Primary emphasis is directed toward the more practical aspects of sensory evaluation and approaches to implementing a respected and credible program, but we do not neglect the fundamental, underlying issues, including but not limited to experimental design, the reliability and validity of results, statistical techniques, and related topics. From a sensory science perspective, reliability and validity are essential to developing a credible program and providing actionable recommendations that enable a company to build a strong brand consistent with its brand strategy (Aaker, 1996). From a business perspective, it is these latter issues that loom as most important because they build trust that others will place on these recommendations and the extent to which managers will act on them. Sensory scientists must communicate information clearly to ensure that superiors, peers, and subordinates understand what was done and what the results mean in terms of meeting that test’s-specific objective and how the research facilitates the product link with the business strategy.

The importance of the relationship between reliability and validity of results and the credibility assigned to sensory recommendations and, in a global sense, the credibility of a sensory program is at stake. Although it is logical to assume that a test result should speak for itself, and that it will be understood, this is not the usual situation. It takes a conscious effort to gain the confidence of a brand manager and consumer insights advisors so that recommendations will be acted on sufficiently. When it succeeds, it is reassuring, but when it does not, it raises questions as to why sensory information is not better understood and used more effectively. There is no single or simple answer. However, it is clear that the ease with which one can evaluate a product makes it easy for a product manager to disregard information that does not satisfy expectations and substitutes one’s own judgment, and that an unwillingness to appreciate the complexity of the judgmental process leads to decision problems and product failures.

However, it is the responsibility of the sensory professional to be sure that tests are organized and fielded using best practices. Replication is an integral part of every sensory analytical test, subjects are qualified and understood the task, the scorecard is relevant to the test objective, the design and serving orders have been documented and followed, and were the products sourced, stored, prepared, and coded correctly, etc.

Product evaluation is a complex multistep process in which groups of individuals respond to stimuli (a set of products) by marking a scorecard (electronically preferably) according to a specified set of instructions. There are many steps in the process where errors can occur, and not all of them are obvious. Of particular importance are the matter of whether the test objective is understood, and second, whether the requester can explain how the results will be used; that is, what questions will be answered? Failure to obtain such information is a clear sign of problems when reporting results. Once there is agreement on the objective, a test plan can be established, a design can be prepared, and actual fielding can be initiated.

As already emphasized, when planning a test, there are challenges faced by sensory scientists; some come from requesters specifying the test method to use, type of scale, output that yields a single numerical value, and so forth. Some reflect decisions made by the sensory scientists based on misinformation or a lack of familiarity with the literature. One example is using numbers on a scorecard in the mistaken belief that it will help subjects providing a response. There are two problems: early research had shown a bias toward use or avoidance of certain numbers. For example, some Asian cultures avoid the number 4 while numbers 6 and 8 have positive associations. In some Western cultures, the number 7 has positive associations. A second problem is the potential of subjects thinking they remember a number assigned previously, which affects their response to a subsequent product.

Measurement or scaling is an important part of the testing process and here too, one finds examples of misuse, reflecting a lack of familiarity with the research on how measurement systems were developed. Examples include the optimal number of categories and a second is the use of a universal scale for which there is no evidence that such a scale exists (Garner, 1960; Lawless and Heymann, 2010). In Chapter 3, Measurement, this topic is discussed in detail.

Subjects, not surprisingly, have a major impact on a program’s success. Here, too, there are differing practices as to how subjects are selected/qualified to participate in a sensory test.

Probably the single most important aspect of subject selection is recognition of individual variability in perception. There is a large body of evidence that recognizes individual variability in perception is real. It is based on genetics, health of the receptors, and motivation differences (Pangborn, 1981).

In addition, the products are variable. No two products are alike. All these sources of variability do not mean one cannot design a test or reach a conclusion; rather it means that the test plan must account for these sources of variability. For those looking for the same value from each test, there will be disappointment. It has particular appeal among those who lack an understanding of human behavior, and the perceptual process. Subjects, no matter what their level of skill or number of years of training and practice, exhibit differences in sensitivity from one another, as well as differences in variability that are unique (to the individual). Some training programs (e.g., see Spectrum analysis in Meilgaard et al., 2006) imply that this sensitivity and accompanying variability can be overcome through training and use of appropriate reference standards that represent absolute intensities. Such training, as much as 10 hours per week and often lasting 4 or more months, has considerable appeal; however, there is no evidence that such an approach has scientific merit. Lawless and Heymann (2010) stated, We are somewhat skeptical of this claim since there are no published data to support it (p. 360). It is inconsistent with our knowledge of human perception and the physiology of the senses. There are no peer-reviewed results reported in the literature, and one should not expect to obtain invariance. Theoretically, if panelists were in complete agreement one would expect the standard deviation for any specific product-attribute combination to be close to zero. However, most Spectrum studies have attributes with non-zero standard deviations indicating that the panel is not absolutely calibrated (Lawless and Heymann, 2010, p. 361). Such an approach is a form of behavior modification rather than a means of capturing responses as a function of a stimulus (whether that stimulus is a purified chemical or a consumer-ready beverage). As noted, subjects are changing, products are changing, and the experiment itself is changing as the test progresses. Replication allows one to measure responses across trials and partition the sources of variability and to determine whether the obtained differences were perceived by a sufficient number of subjects to conclude that the difference was detected.

It is risky to assume that a human can be invariant, just as it is misguided to assume that an instrument yields the same value every time it is used. In each instance, the end result is to overcome what appear to be limitations of the sensory process in the mistaken belief that they will make results more acceptable. Short of directly telling an individual what an answer should be, there will always be some variability. Nonetheless, the requestor of a test expecting that test to yield an invariant result (the same response or numerical value every time) is disappointed and concerned about this unique information source. This disappointment is also accompanied by reservations as to any conclusions and recommendations; that is, the issue of credibility arises. Alternatively, when results are not in agreement with information obtained elsewhere (and often not sensory information) and no attempt is made to understand and explain the basis for any disagreement, further erosion of program credibility will occur. The precision of weighing and volumetric devices, flow through pumps, and heating and cooling equipment varies so that the finished product will reflect this variation. To somehow expect a subject to provide the same judgment each time, a product is presented is naive, at best.

Sensory professionals must not ignore fundamental scientific truths in our data collection instrument: the human. If sensory is indeed a science, we must embrace the knowledge of genetic differences at the receptor level, perceptual differences, and behavioral differences, and provide scientifically derived quantitative data to measure and understand perception.

The success of a sensory program, and particularly its credibility, begins with having a plan and an organized effort and making sure that the testing process follows accepted procedures and practices—what method was used, who the subjects were and how they were selected, what test design was used, how the data were analyzed, including the evidence of reliability, and so forth. In a business sense, it begins with an explanation of what sensory information is and is not, how results are communicated and whether they are understood, and end with actionable recommendations being implemented to everyone’s satisfaction. Although these issues are discussed in detail in succeeding chapters, their inclusion here is to provide an initial perspective to the issue of business credibility and direct involvement of sensory evaluation in product decisions. Without an organized product evaluation effort and demonstrated reliable and valid results that are communicated in a way that is readily understood, one is returning to reliance on product experts, the "N of 1," who made product decisions by virtue of their expertness and not because there were data to support their judgments. Decisions derived in this manner are neither new nor unusual; however, they make it very difficult for individuals trying to organize and operate a credible sensory test program. As noted by Stone et al., (2012), sensory evaluation must develop a strategy for success. It must reach out to its customers and must educate them about the benefits that can be realized from using sensory information. At the same time, it must gain management support through an active program of selling its services and how the company can benefit from those services.

This book is not intended as a comprehensive introduction to the topic of sensory evaluation. Nonetheless, for some aspects of sensory evaluation, considerable detail is provided in an almost stepwise manner. Readers, however, will probably derive more from this book if they have a basic understanding of sensory evaluation, experimental design and statistics, and especially the perceptual process.

Where appropriate, background information sources are cited in this text and should be considered recommended reading. In addition to its benefit to the sensory professional, this book is intended to serve as a guide for the research and development executive seeking to have a capability in sensory evaluation and to develop a more efficient and cost-effective product development program. It should also be of interest to marketing, market research/consumer insights, technical managers, and quality control, all of whom have an interest in their company’s products and their quality as measured by consumer responses and through sales, market share, and profitability.

1.2 Historical background

Of the many sectors of the consumer products industries (food and beverage, cosmetics, personal care products, fabrics and clothing, pharmaceutical, etc.), the food and beverage sectors provided much early support for and interest in sensory evaluation. During the 1940s and through the mid-1950s, sensory evaluation received additional impetus through the U.S. Army Quartermaster Food and Container Institute, which supported research in food acceptance for the armed forces (Peryam et al., 1954). It became apparent to the military that adequate nutrition, as measured by analysis of diets or preparation of elaborate menus, did not guarantee food acceptance by military personnel. The importance of flavor and the degree of acceptability for a particular product were acknowledged. Resources were allocated to studies of the problem of identifying what foods were more or less preferred as well as the more basic issue of the measurement of food acceptance. These problems and their solutions were overlooked or forgotten during the 1960s and early 1970s when the federal government initiated its War on Hunger and Food from the Sea programs. The government’s desire to feed the starving and malnourished met with frustration when product after product was rejected by the recipients primarily because no one bothered to determine whether the sensory properties of these products were acceptable to the targeted groups. This is not to suggest that each country’s ethnic and regional food habits and taboos were not important but, rather, in the context of these government programs, there was scant attention given to the sensory evaluation of the products as they were being developed. Unfortunately, this situation continues to exist because there remains a fundamental lack of appreciation for the importance of the sensory properties of consumer-packaged goods (CPG) being developed. There is no question that providing nutrients to those in need is the first priority, and in extreme situations that is about all there is. However, large giveaway programs often yield less than satisfactory results that are well below everyone’s expectations, in part because the sensory properties fail to come close to the expectations of a culture or intended consumer group. In today’s connected world, the idea of using Artificial Intelligence (AI) and big data are looked on as a way of overcoming past failures; however, it is too early to know how successful these approaches and knowledge sources will be.

In the 1950s, the food and beverage industry, possibly taking a cue from the government’s successes and failures in sensory evaluation (Peryam et al., 1954), provided support for this emerging science. Although many industries have since recognized its value in formulating and evaluating products, general appreciation for sensory evaluation as a distinct function within a company remained relatively minimal until approximately the past three decades. In general, there is an agreement on the role of sensory evaluation in industry but not necessarily how sensory evaluation should be organized and how it should operate within a company. As with any discipline, divergent opinions and philosophies on sensory evaluation exist both within and outside the field. It is not necessary that we examine all these opinions and philosophies in detail; however, some discussion is appropriate to enable the reader to gain a greater appreciation for the challenges involved in the organization and operation of a sensory program.

The food and beverage industry traditionally viewed sensory evaluation in a technical context, like the company expert (the N of 1) who through years of accumulated experience was able to describe company products and set standards of quality by which raw materials would be purchased and each product manufactured and marketed. Examples of such experts include the perfumer, flavorist, brewmaster, winemaker, and coffee and tea tasters. In the food industry, experts provided the basis for developing the cutting sessions and canning bees (Hinreiner, 1956). In the canning industry, products usually were evaluated on a daily basis and in comparison with the previous day’s production, competitive products, new products, etc. In addition, there were industrywide cutting bees to assess general or category product quality. These sessions enabled managers and experts to assess product quality at their own plants as well as to maintain a familiarity with all other companies’ products. This process continues today in most companies as well as in trade associations seeking to solve common problems that are usually related to product quality. In recognizing the purpose of the cutting bee and its overall function, Hinreiner described the efforts undertaken to improve the quality of the information derived from one group, the olive industry. The Processed Products Branch of the Fruit and Vegetable Division of the U.S. Department of Agriculture has updated its guidelines for approved illumination for cutting rooms, an action that recognizes the importance of providing a more standardized environment for product evaluations. In addition to the evaluation, record-keeping was formalized, making it possible to compare results from one year with those of another and thus provide for a great degree of continuity. It is important to remember that the industry recognized a problem, and with assistance from sensory scientists, it took some action to improve its product quality information. This activity continues today, especially in industries that rely on basic agricultural products that do not experience substantial value-added processing—for example, the wine industry, the dairy industry, fresh fruits and vegetables, and olive oil. The enormous attention given to the Mediterranean diet and emphasis on olive oil has resulted in a major increase in consumption of olive oil, specifically extra virgin olive oil. This in turn has resulted in renewed focus on how olive oil quality is determined, along with conferences about olive oil, its nutritional benefits, and the criteria by which quality is determined. As such, extra virgin olive oil is an interesting case study to illustrate new challenges with combining expert evaluations, U.S. Department of Agriculture (USDA) grading, and sensory sciences. This topic is discussed in more detail later in this book.

Cutting sessions and the work of product experts have continued; however, the impact of both has been reduced but not entirely eliminated. In retrospect, the results accomplished (and in many cases failed to accomplish) by these experts and the cutting sessions were quite remarkable. By and large, experts determined which alternatives—from among many alternative ingredients, products, and so forth—were appropriate for sale to the consumer. Their success reinforced their role for establishing quality standards for particular products, such as canned fruits and vegetables, and these standards in turn received additional support through official USDA standards that referenced these results. As long as the food industry was involved solely in preserving a basic agricultural crop (e.g., frozen peas, canned fruits and vegetables, juice, or oil such as extra virgin olive oil), it was relatively easy (or uncomplicated) for the product expert to understand a particular product category and to make reasonable sound recommendations.

In the early stages of the growth of the food-processing industry and where competition was primarily regional in character, such standards and evaluative procedures by experts were extremely useful. In most industries, experts also created an array of scorecards and unique terminologies to serve as a basis for maintaining records and presenting a more scientific process. Subsequently, numerical values were assigned to various attributes being measured, as described by Hinreiner (1956). These scores soon became targets or standards; for example, the 100-point butter scorecard, the 100-point beer scorecard, the 10-point oil quality scale, and the 20-point wine scorecard; all had specific numbers that connoted the levels of product acceptance (equated with quality). All of these and others continue to be used within their respective industries. Certain values became fixed in people’s minds, and they were transposed inappropriately into measures of consumer acceptance, creating a multitude of new problems. That some of these scorecards have survived virtually intact after 50+ years is remarkable, considering their weaknesses. Where they have not survived, one can usually find the concept still alive, particularly that of the single number being equated with quality and the associated belief of the invariance of the expert. Although it is more common in quality control, the reemergence of experts in sensory evaluation is somewhat surprising but understandable. Reliance on one or two experts reflects a desire for simple numbers to make any decision easy to reach and to avoid statistics, which complicates this process especially for those unfamiliar with statistical thinking. Sensory scientists should be careful about taking a position on this process without determining the background and also considering the alternative. In one sense, the activities associated with product grading, merit awards in product competition, obtaining the right to label a product as coming from a particular growing region, and so on are linked with sensory evaluation but not in the mainstream of how sensory professionals define the work that they do. These systems provide a basis for assuring the buyer that the particular product meets the previously established criteria. They also have value when buying and selling goods; however, they do not reflect the marketplace, which is dynamic, whereas grading systems are static. The problem arises when the leap is made to assume that highest grade equates with highest quality and with product best liked by the consumer. There is more to product success than simply a designated emblem of best quality.

With the growth of the economy and competition and the evolution of processed and formulated foods, experts faced increasing difficulty in maintaining an awareness of all developments concerning their own product interests. As a further complication, product lines expanded to the extent that it was virtually impossible for an expert to have detailed knowledge about all products, let alone the impact of different technologies. While the expert was required to continue making finite decisions about product quality, consumer attitudes were changing in ways that were not fully appreciated. With the development of contemporary measurement techniques and their application to sensory evaluation, it became evident that reliance on a few experts was questionable. To deal with this problem, some companies turned to sensory evaluation (which was usually referred to as organoleptic analysis in the early literature). In truth, companies did not turn directly to sensory evaluation as a solution to the inadequacies of experts; rather, the marketplace created opportunities. As competition increased and became more national (and eventually international) in scope, the need for more extensive product information became evident. Managers were disappointed with results from some types of consumer tests and/or costs became increasingly difficult to justify to management, and now they were more willing to consider alternative sources of product information. For those companies in which there were sensory resources, opportunities developed, and in some instances considerable success was achieved. To that extent, sensory evaluation represented a new and as yet untried resource. Before discussing this contemporary view, it is necessary to further explore the earlier developments of sensory evaluation. This topic is discussed in more detail in Chapter 8, Strategic Applications.

As noted previously, sensory evaluation was of considerable interest in the late 1940s and into the 1950s, prompted in part by the government’s effort to provide more acceptable food for the military (Peryam et al., 1954), as well as by developments in the private sector. For example, the Arthur D. Little Company introduced the Flavor Profile Method (Caul, 1957), a qualitative form of descriptive analysis that minimized dependence on the technical expert. Although the concept of a technical expert was and continues to be of concern, the Flavor Profile procedure replaced the individual with a group of approximately six experts (whom the technical expert trained) responsible for yielding a consensus decision. This approach provoked controversy among experimental psychologists who were concerned with the concept of a group decision and the potential influence of an individual (in the group) on this consensus decision (Jones, 1958). Nonetheless, at that time and continuing to the present, the method provided a focal point for sensory evaluation, creating new interest in the discipline, which stimulated more research and development into all aspects of the sensory process. This topic is covered in more detail in the discussion on descriptive methods in Chapter 6, Descriptive Analysis.

By the late 1950s, the University of California at Davis was offering a series of courses on sensory evaluation, providing one of the few academic sources for training of sensory evaluation scientists. Soon after other universities, including Oregon, Rutgers, Cornell, Illinois, Kansas, and Louisiana, to name a few, offered programs and degrees in sensory evaluation. This also is reflected in the volume of sensory research published. The literature, even before formal training in the sensory sciences, has experienced growth; for example, see publications by Boggs and Hansen (1949), Giradot et al. (1952), Baker et al. (1954), Harper (1950), Pangborn (1964), and Stone and Pangborn (1968) and related ASTM publications on sensory evaluation. These studies stimulated and facilitated the use of sensory evaluation in the industrial environment.

The early research was especially thorough in its development and evaluation of specific test methods. Discrimination test procedures were evaluated by Boggs and Hansen (1949), Giradot et al. (1952), and Peryam et al. (1954).

In addition to discrimination testing, other measurement techniques also were used as a means for assessing product acceptance. Although scoring procedures were used as early as the 1940s (Baten, 1946), primary emphasis was given to use of various paired procedures for assessing product differences and preferences. Rank-order procedures and hedonic scales became more common in the mid- to late 1950s. During this time period, various technical and scientific societies, such as Committee E-18 of the American Society for Testing and Materials (now ASTM International), the Food and Agriculture Section of the American Chemical Society, the European Chemoreception Organization, and the Sensory Evaluation Division (now the Sensory and Consumer Sciences Division) of the Institute of Food Technologists, organized programs about sensory evaluation. Accessing any current publications that contain research on sensory evaluation will also list numerous conferences on sensory evaluation attesting to its growing importance.

1.3 Development of sensory evaluation

It would be difficult to identify any one or two developments that were directly responsible for the emergence of sensory evaluation as a unique discipline and its acceptance (albeit, on a limited basis) in product business decisions. Certainly the international focus on food and agriculture in the mid-1960s and into the 1970s (and that continues today), the energy crisis, food fabrication, the cost of raw materials (Stone, 1972), competition, product development from alternative sources, globalization of the marketplace has, directly or indirectly, created opportunities for sensory evaluation. For example, the search for substitute sweeteners stimulated new interest in the measurement of perceived sweetness along with an interest in time-intensity measures. This, in turn, stimulated development of new measurement techniques (Inglett, 1974) and indirectly stimulated interest in development and use of direct data entry systems as a means for evaluating the sweetness intensity of various ingredients (for more information about the latter topic, see Anonymous, 1984; Gordin, 1987; Guinard et al., 1985; Winn, 1988). Today, this situation has changed dramatically with respect to capturing responses and real-time data analysis; other opportunities remain to be satisfied.

Whether companies are developing new products, attempting to enter new markets, or attempting to compete more effectively in existing markets, the need for sensory information remains (Stone, 2018).

Although much more could be written on and speculated about these opportunities and their antecedents, it is more important that our attention be focused on how this sensory resource should be structured so that it can function more effectively today and in the future.

After a long and somewhat difficult gestation, sensory evaluation has emerged as a distinct, recognized scientific specialty (Sidel et al., 1975, see also Sidel and Stone, 2006; Stone and Sidel, 1995). Although the focus of the former article was on the use of sensory evaluation in the development of fabricated foods, there were implications for sensory evaluation in general. As a unique source of product information, it had important marketing consequences, providing direct, actionable information quickly and at low cost. It was proposed that organizing of sensory evaluation test services along well-defined lines (e.g., formal test requests, selection of a test method based on an objective, and selection of subjects based on sensory skill) would increase the likelihood of such services being accepted as an integral part of the product information resources useful to other business units within a company. It has become clearer that without an organized approach, a management-approved plan, and an operational strategy, sensory resources are rarely used effectively and are less likely to have a significant, long-term impact.

In a short course given several decades ago, Pangborn (1979) called attention to misadventures that have occurred in sensory evaluation. The article was one of several by this author as part of her continuing efforts to improve the quality of the research being done and indirectly enhance the skill level of sensory professionals in companies. The three issues of particular concern were the lack of test objective, adherence to a test method regardless of application, and improper subject selection procedures. These three issues remain in today’s business environment. These are not the sole property of the sensory literature (and by default many in teaching roles) but also are quite commonplace in the business environment. It is clear that much more needs to be done to improve the quality of sensory information.

Short courses and workshops continue to be offered. While the emphasis has shifted away from the basic principles, it reflects the needs of new professionals in working in companies that have their own ways of testing. Universities also have developed distance learning certificate programs in sensory and consumer sciences. Our own experience in offering courses during the past five decades reflects a continued interest in sensory evaluation, especially the more practical issues of developing a program within a business environment. Some of the material presented in this book evolved from workshop material that has proven especially beneficial to participants. Newspapers and other public information sources present articles about sensory evaluation (not regularly, but often enough to be noticeable). These articles usually include some impressive revelations (to attract the reader) about the special tongue or nose of certain individuals who are claimed to have almost mystical powers. These individuals are generally associated with products such as wine, beer, coffee, and fragrance, or they function as wine and food writers. Still other approaches convey an impression that the subconscious mind is being tapped by a new sensory technique, with the end result being the ideal consumer product. A recent example is the development of methodology described as emotional profiling about which more can be found in Chapter 6, Descriptive Analysis. Emotional profiling is, in effect, using the descriptive process to identify consumer language with which consumers associate with their feelings about a brand (Aaker et al., 2010). It is a potential tool for sensory programs provided it is recognized by marketing and advertising functions within a company. After all, it is very easy to obtain a response from a consumer/subject; it does not mean that the question was understood or that the responses will yield information that is understood by the requester. These developments, and a greater awareness of sensory evaluation, appear to have coincided with a dramatic shift by the CPG industry toward a more consumer-oriented environment and away from the more traditional manufacturing/production-oriented environment. By that we mean a recognition that understanding consumer attitudes and behavior is an essential information source and ought to be known before one formulates a product rather than manufacturing a product and looking to others (e.g., marketing) to convince the consumer to purchase that product.

Opportunities for sensory evaluation continue to develop primarily as a result of significant changes in the marketplace and to a much greater extent than changes in sensory evaluation methodology. Mergers, leveraged buyouts, and other financial restructuring activities and the internationalization of the marketplace have created even greater challenges in the consumer products industry. There are numerous choices in terms of brands, flavor alternatives, convenience, pricing, new products, and combinations not thought of a decade ago (e.g., yogurt beverages, plant-based nonmeat alternatives). Many companies have determined that new products at competitive prices are essential for long-term growth and success. However, this has presented its own unique challenges and risks (Meyer, 1984; Buss, 2018; Martindale et al., 2019; Davis, 2018). New product development and the proliferation of choices within a product category accelerated in the 1980s at a rate neither appreciated nor believed possible in the past. This acceleration was accompanied by considerable financial risk (Anonymous, 1989; Martindale et al., 2019). In separate 1977 publications on the topic, Crawford and Carlson determined that the failure rate of new products has, at times, been as high as 98% for all new products. Today, one can find similar reports; clearly, this situation has not changed very much and certainly not for the better. From a business perspective, this risk challenges creative skills and available technical resources and has provided renewed interest in resources such as sensory evaluation (Stone, 2018). While companies today are now more receptive to new approaches and to new ways of anticipating and measuring the potential for a product’s success in the marketplace, there is a lack of appreciation for sensory evaluation because many of the new developments come from nonfood entrepreneurs. Also, there is a sense that electronic-derived information: for example, the e-nose and the e-tongue combined with AI will provide the answers. At a recent meeting attended by one of the authors (HS) heard a company executive extoll their success is being achieved through chef endorsement. This may yet prove correct but when surveys time and again cite taste as most important to their purchase, the need for sensory information remains. Companies are now more aware of sensory evaluation; however, the organization and operation of sensory business units with full management support still lag other related activities such as consumer insights (a successor to marketing research). Nonetheless, the fact that some programs are fully operational bodes well for the future.

Although much progress has been made, considerably more remains to be achieved, particularly within the business environment. In the next chapter, organizational issues are more fully explored, with particular emphasis on structural issues and their integration with the other product information resources—that is, how methods and subjects are developed and used to solve specific problems and maximize sensory science’s benefits to a company.

1.4 Defining sensory evaluation

As a first step, it is helpful if we consider what does sensory evaluation mean, how does one explain what sensory scientists do? In 1975 the Sensory Evaluation Division of the Institute of Food Technologists (Anonymous, 1975), developed the following definition:

Sensory evaluation is a scientific discipline used to evoke, measure, analyze, and interpret reactions to those characteristics of foods and materials as they are perceived by the senses of sight, smell, taste, touch, and hearing.

This definition represented a conscious effort to be as inclusive as is possible within the framework of food evaluation, with the word food considered global; that is, an ingredient is a food, a beverage is a food, and so forth. Similarly, materials can be products for the home such as furniture polish, a product for personal care such as a shampoo, a hair colorant, or a lipstick, etc. It can be argued that this definition is too narrow or the converse, it is too broad, based on one’s perspective. The main purposes for the definition were to make clear that sensory evaluation encompasses all the senses and is not limited to food. The concern about all the senses is particularly important because request for test services often assume that established methods are not appropriate or that certain modalities should be ignored. For example, instruct the subjects to ignore product appearance and just measure taste as if the brain works in some kind of compartmentalized way. Although the subjects may agree to this plan, it is unlikely that their responses will reflect this instruction. If there is no place on the scorecard to record responses to these other attributes, they are likely to be embedded in the flavor response. This will lead to a confounding of the response and potential misinterpretation of the results. A product’s appearance will impact an individual’s response to that product’s taste, etc. Regardless of what one may like to believe or has been told, responses to a product are the result of interactions of various sensory messages, independent of the source. To avoid obtaining incomplete product information, it is important to design studies that take this knowledge into account. The familiar request to field a test but tell the subjects to ignore the color as that will be corrected later is a sure sign of future problems. This issue will be discussed in a subsequent chapter but is mentioned here to emphasize its importance in the overall evaluation strategy and the seeming lack of appreciation of its consequences for both the sensory staff and the requestor. When the sensory professional reports results to managers who will not ignore the color, what kind of response will be offered? The definition also reminds everyone that sensory evaluation is derived from several different disciplines but emphasizes the behavioral basis of perception. This involvement of different disciplines may help to explain the difficulty entailed in delineating the functions of sensory resources within a business environment. These disciplines include experimental, social, behavioral, and physiological psychology, statistics, home economics, marketing research, and a working knowledge of product technology.

As the definition implies, sensory evaluation involves measuring all of a products’ properties because there will be interactions that may be more important than just responses to a few characteristics. It