Food Research International 64 (2014) 289–297

Contents lists available at ScienceDirect

Food Research International

journal homepage: www.elsevier.com/locate/foodres

Balanced incomplete block design: an alternative for data collection in

the optimized descriptive profile
Rita de Cássia dos Santos Navarro da Silva a,⁎, Valéria Paula Rodrigues Minim a, Alexandre Navarro da Silva b,
Andréa Alves Simiqueli a, Suzana Maria Della Lucia c, Luis Antonio Minim a
a
Departamento de Tecnologia de Alimentos, Universidade Federal de Viçosa (UFV), 36570-900, Viçosa, Minas Gerais, Brazil
b
Departamento de Engenharia de Produção e Mecânica, Universidade Federal de Viçosa (UFV), 36570-900, Viçosa, Minas Gerais, Brazil
c
Departamento de Engenharia de Alimentos, Universidade Federal do Espírito Santo (UFES), 29500-000, Alegre, Espírito Santo, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: The optimized descriptive profile (ODP) is a new descriptive methodology which proposes an optimized sample
Received 9 April 2014 evaluation by no trained assessors. To permit that assessors with a low degree of training quantitatively evaluate
Accepted 28 June 2014 samples consistently, this method proposes the presentation of reference materials ("weak" and "strong")
Available online 6 July 2014
together with all set of samples at the time of evaluation. The attribute-by-attribute protocol is recommended
on the ODP technique to allow for comparison among the samples and avoid fatigue, since only one attribute
Keywords:
No trained assessors
is evaluated at a time. Simultaneous presentation of the samples, together with the reference materials, is the
New techniques "key point" of the ODP, and therefore the number of samples may be a limiting factor in this technique. The
Quantitative data purpose of the present study is to determine if the balanced incomplete block design (BIBD), used in descriptive
Attribute-by-attribute protocol techniques traditionally, may be an alternative for data collection on this specific methodology, the optimized
Design for sample presentation descriptive profile (ODP). The BIB design does not allow for comparison among all samples together, but only
among a fraction of them. Therefore an assessment of the data quality obtained through this design is necessary
to increase the applicability of the ODP. In this experiment a specific set of grape juice formulations was used,
which presented variations in pulp and sugar contents. Sensory evaluation was conducted with two independent
panels, where each panel evaluated the samples using the ODP protocol in a specific design. Thus, one panel
evaluated the samples using the balanced complete blocks design (BCBD) and the other panel the balanced
incomplete blocks design (BIBD). The data obtained by the two designs were compared in relation to analysis
of variation, F-test power and regression models. Data obtained by the ODP conducted in the BIBD was very
similar to that of the ODP in the BCBD. In the ANOVA, the formulations differed (p b 0.05) in relation to all attri-
butes, showing a test power (1 - β) greater than 0.95. The regression models obtained in the different designs
indicated overlapping of the confidence intervals (CI 95%). Therefore, the BIBD presents itself as an alternative
for data collection in the ODP without loss in quality of the results.
© 2014 Elsevier Ltd. All rights reserved.

1. Introduction
The optimized descriptive profile (ODP) is a descriptive methodolo-
gy that was recently proposed (Silva et al., 2012) in order to reduce the
time of sensory testing, and at the same time provide quantitative
information on the sensory attributes in foods. The obtaining of quanti-
tative data highlights the ODP among the fast methodologies published
recently, such as: flash profile (Dairou & Sieffermann, 2002; Delaure &
Sieffermann, 2004), napping (Pagès, 2005), free sorting task (Cartier
et al., 2006), check-all-that-apply (Adams, Williams, Lancaster, &
⁎ Corresponding author.
E-mail addresses: rita.navarro@ufv.br (R.C.S.N. Silva), vprm@ufv.br (V.P.R. Minim),
alexandre.navarro@ufv.br (A.N. Silva), andrea.simiqueli@ufv.br (A.A. Simiqueli),
smdlucia@yahoo.com.br (S.M. Della Lucia), lminim@ufv.br (L.A. Minim).
Foley, 2007; Ares, Giménez, Barreiro, & Gámbaro, 2010), pivot profile
(Thuillier, 2007), ultra-flash (Perrin et al., 2008), ranking descriptive
analysis (Richter, Almeida, Prudencio, & Benassi, 2010) and their
respective variations.
In ODP technique a panel of no trained assessors evaluated the
intensity of sensory stimuli on an unstructured scale, anchored at the
extremities by the terms “weak” and “strong” which are represented
by quantitative reference materials. Product evaluation by the panel
was performed via the attribute-by-attribute protocol, which was
recommended by Ishii, Chang, and O’Mahony (2007) for untrained
assessors.
Attribute-by-attribute protocol recommends that the assessors
receive all samples at once. In this protocol, they are asked to the asses-
sors to compare the samples with each other, in relation to a single
attribute, and indicate the stimulus intensity on an unstructured scale.
In technique, re-tasting of the products and re-evaluation of the score

http://dx.doi.org/10.1016/j.foodres.2014.06.042
0963-9969/© 2014 Elsevier Ltd. All rights reserved.
290 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297
may be performed until the judge is sure that the scores represent the
correct “space” between intensities (Ishii, Stampanoni, & O’Mahony,
2008).
In the OPD, the attribute-by-attribute protocol is combined with the
presence of reference materials at the time of product evaluation. The
quantitative references are presented together with the samples to
allow assessors to compare the samples with the references, which
facilitates allocation of the attribute intensities on the unstructured
scale (Silva et al., 2012). This structuring of the test (all samples submit-
ted together along with the presence of reference materials) is the “key
point” of the ODP, because it allows for assessors with a low degree of
training to consistently assess the intensity of the products, as verified
in the studies of Silva et al. (2012, 2013a).
The evaluation of a single attribute per session is recommended in
the ODP in order to avoid the effect of sensory fatigue, since the judge
must test all samples and reference materials (“weak” and “strong”)
before the allocation on the stimulus intensity scale. Re-tasting may be
necessary in some cases, possibly enhancing the effect of sensory fatigue
in the case of a large number of samples. Silva et al. (2012) reported that
the number of samples is a possible limitation of the ODP methodology,
since the simultaneous presentation of a high number of samples and
the references may cause sensory fatigue and not represent the actual
intensity of sensory stimulus.
Therefore, the objective of the present study is to assess whether
the balanced incomplete block design may be an alternative design to
evaluate a larger number of samples in this specific methodology, the
optimized descriptive profile. To this end, this research evaluated the
same number of samples using two designs (complete and incomplete)
to verify agreement between the data.
2. Designs used for presentation of samples in conventional
descriptive sensory analysis
2.1. Balanced complete block design (BCBD)
The balanced complete block design is the most widely used design
in conventional descriptive tests and is recommended when the
number of samples is small and sensory fatigue is not a concern
(Meilgaard, Civille, & Carr, 2006). In this design, the assessors are the
“blocks”, which assess all samples in r repetitions. Thus, the number of
assessors corresponds to the number of blocks (b) and the judging effect
is partitioned from the total variance in the analysis of variance
(ANOVA). Samples are presented to the assessors at random, and
evaluation repetitions by the assessors (intra-block repetitions) allow
for estimation of the interaction effect between the assessors and
samples, adopting the model described in Eq. (1) (Stone & Sidel, 2012).
Model : Samples þ Assessors þ Interaction ðsamples x assessorsÞ
þ Error ð1Þ
The methods of presenting the samples to the assessors in the BCBD
may follow two different protocols, which are: (i) simultaneous
protocol: all samples are presented simultaneously to the assessors in
the same session, making it possible to compare them in relation to
the complete list of attributes; (ii) monadic protocol: samples are
presented monadically, i.e., individually. In the monadic protocol, com-
parison is not allowed, and the judge evaluates a sample in relation to all
attributes listed before another sample is presented. In both protocols,
the order of sample presentation, including the repetitions are random-
ized and balanced.
In the studies conducted by Silva et al. (2012, 2013a) with the ODP
methodology (no trained assessors), evaluations of the samples were
performed using the balanced complete block design and followed the
attribute-by-attribute protocol, which is similar to the simultaneous
protocol, however only one attribute is evaluated per session. Thus,
the number of sessions required corresponds to the number of
attributes multiplied by the number of repetitions of the evaluations.
2.2. Balanced incomplete block design (BIBD)
Balanced incomplete block designs are recommended for sample
presentation in conventional descriptive methods when the total
number of samples under study is greater than the number that can
be evaluated in the session before sensory fatigue (Best, Rayner, &
Allingham, 2011; Gacula, Singh, Bi, & Altan, 2008). In the BIBD, in a
single session the assessors evaluate only a fraction (k) of the total
number of samples (t), where k b t, which are presented simultaneously
(Stone & Sidel, 2012). In this design all samples are evaluated the same
number of times (denoted by r), and the number of times that a pair of
samples is evaluated together is the same for all pairs and is denoted by
λ. The fact that r and λ are constant for all samples in the BIBD ensures
that each treatment average is estimated with the same accuracy and
that all paired comparisons between two treatment averages are equal-
ly sensitive (Montgomery, 2001). The number of blocks required to
complete one repetition of the basic design is detonated by b, and the
number of repetitions of the basic design is detonated by p. A list of
BIB designs is presented by Cochran and Cox (1981) and is useful for
selecting a specific design for the study.
According to Meilgaard et al. (2006), two of the BIB designs may
be adopted according to the number of blocks, which are: (i) a high
number of blocks: each judge evaluates only one block (b) of samples,
and therefore pb assessor are needed to complete the basic design, (ii)
small number of blocks: each judge returns several times until complet-
ing all repetitions of the basic design, thus each judge evaluates all
sample blocks (b), and the number of repetitions (p) is equal to the
number of assessors.
The method of using the analysis of variance (ANOVA) to analyze
data from the BIBD depends on how the design is administered. Thus,
if each assessor evaluates only one block of the basic design, as in case
(i), then the assessors' effect will be confounded (“mixed”) with the
block effect. In case (ii), in which the assessors evaluate all blocks of
the basic design, the assessors' effect can be partitioned from the total
variability, where the ANOVA may be adopted as described in Eq. (2)
(Meilgaard et al., 2006; Stone & Sidel, 2012).
Model ¼ Samplesðadjusted for blocksÞ þ Assessors
þ Blocksðwithin assessorsÞ þ Interactionðsamples x assessorsÞ
þ Error: ð2Þ
The ODP methodology has not been evaluated according to the BIBD,
and therefore presents a question as to whether the technique provides
accurate and precise results when data are collected via this design. The
purpose of the ODP is to make comparisons between the samples and
the references before allocation of intensity of the stimuli on the
unstructured scale. In the BIB design it is not possible to make com-
parisons between all samples, but only among a fraction of them, and
therefore to assess the feasibility of this design for data collection in
the ODP it is necessary to increase the applicability of this technique.
3. Materials and methods
3.1. Samples
Five grape juice formulations were used for descriptive analyses.
This specific number (small) of samples was used for permitting the
comparison of the BCB and BIB designs in the ODP protocol. Therefore,
this small number of samples was used purposely to allow comparison
of the two designs using the same assessment protocol (ODP), without
causing fatigue.
 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297
Table 1
Grape juice formulations.
Samples Pulp Sucrose
Codification Proportion (pulp:water) Concentration (%) Codification
Juice-A +1 1:1 0 −1
Juice-B +1 1:1 2 +1
Juice-C −1 1:3 2 +1
Juice-D −1 1:3 0 −1
Juice-E 0 1:1.7 1 0
The formulation samples presented different levels of pulp and
sugar, which were obtained according to a factorial design with a central
point (Gacula et al., 2008), including two factors at two levels plus a
central point (22 + 1) to total five formulations (Table 1). It was used
as a product that would be easy to characterize, which could be sensory
characterized by a small number of attributes. This decision was taken
due to the number of evaluation sessions required in ODP protocol
corresponds to the number of attributes multiplied by the number of
repetitions of the evaluations. Therefore, the characterization of a
complex food matrix requires a high number of sessions, a situation
that was not studied to the ODP technique, bringing concerns if the
technique, by itself, would be capable to obtaining reproducible results.
This scenario could hamper the comparison of the design sample
presentations specifically.
The formulations were prepared with whole grape-flavored pulp of
the commercial brand Bella Ischia® and commercial refined sugar
(União®). The pulp was diluted in water at the desired proportions
and sugar added at the previously defined proportions. Juices were
maintained refrigerated and served at a temperature of 4 ± 1 °C.
3.2. Sensory procedure
Sensory characterization of the grape juices was performed by two
independent panels, and each panel evaluated the samples based on a
specific design. Thus, the both panels evaluated the samples via the
ODP technique, where one panel used the BIB design (panel ODP/
BIBD) and the other panel used the BCB design (panel ODP/BCBD).
First, sixty assessors were recruited using structured questionnaires,
as recommended by Meilgaard et al. (2006). After recruitment, these
judges were pre-selected based on their ability to discriminate two
grape juices (juice I and II). Therefore, a series of four triangular tests
were performed, and the judges who passed at least 75% of the tests
were selected (thirty-four assessors). The proportions used in preparing
the formulations were: juice I) pulp:water ratio of 1:1, adding 1% sugar;
juice II) pulp:water ratio of 1:3, with no sugar added. After pre-selection
of the participants based on discriminatory power, the attributes
describing the test juices were determined using the open discussion
technique, as described by Damásio and Costell (1991). The sensory
attributes selected by the panels were: violet color, grape aroma, sweet-
ness, sourness, grape flavor and astringent feel. For preparation of the
reference materials that anchored the extremes (“weak” and “strong”)
of the 9 cm unstructured scale, formulations containing pulp and
sugar allowed for obtaining significantly different intensities and
extremes compared to the formulations under evaluation.
After this procedure, the assessors were divided into two homoge-
neous panels for product evaluation. Thus, one group of seventeen
panelists evaluated the juice via the ODP methodology according to
the BIB design, another seventeen assessors using the ODP according
to the BCB design, and according to the recommendation of the ideal
number of assessors for ODP technique (Silva, Minim, Silva, Peternelli,
& Minim, 2014). The ODP/BCBD panel was comprised of eight women
and nine men ranging in age from twenty-one to thirty-two years old.
The other panel (ODP/BIBD) was comprised of ten women and seven
men, with ages ranging between twenty and thirty years old.
291
3.2.1. Optimized descriptive profile: evaluation by the balanced complete
block design
The seventeen pre-selected assessors of the ODP/BCBD panel
participated in a familiarization session with the descriptive terms and
their reference materials. In this session, the objective was to standard-
ize the evaluation form and clearly state which sensory stimulus
referred to each term, in addition to anchoring the ends of the unstruc-
tured scale (“weak” and “strong”). The assessors were presented with
the descriptive terms and their respective reference materials in
individual booths. The assessors were asked to read the definition of
the sensory attribute and taste the references. After this step, evaluation
of the test juices was started.
Evaluations of the grape juices were conducted according to a
balanced complete block design (BCBD), so that the assessors evaluated
all formulations in the same session. The attribute-by-attribute protocol
was used, where the samples were served simultaneously and only one
attribute was evaluated per session. The five juices were provided to the
assessors in disposable cups (30 mL) coded with three random digits.
Juices were presented in a random and balanced sequence. Three
repetitions of the evaluations were performed per judge, totaling 18 ses-
sions for evaluating products for the six attributes in three repetitions.
In the evaluation procedure, the assessor was asked to compare the
samples among themselves and with the reference materials, and thus
allocate the stimulus intensity on the intensity scale. The evaluation
form was organized by attributes and contained the 9 cm unstructured
scale (interval) associated with each formulation.
3.2.2. Optimized descriptive profile: evaluation by the balanced incomplete
blocks design
The seventeen pre-selected assessors of the ODP/BIBD panel
participated in a familiarization session with the descriptive terms and
their reference materials as described in the previous section, where
this section was followed by evaluation sessions of the samples.
Evaluations of the grape juices were conducted according to the
balanced incomplete block design (BIBD), so that the assessors evaluat-
ed only a fraction (k = 3) of the samples in each session. The design
utilized is shown in Table 2, which was proposed by Cochran and Cox
(1981), considering the following parameters:
✓ number of treatments (t): 5
✓ number of samples evaluated in each block (k): 3
✓ number of repetitions (r): 6
✓ number of times that a pair of samples is tested together (λ): 3
✓ number of blocks (b): 10
Each assessor completed evaluations of the samples in ten blocks
for each sensory attribute, evaluating only one attribute per session,
totaling sixty sessions. The blocks were assigned to the assessors in a
random order.
Samples from each block were served to the panelists according to
the attribute-by-attribute protocol, i.e., served simultaneously. The
assessors received the three samples in disposable cups (30 mL)
Table 2
Balanced incomplete block design, considering t = 5, k = 3, r = 6, λ = 3, b = 10.
Blocks Samples
A B C D E
1 x x x
2 x x x
3 x x x
4 x x x
5 x x x
6 x x x
7 x x x
8 x x x
9 x x x
10 x x x
292 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297

coded with three random digits. The juices were presented in a random
order within each block.
In the evaluation procedure, the assessor was asked to compare the
samples among themselves and with the reference materials, and thus
allocate the stimulus intensity on the intensity scale. The evaluation
form was organized by attributes and contained the 9 cm unstructured
scale (interval) associated with each formulation.
3.3. Statistical analysis
Descriptive data were analyzed by means of the analysis of variance
(ANOVA) and multiple regression analysis, which were performed
using the R software (R Development Core Team, 2005).
In the ANOVA, the models described in Eqs. (1) and (2) were consid-
ered, using the appropriate design. In the case of significant interaction,
the F-test for samples was calculated using MSinteraction as the denomi-
nator, as recommended by Stone and Sidel (2012). The significance
level (p-value) of the variation source of interest (sample), obtained
by ANOVA (probability of FSAMPLE), was observed as a criterion for
evaluating the discrimination power of the samples in the different
designs. The power analysis of the F-test was also performed to better
estimate the discrimination power of samples for each design. The
power of the test was calculated by the probability function [1–
P(type II error)], fixing the significance level (type I error) at 5%, as
described in O’Brien and Lohr (1984).
The multiple regression analysis was used to analyze the influence of
quantitative factors (pulp and sugar concentrations), independent
variables, on the dependent variables (sensory attributes of the juice).
Fitting of the regression models was performed as recommended by
Montgomery (2001), assessing the lack of model fit and the significance
of the parameters, at 5% probability. The coefficient of determination
was also obtained.
the PCA were confirmed by the cluster analysis, using the k-means
clustering procedure (Johnson & Wichern, 2007).
4. Results
4.1. Comparison of the designs (BIB and BCB)
The mean scores of the sensory attributes obtained using the both
designs showed significant difference (p b 0.05) based on the F-test
(Table 3) for only four attributes. There was a significant difference
between the mean scores obtained in the sensory characterization of
juice-A for the attributes grape aroma, sourness and astringent. Differ-
ence between the two designs for the grape flavor attribute in juice-E
was also observed. For other attributes and formulations both designs
provided similar mean scores (p N 0.05).
The ODP conducted in the balanced incomplete block design
presented similar mean scores compared to using the same methodolo-
gy conducted in the balanced complete block design. There were only
four exceptions, and the average score obtained in the incomplete
design (BIBD) were higher than the score assigned by the panel when
they evaluated the same juice through complete design (BCBD).
4.1.1. ANOVA: BIB and BCB designs
A summary of the ANOVA, considering the different designs, is
described in Table 4. In the BCB design, only four sources of variation
were partitioned from the total variation, where the “block effect” is
represented by the variation among assessors. In the BIBD there is an
extra source of variation, which estimates the variability in evaluation
Table 3
ANOVA for comparison of designs BIB and BCB.
Attributes BIB design BCB design

3.3.1. Comparison of designs Juice-A

To compare the mean scores obtained by two designs (ODP/BIBD Violet color 6.6a 6.7a
Grape aroma 5.8a 5.1b
and ODP/BCBD), a one-way ANOVA was performed considering each Sweetness 3.0a 3.2a
sensory attribute, for each juice. Sourness 6.2a 5.3b
Grape flavor 5.9a 5.4a
3.3.1.1. Comparison of ANOVA models and discrimination power. The Astringent 6.3a 5.5b
similarity between the ANOVA models was compared on a qualitative Juice-B
way. Also, compared was the magnitude of each variance partitioned Violet color 6.5a 6.6a
by the Hatley test, which evaluates the significance of the ratio between Grape aroma 6.4a 6.0a
Sweetness 6.6a 6.5a
higher and lower variance (Montgomery, 2001).
Sourness 5.2a 5.2a
Significance of F value on the ANOVA for each attribute, on two Grape flavor 6.8a 6.7a
designs, and power of test F was observed as discrimination power Astringent 5.7a 5.5a
criteria.
Juice-C
Violet color 1.8a 1.8a
3.3.1.2. Comparison among the regression models. Similarity of the Grape aroma 3.0a 2.8a
multiple regression models adjusted to the sensory description data in Sweetness 4.1a 4.1a
different designs was verified by the overlapping of confidence intervals Sourness 2.4a 1.9a
Grape flavor 3.3a 3.1a
for the parameters, with 95% confidence (Schenker & Gentleman, 2001; Astringent 2.5a 2.3a
Van Belle, 2002).
Juice-D
Violet color 3.5a 2.0a
3.3.2. Study of the sample sub-groups in the DBIB Grape aroma 1.3a 1.3a
A study was conducted to determine if there was a difference Sweetness 1.6a 1.6a
between the mean scores of juices when they were evaluated in differ- Sourness 1.6a 1.5a
ent blocks (b), i.e., the different groupings of k samples, similar to the Grape flavor 2.1a 1.9a
Astringent 2.4a 1.8a
procedure performed by Villanueva, Petenate, and Da Silva (2005).
The purpose of this evaluation was to determine if there was a Juice-E
significant change in mean scores attributed to a given sample when Violet color 3.5a 2.0a
Grape aroma 3.6a 3.5a
compared with different sample groups, verifying the contrasts
Sweetness 4.1a 4.5a
between stimuli. Sourness 4.3a 3.7a
The comparison of mean sample scores, when evaluated in different Grape flavor 4.3a 4.0b
blocks, was performed by the PCA and the Tukey's means test, at 5% Astringent 3.4a 3.6a
probability. In the PCA, each juice formulation had six average scores Average values followed by the same letter in the row are not significantly different at 5%
plotted, each referring to a repetition. The groups suggested by of probability.
 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297 293

Table 4
Summary of the ANOVA obtained for each design.

Design Source of variation Degree of freedom Mean square (MS)

Violet color Grape aroma Sweetness Sourness Grape flavor Astringent

BIB Sample (adjusted for blocks) 4 568.5455 327.7806 371.8679 292.9823 416.4389 314.5753
Assessors 16 9.7541 12.6985 10.5825 13.3764 11.7441 17.1937
Blocks (within assessors) 153 3.1057 3.2091 2.6397 3.4986 2.9241 3.5706
Interaction 64 3.2133 6.0704 4.0061 6.5063 5.8122 5.0995
Error 272 0.4811 2.0811 1.3557 2.0248 1.0475 2.2097
BCB Sample 4 299.1208 134.2522 168.5611 151.1814 191.4267 124.3403
Assessors (blocks) 16 5.3929 4.3889 5.3590 14.0418 5.6429 15.9219
Interaction 64 1.9738 4.2534 4.0730 3.9950 4.0688 3.6597
Error 170 0.7912 2.9840 1.6660 2.6401 2.0709 2.8607

of each judge due to repetitions in the different sample blocks. The
number of degrees of freedom for the sources of variation was similar
in both designs. Only the residual variation observed a higher value
for the incomplete design (DFerror = 272) compared to the complete
design (DFerror = 170). This difference between the residual degrees
of freedom occurs because on the incomplete design the panel was
required six repetitions to complete the assessment of the entire block
of samples, while three replications of evaluations were performed on
the complete design, as recommended in the literature for descriptive
methods (Stone & Sidel, 2012).
Hatley test was used to compare the magnitude of common vari-
ances between the designs (sample, assessors, interaction and error),
presented in Table 5. Variance of samples effect was similar between
designs for all sensory attributes, indicating that the formulations
were similarly discriminated in both designs. On the other hand, there
was significant difference (p b 0.05) to assessors' effect in the evaluation
of grape aroma, where in the incomplete design the estimated variance
was higher (Table 4). Regarding the interaction effect, the variability
of this effect was significantly higher in the incomplete design to
violet color and sourness. The residual variance was significantly
lower (p b 0.05) in the incomplete design for all attributes, except for
sweetness, for which variances were similar (p N 0.05).
4.1.2. Discrimination power
Discrimination power of samples on the ODP technique in both
designs was evaluated using the significance level (p-value) at which
the samples were discriminated by ANOVA (probability of Fsample).
The power analysis of F-test [1–P (type II error)] was also evaluated as
a criterion for discrimination of samples.
In both designs, significant effect was found for the interaction
between assessors and samples (p b 0.05) in all attributes. In this case,
Fsample was calculated using the MSinteraction as the denominator (Stone
& Sidel, 2012). Samples were discriminated at the same level
of significance (p b 0.0001) in both designs, considering all sensory
attributes. The power of the F test presented greater than 0.95, with
probability of type II error lower than 0.05.
There was no difference in discrimination of samples when com-
paring two different designs of sample presentation using the ODP
technique, showing no alteration on the discriminatory power of the
technique when the design was different.
4.1.3. Influence of the pulp and sugar concentrations on the sensory profile:
regression models
The pulp and sugar contents influenced the sensory characteristics
of the juices, and their effects were modeled using the linear model
(Table 6). Significance and estimates of the regression parameters
were similar in both designs (ODP/BIBD and ODP/BCBD), as shown in
Fig. 1.
The pulp concentration was significant (p b 0.05) in all sensory
attributes; whereas the sugar content only influenced the attributes
“sweetness”, “grape aroma” and “grape flavor”. All adjusted regression
models showed high determination coefficients (greater than 0.85).
The highest values of determination coefficients were observed for the
data obtained from the ODP using the BCBD, showing that the explana-
tion of data can be better using the complete design due the exposition
of all samples at the same time.
4.2. BIBD: comparison among the sub-group samples
In the PCA, Fig. 2, it was noted that evaluations of juice samples were
very similar when they were assessed in the different blocks. The group-
ings suggested by the PCA were confirmed by the cluster analysis using
the k-means clustering procedure with 1,000,000 iterations, where the
repetitions of each sample were grouped. This indicates that the average
score assigned to the samples in a first evaluation was not altered when
this same sample was presented for the second, third or more times. Re-
gardless of the number of times that the sample was presented, the in-
tensity assigned to the sample was similar in all sessions.
Although the spatial distribution of the samples in the PCA was
similar in the repetitions of evaluations, when the average scores
of the repetitions were compared by the Tukey test (Table 7), some
significant differences (p b 0.05) were detected between the repetitions
due to contrasts among sensory stimuli.
Of the thirty groups formed (blocking), in only seven groups
there was a significant difference (p N 0.05) between the repetitions
performed in the different groupings. Sensory perception for juice-A

Table 5
Comparison between estimates of variances by Hatley test (α = 0.05).

Source of variation Hatley test

Violet color Grape aroma Sweetness Sourness Grape flavor Astringent

F p-value F p-value F p-value F p-value F p-value F p-value

Sample 1.90 0.2746 2.44 0.2042 2.21 0.2312 1.94 0.2687 2.18 0.2351 2.53 0.1953
Assessors 1.81 0.1233 2.89 0.0204 1.97 0.0922 1.05 0.4619 2.08 0.0767 1.08 0.4399
Interaction 1.63 0.0267 1.43 0.0787 1.02 0.4737 1.63 0.0266 1.43 0.0782 1.39 0.0936
Error 1.64 0.0001 1.43 0.0041 1.23 0.0655 1.30 0.0259 1.98 0.0001 1.29 0.0292

Bold values indicate significance at 5% of probability.

294 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297

Table 6
Multiple regression models adjusted for each attribute on the two designs.

Sensory attributes BIB design BCB design

Violet color Y^ color ¼ 4:11 þ 2:40P Y^ color ¼ 4:18 þ 2:43P

R2 = 0.9632 R2 = 0.9831
Grape aroma Y^ aroma ¼ 4:30 þ 1:77P þ 0:35S Y^ aroma ¼ 3:99 þ 1:63P þ 0:44S
R2 = 0.9078 R2 = 0.9575
Sweetness Y^ sweet ¼ 3:74 þ 1:06P þ 1:60S Y^ sweet ¼ 3:75 þ 1:11P þ 1:51S
R2 = 0.9097 R2 = 0.9801
Sourness Y^ sour ¼ 3:89 þ 1:66P Y^ sour ¼ 3:63 þ 1:75P
R2 = 0.8140 R2 = 0.9498
Grape flavor Y^ flavor ¼ 4:36 þ 1:92P þ 0:62S Y^ flavor ¼ 4:09 þ 1:85P þ 0:70S
R2 = 0.9437 R2 = 0.9755
Astringent Y^ astringent ¼ 4:15 þ 1:73P Y^ astringent ¼ 3:77 þ 1:57P
R2 = 0.8572 R2 = 0.9431

P: pulp content; S: sugar content; R2: determination coefficient.

showed a difference (p b 0.05) in mean scores attributed in the different
sessions only in relation to “violet color”. The score assigned to the sam-
ple in block ABC differed from that assigned when it was in comparison
with samples B and D. Substitution of formulation C for sample D in the
block altered the perception of visual stimuli of juice-A. The contrast be-
tween A and D, formulations with opposing sensory profiles (Table 3),
caused a suppression in the perception of the stimulus, which was
then less intense (Lim, 2011; Silva et al., 2013b).
Intensity of juice-B was altered for the attribute sourness. When the
sample B was tested in comparison with juices-C and D, the average
perception of acidity was more intense than when the same juice was
evaluated with formulations A and D. The change from sample C, with
lower intensity of sourness, to juice-A with the highest stimulus inten-
sity, caused a decrease in sensory perception.
Juices-C and D were the formulations most affected by the blocking,
presenting difference in two attributes, which were: violet color and
sourness for juice-C, and sourness and astringent feel for juice-D. Similar
behavior was observed for juices-C and D when tested in comparison
with samples E and B. When these were compared with juice-E (more
dilute), the perception of acid taste was more intense than when tested
in combination with the more concentrated sample (juice-B).
Juice-E, to which no sugar was added, had its perception altered in
relation to the sweetness. When this juice was tested together with
the formulations with low sucrose content (Juice-A and Juice-D), the
sweetness of this juice was perceived more intensely. However, when
the same juice was evaluated in conjunction with the juices presenting
higher sucrose concentrations, the sweetness was perceived more
lightly.
5. Discussion
The optimized descriptive profile technique conducted according
to the BIB design presented a sensory profile of the grape juices very
similar to the profile obtained when the same method was conducted
in the complete design (BCBD). The average scores obtained to the
five formulations of grape juice were very similar when compared by
ANOVA. There was a significant difference (p b 0.05) only for four
attributes (three attributes for juice-A and one attribute for juice-E). In
BIB design, the mean score was higher compared to the complete design
showing that the assessors have tended to assign values closer to the
anchor when a smaller number of samples were presented to them.
On the other hand, the magnitude of the difference between the scores
obtained by the two different designs was small. The smallest significant
difference was 0.3 cm for grape flavor in the juice-E, and the biggest
significant difference was equal to 0.9 cm for the attribute sourness in
the juice-A.

Fig. 1. Representation of the confidence intervals (CI95%) of the linear model parameters for the two designs.
 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297 295

lower, showing greater accuracy in the evaluation data. In the study

Fig. 2. PCA considering the different blocks of the BIBD. Representation of the five
formulations of grape juice (A, B, C, D and E) in the six replicates of evaluation.
for the determination of ideal number of judges for the ODP, Silva
et al. (2014) found that the greatest impact factor in the number of
judges was the magnitude of the experimental error (MSerror). The
lowest magnitude of this variance observed in the incomplete design
can permit even a reduction in the required number of judges in the
ODP.
In relation to the discrimination of samples, the magnitude of the
variance related to the samples (MSsamples) were similar in both designs,
indicating that the formulations were discriminated similarly, regard-
less of the design as they were presented to the panel. Additionally,
the discrimination of samples obtained by the BIB and BCB designs
was similar when compared the decision obtained in the ANOVA, show-
ing high sensitivity in differentiation of stimuli (p b 0.0001) in both
designs. The probability of correctly detecting the difference between
the samples was also high (greater than 0.95), allowing for minimizing
the type II error to probabilities lower than 5%.
In analysis of the influence of quantitative factors (pulp and water)
on the intensity of sensory attributes by multiple regression, the same

Table 7
Tukey's test (α = 0.05) for the average scores obtained in the different blocks of the BIB
The BIBD is an alternative for data collection in the ODP technique design.

without causing alteration in the mean scores assigned to the formula- Juice-A Juice-B Juice-C Juice-D Juice-E
tions. This shows that the presence of the reference materials (“weak”
Block Mean Block Mean Block Mean Block Mean Block Mean
and “strong”) at the time of evaluating the samples provides essential
Violet color
support for the allocation of stimulus intensity on the evaluation scale.
ABC 7.3a BAC 7.0a CAB 1.3b DAE 1.4a EAB 3.1a
Silva et al. (2013a) also verified the importance of reference materials ABE 6.3ab BAE 6.3a CBD 2.7a DBC 1.9a EAD 4.0a
to represent the anchors of the unstructured scale. The authors ob- ADE 7.1ab BCD 6.1a CDE 2.0ab DCE 2.3a ECD 4.6a
served that in the CP (trained panel and lack of references at the time ABD 6.0b BAD 6.3a CAD 2.0ab DAB 1.5a EAC 4.3a
of assessment) the rating scale was not homogeneously used, and the ACD 6.7ab BCE 6.5a CAE 2.0ab DAC 1.8a EBC 4.1a
ACE 6.5ab BDE 6.3a CBE 1.3b DBE 1.4a EBD 4.1a
assessors tended to “anchor” the samples, more often using the ex-
tremes of the intensity scale. In contrast, when the samples were evalu- Grape aroma
ated by the ODP technique (presence of anchors samples), the ABC 6.3a BAC 6.1a CAB 2.7a DAE 2.6a EAB 4.4a
ABE 5.5a BAE 6.3a CBD 3.9a DBC 1.9a EAD 4.7a
unstructured scale was used more homogeneously.
ADE 6.1a BCD 6.0a CDE 2.3a DCE 2.5a ECD 5.4a
The ODP technique when administered in the incomplete design ABD 5.1a BAD 6.5a CAD 3.3a DAB 1.7a EAC 4.7a
presented smallest variability of experimental error when compared ACD 6.4a BCE 6.4a CAE 3.0a DAC 2.5a EBC 3.6a
with the same method conducted in the complete design for samples ACE 5.4a BDE 6.8a CBE 2.5a DBE 1.6a EBD 3.6a
presentation. On the other hand, in the incomplete design there was Sweetness
the largest interaction effect between sample and assessors. These ABC 2.9a BAC 6.6a CAB 3.9a DAE 1.4a EAB 3.5ab
results suggest that when the technique was conducted following the ABE 2.4a BAE 6.9a CBD 4.2a DBC 1.7a EAD 4.6a
ADE 3.8a BCD 7.0a CDE 4.6a DCE 1.1a ECD 3.9ab
BIB design the evaluation procedure has become more difficult for the
ABD 3.0a BAD 7.2a CAD 4.6a DAB 1.3a EAC 3.2ab
assessors, which were more influenced by the samples at the time of ACD 3.1a BCE 6.2a CAE 4.2a DAC 1.5a EBC 2.8b
allocation of intensities for the violet color and sourness attributes. At ACE 2.8a BDE 5.9a CBE 2.8a DBE 1.2a EBD 3.2ab
the same time, random variation was also reduced when the evalua-
Sourness
tions were conducted in the BIBD, showing that the variability between ABC 6.4a BAC 4.8ab CAB 2.7ab DAE 2.0ab EAB 2.7a
repetitions of the evaluations were lower than when the whole set of ABE 6.0a BAE 6.0ab CBD 2.7ab DBC 2.5ab EAD 4.0a
samples was presented simultaneously (BCBD). The presentation of ADE 6.3a BCD 6.3a CDE 3.3a DCE 3.5a ECD 4.0a
only a fraction of the samples may complicate the analysis for the no ABD 5.5a BAD 4.1b CAD 2.2ab DAB 2.4ab EAC 3.3a
ACD 6.2a BCE 4.8ab CAE 2.2ab DAC 2.6ab EBC 2.7a
trained assessors, but the presence of references materials (anchors)
ACE 6.5a BDE 5.0ab CBE 1.4b DBE 1.0b EBD 3.3a
can help in score attribution and orient the assessors, enabling a more
consistent assessment. The results suggest that even the assessors did Grape flavor
ABC 5.8a BAC 6.8a CAB 3.3a DAE 1.6a EAB 4.1a
not have access to all the samples to perform an effective comparison
ABE 5.5a BAE 6.4a CBD 4.0a DBC 1.8a EAD 4.2a
between them, the smaller number of samples in evaluation on the ADE 6.6a BCD 6.7a CDE 3.1a DCE 2.0a ECD 5.1a
BIB design can enable greater attention given to each unit, generating ABD 5.2a BAD 6.9a CAD 3.4a DAB 1.5a EAC 4.6a
more accurate answers (less variability between repetitions of evalua- ACD 6.3a BCE 6.8a CAE 3.0a DAC 1.9a EBC 3.8a
tions—MSerror). ACE 5.8a BDE 6.5a CBE 2.5a DBE 1.1a EBD 3.9a

In previous studies conducted with the ODP method (Silva et al., Astringent
2012, 2013a) the complete design was used (BCBD). The authors ABC 6.3a BAC 5.2a CAB 3.3a DAE 2.0ab EAB 3.3a
found that the technique permitted to obtain results very similar to ABE 6.0a BAE 6.0a CBD 3.5a DBC 2.6ab EAD 5.0a
ADE 6.6a BCD 6.3a CDE 2.6a DCE 3.8a ECD 4.2a
the profile constructed by a trained panel and also enabled a reduction ABD 6.1a BAD 5.3a CAD 2.3a DAB 2.6ab EAC 3.3a
in the effect of interaction between assessors and samples. In the ACD 6.1a BCE 5.9a CAE 2.3a DAC 2.8ab EBC 3.7a
present study, it was verified that when the samples are presented at ACE 6.9a BDE 5.7a CBE 1.6a DBE 1.3b EBD 3.2a
an incomplete design, the interaction effect can appear more acutely. Same letters in the column, to each attribute, are not significantly different at 5% of
In contrast, in the BIB design the variance of experimental error was probability.
296 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297
Table 8
Comparison between the designs in relation to the number of steps, time required and quality of data.
Criteria
Number of sessions in each steps:
Pre-selection (8 min/session)
Definition of attributes (30 min/session)
Presentation of reference materials (5 min/session)
Evaluation of samples
Total number of steps
Estimated time required for complete evaluation's session
Results of ANOVA
Number of variances partitioned
Discrimination of samples effect
Interaction effect
Error variance
Regression models
Significance of parameters
Estimative of parameters
Determination coefficients
mathematical model was obtained (p N 0.05) for the two designs
utilized.
It is important to note that in the BIBD sixty sessions were neces-
sary to complete the evaluation of the samples and a greater number
of repetitions of the evaluations to obtain all paired comparisons.
The time required to complete the evaluations may have contributed
for “calibration” of the sensory memory of the panel, constituting
“panel training”. In contrast, when comparing the mean scores
assigned to samples in the different evaluation repetitions, i.e.,
along the execution time of the experiment there was similarity be-
tween the repetitions (Fig. 2). The difference observed between rep-
etitions in the Tukey's means test was due to the contrast effect
between stimuli (Lim, 2011; Silva et al., 2013b), which occurred
when a given sample was tested in conjunction with
other(s) formulation(s) with higher stimulus intensity, where sen-
sory perception is suppressed and a lower sensory score is attribut-
ed. However, when a same formulation was tested against
formulations with more similar intensity to the sample under evalu-
ation, the stimulus was more noticeable and a higher score assigned.
In the BCBD this effect is minimized since all stimuli are presented
simultaneously.
Table 8 shows a comparison between the designs with respect to the
number of sessions and time required, besides the quality of responses
obtained by the method when conducted in both designs. It is noted
that the ODP, when conducted under the BCBD, permitted a 37% reduc-
tion in the number of sessions in relation to the BIBD, representing a 42%
reduction in the time required. The ODP conducted in the BIBD showed
longer time required to complete the evaluations, being less economical
than the same method conducted in the complete design for sample
presentation. On the other hand, when the ODP was conducted under
the BIB design, it enabled the evaluation of the same number of samples
without altering the sensory profile of the product, discrimination of the
samples, and in functional relationship between the independent and
dependent variables. Thus, the number of samples is not a limiting
factor for the ODP, once that this evaluation can be done by an incom-
plete design.
By another side, when the incomplete block design is used, the
reduction in time of the technique is smaller. Thus, the greater the
number of samples to be evaluated, the greater the number of blocks
required to make comparisons between the treatments and, therefore,
more time required to complete the evaluation. On the other hand,
training of the panel for a large number of samples should also be
more intense and the number of sessions for evaluation of all samples
and repetitions should also increase, since it is not recommended
to evaluate more than six samples per session in the conventional
technique (Stone & Sidel, 2012).
Design
BIBD BCBD
4 4
2 2
1 1
60 18
67 25
~5 minutes ~7 minutes
5 4
Equals Equals
Higher Lower
Lower Higher
Equals Equals
Equals Equals
Lower Higher
6. Conclusion
The presence of all set of samples simultaneously on the evaluation
session is a requisite to optimized descriptive profile technique. This is
necessary so that the no trained assessors can evaluate the samples
consistently and present quantitative responses. However, the presen-
tation of all set of samples at the same time and the presence of
reference materials ("weak" and "strong") brought a question regarding
the evaluation of a larger number of samples. The number of samples
could be a limiting factor in the application of ODP? This answer is no!
This study found that the evaluation of the same number of samples
conducted under the conventional design (BCBD) and the incomplete
design (BIBD) for presentation of samples has provided similar results,
showed a high discrimination power of the samples and allowed for
obtaining the functional relationship between the independent and
dependent variables on both designs. The balanced incomplete block
design was tested in this study to evaluate juices using the ODP and
presented very similar results when the same technique was adminis-
tered in the complete design. The use of this experimental condition
(ODP/BIBD) in other food matrices such as more complex products
(greater number of attributes descriptors) can be necessary.
The BIBD is presented as an alternative for the evaluation of a larger
number of samples, showing that the number of samples is not a limit-
ing factor for the ODP. For this number of samples there was no loss of
information in the BIB compared to the BCB design. Thus, a larger
number of samples may be extrapolated to the BIBD on ODP protocol.
Acknowledgment
The authors would like to acknowledge the CNPq (Centro Nacional
de Desenvolvimento Científico e Tecnológico) and Fapemig (Fundação
de Amparo à Pesquisa de Minas Gerais) for their financial support.
References
Adams, J., Williams, A., Lancaster, B., & Foley, M. (2007). Advantages and uses of check-all-that-
apply response compared to tradicional scaling of attibutes for salty snacks. 7th Pangborn
Sensory Science Symposium. Minneapolis, USA, 12-16, August, 2007.
Ares, G., Giménez, A., Barreiro, C., & Gámbaro, A. (2010). Use of a open-ended question to
identify drivers of liking of milk desserts: Comparison with preference mapping
techniques. Food Quality and Preference, 21, 286–294.
Best, D. J., Rayner, J. C. W., & Allingham, D. (2011). A statistical test for ranking data from
partially balanced incomplete block designs. Journal of Sensory Studies, 26(1), 81–84.
Cartier, R., Ritz, A., Lecomte, A., Poblete, F., Krystlik, J., Belin, E., et al. (2006). Sorting
procedure as an alternative to quantitative descriptive analysis to obtain a product
sensory map. Food Quality and Preference, 17, 562–571.
Cochran, W., & Cox, G. M. (1981). Diseños experimentales (7th ed.). México: Editorial
Trillas.
 R.C.S.N. Silva et al. / Food Research International 64 (2014) 289–297
Dairou, V., & Sieffermann, J. M. (2002). A comparison of 14 jams characterized by convention-
al profile and a quick original method, the flash profile. Journal of Food Science, 67,
826–834.
Damásio, M. H., & Costell, E. (1991). Análisis sensorial descriptivo: Generación de
descriptores y selección de catadores. Revista Agroquimica de Technologia de
Alimentos, 31, 165–178.
Delaure, J., & Sieffermann, J. M. (2004). Sensory mapping using flash profile. Comparison
with a conventional descriptive method for the evaluation of the flavour of fruit dairy
products. Food Quality and Preference, 15, 383–392.
Gacula, M. C., Jr., Singh, J., Bi, J., & Altan, S. (2008). Statistical methods in food and consumer
research (2nd ed.). Academic Press.
Ishii, R., Chang, H., & O’Mahony, M. (2007). A comparison of serial monadic and attribute-
by-attribute protocols for simple descriptive analysis with untrained judges. Food
Quality and Preference, 18, 440–449.
Ishii, R., Stampanoni, C., & O’Mahony, M. (2008). A comparison of serial monadic and
attribute-by-attribute descriptive analysis protocols for trained judges. Food Quality
and Preference, 19, 277–285.
Johnson, R. A., & Wichern, D. W. (2007). Applied multivariate statistical analysis (6th ed.).
London: Pearson Education. Inc, 773.
Lim, J. (2011). Hedonic scaling: A review of methods and theory. Food Quality and
Preference, 22, 733–747.
Meilgaard, M. C., Civille, G. V., & Carr, B. T. (2006). Sensory evaluation techniques (4th ed.).
Boca Raton: CRC Press.
Montgomery, D. C. (2001). Design and analysis of experiments (5th ed.). New York: John
Wiley and Sons, 699.
O’Brien, R., & Lohr, V. (1984). Power analysis for linear models: The time has come.
Proceedings of the ninth annual SAS User’s Group international conference
(pp. 840–846).
Pagès, J. (2005). Collection and analysis of perceived product inter-distances using
multiple factor analysis: Application to the study of 10 white wines from the Loire
Valley. Food Quality and Preference, 16, 642–649.
297
Perrin, L., Symoneaux, R., Maître, I., Asselin, C., Jourjon, F., & Pagès, J. (2008). Comparison
of three sensory methods for use with the napping procedure: Case of tem wines
from Loire valley. Food Quality and Preference, 19, 1–11.
R Development Core Team (2005). R: A language and environment for statistical computing.
Vienna, Austria: R Foundation for Statistical Computing3-900051-07-0 (http://www.
R-project.org).
Richter, V. B., Almeida, T. C. A., Prudencio, S. H., & Benassi, M. T. (2010). Proposing a
ranking descriptive sensory method. Food Quality and Preference, 21, 611–620.
Schenker, N., & Gentleman, J. F. (2001). On judging the significance of differences by
examining overlap between confidence intervals. The American Statistician, 55,
182–186.
Silva, A. N., Silva, R. C. S. N., Ferreira, M.A.M., Minim, V. P. R., Costa, T. M. T., & Perez, R.
(2013a). Performance of hedonic scales in sensory acceptability of strawberry yogurt.
Food Quality and Preference, 30, 9–21.
Silva, R. C. S. N., Minim, V. P. R., Carneiro, J.D. S., Nascimento, M., Della Lucia, S. M., &
Minim, L. A. (2013b). Quantitative sensory description using the optimized descrip-
tive profile: Comparison with conventional and alternative methods for evaluation
of chocolate. Food Quality and Preference, 30, 169–179.
Silva, R. C. S. N., Minim, V. P. R., Silva, A. N., Peternelli, L. A., & Minim, L. A. (2014).
Optimized descriptive profile: How many judges are necessary? Food Quality and
Preference, 36, 3–11.
Silva, R. C. S. N., Minim, V. P. R., Simiqueli, A. A., Moraes, L. E. S., Gomide, A. I., & Minim, L. A.
(2012). Optimized descriptive profile: A rapid methodology for sensory description.
Food Quality and Preference, 24, 190–200.
Stone, H., & Sidel, J. L. (2012). Sensory evaluation practices (4th ed.). New York: Academic.
Thuillier, B. (2007). Rôle du CO2 dans l’Appréciation Organoleptique des Champagnes –
Expérimentation et Apports Méthodologiques. Reims, France: Thèse de l’URCA.
Van Belle, G. (2002). Statistical rules of thumb. New York, NY: John Wiley and Sons.
Villanueva, N. D.M., Petenate, A. J., & Da Silva, M.A. A. P. (2005). Performance of the hybrid
hedonic scale as compared to the traditional hedonic, self-adjusting and ranking
scales. Food Quality and Preference, 16, 691–703.