Certification Course on

Quality Assurance and Statistical Quality Techniques

Course Level A Measurement Systems Analysis Code 1.08: MSA
Issue No.: 01
Effective Date: 15-04-2014

Measurement systems Analysis is a tool that evaluates the statistical properties of a measurement
process to determine the extent of bias in the measurement result in relation to the true value of the
characteristic measured.

The purpose of doing MSA studies is to statistically verify that the measured values are correctly
representing the true value of the characteristic we are measuring, the results have no or very low bias
and where variability exists during measurements, to reduce it to the minimum. This helps in:

a) Making correct decisions on conformance

b) Eliminating incorrect assessment of process capability

MSA studies are universally applied in all industries, laboratories and even in the service sector when
measuring characteristics like time or process errors.

Components of the measurement system

A measurement system comprises of several factors, that may influence the accuracy of the results
obtained.

It includes:

• Instruments or gages

• Reference Standards against which the instruments was calibrated;

• the methods used:

• test methods,
• sample preparation methods,
• data collection methods,
• calculation methods
• reporting methods
• the fixtures associated with the instrument, for example dial gauge fixtures
 • software that is increasingly becoming integral to a host of measuring devices,

• the environment in which the instrument is used or the test is performed,

• the persons who perform the tests and whose knowledge and skill levels play a very important
role

Terms used in MSA

Term Definition*
measurement Process of experimentally obtaining one or more quantity values that
can reasonably be attributed to a quantity

measurand Quantity intended to be measured e.g length, temperature, hardness

under the specified measurement conditions
measurement result Set of quantity values being attributed to a measurand together with
any other available relevant information, e.g measurement
uncertainty

Measurement accuracy Closeness of agreement between a measured quantity value and a

true quantity value of a measurand

Alternate definition: The difference between a measurement reading and the

true value of that measurement.
Measurement precision Closeness of agreement between indications or measured quantity
values obtained by replicate measurements on the same or similar
objects under specified conditions

Alternate definition : The degree to which an instrument will repeat the same
measurement over a period of time under specified conditions

measurement error measured quantity value minus a reference quantity value

Alternate definition: The actual difference between a measurement value and

the known standard `value.
systematic measurement error measurement error that remains constant or varies in a predictable
manner during replicate measurements

Systematic errors are predictable and expected.

measurement bias estimate of a systematic measurement error

Alternate definition: The predicted difference on average between the

measurement and the true value.

Bias can occur from instrument fault, incorrect method or operator error
stability property of a measuring instrument, whereby its metrological
properties remain constant in time

linearity the difference in the accuracy values through the expected operating
 range of the equipment
Linearity is also the amount of deviation from an instrument's ideal
straight-line performance.
Measurement repeatability measurement precision under a set of repeatability conditions of
measurement
repeatability conditions of measurement are those set of conditions
that includes the same measurement procedure, same operator,
same measuring system, same operating conditions and same
location, and replicate measurements on the same or similar objects
over a short period of time
Measurement reproducibility measurement precision under reproducibility conditions of
measurement
reproducibility conditions of measurement are those set of
conditions that includes different locations, operators, measuring
systems, and replicate measurements on the same or similar objects
measuring instrument A device used to inspect, measure, test, or examine parts in order to
determine compliance with required specifications.
specified range of measurement The limit of measurement values that an instrument is capable of
reading. The dimension being measured must fit inside this range.
discrimination (Least Count) The distance between two lines on a scale or the fineness of an
instrument's divisions of measurement units.
resolution The smallest change in a measured value that the instrument can
detect. Resolution is also known as sensitivity.
calibration The comparison of a device with unknown accuracy to a device with a
known, accurate standard to eliminate any variation in the device
being checked.
correction factor The amount of deviation in a measurement that is accounted for in
the calibration process. You can either add the correction factor to
the measured value or adjust the measuring instrument.
hysteresis The delay between the action and reaction of a measuring
instrument. Hysteresis is the amount of error that results when this
action occurs.
drift The actual change in the measurement value when the same
characteristic is measured under the same conditions, same operator,
at different points in time. Drift indicates how often a measurement
needs recalibration.
stability The ability of a measuring instrument to retain its calibration over a
long period of time. Stability determines an instrument's consistency
over time.
linearity The amount of error change throughout an instrument's
measurement range. Linearity is also the amount of deviation from
an instrument's ideal straight-line performance.

reproducibility measurement precision under a set of conditions that includes

different locations, operators, measuring systems, and replicate
 measurements on the same or similar objects
rule of ten The inspection guideline stating that a measuring instrument must be
ten times more precise than the acceptable tolerance of the
inspected part feature.
standard A recognized true value. Calibration must compare measurement
values to a known standard.
*Source : VIM JCGM

Understanding Measurement Accuracy & Precision

Measurement accuracy is the difference between a known or true value of any measurand and what our
measurement system is giving us.

Measurement precision is used to define measurement repeatability and measurement reproducibility.

Precision is how close are the observed values to each other on repeated measurements on the same
test item under the same conditions.

Very often, measurement precision is a function of the instrument class itself such as precision class of
slip gauge sets. It may be possible to improve the measurement accuracy of measuring device or a test
setup by proper adjustments, but most often measurement precision is a function of the instrument
class itself and may not be improved until the measuring equipment is changed.

More about Precision

MSA is centered around precision of measurements, therefore it is necessary to understand the basics
of precision.

a) Precision depends upon the distribution of random errors and does not relate to the true value or
the specified value of the measurand. It means Precision is a function of variation when repeated
measurements are taken using the instrument

b) The measure of precision is usually computed as a standard deviation of test results. Less precision
is indicated by a higher standard deviation.

c) Quantitative measures of precision depend critically on the stipulated conditions. Repeatability and
reproducibility conditions are particular examples of extreme stipulated conditions. The variation
itself is a function of how and where the measurements are made. Repeat measurements taken by
the same person in the same laboratory will not be the same if measurements were taken by
different persons in the same laboratory. It will be even more different if the measurements are
taken at different laboratories. The changes produce conditions of repeatability and reproducibility.
Understanding repeatability conditions

For MSA studies we take repeatability as variation in measurements obtained with one measuring
instrument when used several times by an appraiser while measuring the identical characteristic on
different pieces of the same part.

Repeatability conditions are generated within a short period of time.

Repeatability standard deviation is the standard deviation of test results obtained under repeatability
conditions. The standard deviation will not include variation across different operators or across
different instruments. It is a measure of the inherent variability of the process.

Causes of poor repeatability may arise due to operator’s skills in handling the equipment, especially
when the measurement is sensitive to handling errors, or due to aspects related to equipment such as
improper clamping, or lack of maintenance.

Poor repeatability is reflected in Gage R & R study as high equipment variation (EV) factor.

Understanding reproducibility conditions

For MSA studies, we take reproducibility as the variation in the average of the measurements made by
different appraisers using the same gage when measuring a characteristic on one part

Reproducibility standard deviation is the standard deviation of test results obtained under
reproducibility conditions.

Causes of Poor Reproducibility mostly arise due to lack of proper training of operators in using and
reading gages. For tests requiring manual skills, variation may occur due to physical attributes such as
unsteady hands.

Impact of poor repeatability & reproducibility on Process capability evaluation

It is normally assumed, when preparing control charts or determining process capability, that the spread
or variation denoted by plus minus three sigma is an accurate estimate based on the values of
individual measurements taken. In practical terms, it is quite likely that the measurements taken are
themselves subject to variation due to equipment bias (repeatability) and operator variation
(reproducibility). This implies that the estimated control limits include not only the process variation but
also the variation occurring due to measurements. This further implies that in order to know the correct
control limits or process capability, the variation due to measurement systems must be excluded.

Figure 1 demonstrates the impact of variation of measurement system on the estimate of process
capability. If you look at each of the three curves, you will notice the element of repeatability variation,
i.e equipment bias around the process mean for each operator. You can also see how the process mean
shifts due to reproducibility variation when we observe the results for different operators. Both
variations combine to artificially expand the process spread on both sides.
 Fig 1

MSA process flow

MSA study involves some preliminary analysis before the actual Gage R & R is computed. These are:

1. Preparation for the study

The first step in a MSA study is to decide which measurements are critical and subject to measurement
bias. The scope of the study is based on this. Although the study is on the measurement process, usually
it is linked to a product characteristic such as shaft diameter or surface roughness measurement.

The second step is to decide which measurement method is to be studied. For example a shaft OD may
be measured by Vernier calipers, or screw micrometer, or digital micrometer or dial gauge fixture or
even a pie tape. We also need to specify how the readings will be taken for example at 90 degree
rotations per part or at 45 degree rotations.

The third step is to establish the number of sample parts, the number of repeated readings, and the
number of operators that will be used. The number of parts can vary from 2 to 10. It is important that
the parts are identical in design, even though there will be natural variation in their dimensions.

The fourth step is to specify how many repeat readings are to be taken by each appraiser on each part.

The fifth step is to decide on the number of appraisers who will take part in the study.

Finally we need to select the operators and sample parts that will form the part of the study

2. Evaluation of measurement process stability

An important aspect of measurement systems is to establish whether they are stable over time, that is
whether they continue to produce consistent reliable results with use and passage of time.

The following process is followed for evaluating process stability

1. Choose a sample standard from the process.

When different ranges are under study, the measurement system stability is to be separately
assessed for each standard sample.

2. Accurately determine the reference value of the sample standard using reference standards

3. Measure sample standard three to five times each day over 3 to 4 weeks (15 – 20 sub groups)

4. Plot the data on a x-bar and R chart where X bar is the average and R is the range of
measurement results obtained each day

Determine if the measurement process is in statistical control from the X bar and R charts. Both charts
should be in control. If x-bar chart is not in control this could indicate that there has been a change in
environment, improper fixturing, change in the gauge, etc. If the R chart is not in control this indicates
that the measurement process variability is not stable.

If the process is unstable you will need to determine and correct the cause before proceeding further.

3. Evaluation of resolution

We need to evaluate measurement resolution to determine if the measurement system can identify and
differentiate between small changes in the given characteristic. We need to have a certain number of
discreet values to compute the Gage R & R. Very often in Gage R & R studies, all operators return almost
identical values over all the trials for the same part, as shown in Fig 2. This is an undesirable trait and is
a symptom of inadequate discrimination. This can happen when using a low resolution instrument. In a
Gage R & R study, there must be variation present.

The following process is followed for evaluating resolution

1. Choose a sample standard.

2. Measure the sample standard three to five times.

3. Repeat the process 10 to 25 times.

4. Plot data on a R chart. We can use the same R chart as we used for
measurement system stability. Fig 2

The resolution is inadequate if there are only one, two, or three possible values for the range, or there
are only four possible values for the range when n >= 3.
A thumb rule of measurement is the one-tenth rule, that is the least count or resolution of the
measuring equipment should be one tenth the resolution of the measurand.

4. Determination of bias

A very important preliminary examination is the determination of measurement bias, which is the
variation between an observed measurement and the actual measurement of a part under repeatability
conditions

The following process is followed for evaluating measurement bias.

1. Choose a sample standard with known reference value

2. Measure the sample standard 10-15 times (n) using the same measuring device, the same operator,
and the same setup.
3. Calculate x-bar (average of n measurements taken)
4. Calculate bias:
Bias = Average – Reference Value
5. Calculate standard deviation σ
5. Calculate the upper and lower 95% confidence limit (CL) – 2 sigma limits
UCL = X bar + 1.96 σ / √n LCL = X bar - 1.96 σ / √n
The X bar, should fall within the 95 % confidence limits calculated at plus minus 1.96 sigma limits
If reference value is within the 95% CL then the bias is insignificant.
If reference value is outside the 95% CL then the bias is significant and measurement system must be
recalibrated
If it is within the limits, we can be assured that the bias is insignificant , however if it is more, then we
need to investigate the causes. The remedy may lie in re-calibrating the gauge, resetting for zero temp
or correcting the measurement techniques.

5. Evaluation of linearity

Linearity is determined to evaluate whether a significant difference exists between the obtained value
and a reference value using the same instrument over the entire measurement range.

The possible causes of gauge non-linearity are - Gauge not properly calibrated at the lower and upper
ends of the operating range; Error in the value of measurand at the maximum or minimum range; Worn
out gauge ; Internal design problems etc.

For evaluating linearity, several sample standards of known reference value are required to cover the
entire range or scale of the measurement.
The following process is followed for evaluating linearity:

1. Decide the measurement range in which the instrument is used

2. Choose three to five different sample standards of known values that cover the full measurement
range.

3. Measure each sample standard 15 to 25 times, using the same measuring device, the same operator,
and the same setup

4. Calculate the average of the observed readings for each sample.

5. Calculate bias (reference value - average observed value) for each sample

5. Plot values on x-y graph – bias (Y) on the y axis and reference value (X) of each sample on the x axis
6. From the values, calculate the equation of the linear regression line Y = bX + c (you can use the
spreadsheet template : ‘Correlation & Regression’ for determining the equation; you will read more
about regression line in the next chapter)
In the equation, b is the slope of the line that represents linearity and c is the bias, which is the intercept
on the Y axis
When, the slope is zero or tending to zero it indicates low linearity. The closer the slope is to zero, the
better the instrument.
If the slope is significant, you will need to readjust, rectify and re-calibrate the gage to reduce linearity. A
less recommended method is to adjust the observed values by feeding them in the equation, to get the
correct values when the instrument is used for Gage R & R study or during actual use.

6. Planning the Gage Repeatability and Reproducibility study

In a Gage R & R Study we study the overall impact of variation on the measurement results, called Total
Variation.

Total Variation is a combination of three distinct variations:

a) Equipment Variation (EV) that represents repeatability

b) Appraiser Variation (AV) that represents reproducibility

c) Part variation (PV) that represents process variation.

We have to set up the study in such a way that we can calculate all the three variations with sufficient
reliability for which we need a good number of data points.

In most cases, a single instrument is used for performing the Gage R & R study. Sometimes, when
identical gages are in use, and which are used interchangeably by different operators, we can use more
than one gage, ensuring that each gage meets the requirements for resolution, stability, bias and
linearity.
A minimum of two operators is essential, while the preferred number is 3 or 4.
If possible, the number of parts or samples should be 10. The sample parts should be selected on
random basis, from the same production lot. We do not need to know their reference or true values.

A general rule is (no. of samples,n) x (no. of operators,k) >15

Each operator measures the parts multiple times. In most cases, the number of trials (r) must be greater
than or equal to two. Short term MSA study can be conducted with 1 trial, provided n x k >15. When it
is not possible or practical to get more samples or operators, we will need to increase the number of
trials to get adequate number of data points. The following guidelines can be used for deciding the
number of trials

• 1 trial where Number of operators x No of parts > 15

• 2 trials where Number of operators x No of parts  15

• 3 trials where Number of operators x No of parts  7

• 4 trials where Number of operators x No of parts  4

• 5 trials where Number of operators x No of parts  3

Normally, the Gage R&R study is performed with single measurement per part during each trial.
However, if the measurement result is based on the average of several measurements – then the same
procedure should be followed ensuring that all operators measure the parts exactly in the same
manner.

Performing Gage R & R Study

(Range method)

The following process is followed for calculating Gage R & R, where we will measure three types of
variations – AV, EV and PV:

1. Each sample is identified with a unique number

2. The First operator measures all samples once in random order

3. The Second operator also measures all samples once in random order

4. This is continued till all operators have measured all samples in the first trial

5. Steps 2-4 are repeated for the required number of trials, making sure the previous results are
not known to other operator
 The formulas for Gage R & R are as given below

Equipment Variation (EV) = ̿ x K1

where ̿ is the grand average of range average of all operators, range here being maximum and
minimum values recorded by each operator for each part during various trials. K1 is a constant
selected from Table 1 based on number of trials.

Appraiser Variation (AV) = √ {( ̅ x K2)2 – (EV2/nr)},

where ̅ is calculated as the range between averages of all the part measurements by each
operator, n is number of parts and r is number of trials. K2 is a constant selected from Table 1 based
on number of appraisers.

R & R =√ { EV2 + AV2}

No of k1 No. of k2 No of k3
trials appraisers parts

2 4.56 2 3.65 2 3.65

3 3.05 3 2.70 3 2.70
4 2.50 4 2.30 4 2.30
5 2.08
6 1.93
7 1.82
8 1.74
9 1.67
10 1.62

Table 1 – Constants used in Gage R & R Study

The Constants in tables are used as factors to give the values in a confidence band of 99 %.

Eliminating out of limit variations in Repeatability

It is possible that a single value recorded by an operators may lie well outside the normal range of values
due to an unintended measurement error. As these values may substantially influence the equipment
variation (EV) and consequently the overall R & R value, it is better to check the significance of
suspected out of range values. For this we use the following formula to draw the limit within which all
the range values should lie:
 ̅

UCLR represents the upper control limit if we were drawing a Range control chart. The value of D4 is
taken from the table of constants used for variable control charts. If any range value (max – min for any
given part by an individual operator) falls outside this limit, we can use the option of repeating these
readings using the same appraiser and part as originally used or discard the values and re-average and
recompute R. We may have to repeat the significance test a second time too if any value falls outside the
recomputed UCLR.

Evaluating significance of calculated R & R

Having obtained the calculated value of R & R , we need to evaluate what is the significance of the
calculated Gage R & R. This can be done in different ways:

a) As a percentage of the specification tolerance limits. If the specification limits are known, in
most cases the Gage R & R will be calculated with this method.

% Gage R & R =

b) As a percentage of known Process control limits +3 ̿ / d2 from a control chart. This method
should be used when the specification tolerances are one sided

% Gage R & R =

c) By calculating as a percentage of total variation estimated from the Gage R & R study using part
variation as a third variable

% Gage R & R =

Total Variation (TV) = √ { R&R2 + PV2}

PV or Part variation is calculated as

PV = k3 X Rp

Where Rp is the average of ranges of the values observed for each part, including all trials by all
operators (Thus if there are 3 operators, and we conducted 3 trials on each part, Rp will be the
range for all the 9 values). K3 is the constant taken from Table 1 for the number of parts used
(max 10 parts)

Note : To cross examine if all calculations are correct, we can use the following equation. If the result is 1
or 100 %, the calculations are correct:

EV/TV)2 + (AV/TV)2 + (PV/TV)2 = 1 (or 100 %)

The Excel Spreadsheet given as a part of this course, contains a readymade example showing all
formulas mentioned above and indications of how the respective range values are calculated.

You can also enter data from your work area in the template to see the results and how they are
calculated.

Using the Gage R & R results

Having determined the % Gage R & R value, we need to know what is acceptable.

As we learnt earlier, the observed process variance is a function of actual process variance and the
variance due to measurement system.

The relationship between observed process variance, actual variance and the variance due to MSA is
given by the equation

σ2observed = σ2actual + σ2msa

Table 2 shows how the actual process capability appears as reduced due to % gage R & R, using a cp
value of 1.3 for reference. Higher the MSA, the lower the process capability will appear to be.

Actual Observed Cp (based on tolerance)

Cp
GRR (in %)

10 20 30 40 50 60 70 90

1.3 1.29 1.26 1.20 1.11 0.99 0.81 0.54 never

Table 2 : Influence
of Gage R & R on Cp

The decision for what is acceptable as % gage R & R will largely depend on the actual process capability
and also the purpose of doing the MSA study, for example:

a) the need to ensure that the measurement system gives highly accurate results
b) the need to demonstrate process capability, for example to a customer

In the first case, if the process capability is very high, say 2 or above, the impact of measurement
systems error on actual product acceptance may not be very significant, as the process itself is
producing near zero defect, even though the observed Cp may appear to be much lower. However,
when the process capability is lower say around 1 or lower, we are aware that the process is actually
producing defective parts. In such case we need to be very certain about our measurements. It is in
these lower Cp ranges, where we need to take a clear decision what is an acceptable cut off for Gage R
& R. Generally 30 percent is taken as the largest acceptable criterion for Gage R &R. Values less than 10
percent are considered to be good. We must pursue a goal of 20 percent or less.

In the second case, the impact of Gage R & R is not very visible in the lower ranges of Cp upto around
0.8 to 0.9 (see Fig 3). However as the actual Cp improves, the effect of Gage R & R influences its
estimation significantly. A Cp of 2.0 may appear to be 1.3 with a Gage R & R of 50 %. A customer may
not accept process performance if he finds the process capability to be low. This is also the reason why
MSA studies are mandatory when conducting a six sigma project, where the outcome is measured in
actual sigma levels.

Fig 3

Improving Gage R & R

When results of MSA are unacceptably high, we need to make improvements in the measurement
system. Some examples of possible improvements are:
 • Ensure that the measuring device and operators are performing in the best way that is possible.
Review the method of inspection and testing, identify sources of error with the use of the
Ishikawa Cause and effect diagram, and eliminate or minimize them. Examples could be
replacing manual handling with fixtures, proper alignment of equipment, proper lighting, pre-
conditioning of parts, replacing analog displays with digital (ensuring same instrument sensitivity
level), etc,

• If the equipment variation (repeatability) is high, we can also consider switching to a better class
equipment.

• When operators (reproducibility) are the major source of variability, re-training and
standardization of procedures, frequent use of reference (standard) samples help in improving
consistency.

The method described in this reading material is based on the X bar and Range method. Ranges always
have some limitation as compared to standard deviation. Further the Range methods does not take into
account variation due to operator / part interaction. For more precise determination in critical
applications, Gage R & R calculations based on Analysis of Variance technique are carried out. ANOVA
is covered in Level B course and if you pursue your training we will see examples of Gage R & R in the
chapter on ANOVA.
I