1

Model Verification and

Validation
2 2
Outline
Important of Model Verification and
Validation
Model Verification
Model Validation
Model Building, Verification and Validation

3
Important of Model Verification and
Validation
Model building is, by nature, very error prone
The modeler must translate the real-world system into
a conceptual model
Then the conceptual model must be translated into a
simulation models.
This translation is iterative
In this, there is plenty of room for making errors
Verification and validation processes can reduce or
eliminate these errors.

4
Important of Model Verification and
Validation
Model Verification is the process of
determining whether the simulation model
correctly reflects the conceptual model.
Model Validation is the process of determining
whether the conceptual model correctly
reflects the real system
Model Verification and Validation are critical
to the success of simulation project.
5
6
Reason for Neglect
The primary reasons for neglecting this
important activity are:
Time and budget pressures
Laziness
Overconfidence
Ignorance
7
Practices That Facilitate Verification
and Validation
Poor modeling practices
Build models with little or no thought about
being able to verify or validate the model
Models contain spaghetti code that is difficult
for anyone, including the modeler, to follow.
Becomes more acute as models grows in
complexity
8
Practices That Facilitate Verification
and Validation
To create models that ease the difficulty of
Verification and Validation
Reduce the amount of complexity of the model
The code is readable and understandable
Finally, model data and logic code should be
thoroughly and clearly documented
9 3
Objectives of Verification and
Validation
To produce a representative model of the
system under study
To increase the model credibility
To gradually refine the model during the
development process
10
Model Verification
Model Verification is the process of
determining whether the model operates as
intended (it runs correctly).
Building the model right
Tries to detect unintended errors in the model
data and logic and removes them.
Verification is the process of debugging the
model



11
Model Verification
Errors or bugs in a simulation model are of
two types:
Syntax errors grammatical errors
Semantic errors
Associated with the meaning of the modeler and
are therefore difficult to detect.
Often they are logical errors

12
Model Verification
Preventive Measures
Get it right the first time (models with no bugs
the first time)
In practice, this isnt always possible as bugs
are often sneaky and difficult to prevent
Practices minimizing bugs are:
BE CAREFUL

13

Preventive Measures

Use structured programming
Five principles of structured programming
Top-down design
Modularity
Compact modules
Stepwise refinement
Structured control
14
Establishing a Standard for
Comparison
One simple standard is a common sense
If your simulation is giving totally bizarre results
To construct an analytic model of the problem
(with simplified assumption) if at all possible,
e.g.,
To run simulation without downtime
A queuing system without balking


15
Verification Techniques
Conduct model code reviews
Check the output for reasonableness
Watch simulation for correct behavior
Use the trace and debug facilities provided
with the software
16
Model Validation

Model Validation is the process of determining
whether the model is meaningful and accurate
representation of the real system (Hoover and
Perry, 1990)
Building the right model
Stakeholders and customers should become
heavily involved in the validation process
Can be very time consuming


17
Model Validation
Not interested in achieving absolute validity
but only functional validity.
Functional validity: The process of
establishing that the models output behavior
has sufficient accuracy for the models
intended purpose over the domain of the
models intended applicability (Sargent, 1998)
18
Model Validation
Through simulation we convey the user or
consumer that the simulation results can be
trusted and used to make a real world decisions.
For existing systems, the model behavior should
correspond to that of the actual system.
In case of a new system, the input data should
accurately reflect the design specification of the
system.

19
Model Validation
Determining Model Validity
There is no simple test to determine the
validity of a model.
Validation is an inductive process
As with model verification, it is common to
use a combination of techniques when
validating a model.
20
Model Validation
Techniques for validating a model
Watching the animation
Comparing with other models
Conducting degeneracy and extreme condition test
Checking for face validity
Testing against historical data
Performing sensitivity analysis
Running traces
Conducting Turing test

21 9
Steps in Validation
1.- Build a model with high face validity.
2.- Validate model assumptions.
3.- Compare the models input-output
transformations against those in the real
system.
22 10
Face Validity
Face validity is concerned with the
reasonableness of the model to knowledgeable
peers.
Sensitivity analysis can help checking for face
validity.
23 11
Validating Model Assumptions
Types of Assumptions
Structural
Data
Structural assumptions must be checked
against the real system.
Data assumptions must be checked by
statistical testing.
24 12
Validating Transformations
The range of outputs of the model for a given
range of inputs must resemble the one
observed in the real system.
Use historical data.
Validate on the main response variables.
What to do if the model represents a non-
existing system?
25
Maintaining Validation

We need to maintain validation as system
specifications tend to evolve right up to, and
often even after system implementation
We need to maintain validation (keep
updating the model to continually reflect
current system design specification)

26
Optimum Level of Validation
27 5
Model Building, Verification and
Validation
Steps in Model Building
1.- Observe the real system
2.- Construct conceptual model and perform
conceptual validation
3.- Translate conceptual model into a computer
model and perform verification
4.- Calibrate, verify and validate at every step
28 4
Verification and Validation
Verification
Building the model right
Validation
Building the right model
Verification and Validation must be conducted
simultaneously throughout the model
development process
29 8
Validation and Calibration
Validation compares the model to the real
system.
Calibration adjusts the model to make it more
representative of the real system.
Validation and Calibration must be performed
all the time and until the very last minute.
30
Statistical Techniques for Validation
Analysis of variance
Multiple analysis of variance
Confidence intervals
Goodness-of-fit tests (Chi-square test,
Kolmogorov-Smirnov test)
Tests of means (t-test, nonparametric)
Regression analysis
Time-series analysis