Final Draft - 18 May 2006

V&V Plan Title Page

This title page should include the following information, although the arrangement of the
information on the title page is left to the discretion of the organiation preparing it!
1" Date of the Document
2" #dentification of $rogram%$ro&ect%'(ercise%)tudy
*" Document Title
a" M+) ,ame and -ersion
." Document Type /i"e", 0ccreditation $lan, -+- $lan, -+- 1eport, 0ccreditation
1eport2 and -ersion
3" #dentification of document preparer
4" Distri.ution )tatement /if re5uired2
6" 6lassification /if re5uired2

Final Draft - 18 May 2006
Document Version
Version no. Date Changes
Draft 1 1 Sep 01 First draft
Final Draft - 18 May 2006
Table of Contents
V&VPLAN EXECUTIVE SUMMA!""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"# P$%LEM STATEMENT""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"#" Inten&e& Use"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"#' M&S $(er(ie)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"#* M&S A++lication"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"#, Accre&itation Sco+e"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
"#- V&V Sco+e"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""4
'# M&S E.UIEMENTS AND ACCEPTA%ILIT! CITEIA""""""""""""""""""""""""""""""""""""""""4
*# M&S ASSUMPTI$NS/ CAPA%ILITIES/ LIMITATI$NS/ & IS0S1IMPACTS"""""""""""6
*#" M&S Assum+tions"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""6
*#' M&S Ca+abilities"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""6
*#* M&S Limitations"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""6
*#, M&S is2s1Im+acts"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""6
,# V&V MET3$D$L$4!""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""6
,#" Planne& Data V&V Tas2s1Acti(ities""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""7
,#"#" Data Verification Tas2s1Acti(ities""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""7
,#"#' Data Vali&ation Tas2s1Acti(ities"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""7
,#"#* e5uire& Vali&ation Data""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""7
,#, Planne& Conce+tual Mo&el Vali&ation Tas2s1Acti(ities"""""""""""""""""""""""""""""""""""""7
,#' Planne& Design Verification Tas2s1Acti(ities""""""""""""""""""""""""""""""""""""""""""""""""""""""7
,#* Planne& Im+lementation Verification Tas2s1Acti(ities""""""""""""""""""""""""""""""""""""""7
,#- Planne& esults Vali&ation Tas2s1Acti(ities""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
,#6 Planne& V&V e+orting Tas2s1Acti(ities"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
-# V&V ISSUES""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
6# 0E! PATICIPANTS"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
6#" Accre&itation Partici+ants""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
6#' V&V Partici+ants""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
6#* $t7er Partici+ants"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
8# PLANNED V&V ES$UCES"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""8
8#" Planne& V&V Tas2ing an& 9un&ing""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""10
8#' Planne& V&V Timeline"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""10
APPENDIX A M&S DESCIPTI$N"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#" M&S $(er(ie)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#' M&S De(elo+ment an& Structure""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#* M&S Ca+abilities an& Limitations""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#, M&S Use 3istor:"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#- Data"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#-#" In+ut Data""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#-#' $ut+ut Data"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
A#6 Configuration Management""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""11
APPENDIX % M&S E.UIEMENTS TACEA%ILIT! MATIX"""""""""""""""""""""""""""""""12
3
Final Draft - 18 May 2006
APPENDIX C %ASIS $9 C$MPAIS$N"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX D E9EENCES""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX E AC$N!MS""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX 9 4L$SSA!"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX 4 V&V P$4AMMATICS""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX 3 DISTI%UTI$N LIST""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
APPENDIX I ACCEDITATI$N PLAN""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""12
4
Final Draft - 18 May 2006
V&V PLAN EXECUTIVE SUMMA!
The e(ecuti9e summary pro9ides an o9er9iew of the -+- $lan" #t should .e a synopsis, two to
four pages in length, of the ma&or elements from all sections of the document, with emphasis on
-+- scope, M+) re5uirements and accepta.ility criteria, -+- methodology, and -+- issues"
"# P$%LEM STATEMENT
This section descri.es the pro.lem the M+) is e(pected to address" The pro.lem statement
ser9es as the foundation for the definition of re5uirements, accepta.ility criteria, and ultimately
the accreditation assessment .y documenting!
the 5uestion/s2 to .e answered and%or the particular aspects of the pro.lem that the
simulation will .e used to help address
the decisions that will .e made .ased on the simulation results
the conse5uences resulting from erroneous simulation outputs
The information included in the su.sections .elow is common to all four core documents"
"#" Inten&e& Use
This su.section descri.es the pro.lem to .e addressed .y the M+), including the system or
process .eing represented and the role it plays in the o9erall program"
"#' M&S $(er(ie)
This su.section pro9ides an o9er9iew of the M+) for which this plan is written and discusses
the le9el of configuration control that currently e(ists for the M+)" Detailed M+) information
is pro9ided in 0ppendi( 0"
"#* M&S A++lication
This su.section descri.es how the M+) will .e used in the o9erall program and lists the
program o.&ecti9es the M+) must meet in order to fulfill the intended use"
"#, Accre&itation Sco+e
This su.section descri.es the scope of the accreditation effort .ased on the assessment of the ris:
of using the M+) and the a9aila.ility of resources"
"#- V&V Sco+e
This su.section descri.es the scope of the -+- effort .ased on the assessment of M+)
re5uirements, accepta.ility criteria, and the a9aila.ility of resources"
'# M&S E.UIEMENTS AND ACCEPTA%ILIT! CITEIA
This section descri.es the M+) re5uirements defined for the intended use, the deri9ed
accepta.ility criteria that must .e met to satisfy the re5uirements, the 5uantitati9e and 5ualitati9e
metrics used to measure their success, and the order of their priority The relationship .etween
the re5uirements and the accepta.ility criteria can .e shown either in te(t or in a ta.le as shown
.elow"
5
Final Draft - 18 May 2006
# M&S Requirement Acceptability Criteria
1 1.1
1.2
1.n
2 2.1
2.n
n n.1
*# M&S ASSUMPTI$NS/ CAPA%ILITIES/ LIMITATI$NS/ & IS0S1IMPACTS
This section descri.es factors that constrain the de9elopment and%or use of the M+) or that
impede the --+0 effort, including the assumptions, capa.ilities, limitations, and ris: factors
affecting M+) de9elopment and ris:s associated with using the M+) for the intended use"
*#" M&S Assum+tions
This su.section descri.es the :nown assumptions a.out the M+), the M+) capa.ilities, the data
used in support of the M+), and any constraints placed upon the M+) .y the conte(t of the
pro.lem"
*#' M&S Ca+abilities
This su.section descri.es the :nown capa.ilities of the M+)"
*#* M&S Limitations
This su.section descri.es the :nown constraints and limitations under which the M+) will .e
de9eloped, tested, and used, including constraints on M+) de9elopment that result in limitations
in M+) capa.ility, as well as constraints on M+) testing and e9aluation that result in inade5uate
information regarding M+) capa.ility /e"g", inade5uate resource, inade5uate su.&ect matter and
technical :nowledge e(pertise, una9aila.le data, inade5uately defined M+) re5uirements and
methodologies, inade5uate test en9ironments2"
*#, M&S is2s1Im+acts
This su.section descri.es the ris:s associated with de9eloping and%or using the M+) for the
intended use including the ris:s resulting from identified constraints and limitations and the ris:s
associated with doing and%or not doing 9arious --+0 tas:s as well as the impacts associated
with those ris:s"
,# V&V MET3$D$L$4!
The core of the -+- $lan lies in a step-.y-step road-mapping of how the -+- tas:s will .e
performed" -+- tas:s should .e tailored according to need, 9alued added, and resources" #n this
section, descri.e what -+- tas:s are planned, as well as each tas:;s o.&ecti9es, assumptions,
constraints, criteria, methodology, and how they will .e measured and e9aluated" #dentify what
performance metrics will .e used"
6
Final Draft - 18 May 2006
,#" Planne& Data V&V Tas2s1Acti(ities
,#"#" Data Verification Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9erifying the data within the conte(t of how it
is used in the M+)"
,#"#' Data Vali&ation Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9alidating the data within the conte(t of how
it is used in the M+)"
,#"#* e5uire& Vali&ation Data
This su.section descri.es%identifies the data that are needed to implement the tas:s" #t also
descri.es the coordination mechanism and schedule for o.taining the needed data"
,#' Planne& Conce+tual Mo&el Vali&ation Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9alidating the conceptual model" #t should
correlate specific segments of the conceptual model to the M+) re5uirements and accepta.ility
criteria as well as identify which authoritati9e resources will .e used to esta.lish the 9alidity,
including su.&ect matter e(perts, reference documents, and reference data" For each su.&ect
matter e(pert, the following information should .e pro9ided!
1" name and contact information /e"g", address, phone num.er, email2
2" agency
*" summary of rele9ant e(perience
3" education credentials
4" rele9ant pu.lications
,#* Planne& Design Verification Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9erifying the M+) design" #t should correlate
specific segments of the design to the conceptual model and to the accepta.ility criteria as well
as cite applica.le standards, codes, .est practices, etc" to which the design should adhere and
how adherence will .e e9aluated"
,#, Planne& Im+lementation Verification Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9erifying the M+) implementation" #t should
descri.e how the M+) de9elopment documentation /installation guide, user;s manual and so on2
will .e re9iewed and e9aluated as well as state how completeness, correctness, and consistency
of functional re5uirements will .e measured"
This su.section should include a discussion of the planned scenarios, test cases, and sample sie
re5uired for the simulation tests designed to 9erify that the software code is error free and to
9erify that there is successful integration of all components into a single simulation system,
system of systems, or federation" This su.section should discuss what organiation will run the
tests, what organiation will analye the results, the time re5uired to do so, and the schedule for
accomplishing the runs" 0n e(ample of the type of information to document follows!
7
Final Draft - 18 May 2006
#mplementation -erification Test Description!
1" #dentify the test .y name, date and time"
2" #dentify test conductor;s name, organiation, phone and email address"
*" Descri.e the hardware%software architecture"
3" )tate purpose relati9e to the accepta.ility criteria"
4" $ro9ide .rief description"
6" #dentify any prere5uisite conditions that must .e esta.lished prior to performing the
test case"
7" Descri.e test inputs necessary for the test case"
8" #dentify all e(pected results /pre-test predictions2 for the test case
8" Define the test procedure for the test case"
10" #dentify any assumptions made or constraints imposed in the description of the test
case"
11" #dentify the 9erification techni5ue to .e used"
,#- Planne& esults Vali&ation Tas2s1Acti(ities
This su.section descri.es the o9erall approach for 9alidating the M+) results" #t should correlate
M+) results with accepta.ility criteria and M+) re5uirements as well as #dentify all
authoritati9e resources to .e used in e9aluating the M+) results, including su.&ect matter
e(perts< mathematical or statistical techni5ues< and data resources" #t should state how the
resources are to .e applied and how the results are to .e e9aluated" For su.&ect matter e(perts, it
should descri.e the specialied s:ills or :nowledge that is needed"
For the simulation tests designed to 9alidate the results, this su.section should include a
discussion of the planned scenarios, test cases, sample sie, and data re5uired" 0s well as a
discussion of what organiation will run the tests, which organiation will analye the results,
the time re5uired to do so, and the schedule for accomplishing the runs" 0n e(ample of the type
of information to document follows!
1esults -alidation Test Description!
1" #dentify the test .y name, date and time"
2" #dentify test conductor;s name, organiation, phone and email address"
*" Descri.e the hardware%software architecture"
3" )tate purpose relati9e to the accepta.ility criteria"
4" $ro9ide .rief description"
6" #dentify any prere5uisite conditions that must .e esta.lished prior to performing the
test case"
7" Descri.e test inputs necessary for the test case"
8" #dentify all e(pected results /pre-test predictions2 for the test case
8" Define the test procedure for the test case"
10" #dentify any assumptions made or constraints imposed in the description of the test
case"
11" #dentify the 9alidation techni5ue to .e used"
8
Final Draft - 18 May 2006
,#6 Planne& V&V e+orting Tas2s1Acti(ities
This su.section descri.e the plans for producing and deli9ering the -+- 1eport and
0ccreditation $ac:age"
-# V&V ISSUES
This section discusses the important unresol9ed issues rele9ant to this stage of the --+0 effort,
including administration, coordination, and e(ecution" 1eport acti9ities underway to address
these issues and the pro.a.ility of each acti9ity;s success" 0s the -+- effort is .oth iterati9e and
dependent on the products of the M+) de9elopment process, the -+- process will most li:ely
unco9er se9eral unresol9ed issues throughout the --+0 effort" 0lthough these open-ended
areas of concern are common, it is important to document all issues early on and formulate what
acti9ities are .eing e(ecuted, or will .e conducted, to address each issue, along with the
pro.a.ility of their success"
6# 0E! PATICIPANTS
This section identifies the participants in9ol9ed in the --+0 effort as well as the roles that they
are assigned and their :ey responsi.ilities within that role" 1oles and :ey responsi.ilities are
defined during initial planning< names and contact information of the actual participants are
added when they are determined" For each person ser9ing as a )u.&ect Matter '(pert /)M'2,
include a listing of the person=s 5ualifications"
6#" Accre&itation Partici+ants
This su.section lists the participants in9ol9ed in the accreditation effort, including their contact
information, assigned role, and the :ey responsi.ilities associated with that role" Typical
accreditation roles include 0ccreditation 0uthority, 0ccreditation 0gent, 0ccreditation Team, and
)u.&ect Matter '(perts /)M's2"
6#' V&V Partici+ants
This su.section lists the participants in9ol9ed in the -+- effort, including their contact
information, assigned role, and the :ey responsi.ilities associated with that role" Typical -+-
roles include M+) $roponent, -+- 0gent, -+- Team, -alidation 0uthority, Data )ource, and
)u.&ect Matter '(perts"
6#* $t7er Partici+ants
This su.section identifies the mem.ers of the application program and model de9elopment effort
with -+- or accreditation responsi.ilities as well as others who ha9e a role in the --+0
process" The information should include their position or role, contact information, and --+0
responsi.ilities" Typical roles include M+) $rogram Manager, 0pplication )ponsor%>ser, M+)
De9eloper, Data )ource, Milestone Decision 0uthority, $rogram ?ffice, M+) De9elopment
Team, >ser @roup, 6onfiguration 6ontrol Aoard, and )u.&ect Matter '(perts"
8# PLANNED V&V ES$UCES
This section discusses the resources planned for the -+- effort, such as man-hours, materials,
and funding" 0s with any programmatic effort, the documentation of schedule, manpower, and
funding plans is not only wise, .ut re5uired" Fiscal responsi.ility for -+- dictates careful
planning" Thus, the -+- $lan should include a definition of tas:s and their associated resource
9
Final Draft - 18 May 2006
re5uirements so that $rogram Managers can carefully .udget for -+- in their resource
allocations" For clarity, it is .eneficial to map out the schedule, manpower, and funding
re5uirements in ta.les and to identify any resource shortfalls"
8#" Planne& V&V Tas2ing an& 9un&ing
This su.section identifies the resources needed to accomplish the 9erification and 9alidation
planned" The information pro9ided here should include the name of the acti9ity, tas:, or e9ent,
the list of re5uired resources /e"g", )M's, e5uipment, TDB funding2 needed to accomplish it,
milestones, deadlines, and the $?6" The acti9ities, tas:s, and e9ents, and their associated
milestones, products, and deadlines should .e consistent with information pro9ided elsewhere in
this plan"
8#' Planne& V&V Timeline
This su.section identifies when each :ey e9ent descri.ed in the -+- Tas:ing and Funding
su.section should .e met"
10
Final Draft - 18 May 2006
APPENDIX A M&S DESCIPTI$N
This appendi( contains pertinent detailed information a.out the M+) .eing assessed"
A#" M&S $(er(ie)
This section pro9ides a description of the M+) including the type of model /e"g", stochastic,
deterministic, high resolution, low resolution, human in the loop CD#TEF, hardware in the loop
CDG#EF, stand-alone, engineering, aggregated2, and what types of pro.lems it is intended to
support /e"g", training, force structure analysis, command and control, e(perimentation, system
analysis, analysis of alternati9es2"
A#' M&S De(elo+ment an& Structure
This section pro9ides information a.out how the M+) is organied and%or constructed /e"g", the
M+) design2, hardware and software specifics, and technical statistics /e"g", runtime speed,
capacity, .andwidth2" For M+) under de9elopment, this section includes the M+) de9elopment
plan, including the de9elopment paradigm .eing followed /e"g", spiral de9elopment, model-test-
model2, and .asic assumptions a.out its e(ecution"
A#* M&S Ca+abilities an& Limitations
This section summaries the capa.ilities and the limitations of the M+) with respect to the
intended application"
A#, M&S Use 3istor:
This section .riefly descri.es how and when the model has .een used in the past as well as
references rele9ant historical use documents"
A#- Data
A#-#" In+ut Data
This su.section identifies the data re5uired to populate and e(ecute the simulation, including
input data sets, hard-wired data /constants2, en9ironmental data, and operational data"
Descripti9e metadata, metrics, and authoritati9e or appro9ed sources are pro9ided for each"
A#-#' $ut+ut Data
This su.section identifies the M+) output data, including a definition, the unit of measure, and
the range of 9alues for each data item"
A#6 Configuration Management
This section includes a .rief description of the M+) configuration management program, lists
the M+) artifacts and products that are under configuration management, identifies
documentation and reporting re5uirements that impact the --+0 effort, and pro9ides contact
information"
11
Final Draft - 18 May 2006
APPENDIX % M&S E.UIEMENTS TACEA%ILIT! MATIX
This appendi( traces /e"g", in a ta.le2 the accepta.ility criteria and their measures and metrics to
each M+) re5uirement" The M+) 1e5uirements Tracea.ility Matri( pro9ides a 9isual
demonstration of the degree to which a relationship can .e esta.lished .etween the M+)
re5uirements and associated accepta.ility criteria for accreditation /and e9entually will 9isually
demonstrate the totality of e9idence collected during -+- implementation2" 0s the M+)
re5uirements pro9ide the .ac:.one of accreditation and its associated -+- effort, the clearer the
articulation and detailed le9el of re5uirements and accepta.ility criteria, the more accurate and
rele9ant the -+-" #n this light, an M+) 1e5uirements Tracea.ility Matri( can .e an in9alua.le
tool in capturing the Hone to oneI tie .etween the 0ccepta.ility 6riteria and the M+)
1e5uirements"
APPENDIX C %ASIS $9 C$MPAIS$N
This appendi( descri.es the .asis of comparison used for 9alidation" The .asis for comparison
ser9es as the reference against which the accuracy of the M+) representations is measured" The
.asis of comparison can come in many forms, such as the results of e(periments, theory
de9eloped from e(periments, 9alidated results from other M+), and e(pert :nowledge o.tained
through research or from )M's"
APPENDIX D E9EENCES
This appendi( identifies all of the references used in the de9elopment of this document"
APPENDIX E AC$N!MS
This appendi( identifies all acronyms used in this document"
APPENDIX 9 4L$SSA!
This appendi( contains definitions that aid in the understanding of this document"
APPENDIX 4 V&V P$4AMMATICS
This appendi( contains detailed information regarding resource allocation and funding"
Planned Resource Allocations and unding
V&V Acti!ity
Required
Resources
unding
Source
"#$
%&
% "#$
%&
"#$
%&
"#$
%&
APPENDIX 3 DISTI%UTI$N LIST
This appendi( pro9ides the distri.ution list for hardcopies of the appro9ed document"
APPENDIX I ACCEDITATI$N PLAN
This appendi( pro9ides a copy of or a reference to the 0ccreditation $lan for the simulation for
which this document has .een prepared"
12