NI TestStand Verification and Validation Best Practices

Publish Date: Oct 31, 2013

Overview

An on-going challenge of todays test world is ensuring that test systems are designed correctly and that they continue to function
properly. While developing a test system, engineers create test procedures and set measurement limits to correctly detect
defective products. Once the test methods have been determined, the test system is developed to confirm that the software and
hardware performs the tasks correctly. When the system is completed and implemented, changes might arise in the company,
product, or test environment that must be incorporated into the test system. Verification and Validation (V&V) processes formally
ensure that the test system is developed correctly and accomplishes its intended purpose.
V&V primarily affects businesses governed by ISO or FDA procedures or good practices that manufacture products such as
pharmaceuticals or medical devices, or products for automotive and aeronautical use. Since such products are highly critical to
health and safety, these industries are subject to formal oversight, including well-defined V&V processes. Some companies
voluntarily invest in formal V&V processes to reduce costs, or for competitive reasons. If a companys competitiveness is based on
quality or reliability, investing in a rigorous V&V process might pay for itself.
NI TestStand software helps engineers develop test systems by providing tools to create test sequences that call code modules
written in many different programming languages. TestStand also helps test engineers manage test limits and configuration.
TestStand components are critical to any V&V effort.
This document discusses V&V as it applies to test systems developed with TestStand. First, this document introduces the
concepts of verification, validation, and impact analysis as applied to TestStand. Second, this document examines the
components of TestStand and describes some best practices to help streamline V&V for each component. Finally, this document
discusses some general purpose best practices for V&V.
Table of Contents

1.
2.
3.
4.
5.
6.
7.

Validation and Verification

Impact Analysis
Components of TestStand
General Best Practices for Validation and Verification
Conclusion
View Additional Sections of the NI TestStand Advanced Architecture Series
About the Author

1. Validation and Verification

The governing principles of V&V are well-defined for many industries, and are outlined by disciplines like Good Manufacturing
Practices (GMP) or by regulation such as ISO-9000, FDA's 21 CFR, or IEEE Standards. Each V&V system is similar but uses
slightly different terminology to explain the generic requirements of the two processes. Specific requirements are usually not
defined. This document explores V&V processes for automated test systems.
Learning the difference between verification and validation is an important step in preparing to develop and certify a test system.
Verification is one or many formal audits to determine if a test system is built according to specifications provided in a design,
drawing, statement of work, or other similar guideline. Verification allows an auditor to test at the end of a development phase if
the item being produced meets the conditions imposed on it. Verification tests can be performed at several milestones of a
product development, and can be performed on assemblies and subassemblies.
Validation is the process to evaluate if the system, having already passed verification to one or more specifications throughout
several phases, is complete and now accomplishes its purpose or intent. Validation might include a formal pass/fail test
procedure, or it might be a subjective form of a usability study performed with customers, users, or patients. Validation often
involves some subjective requirements, such as rejects defective products or has an easy to use interface.
To illustrate the difference between verification and validation, suppose a test department builds a simple test fixture for measuring
the electrical current consumption of a unit under test (UUT). The test system must compare the current of the UUT current
against test limits that require the UUT to consume less than 500mA. If the system performs the measurement correctly with
acceptable repeatability, the system passes verification.
1/8

www.ni.com

If a known failure mode exists in manufacturing, such as a diode installed in reverse, that causes some parts of the circuit not to
activate, the current consumption might be excessively low, perhaps 150mA. Unfortunately, in this example the UUTs pass
testing on the verified fixture, and might be shipped. Though the test system was built correctly according to the specifications, the
system does not serve the purpose it was commissioned to fulfill and therefore fails validation. The specification and the test
system must be modified to incorporate an upper and a lower measurement limit, such as 400-500mA.
Performing system verification can be relatively easy based on a well-written specification, drawing, or statement of work, and test
methods can be very straightforward so that defects are easy to find, but validation can be more challenging, as shown by the
previous example.
In comparison, validation requirements that relate specifically to medical devices in the FDAs 21 CFR are vague, including
phrases that specify that a medical device must be validated to conform to user needs and intended uses, so the quality system
manager must define the needs and oversee validation testing. Selecting the validation methods is not necessarily very easy. For
test systems, one method of defining a validation test might be as simple as keeping track of known failure modes, and having
good and bad product samples available to help ensure the system detects known defects. Another method might use a trusted
manual test procedure or include another automated system to validate the results of the new test system.
Some instances of validation can be extraordinarily thorough, such as those that surround the aeronautics, pharmaceuticals, and
medical device industries, because the validation processes involve extremes cases of safety, quality, or cost. A thorough system
validation might take weeks or months to define and perform. For example, if a test system uses a switch matrix in a 16x32
configuration, the test engineer might be required to test every possible combination of connections using a continuity tester and
ensure that no restricted connections are ever made. Another example might consist of validating a communication system in
which every possible command and sequence of commands must be tested and validated. Although such validation processes
might seem extreme, is imperative that no damage, injury, or incorrect results arise under any circumstance.
No written procedures exist to explain what must be verified or validated, or to define how testing must be accomplished. The
same is true for reverification or revalidation if changes are made to a test system. The organization must appoint someone to
make recommendations about test procedures and review and approve them. Although each company must decide and define
how to implement design controls and change management in their products and test systems, this document provides some
ideas and best practices to help with defining such policies.
2. Impact Analysis

Impact analysis concerns the understanding of consequences of any change that alters the way in which a test system performs,
such as replacing a failing instrument or modifying an algorithm or setting. Changes might be due to maintenance or repair, or
trying to improve or correct the performance of the system. A change might cause the system to operate improperly in a
noticeable or even unnoticeable manner, and might result in product recalls, production stoppages, or other interference with
business. A change might also cause the system operate properly but affect the outcome or test results, and cause incorrect
decisions about tested products.
The cost of a missed change or an incorrect validation can be extremely large, such as shipping bad products to customers. In
some industries, shipments of faulty products can result in a product recall. The FDA reserves the right to take regulatory action.
One company reported that they take extra measures beyond typical V&V processes because even a tiny reduction in stock price
might cost them billions of dollars.
Assume that every change you make will require revalidation. No simple rule exists for deciding what you can change, how you
can make changes, or what effects will arise from any change. Understanding the components of TestStand and the impact of
changes made to the components can help make changes easier to plan for, easier to recognize, and can minimize the scope and
difficulty of testing the changes.
3. Components of TestStand

TestStand is not a traditional programming language, and therefore possesses distinct advantages that facilitate V&V processes.
TestStand includes a very modular architecture made up of basic building blocks that allow your test system to operate however
you want. The following sections describe the TestStand components as methods to facilitate design decisions to better manage
V&V processes.

2/8

www.ni.com

Sequences & Steps

The most fundamental part of a TestStand system is a sequence. In a sequence, you define the overall behavior of the system
and the settings and limits applied to each test. Even in the simplest sequence, much of the work involves creating and
configuring steps in the sequence. You can add sequencing actions, such as looping properties, preconditions, and other
configurations that directly influence which steps execute and in what order. For each step, you can select a number of settings,
such as how to record results, what expressions to evaluate, or what actions to take if the step fails. Both built-in and custom
steps have options that determine if a step passes or fails, and if the UUT passes or fails. Some steps determine failure from
measurement limits that you set. For example, for the numeric limit step you can require that a measurement be within a range.
Changes to the steps in a TestStand sequence fundamentally modify how a test system operates. Changes include renaming
steps, modifying the preconditions that allow a step to run, or even deleting an obsolete step. Such changes might be critical and
require revalidation of the system. Other changes might be somewhat minimal and require minimal validation. Each step has
some amount of interaction with the steps before and after it. Determining the effect of a change requires a specific study of how
those steps work together and where each on gets information it needs to run.
The following are some best practices to help you make changes to sequence files easier to recognize, understand, and test:
Use Case steps instead of preconditions. Case steps are more visible in the TestStand environment, are expandable, and can
be modified to run different options and include more than one step per condition. Preconditions are partially hidden and can
only determine whether to run a single step.
Use Switch Steps instead of using the switching options for a step. Switch steps are more visible in a sequence and make the
sequence somewhat more self-documenting. The switching options for a step are convenient but are partially hidden and can
only be set to start and stop in relation to a single step. Create written procedures and TestStand sequences that relate to
each other directly. For example, if a written procedure requires that you first power on a device and wait for the device to
boot, create two instruction steps in the procedure to correlate to two steps in TestStand.
Name every step carefully as a form of documentation within the sequence. The step name should describe what the step
does and why it performs an action, if possible. For example, instead of naming a step Wait, name the step Wait 2 second for
system to boot. If the name requires more information, use the Description property of the step to specify additional details.
Organize the sequence so that the Main sequence is composed almost entirely of Sequence Call steps. Each Sequence Call
step can relate to a single test that can be given a name, such as Power and Current Test sequence call followed by the Audio
Test sequence call, and so on for each test in the test procedure. This helps logically organize the sequence similar to how
someone might describe the sequence verbally, saying The procedure starts by doing a power and current test, then it does
an audio test.
Create each subsequence to stand alone and prepare itself to run, including the setup of all instruments and devices for the
subsequence. For example, if you make changes to the Audio Test, you might want to run the subsequence over and over
before validation to ensure it runs properly and to avoid having to run the entire sequence. The test subsequence should
power on the device, set up all necessary switches or instruments, and execute with noticeably good or bad feedback from the
Audio Test results. If each test runs independently, testing and retesting can become much easier.
Avoid using the PreviousStep property and use variables instead. The PreviousStep propertyrefers dynamically at runtime to
the step that last ran and that retrieves information from the step. The source of the information can change if you rearrange or
delete steps, thus resulting in erroneous or missing data. Set a variable using a the PostExpression property of the step and
retrieve data from the variable as needed.
Code Modules

TestStand does not control instrumentation and automation hardware directly, but instead calls code modules to perform these
tasks. TestStand relies heavily on the design and performance of code modules to operate properly. Code modules are small
pieces of code written in a number of languages, including LabVIEW, C++, C#, and Microsoft Visual Basic. A test system
developer or coworker might create many of the code modules, while a third party might create other code modules. Additionally,
many steps included with TestStand are actually code modules written in C and compiled into DLLs. Changing a code module
used in a test sequence fundamentally affects how TestStand performs actions, but this can be advantageous because TestStand
is a fixed and compiled application that cannot be changed, and only the code modules TestStand executes can be modified.
Because a code module must only receive data from TestStand, execute a specific action, and return data to TestStand, you can
limit testing caused by a code module change.
The following are some best practices for using code modules:
Plan ahead when you select parameter inputs and outputs for the code dodule. Do not rearrange or rename inputs unless no
alternative exists. For example, when you select a connector pane for a LabVIEW code module, select a connector pane with
extra inputs and avoid changing the connector pane. Any of these actions might cause the step to not execute in TestStand or
might cause the step to execute but return results improperly or out of order.
Design or implement code module testers that can test all possible inputs and record the output values for each set of inputs.
This can help prove that a single code module executes correctly as it did before the change, but you must still prove that new
software works with TestStand, and that it executes and returns correct results. If the code module passes module testing and
can still be called from TestStand, no change or effect to the TestStand sequence exists, and you can limit testing.
Avoid using the Sequence Context to create variables or to set the value of variables dynamically. Instead, pass values from
code modules to TestStand using the Module Adapter inputs and outputs to make setting variables more obvious to see and
easy to track.This also allows you to use the Test Expression button to verify the datatype or expression and ensure that the
data type or expression evaluates correctlyto help avoid setting the wrong variable with the wrong data.
Data Types and Process Models

Data types and process models other characteristic components of TestStand that you can change to allow advanced
3/8

www.ni.com

Data types and process models other characteristic components of TestStand that you can change to allow advanced
programmers to fine tune performance and behavior. Many users employ these components in the background without realizing
they have done so, and without modifying the components intentionally. For example, aata types, such as strings, numbers, and
arrays, define the characteristics of the variables and constants throughout the system. A process model is a sequence within a
sequence that is generally unseen and that governs how your sequence starts, executes, stops, and reports data, and manages
batch or parallel testing. You can modify data types and process models, but leaving them in their default state guarantees
smooth V&V processes. You might think of these components as being similar to the registry within Microsoft Windows, in which
developers must thoroughly understand changes to avoid unintended changes to the system. Although some changes to the
process model might seem relatively simple, the complexity of the changes can affect multiple components in the system, thus
making V&V processes difficult to plan.
The following are some best practices for using data types and process models:
Treat changes to the process model as seriously as you do when you make a registry change or large operation system
change. Consider creating a backup of the sequence file and process model files, and perhaps a backup image of the entire
hard drive if you might undo the change. .
When you create a new data type or process model, copy an existing version and modify it in minimal increments, changing
some features and testing it at each phase to ensure that the change occurs as you expected without many unexpected
consequences.
TestStand Settings

TestStand includes several options and preferences that you can access from the menus in the TestStand Sequence Editor or in a
TestStand User Interface. Some settings can be easily changed without any noticeable effects in operation, and cause no
noticeable change in managed or verified source code files or sequence files, but changes to other settings might significantly
affect the behavior of the system. One such setting is the database schema that handles how data is stored in a database, and
how TestStand modifies measurements in preparation for storage. Another setting, which is as simple enabling a checkbox, can
disable all reporting and result in serious implications to any test system. You must be aware of what the TestStand settings
accomplish, how you intend to use the settings, and how the settings affect your test system.
The following are some best practices for using the options and preferences in TestStand:
Review all available settings early and decide how to set them before the system is verified. If possible, include the settings
you want to use in an early draft of the specification, so that the settings are less likely to change later. For example, decide
what kind of reports you require and select the appropriate settings. Try to imagine any changes that you might have to make
to those settings over time. If not conceived early, a change to a setting would require revalidation. Pay attention to database
and report settings, especially as they relate to hard drive space or database size. A system that tests 1000 units per day and
generates a report for each unit can become difficult to browse or might fill up the hard drive on the computer. If a customer
must retain all reports but does not use them on a daily basis, keep the size of the reports as small as possible. If the customer
uses reports only in case of failure on a test run, but never uses the reports again, you can instruct TestStand to reuse the
filename and overwrite old reports.
You can keep the size of reports as small as possible by turning off reporting for as many steps as possible. If reporting is
activated for every step, test with just five key measurements might include over one thousand entries in the database and
reports. Many steps, such as DoWhile, End, For, and Break, do not provide much useful information in databases or reports.
Steps inside loops write multiple records for each loop. In fact, many consumers of the reports and database, such as
operators and quality engineers, will likely complain that the reports are very difficult if they only want to find what failed on a
single test run.
Use search directories to eliminate the possibility of finding code in an unexpected place.
Use relative search paths to locate code if you decide to move or rearrange folders and files.
Factors External to TestStand

TestStand developers must realize that numerous factors can affect a test system, including a company's network configuration,
the computer name, or the correct date and time. Some of the most obvious but often overlooked settings are those for
instruments and the computer itself. Instrument dettings might include NI-DAQmx settings made in NI Measurement & Automation
Explorer (MAX), or GPIB and COM settings for a device. In Windows, a setting might include a network drive mapping, or a
screen saver setting. Changes to such settings can interrupt or modify the operation of a test system and might be partially
undetectable, such as an instrument failing to take valid data, or a motor failing to move. Such settings can be numerous and
validation tests difficult to design.
The following are some best practices for handling external factors:
If you can set any options dynamically, consider using the Setup step group in a sequence to ensure the are set correctly.
If possible, query each setting to ensure that the setting was accepted by the instrument or program. If the setting cannot be
queried, find where the setting is stored and read it from a text, INI, or XML file. The system can verify and even record the
state of items outside of TestStand and keep them under control.
Find files that contain the settings and manage them using source code control (SCC).
The preceding sections provided a brief introduction to the TestStand components that you can change, resulting in reverification
or revalidation of a test system. Use discipline and best practices to plan ahead and save unnecessary delay and cost.
4/8

www.ni.com

4. General Best Practices for Validation and Verification

This section describes more general best practices that do not apply to any specific TestStand component. Although not
completely exhaustive, the ideas below can help educate and remind you of items you might want to manage while developing
any test system subject to V&V processes.
Documenting and Gathering Requirements

V&V processes center on well-defined specifications. Validation can also be subject to some objective issues, such as loosely
defined needs of the marketplace or the end user. The most important first step for any test system is to research and document a
good working specification and V&V requirements. Testing can be difficult if specifications are not specific or leave room for
interpretation or ambiguities. Testing can be halted if the auditor or observer finds a setting undocumented or implemented
incorrectly. A well-written specification implemented with care and attention can help ensure a painless V&V process.
Verification requires one or more design documents or drawings to govern what the system must accomplish. The documents
and drawings might cover a component, assembly, or entire system. The specification and test methodology for verification must
be a throughly detailed document with as much information as necessary to create a correct test system.
Thoroughly record any changes to specifications, whether devised by a customer, by engineers, or as the result of learning and
discovery. Employ a change order process to record the change and the reason, and to make the change official. Verification
only passes if you match instructions, settings, and test limits to the correct document. You might also need to record less obvious
changes, such as Quality Manager agrees to reduce measurement resolution to reduce test time, which will increase throughput
for Manufacturing Manager. You might want to debate, test, agree, and release such a change under a change process to ensure
all the changes are correctly implemented.
Designing a validation test might seem like more of an art than a science, and although wisdom and experience might seem like
the only tools for validation design, remember that gathering requirements can be revealing and useful. Techniques might include
reviewing past performance of other test fixtures or products, interviewing operators and their supervisors, and studying past
measurement data. One company that commonly outsources to test system integrators performs a detailed review of each project
to find ways to make the next project better, and places those ideas in a checklist for the next project.
For example, if you manufacture a product as simple as a cooling fan for electronic enclosures,compliance requirements might
include electrical emissions testing and describe the way the emissions must be tested. For the same product, an audible fan
noise might not exist in the verification spec or test procedure. Assume that a study of product warranty data reveals that annoyed
customers historically return noisy fans that are too loud in home or office devices, and the noisy fans came in batches. Perhaps
the test system could perform random sampling of audio information to save test time while still inspecting the noise level of the
batch. If you treat requirements gathering with meticulous notes, fact-checking, and curiosity, you can help design a system that
can more easily pass validation.
Take Advantage of TestStand Modularity

When a system is modular, the system consists of several modules or components. In comparison to manufacturing, many parts
or subcomponents have their own V&V processes so that the product can pass through validation with reduced effort and can
incorporate changes and improvements. For example, in automobiles if an airbag module changes, the module can undergo
some electrical tests to ensure that the next version of the airbag is functionally equivalent to the previously validated airbag
version. Some non-electrical requirements, such as speed of deployment, size, shape, and other factors, can all be verified
outside the automobile. Because the airbag has no effect, for example, on fuel modules and exhaust systems, no reason exists to
reverify or revalidate the subsystems to approve the vehicle design as a whole. A revalidation plan might have the fuel system,
braking systems, and others marked as no change but the airbag, the steering system, and bumper sensors might be marked for a
simplified V&V process. You can apply such modularity to test systems.
In the case of TestStand, you can plan code modules to be modular in order to streamline V&V processes. For example, if you
have successfully validated version 2.5 of a sequence and version 1.8 of code moduels, consider the effort involved to change a
measurement limit. If the limit is hard coded in LabVIEW, you must modify and recompile the code, which requires you to
revalidate the new code, version 1.9. If you can make the changes outside of the source code, such as loading limits from a text
file, then the validations of versions 1.8 and 2.5 of your code and sequence file are still current. You might still need to check that
the new text file is correct, and that it is compatible with the code and sequence.

5/8

www.ni.com

Limit the Interaction between Two Subsystems

One strategy to limit the amount of revalidation is to include distinct lines of independence in your system. For example, if you
have a medical device with firmware loaded to it, followed by a number of functional tests, and a customer-specific file is loaded
into memory and tested before shipping, you can create three separate test systems that perform these tasks independently of
each other. Changes to one test station do not affect the others. In the same way you can create a single test system that
performs the work of all three, each of which can be written into its own sequence file and the sequence files can be called one
after the other from a master sequence file. IN this way you can easily show which files are modified so you need to only reverify
the files that change.
Some dependence still exists in the independent systems, whether they are separate stations or not. The functional test is
dependent on the previous firmware test working properly. Although they are interdependent, you want to limit the effects of a
change. In this example, the next step can continue as long as the previous step worked properly. You can use a step in the
functional test to verify that the loaded firmware is compatible. Because only one output value affects the next step, validation can
be minimized to ensure that the requirement is met after any changes.
For example, if test 3 is an Audio Test that includes a volume test at Low, Medium, and High volume and test 4 performs audio
quality testing that must be done at Medium volume, you might want to power off and reset the audio volume between tests and
design test 4 to set itself to medium volume. If test 4 is now completely independent on the design and settings of test 3, a change
to test 3 can result in a revalidation limited to only that single test.
Manage Files Using Source Code Control Tools

Computer-based test systems consist of numerous files. Source code, sequences, configuration files, and others change as a test
system is developed or modified.
Code Modules are stored in a format specific to their platform. For example, LabVIEW generates Virtual Instrument files called VIs
(.vi and control and type Definition files (.ctl), and organizes those files in LabVIEW project files (.lvproj). An application written
using applications such as C or Visual Basic includes a number of source files and compiled DLLs.
TestStand sequence files (.seq) are created within TestStand and you must manage them in the same way you manage software
code modules. You can also store sequence files in binary, text, or XML formats. You can organize sequence files along with your
code modules.
Generally, the system designer creates, saves, and manages the files mentioned in this. One best practice is to consolidate the
files in a single location, or use a management tool, such as a LabVIEW project or TestStand workspace, to reference them in
various locations. You must track and archive all files in support of a given specification.
Files other than those deliberately modified and collected might include .ini files, XML files, or binary files that TestStand or
another third-party application stores. For example, TestStand and LabVIEW both use .ini files to store some configurations and
settings made within options and menus. MAX uses a file designed to be hidden from tampering but that contains information
about your instruments configuration. You can export most settings from MAX to a backup file (.mce) for users to manage. Most
applications, drivers, or other tools generate files, and changes to any of these files can affect V&V processes.
The industry-accepted method for monitoring, controlling, and storing such files is Source Code Control (SCC) programs such
Subversion (SVN), Perforce, and Microsoft Visual Source Safe. Many of these programs are designed to conform to the Microsoft
SCC interface and you can use them from within TestStand or LabVIEW. In some cases, you cannot modify a file without taking
temporary ownership of it and documenting your changes in order to save them. These programs can often tell you which files
changed as well as analyze the old and new files to highlight the changes to help simplify verification. For example, Figures 1 and
2 shows an example in Subversion, in which only one file, MySettings.xml, has changed, and in that file only the Sheath Flow
value was modified. Using these programs helps simplify validation because the programs can demonstrate that all the other
settings have not changed.

6/8

www.ni.com

Figure 1. Subversion Showing Only One Modified File in the Working Directory

Figure 2. Comparison in Subversion Showing Only One Modified Value in the File
Verify Files at Run Time

After you complete and verify system configuration , you must archive the system in such a way that you can recall and restore,
verify, and distribute the system as the proper specification. For example, if the MySettings.xml file from the previous example
passes V&V along with a certain TestStand sequence file, you can commit both files to source code control and submit the files to
an independent document control entity. You might want to self-audit to verify that the modules are running on the system. Allow
TestStand to verify everything possible during testing, and even record confirmation of dependant files alongside the test results in
a database or report. For example, you might accomplish this using a step that can read the checksum of a file and ensure that it
never changes, as shown in Figure 3. Using file utilities and storing the results along with other test data creates a complete
record that verified files are being used correctly.

Figure 3. Third-Party Add-on for TestStand to Verify and Record the Checksum at Run Time
Hardware Configuration Management

The requirements for hardware configuration management are no different than for software. You must properly select, install,
program, and configure the instruments not only for the system but also for each individual test. For example, a digital multimeter
(DMM) or oscilloscope has several options to configure for proper communication and signal acquisition that must be verified and
validated at the completion of a test system and for changes to hardware in the future.
Use software to help validate hardware at run time. By reading and storing settings or other factors at run time you can have
confidence that items that must be validated along with the software are set and operating as intended. For example, your
TestStand steps might query the calibration dates of an instrument to ensure the calibration is current, and can verify the model
number and serial number of the instrument attached to a COM port to ensure that the instrument has not been replaced.
Designing your test sequence and even purchasing instruments with these considerations in mind can help simplify V&V
processes.
If you anticipate that your hardware must change, you must consider the change in a V&V process. If an instrument fails and
another instrument of the same make and model is inserted, think about what you must accomplish to verify if it operates correctly,
and design a test to ensure that the change was successful. If the instrument settings are critical from one instrument to the next,
create a sequence of steps that automatically sets up the instrument when the sequence detects a new instrument. If another
make or model is required, but the application is hard-coded to use that instrument, then you cannot avoid revalidation. Using
7/8

www.ni.com

make or model is required, but the application is hard-coded to use that instrument, then you cannot avoid revalidation. Using
Interchangeable Virtual Instrument (IVI) drivers and interfaces for instrument setup can help simplify the transition between two
instruments of the same make or model, or between two instruments of dissimilar make or model.
Understand and Manage Software Upgrades

A common question concerns whether you can upgrade LabVIEW, TestStand, or any other program to take advantage of new
features as they become available. Making such a software upgrade is always a trigger for a revalidation and reverification. Treat
a potential upgrade as a return on investment (ROI) exercise. For example, to gain a streamlined development interface, you
might want to upgrade during development but not after the system is deployed. However, as is the case with recent TestStand
upgrades, improved execution speed can result in shorter test time, greater throughput, and greater revenue. In both cases, the
cost of revalidation is the deciding factor, but the cost can also provide a positive ROI and is therefore worth the expense and
effort.
Another easily overlooked upgrade is Microsoft Update, which can help to protect a system from attacks and help repair bugs
discovered in Windows. By default, Microsoft Update occurs automatically on most computers. Other companies, such as Sun,
Apple, and Adobe, also use web-based automatic updates. Unequivocally, you must disable any automatic changes and
upgrades on any system that is subject to V&V processes. The changes that automatic updates make are not predictable and can
have unknown effects on operation and settings. Some updates automatically reboot your computer after installing.
Your Internet Technology (IT, MIS, or other) department might have a general policy that they control computers within the
company, including using virus scanning software, setting security policies like screen savers, and installing patches and upgrades
as needed. A manufacturing department must work with the IT Department to help manage TestStand systems by leaving them
absolutely untouched. These computers are not typically used to read email or browse the Internet and are therefore less
susceptible to worms or viruses. You must decide what items specifically affect your computers, but your needs might contrast
with IT policy, such as removing virus scanners, turning off screen savers, and exempting from company-wide upgrades or
patches.
5. Conclusion

Significant challenges can exist with V&V processes for any test system. The best practices described in this document are just a
sample of the methodology that simplifies V&V processes. You must tailor any of these practices to your company, because some
might with your particular environment. Enforcing some or all of these disciplines can improve your product quality, yield, rework
efforts, and many other tangible and intangible considerations in order to reduce costs, turn greater profit, and please customers.
6. View Additional Sections of the NI TestStand Advanced Architecture Series

Click to view additional documents within the NI TestStand Advanced Architecture Series covering topics of interest to advanced
NI TestStand developers
7. About the Author

Joe Spinozzi Cyth Systems LLC

Joe Spinozzi is the co-founder and Director of Operations at Cyth Systems. Equipped with 12 years of experience helping dozens
of companies develop products and test systems, he now leads a team of engineers to plan, architect, and complete projects
across many industries. Operating from San Diego, California, Cyth Systems is responsible for test systems large and small
currently operating in the United States, Europe, and Mexico.

8/8

www.ni.com