c




 

 

 

The design and production of Medical Devices is a highly regulated industry. A
developer/manufacturer must understand and follow a large number of laws and regulations.
The introduction of computer controls, software, and increasingly more sophisticated Devices
has resulted in the regulation of how Medical Products are designed as well as manufactured.
For a software engineer working on a Medical Device, the daily activities needed to develop the
code are well documented and controlled. This includes the type and amount of testing that
needs to be performed.

c 
This paper will explore:
— What a Medical Device and how it is classified
— Why Medical Devices are regulated
— The Regulatory Agencies involved
— The regulations as they pertain to Medical Device software / software testing
— The regulations as they pertain to off the shelf software
— Common Medical Device software issues
— The difficulties in discovering/troubleshooting Medical Device failures.

This paper will not explore the European regulations that need to be followed for Medical
Devices to have the CE mark for sale overseas.

ë 






The Federal Food, Drug, and Cosmetic Act defines a medical device as:

"an instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other
similar or related articles, including any component, part, or accessory, which is:
— Recognized by the official National Formulary, or the United States
Pharmacopoeia (USP), or any supplement to them.

— Intended for use in the diagnosis of disease or other conditions, or in the

cure, mitigation treatment, or prevention of disease, in a man or other
animals.

— Intended to affect the structure or any function of the body of a man or other
animals, and which does not achieve any of its principal intended purposes
through chemical action within or on the body of a man or other animals and
which is not dependent upon being metabolized for the achievement of its
principal intended uses." Z

£ 






The FDA uses three classes to divide up medical devices based on the amount of risk the
device imposes and the corresponding depth of regulation needed.

— Class I = General Controls

ÿ Minimal potential for harm
 ÿ Examples: crutches, elastic bandages, handheld surgical instruments
ÿ Represents about 45% of all Medical Devices

— Class II = General Controls and Special Controls

ÿ Devices for which Class I is insufficient to insure safety.
ÿ Examples: powered wheelchairs, pregnancy test kits, infusion pumps
ÿ Represents about 47% of all Medical Devices

— Class III = General Controls, Special Controls, and Premarket Clearance

ÿ Sustain or support life, are implanted, or present potential hazards.
ÿ Requires clinical demonstration that the device is safe and effective
ÿ Examples: implantable pacemakers, heart valves, breast implants
ÿ Represents about 8% of all Medical Devices

 

 
The U.S. Food and Drug Administration is the primary regulatory agency for Medical Devices.
From the FDA Modernization Act of 1997, its mission in part is to:

— To promote the public health by promptly and efficiently reviewing clinical research and
taking appropriate action on the marketing of regulated products in a timely manner, and
monitoring these products for continued safety.

— With respect to such products, that there is reasonable assurance that those intended for
human use are safe and effective.

The FDA can enforce its regulations through a series of actions.

— FDA-Initiated Recall - The product is removed from the marketplace.

— Voluntary Recall - Again, the product is removed from the marketplace.
— Warning Letter - Correspondence indicating that more severe sanctions may be
imposed if the violations described are not corrected. Precedes the more severe
sanctions, listed below.
— Citation - Formal warning of intent to prosecute if violations are not corrected. It
provides the firm an opportunity to convince FDA not to prosecute.
— Prosecution - A criminal action alleging violation of the law.
— Seizure - A civil court action to remove a specific quantity of goods from the
marketplace.
— Civil Money Penalties - Fines for violations of the law. Typically imposed in addition
to other the other actions listed above.

Underneath the umbrella of the FDA is the Center for Devices and Radiological Health (CDRH)
with the charter (from its website) of:

³Protecting the public health by providing reasonable assurance of the safety and effectiveness
of medical devices and by eliminating unnecessary human exposure to radiation emitted from
electronic products."

The CDRH is the ³arm´ of the FDA that actually does all the work regarding Medical Devices.
This includes:
 — Device Evaluation.
— Performance Standards.
— Training, Technical Assistance
— Guidance Publications

* 

 

Medical Devices are regulated by the FDA to ensure that they are safe and effective. Although
this seems obvious, it can be stated a different way ± Medical Devices have failed in the past
and humans were hurt or killed in the process. The government must protect its citizens and
thus the design and manufacture of Medical Devices is now heavily regulated. Some examples
of past failures are presented here. As this paper is about software in Medical Devices the
examples will be restricted to software failures.

The Therac 25

The Therac 25 example is old considering the advances in technology since 1987, although it
still appears to be the most famous and widely referenced. A few more recent examples will
demonstrate the difficulties with today¶s software.

The MAUDE Database

The FDA/CDRH maintains the ³Manufacturer and User Facility Device Experience Database´
(MAUDE). MAUDE contains reports of adverse events involving medical devices and includes
an on-line search capability.

Examples of Medical Device Failures from MAUDE

1. Device: Ultrasound GE-400

Date: July 1999
Description: The system's software calculates the "size" of a fetus and a corresponding
ratio. The ratio was invalid for some stages of the pregnancy and an "Abnormal"
indicator had the potential to mislead Doctors. There were no reports of any harm as a
result of the problem.

2. Device: Acuson Diagnostic Ultrasound

Date: March 1995
Description: The system's software incorrectly calculates the "tricuspid valve regurgitant
orifice area". Customers were notified and a software patch needed to be issued. There
were no reports of any harm as a result of the problem.

3. Device: External DC Defibrillator/Cardiac Monitor

Date: May 1999
Description: The systems' software did not properly recognize a low battery mode,
resulting in the system's battery completely discharging with no auditory replace battery
alarm. This could have resulted in a delay of treatment if a device had a dead battery
when it was needed. There were no reports of any harm as a result of the problem
 4. Device: Plum A+ Infusion Pump
Date: November 2000
Description: The system was running in a surgical ICU when it alarmed twice and then
turned off. When it was turned back on it displayed a malfunction code. There was a
delay of 5 minutes before a replacement pump could be set up. The patient's blood
pressure dropped substantially during that time but was then stabilized with no after-
effects. The pumps' log indicated an E437 software failure at the time of the incident.

5. Device: Bedside/Central Station Monitor

Date: June 1998
Description: A patient's heart stopped while being monitored. The nurse was in the
patient's room, but the bedside monitor failed to alarm. The central station did alarm, but
the nurse was out of hearing range. The patient died shortly thereafter. The specific
cause of the failure was not given, but a software revision which corrects the bedside
alarm problem was installed.

6. Device: Angiographic X-ray System

Date: January 2001
Description: The system ³lost imaging´ during a cardiac catheterization and could not be
rebooted. The patient went into cardiac arrest and died. Note that this might not be a
software failure, but it is an important example and will be referenced later on in this
paper.

D

The primary regulation for Medical Devices is Title 21 -Food and Drugs, of the Code of Federal
Regulations (CFR). 21 CFR has many different sections dealing with all aspects of food, drugs,
Medical Devices and more. Some examples:

— 21 CFR 101 Food Labeling

— 21 CFR 558 New animal drugs for use in animal feeds
— 21 CFR 820 Quality System Regulation

21 CFR 820, the Quality System Regulation, is the most important regulation for Medical
Devices. It requires the establishment of a quality system that handles the entire life cycle of
the medical devices a manufacturer produces. For example:

— 820.40 Document Controls - Requires procedures to control all documents and records,
including approval, storage, and changes.
— 820.70Production and Process Controls. Requires controls on production and
production processes to ensure that all devices meet their specifications.

D
  
For Software Development section 820.30 Design Controls is the most relevant. It requires that
manufacturers have procedures to control the design of any Medical Device to make sure that it
meets its requirements. This applies to the design of any Class III or Class II Medical Device
plus any Class I Device using software.

Inside of 820.30 there are few references to software, but it all applies, as a Medical Device
could be purely software. Some of the more important sections:
820.30(c) Design Input. Requires procedures to ensure that the design requirements of a device
are appropriate and address the intended use of the device. A way to address incomplete,
ambiguous, and conflicting requirements is also required.

820.30(d) Design Output. Requires procedures for defining and documenting Design Output to
allow an adequate evaluation of conformance to Design Input requirements. This includes
acceptance criteria and identification of those design outputs that are essential for the proper
functioning of the device.

820.30(e) Design Review. Requires procedures for formal documented reviews at appropriate
stages of the device's design development.

820.30(f) Design Verification. Requires procedures for verifying the device design, thus
confirming that the Design Output meets the Design Input requirements.

820.30(g) Design Validation. Requires procedures for validating the Medical Devices design to
show that the design conforms to user needs and intended uses. This includes software
validation when software is a part of the design.

820.30(i) Design Changes. Requires procedures for the identification, documentation,

validation, verification, review, and approval of design changes before implementation.

For software, what 820.30 essentially requires is a fully documented software development
cycle. It does not specify which one to use, nor does it specify what to do. It only requires that
you have a process and procedures, and that they be fully documented and followed. This gives
manufacturers the freedom of developing as simple or as complex a system as needed for the
given software they wish to develop. For Software Testing, sections 820.30(f) & (g) require both
the Verification and the Validation of any software in a Medical Device

The CDRH publishes guidances to help manufacturers with the regulations. Several of these
focus on software development and testing:

— General Principles of Software Validation; Final Guidance for Industry and FDA Staff
— Off-The-Shelf Software Use in Medical Devices

D
  
This CDRH Guidance has sections that further explains the requirements for software testing.

D

 c
c
!c"

The incorporation of Off-The-Shelf computer hardware into Medical Devices (such as
motherboards or even a whole PC) means that the use of Off-The-Shelf software in Medical
Devices is also becoming more prevalent.
The use of OTS software allows the Software Developers to focus on the additional, specific
functionality the Device needs. The downside is the loss of control of the development and
versioning of the OTS software while still bearing the responsibility for producing a safe Medical
Device.

The CDRH is therefore concerned that general purpose OTS software may not always be
appropriate for use in a Medical Device. This had led to the published guidance entitled ³Off-
The-Shelf Software Use in Medical Devices´.

This guidance states that for OTS software the developer needs to generate a Hazard Analysis.
Similar to a Risk Analysis, the Hazard Analysis focuses on the harm the OTS software might
cause if it fails instead of the probability of occurrence and failure rates. The Hazard Analysis
should categorize the OTS software into one of three defined Levels of Concern:

— Major - A failure could result in death or serious injury to a patient, operator or bystander.
— Moderate - A failure could result in non-serious injury to a patient, operator or bystander.
— Minor - A failure is not expected to result in any injury to a patient, operator or bystander.

The Level of Concern identified for a particular piece of OTS software guides the amount and
type of testing required.

For Minor and Moderate Levels, in addition to the normal testing, verification and validation
required to appropriately test the Medical Device:

— It must be shown that the OTS Software is appropriate for use in the Medical Device.

— The exact OTS Software (title and version) to be used must be identified and
only that specific software may be used in testing and integration.

— If multiple versions of OTS Software are to be used in the Medical Device, then
each version needs to be validated separately.

For Major, in addition to the requirements for Minor and Moderate above:
— An audit of the OTS Software developer¶s design control methodologies must be
performed to show that they are sufficient to allow the OTS Software to be used in the
Medical Device. If an audit cannot be performed the OTS Software should probably not
be used.

— Show that the Verification and Validation performed on the OTS Software by both the
OTS Software developer and the Medical Device manufacturer meet the requirements of
the Medical Device for safety and effectiveness.

One final issue with OTS software is what to do when a new revision is offered. The guidance
states that Black Box testing of the OTS Software is insufficient to assess the full impact of the
upgrade on the Medical Device. A full regression test is recommended as a part of the normal
Verification and Validation activities associated with an update to a Medical Device.

r ## c
$ 
Software Engineers working on Medical Devices need to be aware of the more common
software faults associated with the failure of these devices. This knowledge allows the software
to be scrutinized with the goal of eliminating commonly seen problems.

A study entitled ³Lessons from 342 Medical Device Failures´ addressed this issue. The authors
searched the FDA Medical Device failure databases (MAUDE plus some older ones) and
gathered those failures that:

— Occurred from 1983 to 1997.

— Were caused by a software failure.
— Resulted in the recall of the device.
— Had no associated patient injuries or deaths.

The failures were characterized into thirteen primary symptoms such as Behavior (The system
performs an incorrect action), Data (Loss or corruption), and System (The overall system). The
symptoms were than analyzed and categorized into Fault classes. It¶s these fault classes and
the percentage of failures in each class that are of interest. The classes are described along
with suggested methods for Prevention (during development before testing) and Detection
(during Testing).

The two largest Fault Categories were Logic (43%) and Calculation (24%).

Logic - This is a catchall category and includes all failures resulting from incorrect logic. A good
example from MAUDE of a logic failure is when the Defibrillator failed to recognize and alarm for
the low battery condition. (Example 3, above)

An important point is that a logic fault can originate from a number of sources. Some are certain
to be true logic errors, but many are the result of problems with the requirements.
The Requirements -
The Design ± The requirements were not understood or design incorrectly.
The Code - The design was coded incorrectly.

Suggested Prevention - Traceability analysis from design to code.

Suggested Detection ± Code Review, Code Inspection.

Calculation ± Calculation errors result from various algorithmic and computational errors,
including incorrect algorithms, unit mismatch, ranges and expressions. Examples from MAUDE
include the ultrasound systems that incorrectly calculated the fetal size and the orifice area.
(Examples 1-2, above)

Prevention and Detection depends on the type of calculation being performed.

For errors involving Algorithms

For errors involving Constants:

Suggested Prevention - Traceability analysis from design to code.
Suggested Detection ± Code Reading, Code Inspection, Unit Test

The next highest category Change Impact (6%) also deserves mentions it has been specifically
mentioned by the FDA as a problem. Change impact means that a software error is introduced
as the result of a failure to understand the impact to the overall software of some
change/modification to it. Note that there is not enough public information in the MAUDE
database to determine if any of the problems reported were due to change impact.

Suggested Prevention - Traceability analysis, Change Impact Analysis.

Suggested Detection ± Change Inspection, Regression testing.

It should be noted that the study is somewhat incomplete. The authors should have included all
software faults, including those resulting in death or injury. Additionally, the Device
manufacturers needed to be more forthcoming with information regarding the fault. In some
instances the authors were forced to guess as to the fault category, and they admit that the
³Logic´ fault class would likely have a lower percentage with better information.

‰




 

 
 %# 
 

 &c

With Medical Devices there is a unique phenomena with the user community that is unlike any
other industry. When Companies like Microsoft and Mentor Graphics release a new software
version, the user communities install, assess and call with complaints and difficulties right away.
With the medical community there appears to be a reluctance to report problems and near
misses with Medical Devices. They tend to get swept under the table rather than reported to try
and prevent malpractice charges.

Several examples of this can be found in the MAUDE Database. First, recall the earlier example
of the Angiographic X-ray System that lost imaging. The hospital¶s lawyers reported this on
1/31/2001 to the manufacturer only as the result of a pending lawsuit. The incident/ patient
death actually occurred in 1999. The length of time between incident and report hampered the
company¶s ability to troubleshoot the device as well as the limited information released by the
lawyers regarding the exact identity of the device.

In other cases from MAUDE the patient¶s information, including the equipment¶s log files that
could be used in troubleshooting, were deleted when the patient was discharged immediately
following death.

'

#
  

 



When a software failure could cause a serious accident, such as the case of the Therac 25,
hazard mitigation is best done at the system level. As an example, consider the Therac 20. A
slightly different version from the Therac 25, it nonetheless had the exact same software bug.
The difference is that no one was ever killed or injured by the Therac 20 since it had hardware
interlocks. The bug was just a nuisance, as a fuse would blow whenever it occurred.

Hardware can even used to mitigate lesser risks. Ultrasound probes need to transmit a great
deal of sound energy and consequently can get quite hot. The software is designed to shut
down the probe if it overheats, but there is typically a thermistor or thermocouple as a hardware
backup.