Learning Objectives:

Quality Assurance Procedures for Digital

Radiography
• Review components of a QA
program and show how they
apply to DR.
• Understand how some
conventional tests should be
Charles E. Willis, Ph.D., DABR modified for a digital
Associate Professor radiographic system integrated
Department of Imaging Physics into an electronic image
The University of Texas M.D. Anderson Cancer Center
management system.
Houston, Texas
• Identify key references and
standards that can be useful in
QA of DR.

Quality Assurance (QA) is … Alternate definition of Quality Assurance (QA)

• All activities that ensure consistent, maximum

Quality Assurance Are we operating the devices
performance from physician and imaging facility properly?
(NCRP 99; 1988)
• Mandated in radiology by ACR Standards Are the devices, themselves,
• Often confused with Quality Control (QC) Quality Control operating properly?
• AKA QI,CQI, PI, TQM = constantly seeking
Are the devices properly
improvement Medical supported?
• Vehicle for providing highest quality medical care Maintenance
(scheduled)
(unscheduled)
(admin and technical
support services)
 Some traditional components of
a QA Program Quality Control is …

• Most tangible aspect of QA

• QA Committee • “…a series of distinct technical procedures
• Policies and Procedures which ensure the production of a satisfactory
• Reject Analysis product.”
• Radiologist Film Critique • Four major aspects:
• Operator QC Activities – Acceptance testing of new equipment or post
major repair
• Service Events
– Establishment of baseline performance
• Technologist Inservice training – Diagnosis of changes in performance before
• Medical Physicist QC Activities radiologically apparent
• Incident investigation/troubleshooting – Verification of corrective action

Who is responsible for QC?

(“It takes a village …” Sec. of State H. Clinton, Health “What’s my motivation?”
Care Expert) (unknown screen actor)

• Regulatory Compliance
• Physician responsible for clinical service is – Title 12, Code of Federal Regulations (CFR) Part 20, Standards for Protection
against Radiation
ultimately responsible – State regulations http://www.tdh.state.tx.us/radiation/
• Standards of Care
• Medical Physicist oversees the program – ACR Standard for Diagnostic Medical Physics Performance Monitoring of
Radiographic and Fluoroscopic Equipment
• QC Technologist makes day-to-day – ACR Radiography and Fluoroscopy Accreditation Program (now defunct)
– M. B. Williams, E. A. Krupinski, K. J. Strauss, W. K. Breeden III, M.
measurements, verify post-repair integrity S.Rzeszotarski, K. Applegate, M. Wyatt, S. Bjork, and J. A. Seibert, “Digital
radiography image quality: Acquisition,” J. Am. Coll. Radiol. 4, 371–388
2007.
• Service engineer carry out repairs, PM, – NCRP Report No. 99 “Quality Assurance for Diagnostic Imaging”
calibrations – Nationwide Evaluation of X-ray Exposure Trends (NEXT)
– Reference Values1
• Providing the highest quality medical care
• MANAGING RADIATION DOSE!!!
1Gray JE, Archer BR, Butler PF, Hobbs BB, Mettler FA, Pizzutiello RJ, Jr.,Schueler BA,
Strauss KJ, Suleiman OH, and Yaffe MJ.(2005) "Reference Values for Diagnostic
Radiology: Application and Impact " American Association of Physicists in Medicine Task
Group on Reference Values for Diagnostic X-Ray Examinations. Radiology; 235:354-358.
 Many factors affect image quality and patient dose
Where can we find instructions for how to
Wolbarst. Physics of Radiology (1993) Table 19-1
perform QC tests?
Factor Contrast Resolution Noise Patient Dose

Focal spot size X

• AAPM Report 74: Quality Control in Diagnostic
Off-focus radiation x (x) x
Radiology (2002)
Beam filtration x X • AAPM Monograph 20: Specification, Acceptance
Voltage waveform (x) x x
Testing and Quality Control of Diagnostic X-ray Imaging
Equipment (1991)
kVp X (x) X • AAPM Monograph No. 30: Specifications, Performance
mA (x) Evaluations and Quality Assurance of Radiographic and
Fluoroscopic Systems in the Digital Era (2004)
S X
• AAPM Report 93 CR Acceptance Testing and QC
mAs (x) X X (2007)
SID X X
• IPEM Report 91 Recommended Standards for the
Routine Performance Testing of Diagnostic X-Ray
Field size X X Imaging Systems (2005)
Scatter rejection X X

Medical Physicist’s Worst Nightmare Your first thoughts …

• “They’re installing the • “What the heck is a DEMI-RAD™ ?”

new DEMI-RAD™ • “How bad do I need this job?”
system tomorrow.” • “Where is that monograph from the AAPM
• “We need you to come 2004 Summer School?”
tell us if it’s okay to use
with patients.”
• “BTW, we’re
scheduling patients on
it for Monday.”
 Is this a plausible scenario? What is “Acceptance”?

• 3 categories of DR • Acceptance is a process whereby a customer

plus CR determines whether …
– “newly installed imaging equipment is functioning as
• 17 DR designed,
manufacturers of 37 – “complies with regulatory standards, and
– “produces high quality images.1”
products plus 5+ CR
• Data gathered during acceptance testing establishes
vendors a baseline for later quality control (QC) testing.
• This was 5 years • There are legal, financial, and warranty
ago consequences to acceptance.
1 Gray JE and Stears JG “Acceptance Testing of Diagnostic x-ray Imaging

Equipment: Considerations and Rationale” Specification, Acceptance Testing

and Quality Control of Diagnostic X-ray Imaging Equipment. Seibert JA, Barnes
GT and Gould RG. Eds American Association of Physicists in Medicine. Medical
Physics Monograph No. 20. pp 1-9. (1994).

Acceptance testing (AT) could be as

Acceptance testing is an opportunity … simple as an inspection and inventory.

• To identify and resolve discrepancies • Verification that what was purchased was indeed
prior to clinical use delivered and installed.
• Purchasing agent, radiological technologist, or
• To become familiar with the controls biomedical engineer may not recognize missing
and operation of the equipment critical components.
• For Continuing Education on new
technology and products
Advances in Digital Radiography: Categorical
Course in Diagnostic Radiology Physics. Eds
Samei E and Flynn MJ. RSNA 2003. 252 pp.
 What about functional tests? Clinical Acceptability is the trump card!
• May test all operator controls to determine if they function.
• May test the manufacturer’s claims of performance.
• Any Diagnostic Radiographic
• May test specific performance that was crucial to the selection of Imaging System must produce
this equipment. images of sufficient quality to
– May or may not be contract provisions support clinical diagnosis at
– Ex: Throughput reasonable radiation dose to the
• May test compliance/conformance with industry standards of patient.
practice. – Physician defines diagnostic quality
– Ex: DICOM, IHE – Regulatory bodies may define
• May test whether manufacturer’s installation instructions were reasonable dose, else comparison to
followed. standard of care
• May collect “engineering data” for later reference. • Humans must be able to safely
operate the equipment

Machines that produce radiation are Non-invasive kVp measurement of a

subject to government regulations DR system
• Irrespective of the detector technology, you must assess the degree to
which the x-ray generator allows the precise and reproducible control of
the primary imaging technique factors
– kilovoltage (kVp)
– tube current (mAs)
– exposure duration (msec)
• Evaluation of Automatic Exposure Control (AEC) devices differs
because “consistent and reproducible Optical Density (OD)” is no
longer an appropriate criterion!
– Christodoulou EG, Goodsitt MM, Chan HP, and Hepburn TW (2000)
Phototimer setup for CR imaging. Med Phys 27 2652-2658.
• Evaluation of focal spot size (“measure me first!”) and
“congruence”/positive beam limitation may differ
– Rong XJ, Krugh KT, Shepard SJ and Geiser WR (2003) Measurement of
focal spot size with slit camera using computed radiography and flat-panel Sensors in beam No sensors in beam …
based digital detectors. Med Phys 30 1768-1775.
• Total filtration (HVL) and leakage radiation are measured the same.
Lesson #2. Tests that involve production of large amounts of radiation require
Lesson #1: Tests that rely on the receptor to assess generator performance must be modified. protection of the image receptor.
 It might be nice to have the DEMI-RAD™ Let’s consider the “DEMI-RAD™”
service engineer present during testing system to be a “black box”
• To assist you with operation of the
machine
• Gain
– Test modes • Characteristic
– Vendor-supplied tests
• To provide technical references • Uniformity
such as the service manual or Input Output
installation instructions • Contrast
DEMI-RAD™
• To observe your measurements • Sharpness
– to “share the experience”
– in case of “questions” from the • Noise
factory
• To correct deficiencies on-the-spot • Artifacts
when possible • Dose

How can I test the imaging functions

of a “black box”? What is “output”?
• Could be laser-printed film
• A fixed input should produce a specific output (aka Gain).
– Measure with densitometer
• Output should bear a specific relationship to input (aka
Characteristic function). • Could be luminance from monitor
• Input that is uniform in two dimensions should produce uniform – Measure with photometer
output (aka Flat-field). • Could be digital values
• Projected details will be represented in the output with a – Measure with Region of Interest (ROI) or Pixel tool by viewer
particular contrast and sharpness. software
• Output will contain noise related to noise in the input and – Code values (CV) = Pixel values (PV) = grayscale values (GY)
internal sources of noise. = quantization levels (QL)
• Output should be free from artifacts. • Could be derived indicator of exposure
• Identical black boxes should produce similar output. • Includes “metadata” from the DICOM header
• Output should be free from signal from previous output Must address calibration of both output device and measurement
(erasure). device before collecting acceptance data
• Output involves a penalty, that is, radiation dose to the patient
 Important information about DR acquisition
Gain
and processing is in metadata
• Set technique factors according to
• CR vs. DX object manufacturer specification
• Mandatory vs. optional
vs. private tags • Measure/calculate the radiation
• Automatic vs. manual exposure to the detector
entry of data • Measure the output of the system
• PACS interpretation of • Complications
metadata – Auto-ranging
– Bucky factor
Lesson #3: Assessment of DR performance likely involves access to DICOM images

Exposure indicators in Computed Radiography Exposure indicators in Direct Radiography

– exposure delivered to detector – exposure delivered to patient
• GE
• Fuji • DAP, Dose Area Product, dGy-cm2
• S number, Sensitivity Number • “ESE”, Entrance Skin Exposure, mGy, at 25 cm (default)
• 1 mR at 80kVp => 200 • DEI (new)
• 200/S X • Philips/Seimens/Thompson (Trexel)
• CareStream • DAP
• EI, Exposure Index, (mbels) • EI, Exposure Index or Indicator, similar to S (Philips - exception)
• 1mR at 80kVp +1.5mm Al and 0.5mm Cu => 2000 • EXI (Seimens –exception)
• +300 EI = 2X and –300 EI = 1/2X • Canon (exception)
• Agfa • REX, Reached Exposure Value, f(Brightness, Contrast)
• lgM, logarithm of the Median of the histogram, (bels) • EI (new)
• 20 µGy at 75 kVp +1.5mm Cu => lgM= 2.56 • Hologics (semi-exception)
• +0.3 lgM = 2X and –0.3 lgM = 1/2X • Exam Factor, Center of Mass of log E Histogram, old
• Konica • DAP and “Accumulated Dose” for exam, new
• S value, similar to Fuji • SwissRay
• mA, sec, field size, kVp, no exposure indicator, old
• New: similar to Agfa lgM
 DR has wide dynamic range (latitude)
3 10000 Auto-ranging
2.5
1000
Intensity (rel)
Density (OD)

2
Film/screen
1.5 100 PSL Fuji Autora nging Spe cification
1
10 10000
0.5
0 1 67%
0.01 0.1 1 10 100 S=200, G=1.0
1000

Sensitivity (S)
1023
EDR Signal

Exposure (mR) S=180, G=0.9

S=220,G=0.9
S=180, G=1.1
100
S=220, G=1.1
Histogram re-scaling High kV
L=2.2, S=50
Over-Exposed 67%
10
00.1 mR 0.1 1 10
Raw Plate Exposure 1000 mR
Low kV, L=1.8, S=750 Expos ure (m R)
Under-Exposed

There is a documented tendency to

overexpose in CR and DR How much exposure was used?

• Oversight of exposure factor selection is impossible without

an exposure indicator
Freedman M, Pe E, Mun SK, Lo SCB, Nelson M (1993) the potential for unnecessary
patient exposure from the use of storage phosphor imaging systems. SPIE
1897:472-479.
Gur D, Fuhman CR, Feist JH, Slifko R, Peace B (1993)Natural migration to a higher
Actually “EXI”
dose in CR imaging. Proc Eighth European Congress of Radiology. Vienna
Sep 12-17.154.

• Seibert, et al Acad Radiol (1996) 4: 313-318

– QA based on exposure indicator reduces doses
• Willis Ped Radiol (2002) 32: 745-750
– 33% dose reduction if exposure indicator target followed
• AAPM Task Group #116 is effort to standardize indicators
Shepard, Wang, et al. 2009 Med Phys 36(7) 2898-2914
Barry Burns, UNC
 Exposure Indicator
from image of calibrated stepwedge, REX adjusted until
each step disappears Characteristic function

Canon CXDI-22 • Vary the input

700 – Change mAs
600 – Stepwedge
Reached Exposure (REX)

• Measure output
500

400

300
y = 115.08x - 9.1053
R2 = 0.998
• Complications
200
– Digital Look-up Tables (LUT)
100

0
– Auto-ranging
0.00 1.00 2.00 3.00
Ex posure (mR)
4.00 5.00 6.00
– Energy dependence of code values: Beam
hardening

A very fancy calibrated stepwedge

Spectral dependence of
characteristic function
80kVp no filter
GE DR CT CHEST
80kVp 3/4" Al filter
16384
y = 1425.2x - 47.347 125kVp no filter
14336 R2 = 1 y = 1042x - 30.611
R2 = 0.9998
80kVp 3/4" Al w/grid
12288 y = 1453.3x + 18.635
R2 = 1
10240 80kVp no filter w/grid
Code Value

8192 y = 811.41x + 28.054 125kVp no filter w/grid

R2 = 0.9996
6144 125kVp LucAl w/grid

4096 Linear (80kVp 3/4" Al filter)

2048
Linear (125kVp no filter)
0
Linear (80kVp no filter)
0.000 5.000 10.000 15.000 20.000
Detector Exposure (mR) Linear (125kVp LucAl
w/grid) AGFA Test Object 75 kVp +1.5 mm Cu, 47 µGy exit
 “Linear” Display processing Look-up Table
Display processing curve for Chest
from ROI of each step of image of calibrated stepwedge (LUT) is actually log-linear

Canon (Linear)
Canon CXDI-22
2048

2048 y = 199.14Ln(x) + 1233.7

1536 R2 = 0.9953
1536

Pixel value
Pixel value

1024
1024

512 512

0 0
0.01 0.10 1.00 10.00 0.01 0.10 1.00 10.00
Ex posure (mR) Exposure (mR)

REX depends strongly on Brightness

and Contrast setting! Flat-field

• Using large Source-to-image Distance (SID),

Canon (Chest)
produce a uniform input.
2048
• Inspect and measure the uniformity of the
1536 output.
Brightness 26,
• Complications
Pixel value

Contrast 12
1024
– Heel effect: if possible, rotate detector 180o
Brightness 16,
Contrast 10
512
– Backscatter: Pb backing or tabletop
0 – Fixed SID
0.01 0.10 1.00 10.00
Exposure (mR) Seibert JA, Boone JM, Lindfors KK. Flat-field correction
technique for digital detectors. Proc. SPIE 1998; 3336
3336: 348-354.
Lesson #4: Assessment of Detector requires access to “for processing” image
data as well as processed image data.
 Uncorrected DR image is inherently Non-uniformities are corrected by
non-uniform “flat-fielding”

How many defects are acceptable?

Lesson #5: Assessing the receptor may require access to uncorrected image.

Artifacts related to gain and offset correction

GE DR Canon DR

(pretty ! )
Willis CE, Thompson SK and Shepard SJ. Artifacts and Misadventures in
Digital Radiography. Applied Radiology pp. 11-20, January 2004.
 (pretty darn uniform) (pretty darn …hmm)

Contrast: what kind?

• Contrast
– slope of detector characteristic
• Contrast resolution
– Detector ability to distinguish features of similar
signal level
– Grayscale bit depth
• Contrast detectability
– Observer ability to distinguish features of similar
(darn!)
signal levels
Lesson #6. A grayscale histogram is also helpful in assessing the receptor.
 Identical machine,
Same exposure conditions same exposure conditions

Calibrated step wedge:

ROI indicates loss of latitude

SwissRay

1536
Pixel value

1024
1st Floor
2nd floor
512

LucAl Chest phantom

0
0.01 0.1 1 10
w/QC object
10X Exposure
100X
(m R)
 Where d is the del dimension …

2d lpxy=1/d 2
Sharpness

2
d
lpx=1/2d
• Spatial resolution lpx 1/2d
lpxy =
– f(digital matrix size), i.e. pixel dimensions 1/d 2

– Nyquist frequency = ½ sampling rate lpx d 2

=
(need two pixels to represent a line pair) lpxy 2d

• Bar patterns oriented orthogonal to matrix,

lpx 2 = lpxy 2
else 1.414 factor high
2d lpx 2 = lpxy

Leeds Test Object TODR[CR]

lpy=1/2d

Practical resolution is less than the Swissray DR

Nyquist frequency

• Factors besides sampling

compromise sharpness
– X-ray focal spot dimensions
– Blur in Indirect DR and CR
– Optical and mechanical
imprecision in IDR and CR
– Afterglow in fast-scan
dimension in CR
• Limit of resolution is where
Modulation Transfer
Function (MTF) has
decreased to 10% AGFA CR Test Object
 Noise

• Primary, unavoidable source of noise in

radiographic imaging is quantum noise
• Absolute magnitude of quantum noise
increases with 4D
• Standard deviation of ROI is an
indication of noise
• Complication
– Non-linear Characteristic function

When pixel value is proportional to log D,

Combination of quantum noise and
anatomic noise limits low contrast detection SD of ROI should be proportional to D-1/2

Noise indicators (simulation)

10 0.1 1/SNR
y = 2.795x -0.4937
R2 = 0.9983 SD mR/Ave mR
SD mR/mR
SD pixel

1 0.01 SD pixel
y = 0.0126x -0.4943
R2 = 0.9982 Power (SD pixel)

0.1 0.001 Power (SD mR/Ave

0.1 1 10 mR)

Exposure (mR)

DR Image CT Image Christodoulou EG, Goodsitt MM, Chan HP, and Hepburn TW (2000)
Phototimer setup for CR imaging. Med Phys 27 2652-2658.
 Variation in Exposure-dependent SNR is
SNR should improve with exposure improved by gain and offset calibration

Eleven GE DR systems, LucAl Chest phantom at 125 kVp

ADC70 Noise Measurements
SNR from central ROI of “for processing” image
27 95% CI 95% CI
250
250 95% CI 95% CI
24 A4 A4
200
200 A6 E1
E1 E2
21 150
150
E2

SNR
I1

SNR
A1 100 I10
SNR (dB)

18 100
F12 A1

50 I1 50 A6
15 A cceptance level at PS I10 C3
0 0 C1
7/29/1998 0.1 1 10
C3
100 0.1 1 10 100
C2
12 Log. (7/29/1998) Dete ctor Exposure (m R)
C1 De tector Expos ure (mR)
C2 F12

9 y = 2.4057Ln(x) + 19.987
R2 = 0.9984
Before calibration After calibration
6
0.01 0.10 1.00 10.00
Expos ure (m R)
Lesson #7. Performance data on large numbers of DR systems under
simulated clinical conditions are needed to establish action limits

New artifacts
from the discrete
Configuration management
nature of DR

• Interference
pattern between
fixed grid lines and
down-sampling
rate for display
• Disappeared on
zoom
• Bad choices
– Display default
magnification
factor
– Line rate of grid

Main Department Orthopedic Department

 Entrance Exposure Erasure
• Position representative material between
tube and detector.
– CDRH phantoms • Re-usable image media
– ANSI/AAPM phantoms (RIM)
– ACR Phantoms • Consequences of poor
– Acrylic/lucite blocks erasure
– Cu or Al filter on collimator => scatter-free – “Ghost” structures
• Use appropriate clinical technique settings. – Noise
• Use AEC if appropriate.
• Immediately subsequent to
• Measure entrance exposure and record
output. normal exposure, produce
• Compare to regulations, national trends, or image with no input and
reference levels. high gain setting. Inspect
Standardized Methods for Measuring Diagnostic output.
X-ray Exposures. AAPM Report No. 31 July 1990.

When is an anthropomorphic phantom not

Anthropomorphic phantoms anthropomorphic?

• Approximate clinical subject “Lawyer” Phantom

• Complication: non-human histogram

Inadequate subject contrast

Before calibration Post calibration S = 895, L = 1.6 S = 283, L=1.8

 Phantoms may not adequately represent
radiographic projections of human anatomy Pass/fail criteria: How do you know?

• Government regulations
• Specifications and service manuals
• Scientific literature
Mah E, Samei E, Peck DJ “Evaluation of a – Medical Physics, SPIE Proceedings, Journal of Digital
quality control phantom for digital chest Imaging
radiography” JACMP 2(2) 90-101. – Samei E, Seibert JA, Willis CE, Flynn MJ, Mah E, and Junck
KL. Performance evaluation of computed radiography
systems. Medical Physics 28(3):361-371, 2001.
• Comparison with other devices or customer
experience

Summary of four additional tests A postscript on Quality Control …

• Flat-field => Gain and uniformity • Still necessary with digital radiography
– Manufacturer’s conditions
– Measure exposure • Repeat acceptance tests periodically
• Calibrated Stepwedge => detector and incidental to service events
characteristic, display processing, contrast,
noise • Routine QC must be performed by
• Bar patterns => spatial resolution operators/supervisors of system
• Erasure => “base plus fog”
• Entrance exposure => patient dose
– Not an extra test!
 Institute processes to detect, correct, Perform and document cleaning and
report, and document errors. maintenance on a regular basis.

• Check images before release

and archive.
• Exercise vigilance over
rejected images.
– Analyze reasons for repeated
exams
– Take action based on the analysis

Automated evaluations of the image receptor

What do you do with the QC data?
XQi C1 y = -0.0052x + 218.2

• Because systems are

2
R = 0.8897
25

relatively new, 24
23

manufacturers are
MTF @ 2.5 lp/mm

22
21

uncertain about 20
19

longitudinal data 18
17

• Lower limit for test is MTF

16
15
2/13/2002 9/1/2002 3/20/2003 10/6/2003 3 years
4/23/2004 11/9/2004 5/28/2005 12/14/2005

@ 2.5 lp/mm = 17% Date

• CsI(Tl) is hygroscopic – A6 QAP data y = -0.0023x + 104.85

R 2 = 0.2349

columnar structure is
25

24

degraded 23
Spatial MTF at 2.5 lp/mm

22

• Both systems depicted 21

required detector
20

19

replacement 18

17

16

15 2 years
3/20/03 6/28/03 10/6/03 1/14/04 4/23/04 8/1/04 11/9/04 2/17/05 5/28/05 9/5/05
Date
 Involve all local resources in a team
approach to the QC effort. References:

• Radiologist
– Ultimate responsibility for quality of images
– Department can provide only the lowest quality that is
acceptable to radiologist Comprehensive
• Radiology Administrator QC Plan for CR
– Responsible for efficiency of imaging operations
• Radiology Lead Technologist
– First-line supervision of quality control operations
• Clinical Engineer
– Responsible for equipment life cycle management
• Medical Physicist
– No other person has image quality as first priority