Health Risk Communication Programs Outomes Evaluation Primer - ATSDR USA - 2004

ATSDR Evaluation Primer on Health Risk Communication Programs, TOC
Search | Index | Home | Glossary | Contact Us
Agency for Toxic Substances and Disease Registry

Evaluation Primer on Health Risk Communication Programs
Programs and Outcomes
Environmental Health Policy Committee

Subcommittee on Risk Communication and Education
Note: While the original publication dates on some of ATSDR's documents may not appear to be current, the
information in the documents is valid and may still provide relevant information.
Contents
● Preface
● About the Primer
● Overview of Principles and Techniques for Evaluating Health Risk Communication Activities
❍ Why Evaluate Risk Communication Programs?
❍ Types of Evaluation
❍ Elements of an Evaluation Design
❍ Risk Communication Objectives as a Measure of Effectiveness
❍ Barriers to Risk Communication Evaluation
● What Do We Want to Say? The Role of Formative Evaluation and Research

❍ Designing and Testing the Message
❍ Whom To Include in Review and Pretesting
❍ Guidelines for Selecting Pretest Methods
❍ Pretest Methods for Print Materials
❍ Sample Survey for Assessing Risk Communication Needs
❍ Sample Focus Group Case Study
❍ Guidelines for Analyzing Pretest Results
❍ Guidelines for Using Pretest Results
❍ Evaluating Communications To Special Populations
❍ Evaluation Tool: Risk Message Checklist
● Did It Work? Tips for Evaluating Communication Outcomes and Impacts
http://www.atsdr.cdc.gov/HEC/evalprmr.html (1 of 2)13/11/2004 17:54:54

ATSDR Evaluation Primer on Health Risk Communication Programs, TOC
❍Risk Communication Program Assessment Questions

❍ Evaluation Options Based on Available Resources
❍ Factors To Consider in Midcourse Reviews
❍ Have We Succeeded?
❍ Evaluation Case Example
❍ Evaluation Action Plan
❍ Attributes of an Effective Risk Communication Program
● Selected References
Primer Content Prepared by:
Tim L. Tinker, Dr.P.H., M.P.H.

Agency for Toxic Substances and Disease Registry
Paula G. Silberberg, M.Ed.

U.S. Food and Drug Administration
May 1997
US DEPARTMENT OF HEALTH AND HUMAN SERVICES

Public Health Service
This page last updated on May 28, 2002
Contact Name: Wilma López/ WLópez@cdc.gov
ATSDR Home | Search | Index | Glossary | Contact Us

About ATSDR | News Archive | ToxFAQs | HazDat | Public Health Assessments
Privacy Policy | External Links Disclaimer | Accessibility
US Department of Health and Human Services
http://www.atsdr.cdc.gov/HEC/evalprmr.html (2 of 2)13/11/2004 17:54:54

ATSDR - Evaluation Primer, Section 1
Preface
Among agencies of the federal government, the U.S. Department of Health and Human Services
(DHHS) and the U. S. Environmental Protection Agency (EPA) share the broadest set of
responsibilities for determining and communicating health risks to the public. Risk assessment and
communications activities have spanned a very broad range over recent years, as illustrated by the work
of the National Cancer Institute (NCI) to determine behavioral risk factors for cardiovascular
disease and cancer and public outreach efforts of the Centers for Disease Control and Prevention
(CDC) to inform the public about the adverse human health effects of environmental tobacco
smoke (ETS).
A growing emphasis within the U.S. Public Health Service (PHS) and other federal agencies on
determining the value and utility of public health information has created an increasing need for
innovative and cost-effective evaluation strategies. As with other health education efforts, health risk
communication programs must be evaluated to understand which strategies are helpful and which are
not. Health risk communication messages enter our lives in many forms (e.g., advertisements,
solicitations, health campaigns, medical care requirements, and word of mouth). This suggests many
compounding factors, many problems, many challenges, and complexities of beliefs, values, and
behaviors.
The Environmental Health Policy Committee (EHPC) is a standing DHHS committee. It is chaired by
the Assistant Secretary for Health and has representation from the PHS agencies EPA, U.S. Department
of Energy, U.S. Department of Agriculture, and U.S. Department of Defense.
In 1993, PHS undertook an analysis of health risk communication policies and practices across its
agencies with the goal of developing recommendations to improve health risk communication. EHPC
charged its subcommittee on risk communication with the analysis. Member agencies of the
subcommittee submitted examples of health risk communication activities or decisions they perceived to
be effective and examples of activities they thought had been less effective. These examples and findings
are contained in the Subcommittee's report "Recommendations to Improve Health Risk Communication:
A Report on Case Studies in Health Risk Communication." A major finding of the analysis was that
evaluation was the least understood of the communication components. Collecting process and anecdotal
information was the preferred method of evaluation, with relatively little or no emphasis on outcome and
impact evaluation.
http://www.atsdr.cdc.gov/HEC/evalp1.html (1 of 12)13/11/2004 17:55:24

The Subcommittee on Risk Communication and Education developed A Primer for Evaluating Health
Risk Communication to assist federal health risk communication practitioners and decisionmakers to help
improve their effectiveness in evaluating health risk messages and campaigns. The primer encompasses
evaluation principles and practices as a central means for ensuring appropriate goals, content, and
outcome of our health risk communication programs.
To address the perceived evaluation needs of various PHS agencies, the subcommittee recommended
that each PHS agency consider developing a set of generally accepted practices and guidelines for
effective evaluation of communication activities and programs. This primer presents key principles and
techniques to assist federal decisionmakers and health risk communicators to improve their overall
effectiveness in evaluating health risk messages and materials.
[Table of Contents]
About the Primer

Target Audience
The principles and techniques provided in the evaluation primer are designed to improve the capacity of
risk communication practitioners and decisionmakers in PHS and non-PHS agencies to evaluate the
efficiency and effectiveness of health risk communication messages, materials, and campaigns.
Purpose
Use of the primer can facilitate planning evaluations for health risk communication programs in several
key areas.
● Informs decisionmakers about what should be communicated, in what form, to whom, and with
what expected outcome.
● Identifies performance indicators used in assessing or measuring communication goals. For

example, the Seven Cardinal Rules of Risk Communication, as identified by EPA, are
1. accept and involve the public as a legitimate partner

2. plan carefully and evaluate your efforts
3. listen to the public's specific concerns
4. be honest, frank, and open
5. coordinate and collaborate with other credible sources
6. meet the needs of the media

7. speak clearly and with compassion.
● Provides guidance on how to most effectively use target audience ideas and opinions to shape the
risk communication message.
Organization
The primer excerpts from the research of leading risk communication experts, including (1) issues and
guiding principles to consider in evaluating health risk communication activities, and (2) case examples
illustrating proven evaluation methods and tools.
[Table of Contents]
Why Evaluate Risk Communication Programs?

(Regan and Desvousges 1990)
Evaluation is a purposeful effort to determine effectiveness. It is essential to communication because it

provides feedback about whether risk messages are received, understood, and internalized by those for
whom they are intended. Without evaluation, it is impossible for communicators to chose those messages
and channels that use limited resources more effectively. Instead, communicators are left to their own
subjective interpretations about what works and what does not. A lack of evaluation, therefore, affects
both quality of the specific risk communication effort and the primary goal—improving public health.
The ideal way to apply evaluation findings is to improve ongoing risk communication activities. In
addition, evaluation is valuable for other uses.
● To provide evidence of need for additional funds or other resources

● To increase institutional understanding of and support for risk communication activities
● To encourage ongoing cooperative ventures with other organizations
● To avoid making the same mistakes in future risk communication efforts
[Table of Contents]
Types of Evaluation
(NCI 1992)

The following types of evaluation have been adapted to serve the goals of evaluating risk communication
programs.
Formative. Evaluation during the formative stages of a risk communication effort assesses the strengths
and weaknesses of materials or campaign strategies before implementation. It permits necessary
revisions before the full effort goes forward. Among other things, materials can be tested for the
following characteristics:
● clarity
● tone
● comprehensiveness
● anticipated reactions.
Process. Process evaluation examines the procedures and tasks involved in implementing an activity.
This type of evaluation also can collect the following information about the administrative and
organizational aspects of the overall effort:
● number of staff working on the project

● schedule of activities
● number of materials distributed
● attendance at meetings
● number of calls to a hotline
● number of public inquiries received as a result of a public service
● announcement
● articles printed.
Outcome. Outcome evaluation is used to collect and present information needed for judgements about
the effort and its effectiveness in achieving its objectives. Not all risk communication efforts are suitable
for outcome evaluation. Outcome evaluation is most suitable when the program has clear and measurable
goals and consistent replicable materials, organization, and activities. Outcome evaluation can obtain
descriptive data on a project and document the immediate effects of the project on the target audience (e.
g., percent of the target audience showing increased awareness of the subject).
An outcome evaluation can collect the following information about the program:
● changes in knowledge and attitudes

● expressed intentions of the target audience
● changes in behavior.
Impact. Impact evaluation focuses on the long-range results of the program and changes or
improvements in health status as a result. It is designed to identify whether and to what extent a program
contributed to accomplishing its stated goals (more global than outcome evaluation). In a "real world"

environment, there are many factors that influence an individual's health behavior, including peer
support and approval, self-esteem and other individual characteristics, advertising and mass media
coverage of health, and community and institutional factors (such as availability of services). It is usually
extremely difficult to separate the impact of a health risk communication program from the effects of
other confounding variables on an individual's behavior. Thus, the results of an impact evaluation often
cannot be directly related to the effects of an activity or program because of the other (external)
influences on the target audience which occur over time. Impact evaluations are rarely possible because
they are frequently costly, involve extended commitment, and may depend upon other strategies in
addition to communication. Information obtained from an impact study may include the following:
● changes in morbidity and mortality

● changes in absenteeism from work
● long-term maintenance of desired behavior
● rates of recidivism.
[Table of Contents]
Elements of an Evaluation Design

(NCI 1992)
Every formal design—whether formative, process, outcome, impact, or a combination—must contain

eight basic elements.
1. A Statement of Communication Objectives

Unless there is an adequate definition of desired achievements, evaluation cannot measure them.
Evaluators need clear and definite objectives in order to measure program effects.
2. Definition of Data To Be Collected

This is the determination of what is to be measured in relation to the objectives.
3. Methodology
A study design is formulated to permit measurement in a valid and reliable manner.
4. Instrumentation
Data collection instruments are designed and pretested. These instruments range from simple tally sheets
for counting public inquiries to complex survey and interview forms.
5. Data Collection
The actual process of gathering information.

6. Data Processing
Putting the information into a usable form for analysis.
7. Data Analysis
The application of statistical techniques to the information to discover significant relationships.
8. Reporting
Compiling and recording evaluation results. These results rarely pronounce a program a complete
success or failure. To some extent, all programs have good elements and bad. It is important to
appreciate that lessons can be learned from both if results are properly analyzed. These lessons should be
applied to altering the existing program or as a guide to planning new efforts.
[Table of Contents]
Risk Communication Objectives as a

Measure of Effectiveness
(Covello, Slovic, and von Winterfeldt 1987)
Program planning of any sort should begin with an explicit statement of objectives and how they will be
evaluated. The following are the four major types of risk communication programs, categorized
according to their primary objectives.
TYPE 1: Information and Education
● Informing and educating people about risks and risk assessment in general.
EXAMPLE: Using statistical comparisons of risks from different energy production technologies.
TYPE 2: Behavior Change and Protective Action
● Encouraging personal risk-reduction behavior.
EXAMPLE: Using advertisements to encourage people to wear seat belts.
TYPE 3: Disaster Warnings and Emergency Information
● Providing direction and behavioral guidance in disasters and emergencies.

EXAMPLE: Using sirens to indicate an accidental release of toxic gas from a chemical plant.
TYPE 4: Joint Problem Solving and Conflict Resolution
● Involving the public in risk management decisionmaking and in resolving health, safety, and
environmental controversies.
EXAMPLE: Using public meetings to inform citizens about a possible hazardous waste site.
[Table of Contents]
Barriers to Risk Communication Evaluation

(Arkin 1991)
Every program manager faces constraints to undertaking optimal evaluation tasks, just as there are
constraints to designing other aspects of a risk communication program.
These constraints may include the following:
● limited funds
● limited staff time and capabilities
● length of time allotted to the program
● limited access to computer facilities
● agency restrictions to hiring consultants or contractors
● policies limiting the ability to gather information from the public
● management perceptions regarding the value of evaluation
● levels of management support for well designed evaluation activities
● difficulties in defining (or establishing) agency consensus regarding the objectives of the program
● difficulties in designing appropriate measures for risk communication programs
● difficulties in separating the effects of program influences from other influences on the target
audience in "real world" situations.
[Table of Contents]
Designing and Testing the Message

(AED, JHU, and PN 1993)
A crucial step in creating and assessing the effectiveness of health risk communication activities is
determining what message ideas or concepts have the best chance of "connecting" with the target
audience and influencing them to change behavior if behavior change is the stated objective. This
process begins with using formative research and evaluation, a combination of techniques designed to
help develop effective messages.
Literature reviews, in-depth interviews, and focus groups are examples of formative research tools that
can be used to help determine if one concept is more salient to an audience segment than another, and
which concepts should eventually be developed into specific messages. The general approach to
pretesting concepts is to share them with members of the target audience and gauge their reactions.
Pretesting is conducted while materials are in draft form, to allow changes to be made without great
expense if testing reveals ways to improve the messages or materials. Methods of pretesting include
intercept interviews with members of the target audience and focus groups. Pretesting helps determine
whether the messages and formats are appropriate, understandable, clear, attention-getting, credible,
relevant, and have the desired effect (e.g., to raise awareness about an issue).
[Table of Contents]
Whom To Include in Review and Pretesting

There are four groups to consider for pretesting and review.
1. Target Audience
❍ To identify current knowledge, attitudes, and behavior related to the subject to identify
whether and what kind of new information is needed
❍ To identify myths and misconceptions about the topic
❍ To assure appeal, appropriateness, understanding, clarity, and personal relevance of
materials
❍ To check for comprehension and cultural appropriateness
2. External Experts

❍ To verify appropriateness of materials based on proven models and theories of

communication
❍ To verify accuracy and appropriateness of information in the materials
3. Gatekeepers (e.g., print and broadcast media, religious leaders, political and legal groups,
legislators, and other key policy- and decisionmakers)
❍ To assure that they will support, not block, use of materials

❍ To increase "ownership" of the materials
❍ To identify problems based on gatekeepers' experiences with the target audience. If any
problems are identified, they should be verified through pretesting directly with the target
audience
4. Clearance officials
❍ To obtain approvals prior to printing
[Table of Contents]
Guidelines for Selecting Pretest Methods

You must choose the method that works best. The following are points to consider in determining the
best method.
● The material's format and complexity (e.g., longer and more complex materials may require
having an interviewer to ensure the participants complete the pretest)
● The material's degree of sensitivity (e.g., youth may be more comfortable with a self-administered
questionnaire on topics such as sex and AIDS)
● Target audience characteristics (e.g., interest and willingness to respond, lack of reading skills or
other limitations)
● Where materials can be accessed (e.g., clinic, school, home)
● Purpose of the pretest (e.g., is more in-depth information about target audience perceptions of the
topic desired?)
● Resources available for pretesting (e.g., using trained interviewers and companies to recruit for
and conduct focus groups are more expensive options; self-administered pretests can be more
inexpensive)
● Consider using a combination of methods to meet pretest purpose and budget (e.g., use a larger
number of self-tests, followed by a limited number of individual interviews to clarify findings or

explore problematic areas)
[Table of Contents]
Pretest Methods for Print Materials

1. Self-administered Surveys (mailed or personally delivered)
Purpose: To obtain individual reactions to draft materials
Application: Print or audiovisual materials
Number of: Enough to see a pattern of response
Respondents: (Minimum 20; ideal 100-200)
Resources Required: List of respondents; draft materials; questionnaire; postage (if mailed)
Pros: Inexpensive. Does not require staff's time to interact with respondents if mailed; can be
anonymous for respondents; can reach homebound, rural, or other difficult to reach groups; easy and
(usually) quick for respondents
Cons: Response rate may be low if mailed; may require followup; may take long time to receive
sufficient responses; respondents self-select (potential bias); exposure to materials isn't controlled; may
not be appropriate if audience has limited writing skills
Sample Survey for Assessing Risk Communication Needs

(Chess and Hance 1992)
1. Approximately what percentage of your on-the-job time, on average, do you spend interacting
with the public?_____
2. What are the different publics you communicate with? (check all that apply)
[ ] community members [ ] local government officials [ ] state government officials [ ] reporters
[ ] environmental groups [ ] health professionals [ ] other, please
specify_____________________________
3. What do you think will most help you improve your communication with the public?

4. What do you think will most help your organization improve its communication with the public?
5. What kind of assistance in dealing with the public would you most like to have?
6. If your organization provides training on communicating with the public, what should it be sure
to include?
7. What should it be sure to avoid?
8. Any other comments?
2. Individual Interviews (telephone or in person)
Purpose: Can answer questions similar to self-administered questionnaires; probe for individual's
responses, and beliefs; discuss range of issues
Application: Develop hypotheses, messages, potentially motivating strategies; discuss sensitive issues
or complex draft materials
Number of Respondants: Dependent on variables of issue urgency and complexity, time, and money.
For a ballpark figure, get 10 opinions.
Resources Required: List of respondents; discussion guide/questionnaire; trained interviewer, telephone

or quiet room, tape recorder (optional)
Pros: In-depth responses may differ from first response; can test sensitive or emotional materials; can
test more complex/longer materials; can learn more about "hard-to-reach" audiences; can be used with
individuals who have limited reading and writing skills
Cons: Time consuming to conduct/analyze; expensive; may yield to firmer conclusions or consensus
3. Focus Group Interviews
Purpose: To obtain insight into the target audience's perceptions, opinions, beliefs, and attitudes about
draft materials. Readability and understandability of print materials can also be addressed.
Application: Testing broad concepts, issues, audiovisual or print materials, and logos or other artwork.
Number of Respondants: 8-10 group. Usually, the number of groups is dependent on program needs
and resources. Minimum 2 groups per type of respondent. When target audience perceptions are
comparable, additional focus group sessions are not necessary.
Resources Required: Participants representative of the target audience, recruitment screening

instrument, moderator's guide, trained moderator, focus group facility with one-way mirror and audio-
and videotape capability (optional).

Pros: Capture of real-life data in a social environment where the moderator can interact directly with
respondents; group interaction and length of discussion can stimulate more in-depth responses than
individual interviews; can discuss concepts prior to materials development; can gather more opinions at
once than individual interviews; can cover multiple topics; flexibility and ability to probe for more
information; high face validity and an easily understood technique compared to sophisticated survey
research employing complex statistical analyses; provision of data more quickly than individual
interviews; and richness of data as the group participants react and build upon the responses of others in
an open format.
Cons: Too few responses for consensus or decisionmaking; no individual responses (group influence)
unless combined with other methods; can be expensive; respondents choose to attend and may not be
typical of the target population; less control of the responses by the moderator than in individual
interviews; difficult analysis of data (e.g, summarization, interpretation); special skills are required of
moderators and moderator bias may occur; troublesome differences between groups (e.g., opposite
responses); difficulty in recruiting participants; can be expensive; and logistical problems (e.g., arranging
location, dates, and times, incentive payments, and refreshments).
[Table of Contents]

U.S. Department of Health and Human Services

Sample Focus Group Case Study:

Four Focus Groups Among Women on Their Reactions to Two
Contraceptive Efficacy Tables for Uniform Contraceptive
Labeling
Background
In keeping with the goal of the U.S. Food and Drug Administration (FDA) to develop uniform
contraceptive labeling, focus groups were conducted to determine a presentation format most useful to
the consumer for contraceptive pregnancy rates. The focus group study obtained consumer reactions to
several model contraceptive efficacy tables and graphs. Using findings from the study, the Center for
Devices and Radiological Health developed for further testing two prototype, composite tables
with the characteristics the focus group participants favored for uniform contraceptive labeling.
Study Purpose
To utilize focus groups to garner reactions (perceptions, opinions, beliefs, and attitudes) and preferences
from women about the format and content of the two different model contraceptive efficacy tables.
Participant Characteristics
The single and married premenopausal women participating in the study were currently using or
considering using a birth control method; had not earned a bachelor's degree or above; had no more than
two health science or mathematics classes beyond the high school level; and had not participated in a
focus group in the past 12 months.
Recruitment Method
Women who had previously expressed interest in participating in focus groups were recruited by phone.
Sample Questions from the Moderator's Guide
(1) What do you know about pregnancy rates?

(2) Where have you seen (or heard) information telling about pregnancy rates for various birth control
methods?
(3) Has this information about pregnancy rates been useful to you in deciding on a particular birth
control method?
(4) After showing each table:
● (a) What does this table tell you?

● (b) How easy to understand is this table?
● (c) What information here is new to you?
● (d) What did you especially like about this table?
● (e) What did you dislike about this table?
● (f) How useful is this information to you?
● (g) What might you change, delete, or add to this table?
(5) How would you describe how well these tables help you understand how effective your birth control
method is compared to other birth control methods?
(6) Is there any other format you would suggest?
(7) Is there any other information you would like to see in these tables?
(8) Is there anything else you want to say about the tables you've seen today?
Major Findings
Participants were equally divided between their preference for each of the two tables. Although they did
not clearly demonstrate an overwhelming preference for one table over the other, most participants found
that the tables met the goal of helping them to understand how effective their birth control method was in
comparison to other methods.
[Table of Contents]
Guidelines for Analyzing Pretest Results

Remember these guidelines as you first plan, and then consider the meaning of the pretest findings.

● Write a clear statement of purpose prior to the pretest.

❍ Make sure that the questions asked relate to the purpose.
❍ Then use the purpose statement to analyze the findings.
● Consider what was found and what was not found—sometimes what is not stated by participants
is very important.
● If the answers to some questions are unclear, consider more or different types of pretesting.
[Table of Contents]
Guidelines for Using Pretest Results

Qualitative research can be predictive, but it is not precise. Interpreting and applying the results of
pretesting and materials review will require certain actions.
● Give more weight to target audience and reviewers' responses that are within the context of their
roles regarding the materials.
● Temper their responses with your own professional judgment, while considering the lay public as
respected equals with a right to the information.
● Do not feel obligated to respond to every individual's every comment or complaint.
● Remember that gatekeepers can block target audience access to print materials, so their opinions
are important too.
[Table of Contents]
Evaluating Communications To Special Populations

Formative research findings may reveal special communications needs of the audiences you're trying to
reach with prevention information and education. For example, it may be important to tailor content,
layout, and use of visuals to the needs of a reader with poor reading and communication skills. What
does this mean in concrete terms? Low-literacy experts have identified key principles for developing
effective materials for this audience. These principles are summarized in the checklist below. You can
use this list as you are developing a new publication and doublechecking product drafts.
Checklist: Key Principles of Effective Low-Literacy Print Materials

(Gatson and Daniels 1988)
Content/Style

___ The material is interactive and allows for audience involvement.

___ The material presents "how-to" information.
___ Peer language is used whenever appropriate to increase personal identification and improve
readability.
___ Words are familiar to the reader. Any new words are defined clearly.
___ Sentences are simple, specific, direct, and written in the active voice.
___ Each idea is clear and logically sequenced (according to audience logic).
___ The number of concepts is limited per piece.
___ The material uses concrete examples rather than abstract concepts.
___ The text highlights and summarizes important points.
Layout
___ The material uses advance organizers and headers.

___ Headers use simple and close to text.
___ Layout balances white space with words and illustrations.
___ Text uses upper and lower case letters.
___ Underlining or bolding rather than caps give emphasis.
___ Type style and size of print are easy-to-read; type is at least 12 point.
Visuals
___ Visuals are relevant to text, meaningful to the audience, and appropriately located.
___ Illustrations and photographs are simple and free from clutter and distraction.
___ Visuals use adult rather than childlike images.
___ Illustrations show familiar images that reflect cultural context.
___ Visuals have captions. Each visual illustrates and is directly related to one message.
___ Different illustration styles, such as photographs, shaded line drawings, and simple line drawings,
are pretested with the audience to determine which is understood best.
___ Cues, such as circles or arrows, point out key information.
___ Colors used are appealing to the audience (as determined by pretesting).
Readability
___ Readability analysis is done to determine reading level.
[Table of Contents]
Evaluation Tool: Risk Message Checklist

(National Research Council 1989)
Information About the Nature of Risks
1. What are the hazards of concern?

2. What is the probability of exposure to each hazard?
3. What is the distribution of exposure?
4. What is the probability of each type of harm from a given exposure to each hazard?
5. What are the sensitivities of different populations to each hazard?
6. How do exposures interact with exposures to other hazards?
7. What are the qualities of the hazard?
8. What is the total population at risk?
Information About the Nature of Benefits
1. What are the benefits associated with the hazard?

2. What is the probability that the projected benefit will actually follow the activity in question?
3. What are the qualities of the benefits?
4. Who benefits and in what ways?
5. How many people benefit and how long do benefits last?
6. Which groups get a disproportionate share of the benefits?
7. What is the total benefit?
Information on Alternatives
1. What are the alternatives to the hazard in question?

2. What is the effectiveness of each alternative?
3. What are the risks and benefits of alternative actions and of not acting?
4. What are the costs and benefits of each alternative and how are they distributed?
Uncertainties in Knowledge About Risks
1. What are the weaknesses of available data?

2. What are the assumptions on which estimates are used?
3. How sensitive are the estimates to changes in assumptions?
4. How sensitive is the decision to changes in the estimate?
5. What other risk and risk control assessments have been made, and why are they different from
those being offered?
Information Management
1. Who is responsible for the decision?

2. What issues have legal importance?

3. What constrains the decision?

4. What resources are available?
[Table of Contents]
Risk Communication Program Assessment Questions

(Arkin 1991)
How many people were reached? (process evaluation)
● Amount of time on radio and television and estimated audience for those times
● Print coverage and estimated readership
● Number of educational materials distributed
● Number of speeches and presentations and size of audience
● Number of other organizational and personal contacts
Did they respond? (process evaluation)
● Number of in-person, telephone, and mail inquiries (location of person inquiring, where inquirer
heard of the program, and what was asked)
● Number of new organizations, businesses, and media outlets participating in the program
● Response from presentations (e.g., filled-out evaluation forms)
Who responded? (outcome evaluation)
● Demographics of responders (e.g., gender, education, income, and geographic residence)
Was there change? (outcome evaluation)
● Changes in knowledge and attitudes

● Changes in intentions (e.g., intentions to modify diet)
● Actions taken (e.g., increased enrollment in smoking cessation clinics)
● Policies initiated or other institutional changes made
[Table of Contents]

Evaluation Options Based on Available Resources

(NCI 1992)
Note: The following chart uses an additive matrix from left to right (i.e., each ascending program
level could be expected to include the evaluation technique described at lower levels in addition to
those described at the higher level).
Program Levels
Types of Minimal Modest

Substantial
Evaluation Resources Resources
Resources
Formative Readability test Central location

Focus groups
Intercept interviews
Individual indepth interviews
Process Recordkeeping Program checklist

Management audit
(e.g., monitoring (e.g., review of adherence
(e.g., external management
activity timetables) to programs plans)
review of activities)
Outcome Activity assessments Progress in attaining

Assessment of target
(e.g., numbers of objectives monitored
audience for knowledge
health screenings (e.g., periodic calculation
gain (e.g., pretest and
and outcomes or of percentage of aware,
posttest of change in
program attendance referred, participating)
knowledge)
and audience response)

Impact Print media review Public surveys

Studies of public
(e.g., monitoring of (e.g., telephone
behavior/health
content of articles surveys of self-
change (e.g., data on
appearing in reported behavior)
physician visits or changes
newspapers)
in public's health status)
[Table of Contents]
Factors To Consider in Midcourse Reviews

(Lum 1991)
In designing and implementing a midcourse review for risk communication program activities, several
factors should be considered.
● Is the process of implementation formal or informal?

● Is control centralized or decentralized?
● Is management authoritarian or participatory?
● Is the program structure hierarchical or egalitarian?
● Is the community divisive or cohesive?
● Is the program isolated or community oriented?
● Are the methods of communication standardized or individualized?
● Is response and interaction controlled or expressive?
● Are strategies partitioned or integrated?
[Table of Contents]
Have We Succeeded?
(Chess and Hance 1992)
To help make adjustments, duplicate successes, and avoid repeating failures, you should plan for

feedback and evaluation of your communication. This might include taking the following actions:
● pretesting of materials before they are printed or distributed

● handing out evaluation forms to get feedback after meetings
● debriefing agency staff after meetings or other interactions
● tracking news coverage
● asking for feedback from key people outside the agency on written
● materials, meeting agendas, and communication plans
● documenting communication efforts through up-to-date files and summaries
● summarizing lessons learned from communication efforts
● conducting rigorous evaluations on large-scale efforts.
[Table of Contents]
Evaluation Case Example:

Communicating the NCI Mammography Screening Guidelines:
The Practicing Physician's Perspective
(NCI 1994)
Background
Before announcing its decision to drop the mammography guideline for women under age 50, the Office
of Cancer Communications (OCC) at the National Cancer Institute (NCI) conducted a series of
seven focus groups with obstetrician/gynecologists (ob/gyns) and primary care physicians. The purpose
of the study was to (1) assess the likely impact of a change in the NCI mammography screening
guidelines on physician decision making with regard to mammogram referrals, and (2) explore ways that
NCI can best support physicians in their use of the new guidelines and in their communications with
patients. Obtaining a better understanding of the environment in which physicians practice, including
their attitudes and concerns about the new guidelines, was considered a prerequisite for developing
effective communication materials.
Evaluation Approach
To achieve diversity in the seven focus groups, physicians were recruited based on a number of different
variables: their practice setting (i.e., managed care and fee-for-service), length of time in practice
(minimum of 5 years), patient base (minorities, low income, etc.) and sex. Physicians who had
mammography equipment in their offices were excluded from participation in the study. Three of the
groups were composed of full-time practicing ob/gyns or gyns; four groups were composed of primary

care physicians who were either family physicians or internists. Fifty-five physicians participated;
approximately one-fourth were members of a minority group.
The focus groups were convened in four locations: Baltimore, Maryland; Chicago, Illinois; Omaha,
Nebraska; and Newark (Wyckoff), New Jersey. Overall, these sites were selected for their geographic,
ethnic, racial, and socioeconomic diversity. During the focus groups, participants were asked to react to a
number of sample print materials designed to support the introduction of the revised guidelines.
Findings and Lessons Learned
● Physicians have a strong sense of responsibility with regard to screening patients for breast
cancer. This reflects the high level of concern they see in their patients; in part, it reflects their
concerns about liability. However, their own commitment to fighting a deadly disease has
touched them professionally and in some cases personally.
● Physicians firmly believe that mammography is the only effective means of detecting breast
cancer at its earliest possible stage. Moreover, physicians are convinced that early detection is the
single greatest factor influencing survival.
● Physicians prefer to take a conservative approach with respect to mammography. They follow the
most stringent guidelines in screening low-risk patients. For high-risk patients, they order
mammograms earlier and more frequently than the guidelines specify, as is evident from their
desire for more explicit recommendations for high-risk women. Physicians do not like operating
"in the gray zone" where they must rely primarily on self-judgement.
● NCI's guidelines are largely unknown to physicians, compared with those of the American Cancer
Society (ACS) and the American College of Obstetricians and Gynecologists
(ACOG). Lack of knowledge and familiarity with NCI may be responsible in part for
physicians' questioning whether the guideline change is politically or scientifically motivated.
● Most physicians are highly resistant to decreasing their use of mammography, particularly in
women under 50 years old. They indicate they would need strong, compelling evidence that
mammography is either useless or harmful before they would change the way they currently
practice. The scientific evidence, as presented to them, is seen as inconclusive; it does not
convince them that a change in the guidelines is warranted at this time.
● It is important to physicians that their professional organizations endorse any change that takes
place in the recommendations for mammography screening. Ob/gyns in particular are unlikely to
support a change unless ACOG endorses it.
● Overall, the findings from the study suggest that a majority of physicians will simply ignore the
new NCI guidelines and continue to practice as usual. As some physicians have realized, they can
follow their current practices and still be in compliance with the new guidelines.
● Given that the communications environment is characterized by skepticism and resistance, it is
likely that any communications program designed to introduce and support the guidelines will
face stiff challenges. The sample materials presented in this study were largely unsuccessful
because they could not overcome physicians' objections to the revised recommendation. New
strategies based on what was learned about physicians as consumers in this study are now being
developed.

[Table of Contents]
Evaluation Action Plan

Purpose
The purpose of an evaluation action plan is to assist you in identifying specific evaluation strategies for
assessing how well your messages, materials, and activities were implemented and received by the target
audiences.
Instructions
Ideas for Actions. Write down ideas and strategies that you might want to try in evaluating your own
health risk communication program.
Action Plan. Choose two or three of your evaluation ideas and develop specific actions to implement
them. Write these ideas on an action plan form. When developing your action plan, ask yourself the
following questions:
Planning
● Is the action specific (Are action verbs used)?

● How much time will it take?
● Is it written so that I or someone else will know when the action occurs?
Resources
● Are the skills and resources for carrying out the action available?
● Who will do it?
● Are special materials or equipment necessary?
● Is continual monitoring or followup required?
Implementation
● Do I have the authority to implement the action? If not, who does?

● How can I go about getting approval?
● What is the degree of support for my idea?
● Besides my supervisor, who will I need to sell my idea to?
● What in my organization environment might interfere with doing this?

Evaluation
● Whom will this action affect?

● How will it affect them?
● How will I measure the effects? (evaluation plan)
[Table of Contents]
Attributes of an Effective Risk Communication Program

(Covello, Slovic, and von Winterfeldt 1987)
When designing and evaluating communication programs' effectiveness, the following maxims may be
helpful.
● The communication should be clear, understandable, informative, accurate, and concrete.

● The source of the communication should be perceived as credible and reliable.
● When the target population is not homogeneous, the message should be presented in several
ways, each specifically designed for one segment of the target population.
● Whenever possible, the target population or representatives of it should be closely involved in the
planning and implementation of the program. The earlier the involvement, the better. This advice
particularly refers to occupational hazards, where the involvement of unions is considered
essential.
● Multimode presentation is considered to be more effective than single mode presentation. This
applies both to mass-media programs and to programs designed for smaller target populations. In
the latter situation, face-to-face, two-way communication is also advocated.
● Feedback about the behavior change and its consequences in lowering risk is highly effective, and
should be used whenever possible.
● Incentives or rewards are thought to be effective in inducing change.
● Repetitions of the message are desirable to a point; too many repetitions are ineffective or even
deleterious. The optimal number of repetitions is not known.
● Certainly the message should be interesting; vividness has been shown to aid learning. But how
arousing should the material be? Research results are mixed concerning the effectiveness of fear
arousal.
[Table of Contents]



Selected References
Arkin E. 1991. Evaluation for risk communicators. In: Ann Fisher, Maria Pavolva, and Vincent
Covello, editors. Evaluation and effective risk communications workshop proceedings. Washington,
DC: U.S. Environmental Protection Agency, Pub. no. EPA/600/9-90/054, pgs. 17-18.
Arkin E. 1992. Health communications for consumers: the role of the federal government. Testimony
before the Subcommittee on Housing and Consumer Interests, House Select Committee on Aging.
Washington, DC, August 5, 1992.
Centers for Disease Control and Prevention (CDC). 1993. Health communication at CDC—CDC's
health communication wheel. Atlanta: U.S. Department of Health and Human Services, Public Health
Service.
Academy for Educational Development; Johns Hopkins University; Porter/Novelli (for National AIDS
Information and Education Program, Centers for Disease Control and Prevention). 1993. A workshop in
developing effective educational print materials, April 1993. Washington, DC: Academy for
Educational Development; John Hopkins University; Porter/Novelli.
Chess C and Hance BJ. 1992. Communicating with the public: ten questions environmental mangers
should ask. New Jersey: Center for Environmental Communication, Rutgers University.
Chess C and McCallum DB. 1992. Needs assessment for ATSDR risk communication focus groups.
Atlanta: U.S. Department of Health and Human Services, Public Health Service.
Covello VT, McCallum DB, and Pavolva MT. 1989. Principles and guidelines for improving risk
communication. In: Effective risk communication: the role and responsibility of government and
nongovernment organizations. New York: Plenum Press.
Covello VT, Slovic P, and von Winterfeldt D. 1987. Risk communication: a review of the literature.
Draft, pgs. 5, 58-9.
Gatson N and Daniels P. 1988. Guidelines: writing for adults with limited reading skills. Washington,
DC: U.S. Department of Agriculture, Food and Nutrition Service.
Lum M. 1991. Benefits to conducting midcourse reviews. In: Ann Fisher, Maria Pavolva, and Vincent

Covello, editors. Evaluation and effective risk communications workshop proceedings. Washington,
DC: U.S. Environmental Protection Agency, Pub. no. EPA/600/9-90/054, pg. 97.
National Cancer Institute. 1992. Making health communication programs work: a planner's guide.
Washington, DC: National Cancer Institute, NIH Publication no. 92-1493, pgs. 64-65.
National Cancer Institute. 1994. Communicating the NCI mammography screening guidelines: the
practicing physician's perspective. In: Tim Tinker, editor. Case studies of applied evaluation for health
risk communication (workshop proceedings). Washington, DC: U.S. Department of Health and Human
Services, Public Health Service, pgs. 21-22.
National Research Council. 1989. Improving risk communication. National Academy Press,
Washington, DC, pg. 175.
Regan MJ and Desvousges WH. 1990. Communicating environmental risks: a guide to practical
evaluations. Washington, DC: U.S. Environmental Protection Agency, Pub. no. 230-01-91-001, pgs. 2-
3.
U.S. Department of Health and Human Services (DHHS). 1986. Determining risks to health: federal
policy and practice. Dover, Massachusetts: Auburn House Publishing Company.
[Table of Contents]


ATSDR - Glossary of Terms
CONTENTS ATSDR Glossary of Terms

En Español
The Agency for Toxic Substances and Disease Registry (ATSDR) is a federal
General Terms public health agency with headquarters in Atlanta, Georgia, and 10 regional
·ABC offices in the United States. ATSDR's mission is to serve the public by using the
·DEF best science, taking responsive public health actions, and providing trusted health
·GHI information to prevent harmful exposures and diseases related to toxic
·JKL substances. ATSDR is not a regulatory agency, unlike the U.S. Environmental
·MNO Protection Agency (EPA), which is the federal agency that develops and enforces
·PQRS environmental laws to protect the environment and human health.
·TUV
·WXYZ This glossary defines words used by ATSDR in communications with the public.
It is not a complete dictionary of environmental health terms. If you have
questions or comments, call ATSDR's toll-free telephone number,
1-888-422-8737.
General Terms
Absorption
The process of taking in. For a person or an animal, absorption is the process of a
substance getting into the body through the eyes, skin, stomach, intestines, or
lungs.
Acute
Occurring over a short time [compare with chronic].
Acute exposure
Contact with a substance that occurs once or for only a short time (up to 14 days)
[compare with intermediate duration exposure and chronic exposure].
Additive effect
A biologic response to exposure to multiple substances that equals the sum of
responses of all the individual substances added together [compare with
antagonistic effect and synergistic effect].
Adverse health effect

A change in body function or cell structure that might lead to disease or health
problems
http://www.atsdr.cdc.gov/glossary.html (1 of 17)13/11/2004 17:57:45

Aerobic
Requiring oxygen [compare with anaerobic].
Ambient
Surrounding (for example, ambient air).
Anaerobic
Requiring the absence of oxygen [compare with aerobic].
Analyte
A substance measured in the laboratory. A chemical for which a sample (such as
water, air, or blood) is tested in a laboratory. For example, if the analyte is
mercury, the laboratory test will determine the amount of mercury in the sample.
Analytic epidemiologic study

A study that evaluates the association between exposure to hazardous substances
and disease by testing scientific hypotheses.
Antagonistic effect
A biologic response to exposure to multiple substances that is less than would be
expected if the known effects of the individual substances were added together
[compare with additive effect and synergistic effect].
Background level
An average or expected amount of a substance or radioactive material in a
specific environment, or typical amounts of substances that occur naturally in an
environment.
Biodegradation
Decomposition or breakdown of a substance through the action of
microorganisms (such as bacteria or fungi) or other natural physical processes
(such as sunlight).
Biologic indicators of exposure study

A study that uses (a) biomedical testing or (b) the measurement of a substance
[an analyte], its metabolite, or another marker of exposure in human body fluids
or tissues to confirm human exposure to a hazardous substance [also see
exposure investigation].
Biologic monitoring
Measuring hazardous substances in biologic materials (such as blood, hair, urine,
or breath) to determine whether exposure has occurred. A blood test for lead is
an example of biologic monitoring.
Biologic uptake

The transfer of substances from the environment to plants, animals, and humans.
Biomedical testing
Testing of persons to find out whether a change in a body function might have
occurred because of exposure to a hazardous substance.
Biota
Plants and animals in an environment. Some of these plants and animals might be
sources of food, clothing, or medicines for people.
Body burden
The total amount of a substance in the body. Some substances build up in the
body because they are stored in fat or bone or because they leave the body very
slowly.
CAP [see Community Assistance Panel.]
Cancer
Any one of a group of diseases that occur when cells in the body become
abnormal and grow or multiply out of control.
Cancer risk
A theoretical risk for getting cancer if exposed to a substance every day for 70
years (a lifetime exposure). The true risk might be lower.
Carcinogen
A substance that causes cancer.
Case study
A medical or epidemiologic evaluation of one person or a small group of people
to gather information about specific health conditions and past exposures.
Case-control study
A study that compares exposures of people who have a disease or condition
(cases) with people who do not have the disease or condition (controls).
Exposures that are more common among the cases may be considered as possible
risk factors for the disease.
CAS registry number

A unique number assigned to a substance or mixture by the American Chemical
Society Abstracts Service .
Central nervous system

The part of the nervous system that consists of the brain and the spinal cord.

CERCLA [see Comprehensive Environmental Response, Compensation, and

Liability Act of 1980]
Chronic
Occurring over a long time [compare with acute].
Chronic exposure
Contact with a substance that occurs over a long time (more than 1 year)
[compare with acute exposure and intermediate duration exposure]
Cluster investigation
A review of an unusual number, real or perceived, of health events (for example,
reports of cancer) grouped together in time and location. Cluster investigations
are designed to confirm case reports; determine whether they represent an
unusual disease occurrence; and, if possible, explore possible causes and
contributing environmental factors.
Community Assistance Panel (CAP)

A group of people from a community and from health and environmental
agencies who work with ATSDR to resolve issues and problems related to
hazardous substances in the community. CAP members work with ATSDR to
gather and review community health concerns, provide information on how
people might have been or might now be exposed to hazardous substances, and
inform ATSDR on ways to involve the community in its activities.
Comparison value (CV)

Calculated concentration of a substance in air, water, food, or soil that is unlikely
to cause harmful (adverse) health effects in exposed people. The CV is used as a
screening level during the public health assessment process. Substances found in
amounts greater than their CVs might be selected for further evaluation in the
public health assessment process.
Completed exposure pathway [see exposure pathway].
Comprehensive Environmental Response, Compensation, and Liability Act

of 1980 (CERCLA)
CERCLA, also known as Superfund, is the federal law that concerns the removal
or cleanup of hazardous substances in the environment and at hazardous waste
sites. ATSDR, which was created by CERCLA, is responsible for assessing
health issues and supporting public health activities related to hazardous waste
sites or other environmental releases of hazardous substances. This law was later
amended by the Superfund Amendments and Reauthorization Act (SARA).
Concentration
The amount of a substance present in a certain amount of soil, water, air, food,

blood, hair, urine, breath, or any other media.
Contaminant
A substance that is either present in an environment where it does not belong or
is present at levels that might cause harmful (adverse) health effects.
Delayed health effect

A disease or an injury that happens as a result of exposures that might have
occurred in the past.
Dermal
Referring to the skin. For example, dermal absorption means passing through the
skin.
Dermal contact
Contact with (touching) the skin [see route of exposure].
Descriptive epidemiology
The study of the amount and distribution of a disease in a specified population by
person, place, and time.
Detection limit
The lowest concentration of a chemical that can reliably be distinguished from a
zero concentration.
Disease prevention
Measures used to prevent a disease or reduce its severity.
Disease registry
A system of ongoing registration of all cases of a particular disease or health
condition in a defined population.
DOD
United States Department of Defense.
DOE
United States Department of Energy.
Dose (for chemicals that are not radioactive)

The amount of a substance to which a person is exposed over some time period.
Dose is a measurement of exposure. Dose is often expressed as milligram
(amount) per kilogram (a measure of body weight) per day (a measure of time)
when people eat or drink contaminated water, food, or soil. In general, the
greater the dose, the greater the likelihood of an effect. An "exposure dose" is
how much of a substance is encountered in the environment. An "absorbed dose"

is the amount of a substance that actually got into the body through the eyes,
skin, stomach, intestines, or lungs.
Dose (for radioactive chemicals)

The radiation dose is the amount of energy from radiation that is actually
absorbed by the body. This is not the same as measurements of the amount of
radiation in the environment.
Dose-response relationship
The relationship between the amount of exposure [dose] to a substance and the
resulting changes in body function or health (response).
Environmental media
Soil, water, air, biota (plants and animals), or any other parts of the environment
that can contain contaminants.
Environmental media and transport mechanism

Environmental media include water, air, soil, and biota (plants and animals).
Transport mechanisms move contaminants from the source to points where
human exposure can occur. The environmental media and transport mechanism
is the second part of an exposure pathway.
EPA
United States Environmental Protection Agency.
Epidemiologic surveillance [see Public health surveillance].
Epidemiology
The study of the distribution and determinants of disease or health status in a
population; the study of the occurrence and causes of health effects in humans.
Exposure
Contact with a substance by swallowing, breathing, or touching the skin or eyes.
Exposure may be short-term [acute exposure], of intermediate duration, or long-
term [chronic exposure].
Exposure assessment
The process of finding out how people come into contact with a hazardous
substance, how often and for how long they are in contact with the substance,
and how much of the substance they are in contact with.
Exposure-dose reconstruction
A method of estimating the amount of people's past exposure to hazardous
substances. Computer and approximation methods are used when past
information is limited, not available, or missing.

Exposure investigation
The collection and analysis of site-specific information and biologic tests (when
appropriate) to determine whether people have been exposed to hazardous
substances.
Exposure pathway
The route a substance takes from its source (where it began) to its end point
(where it ends), and how people can come into contact with (or get exposed to)
it. An exposure pathway has five parts: a source of contamination (such as an
abandoned business); an environmental media and transport mechanism (such as
movement through groundwater); a point of exposure (such as a private well); a
route of exposure (eating, drinking, breathing, or touching), and a receptor
population (people potentially or actually exposed). When all five parts are
present, the exposure pathway is termed a completed exposure pathway.
Exposure registry
A system of ongoing followup of people who have had documented
environmental exposures.
Feasibility study
A study by EPA to determine the best way to clean up environmental
contamination. A number of factors are considered, including health risk, costs,
and what methods will work well.
Geographic information system (GIS)

A mapping system that uses computers to collect, store, manipulate, analyze, and
display data. For example, GIS can show the concentration of a contaminant
within a community in relation to points of reference such as streets and homes.
Grand rounds
Training sessions for physicians and other health care providers about health
topics.
Groundwater
Water beneath the earth's surface in the spaces between soil particles and
between rock surfaces [compare with surface water].
Half-life (t½)
The time it takes for half the original amount of a substance to disappear. In the
environment, the half-life is the time it takes for half the original amount of a
substance to disappear when it is changed to another chemical by bacteria, fungi,
sunlight, or other chemical processes. In the human body, the half-life is the time
it takes for half the original amount of the substance to disappear, either by being
changed to another substance or by leaving the body. In the case of radioactive

material, the half life is the amount of time necessary for one half the initial
number of radioactive atoms to change or transform into another atom (that is
normally not radioactive). After two half lives, 25% of the original number of
radioactive atoms remain.
Hazard
A source of potential harm from past, current, or future exposures.
Hazardous Substance Release and Health Effects Database (HazDat)

The scientific and administrative database system developed by ATSDR to
manage data collection, retrieval, and analysis of site-specific information on
hazardous substances, community health concerns, and public health activities.
Hazardous waste
Potentially harmful substances that have been released or discarded into the
environment.
Health consultation
A review of available information or collection of new data to respond to a
specific health question or request for information about a potential
environmental hazard. Health consultations are focused on a specific exposure
issue. Health consultations are therefore more limited than a public health
assessment, which reviews the exposure potential of each pathway and chemical
[compare with public health assessment].
Health education
Programs designed with a community to help it know about health risks and how
to reduce these risks.
Health investigation
The collection and evaluation of information about the health of community
residents. This information is used to describe or count the occurrence of a
disease, symptom, or clinical measure and to evaluate the possible association
between the occurrence and exposure to hazardous substances.
Health promotion
The process of enabling people to increase control over, and to improve, their
health.
Health statistics review

The analysis of existing health information (i.e., from death certificates, birth
defects registries, and cancer registries) to determine if there is excess disease in
a specific population, geographic area, and time period. A health statistics review
is a descriptive epidemiologic study.

Indeterminate public health hazard

The category used in ATSDR's public health assessment documents when a
professional judgment about the level of health hazard cannot be made because
information critical to such a decision is lacking.
Incidence
The number of new cases of disease in a defined population over a specific time
period [contrast with prevalence].
Ingestion
The act of swallowing something through eating, drinking, or mouthing objects.
A hazardous substance can enter the body this way [see route of exposure].
Inhalation
The act of breathing. A hazardous substance can enter the body this way [see
route of exposure].
Intermediate duration exposure

Contact with a substance that occurs for more than 14 days and less than a year
[compare with acute exposure and chronic exposure].
In vitro
In an artificial environment outside a living organism or body. For example,
some toxicity testing is done on cell cultures or slices of tissue grown in the
laboratory, rather than on a living animal [compare with in vivo].
In vivo
Within a living organism or body. For example, some toxicity testing is done on
whole animals, such as rats or mice [compare with in vitro].
Lowest-observed-adverse-effect level (LOAEL)

The lowest tested dose of a substance that has been reported to cause harmful
(adverse) health effects in people or animals.
Medical monitoring
A set of medical tests and physical exams specifically designed to evaluate
whether an individual's exposure could negatively affect that person's health.
Metabolism
The conversion or breakdown of a substance from one form to another by a
living organism.
Metabolite
Any product of metabolism.

mg/kg
Milligram per kilogram.
mg/cm2
Milligram per square centimeter (of a surface).
mg/m3
Milligram per cubic meter; a measure of the concentration of a chemical in a
known volume (a cubic meter) of air, soil, or water.
Migration
Moving from one location to another.
Minimal risk level (MRL)

An ATSDR estimate of daily human exposure to a hazardous substance at or
below which that substance is unlikely to pose a measurable risk of harmful
(adverse), noncancerous effects. MRLs are calculated for a route of exposure
(inhalation or oral) over a specified time period (acute, intermediate, or chronic).
MRLs should not be used as predictors of harmful (adverse) health effects [see
reference dose].
Morbidity
State of being ill or diseased. Morbidity is the occurrence of a disease or
condition that alters health and quality of life.
Mortality
Death. Usually the cause (a specific disease, a condition, or an injury) is stated.
Mutagen
A substance that causes mutations (genetic damage).
Mutation
A change (damage) to the DNA, genes, or chromosomes of living organisms.
National Priorities List for Uncontrolled Hazardous Waste Sites (National

Priorities List or NPL)
EPA's list of the most serious uncontrolled or abandoned hazardous waste sites in
the United States. The NPL is updated on a regular basis.
National Toxicology Program (NTP)

Part of the Department of Health and Human Services. NTP develops and carries
out tests to predict whether a chemical will cause harm to humans.
No apparent public health hazard

A category used in ATSDR's public health assessments for sites where human

exposure to contaminated media might be occurring, might have occurred in the

past, or might occur in the future, but where the exposure is not expected to
cause any harmful health effects.
No-observed-adverse-effect level (NOAEL)

The highest tested dose of a substance that has been reported to have no harmful
(adverse) health effects on people or animals.
No public health hazard

A category used in ATSDR's public health assessment documents for sites where
people have never and will never come into contact with harmful amounts of site-
related substances.
NPL [see National Priorities List for Uncontrolled Hazardous Waste Sites]
Physiologically based pharmacokinetic model (PBPK model)

A computer model that describes what happens to a chemical in the body. This
model describes how the chemical gets into the body, where it goes in the body,
how it is changed by the body, and how it leaves the body.
Pica
A craving to eat nonfood items, such as dirt, paint chips, and clay. Some children
exhibit pica-related behavior.
Plume
A volume of a substance that moves from its source to places farther away from
the source. Plumes can be described by the volume of air or water they occupy
and the direction they move. For example, a plume can be a column of smoke
from a chimney or a substance moving with groundwater.
Point of exposure
The place where someone can come into contact with a substance present in the
environment [see exposure pathway].
Population
A group or number of people living within a specified area or sharing similar
characteristics (such as occupation or age).
Potentially responsible party (PRP)

A company, government, or person legally responsible for cleaning up the
pollution at a hazardous waste site under Superfund. There may be more than one
PRP for a particular site.
ppb
Parts per billion.

ppm
Parts per million.
Prevalence
The number of existing disease cases in a defined population during a specific
time period [contrast with incidence].
Prevalence survey
The measure of the current level of disease(s) or symptoms and exposures
through a questionnaire that collects self-reported information from a defined
population.
Prevention
Actions that reduce exposure or other risks, keep people from getting sick, or
keep disease from getting worse.
Public availability session

An informal, drop-by meeting at which community members can meet one-on-
one with ATSDR staff members to discuss health and site-related concerns.
Public comment period

An opportunity for the public to comment on agency findings or proposed
activities contained in draft reports or documents. The public comment period is
a limited time period during which comments will be accepted.
Public health action

A list of steps to protect public health.
Public health advisory

A statement made by ATSDR to EPA or a state regulatory agency that a release
of hazardous substances poses an immediate threat to human health. The
advisory includes recommended measures to reduce exposure and reduce the
threat to human health.
Public health assessment (PHA)

An ATSDR document that examines hazardous substances, health outcomes, and
community concerns at a hazardous waste site to determine whether people could
be harmed from coming into contact with those substances. The PHA also lists
actions that need to be taken to protect public health [compare with health
consultation].
Public health hazard

A category used in ATSDR's public health assessments for sites that pose a
public health hazard because of long-term exposures (greater than 1 year) to

sufficiently high levels of hazardous substances or radionuclides that could result

in harmful health effects.
Public health hazard categories

Public health hazard categories are statements about whether people could be
harmed by conditions present at the site in the past, present, or future. One or
more hazard categories might be appropriate for each site. The five public health
hazard categories are no public health hazard, no apparent public health hazard,
indeterminate public health hazard, public health hazard, and urgent public health
hazard.
Public health statement

The first chapter of an ATSDR toxicological profile. The public health statement
is a summary written in words that are easy to understand. The public health
statement explains how people might be exposed to a specific substance and
describes the known health effects of that substance.
Public health surveillance

The ongoing, systematic collection, analysis, and interpretation of health data.
This activity also involves timely dissemination of the data and use for public
health programs.
Public meeting
A public forum with community members for communication about a site.
Radioisotope
An unstable or radioactive isotope (form) of an element that can change into
another element by giving off radiation.
Radionuclide
Any radioactive isotope (form) of any element.
RCRA [see Resource Conservation and Recovery Act (1976, 1984)]
Receptor population
People who could come into contact with hazardous substances [see exposure
pathway].
Reference dose (RfD)

An EPA estimate, with uncertainty or safety factors built in, of the daily lifetime
dose of a substance that is unlikely to cause harm in humans.
Registry
A systematic collection of information on persons exposed to a specific

substance or having specific diseases [see exposure registry and disease registry].
Remedial investigation
The CERCLA process of determining the type and extent of hazardous material
contamination at a site.
Resource Conservation and Recovery Act (1976, 1984) (RCRA)

This Act regulates management and disposal of hazardous wastes currently
generated, treated, stored, disposed of, or distributed.
RFA
RCRA Facility Assessment. An assessment required by RCRA to identify
potential and actual releases of hazardous chemicals.
RfD [see reference dose]
Risk
The probability that something will cause injury or harm.
Risk reduction
Actions that can decrease the likelihood that individuals, groups, or communities
will experience disease or other health conditions.
Risk communication
The exchange of information to increase understanding of health risks.
Route of exposure
The way people come into contact with a hazardous substance. Three routes of
exposure are breathing [inhalation], eating or drinking [ingestion], or contact
with the skin [dermal contact].
Safety factor [see uncertainty factor]
SARA [see Superfund Amendments and Reauthorization Act]
Sample
A portion or piece of a whole. A selected subset of a population or subset of
whatever is being studied. For example, in a study of people the sample is a
number of people chosen from a larger population [see population]. An
environmental sample (for example, a small amount of soil or water) might be
collected to measure contamination in the environment at a specific location.
Sample size
The number of units chosen from a population or an environment.

Solvent
A liquid capable of dissolving or dispersing another substance (for example,
acetone or mineral spirits).
Source of contamination
The place where a hazardous substance comes from, such as a landfill, waste
pond, incinerator, storage tank, or drum. A source of contamination is the first
part of an exposure pathway.
Special populations
People who might be more sensitive or susceptible to exposure to hazardous
substances because of factors such as age, occupation, sex, or behaviors (for
example, cigarette smoking). Children, pregnant women, and older people are
often considered special populations.
Stakeholder
A person, group, or community who has an interest in activities at a hazardous
waste site.
Statistics
A branch of mathematics that deals with collecting, reviewing, summarizing, and
interpreting data or information. Statistics are used to determine whether
differences between study groups are meaningful.
Substance
A chemical.
Substance-specific applied research

A program of research designed to fill important data needs for specific
hazardous substances identified in ATSDR's toxicological profiles. Filling these
data needs would allow more accurate assessment of human risks from specific
substances contaminating the environment. This research might include human
studies or laboratory experiments to determine health effects resulting from
exposure to a given hazardous substance.
Superfund [see Comprehensive Environmental Response, Compensation, and

Liability Act of 1980 (CERCLA) and Superfund Amendments and
Reauthorization Act (SARA)
Superfund Amendments and Reauthorization Act (SARA)

In 1986, SARA amended the Comprehensive Environmental Response,
Compensation, and Liability Act of 1980 (CERCLA) and expanded the health-
related responsibilities of ATSDR. CERCLA and SARA direct ATSDR to look
into the health effects from substance exposures at hazardous waste sites and to

perform activities including health education, health studies, surveillance, health

consultations, and toxicological profiles.
Surface water
Water on the surface of the earth, such as in lakes, rivers, streams, ponds, and
springs [compare with groundwater].
Surveillance [see public health surveillance]
Survey
A systematic collection of information or data. A survey can be conducted to
collect information from a group of people or from the environment. Surveys of a
group of people can be conducted by telephone, by mail, or in person. Some
surveys are done by interviewing a group of people [see prevalence survey].
Synergistic effect
A biologic response to multiple substances where one substance worsens the
effect of another substance. The combined effect of the substances acting
together is greater than the sum of the effects of the substances acting by
themselves [see additive effect and antagonistic effect].
Teratogen
A substance that causes defects in development between conception and birth. A
teratogen is a substance that causes a structural or functional birth defect.
Toxic agent
Chemical or physical (for example, radiation, heat, cold, microwaves) agents
that, under certain circumstances of exposure, can cause harmful effects to living
organisms.
Toxicological profile
An ATSDR document that examines, summarizes, and interprets information
about a hazardous substance to determine harmful levels of exposure and
associated health effects. A toxicological profile also identifies significant gaps
in knowledge on the substance and describes areas where further research is
needed.
Toxicology
The study of the harmful effects of substances on humans or animals.
Tumor
An abnormal mass of tissue that results from excessive cell division that is
uncontrolled and progressive. Tumors perform no useful body function. Tumors
can be either benign (not cancer) or malignant (cancer).

Uncertainty factor
Mathematical adjustments for reasons of safety when knowledge is incomplete.
For example, factors used in the calculation of doses that are not harmful
(adverse) to people. These factors are applied to the lowest-observed-adverse-
effect-level (LOAEL) or the no-observed-adverse-effect-level (NOAEL) to
derive a minimal risk level (MRL). Uncertainty factors are used to account for
variations in people's sensitivity, for differences between animals and humans,
and for differences between a LOAEL and a NOAEL. Scientists use uncertainty
factors when they have some, but not all, the information from animal or human
studies to decide whether an exposure will cause harm to people [also sometimes
called a safety factor].
Urgent public health hazard

A category used in ATSDR's public health assessments for sites where short-
term exposures (less than 1 year) to hazardous substances or conditions could
result in harmful health effects that require rapid intervention.
Volatile organic compounds (VOCs)

Organic compounds that evaporate readily into the air. VOCs include substances
such as benzene, toluene, methylene chloride, and methyl chloroform.
Other glossaries and dictionaries:

Environmental Protection Agency
National Library of Medicine (NIH)
For more information on the work of ATSDR, please contact
NCEH/ATSDR Office of Communication, Information Services Center

1600 Clifton Road, N.E. (MS E-29)
Atlanta, GA 30333
Telephone: 1-888-422-8737
Update June 21, 2004

R. Searfoss / Questions? - Call the ATSDR Information Center toll free at 1-888-422-8737, or e-mail


Health Risk Communication Programs Outomes Evaluation Primer - ATSDR USA - 2004

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Health Risk Communication Programs Outomes Evaluation Primer - ATSDR USA - 2004

Încărcat de

Drepturi de autor:

Formate disponibile

ATSDR Evaluation Primer on Health Risk Communication Programs, TOC

Search | Index | Home | Glossary | Contact Us

Agency for Toxic Substances and Disease Registry

Environmental Health Policy Committee

❍ Elements of an Evaluation Design

❍ Risk Communication Objectives as a Measure of Effectiveness

❍ Barriers to Risk Communication Evaluation

● What Do We Want to Say? The Role of Formative Evaluation and Research

❍ Whom To Include in Review and Pretesting

❍ Guidelines for Selecting Pretest Methods

❍ Pretest Methods for Print Materials

❍ Sample Survey for Assessing Risk Communication Needs

❍ Sample Focus Group Case Study

❍ Guidelines for Analyzing Pretest Results

❍ Guidelines for Using Pretest Results

❍ Evaluating Communications To Special Populations

❍ Evaluation Tool: Risk Message Checklist

● Did It Work? Tips for Evaluating Communication Outcomes and Impacts

http://www.atsdr.cdc.gov/HEC/evalprmr.html (1 of 2)13/11/2004 17:54:54

❍Risk Communication Program Assessment Questions

❍ Factors To Consider in Midcourse Reviews

❍ Evaluation Case Example

❍ Evaluation Action Plan

❍ Attributes of an Effective Risk Communication Program

Primer Content Prepared by:

Tim L. Tinker, Dr.P.H., M.P.H.

Paula G. Silberberg, M.Ed.

US DEPARTMENT OF HEALTH AND HUMAN SERVICES

This page last updated on May 28, 2002

Contact Name: Wilma López/ WLópez@cdc.gov

ATSDR Home | Search | Index | Glossary | Contact Us

http://www.atsdr.cdc.gov/HEC/evalprmr.html (2 of 2)13/11/2004 17:54:54

Search | Index | Home | Glossary | Contact Us

http://www.atsdr.cdc.gov/HEC/evalp1.html (1 of 12)13/11/2004 17:55:24

About the Primer

● Identifies performance indicators used in assessing or measuring communication goals. For

1. accept and involve the public as a legitimate partner

http://www.atsdr.cdc.gov/HEC/evalp1.html (2 of 12)13/11/2004 17:55:24

7. speak clearly and with compassion.

Why Evaluate Risk Communication Programs?

Evaluation is a purposeful effort to determine effectiveness. It is essential to communication because it

● To provide evidence of need for additional funds or other resources

http://www.atsdr.cdc.gov/HEC/evalp1.html (3 of 12)13/11/2004 17:55:24

● number of staff working on the project

● changes in knowledge and attitudes

http://www.atsdr.cdc.gov/HEC/evalp1.html (4 of 12)13/11/2004 17:55:24

● changes in morbidity and mortality

Elements of an Evaluation Design

Every formal design—whether formative, process, outcome, impact, or a combination—must contain

1. A Statement of Communication Objectives

2. Definition of Data To Be Collected

http://www.atsdr.cdc.gov/HEC/evalp1.html (5 of 12)13/11/2004 17:55:24

Risk Communication Objectives as a

TYPE 1: Information and Education

TYPE 2: Behavior Change and Protective Action

● Encouraging personal risk-reduction behavior.

EXAMPLE: Using advertisements to encourage people to wear seat belts.

TYPE 3: Disaster Warnings and Emergency Information

● Providing direction and behavioral guidance in disasters and emergencies.

http://www.atsdr.cdc.gov/HEC/evalp1.html (6 of 12)13/11/2004 17:55:24

TYPE 4: Joint Problem Solving and Conflict Resolution

Barriers to Risk Communication Evaluation

These constraints may include the following: