19

Measuring the
Effectiveness of the
Promotional Program

McGraw-Hill/Irwin Copyright 2009 by The McGraw-Hill Companies, Inc. All rights reserved.
Pros and Cons of Measuring Effectiveness

Advantages Disadvantages

Avoid costly mistakes Cost of measurement

Evaluate alternative Research problems

strategies

Increase efficiency in Disagreement on

general what to test

Determine if objectives are Objections of creatives

achieved

Time
Evaluating Alternative Strategies
Measuring Effectiveness: Marketers Views

Issue Agree
MPM is a key priority for todays 90%
technology companies.

I am dissatisfied with our ability to 80%

demonstrate marketing programs
business impact and value.
It is important to define, measure, and 61.5%
take concrete steps in the area of
advertising accountability.
I can forecast the impact on sales of a 37%
10% cut in marketing spending.
 Measuring Advertising Effectiveness

What to test Where to test

Source factors Laboratory tests
Message variables Field tests
Media strategies
Budget decisions

How to test When to test

Testing guidelines Pretesting
Appropriate tests Posttesting
 Pretesting Methods

Laboratory Field
Consumer Juries Dummy Ad Vehicles

Portfolio Tests On-air Tests

Physiological Measures

Theater Tests

Rough Tests

Concept Tests

Readability Tests
Comprehension and
Reaction Tests
 Posttesting Methods

Recall Tests

Tracking Association
Studies Measures

Methods
Single-
Recognition
Source
Tests
Systems

Inquiry Tests
Posttesting the Value of a Sponsorship
 Test Your Knowledge

Laboratory methods of testing for advertisement

effectiveness generally offer high control but low
_____.
A) realism
B) external validity
C) costs
D) generalization
E) value
Where to Test

In the Field
In the Lab
 Positioning Advertising Copy Test (PACT)

1. Provide measurements relevant to objectives of advertising

2. Require agreement on how results will be used
3. Provide multiple measures
4. Be based on a model of human response to communications
5. Consider multiple versus single exposure to the stimulus
6. Require alternative executions to have same degree of finish
7. Provide controls to avoid biasing effects of exposure context
8. Take into account basic considerations of sample definition
9. Demonstrate reliability and validity
 Test Points

1.Concept Testing

2.Rough Testing
Occurs at
Various Stages 3.Finished art or
commercial pretesting

4.Market testing
(posttesting)
 Concept Testing

Explores consumers responses to ad

Objective concepts expressed in words, pictures, or
symbols

Alternatives are exposed to consumers who

match the target audience

Method Reactions & evaluations sought through focus

groups, direct questioning, surveys, etc.

Sample sizes depend on the number of

concepts and the consensus of responses

Qualitative and/or quantitative data

Output evaluating and comparing alternative
concepts
Rough Art, Copy, and Commercial Testing

Comprehension and Reaction Tests

Consumer Juries

Advantages Disadvantages
Consumer may become a
Control
self-appointed expert
Number of ads that can be
Cost effectiveness evaluated is limited
Endorsements by A halo effect is possible
independent third parties
Preference for ad types
Achievement of credibility may overshadow objectivity
 Rough Testing Terms

Animatic Rough

Terms Photomatic Rough

Live-action Rough
An Animatic Example from Airwalk

*Click outside of the video screen to advance to the next slide

Spot Resulting from Animatic Rough

*Click outside of the video screen to advance to the next slide

 Pretesting Finished Print Ads

A laboratory method

Portfolio Includes test and control ads

Tests
Portfolio test have problems

Based on syllables per 100 words

Readability
Tests
Other factors also considered

Dummy Distributed to random sample homes

Advertising
Vehicles Product interest may still bias results
 Test Your Knowledge

_____ is a method of testing ads by placing them

randomly in certain copies of regularly distributed
magazines.
A) Vehicle source testing
B) Burke's reflection test
C) A Flesch test
D) Dummy advertising vehicle testing
E) A contextual test
 Pretesting Finished Broadcast Ads

Theater Tests On-Air Tests

Measures changes in Insertion in TV programs

product preferences in specific markets
May also measure . . . Limitations are imposed
Interest in and reaction by day-after recall
to the commercial Physiological Measures
Reaction from an
adjective checklist
Recall of various
aspects included
Interest in the brand
presented
Continuous reactions
 Physiological Measures

Galvanic skin
Pupil dilation
response

Testing

Brain waves Eye tracking

 Market Testing Print Ads

Inquiry Tests Recognition Tests

Testing

Tracking Studies Recall Tests

Starch-Scored Sports Illustrated Ad
 Posttests of Broadcast Commercials

Day after recall Persuasive

tests measures

Tracking
Diagnostics
studies
Testing

Single-source Comprehensive
tracking measures

Test marketing
Comprehensive Testing by Ipsos-ASI
Problems With Current Research Methods
 Essentials of Effective Testing

Use a consumer
response model

Establish
Use pretests and
communications
posttests
objectives Testing

Understand and
Use multiple
implement
measures
proper research
 Test Your Knowledge

Good tests of advertising effectiveness must address

the nine principles established by PACT. One of the
easiest ways to do this is to follow a decision sequence
model. The first step in the model is to:
A) Understand the appropriate research
B) Create a model that uses multiple
measures
C) Establish communication objectives
D) Decide whether to use posttests or
pretests
E) Develop a consumer response model
Measuring Effectiveness of Other Programs

Sales
promotions
Shopping cart signage

Ski resort-based media

Non-
traditional
media In-store radio and television

Other media

Exposure methods
Sponsor-
ships
Tracking measures
Measuring Effectiveness + Efficiency