ITS-Teaching and Learning

University of North Carolina-Chapel Hill

2006-2007 Classroom Response System Evaluation

This evaluation recommends a single classroom response system as a campus standard for those
courses requiring students to purchase the devices themselves. For these courses, we recommend
adopting eInstruction CPS. Its features, its ability to integrate with existing campus systems, and
the quality of support provided by the vendor meet our selection criteria.

Background
With an objective of increasing the interactivity of large enrollment courses, individual
instructors and departments at UNC-Chapel Hill have been using classroom response systems. A
response system is used by an instructor to pose questions to the class. Students use handheld
devices, commonly referred to as clickers, to transmit their answers to a transceiver attached to
the instructor’s presentation computer. Student’s answers to questions are tallied almost
instantly. The instructor can choose to display to the class graphs of the collected responses.

A few instructors began using these systems as early as the 2000-2001 academic year. Early
adopters struggled with reliability issues as the technologies employed by response systems have
evolved.

As various textbook publishers and other response system vendors continue to vie for individual
instructors and departments as customers, some students have reported having to purchase as
many as three different devices during the course of their undergraduate careers.

Educational institutions who either agree to recommend a single vendor’s system as a standard or
to purchase a certain number of student devices from a vendor receive discounted pricing on
these products.

Objective
The purpose of this evaluation was to recommend a single system as a campus standard for those
courses requiring students to purchase the devices themselves. The recommended system strikes
a balance between cost to students, quality and breadth of vendor support, and features making it
an effective instructional tool.

Products
As of Fall 2006, all competitive vendors offered radio-frequency devices. The five systems in
our evaluation all use this technology, with equivalent reliability: TurningPoint, eInstruction,
iClicker, Interwrite PRS, and Quizdom. (A product comparison matrix is included in this
document.)

During the Fall 2006 semester, TurningPoint, eInstruction, and iClicker were used in instructor-
led courses with large enrollments (75-400 students).

ITS-Teaching and Learning

16/03/07 UNC-Chapel Hill
Selection criteria
To develop and refine selection criteria, ITS-Teaching and Learning distributed an online survey
to faculty currently using a response system and consulted with Student Government about the
usability of the five devices and pricing models used by the vendors. ITS-TL also contacted other
universities who had or were in the process of evaluating response systems themselves.

The primary selection criteria are listed below. Refer to the product comparison matrix for
detailed information about each product.

Available question types. Of those eight faculty who responded to our faculty survey, four
reported using a variety of question types (refer to comparison matrix for a description of these
types).

Ease of use for instructor. Both the usability of the product’s software and ease of student
registration are factors.

Ease of use for students. The largest factor for ease of use is the design of student devices, with
LCD display being the most significant feature. When students earn course credit for the answers
they submit, confirmation through the device that these answers were successfully received
significantly increases students’ comfort in using the technology.

Blackboard integration. Integration with a course management system facilitates both the
registration of student devices with courses and data exchange from instructor’s software to an
online enterprise system.

Quality and range of support from vendor. In addition to providing text-based user guides,
vendors should have weekday phone support, online help request forms, video tutorials, user-
community discussion boards, and real-time, webcast tutorials.

Cost and pricing options. With four of the systems, a student pays a flat fee for a device. With
eInstruction, there are two fees: the cost for the device and the cost for device registration. The
cost to the student for device registration depends upon whether the student decides to pay a one-
time fee or a semesterly fee (capped at three semesters). Therefore, a student pays only during
those semesters that he or she uses the clicker.

During the 2006-2007 academic year, Student Stores Textbooks received orders from instructors
for two systems, eInstruction and iClicker. Standardizing on either of these systems would have
permitted a buyback programs to be instituted more quickly, in this case by May 2007. With a
buyback program in place at this time, the devices that students purchased this academic year,
before the standardization agreement, could be used during Summer Session 2007 onward or
could be resold to other Carolina students.

ITS-Teaching and Learning

16/03/07 UNC-Chapel Hill
 Information current as of Feb 1, 2007; subject to change

CPS TurningPoint iClicker Quizdom PRS

Non-discounted prices ---------- ---------- ---------- ---------- ----------

Student device $16 net $48 net $25 net $39.90 net $35 net
$15 single semester,
capped at three
Device registration fee semesters; $50 flat rate none none none none
Instructor transceiver $350 $199 $300 $525 $249
discounted prices on discounted prices on
discounted registration student devices and student devices and
fees, free instructor instructor transceivers, free instructor instructor transceivers, discounted prices on kits,
Types of discounts transceivers kits transceivers kits free instructor transceiver

Instructor software ---------- ---------- ---------- ---------- ----------

no (can run from USB
Installation required yes yes key) yes yes
both for standalone
client, PC only for both, plus Linux
Supported OS both both both PowerPoint plug in (programmed in Java)
Core question types (refer to
next page for categories) 2 3 1 4 4
Add. software required no PowerPoint no no no
License proprietary proprietary open-source proprietary proprietary
PowerPoint add-in yes yes no yes yes
Instructor transceiver ---------- ---------- ---------- ---------- ----------
802.15.4, Zigbee, same
freq; self-negiotiating
system 16 channels,
Transmission band 2.4 GHz 2.4 GHz 900 MHz patent pending 2.4 GHz
Power supply USB USB USB USB AC (USB in beta)
yes (student remote, yes (student remote
Instructor remote no no reprogrammed) yes (dedicated remote) reprogrammed)

ITS-Teaching and Learning

16/03/07 UNC-Chapel Hill
Student device ---------- ---------- ---------- ---------- ----------
LCD display yes no no yes yes
Battery type 2 AA 2 coin batteries 3 AA 2 AA 3 AAA
360 hours with normal 20 weeks, based on
Battery life usage 6-12 mo 200 hours 1 year of "normal use" typical usage
vendor's server, Bb
Registration services handled building block to vendor's local Bb server, vendor's server or vendor's server or local Bb server,
by server instructor's client instructor's client instructor's client instructor's client
Virtual device available no yes no no yes

Support ---------------- ---------------- ---------------- ---------------- ----------------

Online docs yes yes yes yes yes
Discussion boards yes no no no yes
Chat yes no no no no
Email/webform yes yes yes no yes
Phone yes yes yes yes yes
Video tutorials instructor only instructor only no no instructor and student
Web conference training admin, instructor instructor no no instructor
instructors who adopt
Ad hoc face to face training McGraw-Hill textbooks no no no no

Blackboard Building Block yes yes no no yes

Publisher/content provider Bedford, Freeman &

partnerships McGraw-Hill Thomson Worth Pearson Wiley, Pearson

Question types
Single-answer select Yes Yes Yes Yes Yes
Check all that apply, any order No Yes No Yes No
Numeric input Yes No No Yes Yes
Sequence/series/ordering No No No Yes Yes
Text entry No No No Q5 device only Yes
Weighted ranking No Yes No No No

ITS-Teaching and Learning

16/03/07 UNC-Chapel Hill