Documente Academic
Documente Profesional
Documente Cultură
Table of Contents
Page
Summary
3-4
4-5
Part 3: Results
Part 4: Discussion
7-8
Appendices
8-26
Summary
Kansas State University Libraries uses Twitter as a tool to inform, educate and
entertain their library patrons. Twitter is a web-based tool and a mobile application that
sends 140 character messages to its users who subscribe to a Twitter feed. The
messages (tweets) sent to the library Twitter followers through @kstatelibraries may
contain text, images and/or links to online content.
The purpose of this evaluation was to determine the efficiency, effectiveness and
impact of using Twitter as an educational tool to communicate with library patrons. With
the assistance of program stakeholders, five objectives were identified and approved,
based on the goals of the K-State Libraries Twitter Program (to inform, educate and
entertain). To determine whether the Twitter Evaluation Survey constituted research
with human subjects, the evaluator and collaborators completed Institutional Review
Board (IRB) training and an IRB application, per university policy. Once the IRB
application was approved, a web-based survey was released to the K-State Libraries
Twitter followers via a tweet.
The survey included three questions: one demographic question to establish the
respondents relationship to K-State, a five point Likert Scale to measure the
usefulness of @kstatelibraries tweets based on the program objectives, and an open
response field for narrative feedback on the Library Twitter Program. The user feedback
from this survey will be used to enhance and improve the implementation of Twitter
program objectives that are most useful to library patrons by focusing on specific library
services and programs in an entertaining, humorous way.
Part 1. Description of the program evaluated
Library marketing departments often seek efficient, unobtrusive methods of
connecting with their library patrons in order to create awareness of library events,
services and resources. In the six years that K-State Libraries have been using Twitter,
they have found a passive means of conveying information to those who choose to
follow @kstatelibraries.
The 2,366 followers of the K-State Libraries Twitter account are students
(undergraduate and graduate), staff, faculty, alumni and members of the public. In
addition to the educational and informative aspects, volunteers who enter into a social
media partnership with K-State Libraries through their Twitter feed will gain a deeper
understanding of the campus environment, its culture, history, and humor.
1a. Program Objectives
Previous to this evaluation, the K-State Libraries Twitter Program functioned without
defined program objectives. With the assistance of stakeholders, five program
objectives were identified and approved. These objectives were based on the goals of
the program: to inform, educate and entertain library patrons.
2b. Procedures
The Library Twitter Program has been functioning with varying intensity over the last
six years and continued its normal function throughout this evaluation process that
involves the Library Twitter Team disseminating tweets (at least 6 a day) Monday-Friday
between the hours of 9 a.m. and 9 p.m.
As was mentioned previously in this report, an objectives-based survey was created
for this evaluation using a web-based survey creation/monitoring tool, Qualtrics. The
evaluator and two collaborators created the anonymous, online survey (see Appendix
A) which contains a demographic question, a Likert Scale question to demonstrate the
value/effectiveness of the program objectives, and a text field feedback question to yield
qualitative data.
Before the survey could be administered, IRB Training had to be completed and an
IRB application submitted. This procedure is university policy when any university entity
is potentially conducting research with human subjects. The IRB application and
approval letter are included in Appendix C and D.
As there were a number of surveys being conducted on campus (a university-wide
climate survey and an in-house technology survey), the evaluator and evaluation
collaborators strategized a survey schedule that would potentially yield the most
favorable participation rates for the Library Twitter Evaluation and received permission
from library administration for said schedule. The survey URL was sent out to
@kstatelibraries followers via Twitter a total of three times: Tuesday, October 28th,
Wednesday, November 5th, and Monday, November 10th. The survey closed on Friday,
November 14th.
2c. Data Sources
The sole data source for this evaluation was an online survey distributed to K-State
Libraries Twitter followers three times during a three week period via a tweet containing
a URL link to the anonymous survey. The ordinal data collected in this survey consisted
of a five point Likert Scale ranging from zero to five; zero being not useful and five
being very useful. This scale was used to measure the effectiveness of program
objectives qualitatively.
The data exhibited in the Likert Scale responses of the Library Twitter Program
Survey confirmed that more than half of the respondents find the librarys use of Twitter
as an educational tool very useful in communicating librarys events and services.
Three of the five objectives had a mean over three points and two objectives had a
mean over four points on the Likert Scale.
Part 3. Results
The primary data source for this evaluation was the @kstatelibraries Twitter follower
survey. The table above lists the participant survey responses for the degree of
usefulness of each program objective.
Overall, 28 of the 55 survey participants found that tweets about library events and
services very useful. This outcome was closely followed by 26 of the 55 participants
appreciating the usefulness of @kstatelibraries entertainment value. These two top-rated
program objectives remain so when examined by the mean results with average scores
of 4.27 and 4.20 respectively.
When the data is sorted by demographic responses, more depth to user needs is
revealed (see tables below). Of the 28 undergraduate student respondents, a greater
mean of 4.39 is assigned to finding out about library events and services and the
entertainment value drop to 4.11. Of the 13 graduate student participants, the opposite is
true in that they have a greater use for diversion and entertainment in their use of Twitter
reflected in its higher mean of 4.23 versus 4.15 of library events and services.
Undergraduate Student responses
In the open feedback response section of the survey, a large amount narrative data
was submitted. (see Appendix B) 16 respondents voluntarily shared comments that
reaffirmed the above listed survey results as six referred to @kstatelibraries
entertainment value and seven directly supported knowing library events and services.
Part 4. Discussion
As this was a formative evaluation of the K-State Libraries Twitter Program in
progress, the main purpose of conducting this evaluation was to use the data generated
to improve the program. Overall, the results of this evaluative study support the
usefulness of @kstatelibraries as an education tool for K-State Libraries to communicate
information with their followers.
The Likert Scale used in the K-State Libraries Twitter Survey asked its respondents to
value the usefulness of the program objectives on a scale of one (not useful) to five (very
useful). As the mean of three objectives were three points (3.42, 3.56 and 3.62) and the
remaining two objectives were given four points (4.20 and 4.27), it is safe to say that KState Libraries Twitter followers find the information disseminated from the feed
moderately to above average in its usefulness.
In the results section of this evaluation report, a difference in the user needs of
undergraduate and graduate students was identified based on their survey responses.
Graduate students value the entertainment value of Twitter more than finding out about
library events and resources. Undergraduate students had the opposite response,
valuing the usefulness of library events and resource knowledge over entertainment.
Logically, the assumed use of a social media tool such as Twitter would be to
entertain and one would expect the evaluative analysis to reflect that assumption. The
fact that the program objective patrons will identify library events and services was found
to be more useful than @kstatelibraries entertainment value, is substantial and
surprising. It demonstrates the growth in the use of Twitter as a tool from something
used as a distraction to a tool that can be useful and impactful. K-State Libraries Twitter
followers enjoy being entertained by library Twitter feed but they also expect to be
informed of useful information that aids in their research and education.
As is illustrated in the narrative feedback section of the survey, those who are
@kstatelibraries followers are very supportive of the content being shared with them
and the efforts of those producing the content. The K-State Libraries Twitter Program
received very respectful mean numbers in this survey that evidenced its usefulness but
there is certainly room for improvement in becoming very useful.
When comparing needs of the students who follow @kstatelibraries, administrators
may want to consider the power of combining objectives. For example, a tweet about
library branches might incorporate humor as well. The feedback section of the survey
offered content suggestions to improve the library Twitter feed including offering
researching tips and tricks and to include more pictures, trivia and random facts. Again,
combining these suggestion with the objectives of the program and using humor can
garner attention and entertainment value to educational objectives.
Now that the program objectives have been established and @kstatelibraries have
demonstrated their value, the program objectives should be clearly communicated to KState Libraries Twitter Team. This creative group of staff members will most likely have
feedback of their own to contribute and ideas on how to incorporate those objectives in
the daily Twitter feed content, increasing the usefulness and impact of the program.
As this is the initial evaluation of the K-State Libraries Twitter Program, developing
regular Twitter survey intervals would be recommended. This evaluation may be used as
a baseline for the efficiency, effectiveness and impact of using Twitter as an educational
tool.
Staff
Total
Evaluator
13 days @ $500/day
$6,500
Per diem
13 days @ $35/day
$455
$6,955
Travel
Estimated mileage: 92 miles (56.5 cents/mile)
$51.98
Technology
Qualtrics Survey Software: two month usage
$417
$15
Total Costs
$7,438.98
Appendices
Appendix ATwitter Evaluation Surv
Response
28
51%
13
24%
16%
0%
4
1
55
7%
2%
100%
Value
1
6
1.95
1.65
1.28
55
10
2. What are the most useful aspects of being a @kstatelibraries Twitter follower?
1 (not
5 (very
Total
#
Question
2
3
4
Mean
useful)
useful) Responses
Understanding
the different
1
ways of
2
8
13
21
11
55
3.56
finding library
help
Finding out
more about
2
2
11
15
16
11
55
3.42
the branch
libraries
It's fun and
3
1
4
4
20
26
55
4.20
entertaining
Finding out
about library
4
0
2
9
16
28
55
4.27
events and
services
Learning
about the
library
5
4
4
16
16
15
55
3.62
resources that
can aid in my
research
Statistic
Min Value
Max Value
Mean
Variance
Standard
Deviation
Total
Responses
Understanding
the different
ways of
finding library
help
Finding out
more about
the branch
libraries
Finding out
about
library
events and
services
1
5
3.56
1.18
1
5
3.42
1.28
1
5
4.20
0.98
2
5
4.27
0.76
Learning
about the
library
resources
that can aid
in my
research
1
5
3.62
1.39
1.08
1.13
0.99
0.87
1.18
55
55
55
55
55
11
3. Any feedback you'd like to share with us regarding our Twitter account?
Text Response
Love the variety of content posted, including the fun facts and the services provided at
Hale. Makes the account much more interesting and keeps my attention.
I like the "random" facts you post. I am a distance learner, so the different ways of
finding library help, library events/services really don't apply to me and I've got a handle
on the library resources...overall, an interesting Twitter feed.
As someone who spends most of their time in the library, I learned the different ways of
researching from experience, so the twitter account isn't too helpful on that; however,
when a new event occurs the twitter account does help me there. I usually don't use
twitter for research. I use it to distract me from homework.
I love the fun facts and pictures! Its a nice study break when I get on the library page.
Great humor!
You guys are fantastic! Keep up the hard work!
I do not have a personal twitter account, because of this I really enjoy the twitter feed on
the library's website. It keeps me with historical tidbits and upcoming library events.
Keep up with the funny tweets of other students living in the library.
I love the variety that is seen in the libraries' Twitter account. I appreciate that it
provides real information about how to use the libraries and how to find help but also
keeps things fun and engaging.
I love what the libraries did on Halloween by showcasing the librarians that work here.
I'd love to get to know more about the people that work here because it personalizes
them when I need help on my research.
I enjoy the tweets about books to read and things to watch etc.
Always fun to read!
Keep up the involvement with students and get the word out that the libraries actually
do have a twitter account!
I definitely enjoy seeing the events that the library is hosting but also some really helpful
research tips or tricks would be appreciated as well.
Very encouraging! And the humor is great! Good work
Whoever mans the Twitter account is awesome!
Statistic
Total Responses
Value
16
12
ADMINISTRATIVE INFORMATION:
Title of Project: (if applicable, use the exact title listed in the grant/contract application)
K-State Libraries Twitter Evaluation
Type of Application:
New/Renewal
Shelly Richardson
Degree/Title:
Library Asst. II
Department:
K-State Libraries
Campus
Phone:
785-532-5968
Campus Address:
Fax #:
N/A
sricha71@k-state.edu
13
Does this project involve any collaborators not part of the faculty/staff at KSU? (projects with nonKSU collaborators may require additional coordination and approvals):
No
Yes
Note: Class Projects should use the short form application for class projects.
Funding Source:
Internal
External (identify
source and attach a copy of the sponsors grant
application or contract as submitted to the funding
agency)
Copy attached
Not applicable
Based upon criteria found in 45 CFR 46 and the overview of projects that may qualify for
exemption explained at http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html , I believe
that my project using human subjects should be determined by the IRB to be exempt from IRB
review:
No
Yes
(If yes, please complete application including Section XII. C. Exempt Projects;
remember that only the IRB has the authority to determine that a project is exempt
from IRB review)
14
If you have questions, please call the University Research Compliance Office (URCO) at 532-3224, or
comply@ksu.edu
Shelly Richardson
K-State Libraries Twitter Evaluation
10/01/14
MODIFICATION
Is this a modification of an approved protocol?
following:
Yes
I.
BACKGROUND (concise narrative review of the literature and basis for the study):
N/A
15
II.
III.
OBJECTIVE (briefly state the objective of the research what you hope to learn from the study):
Whether or not the Library's Twitter objectives are being effectively communicated to its
Twitter followers.
IV.
V.
N/A
N/A
RESEARCH SUBJECTS:
A. Source:
B. Number:
N/A
16
participation)
D. Recruitment procedures: (Explain
how do you plan to recruit your
subjects? Attach any fliers, posters,
etc. used in recruitment. If you plan
to use any inducements, ie. cash,
gifts, prizes, etc., please list them
here.)
VI.
RISK PROTECTION BENEFITS: The answers for the three questions below are central to human
subjects research. You must demonstrate a reasonable balance between anticipated risks to
research participants, protection strategies, and anticipated benefits to participants or others.
A. Risks for Subjects: (Identify any reasonably foreseeable physical, psychological, or social
risks for participants. State that there are no known risks if appropriate.)
no known risks
B. Minimizing Risk: (Describe specific measures used to minimize or protect subjects from
anticipated risks.)
C. Benefits: (Describe any reasonably expected benefits for research participants, a class of
participants, or to society as a whole.)
participants may benefit from winning the gift card drawing and the library may benefit
from the evaluation of its twitter service
In your opinion, does the research involve more than minimal risk to subjects? (Minimal risk
means that the risks of harm anticipated in the proposed research are not greater, considering
probability and magnitude, than those ordinarily encountered in daily life or during the performance of
routine physical or psychological examinations or tests.)
Yes
No
17
your agreement with the volunteer and with their expectations. If possible, it is best if
research subjects identity and linkage to information or data remains unknown.
Explain how you are going to protect confidentiality of research subjects and/or data or
records. Include plans for maintaining records after completion.
Survey participants will be anonymous unless they choose to share their email address for
the random gift card drawing. Those who share their email addresses will be assured that
the library will only use their email for the purspose of drawing and will not share it with
others, protecting participant privacy.
VIII. INFORMED CONSENT: Informed consent is a critical component of human subjects research
it is your responsibility to make sure that any potential subject knows exactly what the project that
you are planning is about, and what his/her potential role is. (There may be projects where some
forms of deception of the subject is necessary for the execution of the study, but it must be
carefully justified to and approved by the IRB). A schematic for determining when a waiver or
alteration of informed consent may be considered by the IRB is found at
http://www.hhs.gov/ohrp/policy/consentckls.html
Even if your proposed activity does qualify for a waiver of informed consent, you must still provide
potential participants with basic information that informs them of their rights as subjects, i.e.
explanation that the project is research and the purpose of the research, length of study, study
procedures, debriefing issues to include anticipated benefits, study and administrative contact
information, confidentiality strategy, and the fact that participation is entirely voluntary and can be
terminated at any time without penalty, etc. Even if your potential subjects are completely
anonymous, you are obliged to provide them (and the IRB) with basic information about your
project. See informed consent example on the URCO website. It is a federal requirement to
maintain informed consent forms for 3 years after the study completion.
Yes
No
18
E. Are subjects debriefed about the purposes, consequences, and benefits of the
research? Debriefing refers to a mechanism for informing the research subjects of
the results or conclusions, after the data is collected and analyzed, and the study
is over. (If no explain why.) Attach copy of debriefing statement to be utilized.
*It is a requirement that you maintain all signed copies of informed consent documents for
at least 3 years following the completion of your study. These documents must be available
for examination and review by federal compliance officials.
IX. PROJECT INFORMATION: (If you answer yes to any of the questions below, you should explain
them
in one of the paragraphs above)
Yes
No
19
i.
j.
k.
l.
m. Is there potential for the data from this project to be published in a journal,
presented at a conference, etc?
n. Use of surveys or questionnaires for data collection
IF YES, PLEASE ATTACH!!
X.
SUBJECT INFORMATION: (If you answer yes to any of the questions below, you should explain
them in one of the
paragraphs above)
Yes
No
Does the research involve subjects from any of the following categories?
a. Under 18 years of age (these subjects require parental or guardian consent)
b. Over 65 years of age
c.
g. Victims
h. Subjects in institutions (e.g., prisons, nursing homes, halfway houses)
i.
Are research subjects in this activity students recruited from university classes or
volunteer pools? If so, do you have a reasonable alternative(s) to participation as
a research subject in your project, i.e., another activity such as writing or reading
that would serve to protect students from unfair pressure or coercion to participate
in this project? If you answered this question Yes, explain any alternatives
options for class credit for potential human subject volunteers in your study. (It is
also important to remember that: Students must be free to choose not to
participate in research that they have signed up for at any time without penalty.
Communication of their decision can be conveyed in any manner, to include
simply not showing up for the research.)
This is a voluntary, mostly anonymous online survey. We have no way of knowing
20
XI.
j.
Are research subjects audio taped? If yes, how do you plan to protect the
recorded information and mitigate any additional risks?
k.
CONFLICT OF INTEREST: Concerns have been growing that financial interests in research
may threaten the safety and rights of human research subjects. Financial interests are not in
them selves prohibited and may well be appropriate and legitimate. Not all financial interests
cause Conflict of Interest (COI) or harm to human subjects. However, to the extent that
financial interests may affect the welfare of human subjects in research, IRBs, institutions,
and investigators must consider what actions regarding financial interests may be necessary
to protect human subjects. Please answer the following questions:
Yes
No
a. Do you or the institution have any proprietary interest in a potential product of this
research, including patents, trademarks, copyrights, or licensing agreements?
b. Do you have an equity interest in the research sponsor (publicly held or a nonpublicly held company)?
c.
21
A. KSU Collaborators list anyone affiliated with KSU who is collecting or analyzing data:
(list all collaborators on the project, including co-principal investigators, undergraduate and
graduate students)
Name:
Department:
Campus Phone:
Campus Email:
Laurel Litrell
K-State Libraries,
Office of Library
Planning and
Assessment
785-532-5467
laurlit@ksu.edu
Sarah Hoyt
K-State Libraries,
Office of
Communications
and Marketing
785-532-7452
shoyt@ksu.edu
B. Non-KSU Collaborators: (List all collaborators on your human subjects research project not
affiliated with KSU in the spaces below. KSU has negotiated an Assurance with the Office for
Human Research Protections (OHRP), the federal office responsible for oversight of research
involving human subjects. When research involving human subjects includes collaborators
who are not employees or agents of KSU the activities of those unaffiliated individuals may be
covered under the KSU Assurance only in accordance with a formal, written agreement of
commitment to relevant human subject protection policies and IRB oversight. The Unaffiliated
Investigators Agreement can be found and downloaded at http://www.k-
state.edu/research/comply/irb/forms/Unaffiliated%20Investigator%20Agreement.doc
C.
The URCO must have a copy of the Unaffiliated Investigator Agreement on file for each nonKSU collaborator who is not covered by their own IRB and assurance with OHRP. Consequently,
it is critical that you identify non-KSU collaborators, and initiate any coordination and/or approval
process early, to minimize delays caused by administrative requirements.)
Name:
Organization:
Phone:
Institutional
Email:
Does your non-KSU collaborators organization have an Assurance with OHRP? (for
22
Federalwide Assurance and Multiple Project Assurance (MPA) listings of other institutions, please
reference the OHRP website under Assurance Information at: http://ohrp.cit.nih.gov/search).
No
Yes
C. Exempt Projects: 45 CFR 46 identifies six categories of research involving human subjects
that may be exempt from IRB review. The categories for exemption are listed here:
http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html. If you believe that your
project qualifies for exemption, please indicate which exemption category applies (1-6).
Please remember that only the IRB can make the final determination whether a project is
exempt from IRB review, or not.
category 4; survey
Exemption
Category:
Yes
No
23
1. After signing into K-State Online, you will be taken to the Export Control Homepage
2. Read the directions and click on the video link to begin the program
3. Make sure you enter your name / email when prompted so that participation is verified
If you click on the link and are not taken to K-State Online, this means that you have
already completed the Export Control training and have been removed from the roster. If
this is the case, no further action is required.
-Cant recall if you have completed this training? Contact the URCO at 785-532-3224 or
comply@ksu.edu and we will be happy to look it up for you.
Post Approval Monitoring: The URCO has a Post-Approval Monitoring (PAM) program to help
assure that activities are performed in accordance with provisions or procedures approved by the
IRB. Accordingly, the URCO staff will arrange a PAM visit as appropriate; to assess compliance
with approved activities.
If you have questions, please call the University Research Compliance Office (URCO) at 532-3224, or
comply@ksu.edu
Shelly Richardson
Title of Project:
XIV. ASSURANCES: As the Principal Investigator on this protocol, I provide assurances for the
following:
A.
24
implementation.
B.
Training: I assure that all personnel working with human subjects described in
this protocol are technically competent for the role described for them, and
have completed the required IRB training modules found on the URCO website
at:
http://www.k-state.edu/research/comply/irb/training/index.htm. I
understand that no proposals will receive final IRB approval until the URCO has
documentation of completion of training by all appropriate personnel.
C.
D.
E.
F.
Adverse Event Reporting: I assure that I will promptly report to the IRB / URCO
any unanticipated problems involving risks to subjects or others that involve
the protocol as approved. Unanticipated or Adverse Event Form is located on
the URCO website at:
http://www.kstate.edu/research/comply/irb/forms/index.htm. In the case of a serious
event, the Unanticipated or Adverse Events Form may follow a phone call or
email contact with the URCO.
G.
Accuracy: I assure that the information herein provided to the Committee for
Human Subjects Research is to the best of my knowledge complete and
accurate.
(date)
25
26