Research Paper

Running head: THE LOYOLA EXPERIENCE
Program Evaluation for The Loyola Experience: Year Two

Caroline Chalker and Tempris Daniels
Loyola University Chicago
THE LOYOLA EXPERIENCE
Table of Contents
Introduction.. p. 4-5
Program Overview... p. 5-14
Theoretical Framework.. p. 5-6
History... p. 6-7
Description.... p. 7
Loyola Experience overall goals. . p. 7-8
Year two overall goals and activities. p. 8
Touch points and milestones of year two. . p. 8-10
Additional programs and activities. .. p. 10
Demographics. .. p. 10
Purpose and Objectives.. p. 10-12
Stakeholders... p. 12-14
Logic Model...... p. 14-18
Inputs. p. 14-16
Outputs..... p. 16
Outcomes. p. 16-17
Assumptions..... p. 18
External Factors........ p.
18
Evaluation Approach.. p. 19-20
Design Purpose and Success..... p. 20
Areas of Investigation . p. 20-21
Quantitative Approach.. p. 21-26
Population and Sampling Frame.. p. 21
Response Rate...... p. 21-22
Research Design.. p. 22
Comparison Group.. p. 22
Survey Instrument... p. 22-23
Pilot Test.. p. 23-24
Survey Administration. p. 24-25
Statistical Analysis... p. 25-26
Results in the Final Report... p. 26
Qualitative Approach..... p. 26-34
Sampling... p. 27
Focus Group Protocol.. p. 28-30
Pilot Test... p. 30
Implementation. p. 30
Moderators, audio recording and note taking... p. 30-31
Focus group details... p. 31
Incentives...... p. 31
Analysis...... p. 31-32
Inter-Rater Reliability....... p. 32
Member Checking........ p. 32
Biases....... p. 32
Triangulation......... p. 33
Limitations........ p. 33
Presentation of Qualitative Results..... p. 33-34
Mixed Methods.... p. 34
Timeline... p. 35
Budget. p. 35-36
Next Steps p. 36
References... p. 37-38
Appendices.. p. 39-68
Logic Model-Appendix A.... p. 39
Survey Construct Map- Appendix B ... p. 40-41
Survey Email Invitation-Appendix C... p. 42
Loyola Experience Second Year Survey- Appendix D.... p. 43- 49
Focus Group Email Invites- Appendix E...... p. 50
Focus Group Demographic Form- Appendix F.... p. 51-52
Consent Form for the Second-Year Experience Focus Group- Appendix G... p. 53-54
Third-Year Experience Focus Group Protocol- Appendix H... p. 55-57
Fourth-Year Experience Focus Group Protocol- Appendix I... p. 58-60
Budget- Appendix J... p. 61
Timeline- Appendix K.. p. 62
PowerPoint- Appendix L.. p. 63-68
Program Evaluation for The Loyola Experience: Year Two

Loyola University Chicago (LUC), a Jesuit Catholic University, uses their mission
statement as the core for creating an inclusive environment. The Loyola mission statement reads
We are Chicago's Jesuit, Catholic University-a diverse community seeking God in all things and
working to expand knowledge in the service of humanity through learning, justice and faith.
(Loyola University Chicago, A1). LUC uses its mission and vision statement Loyola University
Chicago is the school of choice for those who wish to seek new knowledge in the service of
humanity in a world-renowned urban center as members of a diverse learning community that
values freedom of inquiry, the pursuit of truth and care for others (Loyola University Chicago,
A1) to create a four-year plan that maps out the goals and touch points that the institution hopes
that its students will reach as they progress through their time as an undergraduate student. With
this came the creation of the Loyola Experience, this is a living document that informs students,
staff, faculty and all whom are involved in the development of students, how LUC envisions
students progressing. The Loyola Experience connects back to many Jesuit values and provides
guidelines to academic advancement.
In order to develop the Loyola Experience, LUCs Division of Student Development and
Academic Affairs collaborated to create this four-year-plan document. The Loyola Experience is
the larger four-year plan that encompasses the entire undergraduate experience for students at
LUC. This is a document and plan for how undergraduate students should move through their
time at Loyola, and outlines the experiences that LUC hopes that the students experience and
take part in. The Second Year Experience is just one part of The Loyola Experience. This is the
smaller outline for second-year students and includes the touch points and goals for that year.
LUC created the four-year-plan in order to build a very clear vision for the university to
publish that anyone could look at and understand the conceptual framework of an undergraduate
students progression through their time at Loyola (S. Howes, personal communication,
September 24, 2014). Shannon Howes the director of the Office of Second Year Experience,
informed us that Loyola did not create this document because there was any specific problem,
rather they felt that there was something missing at the university and they wanted to improve
understanding around what an undergraduate students experience may look like in alignment
with the universities mission and values.
The Loyola Experience is a distinctive process helping students discover their gifts and
talents. Within the four-year-plan LUC hopes that, students will continue through their journey
of building community (during year one), making commitments such as declaring a major (year
two), engaging Chicago and the world (year three), and creating the future (year four) (Loyola
University Chicago, B). The purpose of this evaluation is to determine the success of year two,
making commitments, where we hope to explore whether the touch-points are aiding the
development of students to provide a more holistic experience. We are focusing on year two
specifically because we found when reviewing the four-year-plan that many aspects of year two
include programming, an interest area we both hold. We also noticed that during year two there
are some major milestones such as declaring a major that can affect a students college
experience dramatically
Program Overview
Theoretical Framework
The Loyola Experience is a program created for the undergraduate student population, it
works to help create a positive student transformation and contains seven areas of focus: Build
your skills, connect with community, expand your knowledge, commit to faith, justice and
service, engage in Chicago and the world, focus on your well-being and lead with values (Loyola
University Chicago, B). The four-year-plan does not specify whether this plan is inclusive of
transfer students or students native to LUC, but this will be a part of the evaluation we will look
to find out. Coinciding with the student promise that promotes care for self, others, and
community, the experience incorporates academic, spiritual, and social activities that are unique
to each student (Loyola University Chicago, B). Specific to LUC, the Loyola Experience seeks
to facilitate reflection, help students mark their milestones, set goals, and shape their own
experience. Because this program is unique to LUC and seems to be of interest to other
institutions, administers have presented this plan at various national conferences over the last few
years (S. Howes, personal communication, September 24, 2014). The Loyola Experience has
shifted the expectations for faculty, staff, administrators, and students creating a common goal
towards helping students develop holistically in the areas that LUC has outlined.
History
The Loyola Experience was established in the early 2000s to assist in creating a holistic
environment for students to flourish. LUC wanted to make the outline and plan for undergraduate
students clear for them to understand. The implementation committee for the Loyola Experience
consists of members from both Academic Affairs and the Division of Student Development,
allowing for an equal partnership. Using the transformative education document an article that
describes Jesuit theory and values in the context of higher education (Hartnett, 2009), the
committee worked to establish a solid foundation of the program using theories and Jesuit
traditions. During the early stages of development, committee members found many of the
changes they were looking to create coincided appropriately with the institutional values.
Surprisingly, the concept of having this experience laid out for the student body is innovative to
Loyola and is not widely occurring at other institutions. This factor made the shift in culture
challenging, as research on the topic was limited.
However from a critical approach, in order for the program to make improvements, both
Academic Affairs and The Division of Student Development need to engage students in the
process of understand what this document is, why it is in place, and what it means to them as
students at LUC. The original planning did not include student input (S. Howes, personal
communication, September 24, 2014) a downfall we see as a part of this document and we
question why students were not a part of this. Additionally, because this program is a work in
progress, there is not a set deadline in terms of when decisions need to be made. Furthermore,
because the foundation of the program has already been set in place, small changes will not
greatly affect the project. The Loyola Experience was designed to serve students better, and
create positive connection points for all students. Although the Loyola community did not find a
common problem throughout campus, they did think student experiences could be better fulfilled
(S. Howes, personal communication, September 24, 2014).
Description
Loyola experience overall goals.
As mentioned before, The Loyola Experience was designed to accommodate the
traditional four-year student academic path. While each year has a specific goal, there are seven
repeating themes including the following: Build your skills, connect with community, expand
your knowledge, commit to faith, justice, and service, engage in Chicago and the world, focus on
your well-being and lead with values (Loyola University Chicago, B).
For the purpose of our project, we will focus in on year two of the Loyola Experience.
Before we describe year two, it is important to understand the context of the other years. The
first year focuses on building communities. Within this year there is a strong emphasis on
developing strong habits and getting involved on campus. The third year centers on engaging in
Chicago and the world, where some of the goals include building relationships, and expanding
students horizons. The final year centers on creating a positive future for students. Some of the
main goals include finding time for reflection and making contacts beyond Loyola University
Chicago.
Year two overall goals and activities.
There are five main goals the institution hopes second-year students achieve. The first
goal is to refine strong habits, a necessary action to help students engage in thought-provoking
and intellectual work (Loyola University Chicago, B). Secondly, students needs to be able to
state their values so that they are able to articulate them and discern how they have formed their
personal, academic, and professional goals and decisions (Loyola University Chicago, B).
Reflection on what students have learned is the third objective where the institution hopes
students will continue to integrate their learning through personal reflections at retreats and/or on
their own time of their LUC experience (Loyola University Chicago, B). The fourth objective is
to build relationships with faculty, staff, and other members of the LUC community (Loyola
University Chicago, B). Building these relationships will provide students with meaningful
relationships that they will be able to hold on to after graduation. The last objective for secondyear students is to help students find time for faith by providing opportunities for students to
deepen the understanding of their own faith and learn about other religions and worlds views
(Loyola University Chicago, B).
Touch points and milestones of year two.
In terms of year two, the overarching theme is titled: Making Commitments. The seven
repeating themes reflect the yearly touch-points seen through the year. The first touch-point is to
commit to faith, justice, and service and this is executed by celebrating ones own faith with the
community around you. The second point is to connect with the community. There are three
reflection questions students can use as a guide throughout the year:
1. How can you maximize your intellectual, social, and spiritual growth?
2. How are you integrating knowledge from your Core, major, and experiences outside of
the classroom?
3. How did you grow in your first year? How would you like to grow in your second year
(Loyola University Chicago, A6)?
Furthermore, the third touch point of the year two plan emphasizes engagement in Chicago and
the world. Conversations are had to prepare students for future internships and to create dialogue
with the Office of International Programs for future study abroad opportunities. Expanding
students knowledge is the fourth point, where direction is given to students about the Core
curriculum, experiential learning, teaching, and academic excellence. The final touch-point is to
assist students to lead with values. This point includes becoming a better leader and promoting
positive change. With these touch points, students are able to engage with each other and gain
multiple perspectives of the world.
Second-year students are also given the opportunity to complete a checklist of
accomplishments. The following are the list of milestones students hope to reach:
1.
2.
3.
4.
5.
6.
Earn at least 60-plus credit hours.

Declare majors and minors; generate your academic requirement report in locus.
Create plans for internships, study abroad, and research with faculty.
Complete your core curriculum courses.
Update your resume and e-portfolios
Participate in the Halfway to Graduation celebration (Loyola University Chicago, B).
Additional programs and activities.
To help students achieve these goals, some of the key characteristics used to support students
include facilitating intentional connections with faculty, staff, students, and community
10
members. Additionally, reflection retreats such as Unwritten provide students with time and
space to relax and think (S. Howes, personal communication, September 24, 2014). Welcome
Back Week is another experience for students where they receive continued support by the
institution. While the needs for second-year students are different than during their first-year
orientation, there is continued encouragement for students to continue to make strong
connections (S. Howes, personal communication, September 24, 2014).
A few additional key activities for second-year students include: sophomore supper, student
organizations, and informative time with faculty (S. Howes, personal communication, September
24, 2014). The office that organizes many of these activities is the Office of Second Year
Experience. Loyolas hope was to allow progress for all students to learn and engage in a diverse
society (S. Howes, personal communication, September 24, 2014), which is shown by creating
spaces that are needed.
Demographics.
The demographics and geographical locations where these activities occur, vary
depending on the student, but learning can take place everywhere. The program for sophomore
students is a yearlong process until the students move on to year three. This program is not a
step-by-step process because there is no follow up, yet the program flows differently based on
each students individual needs. Although the description of the program is listed, there is not a
particular order, yet just a set of goals and milestones.
Purpose and Objectives
The purpose of implementing the Loyola Experience was to better serve the students at
the institution. The mission of the university states; We are Chicagos Jesuit, Catholic
University a diverse community seeking God in all things and working to expand knowledge
11
in the service of humanity through learning, justice, and faith (Loyola University Chicago, A1).
Utilizing the mission and building on the overall strategic plan, the board felt that at The Loyola
Experience would help enhance students overall satisfaction (S. Howes, personal
communication, September 24, 2014). Although students were able to engage with classmates
during their first year, administrators learned that students were not connecting with their peers
once they declared a major (S. Syversen, personal communication, September 24, 2014). The
Loyola Experience provided conditions for students to have common experiences with each
other even if in different majors because of the milestones given and assignments like eportfolios. Additionally, it was important to the institution that students are prepared for life after
graduation and that they develop holistically. The LUC community felt as though students were
thriving on campus, but wanted to ensure that students could prosper in their future careers and
would be culturally competent.
The second year was looked at as the forgotten year before the Loyola Experience was
established (S. Howes, personal communication, September 24, 2014). One of the main goals
for a students second year was to provide higher retention rates by building affinity, where
students felt welcomed and wanted by the community (S. Howes, personal communication,
September 24, 2014). The second year is an important year for students because if they commit
to a major, and return for their third year, the likelihood of graduating is much higher than
students who do not return after their first year (S. Howes, personal communication, September
24, 2014). Additionally, an emphasis on the four-year experience is something that is pushed in
the second year. LUC wants their students to commit so that they achieve their goals in four
years versus five or six years. Urging students to achieve this goal will help them stay on a path
for academic support and success (S. Howes, personal communication, September 24, 2014).
12
Stakeholders
There are five stakeholders who play a large part in the past and current success of year
two of the Loyola Experience. The first stakeholder is Terri Thomas, the Director of Student
Academic Success. As a higher education administrator with over 12 years of professional
experience, Terri is dedicated to providing students with high-quality developmental services that
attribute to their success (T. Thomas, personal communication, 2014). Terri has been a member
of the Loyola community since September 2006 serving as an Academic Advisor. Terris current
position has allowed her to co-chair the implementation committee of the Loyola Experience,
collaborating with both Division of Student Development and Academic Affairs.
Our second stakeholder is Shannon Howes the Director of Leadership Development and
Second Year Experience. As the director, she focuses on strategic planning, program
development, and assessment (Loyola University Chicago, A4). Shannons office provides
students with the opportunities to make commitments concerning their career path and
communities (Loyola University Chicago, A4). Additionally, the Office of Second Year
Experience encourages students to take ownership of their passions and goals while providing
them with different ways to make intentional connections with the community, through activities
like the sophomore retreat titled Written, Welcome Back Week, sophomore supper, and halfway
to graduation (S. Howes, personal communication, September 24, 2014).
Our third stakeholder works with second-year students through academic affairs. The
Department of First and Second Year Advising works with students as they explore potential
majors. Specifically, Sarah Syversen, Assistant Director of Second-Year Advising, works with
additional academic advisors to encourage students to build experience into their four-year plan
such as studying abroad and exploring opportunities for research and internships. Additionally
13
the department conducts the sophomore file review where it checks second-year students
academic history and ensures they are on the correct path to graduation. Furthermore the office
assists students as they explore options for majors and minors and then make a commitment to
their academic endeavors. They also assist students in obtaining credits for courses taken while
abroad, and participates in the halfway to graduation ceremony (Loyola University Chicago, A2).
Our fourth stakeholder is the Assessment Committee chaired by Michael Beazley, Ph.D.,
Director of Assessment in the Division of Student Development. The assessment committee is
membered with both the Division of Student Development and Academic Affairs. According the
LUC assessment committee link on the universities website, the assessment committee is
responsible for collecting, reporting and working in conjunction with the offices in the Division
of Student Development to evaluate learning outcomes and programs (Loyola University
Chicago, A5).
The final stakeholders are the students who are directly affected by the second-year
Loyola Experience. These students will hopefully proceed through this year exercising the
habits to engage in challenging intellectual work and integrate their learning through reflection
on their experiences (Loyola University Chicago, A5).
With these main stakeholders in mind, it is important to realize that there are groups that
indirectly benefit from the second-year Loyola Experience. Some of these groups include the
other students attending the university, as they are able to see how transformative a students
second year can be, as well as the Rogers Park and general Chicagoland community. These
individuals/groups are important because they create the community at large and while there is
the second-year community, it is important that students not only make commitments to
themselves but the people surrounding them. Because one of the goals that the second-year plan
14
includes engaging in Chicago and the world, students may take some time during their second
year to work in the Rogers park community or other neighborhoods in Chicago. LUC is
connected to many different resources in the city, and students may be told about different
community resources if they are interested in areas such as community service, community
engagement, local churches, outreach centers etc.
Logic Model
This logic model (Appendix A) created for year two of the Loyola Experience, outlines
the flow of the essential parts of this program. It is set up in a way that allows the program to be
broken down into its most foundational pieces, and to be processed in a way that goes from start
to finish (Wholey, Hatry, & Newcomer, 2010). Because the nature of this program is very large
and is set up to serve the entire second-year population of students at Loyola, this logic model is
inclusive of all of those pieces.
Inputs
In order for this to happen, Loyola depends on the students to be participants in this
process, where they must partake in the programs and activities outlined by the Loyola
Experience for this plan to be successful. The second largest group that this program depends on
is the university staff, faculty and administrators. Everything outlined and provided for the
second-year students in this plan are run, coordinated, organized and implemented by those at
LUC who work directly with students. Without their efforts and willingness to partake in this
program, the second-year experience would not be possible. Time is a key component and input
for this program, because a large percentage of the activities and programmatic initiatives in the
second-year experience are implemented in that academic year, from August to May. Students
15
only have a short amount of time to complete or be a part of all of the experiences laid out for
them in their second year.
Perhaps the most fluctuating input, money, is a major component of the second-year
experience. Loyola must set aside money in order for programs to be run, and students must
produce funds if they wish to partake in some of the outlined activities such as study abroad. As
the years go by, budgets fluctuate and impact the degree to which these initiatives set forth by the
university can run properly. The same goes for the student population, as different classes are
enrolled at Loyola, their financial needs and abilities change. Each students finances are unique,
therefore adding a very complex layer to this program and how students engage in it.
Most departments on campus are either directly or indirectly connected to the experience
of second year students at Loyola, but there are a few that are dedicated to serving this
population and are informed by the outline of the Second Year Loyola Experience. The first
office is Leadership Development and Second Year Experience, an office is grounded by
leadership opportunities and programs specifically designed to serve the current second year
students. The second office is First and Second Year Advising where their focus lies in guiding
this population of students through their second year at Loyola.
There are a lot of materials and supplies that go into this second year experience, and
much of that is reliant on budget. In order to make these events and initiatives successful they
need pamphlets, flyers, t-shirts, dinners, academic supplies etc. Much research was needed in
order to design and create this model of the Loyola Second Year Experience. The research that
was necessary was needed to allow for the most progressive, current and reliable information
that informed the creators of the concept of what a program of its kind would look like, and how
it would affect the students. Lastly, technology is a part of everyday life that plays a role in the
16
experience of second year students at Loyola (S. Howes, personal communication, September
24, 2014). Technology was needed to create this program, implement it, utilize it, and
understand it. Without the daily use of technology, this program would not be able to function.
Outputs
The outputs in this logic model derive from multiple, if not all of the inputs that create the
substance of this program. The university along with some key department collaborators, invest
a lot of time during the academic year to provide the second-year student population with a
multitude of activities and programs, to make sure that all of the goals and touch points, in which
they hope that most second year students will reach are obtainable. Some of the major
components are retreats hosted by the Leadership Development and Second Year Experience
office, the academic advising office assisting with choosing a major, and the Half way to
Graduation Celebration. These activities and many more are provided so that the second year
students feel supported and encouraged to continue through their academic careers.
Outcomes
The outcomes of all of these programs and initiatives are split up between short, medium
and long term. After students second year has come to a close, the short-term goals LUC hopes
for include a deeper understanding of values and finding a sense of commitment to them, such as
joining a club or organization that aligns with ones values. They also hope that you gain the
ability to engage in asking Big Life Questions. Although Big Life Questions are different for
everyone they mainly surround questions of your academic, social, spiritual and personal
development (S. Howes, personal communication, September 24th 2014) especially after you
have declared your major. Lastly, among other goals, they hope that you feel a sense of support
from the university and are prepared to enter your third year with a good sense of direction.
17
The medium term goals Loyola has set up include building strong bonds with faculty.
Through the work of particular departments on campus such as Academic Advising, they hope
that students have found a faculty member in the department they are a part of, and feel
comfortable working alongside them and approaching them if you need support. After students
second year, the university hopes that they have gained a long-term affinity with their graduating
class. LUC hopes that students can create a sense of connection to their peers that will carry with
them through graduation. Time to discover passions and interest areas at the institution are also
of much importance after students second year. The objective is for students to feel at home
and find a place on campus where they can thrive and find a sense of community.
The long-term goals are skills and values that Loyola hopes students can carry with them
not only through the rest of their time as an undergraduate student, but for the rest of their life.
An understanding of local, national, and international community is of key importance. A
connection to the university is a central piece of what the institution hopes students have come
to build after their second year, grow stronger during their third and fourth, and then carry that
with them as alumni. Lastly, stronger social and communication skills are a component of
personal development the university hopes students can foster in students. The hope is that
through the experiences that have been offered, second year students can see a positive change in
their communication skills.
Assumptions
In order to evaluate this program in a truthful and honest manor, there are some
assumptions about The Second Year Experience that need to be acknowledged. One of the main
assumptions is that second year students are allotted enough time during the academic calendar
in order to actually complete these touch points laid out for them. The supposition is that these
18
goals are realistic but that may not hold true for all students. There is also the belief that the
needs of the students match the outcomes that the university has set forth. This theory may
change as higher education and students transform. Additionally, since we know that financial
resources are continually changing, there is an assumption that the university will year after year,
be able to afford the necessary activities and initiatives in order for this second year experience
program to continue and remain long lasting.
External Factors
In order for the second year plan to continue to grow, it must be mindful that for better or
for worse there are outside factors that can change the way in which this program works, for
better or for worse. The economic climate of not only the state of Illinois, but also of the country
can affect the programs success. If funding for the university is cut dramatically or the students
entering their second year of Loyola are being impacted by something like a recession, than the
opportunities the university had hoped to engage students in might not happen. The influence of
family pressure is also an external factor that would need to be taken into consideration with the
evaluation of this program. The expectations and needs of students, families and close
relationships, may be of competing importance with the Loyola expectations and goals. This can
add a more complex level of understanding the different dynamics to this evaluation.
Evaluation Approach
The goal of this evaluation is to provide an understanding about whether the Loyola
Experience: Year Two is working, and if it is not working, what exactly is causing that problem.
Specifically, we hope to answer whether second-year students emerge with the types of learning
19
and developmental outcomes outlined by the Loyola Experience second- year plan. Since an
evaluation has not been done before surrounding these questions, it will provide a more detailed
and specific understanding of how this program is going, and what areas could be improved
upon. In order to work on finding those answers this evaluation will take a formative approach.
We are looking to be able to provide feedback to the stakeholders, a picture of what can be
improved upon, and where the potential problem(s) lie. In taking this formative look at the
Second Year Experience, a combination approach of process and outcomes would be most suited
to answer some of these questions.
Furthermore, we are looking to answer overarching evaluation questions. First our hope
is to find out if the Loyola Experience is working and what are some strengths and weaknesses?
Next, is the second-year plan serving the needs of second- year students? Third, are students
reaching the touch-points that are laid out in the second-year plan? Fourth, is the second-year
plan aiding the development of students to provide a more holistic experience? Finally, we are
looking to examine whether students are aware of the Loyola Experience?
Due to the evaluation questions, a combination approach is best suited for the design of
our evaluation. It will allow us to understand if the connections are being made between what the
university has planned and stated, and whether the students in their second year are truly being
affected in the way the university has hoped (Fitzpatrick, Sanders, & Worthen, 2011), while also
looking at whether the delivery of the programs are positively impacting the students. They also
focus on the characteristics of the students receiving this program and what the actual activities
look like (Fitzpatrick et al., 2011). Because this is such a large program that affects so many
students, it will be crucial to understand how the initiatives are being executed.
Design Purpose and Success
20
A formative design, with a combination approach, will allow us to understand what is

working, and what is not. It will also allow us to understand the experience of second year
students, and how they are receiving and being impacted by this program. This combination
approach will be a strong way to go about this evaluation because it will allow us to utilize
different aspects of evaluation in order to understand as many different parts of this large
program as possible.
One of the weaknesses we can expect to encounter is a lack of true congruency with the
process of understanding how this program affects the students. Because second year students
were not included as a part of creating this plan, their voice (arguably the most important) was
left out (S. Howes, personal communication, September 24, 2014). Therefore, the evaluation
might be able to help administrators and staff more than it can truly identify if this process has
helped students, because of its existence.
Areas of Investigation
The outcome we plan to understand first, is if the goals and areas of growth that LUC has
laid out for its second year students can be achieved by the ways in which the institution has
planned them to work. This allows us to look at the design of the program and its relationship to
actualizing the outcomes. We are also interested in understanding specifically what is allowing it
to work, or what is hindering this process from being successful. Second, we would like to look
specifically at second year students or those who have just completed their second year, to
understand what their experience was like going through this process. Since no formal evaluation
has been done on the second year experience, this will provide a good foundation for any more
specific evaluations to be completed in the future. These questions are the most basic, yet what
we see as the most fundamental to support any forward movement to be done in the future.
21
Quantitative Approach
Population and Sampling Frame

We are distributing the survey to 2,497 second-year LUC students through email (Loyola
University Chicago, A3). Intentionally, we understand that every student will not fill out the
survey but our hopes are that we are able to get a wide-range of perspectives that representative
of the entire second-year class. Each student can access a computer and/or Internet through the
institutions library or through his or her personal computers. Additionally, due to the last
attempt to survey a past second-year class where less than ten students responded (personal
communication, M. Beazley, October 1st 2014), our hope is that including raffle prizes will
encourage students to participate. For the purpose of this study, staff and faculty will not have
the opportunity to fill out the survey.
The sample frame we will use for this survey is a census sampling approach (Schuh &
Associates, 2009). A census approach is a measurement of the entire sampling population. For
the purposes of our study, our sampling population is entire second-year student class. By using
this approach, we hope that our results will represent majority of the class (Schuh & Associates,
2009).
Response Rate
The anticipated response rate is 25% where out of the 2,497 students enrolled in the Fall
2013-2014 first-year class, 624 students would need to complete the survey (Loyola University
Chicago, A3). The current research team found that this would be a high-response rate based on
previous studies. One hypothesis states that students are more invested in the Loyola Experience
because the program has been around longer (personal communication, T. Thomas, October 9th
2014) and students are more connected with the idea of Loyola Experience. Additionally, a
22
department was built to help enrich the second year experience to allow the opportunity for
students to become more aware of the different opportunities for them.
Research Design
The research design used for this study is a pre-experimental design. We are evaluating a
treatment, the second-year class, during one point in time. Evaluation takes place at the
conclusion of the second-year students experience but there is not a control group (Creswell,
2014). The measurement is taken through a single snapshot where regression analyses helps one
understand the relationships between explanatory variables like program participation,
demographics, and program outcomes evaluators are measuring.
Comparison Group
For this particular evaluation we will not be using a comparison group. In order to
evaluate an entire class of students it will take an enormous effort to just handle that volume of
numbers and statistics. A comparison group is not a possibility at this point. If we were looking
at a subset of the second year students that might be possible, but seeing as though this is the first
evaluation to be prepared there is not much to work off of.
Survey Instrument
The survey instrument we have created (Appendix D) is a compilation of foundational
questions related to the short term and medium term outcomes we are looking to assess along
with demographic questions. There are questions that relate back to short-term and medium-term
outcomes. Short-term outcomes were defined as results that could be measured by the end of a
students second-year. Medium-term outcomes were defined as answers students could give after
reflecting on their second-year experience. For purposes of this evaluation, only short and
23
medium term outcomes were measured based on what outcomes students might have found
impactful at this point during their educational journey and overall time.
All of the questions asked in the survey relate back to the outcomes that are being
analyzed based on the five goals of year two of the four-year plan. Survey questions are based
off of the milestones and goals set by the University for second-year students. It is our hope that
we gain a better sense for how helpful and supportive University departments were to students
during their second-year. In addition, we are looking to see how committed students are to their
declared major as well as the bonds they feel they have built with faculty members. While all
students may not take a religious stance, through the survey we are examining to see if students
think they are provided with opportunities for reflection and/or to gain a better understanding of
their faith. The survey will also help us see if students are developing holistically (academically,
professionally, personally, and spiritually) during their second-year experience. Finally, our both
is to better understand how students think they are building bonds with others students and feel a
sense of belonging to the University.
Pilot Test
We will be pilot testing our survey. This will allow us to gauge the accuracy of our
questions and understand the point of view of a current second-year student who is living out
their second year at Loyola. It is our hope that by pilot testing our survey we will be able to gain
feedback from current third-year students about any changes that may need to be made on the
survey. We hope that they will let us know if there is any major parts of the second-year
experience that we have left out or if some questions on the survey are not relative to the second
year experience of a student at Loyola. We will also welcome students to annotate any
suggestions or notes they have on the sides of the pilot test survey. The testing will occur in a
24
more informal setting so that the students involved do not feel any pressure. We want them to
take their time and be honest in their responses. We will ask four current third-year students to be
involved in the pilot test. They will both receive the exact same survey. Our choice to include
current third year students is intentional because we also want to have feedback from students
who have completed their second year and can reflect back on their recent experiences. This will
supplement the feedback we receive from current third-year students who may have experienced
some parts of the Year Two experience laid out by Loyola, but may also be in the process of
experiencing other parts. The most important part of this pilot test will be to ensure that the
survey is user-friendly and that the respondents understand the questions. The feedback obtained
from the pilot testing will then be evaluated by Caroline Chalker and Tempris Daniels then taken
into consideration when making edits.
Survey Administration
The survey will be administered to students at the completion of their second year at
Loyola, during the last month of the semester. In order to collect as much data as possible the
survey will be handed out during the last month of the semester. We hope this will allow a large
number of students to participate and not confine them to a certain time and date. In order to
administer the survey to as many students as possible, students will complete the survey online
through a link that they will receive in an email from the university (Appendix E). The survey
will be distributed by the Dean of Students office through email (LoyolaExperience@luc.edu).
Students will receive reminder emails throughout the last month of the semester asking for
participation so that they do not forget about the survey or lose the email. The goal is that
through the reminder emails, students will see the importance of completing the surveys and
25
view this survey as a priority. The link will then close on the last day of the semester and we will
begin to look at the results.
A web-based survey will also allow us to collect a large amount of data quickly and
smoothly. Because the students will receive the survey through their email, there will not be
anyone present to administer the survey, but we will provide our email addresses for anyone who
has any questions or comments about the survey. Although our survey is distributed online, we
are accommodating to all students and can distribute the survey in a way that is most comfortable
for the student. There will be an incentive involved in this survey. Every second year student
who completes the survey will be entered into a raffle to win one of the four $25.00 gift cards to
the Loyola bookstore to help pay for their books the following year. There will be a total of four
single gift cards given away.
Statistical Analysis
After the survey closes during the final month of the semester, we will begin by using
SPSS to run frequencies for categorical variables based off of our survey using demographics
such as race, gender, age, first-generation college student status, and student living arrangement
in order to get a general understanding of who completed our survey. In addition, a means likertscale is used to gain a better perspective of the range of students completing the survey. Using
this breakdown will allow us to see if there is a particular demographic of student that has not
filled out the survey and therefore may skew our data in one way or another based off the
demographic breakdown of the entire second-year class. A frequency analysis will also help us to
see how students feel about their connection with university faculty and staff across the entire
demographic categories. We will then use regression tests to explain multiple variables and their
connection to each other or lack thereof using independent variables such as demographics relate
26
to dependent (outcome) variables like connectedness and a sense of belonging to the university.
Looking at frequencies will allow us to look at how representative the sample population is to the
entire population of second year students.
Results in a Final Report
After running the statistical analysis through SPSS we will present the results in two
forms, tables and charts. The tables will allow us to show the results of the descriptive statistical
analysis that we run. The use of tables will allow us to show numbers such as the different
means, medians and modes in a visual. We will also present data in a table format and include
values such as Beta coefficients, R-square values, and standard errors for the independent
variables tested in order to describe the regression analysis and frequencies.
Qualitative Approach
While our quantitative assessment will assist in describing statistics related to the survey
results, our qualitative assessment will go beyond that by providing enriching stories from our
survey respondents. With the use of questions related to the given topics, our focus groups will
look to highlight memorable moments of the second-year experience. Our qualitative approach
will allow us to see how students are interacting on campus and their level of involvement. In
addition, this approach will be shaped based on the quantitative survey questions and will help
evaluators learn through students stories how the second-year experience aids students in their
development. Using a focus group approach, our data collected will help investigate our
processes and outcomes. A focus group is used during the qualitative approach to collect data
through a semi-structured conversation with students and the group leader (Schuh, 2009).
A major way data collection will link to our processes and outcomes is by addressing the
correlation we find through survey results and focus groups. From this we are able to hear the
process in which students take to reach the learning outcomes. The outcome approach will help
27
make improvements for the second-year experience. Through the shared experiences, assessment
can be done on how to improve and make the second-year more impactful. The process approach
will help researchers investigate how the second-year experience shaped the students.
Sampling
For the qualitative approach, we would like to create opportunities for focus groups
where students will have the opportunity to share their experiences. This will help us get a sense
of what a students second-year experience may generally look like. We will use a purposeful
sampling method for our focus groups, which will be chosen based on the survey responses we
receive in order to maintain maximum variation across multiple demographics (Schuh, 2009). By
utilizing purposeful sampling, we are able to filter student participation based on different
demographics such as race/ethnicity, age, gender, parental/guardians educational status, and
student living arrangements. We hope to utilize these demographics to ensure we have a range of
students represented during our focus groups. In addition, we find that it will be beneficial to
have a range of students who are and who are not committed to their declared major as well as a
range of a students level of involvement on-campus. In relation to our sampling population we
are looking to have a range of students who both completed and did not complete the milestones
set for second-year students.
Focus Group Protocol
Our focus group protocol is divided into seven different sections: welcome, reflection of
the most important parts of your second-year, declaring a major, development of leadership skills
and engagement on campus, exploration of service learning opportunities, exploration of faith
and social justice opportunities, and critiques and suggestions. The above group topics were
chosen based on the milestones set for second-year students to achieve. Each topic is connected
28
to a milestone in the hopes that students have an overall second-year experience that is
meaningful. In addition, these topics will build upon our quantitative survey and allow us to
answer our original evaluation question by hearing students stories on how these topics are
determining the success of year two, making commitments, where we hope to explore whether
the touch-points are aiding the development of students to provide a more holistic experience.
The welcome and icebreaker are part of our focus group so that students have a chance to
meet each other and converse before the actual focus group in order to build affinity and talk
about non-focus group related items. The icebreaker will consist of students sharing their name,
declared major, and one thing about themselves. After every student has done this, they will have
about 30-minutes to eat dinner before the focus group starts. The focus group will officially start
with an introduction, overview of the night, and consent form procedure (Appendix H & I).
For the topic titled reflection of the most important parts of your second-year, we hope to
gain stories surrounding students most significant pieces of their second-year and why their
experiences are so meaningful. The time it will take to speak about this topic is 15-minutes. The
focus of this section is to get students to start talking about their second-year experience in a
broader sense. Through these conversations we are able to relate a students experience to the
overall milestones of a second-year experience and see which experiences students discuss when
thinking of their most memorable ones and to what extent the Loyola Experience programming
connects with these experiences. During this focus group segment, we would like students to
reflect on their overall experience and identify what points stand out (Appendix H & I).
The next topic discussed is declaring a major. This section will take 15-minutes,
transitioning to student academic stories. This section of the focus group is significant for us,
where the most noted milestone for students is declaring a major. The process of declaring a
29
major must be completed by the end of a students second-year. The focus of this section is to
gain a better understanding of the journey students take to explore their major. We are looking to
examine whether in-class experiences, out-of-class experiences, faculty, and peers play a role in
a students exploration. This is also a time where students can reflect on whether various
coursework activities played a part in the process of exploring and declaring their major.
Concluding with declaring a major, developing of leadership skills and engagement on
campus is the next 15-minute topic. The focus of this topic is to engage in conversation
concerning co-curricular involvement. The questions and probes we ask (Appendix H & I) focus
on student involvement, leadership, and leadership skills gained through involvement. Our hopes
are that students will reflect on both formal and informal involvement and how that may have
impacted their second-year experience as a whole. As part of the Loyola Experience, second-year
students are encouraged to make commitments to their organizations and become a better leader.
The next 15-minute section of the focus group is exploration of service learning
opportunities. During this timeframe, students will engage with each other about any
opportunities they had to participate in service learning. In addition, we will invite students to
consider whether their experiences have changed their perceptions (Appendix H & I). These
questions will be asked because as a second-year student, one of the milestones for the Loyola
Experience is to explore the world through different service opportunities.
Following the discussion around service learning opportunities, we will shift gears to
explore our final topic, faith and social justice opportunities. With Loyola being a Jesuit
institution, this 15-minute section will explore whether students wanted to participate in
faith/religious experiences as well as social justice related opportunities (Appendix H & I).
Milestones set surrounding these topics for second-year student may be conflicting because
30
although students are attending a religiously-affiliated institution, they may not hold the same
values. Through our conversations, we hope to gain knowledge on what students consider a
commitment to faith, if at all, and what expanding ones faith may look like.
Pilot Test
A pilot test on the focus group protocol will be done during the same time our survey is
distributed to the second-year students. While students who pilot test in our focus group will
have the opportunity to complete our survey questions, these students will not be able to
participate in the actual focus group, to ensure validity. We will ask our pilot test students to go
through the focus group process and give honest feedback about the questions asked,
environment setting, and any other critiques to ensure the focus groups are ran smoothly in the
future.
Implementation
Moderators, audio recording and note taking.
The next steps will be to carry out the focus groups that we have planned for the
qualitative portion of the evaluation. Both evaluators will serve as the moderators for the focus
groups; we will switch on and off. One evaluator will be moderating for two of the focus groups
and the other evaluator will be the moderator for the third. While one person is the moderator the
other will serve as a note taker and make sure that the conversation is recorded. In order to make
sure that we can look back at the focus group information we will be audiotaping the focus group
sessions as well as taking notes. The person who is not the moderator during a particular focus
group will start the audiotape and then take notes as the conversation goes on.
Focus group details.
31
These focus groups will occur at the end of the second-year experience, but because we
are asking students to fill out their surveys at the end of the semester, they will then all be
leaving for summer break, so we will not be able to host these focus groups until the students
return back for the beginning of the next academic year. Upon arriving back in the fall the
second-year students will attend the third-year focus group (Appendix H) and the third-years will
attend the fourth-year focus group (Appendix I). The focus groups will be held in conference
rooms in the Damen Student Center on Loyolas Lake Shore Campus. We will reserve rooms that
have enough space for large tables and plenty of chairs to accommodate all the students
involved. Each focus group will last for approximately ninety minutes; we hope not to have
students for much longer than that.
Incentives.
Since it will be the beginning of a new semester when we host these focus groups we will
be hosting them during a weekday evening hour. Our hope is that by providing dinner and
hosting it in the Damen Student Center this will make for a more comfortable environment for
the students and they will not feel as though this is another stressful event to add to their already
busy lives. The dinner is catered and it will also include soft drinks; we will not be providing any
other incentives beyond the free dinner.
Analysis
In order to analyze all of the data that the evaluators will collect through the focus
groups, we will utilize a descriptive coding method. We will go about this coding by a mixture of
the two options between developing codes before we read through the transcriptions, and also
change/edit the codes as we continue to work through the transcriptions (Rogers & Goodrick,
2010). Because we plan to host multiple focus groups each lasting approximately ninety minutes
32
long we recognize that this will produce a very large amount of transcription data. Once all of the
transcribing has been completed we will scan through the transcriptions and utilize the pawing
method in order to sift through and find any repeating themes or concepts.
Inter-Rater Reliability
Evaluators will test for inter-rater reliability in order to make sure that the coding system
we will be utilizing while sifting through the transcriptions are similar. We also want to do this to
make sure that we are on the same page when it comes to definitions of codes, so that we
understand what each code means to each other and have matching understanding/definitions of
each code. Consistency is going to be very important in our study because of the massive amount
of students involved and data that will be collected.
Member Checking
We will also utilize member checking in order to make sure that the evaluators are truly
getting to the core themes that may be highlighted in all of the data. It will be important for us to
go back to some of our respondents and focus group members to make sure that the results we
are finding actually resonate with our respondents.
Biases
Neither evaluator attended LUC for our undergraduate studies so we do not have any
particular bias that may shape how we analyze the data. If we find that when we receive the
surveys back and we are looking over the results that we feel we have a prominent bias that we
had not formally recognized then we will include this into the evaluation narrative.
Triangulation
In order for the results and data to be as accurately interpreted as possible we will be
using triangulation, which include the use of data collection methods and multiple investigators
33
to collect data (Schuh, 2009). This method will allow us to cross check all the information we
have collected from various sources in order to be as sure as possible when making
generalizations about the experiences of students at LUC. This triangulation will include
crosschecking the surveys, the focus group data and all of our notes and recordings that we
collected along the way.
Limitations
One of the limitations that we may encounter along the way is that we may not interpret
the data as accurately as possible. It is going to take a lot of rigorous checking and rechecking on
our parts to make sure that all of the coding and reading over of data is done with as much
common understanding between us as possible. Misinterpretation of data would be the most
significant threat to validity in our evaluation over-all.
In order to combat this as much as possible, evaluators will be working very close
together in order to make sure that the analysis of our data is done with as much communication
with each other as possible and a strong understanding of our common definitions. Because we
are using descriptive coding this will help to add more codes or make any edits necessary as the
process moves along. It is our hope that this will help with getting caught in the cycle of codes
that do not accurately depict students experiences.
Presentation of Qualitative Results
In order to display the results that we gather in the most clear and effective manor, the
evaluators will incorporate graphs, tables and quotations to display the findings. These results
will show what second-year students are actually experiencing at LUC, and whether the ways in
which the goals of the Loyola Experience Plan are being lived out. The tables and graphs will
also be a clear representation for demographic breakdowns. These results will display how
34
certain groups of students experience their second year at Loyola, and how their identities may
impact their experience. Our hope is that much of the power of the data that we collect from the
qualitative work will be a collection of quotes and voices of students. In the qualitative results,
the quotations will allow the readers to understand the students experiences and some of their
most salient moments. These results will help to illustrate the lived experience of second-year
students at Loyola. Lastly, it will provide a way to see if their time during their second year
matches with the goals and expectations set by the university.
Mixed Methods
This evaluation calls for a mixed methods design, it allows for the most precise and
concrete understanding of the experiences of second year students at LUC. As Taylor and
Trumbull (as cited in Schuh, 2009) assert The skilled researcher can draw the best from both
approaches and combine them. Researchers should not assume that one approach is superior to
the other, or that qualitative data are easier to use. First, the quantitative approach will allow for
survey results to be collected and quantified. These results will garner descriptive data and
inform researchers and stakeholders of averages and how different groups of students
experienced their second year. Next, our quantitative approach will inform us of how to create
our focus groups, a purposeful sampling approach. The focus groups will help to gain a more in
depth understanding of their experiences and help to understand the quantitative data. The
qualitative focus groups will allow second-year students to tell their stories in a more in depth
manor.
Timeline
Because the large size of this evaluation, the timeline from start to finish is quite lengthy
(Appendix K). This process will begin in the spring of 2015 with the first email sent out to
35
students asking them to participate in the survey. There will then be two more email reminders
sent out after that in late April and then in May. During the summer of 2015 all of the data that
was collected through the surveys will be analyzed and preparations will be made for the focus
groups. Then, once the students return back in the fall for the next year of classes the focus
groups will be conducted, this will take approximately three weeks. Once all of our focus group
data has been collected it will be sent out to be transcribed during October and November. The
hope is that with such a large number of students that may participate and be involved in the
focus groups, that the transcribing process will take a long time. Then, once we receive the
transcription data back from our outside hired transcribers we will combine and integrate the data
we have from the surveys (quantitative) and the data from the focus groups (qualitative). This
integration process will be the final culmination of the mixed methods approach. Lastly, from
November 2015 through January of 2016, the stakeholders and evaluators will be involved in the
decision making process and working together to think of how to best move forward with this
assessment.
Budget
The budget for this assessment is on the larger end of the scale because of the number of
students that we hope will participate and therefore generate a large quantity of data (Appendix
J). The personnel, space and campus labs are all free because they can be utilized on LUCs
campus. The gift cards are the incentive piece we will be giving away to the first students who
participate in the survey. The food amount is allocated for the focus group dinners. We have also
set aside $50.00 for materials in order for our focus groups to function. The largest sum of
money we will need is $10,000 for two people to transcribe all of our data from the focus groups.
36
Lastly, we have set aside an additional $500.00 for any costs that may come up as we work
through this assessment.
Next Steps
Moving forward the next steps would be to implement this evaluation at LUC. In order to
do this, this document would need to be presented to the stakeholders at LUC such as Shannon
Howes, Terri Thomas, Michael Beazley and Senior Level Administrators. A conversation
surrounding feasibility would be of upmost importance. Conversations would need to be had
asking if this evaluation plan is possible at LUC or if it would need to be scaled back. During this
conversation, stakeholders would also need to think back to when they last tried to evaluate the
second-year experience and the shortcomings of that plan.
If stakeholders decided that this evaluation plan should be implemented those involved
would need to come up with ways to promote this evaluation to students. As we know from
conversation with Michael Beazley they have had extremely low participation numbers in
evaluations past. This process would involve looking at past promotions that have been done and
think about ways to change that marketing tactic. Much thought and conversation would need to
happen as well as scheduled meetings with stakeholders in order to assure that everyone on
board is on the same page and has a clear understanding of the forward direction of this
assessment.
References
Creswell, J.W. (2104). Research design: Qualitative, quantitative, and mixed method approaches
37
(4th Edition). Thousand Oaks, CA: Sage Publications.

Hartnett, Father D.F,S.J. (2009). Transformative Education in the Jesuit Tradition.
http://www.luc.edu/media/lucedu/mission/pdfs/Transformative%20Education
%20document.pdf.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2011). Program evaluation: Alternative approaches
and practical guidelines(4th Ed.) New York: Longman.
Loyola University Chicago, A1. (n.d.). College mission statement. Retrieved from
http://www.luc.edu/mission/index.shtml
Loyola University Chicago, A2. (n.d.). First and second year advising. Retrieved from
http://www.luc.edu/fsya/index.shtml
Loyola University Chicago, A3. (n.d.) Office of Institutional Research. Retrieved from
http://www.luc.edu/ir/index.shtml
Loyola University Chicago, A4. (n.d.). Second-year experience. Retrieved from
http://www.luc.edu/secondyear/
Loyola University Chicago, A5. (n.d.). Assessment committee. Retrieved from
http://www.luc.edu/studentdevelopment/about/assessment/committee/
Loyola Univeristy Chicago, A6. (n.d.) The Loyola Experience. Retrieved from
http://luc.edu/loyolaexperience/year2/
Loyola University Chicago, B. (n.d.) The Loyola experience [Brochure]. Chicago, IL.
Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:
Jossey-Bass.
Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program
evaluation (Third Edition). Jossey- Bass: San Francisco.
38
Whitt, E. J., & Edison, M. I. (2001). Influences on students' openness to diversity and challenge
in the second and third years of college. Journal Of Higher Education, 72(2), 172-204.
39
Appendix A
Logic Model
Outputs
Inputs
Participants
(Second Year
Students)
Administrators,
Faculty, Staff
Time (Have to fit
all the outcomes
and programs in
one year)
Money, from
both the student
and Loyola
Departments
nd
(Specifically 2 Year Advising,
Second Year
Experience,
Study Abroad,
etc.)
Materials
(Supplies for
large events)
Research
Technologies
Activities
Provide
Workshops
Advise Students
Develop
Curriculum and
Resources
Deliver Services
Create Retreats
Training Faculty
and Staff
Create Spaces for
Group
Conversations
Half Way to
Graduation
Celebration Out
to Lunch with
Faculty
Sophomore
Supers
Second-Year TShirts (Year Two)
Participation
nd
2 -Year
Students
Faculty and
Staff
Parents, Family,
Guardians,
Community
Members
Administrations
International
Community
Members
(Partners at
International
Campuses)
Assumptions
Students are going to buy into this experience

That we have the resources to evaluate the second-year experiences
There is enough time for students too accomplish these goals in their secondyear
That we have the right resources for the needs of second-year students
We are providing the right opportunities second-year students need
Short
Building
awareness of
resources
available for
second-year
students
Understanding
of ones values
and become
more aware of
them
Engagement in
big life questions
What are my
religious/spiritual
beliefs? (if
applicable)
Prepping for
year three (study
abroad,
internships,
Chicago
involvement, oncampus
leadership
positions)
Outcomes -- Impact
Medium
Build reflective
tendencies
Decisionmaking
(academic
major
discernment)
Asking big
questions about
life
Discovering
passions
Goal-Setting
Affinity with the
University
Building bonds
among faculty
and students
Identity and
affiliation with
their graduating
class
Long
Social and
communication
skills
Global
Understanding
Connection to
the University
Following
career goals
Understanding
of ones large
and small scale
communities
External Factors
Faculty and staff interest

World economy
Fmaily Pressures
Student outside jobs/responsibilites
Changing of universities missions, goals, values, and strategic plan
40
Appendix B
Survey Construct Map
Logic Model Component

S-T OC 1
Build awareness of
the resources
available for
second-year students
S-T OC 2
Understand your values and
become more aware of them
S-T OC 3
Enhance engagement in big
life questions
S-T OC 4
Q#
Question
STOC 1.1
STOC 1.2
STOC 1.3
STOC 1.4
I am aware of the University resources available to me during my second-year.

I know of some key departments that can support me in my second-year.
Staff at the University have helped me to become aware of second-year resources.
My faculty have helped me to become aware of second-year resources.
STOC 2.1
My time thus far at Loyola has helped me become better aware of my values.
STOC 3.1
I learned to become more engaged in big life questions.
Gain a better understanding

STOC 4.1
of my religious/spiritual stance
STOC 4.2
S-T OC 5
Prepping for year three
STOC 5.1
(study abroad, internships,
STOC 5.2
Chicago involvement, on-campus STOC 5.3
leadership positions)
M-T OC 1
Build Reflective Tendencies
MTOC 1.1
(Jesuit spiritual exercises, retreats, MTOC 1.2
in-class reflections, and reflection
with mentors)
M-T OC 2
Decision- Making
MTOC 2.1
(academic major discernment,
MTOC 2.2
leadership positions)
MTOC 2.3
M-T OC 3
Discover
MTOC 3.1
Passions
MTOC 3.2
M-T OC 4
Goal-Setting (academic, career,
MTOC 4.1
personal, spiritual and religious goals) MTOC 4.2
M-T OC 5
Create an affinity with the University MTOC 5.1
Loyola has provided the opportunity to deepen my understanding of my faith and the
faith/spirituality of others.
I have used my time thus far to gain a better understand of my religious/ spiritual stance.
I have taken the time in order to prepare for year three.
I know what resources/options are available for me during year three.
I feel prepared for year three academically, spiritually, professionally, and personally.
Loyola has provided me with options to build reflective tendencies.

I have taken the time to reflect on my Loyola experience with others.
I feel that the University has prepared me with the tools I need to make decisions.
My experiences at the University thus far has allowed me to explore all of my options.
The University has allowed me to feel supported during my decision-making process.
Loyola has provided me with opportunities to discover my passions.
I have utilized University resources to help discover my passions.
I feel the goals I have set will help me in the future.
I have utilized University resources to help set personal goals.
I feel connected to the University.
5
Strongly
Agree
Agree
Neutral
Disagree
1
Strongly
Disagree

M-T OC 6
Build bonds among
faculty and
students
M-T OC 7
Identity and affiliation
with the graduating class
L-T OC 1
Build social and
Communication Skills
L-T OC 2
Strengthen global
understanding
L-T OC 3
Build Long-Term
Career Goals
L-T OC 4
Understanding
of your
large and small
scale communities
41
MTOC 6.1
MTOC 6.2
MTOC 6.3
I have build bonds with faculty members at the University.

I have build bonds with staff at the University.
I have build bonds with students at the University.
MTOC 7.1
I feel a sense of belonging with my graduating class.
LTOC 1.1
LTOC 1.2
I feel I have improved my social and communication skills.

My experience thus far, has allowed me to build my social and communication skills.
LTOC 2.1
LTOC 2.2
I am able to think about issues from a global perspective.

I have the ability to identify global matters.
LTOC 3.1
LTOC 3.2
The University has provided me with opportunities to think about long-term career goals.
I feel prepared to set long-term career goals.
LTOC 4.1
LTOC 4.2
LTOC 4.3
LTOC 4.4
I am knowledgeable about my small scale communities.

I am knowledgeable about my large scale communities.
The University has provided me with options to understand my small scale community.
The University has provided me with options to understand my large scale community.
*Short-term (S-T), Medium-term (M-T), Long-term (L-T), Outcome (OC)
42
Appendix C
Survey Email Invitation
First Email
Subject: Request for Participation in the Year Two Loyola Experience Focus Group
Greetings Second-Year Students,
We hope this email finds you all well as the end of the school year comes to a close. Our names
are Caroline Chalker and Tempris Daniels. We are graduate students in the higher education
program at Loyola. We are conducting an evaluation of the second-year experience as a piece of
the Loyola Experience. Our hope is that your feedback will help us connect whether the
milestones, goals, and experiences developed as part of the Loyola Experience for second years
are being met.
We would like to invite each of you to take part in our 10-minute survey where we ask you about
your second-year experience. The first 50 students who complete the survey will be entered into
a raffle for a chance to win one of our four $25.00 gift cards.
Thank you for your time and support. Your participation in this survey is essential in order to
ensure the Loyola Experience is meaningful for all students. If you have any questions, please do
not hesitate to contact Caroline (cchalker@luc.edu) or Tempris (tdaniels3@luc.edu).
Have a great rest of a year!
Caroline Chalker & Tempris Daniels
Reminder Email
Subject: Reminder Email to Participate in the Year Two Loyola Experience Survey
Greetings Second-Year Students,
This is a friendly reminder that you have until (enter date) to complete the second-year survey.
Please keep in mind that your participation in the survey is vital to ensure that we are achieving
the goals set for second year students.
Thank you again for time and support. If you have any questions, please do not hesitate to
contact Caroline (cchalker@luc.edu) or Tempris (tdaniels3@luc.edu).
Cheers,
43
Appendix D
Loyola Experience Second Year Survey
Thank you so much for participating in our Second Year Engagement (Loyola Experience)
survey. The purpose of this survey is to examine the second-year experience at Loyola University
Chicago. Information from this survey will be used to evaluate year two of the Loyola
Experience.
Your response to this survey will be confidential. Names and email addresses will not be attached
to answers. If you are to ever feel uncomfortable at any point of the survey, please know that you
are able to skip or stop the survey at any time. If you have any questions, feel free to contact
Caroline Chalker (cchalker@luc.edu) or Tempris Daniels (tdaniels3@luc.edu).
In this survey, please rate each question from 1-5, 1 meaning strongly disagree and 5 meaning
strongly agree.
44
Thank you for taking the time to fill-out our second-year engagement (Loyola Experience)
survey. In an effort to understand second-year students needs, we have created a short survey.
The survey should not take more than 10 minutes to complete. We thank you for taking the time
to complete the sruvey and we appreciate your honest feedback.
45
46
Please fill out a few questions surroundings ones understanding of faith and religious stance
based on your second-year engagement.
Please take the time to fill out questions surrounding your decision-making processes and bonds
built during your second-year engagement.
47
48
49
50
Appendix E
FOCUS GROUP EMAIL INVITES
Initial Invitation Email: Focus Group
Subject: Request for Participation in Loyola Experience Focus Group
Greetings Loyola Students,
We hope this email finds you well. We would like to thank you for participating in our Loyola
Experience Second-Year Survey. As we dig deeper into our results we are looking to hear our
students stories as these relate to their second-year experience. Our hope is that your feedback
will help us understand second-year students experiences after participating in various Loyola
Experience programming.
We would like to invite each of you to take part in our 90-minute focus group on (give three
DATES & TIMES) where we ask you about your second year experience. With dinner
provided, we are asking for two hours of your time, with the first 30 minutes dedicated to
dinner.
If you would like to participate, we ask that you reply to this email indicating which time best
works for you. Once we receive confirmations from all by (DATE), you will receive a follow-up
email with location, time, and date of your focus group, which will be held on Lake Shore
Campus.
Thank you for your time and support. Your participation in this focus group is appreciated in
order to ensure the Loyola Experience is meaningful for all students. If you have any questions,
please do not hesitate to contact Caroline (cchalker@luc.edu) or Tempris (tdaniels3@luc.edu).
Cheers!
Follow-Up Email: Focus Group
Subject: Follow-Up Focus Group Reminder
Greetings Loyola Students,
We hope that this email finds you well. We are very excited that you have decided to participate
in our focus group where we will converse about your experiences as second-year students. We
want to remind you that your focus group will take place on the Lake Shore Campus on (fill in
date) at (fill in time).
Thank you again for your time and input. If you have any questions, please do not hesitate to
contact Caroline (cchalker@luc.edu) or Tempris (tdaniels3@luc.edu).
Cheers,
51
Appendix F
FOCUS GROUP DEMOGRAPHIC FORM
Date _______________
Please complete the form below. You may choose to answer any of the items, as you feel
comfortable. If you have any questions, please ask your focus group leaders.
52
53
Appendix G
Consent Form for the Second-Year Experience Focus Group
You have been asked to participate in a focus group for Loyola University Chicagos SecondYear Loyola Experience program. We would like to thank you for volunteering to take part in
this focus group. The purpose of this focus group is to gather information about your experience
at LUC during your second year. Please take some time to read through this form. This form
contains pertinent information about your consent to participate in this focus group. If you have
any questions about the content of this consent form please ask one of the focus group leaders.
Once you have signed this form and agreed to participate, you will be involved in our group
conversation where we will answer questions surrounding your experience as a second-year
student. If at any time you feel uncomfortable answering any of the questions, you are not
required to do so. Please know that throughout this focus group there is no one right answer. We
are simply looking to hear your stories and understand your experience. Your participation in this
focus group is entirely voluntary. If at any time you would like to withdraw you are allowed to
do so without penalty.
In order to secure confidentiality as much as possible your name will not be used in any of our
reporting. Any information that you share with this group today will not be discussed outside of
our evaluation team. Furthermore, we ask that participants in this focus group do not share
anything they hear from the group members outside of this space.
This focus group session will be audio recorded in order to be as accurate as possible throughout
the evaluation process. This evaluation team will only use the recordings for purposes of this
evaluative process.
There is low risk involved in participating in this focus group. Although participants will agree to
keep what is shared within this group, they could break confidentiality and discuss with others
outside of the group. The benefits to participation include a deeper understanding and reflective
time with other students surrounding your experience as a second year student at Loyola
University Chicago.
54
If you find you have any questions about this focus group, feel free to contact Caroline Chalker
at cchalker@luc.edu or Tempris Daniels at tdaniels3@luc.edu. If you would like a copy of this
form please ask one of the facilitators and they will gladly make you a copy.
If you agree to the above terms, please sign and date below.
__________________________
________________________
____________
Participants Signature
Participant Name
Date
__________________________
________________________
____________
Evaluator Signature
Evaluator Name
Date
55
Appendix H
THIRD- YEAR EXPERIENCE FOCUS GROUP PROTOCOL
Focus Group Purpose: To grasp additional knowledge and insights of the second-year student
experience through student stories. The focus group will take place for second-year students
during the first week of their junior academic year.
Focus Group Topics: Reflection of the Most Important Parts of your Second-Year
Declaring a Major
Developing of Leadership Skills and Engagement on Campus
Exploration of Service Learning Opportunities
Exploration of Faith/Social Justice Opportunities
Critiques/ Suggestions
Script
Welcome [5 minutes]
Welcome everyone and thank you for joining us for our third-year focus group. My name is
[moderator name] and this is [name of assistant moderator}, my assistant. Tonight I will be
guiding us through the conversation and my assistant will be taking notes and recording our
conversation. The purpose of this dialogue is to gain an understanding of your experiences as a
second-year student in the final weeks of the academic year that involves participates to look
back at their entire second-year and not just the end of it. Our focus is to understand your
thoughts, personal experiences, and significant occurrences during your second year. This focus
group will take place for 90 minutes. During this time, our hope is that everyone will contribute
to the conversation.
It is important to us that you all know that anything shared in this focus group should remain
confidential among focus group members. In order to allow each member to fully engage in the
dialogue, we hope that everyone will be mindful of the contributions of those in the room. We
will not be using any names or identifiers when looking back at or reporting on the focus group
conversation. At this time, we would ask everyone to read and sign the consent form. Are there
any questions at this time? [Wait and Collect Consent Forms]
Todays conversation will cover the following themes related to your time as a second year
student. First, we would like to gain a better understanding of the important parts of your secondyear. Next, we will discuss the experience of declaring a major. Third, we will talk about the
opportunities to develop leadership skills and engagement on campus. Fourth, we want to learn
about the chance to explore service-learning opportunities. Our last topic is centered-around
exploring ones faith/social justice opportunities. At the end, we will discuss different critiques
and suggestions to improve the second-year experience at Loyola University Chicago.
At this time we would like for everyone to introduce himself or herself, tell us your declared
majors, and one interesting fact about yourself. After we complete introductions we will have
dinner and continue with our discussion.
56
Reflection of the Most Important Parts of your Second-Year [10 minutes]

1. Can you please share with us some of your most significant parts of your second-year here at
Loyola?
Probe: For what reasons were these experiences significant to you?
Probe: Were there any key people other than you involved in these experiences?
2. How would your second-year look if you had not had those significant experiences?
3. Did you expect to have these experiences in your second year? Why or why not?
Declaring a Major [15 minutes]
1. What was the process like to declare your major?
Probe: Tell us about the process you took to explore the major you chose at the
end of your second-year?
Probe: Tell us about the process you took to commit to the major you chose at
the end of your second-year?
2. How has your various coursework influenced your major exploration process?
Probe: Did faculty take a role in influencing your process in the classroom? If so, can
you provide an example?
Probe: Did peers take a role in influencing your process in the classroom? If so, can you
provide an example?
3. What, if any, out-of-class experiences informed your major exploration processes?
Probe: What insights in particular did these experiences provide that related to your major
exploration?
Developing of Leadership Skills and Engagement on Campus [15 minutes]
1. In what ways, if at all, have you been involved on campus?
Probe: Im wondering if you can share more about your choices to get involved in these
opportunities.
2. In what ways, if at all, have you taken on leadership roles (formally or informally) in your
involvement efforts?
opportunities.
3. Are you aware of any leadership skills you have developed?
Probe: Which skills in particular have you developed?
Exploration of Service Learning Opportunities [15 minutes]
1. For what reasons did you or did you not participate in service learning experiences?
2. How have you changed, if at all, through your participation in any service learning
opportunities?
Probe: What aspects of the service learning experiences attribute to your changes?
Exploration of Faith and Social Justice Opportunities [15 minutes]
1. For what reasons did you or did you not participate in faith/religious experiences?
2. Did you participate in any social justice related activities during your time as a second year
student?
57
Probe: How have you changed, if at all, through your participation in any social justice
related activities?
Probe: What aspects of your faith and social justice opportunities attribute to your
changes?
Critiques/ Suggestions/ Conclusion [10 minute]
1. As a student, did you feel you missed out on any opportunities during your second-year?
2. Is there any other type of second-year programming you wish you would have know about or
participated in?
3. Does anyone have any final reflections/insights that they did not yet share related to the
second-year experience?
Thank you all so much for being engaging and sharing your experiences, perspectives, and
thoughts with us. We appreciate the time everyone took out of your day. This focus group will
help us better understand the experiences second-year students are having and the ways we can
make student experiences fuller.
58
Appendix I
FOURTH-YEAR EXPERIENCE FOCUS GROUP PROTOCOL
Focus Group Purpose: To reflect on the lived experiences of second year students as they
embark on their third academic year
Focus Group Topics: Reflection of the Most Important Parts of your Second-Year
Declaring a Major
Developing of Leadership Skills and Engagement on Campus
Exploration of Service Learning Opportunities
Exploration of Faith/Social Justice Opportunities
Critiques/ Suggestions
Focus-Group Script
Welcome [5 minutes]
Welcome everyone and thank you for joining us for our fourth-year focus group. My name is
[moderator name] and this is [name of assistant moderator}, my assistant. Tonight I will be
guiding us through the conversation and my assistant will be taking notes and recording our
conversation. The purpose of this dialogue is to gain an understanding of your experiences as a
second-year student as you embark on your fourth academic year where you all will have the
chance to look back at your entire second-year. Our focus is to understand your thoughts,
personal experiences, and significant occurrences during your second year. This focus group will
take place for one hour and half. During this time, our hope is that everyone will contribute to the
conversation.
It is important to us that you all know that anything shared in this focus group should remain
confidential among focus group members. In order to allow each member to fully engage in the
dialogue, we hope that everyone will be mindful of the contributions of those in the room. We
will not be using any names or identifiers when looking back at the focus group conversation. At
this time, we would ask everyone to read and sign the consent form. Are there any questions at
this time? [Wait and Collect Consent Forms]
Todays conversation will cover the following themes related to your time as a second year
student. First, we would like to gain a better understanding of the important parts of your secondyear. Next, we will discuss the experience of declaring a major. Third, we will talk about the
opportunities you had to develop leadership skills and engagement on campus. Fourth, we want
to learn about the chances you had to explore service-learning opportunities. Our last topic is
centered-around exploring faith and social justice opportunities. At the end, we will discuss
different critiques and suggestions to improve the second-year experience at Loyola University
Chicago.
At this time we would like for everyone to introduce himself or herself, tell us your declared
majors, and one interesting fact about yourself. After we complete introductions we will have
dinner and continue with our discussion.
Reflection of the Most Important Parts of your Second-Year [10 minutes]
59
1. Can you please share with us some of your most significant parts of your second-year here at
Loyola?
Probe: For what reasons were these experiences significant to you?
Probe: Were there any key people other than you involved in these experiences?
2. How would your second-year look if you had not had those significant experiences?
3. Did you expect to have these experiences in your second year? Why or why not?
Declaring a Major [10 minutes]
1. What was the process like to declare your major?
Probe: Tell us about the process you took to explore the major you chose at the
end of your second-year?
Probe: Tell us about the process you took to commit to the major you chose at
the end of your second-year?
2. How has your various coursework influenced your major exploration process?
Probe: Did faculty take a role in influencing your process in the classroom? If so, can
you provide an example?
Probe: Did peers take a role in influencing your process in the classroom? If so, can you
provide an example?
3. What, if any, out-of-class experiences informed your major exploration processes?
Probe: What insights in particular did these experiences provide that related to your major
exploration?
Developing of Leadership Skills and Engagement on Campus [15 minutes]
1. In what ways, if at all, have you been involved on campus?
opportunities.
2. In what ways, if at all, have you taken on leadership roles (formally or informally) in your
involvement efforts?
opportunities.
3. Are you aware of any leadership skills you have developed?
Probe: Which skills in particular have you developed?
Exploration of Service Learning Opportunities [15 minutes]
1. For what reasons did you or did you not participate in service learning experiences?
2. How have you changed, if at all, through your participation in any service learning
opportunities?
Probe: What aspects of the service learning experiences attribute to your changes?
Exploration of Faith/Social Justice Opportunities [15 minutes]
1. For what reasons did you or did you not participate in faith/religious experiences?
2. Did you participate in any social justice related activities during your time as a second year
student?
Probe: How have you changed, if at all, through your participation in any social justice
related activities?
60
Probe: What aspects of your faith and social justice opportunities attribute to your
changes?
Critiques/ Suggestions/ Conclusion [10 minutes
1. As a student, did you feel you missed out on any opportunities during your second-year?
2. Is there any other type of second-year programming you wish you would have know about or
participated in?
3. Does anyone have any final reflections/insights that they did not yet share related to the
second-year experience?
Thank you all so much for being engaging and sharing your experiences, perspectives, and
thoughts with us. We appreciate the time everyone took out of your day. This focus group will
help us better understand the experiences second-year students are having and the ways we can
make student experiences fuller.
61
Appendix J
Budget
BudgetItem
Personnel
Space
Food
Incentives(Giftcards)
MaterialsNeeded for FocusGroup
CampusLab
Two Individualsto Transcribe
Miscellaneous
Total
Price
Free
Free
$1,000.00
$100.00
$50.00
Free
$3,500.00
$500.00
$5,150.00
62
Appendix K
Timeline
TimelineSteps
1st Email Distribution Sent
Reminder Email Distribution Sent
Final Email Distribution Sent
DataAnalysisof SurveyInformation
FocusGroups
DataAnalysisof FocusGroups(Transcription)
DataReporting
MakingDecisions
Date
April 1st 2015
End of April 2015
Mid May2015
June-August 2015
First three weeksof September 2015
October-November 2015
November 1st 2015
November 2015-January2016
PersonPerformingtheTask
Evaluators
Evaluators
Evaluators
Evaluators
Evaluators
Transcribers
Evaluators
Stakeholders& Evaluators
63
Appendix L
PowerPoint Presentation
64
65
66
67
68

Research Paper

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Research Paper

Încărcat de

Drepturi de autor:

Formate disponibile

Running head: THE LOYOLA EXPERIENCE

Program Evaluation for The Loyola Experience: Year Two

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

Program Evaluation for The Loyola Experience: Year Two

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

Earn at least 60-plus credit hours.

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

A formative design, with a combination approach, will allow us to understand what is

THE LOYOLA EXPERIENCE

Population and Sampling Frame

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

(4th Edition). Thousand Oaks, CA: Sage Publications.

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

Students are going to buy into this experience

Faculty and staff interest

THE LOYOLA EXPERIENCE

Logic Model Component

I am aware of the University resources available to me during my second-year.

I learned to become more engaged in big life questions.

Gain a better understanding

Loyola has provided me with options to build reflective tendencies.

THE LOYOLA EXPERIENCE

I have build bonds with faculty members at the University.

I feel a sense of belonging with my graduating class.

I feel I have improved my social and communication skills.

I am able to think about issues from a global perspective.

I am knowledgeable about my small scale communities.

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE

THE LOYOLA EXPERIENCE