Program Evaluation:

ONE MILLION DEGREES

By: Courtney Bishop and Lillianna Franco
Loyola University Chicago

One Million Degrees

(OMD)
Developed in 2006
Helped 25 scholars during their community college years and after they transferred to
a four year college.
Transitioned to helping students only at the community college

Mission
To empower low-income, highly motivated community college students to succeed
in school, in work, and in life.

Qualifications

Be Pell or Star Scholarship eligible

Have at least a 2.0 GPA entering the program and maintain a 2.5 GPA
Be pursuing your first college degree
Be enrolled full-time at an OMD partner community college and graduate no earlier
than Spring 2017

OMD Partner Community Colleges

The 7 City Colleges of Chicago, College of Lake County, Harper College, Prairie State
College, and South Suburban College

OMD Support Services

Academic
Weekly tutoring, meet with academic advisor once a
semester, and create and review an academic plan

Financial
Last dollar scholarships, stipends, and financial literacy
workshops

Personal
Program coordinator assists students through various
challenges and students participate in hands-onlearning about social justice issues

Professional
Gain professional development skills through workshops
and events

Learning Outcomes
Short term

Medium term

Long term

Become familiar with OMD and the resources offered

Comfort with OMD program coordinators and bond with
cohort
Become familiar with community college resources offered
Understanding their academic plan

Become knowledgeable on financial literacy

Able to maintain good academic standing at
community college
Manages all OMD workshops, tutoring sessions, and
individual and small group coaching sessions

Ability to persist from year to year

Completion of Associates degree or transfer to 4 year
college

Our Evaluation Focus

Does the OMD program provide quality and effective
services and does the program have an impact on students
GPAs?
As the program increases its number of scholars there
needs to be a constant need of evaluation to ensure that
the program is providing quality services and benefiting
the students.
GPAs were seen to drop at some of the community
college partners
We will review each community college and administer
surveys and conduct focus groups to find similarities and
differences in the program services offered and see if
there is a correlation to the students GPAs
Or are the students GPAs dropping because of external
factors?
Once the answer is determined through analysis, the
program will take steps to modify the program or provide
additional support services to students

Quantitative Approach
Stratified Random Sampling
Will sample from each community college partner
Include first year, second year, GPA cut offs, and other demographics,
such as sex and race

Paper and Pencil survey

To be completed every semester while student is in the program to show
changes in GPA or quality of services over time
36 questions to be completed in 15 minutes

Analysis
Descriptive analysis-calculate the means of all GPAs
Paired samples T-test- did their grades change over the semesters?
ANOVA-are there differences in GPAs across the different campuses
and other demographics?
Multiple regression- will see if students participation in OMD services
predicts their GPA

Qualitative Approach
Stratified Purposeful sampling
Purposefully sampling will allows our analysis to be diverse but still
focused on categories we find valuable
Categories of GPAs, year in OMD program, and by each community
college

Focus Groups
11 focus groups for each community college partner
Each focus group follows the same protocol that gives students consent
forms and an understanding of the benefits and risks for participating
Duration will last an hour to an hour and half

Analysis
Transcribe all audio recording from the focus groups
Transcripts will be coded using themes related to the topics being
addressed

Limitations
Mortality- participants who drop out during the evaluation
process due to many possible reasons and the outcomes are
thus unknown for these individuals
Researchers presence may bias responses
Students might not answer honestly or feel uncomfortable to
answer
Personal bias and past experiences may influence the
coding process for the evaluators
Not all demographics are included at the time of sampling
One moderator may be overwhelmed with the focus groups
and transcriptions

Timeline

Budget

Next Steps
Present results to appropriate stakeholders and
focus on results that need improvement to
improve students GPA
Give survey and implement focus group
evaluation each year (once a semester) to
collect data that connects to graduation and
transfer rates
If trend is found, we highly recommend the
OMD program remove programming that
hinders student success and improve
programming that helps student success