Documente Academic
Documente Profesional
Documente Cultură
This report aims to study the employability of ASAP trained students in comparison to the
employability of untrained students and thereby understand the impact of the ASAP intervention
on employability of its students. The impact was assessed by comparing the performance of two
groups of students: the control group and the experimental group. The experimental group
comprises of students who have undergone training during ASAP batch of 2014-15 while the
control group comprises of students who did not go through this training but were shortlisted for
selection interviews of ASAP batch 2014-15 and yet were not enrolled. The profile of these
students in terms of socio-economic background is most likely to be similar to those students who
were enrolled under ASAP since their application forms were scrutinized to confirm eligibility to
the selection criteria of ASAP. ASAP training spanned across 30 different courses. Of these thirty
training courses, nine had insufficient sample size in the control group. The similar courses were
clubbed together– Mutual Fund Agent was clubbed with Financial Advisory and Marketing
Services, Plumbing Apprentice Programme was clubbed with Certificate Course in Plumbing
Apprentice Programme and Certificate in Retail Sales Associate was clubbed with Certificate
Course in Retail Sales Associate. Only the remaining three courses were not considered for the
analysis. These courses include Certificate in BPO non-voice, Certificate course in basic level
automation technology and Certificate course in certified warehousing expert.
In order to carry out this employability study, ASAP partnered with Aspiring Minds to use their
flagship product, AMCAT to carry out the assessments of the students. Aspiring Minds is India’s
largest employability solutions company that offers scientific assessments with an innovative
large-scale sourcing model analogous to a GRE-for-job concept. The state-of-the-art assessment
tools developed by Aspiring Minds have been used across industry verticals to help recruit the
right people, develop profile-wise employability benchmarks and assess workforce health.
Aspiring Minds’ intelligent adaptive assessments span across Language, Cognitive skills, Domain
Knowledge and Personality.
2
Aspiring Minds Assessment Suite – AMCAT was administered on a total of 6681 applicants. Out
of the 6681 candidates, 4206 belong to experimental group and the remaining 2475 belong to the
control group. The control group candidates were asked to pick a training course which they would
have liked to undertake. Candidates from above mentioned 3 courses with insufficient sample size
are not considered for analysis. Therefore, henceforth in the report, 3932 candidates belong to
experiment group and 2418 belong to control group. Each candidate was assessed on both generic
as well as domain specific skills. The generic skills included English Comprehension, Information
Gathering and Synthesis and Quantitative Ability. The domain specific skills varied from course
to course and it checked for the aptitude and tacit knowledge required by the candidate to perform
the job related to course undertaken. In addition, the personality profile of the candidate was also
gauged using a psychometric personality tool.
This report provides a methodology to objectively calculate Employability Index (EI) for each
candidate with the intent to study the impact of ASAP in enhancing employability of its students.
For further insights, mean score analysis, shortlist rate and skill gap analysis are included.
3
2 EXECUTIVE SUMMARY
Key findings of the present study are as follows:
Experiment Group has higher shortlist rate than Control Group across all training courses
except Basic Certificate Course in Community Nursing and Palliative Care
Candidates who underwent training at ASAP have higher shortlist rate than those who did not.
Basic Certificate Course in Community Nursing and Palliative Care is the only training course
where Control Group has a shortlist rate of 40% whereas it is 34% for Experiment Group. Overall
it is observed that training has a positive impact on job candidacy. Highest Shortlist Rate is
observed in Certificate Course in Automotive Machining Technician across Experiment Group
and Basic Certificate Course in Community Nursing and Palliative Care across Control Group.
Males & Females have similar shortlist rate across training courses. However, small but significant
difference is observed in Employability Index (EI). Males have higher shortlist rate in courses like
accounting, insurance & welding whereas females are more employable in data entry & customer
care services. Largely, gender doesn’t serve as strong differentiating criteria.
ASAP training curriculum satisfies the industry requirement to a large extent with few areas
of improvement
Based on skill gap analysis, we observed low level of further intervention is required in improving
English communication skill. However, there is some scope of improvement in quantitative ability
and information gathering and synthesis. For quantitative ability, 4 courses require high-level
intervention and 8 require mid-level intervention. For information gathering and synthesis, 17
courses require mid-level intervention and 3 require high-level intervention. The extent of training
required is determined basis the skill-gap percentage in the respective course. Those skills where
more less than 50% of the students possess a particular skill, the level of intervention required is
high in order to train them on that specific skill. For those skills where the percentage of students
who possess the skill is greater than 50% but less than 70%, mid-level interventions are needed.
4
3 TEST INSTRUMENT
The candidates were assessed on these four key aspects that are required in any job: Language
Skills, General Aptitude, Domain Aptitude, and Personality. Aspiring Minds suite of assessments
were used to gauge these aspects. The following sections elaborate further on the test composition.
Language Skills
In today’s world, communication in the English language has become important part of most jobs
and employees require this skill irrespective of the sector in which they work. It is only the degree
of importance that varies from one job to another. A retail salesperson might need to know English
to be able to read the names of the products that he/she is selling. On the other hand, IT personnel
need to know English because his job would involve working on the computer and most of the
information on the internet is in English. Hence, we have considered English Comprehension test
for this study.
The English test uses a variety of internationally standardized resources for framing questions
aimed at determining the candidate’s ability to understand (a) the written text and (b) communicate
effectively through written documents.
General Aptitude
In any job, while the skill to perform the job is critical, a lot of times, candidates are expected to
learn the skill on the job. In order to be able to learn the skills, the candidates ought to have basic
cognitive ability so that they can be trained easily. Meta research has proven that trainability of
employees is directly linked with their cognitive ability. The cognitive ability of the candidates is
measured through two tests: Quantitative Ability and Information Gathering and Synthesis.
5
The Quantitative Ability assesses whether the candidate has understanding of basic number
system, i.e., fractions, decimals, negative, positive, odd, even numbers, rational numbers, etc. The
candidate should know how to do basic operations on these numbers (addition, subtraction,
multiplication, division), understand concepts of factors/divisibility and have good practice on
algebra.
Apart from operations on numbers, the candidate should know how to convert a real-world
problem into equations, which could be solved to find an unknown quantity. The candidate is
tested on word problems representing various scenarios to assess the same.
The Information Gathering and Synthesis test measures a candidate’s ability to gather, comprehend
and evaluate information from single or multiple source(s). It checks the candidate’s ability to
locate relevant information, order and classify data, interpret graphs, charts and tables and to make
rule based deductions.
Competencies assessed:
- Can do basic counting, identification and ordering of objects/numbers as stated.
- Can compare, categorize, match, segregate and verify objects/symbols based on basic
rules. Can do actions based on simple 2-3 line instructions.
- Can collate, comprehend and evaluate information such as detailed instructions with
textual/numerical/flowchart based rules, etc. May show some ability to read graphs,
tables and figures.
- Can collate, comprehend and evaluate information from multiple sources including
graphs, figures, tables and texts to make objective inferences and determine the right
action.
Domain Aptitude
While candidates may possess the basic aptitude i.e. the required logical ability and mathematical
skills, they may not be able to apply them in context of their job. Hence, they could be slow learners
on the job. In order to ensure that the candidates are able to understand their job specific situations
and make informed decisions while performing their tasks, it is important to measure their domain
aptitude. This was tested through different combinations of tests based on the course of the
candidate. They largely involved situational questions in context of the job that one would take up
post the training in that course. The detailed list of the tests administered the candidates is given
in the below table.
6
Training Course AMCAT Domain Module 1 AMCAT Domain Module 2
Certified Banking and Finance Financial Products/ Services Situation
Data Recording
Professional Test
Certificate Course in Business
Banking Service Situation Test Data Recording
Correspondent
Certificate Course in Banking
Banking Service Situation Test Data Recording
Correspondent and Business Facilitator
Certificate in Accounting Technicians MS Excel Data Recording
Global Business Foundation Skill Course Customer Service Situation Test Basic Computer Literacy
7
Personality
The candidate’s personality could play a critical role in any job. A lot of times, we observe that
candidates possess the required technical skills and are able to clear the technical selection rounds
easily. However, they fail to qualify for the job in the HR round or the Behavioural round. This is
because of issues in their personality fit for the job under consideration. Hence, we decided to
measure personality of the candidates using Aspiring Minds Personality Inventory (AMPI). AMPI
is uniquely constructed to remove cultural and linguistic biases and is ideal for developing
economies. This tool is based on the universally acclaimed Big Five model. The five traits that are
measured in this are: Agreeableness, Conscientiousness, Emotional Stability, Extraversion and
Openness to Experience. We have used a shorter version of AMPI that measures only the first
three traits. In terms of performance, Conscientiousness1 is the strongest predictor across all job
types followed by Emotional Stability. For jobs with strong interpersonal component,
Agreeableness becomes more desirable for predictive performance2. Hence, these three personality
traits were studied. Extraversion and Openness to Experience were not included because they were
not considered as important for the jobs under consideration.
Conscientiousness(C) - It has two components: striving for achievement and dependability. The
former is related to hard work and perseverance. The latter is characterized by being organized
and responsible.
Emotional Stability (ES) - Emotionally stable people are even tempered and relaxed. They can
face stressful situations without getting upset.
Agreeableness (A) - It refers to social conformity, friendliness, compliance and altruism.
Agreeable people are sympathetic to others, help others and trust others to help them too in return.
1
http://www.scontrino-powell.com/2014/personality-and-job-performance-a-review/
2
Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in
organizational settings. Personnel Psychology, 60, 995-1027.
8
4 APPROACH AND METHODOLOGY
The aim of the report is to calculate employability index for each candidate to further carry out
comparative studies between the candidates in the ASAP group (experimental) vs. the candidates
in the non-ASAP group (control).
Employability Index (EI) was calculated to measure the employability of the candidate in the
selected training course. It is a percentile score and hence it is on a scale of 0-100.In order to arrive
at the Employability Index (EI), we calculated an aggregate score from the individual test scores.
The EI is the percentile of the candidate’s aggregate score with reference to the control group.
Post administration of AMCAT, we obtained scores for all candidates on English, Quantitative
Ability, Information Gathering & Synthesis and domain tests. The scoring scale for all the tests is
0-100. The aggregate score is a weighted sum of the individual test scores. Below are the steps of
how we arrived at the weights for the individual test scores.
We used a combination of primary as well as secondary research for the same. We used O*NET
database as a guidance and involved subject matter experts to arrive at the final weights for each
score
The Occupational Information Network (O*NET) is a free online database that contains
hundreds of occupational definitions to help students, job seekers, businesses and workforce
development professionals to understand today's world of work. The O*NET mapping is
internationally3 accepted by various organisation and institutions.
For any given job role, O*NET provides a list of knowledge, skills and abilities (KSAs) along with
the relative importance of the same in the job. For example, refer to the job role of Food and
Beverage Service - Steward: http://www.onetonline.org/link/details/35-3031.00
From the given list of KSAs, we look for the importance of the skills that we have tested using
AMCAT. For Food and Beverage Service - Steward, the corresponding values of the importance
are given below:
Based on this, the relative importance of the test scores are as below:
Food and Beverage > Information Gathering and Synthesis > English > Quantitative Ability
This information was used as a premise to conduct the Delphi method of arriving at the weights
for each of the tests. A panel of 4-5 subject matter experts (working professionals in this particular
3
https://www.onetcenter.org/dl_files/paw/Products_at_Work.pdf
9
course) was asked to assign weights to each test score basis their relative importance percentages
as obtained from O*NET. The final weights were based on the consensus of the panel. The weights
should add up to a total of 100. In the current example, the weights were finalized as given below.
Further the Employability Index was calculated as the percentile of the candidate when we
considered the Control Group as the norm group. The mean (µ) and standard deviation (σ) of the
aggregate scores of the Control Group are used as the benchmark to determine the Employability
Index (EI) for each candidate.
where
x: aggregate score
µ: mean of aggregate score for a training course
σ: standard deviation of aggregate score for a training course
In MS-Excel, this formula translates to:
NORMDIST(x, µ, σ, cumulative)
In addition to the employability index, each candidate was classified as either ‘Qualified’ or ‘Not
Qualified’ for the job that he/she could pursue after the respective course. In order to qualify for
the job, the candidate had to clear a minimum threshold cut-off in each test. While we could have
set a threshold on the overall employability index, it would not have been a true representation of
whether the candidate possessed all the required skills. A candidate may have a high employability
index because of very high scores in three of the four skills required and might have scored poorly
in the fourth skill while that is also important and required for the job. Hence, applying cut-offs on
10
individual skills is a better approach to determine whether a candidate has qualified or not for the
job.
The hiring benchmarks i.e. the cut-offs on the individual tests are established by Aspiring Minds
for different entry level job profiles. This is achieved through an algorithm called the Particle
Swarm Optimization technique. This technique is based on the premise that both Type I error and
Type II error are minimized in the hiring process for a given job profile. Type I error refers to the
selection of bad candidates and Type II error refers to the rejection of good candidates. The aim is
to arrive at an ideal set of cut-offs such that both elimination of unsuccessful candidates for a job
(elimination of type I error) and inclusion of all candidates who will be successful in the given job
(elimination of type II error) are taken care of. However, it is impossible to achieve complete
elimination of both the errors. If you use relaxed cut-offs to ensure all good candidates are selected,
there is a possibility for a few bad candidates also to be selected. If you use stringent cut-offs to
restrict the selection of bad candidates, a few good candidates will also be rejected. Hence these
two errors are inversely related. With the decrease in Type I error, the probability of Type II error
increases, and vice-versa. The terms ‘good’ and ‘bad’ used in this paragraph signifies the
candidates’ employability or on-job performance. For those job profiles, where the cut-offs did not
exist readily with Aspiring Minds, the Delphi method described earlier was used wherein the same
panel of experts provided their best estimates of the threshold score on each test and a consensus
approach was taken to arrive at the final cut-offs.
Sample cut-offs used for the role of Food and Beverage Service – Steward are given below.
Food & Beverage Situations: 37.5
Information Gathering & Synthesis: 37.5
English: 37.5
Quantitative Ability: 25
On applying cut-offs on the individual skills, the given candidate does not clear the minimum
threshold for Information Gathering & Synthesis and hence is classified as ‘Not Qualified’ for
given job role.
Course specific cut-offs are used to determine the Shortlist Rate for that particular course. It is
defined as the number of candidates who cleared all the individual threshold cut-offs divided by
the total number of candidates. Shortlisted candidates are likely to perform better on job of their
respective course. Shortlist Rate is determined for various demographics such as gender, region,
courses etc. The candidates with scores more than applied cut-offs in all tests are considered
qualified, while the rest are not-qualified.
11
For instance, the total number of candidates in Experiment group of Food & Beverage Service is
171. Out of these, 77 candidates cleared all the individual cut-offs. Therefore, the shortlist rate is
45%.
Further skill gap analysis was carried out as part of this study. This is an analysis carried out for
each skill individually. A skill gap exists if the candidate does not meet or exceed the required
threshold/cut-off in that particular skill. Hence, a candidate can have a skill gap in one or more
skills. In this study, we have calculated the percentage of students who meet the required threshold
score in a particular skill in the experimental group and accordingly advised on the level of training
need. The level of training intervention required is a function of percentage of students above the
given threshold.
For example, in Food and Beverage Services, out of 171 candidates in the experiment group, 114
clear the individual cut-off of Quantitative Ability, i.e., 67% candidates meet the required threshold
criteria in Quantitative Ability. Hence, a mid-level training intervention is required for this
particular skill in the Food and Beverage Services course.
To further elaborate as to how the skill gap is different from shortlist rate, consider this example.
A person needs to meet threshold cut-off in four different skills in order to be qualified. If he/she
does not meet the cut-off in one or more skills, he/she has a skill gap for those respective skills
and also does not qualify for the job and hence not shortlisted. In order for a candidate to be
shortlisted, he/she needs to clear the cut-offs in all the required skill tests and hence he/she would
not have any skill gap in any of the individual skills/competencies.
Statistical significance of scores obtained by the control group and the experiment group is
substantiated by running t-test (assuming unequal variances).
Steps:
i. The first step is to assume a null hypothesis which will state that the two groups
being tested have no statistically significant difference. Hence, the alternate
hypothesis will state that there is significant difference. Also, an alpha level is
chosen, the default alpha is 0.05. This means that there is 95% confidence that the
conclusion of this test will be valid.
ii. The average scores of experimental group and control group are calculated. The t-
test assesses whether the average scores of two groups are statistically different
from each other. The unequal variances are considered because two mutually
exclusive set of candidates are compared with one another.
12
iii. Formula:
iv. The output of t-test gives tobt, t-critical and a significance p-value. If the calculated
tobt is greater than the t-critical, the test concludes that there is a statistically
significant difference between the two groups. The p-value of 0.05signifies that the
null hypothesis can be rejected with 95% confidence.
13
The following tables draw comparisons between control and experiment group across different
parameters:
Shortlist Rate of Experiment Group is noted to be higher than that of Control Group. This
difference is statistically significant with 99% confidence.
The Experiment Group has a significantly higher mean EI than the control group which is
statistically substantiated with 99% confidence.
Mean Scores
Personality Trait P-value
Control Group Experiment Group
Conscientiousness 38 50 <0.01
Agreeableness 41 52 <0.01
The students in the Experiment Group are on an average more conscientious and agreeable than
the students of the control group. This difference is statistically substantiated with more than 99%
confidence. However, there is no difference in the emotional stability levels of the students in the
two groups.
14
6 EMPLOYABILITY INDEX ANALYSIS
This section shows the break-up of Employability Index by training courses across the Control and
Experiment Group and the difference between them. The values of EI and the difference have been
rounded off to the closest integral value.
Certificate in Insurance
114 47 52 64 17 <0.01
Marketing
Certificate Course in Fashion
88 49 56 73 24 <0.01
Retail Program
Certificate Course in Customer
86 48 196 70 22 <0.01
Care Executive
15
Control Group Experiment Group
EI
Training Course Sample Sample P-value
EI EI Difference
Size Size
Certificate in Food and Beverages
Service- Steward 79 49 171 71 21 <0.01
The Experiment Group is seen to have outperformed the Control Group in all courses except in
Certificate Course in 3D Modelling and Texturing for Animation Film Making. However, we are
less confident stating the same in case of Certificate Course in Domestic Data Entry Operator and
Basic Certificate Course in Community Nursing and Palliative Care as the difference is not
statistically significant.
16
b) Employability Index by Gender:
Mean Score
Employability Index Control Group Experiment Group
Male Female P-value Male Female P-value
All Candidates 51 48 <0.05 64 63 <0.05
It is noted that gender has small but significant impact on job candidacy
17
7 SHORTLIST RATE ANALYSIS
This section presents in detail the Shortlist Rate by training courses across the Control and
Experiment Group
Control Experimen
Training Courses P-value
Group t Group
Certified Banking and Finance Professional 24% 31% <0.10
18
Control Experimen
Training Courses P-value
Group t Group
Certificate in Food and Beverages Service- Steward 23% 45% <0.01
The shortlist rate is seen to have improved across all training courses except Certificate Course in
Domestic Data Entry Operator and Basic Certificate Course in Community Nursing and Palliative
Care.
Training courses such as Certificate Course in 3D Modelling and Texturing for Animation Film Making,
Certificate Course in Machinist, Certificate in Insurance Marketing, Certificate in Front Office Operations,
Certificate Course in Welding Technician, Plumbing Apprentice Programme / Certificate Course in
Plumbing Apprentice Programme and Certificate in Retail Sales Associate / Certificate Course in Retail
Sales Associate do not show very significant difference in shortlist rate of Control group vis-a-vis the
Experiment group.
19
b) Shortlist Rate by Gender:
Certified Banking and Finance Professional 26% 22% >0.10 34% 29% >0.10
*NA means no p-value generated due to insufficient sample size of male and/or female candidates
20
Control Group Experiment Group
Training Course
Male Female P-value Male Female P-value
Certificate in Food and Beverages Service-
27% 16% >0.10 52% 37% <0.05
Steward
Certificate Course in Field Technician and AC 22% 0% NA 36% 47% >0.10
*NA means no p-value generated due to insufficient sample size of male and/or female candidates
Insignificant p-values for Control group candidates validate that gender has no notable impact on
shortlist rate. Similarly for experiment group, it is observed that very few courses show significant
difference of impact of training in males and females. In general, we can say that gender is not a
strong differentiating criterion.
21
c) Shortlist Rate by Region:
The following table shows the comparison of shortlist rate of Control and Experiment Group by
region
Control Experiment
City P-value
Group Group
Alappuzha 13% 29% <0.01
Kannur 21% 38% <0.01
Kasaragod 21% 42% <0.01
Palakkad 20% 36% <0.01
Thrissur 17% 36% <0.01
Ernakulam 29% 44% <0.05
Idukki 13% 28% <0.05
Wayanad 16% 33% <0.05
Kozhikode 22% 32% <0.05
Kollam 20% 28% >0.10
Malappuram 28% 31% >0.10
Thiruvananthapuram 29% 34% >0.10
Except for three districts (Kollam, Malappuram and Thiruvananthapuram), across all the other districts,
the shortlist rate of the Experimental Group is higher in comparison to the Control Group and the
difference is statistically significant at 95% confidence level.
22
8 JOB ROLE-WISEANALYSIS
1. Certified Banking and Finance Professional
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 193 312
English Comprehension 57 64 <0.01
Information Gathering & Synthesis 46 51 <0.01
Quantitative Ability 36 38 >0.10
Conscientiousness 47 52 <0.10
Emotional Stability 58 57 >0.10
Agreeableness 51 52 >0.10
Financial Products/Services Situation Test 50 54 >0.10
Data Recording 64 54 >0.10
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 312
English Comprehension 89% Low
Information Gathering & Synthesis 73% Low
Quantitative Ability 43% High
Financial Products/Services Situation Test 80% Low
23
2. Certificate Course in Business Correspondent
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 155
English Comprehension 85% Low
Information Gathering & Synthesis 61% Mid
Banking Service Situation Test 49% High
24
3. Certificate Course in Banking Correspondent and Business Facilitator
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 171
English Comprehension 82% Low
Information Gathering & Synthesis 70% Mid
Banking Service Situation Test 50% High
25
4. Certificate in Accounting Technicians
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 522
English Comprehension 98% Low
Information Gathering & Synthesis 78% Low
Quantitative Ability 52% Mid
MS Excel 70% Mid
26
5. Certificate Course in Automotive Machining Technician
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 104
English Comprehension 87% Low
Information Gathering & Synthesis 60% Mid
Pattern Matching 85% Low
Workplace Health & Safety Situations 78% Low
27
6. Certificate Course in 3D Modelling and Texturing for Animation Film Making
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 72
English Comprehension 79% Low
Information Gathering & Synthesis 64% Mid
Basic Computer Literacy 65% Mid
28
7. Emergency Medical Care Technician
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 111
English Comprehension 77% Low
Information Gathering & Synthesis 52% Mid
Quantitative Ability 95% Low
Medical - Situation Judgement Test 86% Low
29
8. Certificate Course in Automotive Service Technician 2 and 3 wheelers
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 99
English Comprehension 88% Low
Information Gathering & Synthesis 49% High
Quantitative Ability 70% Mid
Pattern Matching 77% Low
Workplace Health & Safety Situations 75% Low
30
9. Certificate Course in Machinist
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 157
English Comprehension 86% Low
Information Gathering & Synthesis 58% Mid
Quantitative Ability 67% Mid
Pattern Matching 83% Low
Work Management 73% Low
31
10. Certificate in Insurance Marketing
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 52
English Comprehension 83% Low
Information Gathering & Synthesis 62% Mid
Quantitative Ability 44% High
Financial Products/Services Situation Test 60% Mid
32
11. Certificate Course in Fashion Retail Program
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 88 56
English Comprehension 49 58 <0.01
Information Gathering & Synthesis 37 46 <0.05
Quantitative Ability 28 32 <0.05
Conscientiousness 33 39 >0.10
Emotional Stability 51 65 <0.01
Agreeableness 34 37 >0.10
Fashion Retail test 56 71 <0.01
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 56
English Comprehension 91% Low
Information Gathering & Synthesis 63% Mid
Quantitative Ability 48% High
Fashion Retail test 96% Low
33
12. Certificate Course in Customer Care Executive
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 196
English Comprehension 88% Low
Information Gathering & Synthesis 69% Mid
Customer Service Situation Test 46% High
*NA means no p-value generated due to insufficient number of candidates took the given module
34
13. Certificate in Food and Beverages Service- Steward
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 171
English Comprehension 80% Low
Information Gathering & Synthesis 64% Mid
Quantitative Ability 67% Mid
F&B Situation Test 92% Low
35
14. Certificate Course in Field Technician and AC
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 135
English Comprehension 93% Low
Information Gathering & Synthesis 60% Mid
Quantitative Ability 61% Mid
Electronics Customer Service Situation Test 68% Mid
Work Management 73% Low
36
15. Global Business Foundation Skill Course
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 320
English Comprehension 91% Low
Information Gathering & Synthesis 73% Low
Customer Service Situation Test 45% High
Basic Computer Literacy 75% Low
37
16. Certificate Course in Domestic Data Entry Operator
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 66 85
English Comprehension 59 55 >0.10
Information Gathering & Synthesis 46 46 >0.10
Quantitative Ability 33 35 >0.10
Conscientiousness 39 47 >0.10
Emotional Stability 54 55 >0.10
Agreeableness 41 53 <0.05
Data Entry 42 41 >0.10
Data Recording - 63 NA
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 85
English Comprehension 88% Low
Information Gathering & Synthesis 51% Mid
Data Entry 52% Mid
*NA means no p-value generated due to insufficient number of candidates took the given module
38
17. Certificate in Welding
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 65 79
English Comprehension 44 59 <0.01
Information Gathering & Synthesis 33 43 <0.01
Quantitative Ability 28 37 <0.01
Conscientiousness 32 48 <0.01
Emotional Stability 55 58 >0.10
Agreeableness 39 45 >0.10
Pattern Matching 62 85 <0.01
Work Management 40 58 <0.01
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 79
English Comprehension 92% Low
Information Gathering & Synthesis 58% Mid
Quantitative Ability 63% Mid
Pattern Matching 90% Low
Work Management 85% Low
39
18. Basic Certificate Course in Community Nursing and Palliative Care
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 55 86
English Comprehension 49 53 >0.10
Information Gathering & Synthesis 39 38 >0.10
Quantitative Ability 28 30 >0.10
Conscientiousness 38 46 >0.10
Emotional Stability 63 54 >0.10
Agreeableness 37 51 <0.05
Nursing Situation Test 61 68 <0.05
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 86
English Comprehension 77% Low
Information Gathering & Synthesis 47% High
Nursing Situation Test 88% Low
40
19. Certificate in Front Office Operations
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 55 80
English Comprehension 56 66 <0.05
Information Gathering & Synthesis 40 50 <0.01
Quantitative Ability 32 41 <0.01
Conscientiousness 43 49 >0.10
Emotional Stability 53 58 >0.10
Agreeableness 47 53 >0.10
Front Office Situation Test 40 45 <0.10
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 80
English Comprehension 90% Low
Information Gathering & Synthesis 63% Mid
Quantitative Ability 81% Low
Front Office Situation Test 66% Mid
41
20. BSE Certified Banking Service Associate Program
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 54 70
English Comprehension 61 75 <0.01
Information Gathering & Synthesis 45 59 <0.01
Quantitative Ability 34 49 <0.01
Conscientiousness 43 62 <0.01
Emotional Stability 60 53 >0.10
Agreeableness 45 61 <0.01
Banking Service Situation Test 45 61 <0.01
Data Recording 62 69 >0.10
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 70
English Comprehension 100% Low
Information Gathering & Synthesis 84% Low
Quantitative Ability 74% Low
Banking Service Situation Test 64% Mid
42
21. Certificate Course in Welding Technician
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Sample Size 50 49
English Comprehension 46 52 >0.10
Information Gathering & Synthesis 34 40 <0.10
Quantitative Ability 29 33 >0.10
Conscientiousness 24 48 <0.01
Emotional Stability 47 60 <0.10
Agreeableness 25 53 <0.01
Pattern Matching 69 80 <0.05
Work Management 42 51 <0.05
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 49
English Comprehension 90% Low
Information Gathering & Synthesis 49% High
Pattern Matching 88% Low
Work Management 88% Low
43
22. Mutual Fund Agent / Financial Advisory and Marketing Services
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 401
English Comprehension 92% Low
Information Gathering & Synthesis 62% Mid
Quantitative Ability 53% Mid
Financial Products/Services Situation Test 74% Low
44
23. Plumbing Apprentice Programme / Certificate Course in Plumbing Apprentice
Programme
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 151
English Comprehension 87% Low
Information Gathering & Synthesis 56% Mid
Quantitative Ability 67% Mid
Workplace Health and Safety 45% High
Work Management 87% Low
45
24. Certificate in Retail Sales Associate / Certificate Course in Retail Sales Associate
Mean Scores
Mean Scores
Module Name P-value
Control Group Experiment Group
Level of training
Shortlist Rate
Module Name intervention
(per modules)
required
Sample Size 298
English Comprehension 83% Low
Information Gathering & Synthesis 62% Mid
Quantitative Ability 39% High
Retail Sector Situation Test 92% Low
46
ANALYSIS SUMMARY
Following inferences are drawn while comparing mean scores of Control and Experiment
Group:
I. In 11 courses, the mean scores of English are higher for Experiment Group in comparison to
the Control Group at 99% confidence level.
Mean Scores
Course Name P-value
Control Group Experiment Group
Certified Banking and Finance Professional 57 64 <0.01
Certificate Course in Business Correspondent 50 59 <0.01
Certificate in Accounting Technicians 59 68 <0.01
Certificate Course in Automotive Service
47 55 <0.01
Technician 2 And 3 wheelers
Certificate Course in Fashion Retail Program 49 58 <0.01
Certificate Course in Customer Care Executive 51 62 <0.01
Certificate Course in Field Technician and AC 48 56 <0.01
Global Business Foundation Skill Course 59 66 <0.01
Certificate in Welding 44 59 <0.01
BSE Certified Banking Service Associate Program 61 75 <0.01
Mutual Fund Agent / Financial Advisory and
55 58 <0.01
Marketing Services
47
II. In 15 courses, the mean scores of IGS are higher for Experiment Group in comparison to the
Control Group at 99% confidence level.
Mean Scores
Course Name P-value
Control Group Experiment Group
Certified Banking and Finance Professional 46 51 <0.01
48
III. In 6 courses, the mean scores of Quantitative Ability are higher for Experiment Group in
comparison to the Control Group at 99% confidence level.
Mean Scores
Course Name P-value
Control Group Experiment Group
49
Following inferences are drawn on the basis of Skill Gap analysis done on the Experiment
Group:
Level of training
Shortlist
Course Name Count intervention
Rate
required
BSE Certified Banking Service Associate Program 70 100% Low
Certificate in Accounting Technicians 522 98% Low
Certificate Course in Field Technician and AC 135 93% Low
Certificate in Welding 79 92% Low
Mutual Fund Agent / Financial Advisory and
401 92% Low
Marketing Services
Certificate Course in Fashion Retail Program 56 91% Low
Global Business Foundation Skill Course 320 91% Low
Certificate in Front Office Operations 80 90% Low
Certificate Course in Welding Technician 49 90% Low
Certified Banking and Finance Professional 312 89% Low
Certificate Course in Automotive Service Technician
99 88% Low
2 And 3 wheelers
Certificate Course in Customer Care Executive 196 88% Low
Certificate Course in Domestic Data Entry Operator 85 88% Low
Certificate Course in Automotive Machining
104 87% Low
Technician
Plumbing Apprentice Programme / Certificate
151 87% Low
Course in Plumbing Apprentice Programme
Certificate Course in Machinist 157 86% Low
Certificate Course in Business Correspondent 155 85% Low
Certificate in Insurance Marketing 52 83% Low
Certificate in Retail Sales Associate / Certificate
298 83% Low
Course in Retail Sales Associate
Certificate Course in Banking Correspondent and
171 82% Low
Business Facilitator
Certificate in Food and Beverages Service- Steward 171 80% Low
Certificate Course in 3D Modelling And Texturing For
72 79% Low
Animation Film Making
Emergency Medical Care Technician 111 77% Low
Basic Certificate Course in Community Nursing and
86 77% Low
Palliative Care
50
II. Information Gathering & Synthesis:
17 courses require mid-level interventions
3 require high-level interventions
Level of training
Shortlist
Course Name Count intervention
Rate
required
BSE Certified Banking Service Associate Program 70 84% Low
Certificate in Accounting Technicians 522 78% Low
Certified Banking and Finance Professional 312 73% Low
Global Business Foundation Skill Course 320 73% Low
Certificate Course in Banking Correspondent and
171 70% Mid
Business Facilitator
Certificate Course in Customer Care Executive 196 69% Mid
Certificate Course in 3D Modelling And Texturing For
72 64% Mid
Animation Film Making
Certificate in Food and Beverages Service- Steward 171 64% Mid
Certificate Course in Fashion Retail Program 56 63% Mid
Certificate in Front Office Operations 80 63% Mid
Certificate in Insurance Marketing 52 62% Mid
Mutual Fund Agent / Financial Advisory and
401 62% Mid
Marketing Services
Certificate in Retail Sales Associate / Certificate
298 62% Mid
Course in Retail Sales Associate
Certificate Course in Business Correspondent 155 61% Mid
Certificate Course in Automotive Machining
104 60% Mid
Technician
Certificate Course in Field Technician and AC 135 60% Mid
Certificate Course in Machinist 157 58% Mid
Certificate in Welding 79 58% Mid
Plumbing Apprentice Programme / Certificate
151 56% Mid
Course in Plumbing Apprentice Programme
Emergency Medical Care Technician 111 52% Mid
Certificate Course in Domestic Data Entry Operator 85 51% Mid
Certificate Course in Automotive Service Technician
99 49% High
2 And 3 wheelers
Certificate Course in Welding Technician 49 49% High
Basic Certificate Course in Community Nursing and
86 47% High
Palliative Care
51
III. Quantitative Ability:
8 courses require mid-level interventions
4 require high level interventions
Level of training
Shortlist
Course Name Count intervention
Rate
required
Emergency Medical Care Technician 111 95% Low
Certificate in Front Office Operations 80 81% Low
BSE Certified Banking Service Associate Program 70 74% Low
Certificate Course in Automotive Service Technician
99 70% Mid
2 And 3 wheelers
Certificate Course in Machinist 157 67% Mid
Certificate in Food and Beverages Service- Steward 171 67% Mid
Plumbing Apprentice Programme / Certificate
151 67% Mid
Course in Plumbing Apprentice Programme
Certificate in Welding 79 63% Mid
Certificate Course in Field Technician and AC 135 61% Mid
Mutual Fund Agent / Financial Advisory and
401 53% Mid
Marketing Services
Certificate in Accounting Technicians 522 52% Mid
Certificate Course in Fashion Retail Program 56 48% High
Certificate in Insurance Marketing 52 44% High
Certified Banking and Finance Professional 312 43% High
Certificate in Retail Sales Associate / Certificate
298 39% High
Course in Retail Sales Associate
Unlike English and Information Gathering & Synthesis, which were considered important across
all job roles, a threshold cut-off on Quantitative Ability was applied only in 15 out of the total 24
courses. This is because it is not as important a skill for the rest of the courses as number skills are
not required to perform those jobs. The above results of the skill-gap analysis should be interpreted
in light of this fact.
52
9 INTER-INSTITUTE COMPARISON
This section compares the mean scores of the Experiment Group with two other institutions which
have opted for similar test suites.
Institute 1 and 2 are non-governmental organization with multiple training programs in diverse
fields from entrepreneurial ability, work ethics and soft skills development to hands-on programs
in electrical, teaching, banking and finance.
Mean Score
Information
Sample English Gathering & Quantitativ
Size Comprehension Synthesis e Ability
ASAP
3932 60 48 36
(Experiment Group)
Institute 1 59 26 34 30
Institute 2 63 31 - 33
The Experiment Group candidates outperform both the institutions in each of English
Comprehension, Information Gathering & Synthesis and Quantitative Ability.
53
10 LIMITATIONS OF THE STUDY
1. Analysis by Education is not feasible due to inadequate sample points. 93% candidates
have twelfth as their education background. Analyzing a big sample set with a small set
may not lead to credible results.
2. The performance of similar profiles in other states cannot be analyzed with ASAP training
courses because all profiles are not tested in a particular state. Few profiles have been
assessed in some states and few have not been assessed.
4.
54
11 RECOMMENDATIONS AND FUTURE INTERVENTIONS
1. The norm group currently used in this study is the control group as the aim of the study is
to gauge the performance of the students in the experimental group (ASAP trained
candidates) with reference to the untrained students. However, an ideal comparison and
benchmarking of employability should be with reference to industry benchmarks obtained
from the samples of working professionals in the respective fields.
2. The employability index is being calculated with the assumption that the candidates
possess the adequate practical skills to perform the job. Ideally, the practical skills of the
candidates should also be considered in calculation of the employability index as the
practical demonstration forms a critical component of on-job performance. Since there was
no evaluation of the candidates on their hands-on skill, this was not included in the
calculation of EI.
55
12 APPENDIX
Weights:
Information
Quantitative Domain Domain
Training Course English Gathering &
Ability Module1* Module2*
Synthesis
Certified Banking and Finance
25 25 25 15 10
Professional
Certificate Course in Business
25 25 15 20 15
Correspondent
Certificate Course in Banking
25 25 15 20 15
Correspondent and Business Facilitator
56
Information
Quantitative Domain Domain
Training Course English Gathering &
Ability Module1* Module2*
Synthesis
Certificate in Welding 10 30 15 25 20
57
Cut-offs:
Information
Quantitative Domain Domain
Training Course English Gathering &
Ability Module1* Module2*
Synthesis
Certified Banking and Finance
37.5 37.5 37.5 33 -
Professional
Certificate Course in Business
37.5 37.5 - 50 -
Correspondent
Certificate Course in Banking
37.5 37.5 - 50 -
Correspondent and Business Facilitator
Certificate in Accounting Technicians 31.25 37.5 37.5 31.25 -
58
Information
Quantitative Domain Domain
Training Course English Gathering &
Ability Module1* Module2*
Synthesis
59