Documente Academic
Documente Profesional
Documente Cultură
Assessment Plan
for Learning Management System in Educational Technology Master Program in Arab Open University (AOU)
Tech6102 | Presented by: Asma AlKalbani| 82934 Fatma Alhinai| 98569 & Mitha Alkhuzami|102477
2
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Table of Content
1. Introduction.3-7
3. Analysis Phase.8-11
3.2 Stakeholder...8-9
3.3 Context.9
4. Design Phase...11-12
5.Development Phase12-13
6. Implementation Phase13
7. Evaluation Phase13-18
7.2 Recommendations...18
8. References..19-20
9. Appendix....21
3
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Introduction
Program evaluation is carefully collecting information about all or some aspect of a program
in order to make necessary decisions related to the program. It is a systematic method for collecting,
analyzing, and using information. It helps evaluator to guide decision-making, identify where support
is needed, provides a picture of what has been accomplished, what needs to be done and enhances the
effectiveness of programs. Evaluators go over evaluation process using scientific methods and
valuable data to ensure high quality of assessment. On this document, we developed an assessment
plan aimed to measure students' satisfaction toward LMS in Educational Technology master program
The Arab Open University (AOU) is a sustainable development and educational non-
profit project. It was founded by HRH Prince Talal bin Abdul-Aziz, Chairman of the AOU
Board of Trustees. In September 2000, AOU was officially declared in the meeting of Arab
ministers of higher education where five Arab countries offered to host the headquarters of
AOU. Among them was Kuwait that was chosen to be the headquarters of AOU in December
2000. Today AOU has eight branches in Kuwait, Lebanon, Jordan, Saudi Arabia, Egypt,
Bahrain, Oman, and Sudan. The (AOU) in Oman focuses on integrated (LMS) in the learning
Vision
A pioneering Arab Open University dedicated to building the science and knowledge
society.
4
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Mission
quality standards without time or geographical barriers for the sake of contributing and preparing
manpower for development needs, and to build science and knowledge society in the Arab
countries.
comprising either taught courses only or taught courses and a dissertation. The program was started
in 2012 and graduated three cohorts; first cohort on 2014, second cohort on 2015, and the third cohort
on 2016. The numbers of student were increased in each cohort (18, 30,43 respectively). The
1. Contribute towards the achievement of the missions and objectives of the AOU, that is
States.
2. Meet the high demand in many of the Arab States for qualified well- trained teachers who
3. Enhance the quality of teacher preparation and teacher training in general, thereby
Arab States
4. Respond to employment market demands for personnel with skills and qualifications in
Instructional Technology, for example, business, mass media, and multimedia production
will avail themselves of excellent upto date teaching and support materials conducive to
self- learning.
Successful candidates will qualify not only for the AOU Educational Technology
which will provide graduates with opportunities to continue their PhD study locally,
regionally or internationally.
It offers career progression in education and, more generally, in the public and private
sectors
The Program is in high demand in all Arab countries, particularly in view of the following facts:
qualified to master's level in education for promotion and enhanced career prospects.
More and more faculties of education and teacher training institutions are being
established in Arab countries, and thus there is a great demand for qualified Masters
National and regional businesses, the mass media, international organizations and
publishing houses also require Arab personnel who are trained in educational technology
The Program will be delivered through the open learning system, which according to the AOU
Interactive distance learning delivered through specially designed teaching and support
materials that are conducive for self-learning, constituting 75% of course credit hours.
6
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
This M. Ed. Program in Instructional Technology will qualify students to be able to:
1. Understand the fundamental concepts and skills for professional use of technology in the
problems.
3. Design methods and tools for the development of innovative learning environments.
4. Use various media to communicate and collaborate effectively with students, colleagues,
and others.
8. Improve the understanding of the impact of (ICT) on the organization of teaching and
learning.
9. Bring to the classroom all the content, motivational and management expertise to capture
10. Use technology to foster students curiosity and creativity, as well as engage students in
13. Stimulate the development of services and systems to ensure access to multimedia
environments.
5 Decide Upon Assessment
Tools (note: these may be
different from the needs
assessment tools)
6 Perform your assessment
We adapted ADDIE model to guide our assessment plan and we match it with the nine steps of
quality assurance plan. Figer1 shows the evaluation model with its steps:
8
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Analysis Phase
We conducted an interview with Dr. Nader Said (the head of the Educational Technology
department in (AOU) to determine the needs to build the assessment plan. We prepared different
questions and recorded the interview by phone for further information, see appendix (1). We
decided to use this tool because it helps us to collect qualitative data as much as possible. After
analyzing the interview, we decide to do an assessment plan to assess student satisfaction toward
using the Learning Management System (LMS) as the department depends heavily on it as they
use it as distance education tool. Also, they did not measure their students satisfaction toward
using LMS.
Stakeholders
9
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
The target audience of this plan is master students at the first and second year in
Educational Technology program in(AOU). The sample of assessment covers 53 students. Most
of them are an employee in the governmental and private sector. They came from different
regions and some of them are foreigners who come with the different culture. they have willing
Context
This plan focuses on studying LMS environment in AOU specifically for those master
administration of training events. The LMS registers users, tracks courses in a catalog and
records data from learners. It also provides reports to management. An LMS is typically
designed to handle courses by multiple publishers and providers like lectures, designer, and
instructional developer. It usually doesn't include its own authoring capabilities; instead, it
factor in measuring the quality of blended learning (Naaj et al, 2012). Students' satisfaction
defined by Sloan Consortium as, Students are successful in the learning and are pleased with
their experience (J. C. Moore, 2009). Another definition of student satisfaction said that
(Sweeney & Ingram, 2001). Both definitions focus on accomplishment and success in learning,
LMS success and as the ultimate target of LMS use is sensible for several reasons. First, previous
10
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
research showed that LMS use supports rather than modifies, existing teaching and learning
approaches. Thus, make sense to redefine LMS success and shift indications of success away
satisfaction is defined as the ability of a service or a product to address customer needs. Even if
student needs are not fully known, it is reasonable to assume that high student' satisfaction is
indicative of success in the sense of whether the LMS responds well to their needs (Naveh,
There are many factors that influence student satisfaction while using LMS. In this
assessment plan, we are measuring student satisfaction towards the LMS in terms of;
Instructional Materials. They refer to the content delivered by LMS. Institutions use the
LMS to supplement traditional face-to-face delivery where faculty members develop and share
digital learning materials via the Internet. In this case, the LMS is used as electronic repositories
of learning materials. In addition, it should be noted that learners rely on learning materials as
their major source of information during the learning process. Consequently, they place great
value on content that is well-organized, effectively presented, interactive, clearly written, in the
right length, useful, flexible, and provide an appropriate degree of breath. There is also a strong
positive relationship between quality of learning materials and overall learners perceived
satisfaction with the LMS. Therefore, learners tend to be disappointed with the LMS when they
find out that uploaded materials are of poor quality and do not provide intended educational
are allowed and encouraged lead to positive learning outcomes. Collaborative learning and
11
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
communication tools can improve student satisfaction in the online learning environment. These
tools allow for group work and immediate feedback. Students can share viewpoints and discuss
them with one another in a virtual environment and with the student and instructor, thereby
gaining insights and perspectives. This type of environment allows for social interaction and
creates meaningful and active learning experiences (Abou Naaj, Nachouki, & Ankit, 2012).
User Interface. Interface or sometimes called the interface design. The interface of the
LMS significantly influences students satisfaction towards LMS. The result of the user -
friendly interface will directly influence student satisfaction towards the use of LMS in blended
learning (Malik, 2009). Therefore, the interface design of LMS is important as its play a vital
role in user interface interaction. In the online environment, users will utilize the computer to
access the content and interact with other online users and instructors. The LMS must be
designed in an appropriate way so that users would not face any trouble when using it. The
problem in LMS design would decrease the user satisfaction when using the LMS. Thus, a
simple LMS user interface must be included for ease the interaction between users and systems
Technical Support. It enhances the satisfaction of users and has a critical effect on users
believes in accepting or rejecting an LMS. When users receive no help from the assistants while
being faced with a problem, they will get the feeling that working with the system is a waste of
time and hence will quit working with it. Technical support in this case, refers to the existence of
assessing staff or faculty to solve problems students encounter when they are working with LMS
(Baleghi-Zadeh, Ayub, Mahmud & Daud, 2017). In addition, disabled users must be considering
in LMS to provide them with suitable materials that work with their skills.
Design phase
12
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Measure the e-learning effectiveness in terms of Student satisfaction toward using the LMS in
How satisfied are the students with the Instructional materials in the LMS?
How satisfied are the students with the Communication in the LMS?
How satisfied are the students with the Technical Support in the LMS?
What are the challenges that students face while using LMS?
Based on students need and purpose of this assessment plan, our team decided to use the survey
as a tool to collect quantitative data to measure students' satisfaction toward LMS in AOU. In
addition, this tool eliminates biases and we can collect much data in less time from a big number
of students.
Development Phase
We developed a survey to assess students satisfaction toward the LMS by adopting American
Society for Training & Development (ASTD) standards for e-learning. Besides that, some items
in the survey, from Bauk, Scepanovic, & Kopp, (2014) study which focused on estimating
students' satisfaction with web-based on learning system in the blended learning environment.
We used Google forms to develop the survey and collect data. The instruments had
been reviewed by our instructor to ensure its validity. In addition, it had been implemented
within a small group of students in AOU to ensure its reliability via SPSS by using Alpha
Cronbachs. The inter-rater reliability of survey achieved an excellent value (0.93). The survey
13
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
consisted of 5 major parts. The first four parts are instructional materials, communications,
interface and technical support using three scales (Agree, Natural and Disagree). The final part is
the open question to determine the problems that student face while using LMS. For further
Implementation Phase
only and we see that this amount of responses is reasonable because it exceeded the half of our
target sample.
Evaluation Phase
Result of survey
Sometimes figures speak better than words. To analyze the responses, we calculated
the mean of responses by using SPSS software. Although the students responses to
Result shows, more than 50% of students agreed that instructional materials in LMS are up to
date, user-friendly, convenient, easy to operate and easy to understand. Naaj, Nachouki, &
Ankit (2012) stated that easy access is one of the most important factors influencing student'
satisfaction. A big number of students considered that instructional material didn't meet
enough different students learning styles. Malik ( 2009) reported that students' satisfaction
and acceptance of LMS is influenced by the teachers teaching style and ways of delivering
the item; The link in the instructional materials direct you to the right references" but in case
of (LMS) failure some supplemental resources are not available for students. Sometimes,
students couldn't access to supplemental resources due to the failure of the internet connection
LMS via five items. Data summarized that students can easily communicate with others. They
received clear, precise and sufficient announcement through LMS. More than 20 students agreed
that they received a regular announcement from their instructor on the courses (LMS). Malik
(2009) stated that when students dont face any problem with teacher interaction their
satisfaction will be improved. Item (4) related to interaction with teachers assigned the highest
mean (0.71) while the lowest score in this group recorded by item (5), " The teachers spend time
on online communication with students (feedback, answer questions etc.)". The present of
teacher via online helps students to answer their inquiry early. It also affects students'
performance when they find teacher follow them and observe their work online especially when
they know that teacher can see how many students login LMS in details; in which time, for what
purposeetc.
1 I can navigate (LMS)freely and easily (example; guidance). 0.71 0.23 0.06
We developed three items in this category. According to students' responses toward the
interface of LMS, they noted that students can navigate (LMS) freely and easily with guidance.
Malik (2009) explained that friendly user interface will directly influence student' satisfaction
towards courses in LMS. For the second item; "Interface provide forms of feedback", 18 students
(55%) answered agree. When students were asked to assess the interactivity of LMS interface, only
16
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
11students (35%) agreed. Interactivity could be affected by the organization of the content inside
We developed four items to measure the level of students' satisfaction toward technical
services in LMS. From the above numbers, it seems that students weren't satisfied with technical
support in LMS. All items in this group had a mean less than 0.39 or equal except item (2); " I
can access technician through phone, email, and fax" which recorded 0.52 and it is still not high
acceptable. Oliveira, Cristiano & Nakayama (2015) stated that there is a need to know where the
students "walk", what they access, what they read when they're doing in LMS to support them
According to the responses to the last question: " What are the challenges that students
face while using LMS?". The major of students responses could be categorized into three issues:
Firstly, LMS doesnt work with all file extension format. Secondly, sometimes students face
problems with the internet so they can't access to LMS and they received the notifications lately.
N Category Mean
Current survey Indicator
result items
1 Instructional materials 0.57 0.50 and above
We calculated the mean average for instructional material, communication, user interface
and technical support and compared them with indicator mean to measure the level of students'
satisfaction in each category. The average mean for student' satisfaction in the first group is 0.57
which is above than indicator mean. That means students were satisfied with the instructional
material in LMS. The results of the second category showed that students were generally
satisfied with communication via LMS (mean=0.59). The lowest mean item in this category was
related to the time spent by the teacher to communicate with students (mean= 0.35) in which
students are less satisfied and require improvement. According to Naaj, Nachouki, & Ankit paper
they found that communication in LMS having the highest mean which indicates that students
are satisfied with the level of communication (Naaj, Nachouki, & Ankit, 2012). That study
agreed with our result. More than 50% of students were satisfied with interface services in LMS
but there were also some areas in which the students were less satisfied specifically in the
interactivity of the interface. Malik, 2009 stated that students positive attitude towards interface
of the online environment will automatically impact on their satisfaction positively and increase
the chances of taking courses' activity on time. The mean average of students' satisfaction toward
18
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
technical support is very low (0.39) because they didnt get enough support from technician
inside or outside class time and disabled students didnt find enough assistive technologies in
LMS. This result doesnt agree with the result of Naveh, Tubin & Pliskin (2012) study. They
explained that the successful implementation and student satisfaction towards LMS could be
Recommendations
program in AOU to
- Differentiate the ways of presenting instructional material in LMS to meet all learning styles.
- Increase the amount of time for online communication to answer all students inquiry in short
time.
Share Results
We shared our results with colleagues in the classroom to exchange our experiences,
instructor of this course, educational Technology program at AOU to get benefits from our
recommendations ,and all interested in developing assessment plan by publishing it over social
media.
Conclusion
There are various alternative learning environments in our country. We need to evaluate
these environments to ensure high quality of services, learning and outcomes. In this document,
found that AOU need to empathize and develop technical support services in the LMS.
19
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
References
Abdul Rahman, N., Hussein, N., & Aluwic, A. H. (2015). Satisfaction on Blended Learning in a
Public Higher Education Institution: What Factors Matter? Elsevier, 768- 775. doi:
10.1016/j.sbspro.2015.11.107
Bauk, S., Scepanovic, S., & Kopp, M. (2014). Estimating Students Satisfaction with Web Based
11. doi:10.1155/2014/731720
Oliveira, C., Cristiano, C., & Nakayama, K., (2015). Learning Management Systems (LMS) and
177520160002000001
Malik, M., (2009). Students Satisfaction toward E-learning: Influential Role of Key Factors.
/All%20papers/STUDENT%20SATISFACTION%20TOWARDS%20E-
LEARNING%20INFLUENTIAL%20ROLE%20OF%20KEY%20FACTORS%20(mahw
ish%20w.%20malik).pdf
Naaj, M. A., Nachouki, M., & Ankit, A., (2012). Evaluating Student Satisfaction with Blended
http://www.jite.org/documents/Vol11/JITEv11p185-200AbouNaaj0979.pdf
Abou Naaj, M., Nachouki, M., & Ankit, A. (2012). Evaluating Student Satisfaction with Blended
http://www.jite.org/documents/Vol11/JITEv11p185-200AbouNaaj0979.pdf
Naveh, G., Tubin, D., & Pliskin, N. (2012). Student satisfaction with learning management
Retrieved from
https://www.researchgate.net/publication/262455665_Student_Satisfaction_with_Learnin
g_Management_Systems_A_lens_of_critical_success_factors
Naveh, G., Tubin, D., & Pliskin, N. (2010). Student LMS use and satisfaction in academic
doi:10.1016/j.iheduc.2010.02.004
Zadeh, S. B., Mohd Ayub, A. F., Mahmud, R., & Daud, S. (2017). The influence of system
journal.org/ojs/index.php/online-publication/article/view/696/338
Ramakrisnana, P., Jaafar, A., Abdul Razak, F. H., & Ramba, D. A. (2012). Evaluation of User
Interface Design for Leaning Management System (LMS): Investigating Students Eye
10.1016/j.sbspro.2012.11.357
21
ASSESSMENT PLAN FOR LEARNING MANAGEMENT SYSTEM
Appendixes
How are satisfied students being with the use of E-learning in post-program in AOU?
in post-program in AOU?
What are the most prominent things that focus on the content that is presented in E-
Is there technical support for students? What kind of support are you provide in E-
learning to post-program students in AOU? What are the challenges that students face in
e-learning?
https://docs.google.com/forms/d/e/1FAIpQLSeHhOQyIXjlWm7YMjL4j_FMK3Lq9e8SET2THa
YLZAX8CXlL5w/viewform?c=0&w=1