Documente Academic
Documente Profesional
Documente Cultură
1 Introduction
Item bank is “a set of items for a special subject constructed by computer system
according to the education measurement theory”. Item bank following the education
measurement theory is an education measurement tool based on strict mathematical
model.
Item bank construction is a complex system engineering. First, establish the
mathematical model. Then set the attributes indexes of the items and the structure of
the test papers, input the scientific effectual items. Following that, evaluate the testing
effect and situation of the students’ leaning. After putting into operation, adjust adap-
tively according to the testing level of the students. To ensure the scientificalness and
effectiveness, the relative attributes of the items and test papers should not only be set
by experts but also be sample tested using large numbers of sampling data, which can
adjust the effectiveness of the parameters. Editing and testing task for a relatively
complete item bank based on the classical test theory is such a formidable task, which
can not be done well by average institutions. [1]
Presently with the development of information technology, item bank are estab-
lished by companies, schools or their cooperation. They have made certain achieve-
ments in practical application, but with some problems.
Z. Pan et al. (Eds.): Edutainment 2008, LNCS 5093, pp. 278–289, 2008.
© Springer-Verlag Berlin Heidelberg 2008
The Design of Web-Based Intelligent Item Bank 279
(1) Independent of the other teaching links, used un-widely, so can not be improved
based on sufficient sample data.
(2) Providing reverence answers without education evaluation and intelligent in-
structing.
(3) The adaptability of the system is not so good, for example: the difficulty index
keeps invariable and papers can not be generated automatically.
(4) Having little characteristics of automatic marking and reasonable statistics analy-
sis functions.
In order to solve the problems above, we improved the existed internet item bank
and developed a general item bank system successfully. It consists of subsystems of
item management, automatic test paper generation, autonomous learning, testing, and
study evaluation, which can provide a series of services: automatic test paper genera-
tion, testing, analysis and evaluation.
2 System Design
In order to solve the problem of running independent of other teaching links, the sys-
tem is designed with the relative systems such as testing, evaluation and autonomous
learning. The overall design is illustrated by figure 1.
Students
database
Strategy
database
Score
database
Knowledge
database
Item
Testing Subsystem
database
Test paper
database
b ᧥ Generating by students: The students can input the parameters of paper genera-
tion (different from the one used by teachers) according to their own condition,
then the system generates the paper for exercise of pertinence which can be
᧥
printed or saved as web pages.
c Static generation strategies: There are some common generation strategies for
the common testing. The teachers can choose the ready generation strategies in-
stead of inputting the complicated parameters.
(3) Testing Subsystem
This subsystem can use the papers generated by the system or the ones generated
by the paper generation subsystem. Students take the test online or take the paper
home via storage device to test, and then submit their answers to the analysis mod-
ules. This subsystem includes three links.
a᧥ On-line examination: The students can choose the special paper via browser,
take the test and submit their answers. The system stores the answers into the
student information database, and then the teachers can take them out and do the
᧥
marking.
b On-line marking: After login in the system, the teachers can choose the paper
they should mark, and then choose the students who took the examination to take
the students’ papers and corresponding answers. The objective questions can be
judged by the system automatically. The non-objective questions are marked by
teachers which are stored into the score database with the marking information
᧥
together after submitted by teachers.
c Results requiring: After login in, the students can choose the papers they tested
and look over the testing contents, standard answer and marking condition, etc.
(4) Autonomous learning subsystem
Autonomous learning subsystem has a close relationship with the testing subsys-
tem. On one hand, it calls the paper generation subsystem to generate the papers, do
exercise autonomously and transfer the results to the analysis modules. On the other
hand, it can give the corresponding advices on knowledge review and learning strat-
egy according the test.
(5) Study Evaluation Subsystem
This subsystem is used to do statistic analysis of questions, papers and the teaching
process according to the history data of the students’ testing which is analyzed by S-P
table. It has two important parameters of difference coefficient and alarm coefficient,
and they can separately provide important information for analysis of teaching situa-
tion and teaching effect. So, this subsystem can analyze the reasonable degree for
individual situation, trends of all the students and improvement of the teaching effect.
It includes the following aspects.
a᧥ Statistics analysis of the papers: The distribution of the items such as students’
points can be shown by line graph and histogram and all the abnormal questions
in the exam can be captured such as all the answers are right or wrong in whole.
Analysis on the following subjects: the reliability, validation and average diffi-
culty of the paper; the maximum and minimum points, the total number of each
grade section, average point and standard deviation ; the original points and the
transformed points for each student.
282 S. Zhong et al.
3 Evaluation Strategies
S-P table is an information processing method of analyzing the teaching based on the
students’ points table of the questions. The method can evaluate the learning situation
of individual student and the trends of all the students; also it can evaluate propriety
of all the questions. Difference coefficient and alarm coefficient are two important
parameters used for S-P analysis. Calculation of them can provide important informa-
tion for analysis of teaching situation and teaching effect.
M students answer n questions, mark 1 when right, mark 0 when wrong, and a matrix
of student – question is created .In the matrix, Uij represents the point student i gets
for answering question j. Because of adoption of 0-1 score method, the matrix is con-
sisted of only 0 and 1.The score matrix can be shown concretely as the condition in
figure 2.
The Design of Web-Based Intelligent Item Bank 283
zTwGt
wX wY wZ w[ w\ w] w^ w_ w` wXW z
zX W X X W X X W X X W ]
zY W X X X X W W W W W [
zZ X X X X X W X X W X _
z[ W X X X X X X X W W ^
z\ X X X X X X X W W X _
z] W X X W W W W X W W Z
z^ X X X X X X X X W X `
z_ X W W X X X X X W X ^
z` X X X X X X X X X X XW
zXW X X X W X X W W W W \
zXX W X W X X X X W W X ]
zXY W X X W X X X X X W ^
zXZ W X X X X X X W X X _
zX[ W X X W X W X W W W [
zX\ W X X X X X X X X W _
j ] X[ XZ XW X[ XX XX ` \ ^
{
The column on the right lists the scores of the students (the number of the right
ones); the bottom lists the number of right answers for each question. Before the ma-
trix is processed, it can only provide the point of each student and the correct rate of
each question. It can provide much significant information after processed by the
following rules.
a) Students will be arranged by descending order according to the scores. Namely,
exchange the students scoring line, making the high score above in the row, low
score firms bottom.
b) Problems will be arranged by descending order according to the number of cor-
rect answers from left to right. Namely, exchange the problems line, making the
problem line of giving more correct answers to it on the left, the problem of low
frequency of correct answers row on the right side.
c) For the same score line, first of all, find out the sum of correct answers which the
student obtained the errors of each problem in the same column. And then, ar-
range the line of the same score according to the sum; line get high score firms
the top.
284 S. Zhong et al.
d) For the same score column, first of all, find out the sum of the student obtained
the errors of each problem. And then, arrange the column of the same score ac-
cording to the sum; column get low score firms the top.
e) Making the S line. Draw the vertical line for each of the students. Making the
number of the problems on the left side of the vertical equal to the students’
score. And then, drawing the striping between lines, link each vertical line to
form a ladder of the curve, called S line. As Figure 3 shows the solid line.
f) Making the P line. Draw the ledgement for each of problems. Making the num-
ber of students above the ledgement equal to the number of correct answers. And
then, drawing the vertical line between columns, link each ledgement to form a
ladder of the curve, called P line. As Figure 3 shows the dashed line.
After processing, the student-question matrix is transformed into ordered list of S
(solid line) and P (dashed line), which is called S-P table.
zTwG{
w\ wY wZ w^ w] w[ w_ wXW wX w` z
z` X X X X X X X X X X XW
z^ X X X X X X X X X W `
zZ X X X X W X X X X W _
zXZ X X X X X X W X W X _
zX\ X X X X X X X W W X _
z\ X X X X X X W X X W _
z_ X W W X X X X X X W ^
zXY X X X X X W X W W X ^
z[ X X X X X X X W W W ^
zX X X X W X W X W W X ]
zXX X X W X X X W X W W ]
zXW X X X W X W W W X W \
zY X X X W W X W W W X [
zX[ X X X X W W W W W W [
z] W X X W W W X W W W Z
j X[ X[ XZ XX XX XW ` ^ ] \
{
Based on the data relationship, it is not difficult to see that the S-P table has the fol-
lowing basic properties.
Because the score of students equals the total time of the right answers, the left-
hand area of S is equal to the area above the P; S is not only the score curve of the
students but also the number curve according to the cumulative scores. P is the accu-
mulating distribution model of the right answer number for each question. S and P are
always intersectant, and when superpositioned, the left end of P is always above S and
the right end of P is always under S; the area gap between S and P is called dispersion
of the two lines which can describe the relationship between the question difficulty
and the students’ response. If S is as same as P, the condition is that the learning state
of all the students is absolute stable.
According to S-P table, holistic analysis can only be done for whole and individual
analysis can only be done for individual.
(1) Holistic Analysis
It includes the difference analysis between the students and the questions and the
distribution analysis between the questions and students’ response. The dispersion of
the two lines reflects the evenness degree of distribution. The increase in dispersion
can be associated with the increase in un-even distribution. The evenness of distribu-
tion can be described as follows:
As for common tests, the value of D ranges from 0.25 to 0.35, and can not over 0.5
usually. If it is over 0.5, the relationship between the questions and students’ response
is abnormal .For example, if a difficult problem is solved by students getting low
points and not solved by students getting high points.
The difference between students and questions can be reflected by the faultage be-
tween S and P. Faultage stands for the beeline range between the domestic wirings.
Figure 4 illustrates the faultage.
If the line S keeps horizontal for quite long, it suggests the danger of polarization.
If the line P keeps vertical for quite long, it suggests that there is sufficient difficulty
difference which may influence the validity of the testing.
(2) Individual Analysis
It evaluates both the learning situation of the students and the pertinence of the
questions. When the dispersion between S and P is quite large, some responses of the
students or the questions are abnormal. As the S-P table in figure 3, the rate of right
answer of Problem P1 is 8/15, which is quite high. But the number who gives a right
answer is just a half both above and below line P, which shows whether the problem
286 S. Zhong et al.
hGGzGs
OP
w
Gsw
G
G
h
OP
Fig. 4. Faultage
can be answered correctly is random distributed, and the students no matter with high
or low points answered the questions correctly random. So the differentiation index of
the problem is quite low and need to be checked. For example, there are two students
who both answered 5 problems right, but the problems for each student distributes
nearly halfly on both sides of line S ,which is also quite random and need to pay at-
tentions. The abnormality degree of the students and questions can be described by
alarm coefficient.
Calculation formula for the alarm coefficient:
COV j ( P)
W j (P) = 1 − (3)
COV j (C )
Wj(P) represents the alarm coefficient of problem j; COVj(P) represents the covari-
ance of the response pattern of problem j and the general response pattern; COVj(C)
represents the covariance of the general response pattern and the complete response
pattern. Complete response pattern means the response pattern that above the curve of
S are all 1s and the blow are all 0s.
Calculation formula for the covariance of the response pattern of problem j and the
general response pattern:
1 m 1
COV j ( p ) = ¦ ( X ij − m PX i )(SX i − v)
m i =1
(4)
In the formula, v means the average score of the students, and it can be calculated
as follows:
1 m
v= ¦ SX i
m i =1
(5)
The Design of Web-Based Intelligent Item Bank 287
COV j (P) ¦ X ij S X i − P X jV
W j (P) = 1 − = 1− i =1
PX (8)
C O V j (C ) j
¦ SX
i =1
i − P X jV
In the formula, COVi(S) presents the covariance of the response pattern and the
general response pattern; COVi(C) represents the covariance of the general response
pattern and the response pattern of the student. Response pattern here means that the
left-side of S are all 1s and the right-side are all 0s. As calculating the alarm coeffi-
cient for problems, calculate COVi(S) and COVi(C), and then Wi(s):
n
¦X
j =1
ij PX j − SX i u
Wi (S ) = 1 − SX i (10)
¦ PX
j =1
j − SX i u
In the formula, u means the average correct time for each question:
1 n
u= ¦ PX j
n j =1
(11)
When the alarm coefficient of the student is high, it suggests that this student
makes mistakes in the relative easy problems but solves the relative hard ones. The
phenomena may result from the problems in the learning conditions .For example,
students are not of high learning enthusiasm such as to answer careless, they don’t
exert the potential abilities or answer just by guessing, etc.
When the alarm coefficient of the student is over high, it suggests that this problem
is answered wrong by students with high points but answered correctly by students
with low points. So the differentiation index of the problem is low and its use value
decreases in terms of score levels.
Experiments indicates that: when the value of Wj(P)or Wi(S) is over 0.6,teacher
should pay sufficient attentions to the problems of the students and courage them to
make great effort. Also the problem should be deleted or modified. [3]
288 S. Zhong et al.
5 Conclusion
At present, the system has already established and been on the test-run stage with the
basic functions. It can solve the shortcomings of running independent of the other
teaching links, lacking of education evaluation, intelligent instructing and statistics
analysis well. Yet further research and development should be made.
(1) Further Application of Information Push Technology
Although the system push information such as evaluation, knowledge points to
review after the functions such as statistics analysis and intelligent instructing,
yet the technology is necessary to extend its application in other aspects. There
are some examples. Push the teaching strategies after mining personalized in-
formation such as students’ appetite on study media and learning regulation;
Push more authoritive tests; Push the adaptive tests to students according to their
knowledge regarding knowledge points. [5]
(2) Componentizing the System
The system provides a series of services of automatic test paper generation, test-
ing, analysis and evaluation, which can provide strong support to the distant
education. Each link of this system is also an important link in the current web-
based distant education. The system can be separated into functional subsystems
based on component, which can integrated into the distant education supporting
platform seamlessly and combine with the network courses closely.
(3) Applying the Item Response Theory to the Adaptive Testing
Compare the IRT with classical test theory, the theoretical system of the former is
built based on more complicate mathematical model with more accurate concepts
and theoretical derivation. It’s suitable for the adaptive testing. The testing system
asks the student one or more questions, then chooses the questions which can
evaluate the student most well and truly by analyzing the completed questions. In
this way, on one hand it can make the test corresponds with the student’s ability
level and then provides the more accurate information to evaluate the students’
ability. On the other hand, it can shorten the testing duration effectively.
The Design of Web-Based Intelligent Item Bank 289
References
1. Yu, S., He, K.: Design and Realization of the Network Test Bank System. J. China Distance
Education 164, 53–57 (2000)
2. Jing, Y.J., Zhong, S.C., Li, X., Li, J.-N., Cheng, X.C.: Using instruction strategy for a Web-
based Intelligent Tutoring System. In: Technologies For E-learning and digital entertain-
ment, vol. 3942, pp. 132–139 (2006)
3. Fu, D.: Education Information Processing. Peking (2001)
4. Zhong, Y.J., Liu, J., Zhong, S.C., Zhang, Y.M., Cheng, X.C.: Programming of informatized
instructional design platform for physics. In: Technologies for E-learning and Digital Enter-
tainment, vol. 3942, pp. 171–177 (2006)
5. Liu, L., Wang, W.: Design of Web-Based Exam-question with Self-study and Adaptive Ad-
justing. J. Computer System Application 4, 45–47 (2006)
Methods on Educational Resource Development and
Application
Z. Pan et al. (Eds.): Edutainment 2008, LNCS 5093, pp. 290–301, 2008.
© Springer-Verlag Berlin Heidelberg 2008
Methods on Educational Resource Development and Application 291
environment, the instructional design basis and so on. Thus, the developed resource
cannot bring advantages of network into play maximally, cannot solve key difficulties
in learning and teaching process. The developed resource and software are too loose
in global area and too centralized in local area, which have no share functions.
Teachers cannot use them smoothly. On rules and approaches of subject-specific
resource development, animation developing tools, resource integration tools and
excellent resource structure, there are lack of normative studies. These lead to the
lower efficiency and quality of resource establishment. On function, structure and
technology of resource and their support software, less work has been done. The
developed resource and software cannot meet teachers’ needs. On categoricalness and
consistency of resource, there exist more problems such as incomplete, incompatible
and conflicting phenomenon [4].
On resource application and service system, there are no more modes and
approaches for the integration of IT and subject, namely, no operating regulation. The
existing application modes and approaches have no enough scientificaalness,
hierarchy, excellent evaluation system (for example, what is the good integration
class?), effective service support system and share system.
Instructional design under the environment of network means to provide students
materials by utilizing network resource. The communication between teachers and
students or among students mainly uses words. There are no communication ways for
the process of solving problems. Instructional activities under the environment of
network lack of support system on learning, directing, supervising, evaluating and
feedback (for example, the supporting system in the course of investigative learning).
More important thing is that the development and change of IT and instructional
modes are so fast that teachers cannot adapt to these dynamically. Clearly, how to
provide teachers effective application support system has become more and more
important [5].
According to the above, this paper presents methods of ERDA in the course of
curriculum implementation, the curriculum resource structure and integration model,
evaluation factors of resource and curriculum integration, and methods on integrating
intelligent resource and establishing distributed resource system by experimental
study.
3 Methods of ERDA
ERDA should be considered from the sufficient integration of IT and curriculum.
Developer should grasp the nature of integration. Methods of ERDA are multifarious,
each school and teacher have their own methods according to their concrete
conditions. Each method has its advantages and shortcomings. At present, there is no
ideal method of ERDA yet. In the following, we contribute a set of ERDA methods
based on the course of curriculum implementation:
Among these steps, instructional mode and process design are the most important
step. If they are not reasonable and scientific, the diagnosis results are unacceptable.
In the course of instructional mode and process design, some factors should be
considered. The following are the key factors:
(1) what are instructional purposes? Where do students’ learning motivations come
from and how to inspire them? Students’ learning motivations come from their needs
of life instinct and curiosity. The learning purposes are to educate abilities, improve
diathesis and master knowledge and information. Concretely, they include mastering
not only basic knowledge but also corresponding knowledge system, logical thinking
approach and essential information based on the frame of abilities. Thus, from the
angle of psychology and education, learning contents that are suitable for the needs of
students’ life instinct and curiosity should be arranged for students.
(2) factors on affecting learning efficiency are multifarious, thereinto, age and
information form are the most important factors among them. Different ages need
different learning modes. People’s means for obtaining information are mainly from
five sense organs. Information forms affect directly the received efficiency. Thus,
from the angle of psychology and education, the effective learning environment,
reasonable instructional process and mode that is subject to the age character should
be made for students.
(3) factors on relationship between knowledge education and innovation education
should be considered. On knowledge education, its goal is to master knowledge and
then change them into skills. Normally, Students educated in this way just can do the
repeated work. However, society needs persons with innovation ability. Thus, the
instructional result should be to make students think about more questions instead of
no questions.
The role that IT plays in the course of curriculum implementation is subject to the
sufficient analysis on IT advantages and features. Analysis on IT advantages and
features should be from multi-angles such as in which parts of instruction IT should
be applied, what kinds of learning environment can be made by IT, etc.
The advantages and features of IT include dynamic interactivity, independent
usability, from stillness to movement, from abstract to concrete, from micro-state to
normal-state, virtuality, simulation, fast transmissibility, real-time, amplitude, etc.
These features are good for inspiring students’ learning interest, integrating excellent
teachers’ and experts’ wisdom together and inheriting effective instructional modes
and approaches.
In the course of IT and curriculum integration, we should do more research on IT
method. IT can set up multifarious instructional environment. Instructional
environment is divided into the following types: multimedia classroom, computer
classroom, campus network and internet. It is necessary to study their features and
application. For each instructional environment, we analyze it from the following
aspects: instructional modes, instructional steps and needed condition, etc.
Methods on Educational Resource Development and Application 295
4.4 Internet
5 Resource Architecture
Definition 6: curriculum resource system means all kinds of resource types, which
includes some based on curriculum criteria, some based on textbook version, some
based on test resource, some based on public resource among curriculums and so on.
Definition 7: textbook version resource means instructional resources organized by
some textbook versions. These resources mainly focus on resolving difficulties that
general instructional means cannot solve during the course of instructional modes
implementation. They are organized by curriculum unit, subject, chapter, passage,
item and so on.
Definition 8: material granularity means knowledge and skill, process and method as
well as the scale of sensibilities value implemented by these materials.
Definition 9: material share means materials that can be used in two or more than two
courseware. There are two means for material share, one is courseware share for the
same content, and another is courseware share for the different content.
Definition 10: curriculum criterion resources mean resources that are organized
through basic knowledge system, encyclopedia information, special topics, test
training questions, learning rules, teaching rules, etc. They don’t depend on
textbook’s version. Some textbook’s resources can be built based on them.
Definition 11: public resource means resource that reflect people’s living environment
and can be shared by subjects. It includes natural and humanistic resources. Typical
public resources include information on nation, region, city, organization and people
as well as some natural environment information on mountain, river, lake, ocean and
so on.
Definition 12 test training resource means resource that can be used to test if students
understand curriculum knowledge or not and to discover where problems exist. It
includes curriculum forward model, curriculum backward model as well as related
test questions and papers.
Figure 2 illustrates the structure of curriculum resources.
Curriculum resource can be divided into three hierarchies from figure 2, namely,
basic hierarchy, category hierarchy, application hierarchy. Resources in basic
hierarchy are the basic curriculum resource types; they have the maximal share
degree, normally, they are 100 percent share. Resources in category hierarchy are
integrated through specific questions; they have the higher share degree. Resources in
application hierarchy are integrated for the concrete applications; they have the lower
share degree.
Methods on Educational Resource Development and Application 297
Resource evaluation should be considered from multiple factors. The following are
key factors:
(1) the proportion of usable resource in each class;
(2) the proportion of share resource in each class;
(3) the granularity of share resource;
(4) the distributing uniformity of total resources;
(5) the reliability of resource design
(6) the validation degree of resource used in instruction;
(7) the freshness of resource update.
(8) the satisfaction degree between resource and curriculum instructional needs ˗
(9) the leading degree of resource.
The function of each factor is different, some of them are sufficient, some of them
are necessary and some of them are sufficient and necessary. Each factor should be
assigned to a weight in the course of evaluation.
BFormula
After getting all information, compute the total score by using the following formula,
0S100 .
(1)
S=p1×p2×p3×(p4×r4+p5×r5+p6×r6+ p7×r7+ p8×r8)
CWeight Assignment
8 Conclusions
In the course of ERDA, it should be to make clear the orientation; study curriculum
types and difficulties existing in instructional process; arrange learning contents that
are suitable for students; select learning process and environment that are helpful for
students; cultivate logical thinking ability while applying visual technology; think
about integration between subjects; establish teachers’ continuing education system;
improve application and management system; take integration as a system
engineering.
Try to form a leading modes and methods system: modes and methods system on
the integration of IT and different content type; on the establishment of resource and
software structure; on the application of resource and software. ERDA needs
educational experts, IT expert, officer and teachers work together to make great
progresses.
Methods on Educational Resource Development and Application 301
References
1. Nan, G.: Several theoretical and practical problems on education informatization. J. E-
Education Research 11, 3–6 (2002)
2. Sang, X., et al.: Strategy on district educational informatization. J. E-Education Research 3,
8–11 (2005)
3. Jin, Y., Zhong, S.: Analysis on modern long-distance instructional modes and support
platform. J. Educational Technology Research 4 (2003)
4. He, K.: Nature of e-Learning–integration of IT and curriculum. J.E-Education Research 1,
3–6 (2002)
5. Gu, X., Zhu, Z.: Implementation mode on teachers professional development. J. China
Educational Technology 3, 5–8 (2005)
6. Wu, J., Yu, S.: General problems and countermeasure for IT and curriculum integration. J.
E-Educational Research 10, 70–72 (2006)
7. Tang, Q.: Analysis on errors of IT and curriculum integration. J. E-Education Research 10,
70–72 (2006)
8. Zhong, S.: Research on resource and software of IT and curriculum integration. J. E-
Education Research 3, 53–56 (2005)
9. Zhong, S.: Model and method on IT and curriculum integration. J. China Educational
Technology Research 9 (2003)