Available online at www.sciencedirect.

com

Computers & Education 51 (2008) 955–967

www.elsevier.com/locate/compedu

Research and trends in the field of e-learning from 2001 to

2005: A content analysis of cognitive studies in selected journals
Meilun Shih a, Jui Feng b, Chin-Chung Tsai a,*,1

a
Graduate School of Technological and Vocational Education, National Taiwan University of Science and Technology,
#43, Sec. 4, Keelung Road, Taipei 106, Taiwan
b
Department of Learning and Digital Technology, Fo Guang University, #160, Linwei Rd., Jiaosi, Yilan County 26247, Taiwan

Received 1 January 2007; received in revised form 7 September 2007; accepted 4 October 2007

Abstract

This paper provided a content analysis of studies in the field of cognition in e-learning that were published in five Social
Sciences Citation Index (SSCI) journals (i.e. Computers and Education, British Journal of Educational Technology, Innova-
tions in Education and Teaching International, Educational Technology Research & Development, and Journal of Computer
Assisted Learning) from 2001 to 2005. Among the 1027 articles published in these journals from 2001 to 2005, 444 articles
were identified as being related to the topic of cognition in e-learning. These articles were cross analyzed by published
years, journal, research topic, and citation count. Furthermore, 16 highly-cited articles across different topics were chosen
for further analysis according to their research settings, participants, research design types, and research methods. It was
found from the analysis of the 444 articles that ‘‘Instructional Approaches,’’ ‘‘Learning Environment,’’ and ‘‘Metacogni-
tion’’ were the three most popular research topics, but the analysis of the citation counts suggested that the studies related
to ‘‘Instructional Approaches,’’ ‘‘Information Processing’’ and ‘‘Motivation’’ might have a greater impact on subsequent
research. Although the use of questionnaires might still be the main method of gathering research data in e-learning cog-
nitive studies, a clear trend was observed that more and more studies were utilizing learners’ log files or online messages as
data sources for analysis. The results of the analysis provided insights for educators and researchers into research trends
and patterns of cognition in e-learning.
Ó 2007 Elsevier Ltd. All rights reserved.

Keywords: E-learning; Cognition; Research trend; Journal analysis

1. Introduction

E-learning has become a widely accepted learning module in recent years (e.g. Cloete, 2001; Hodgson,
2002; McPherson, 2005). Although there are numerous studies that debate its appropriateness, discuss its
applications and practices, and investigate its influences on the facets of today’s education (e.g. Chan,

*
Corresponding author. Tel.: +886 2 27376511; fax: +886 2 27303217.
E-mail addresses: cctsai@mail.ntust.edu.tw, cctsai@mail.nctu.edu.tw (C.-C. Tsai).
1
The authors contributed equally to this paper.

0360-1315/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compedu.2007.10.004
956 M. Shih et al. / Computers & Education 51 (2008) 955–967

Hue, Chou, & Tzeng, 2001; Davies & Graff, 2005; Naeve, Lytras, Nejdl, Balacheff, & Hardin, 2006), the
term itself still lacks a clear definition. Some related terms that share similar characteristics with e-learning
include distributed learning, online learning, web-based learning, distance learning, network learning and
technology-based learning (Wentling et al., 2000). After thoroughly reviewing the definitions of all these
terms as proposed by prior researchers, Wentling et al. (2000) suggest that ‘‘e-learning is the acquisition
and use of knowledge distributed and facilitated primarily by electronic means, p. 5’’. Both its asynchro-
nous and synchronous features allow learners to access it without time or location barriers. In an e-learn-
ing environment, the learning process is more self-paced and self-motivated. This kind of experience is
quite different from what learners might experience in traditional teaching and learning environments.
Learners, on the one hand, have more control and flexibility in their learning, but on the other hand, they
need to take more responsibility for their own learning.
In educational research, cognition in learning has been studied from various perspectives. Because of the
increasing amount of technology integration in the learning environment, more and more studies focus on
examining ‘‘how people learn with technology’’. As Pea (2000) indicated in his book Technology and Learning,
‘‘technology as instruments of thinking, cognitive technologies, are at the heart of the human condition, p.
XV’’. He believes that because computing technology integrates and elaborates the features and uses of other
technological forms, its interactive capabilities provide a new representational form for thinking. Conse-
quently, an analysis of technology-enhanced or e-learning studies from a cognitive perspective is quite essen-
tial to acquire a better understanding of how advanced technology may help learning processes.
Content analysis and citation analysis of published articles in academic journals has been conducted in a
variety of professional fields such as psychology, science education, and instructional technology. For exam-
ple, in the field of psychology, Howard, Cole, and Maxwell (1987) and Smith et al. (1998) reviewed the
research papers published in selected American Psychological Association (APA) journals. Eybe and Schmidt
(2001) and Tsai and Wen (2005) examined research papers from some science education journals; and Rourke
and Szabo (2002) analyzed articles that were published in Journal of Distance Education from 1986 to 2001.
These studies provided insightful information about overall research trends and identified important scholars
and papers in the fields. Topics about cognition in education and e-learning have been discussed by many
researchers for a period of time, and many educators highlighted the importance of studying learners’ cogni-
tion involved in e-learning environments (e.g. Amichai-Hamburger, Kaynar, & Fine, 2007; Hess, 1999; Stahl,
2005). In recent years, the application of e-learning has been enhanced by the rapidly developing online tech-
nologies. We therefore felt that it was necessary to conduct a content analysis for the references of future stud-
ies which are interested in related issues.

2. Research purpose

The purpose of the paper was to investigate the research trends in the field of cognition in e-learning. Arti-
cles that were published by five journals included in the Social Sciences Citation Index (SSCI) database (i.e.
British Journal of Educational Technology, Innovations in Education and Teaching International, Computers and
Education, Educational Technology Research & Development, and Journal of Computer Assisted Learning)
between the years 2001 and 2005 were analyzed by publication year, journal, and research topic categories.
Results for the most and least published research topics can indicate the overall research trends and patterns
in the field. In addition, the citation counts of each article were used to further evaluate the impact of pub-
lished articles and the use made of them (Garfield, 1972; Harter & Nisonger, 1997). This can also provide
researchers with good examples of well-recognized articles in related fields. By analyzing highly-cited articles,
researchers can recognize more influential works in studied areas. Therefore, three highly cited articles from
each research topic category were further analyzed according to their participants, research setting, research
design type, and research methods.
The results of the analysis might outline the research trends and patterns of the articles related to cognition
in e-learning in recent years. It is hoped that the results of this study can provide both experienced and novice
educators and researchers in the fields of cognition, education, and learning technology with some insightful
ideas about the publishing trends of research studies in major journals. Especially for young scholars, this
analysis can help them to not only identify contemporary research topics, methods and trends, but also to
 M. Shih et al. / Computers & Education 51 (2008) 955–967 957

understand influential works and individuals in their major subject domains (Gall, Ku, Gurney, Tseng, & Yeh,
2004; Tsai & Wen, 2005). In addition, knowing the trends of recent research studies can help policy makers in
related fields to make plans to promote further study in the future. The research questions addressed by this
study were therefore:

1. What percentage of the articles published in these journals were related to cognition in e-learning?
2. What research topics related to cognition in e-learning were published in the journals in these five years?
3. What were the relatively popular research topics related to cognition in e-learning published in the journals?
4. What were the highly cited research articles in the SSCI related to cognition in e-learning in these journals?
5. What was the trend of the published cognition in e-learning articles in these five journals in recent years?

3. Method

3.1. Research papers for analysis

This study used papers published by five major educational journals that are included in the SSCI from
2001 to 2005 as the research sample to examine the research trends in the study of cognition in e-learning.
The five journals are: British Journal of Educational Technology (BJET), Innovations in Education and Teaching
International (IETI), Computers and Education (CE), Educational Technology Research & Development
(ETR&D), and Journal of Computer Assisted Learning (JCAL). These journals were chosen because they
are all categorized as being educational-technology-related in the SSCI.
This study only analyzed papers that were identified as ‘‘articles’’ in the SSCI. Other types of papers such as
‘‘book reviews,’’ ‘‘reviews,’’ and ‘‘editorial materials’’ were all excluded from this study. Excluding these other
types of papers, there were a total of 1027 articles published by these five journals from 2001 to 2005. Based on
these articles, a content analysis was carried out, using article abstracts and publication information indexed in
the SSCI. In a few cases in which the publication information was insufficient or missing from the SSCI, the
researchers used the complete articles for analysis. The selection of articles was processed by two post-doctoral
researchers in educational technology and further validated by a professor in the field. This procedure iden-
tified 444 articles that were judged to be relevant to the topic of cognition in e-learning. Descriptive statistics
were used to analyze and report the data.

3.2. Research topic analysis

Based on the methodology of content analysis, the research topics were first categorized related to cognition
in e-learning into seven tentative categories: motivation, information processing, instructional approaches,
learning environment, prior experience, metacognition, and cognitive psychology characteristics. For each cat-
egory, some sub-categories were classified. Definition of cognition in psychology, experts’ opinions, descrip-
tors in the Education Resources Information Center (ERIC), and article keywords in the SSCI were all
used as references in developing tentative categories. During the data analysis process, these categories were
refined continually by using the constant-comparative method (Lincoln & Guba, 1985). Categories and sub-
categories were confirmed until all articles were reviewed.
The final analysis framework consisted of the following research topic categories and sub-categories:

1. Motivation. Articles in this research topic explore some fundamental variables related to learners’ motiva-
tion, such as their beliefs or attitudes toward e-learning. Some articles also discussed the inter-relationships
among participants’ motivation, the usage of technology, and learning outcomes. This category was further
divided into three sub-categories: beliefs, attitudes, and behavioral change.
2. Information processing. This category includes articles that discuss participants’ information processing in
an e-learning environment. Based on different research foci, this category was further divided into six sub-
categories: individual difference, information seeking, information management, critical thinking, decision
making, and problem solving.
958 M. Shih et al. / Computers & Education 51 (2008) 955–967

3. Instructional approaches. This topic of research usually discusses or examines different types of instructional
approaches used in an e-learning environment. Foci include how these approaches might help study par-
ticipants’ to achieve better learning outcomes. It was further divided into five sub-categories: cooperative
learning, collaborative learning, contextual learning, situated learning, and problem-based learning.
4. Learning environment. This category includes studies that outline participants’ learning experiences in an
e-learning environment. The development of innovative e-learning environments was also classified in this cat-
egory. Interactive learning environment and learning community are two sub-categories in this research topic.
5. Prior knowledge. Articles in this research category usually investigate the influences of prior knowledge on
participants’ learning processes and outcomes. There are two sub-categories in this topic: technology
knowledge and technology experiences. While the first explores participants’ ‘‘knowing’’ about technology,
the latter is concerned with their actual experiences of ‘‘using’’ the technology.
6. Metacognition. This topic category typically includes articles that investigate participants’ metacognition
status and its influences during the process of e-learning. There are four sub-categories in this research
topic: planning, visualization, perception (and awareness), and self-evaluation.
7. Cognitive psychology characteristics. Articles that focus on examining participants’ cognitive psychological
characteristics in their e-learning processes and outcomes were classified into this research topic category.
Sub-categories include: schemata, concept maps, mental models, cognitive loads, and cognitive styles.

It should be noted that many articles addressed more than one of the aforementioned topic categories;
therefore, one article might be coded into two (or more) categories (or sub-categories) in this part of the con-
tent analysis. The analysis or coding of these articles was processed by two post-doctoral researchers and
resulted in an agreement of 0.84. The discrepancy was resolved upon discussion. The analysis of selected arti-
cles was further validated by a professor.
In addition, highly-cited papers in each research topic category were further analyzed in detail. Analyzed
items included research participants, research settings, research design, and research methods. Analyzed
results can provide researchers with an opportunity to understand the most influential works and individuals
in related subject majors.

4. Results and discussion

4.1. Percentage of articles related to cognition in e-learning

First, the results of this study revealed that almost half (43.2%, 444 of 1027 articles) of the articles published
in five educational journals from 2001 to 2005 were related to the field of cognition in e-learning. The analysis
by year and journal is presented in Table 1.
From the above analysis, it can be seen that from 2001 to 2005, BJET had 46.8%, IETI had 58.4%, CE had
37.1%, ETR&D had 28%, and JCAL had 42.5% of the published articles related to the study of cognition in e-
learning. 2003 was the year in which the highest percentage of related articles was published (49.8%), while
2002 had the least (37.4%). Among the five journals, IETI had the highest percentage of related articles in
2002, 2003, and 2004. Fig. 1 graphically presents the percentage of the publishing trend of related articles
in the five journals from 2001 to 2005.

Table 1
Number of articles related to the study of cognition in e-learning in each year (from 2001 to 2005) and journal
2001 2002 2003 2004 2005 Total
BJET 24/47 (51.1%) 18/54 (33.3%) 25/52 (48.1%) 30/56 (53.6%) 41/86 (47.7%) 138/295 (46.8%)
IETI 16/37 (43.2%) 20/32 (62.5%) 25/34 (73.5%) 20/30 (66.7%) 13/28 (46.4%) 94/161 (58.4%)
CE 16/42 (38.1%) 19/50 (38.0%) 20/46 (43.5%) 16/47 (34.0%) 15/47 (31.9%) 86/232 (37.1%)
ETR&D 7/22 (31.8%) 6/30 (20.0%) 8/26 (30.8%) 7/21 (33.3%) 7/26 (26.9%) 35/125 (28%)
JCAL 23/43 (53.5%) 16/45 (35.6%) 24/47 (51.1%) 14/41 (34.2%) 14/38 (36.8%) 91/214 (42.5%)
Total 86/191 (45%) 79/211 (37.4%) 102/205 (49.8%) 87/195 (44.6%) 90/225 (40%) 444/1027 (43.2%)
* The number of articles related to the study of cognition in e-learning in each year is highlighted.
 M. Shih et al. / Computers & Education 51 (2008) 955–967 959

80
70
60 BJET
50 IETI
40 CE
30 ETRD
20 JCAL
10
0
2001 2002 2003 2004 2005

Fig. 1. Percentage of articles that were related to cognition in e-learning in the five journals.

4.2. Research topic analysis

In addition to the analysis by year and journal, cognition categories and sub-categories were examined by
frequency. Although many articles discussed only one cognition topic (sub-category), there were still numer-
ous articles which covered two or more sub-categories. Table 2 shows the final frequency counts for each sub-
category.
Table 2 reveals that the most published research topics were ‘‘learning environment-interactive learning
environment’’ (110), followed by ‘‘instructional approaches-collaborative learning’’ (92), and ‘‘metacogni-
tion-perception and awareness’’ (75). These results are quite reasonable, as e-learning can provide adequately

Table 2
Frequencies of cognition category and sub-category in articles
Category Sub-category Frequency count
A. Motivation A.1 Beliefs 31
A.2 Attitudes 65
A.3 Behavioral change 28
B. Information Processing B.1 Individual difference 25
B.2 Information seeking 19
B.3 Information management 20
B.4 Critical thinking 19
B.5 Decision making 4
B.6 Problem solving 11
C. Instructional Approaches C.1 Cooperative learning 21
C.2 Collaborative learning 92
C.3 Contextual learning 26
C.4 Situated learning 16
C.5 Problem-based learning 36
D. Learning Environment D.1 Interactive Learning Environment 110
D.2 Learning community 42
E. Prior Knowledge E.1 Technology knowledge 15
E.2 Technology experience 32
F. Metacognition F.1 Planning 14
F.2 Visualization 13
F.3 Perception and awareness 75
F.4 Self-evaluation 26
G. Cognitive Psychology Characteristics G.1 Schemata 10
G.2 Concept map 18
G.3 Mental Model 7
G.4 Cognitive load 14
G.5 Cognitive style 23
960 M. Shih et al. / Computers & Education 51 (2008) 955–967

Table 3
Percentage of related articles of each category in each journal
Category A (%) Category B (%) Category C (%) Category D (%) Category E (%) Category F (%) Category G (%)
BJET 8.47 5.42 11.53 13.22 4.35 13.22 8.81
IETI 8.7 4.97 26.09 19.88 4.97 18.63 4.97
CE 10.78 12.07 16.81 14.22 3.44 9.91 3.88
ETR&D 8.00 7.2 10.40 9.60 2.40 8.80 8.00
JCAL 11.68 12.15 14.95 13.08 3.27 8.88 7.94
Notes: The top two categories for each journal are highlighted.
Category A: Motivation; Category B: Information processing; Category C: Instructional approaches; Category D: Learning environment;
Category E: Prior knowledge; Category F: Metacognition; Category G: Cognitive psychology characteristics.

interactive and collaborative learning contexts for students, and these characteristics may be the most signif-
icant features of e-learning (Chou & Tsai, 2002; Tsai, 2001). In recent years, researchers in e-learning have
conducted many studies to explore these features. On the other hand, ‘‘information processing-decision mak-
ing’’ (4), ‘‘cognitive psychology characteristics-mental model’’ (7), and ‘‘cognitive psychology characteristics-
schemata’’ (10) were the least published research topics. This is probably because topics such as mental models
and schemata usually involve in-depth analysis, but much e-learning research is still at an early stage. There
were therefore relatively fewer published articles related to these topics.
The published percentages of related articles in each category in the five journals from 2001 to 2005 are
presented in Table 3. For IETI, CE, ETR&D, and JCAL, most cognition in e-learning articles were related
to category C (instructional approaches) and category D (learning environment). However, BJET published
more articles related to ‘‘learning environment’’ (13.22%) and ‘‘metacognition’’ (13.22%). In general, the
results from Tables 2 and 3 indicate that ‘‘Instructional Approaches,’’ ‘‘Learning Environment,’’ and ‘‘Meta-
cognition’’ were the three most popular research topics from 2001 to 2005 in these journals. Although studies
have revealed that both using computers as metacognitive tools and metacognitively active participants could
enhance learning (Azevedo, 2002; Tsai, 2004; Zimmerman, 1989), the number of published articles related to
‘‘Metacognition’’ (except the sub-category of ‘‘perception and awareness’’) on e-learning was still not many in
this study. As the concepts of active learning and lifelong learning have become more and more popular and
important nowadays, the research topic of metacognition requires more attention.

4.3. An analysis of highly cited articles

In order to identify articles of higher quality or impact, the 444 related articles were further analyzed by
their citation counts in the SSCI (as on October, 17, 2006). As explained above, articles with more citation
frequencies are usually those that are better recognized by others in related fields. They probably present more
fundamental ideas about the issues for future research. Three articles with the highest citation counts in each
research topic category were selected for this analysis. Because several of the articles discussed topics from
several categories, a total of 16 articles were finally chosen. Their participants, research settings, research
design type, and research methods (data collection and data analysis) were further analyzed in detail. Table
4 shows the analysis results and SSCI citation counts of these 16 articles.
The citation counts of these 16 articles range from 5 to 18. The most highly cited articles fall into the
research categories of ‘‘information processing/instructional approaches’’ (Oliver & Omari, 2001; citation
counts = 18), focusing on individual difference, collaborative learning, and problem-based learning; and
‘‘information processing/cognitive psychology characteristics’’ (Ford & Chen, 2001; citation counts = 18),
which addressed information seeking and cognitive style; and ‘‘motivation’’ (Tsai, Lin, & Tsai, 2001;
citation counts = 17), which explored learners’ attitudes. All of the top three cited articles were published
in 2001. This is probably because articles that were published earlier have a greater chance of being accessed
and cited by other related studies. However, there was one study in 2004 about ‘‘instructional approaches’’
(Zurita & Nussbaum, 2004) that had 11 citation counts in the SSCI. Considering its year of publication,
it is quite a highly cited article. This result also suggests that papers related to the topics of ‘‘Instructional
Approaches,’’ ‘‘Information Processing’’ and ‘‘Motivation’’ may address some fundamental issues in
 M. Shih et al. / Computers & Education 51 (2008) 955–967 961

Table 4
An analysis of high-citation papers related to the study of cognition in e-learning
Topic Study (author, year, Participants and Research Data Data analysis SSCI
journal, research setting design/type collection citationa
page
information)
Motivation Tsai, Lin, and Tsai, 753 High school Descriptive Questionnaire Descriptive, 17
(2001). Computers & students at eight schools statistical analysis
Education, 37, 41–51 from northern, central, (factor analysis,
and southern Taiwan t-tests, ANOVA)
Motivation/ Mumtaz (2001). 360 Year 3 and year 5 Descriptive Questionnaire Descriptive (SD), 11
Metacognition Computers & pupils selected from statistical analysis
Education, 36, schools in the UK (ANOVA)
347–362
Motivation/ Metzger, Flanagin Study 1: 356 undergrad- Descriptive Questionnaire Descriptive (mean 11
Metacognition and Zwarun (2003). uate students enrolled in and SD), statistical
Computers & introductory communi- analysis (ANOVA)
Education, 41, cation courses in the
271–290 U.S.
Study 2: 436 undergrad-
uate students from the
same university used in
Study 1 and 307 nonstu-
dents recruited using the
‘‘snowball sampling’’
technique
Instructional Chang (2001). 137 participants (tenth Experimental Testing, Descriptive (mean 10
Approaches Journal of Computer graders) attending four Questionnaire and SD), statistical
Assisted Learning, earth science classes in analysis
17, 263–274. Taiwan (MANCOVA,
ANCOVA)
Information Oliver and Omari 240 college students (220 Descriptive Interviews, Descriptive 18
Processing/ (2001). Journal of on-campus and 20 >off- online (frequencies)
Instructional Computer Assisted campus) questionnaires
Approaches Learning, 17, 34–47
Information Ford and Chen 73 post-graduate Experimental Personal survey Statistical analysis 18
Processing/ (2001). British students from a range of (ANOVA andt-tests)
Cognitive Journal of departments at the
Psychology Educational University of Sheffield,
Characteristics Technology, 32, 5–22 UK
Information Papanikolaou, 10 Undergraduate Developmental Participants’ Descriptive 9
processing/prior Grigoriadou, students (with pilot answers to (frequencies)
experience/ Magoulas and study) questions
Learning Kornilakis (2002). embedded in
environment Computers & the scenario
Education, 39, and log files of
333–360 interactions
Instructional Zurita and 48 students ranging 6–7 Descriptive Classroom Descriptive (mean, 11
Approaches Nussbaum (2004). years in one low-income observations SD), statistical
Computers & elementary school of (videotapes), analysis (t-tests)
Education, 42, Santiago de Chile field notes,
289–314 interviews
Instructional Ligorio (2001). 40 primary or secondary Descriptive Chat logs, text- Discourse analysis 7
Approaches Computers & students (aged from 9 to based messages,
Education, 37, 14 years), 18 teachers, photos
103–125 and 6 researchers
(continued on next page)
962 M. Shih et al. / Computers & Education 51 (2008) 955–967

Table 4 (continued)
Topic Study (author, year, Participants and Research Data Data analysis SSCI
journal, research setting design/type collection citationa
page
information)
Learning Rovai (2001). 20 adult learners in Descriptive Questionnaire, Descriptive (mean 9
Environment Educational an online course at a online messages, and SD)
Technology Research large western state overall course
& Development, university in the USA statistical data
49(4), 33–48
Learning Pena-Shaff and 35 participants (22 Descriptive Outline or index Descriptive 8
Environment Nicholls (2004). undergraduates, 8 of BBS
Computers & graduates, and 5 postings,
Education, 42, university employees) transcripts of
243–265 in the U.S electronic
discussions
Learning Brusilovsky (2003). 64 Undergraduate Descriptive, Questionnaire Descriptive 8
Environment/ British Journal of students in a total of Developmental
Meta-cognition Educational two studies
Technology, 34,
487–497
Prior Experience Lazonder, Wilhelm Study 1: 36 secondary Developmental Questionnaire, Discourse analysis, 5
and Ootes (2003). students Study 2: 43 chat logs, Descriptive
Computers & secondary students
Education, 41,
291–308
Prior Experience Steel and Hudson 11 members of the Descriptive Interviews Interpretative 5
(2001). Innovations in full-time teaching
Education and staff at Sheffield
Training Hallam University
International, 38,
103–111
Cognitive Chang, Sung and 48 seventh-grade Experimental Testing, Descriptive (mean 10
Psychology Chen (2001). Journal students selected questionnaire and SD), statistical
Characteristics of Computer Assisted from three classes of analysis (ANOVA,
Learning, 17, 21–33 one junior high ANCOVA, v2 and
school in Taipei post hoc tests)
Cognitive Lin, Liu and Yuan 58 computer science Experimental Testing, Descriptive (mean 9
Psychology (2001). Journal of majors enrolled in a questionnaire and SD), statistical
Characteristics Computer Assisted mandatory analysis (ANOVA
Learning, 17, ‘‘Operating Systems’’ and t-tests)
420–432 course in Taiwan
a
As the record of October 17, 2006.

e-learning, and are therefore likely to gain more citations. These three topics may be perceived as the
research foundation for e-learning studies. It is also interesting to find that for the whole analysis of all
444 articles, ‘‘Instructional Approaches,’’ ‘‘Learning Environment,’’ and ‘‘Metacognition’’ are the three
most prevalent research topics, but the analysis of the citation counts suggests that the studies related to
‘‘Instructional Approaches,’’ ‘‘Information Processing’’ and ‘‘Motivation’’ may have a greater impact on
subsequent research. This is probably because the focus of most e-learning studies is on adapting traditional
teaching and learning approaches into the online environment; therefore, ‘‘Instructional Approaches’’ is
both the most published and the most cited research topic.
Among the 16 most highly cited articles, nine studies had participants who were adult learners, four studies
involved secondary school students, one involved elementary school students, one pre-school students, and
 M. Shih et al. / Computers & Education 51 (2008) 955–967 963

one study involved a combination of elementary, secondary and adult learners. On the whole, college students
(38%) and students in elementary and secondary schools (38%) were the most popular participants among the
16 articles. This result also shows that although one of the important functions of e-learning is its impact on
lifelong learning (e.g. Jameson, Ferrell, Kelly, Walker, & Ryan, 2006), current studies still focus on students in
formal school settings. It is reasonable to suggest that future studies should pay more attention to adult learn-
ers outside of formal school environments.
In terms of the type of research design, these papers were classified into three major types:

1. Experimental research, which typically involved an experimental group and a control group to test hypoth-
eses regarding certain treatments (Ross & Morrison, 1996).
2. Descriptive research, which gathered data from events or participants’ responses to describe, explain, val-
idate or explore a particular issue (Kunpfer & McLellan, 1996).
3. Developmental research, which systematically studied the design, development, and evaluation process of
certain educational interventions (Richey & Nelson, 1996). The development of some innovative e-learning
systems is also viewed in this type.

These are three major research types identified in the Handbook of Research for Educational Communication
and Technology (Jonassen, 1996).
As revealed in Table 4, the majority of studies are classified as descriptive research (10 of the 16 articles).
This result recalls Kunpfer and McLellan’s (1996) prediction that there would be more and more educa-
tional studies using descriptive research as their research method to investigate and explain the complicated
issues within education in recent years. In addition, three out of the four experimental studies’ research top-
ics were about ‘‘cognitive psychology characteristics.’’ This was probably because experimental research has
been a traditional research method in the field of studying learners’ psychology-related issues (Ross & Mor-
rison, 1996). This also suggests that researchers who intend to study learners’ cognitive psychology charac-
teristics may be likely to utilize an experimental research design to gain better research outcomes.
Furthermore, because descriptive research may combine qualitative and quantitative research design fea-
tures, it can better describe the complicated relationships between the different variables involved in a cer-
tain situation. Therefore, most ‘‘learning environment’’ studies used this approach to outline the
‘‘happenings’’ in an e-learning-based environment. The research design type of these 16 articles shows a
valuable reference for researchers who may be interested in doing studies related to these research topics
in the future.
More than half of the studies (10 articles) used the questionnaire as their primary data collection technique
or as one of their data collection methods. In these articles, some other methods, such as online log files, mes-
sages or postings (five studies), interviews (three studies), testing (three studies), survey (one study), and obser-
vation (one study), were also used. Although there are a variety of data collection methods, the use of
questionnaires appears to continue to be the main method of gathering research data in e-learning cognitive
studies. However, there is also a clear trend that many studies in this field used learners’ log files or online
messages as the data sources for analysis. This shows a difference between e-learning and traditional learning,
in that e-learning usually generates electronic log files of users’ learning processes. Future studies about e-
learning could try to make better use of this kind of data.
Finally, Table 4 also displays the methods of analyzing research data used by these articles. Among the
16 studies, seven used both descriptive and statistical analysis to analyze data, five utilized descriptive anal-
ysis, one study used statistical analysis alone, one employed discourse and descriptive analysis, one used dis-
course analysis, and one study employed interpretative analysis. While both descriptive and statistical
methods of analysis are most often used nowadays, the discourse analysis employed in two studies and
the utilization of interpretative analysis in one study also show good examples for qualitatively analyzing
data. Again, this is also probably because the online technology or databases used in e-learning can keep
records about the learners’ learning processes (e.g. discourse, online interactions). Therefore, it is more con-
venient for researchers to carry out discourse or interpretative research in online environments. The growing
value of the qualitative approach in conducting studies of cognition in e-learning may be revealed in the
future.
964 M. Shih et al. / Computers & Education 51 (2008) 955–967

5. Implications and conclusions

In this paper, the researchers categorized and analyzed articles related to cognition in e-learning that were
published in five SSCI journals from 2001 to 2005. Regarding the research topics that have been discussed in
these articles, on the whole, the analysis results indicate that ‘‘interactive learning environment’’ and ‘‘collab-
orative learning’’ are the two dominant topics presented in the five journals. As collaborative learning is usu-
ally delivered through interactive learning environments, and interactive learning environments are also often
designed to encourage learners’ collaboration experiences, many studies actually discussed both topics. Fur-
thermore, four out of the five journals (IETI, CE, ETR&D, and JCAL) had the highest percentage of articles
related to the topic of ‘‘instructional approaches,’’ and among these articles, ‘‘collaborative learning’’ (fre-
quency count rank #2 in the total of 27 sub-categories, shown in Table 2) and ‘‘problem-based learning’’ (fre-
quency count rank #6 in the total of 27 sub-categories) attracted the most research attention. This result
supports the constructivists’ viewpoint that instructional models of collaborative and problem-based learning
are essential in developing the constructivist framework for e-learning (Edelson, Pea, & Gomez, 1996; Savery
& Duffy, 1996).
This study also selected 16 highly-cited papers across different topic categories for further analysis.
Although e-learning focuses on all learners and lifelong learning, it was found that colleges and primary/
secondary schools were the most popular research settings, and that the students in these schools were also
the most common study participants. Therefore, it is suggested that future studies focus more on adult
learners outside of formal school settings. On the whole, in line with Kunpfer and McLellan’s (1996) sug-
gestion that the descriptive research method would become the main trend in educational studies, it was
indeed the dominant research design type of these 16 articles. Although more than half of them (10 articles)
used questionnaires as their primary data collection technique or as one of the methods, still as many as five
studies utilized learners’ log files or online messages as the data sources for analysis. In addition, only four
out of the 16 articles used the experimental research method. This seems to confirm the tendency of edu-
cational research from the 1980s that was suggested by a number of researchers. As Clark indicated in
his series of papers about instructional technology research (e.g., Clark & Snow, 1975; Clark, 1983,
1989), the dilemma of the emphasis between internal validity (control of variables to allow accurate infer-
ence) and external validity (representativeness or generalizability of findings to real life settings) in the nat-
ure of experimental study reduces its status in social sciences, especially in the field related to instructional
technology research. Salomon and Clark (1977) claimed that ‘‘the more it (experimental work) moved into
the deeper layers of understanding media, the farther away it went from the world of education. And in
spite of its improved quality, it nevertheless fell short of accomplishing the objective of improving educa-
tional practice’’, p. 106. This perspective may be the reason for the relatively low frequency of ‘‘experimen-
tal research’’ in the analysis of this study.
There were five studies which used descriptive analysis alone, seven employing it with advanced statistical
analysis, and one using it with discourse analysis. In total, more than 3/4 of the 16 articles used descriptive
analysis as their data analysis method, suggesting that most of these papers had a quantitative feature for pre-
senting research results. However, there were still a few studies which used a more qualitative approach.
Recently, educational researchers have argued about the commonalities between quantitative and qualitative
research, and have advocated mixed method research as a new research paradigm (e.g. Johnson & Onwuegbu-
zie, 2004). Researchers in this field may also conduct studies using mixed methods. In fact, some of the highly-
cited articles reviewed in this paper have taken this approach (e.g. Lazonder, Wilhelm, & Ootes, 2003; Zurita
& Nussbaum, 2004). It may become a major trend in this research field. In addition, because online technology
or databases in e-learning keep records of learners’ learning processes, it allows researchers to make interpre-
tations based on log files, discourse, or interaction records. It seems that the new study field of e-learning has
brought researchers new methods of data collection.
For published journals, half of the 16 articles were published in CE, which has the highest impact factor
ranking in the SSCI among these five journals in terms of educational technology (impact factor = 0.968 in
2005). In addition, 10 out of the 16 articles were published in 2001, which is probably because articles pub-
lished earlier might be more accessible to other researchers and therefore have more opportunities to be cited
by other papers.
 M. Shih et al. / Computers & Education 51 (2008) 955–967 965

In order to further discuss the trends of educational research from a broader perspective, the content of the
American Vision Report 2002 was analyzed. This included 14 papers and was published by the Department of
Commerce of the United States. Several scholars and educational experts were invited to discuss issues such as
the technological environment and its relationship with educational development in the future, from various
points of views. According to the report, one general vision of future education is to create an interactive
learning environment without time or space limitations. Learners are encouraged to learn whatever, wherever
and whenever they prefer. Global collaboration and problem-based learning are the two most widely-used
learning strategies in future classrooms. These predictions are coherent with the analysis results in this study
that ‘‘interactive learning environment’’ and ‘‘collaborative learning’’ are the two primary research topics in
published articles about cognition in e-learning from 2001 to 2005. Because the interrelationship between these
two topics is close, many articles discussed them both.
In addition, because ‘‘student-centered instruction’’ is likely to become the primary trend in education by
2020, motivation was another repeatedly discussed issue in the report. In this study, motivation ranks num-
ber four of the seven main categories that were used to analyzed cognition in e-learning articles. In the
vision report papers, educational scholars and experts not only talked about students’ learning motivation,
but also teachers’ teaching motivation. How to maintain and enhance students’ learning motivation and
teachers’ teaching motivation in this constantly changing educational environment will be an essential issue
for future studies. In the future, educators may need to pay more attention to studying learners’ and teach-
ers’ motivation in e-learning environments. In the report, some papers suggested that ‘‘visualization’’ and
‘‘simulation’’ could be two possible ways to deliver this kind of motivating and interactive instruction.
Because the students themselves will become the center of the whole learning process in future education,
their learning cognition and memory are two related issues that are studied in the report. Therefore, it is
suggested that there be more research effort placed on further studying learners’ cognition and memory
related to e-learning.
Similar perspectives can be found in South Africa’s Draft white paper on education: transforming learning
and teaching through ICT (2003). It also addressed the importance of developing learners’ ‘‘critical thinking,’’
‘‘decision-making,’’ and ‘‘problem-solving’’ skills through ‘‘collaborative learning’’ environments. Learners’
individual differences should also be recognized to promote their motivation in lifelong learning. In addition,
Britain, which is well-known for its success in e-learning developments, also addressed the same ideas. Accord-
ing to its government-commissioned report Teaching and Learning 2020, ‘‘personalizing learning and teach-
ing’’ will become one of the central roles in transforming Britain’s future education. It suggested that in
order to prepare children and young people for the challenges ahead, education should focus on the differences
of each individual learner, helping him/her to develop the essential knowledge, skills, understanding and atti-
tudes which are required in a changing world.
Although the nature of this study is particularly descriptive, it is hoped that the results can provide educa-
tors and researchers in the fields of cognition, education, and learning technology with some insightful ideas
about the publishing trends of research studies in major journals. This type of journal analysis is also very
beneficial for young scholars in the field (Tsai & Wen, 2005). The analysis can help them to not only identify
contemporary research topics, methods and trends, but also to understand the influence and influencers in
their major subjects (Gall et al., 2004). In addition, knowing the trends of recent research studies can help pol-
icy makers in related fields to make future plans in terms of these issues.

6. Limitations of the research

Although citation analysis was considered as one of the objective measurements of paper evaluation, there
were still some disadvantages with this method, such as self-citation, negative citation, and gratuitous citation
(Chu, Hsu, & Yu, 1997). Therefore, it is suggested that a combination of citation analysis and other analysis
techniques can be applied in future studies in this domain. In addition, this study focused primarily on Eng-
lish-based SSCI journals and articles. Since cognition in e-learning is a global issue, it would be interesting to
see future studies include investigation in other contexts or publications in other languages as well. In order to
understand the continuous trends and patterns in this discussed issue, it is also recommended that similar stud-
ies should be conducted with other journals, and be repeated every five years.
966 M. Shih et al. / Computers & Education 51 (2008) 955–967

Acknowledgements

Funding of this research work was provided by the National Science Council, Taiwan, under Grant num-
bers NSC 94-2524-S-009-003, 95-2511-S-011-003-MY3.

References

References marked with an asterisk indicate highly-cited articles in the analysis of this paper

Amichai-Hamburger, Y., Kaynar, O., & Fine, A. (2007). The effects of need for cognition on Internet use. Computers in Human Behavior,
23, 880–891.
Azevedo, R. (2002). Beyond intelligent tutoring systems: Using computers as metacognitive tools to enhance learning? Instructional
Science, 30, 31–45.
*Brusilovsky, P. (2003). Adaptive navigation support in educational hypermedia: The role of student knowledge level and the case for
meta-adaptation. British Journal of Educational Technology, 34, 487–497.
Chan, T. W., Hue, C. W., Chou, C. Y., & Tzeng, O. J. L. (2001). Four spaces of network learning models. Computers & Education, 37,
141–161.
*Chang, C. Y. (2001). A problem-solving based computer-assisted tutorial for the earth sciences. Journal of Computer Assisted Learning,
17, 263–274.
*Chang, K. E., Sung, Y. T., & Chen, S. F. (2001). Learning through computer-based concept mapping with scaffolding aid. Journal of
Computer Assisted Learning, 17, 21–33.
Chou, C., & Tsai, C.-C. (2002). Developing Web-based curricula: Issues and challenges. Journal of Curriculum Studies, 34,
623–636.
Chu, C. Y. Cyrus, Hsu, S. K., & Yu, R. -R. (1997). Proceedings of the National Science Council (Part C: Humanities and Social Sciences).
7(3), 445–473.
Clark, R., & Snow, R. (1975). Alternative designs for instructional technology research. AV Communication Review, 23, 373–394.
Clark, R. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459.
Clark, R. (1989). Current progress and future directions for research in instructional technology. Educational Technology Research and
Development, 37(1), 57–66.
Cloete, E. (2001). Electronic education system model. Computers & Education, 36, 171–182.
Davies, J., & Graff, M. (2005). Performance in e-learning: Online participation and student grade. British Journal of Educational
Technology, 36, 657–663.
Edelson, D. C., Pea, R. D., & Gomez, L. (1996). Constructivism in the collaboratory. In Brent G. Wilson (Ed.), Constructivist learning
environments: Case studies in instructional design (pp. 151–164). Educational Technology Publications.
Eybe, J., & Schmidt, H.-J. (2001). Quality criteria and exemplary papers in chemistry education research. International Journal of Science
Education, 23, 209–225.
*Ford, N., & Chen, S. Y. (2001). Matching/mismatching revisited: An empirical study of learning and teaching styles. British Journal of
Educational Technology, 32(1), 5–22.
Gall, J. E., Ku, H. Y., Gurney, K., Tseng, H. W., & Yeh, H. T. (2004). An analysis of citation patterns in ETR&D. In The proceedings of
association for educational communications & technology (AECT) conference, Chicago, IL, October 19–23, 2004.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.
Harter, S., & Nisonger, T. E. (1997). ISI’s impact factor as misnomer: A proposed new measure to assess journal impact. Journal of the
American Society for Information Science, 48(12), 1146–1148.
Hess, B. (1999). Graduate student cognition during information retrieval using the World Wide Web: A pilot study. Computers &
Education, 33, 1–13.
Hodgson, V. E. (2002). The European union and e-learning: An examination of rhetoric, theory and practice. Journal of Computer Assisted
Learning, 18, 240–252.
Howard, G. S., Cole, D. A., & Maxwell, S. E. (1987). Research productivity in psychology based on publication in the journals of the
American Psychological Association. American Psychologist, 42, 975–986.
Jameson, J., Ferrell, G., Kelly, J., Walker, S., & Ryan, M. (2006). Building trust and shared knowledge in communities of e-learning
practice: Collaborative leadership in the JISC eLISA and CAMEL lifelong learning projects. British Journal of Educational
Technology, 37, 949–967.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational
Researcher, 33(7), 14–26.
Jonassen, D. H. (Ed.) (1996). Handbook of research for educational communications and technology. (2nd ed). Mahwah, NJ: Lawrence
Erlbaum Association.
Kunpfer, N., & McLellan, H. (1996). Descriptive research methodologies. In D. H. Jonassen (Ed.), Handbook of research for educational
communications and technology (2nd ed.) (pp. 1196–1212). Mahwah, NJ: Lawrence Erlbaum Association.
*Lazonder, A. W., Wilhelm, P., & Ootes, S. A. W. (2003). Using sentence openers to foster student interaction in computer-mediated
learning environments. Computers & Education, 41, 291–308.
 M. Shih et al. / Computers & Education 51 (2008) 955–967 967

*Ligorio, M. B. (2001). Integrating communication formats: Synchronous versus asynchronous and text-based versus visual. Computers &
Education, 37, 103–125.
*Lin, S. S. J., Liu, E. Z. F., & Yuan, S. M. (2001). Web-based peer assessment: Feedback for students with various thinking-styles. Journal
of Computer Assisted Learning, 17, 420–432.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.
McPherson, M. (2005). Developing innovation in e-learning: Lessons to be learned. British Journal of Educational Technology, 36,
585–586.
*Metzger, M. J., Flanagin, A. J., & Zwarun, L. (2003). College student Web use, perceptions of information credibility, and verification
behavior. Computers & Education, 41, 271–290.
*Mumtaz, S. (2001). Children’s enjoyment and perception of computer use in the home and the school. Computers & Education, 36,
347–362.
Naeve, A., Lytras, M., Nejdl, W., Balacheff, N., & Hardin, J. (2006). Advances of the semantic web for e-learning: Expanding learning
frontiers. British Journal of Educational Technology, 37, 321–330.
*Oliver, R., & Omari, A. (2001). Student responses to collaborating and learning in a web-based environment. Journal of Computer
Assisted Learning, 17, 34–47.
*Papanikolaou, K. A., Grigoriadou, M., Magoulas, G. D., & Kornilakis, H. (2002). Towards new forms of knowledge communication:
The adaptive dimension of a web-based learning environment. Computers & Education, 39, 333–360.
Pea, R. (2000). The Jossey-Bass reader on technology and learning. New York: Jossey-Bass Publishers.
*Pena-Shaff, J., & Nicholls, C. (2004). Analyzing student interactions and meaning construction in computer bulletin board discussions.
Computers & Education, 42, 243–265.
Richey, R., & Nelson, W. (1996). Development research. In D. H. Jonassen (Ed.), Handbook of research for educational communications
and technology (2nd ed.) (pp. 1213–1245). Mahwah, NJ: Lawrence Erlbaum Association.
Rourke, L., & Szabo, M. (2002). A content analysis of the ‘‘journal of distance education’’ 1986–2001. Journal of Distance Education,
17(1), 63–74.
Ross, S., & Morrison, G. (1996). Experimental research methods. In D. H. Jonassen (Ed.), Handbook of research for educational
communications and technology (pp. 1170–1213). Mahwah, NJ: Lawrence Erlbaum Association.
*Rovai, A. (2001). Building classroom community at a distance: A case study. Educational Technology Research & Development, 49(4),
33–48.
Salomon, G., & Clark, R. (1977). Media research methodology. Review of Educational Research, 47(1), 121–150.
Savery, J. R., & Duffy, T. M. (1996). Problem based learning: An instructional model and its constructivist framework. In Brent G. Wilson
(Ed.), Constructivist learning environments: Case studies in instructional design (pp. 135–148). Educational Technology Publications.
Smith, M. C., Locke, S. G., Boisse, S. J., Gallagher, P. A., Krengel, L. E., Kuczek, J. E., et al. (1998). Productivity of educational
psychologists in educational psychology journals, 1991–1996. Contemporary Educational Psychology, 23, 173–181.
Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21, 79–90.
*Steel, J., & Hudson, A. (2001). Educational technology in learning and teaching: The perceptions and experiences of teaching staff.
Innovations in Education and Training International, 38, 103–111.
Tsai, C.-C. (2001). The interpretation construction design model for teaching science and its applications to Internet-based instruction in
Taiwan. International Journal of Educational Development, 21, 401–415.
Tsai, C.-C. (2004). Beyond cognitive and metacognitive tools: the use of the Internet as an ‘‘epistemological’’ tool for instruction. British
Journal of Educational Technology, 35, 525–536.
*Tsai, C.-C., Lin, S. S. J., & Tsai, M-J. (2001). Developing an internet attitude scale for high school students. Computers & Education, 37,
41–51.
Tsai, C.-C., & Wen, L. M. C. (2005). Research and trends in science education from 1998 to 2002: A content analysis of publication in
selected journals. International Journal of Science Education, 27, 3–14.
Wentling, T. L., Waight, C., Gallaher, J., La Fleur, J., Wang, C., & Kanfer, A. (2000). e-learning-A review of literature. http://
learning.ncsa.uiuc.edu/papers/elearnlit.pdf Retrieved 18.12.06.
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339.
*Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers.
Computers & Education, 42, 289–314.