Higher Education For Medicine, Using VLE

Special Report 6
Review of Virtual Learning

Environments in UK
Medical, Dental and
Veterinary Education
Julian Cook August 2005
ISBN 0 7017 0186 2

Institute for Learning and Research Technology Review of Virtual Learning Environments in UK Medical, Dental and Veterinary Education
Review of Virtual Learning Environments in UK

Medical, Dental and Veterinary Education
Higher Education Academy Subject Centre for Medicine,

Dentistry and Veterinary Medicine: Mini-project Report
1. Document Notes
Author Julian Cook, Institute for Learning and Research Technology, University of Bristol
Date 27 August 2005
Version 2.0
Document Name Higher Education Academy Subject Centre Mini-project Report
2. Summary
This is the report of the Higher Education Academy subject centre for Medicine, Dentistry and Veterinary
Medicine (formerly LTSN-01) Mini-Project ‘Review of Virtual Learning Environments in UK Medical, Dental and
Veterinary Education’. It is an update of the JTAP-623 report carried out in 2001. It presents responses to two
questionnaires (targeted at “academics” and “developers”) circulated in August / September 2004, as well as some
comments collected at a one-day seminar in May 2004.
A total of 35 medical schools responded, along with 9 Dental Schools and 7 Vet Schools. Responses show that
almost half of schools are now using a commercial rather than a bespoke VLE, although bespoke systems are still
felt to be more suitable. Overall usage of VLEs has increased greatly. There has been development of VLEs
towards personalisation and integration with other systems, mainly through moves towards single-sign-on,
although there is still work to do in this respect. A minority of schools are actively involved in sharing content
(mainly questions and interactive materials) and to a lesser extent system code and components. This is mainly
done within formal projects and associations. The teams supporting VLEs vary enormously in terms of their size
and skill sets. Although there has been a broadening in the range of activities carried out with VLEs, their core
function is still delivery of course and administrative information. There is still insufficient data about how VLEs are
affecting teaching and learning practice, although there is a strong belief that they are important in supporting
students on placement, mainly by improving contact with central services.
Julian Cook ISBN 0 7017 0186 2 Page 2

3. Acknowledgments
The author would like to thank the following for their contributions to, and help with, this report.
Academics Developers
Dr David Byrne Ms Sandy Bostock
Dr G Cannavina Dr David Byrne
Prof Jane Dacre Dr John Couperthwaite
Dr David Davies Dr Peter Dangerfield
Dr Reg Dennick Dr Hazel Derbyshire
Prof Susan Dilly Dr Rachel Ellaway
Dr Tim Dornan Dr Neil M Hamilton
Dr Michael Doherty Dr Joan Kemp
Dr Max Field Dr Tony McDonald
Dr Ross Hobson Dr Colin Melville
Ms Sharon Huttly Dr Malcolm Murray
Dr Andrew Jefferies Dr Anthony Peacock
Prof Sam Leinster Dr Andy Pellow
Prof Stephen May Dr Giles Perryer
Prof Jim McKillop Mr Peter Rayment
Dr Jacinta McLoughlin Mr Ash Self
Dr Kieran McGlade Mr Nick Short
Dr Jean McKendree Ms Vivien Sieber
Prof Stewart Petersen Prof Michael Ward
Dr Patricia Reynolds Mr Kim Whittlestone
Prof Trudie Roberts Dr Simon Wilkinson
Dr Susan Rhind Dr Jane Williams
Dr Anita Sengupta
Prof John Simpson Academy Subject Centre
Dr Patsy Stark Dr Megan Quentin-Baxter
Prof David Stirrups
Prof Martin Sullivan ILRT, University of Bristol
Dr Frank Taylor Mr Mike Cameron
Prof Richard Vincent Mr Andy Ramsden
Prof Damien Walmsley Mrs Sue Timmis
Dr Diana Williams
Mr WM Williamson
Mr Jeff Wilson
Thanks also to those who contributed anonymously, and to the reviewers of the text drafts including: Rachel
Ellaway (University of Edinburgh), David Davies (University of Birmingham), Megan Quentin-Baxter (Academy
Subject Centre), and all those who responded to the consultation on the final draft in May 2005.
Thanks to the Higher Education Academy subject centre for Medicine, Dentistry and Veterinary Medicine, and the
Teaching and Learning Technology Programme Facilitated Network Learning in Medicine and Health Sciences
transferability project (project number 86) for financial support of this study.
Page 3
4. Contents
Institute for Learning and Research Technology Review of Virtual Learning
Environments in UK Medical, Dental and Veterinary Education Higher Education
Academy Subject Centre for Medicine, Dentistry and Veterinary Medicine: Mini-project
Report....................................................................................................................................................2
1. Document Notes.........................................................................................................................................................................2
2. Summary ........................................................................................................................................................................................2
3. Acknowledgments .......................................................................................................................................................................3
4. Contents ........................................................................................................................................................................................4
5. Foreword.......................................................................................................................................................................................5
6. Introduction ..................................................................................................................................................................................6
7. Aims ................................................................................................................................................................................................6
8. Methods .........................................................................................................................................................................................6
9. Summary of results .....................................................................................................................................................................7
10. Discussion.................................................................................................................................................................................. 11
11. Recommendations ................................................................................................................................................................... 13
Introduction....................................................................................................................................... 15
Aims .................................................................................................................................................... 16
Methods.............................................................................................................................................. 17
12. Data collection ......................................................................................................................................................................... 17
13. Responses.................................................................................................................................................................................. 18
Results ................................................................................................................................................ 21
14. Case for Medical VLEs............................................................................................................................................................ 21
15. Pressure to standardise.......................................................................................................................................................... 27
16. Changes to VLEs since 2001................................................................................................................................................. 30
17. Growth of VLE usage ............................................................................................................................................................. 34
18. IT and support infrastructure ............................................................................................................................................... 38
19. Sharing of system components and content..................................................................................................................... 42
20. Role of VLEs in supporting student placements .............................................................................................................. 48
21. How VLEs are used................................................................................................................................................................. 50
22. Effect of VLEs on learning and teaching practice ............................................................................................................. 53
Discussion........................................................................................................................................... 54
23. Limitations of this survey....................................................................................................................................................... 60
Recommendations ............................................................................................................................ 62
24. Hosting institutions ................................................................................................................................................................. 62
25. Medical, dental and veterinary schools .............................................................................................................................. 62
26. All developers........................................................................................................................................................................... 62
27. Developers of bespoke systems .......................................................................................................................................... 62
28. Funders / stakeholders ........................................................................................................................................................... 62
Appendices......................................................................................................................................... 64
29. Appendix One: VLE types used by individual schools as main VLE............................................................................ 64
30. Appendix Two: How bespoke VLEs have developed since 2001 ............................................................................... 65
31. Appendix Three: Full system usage data - 2004 .............................................................................................................. 67
32. Appendix Four: Server Descriptions .................................................................................................................................. 68
33. Appendix Five: Interoperability technologies implemented in VLEs .......................................................................... 69
34. Appendix Six: Breakdown of e-learning activities undertaken to any significant extent at UK medical,
dental and veterinary schools............................................................................................................................................... 70
35. Appendix Seven: On-line academics survey ..................................................................................................................... 72
36. Appendix Eight: On-line developers survey ..................................................................................................................... 75
37. Appendix Nine: Glossary....................................................................................................................................................... 82

5. Foreword
Few will doubt that e-learning has now become part of the core business of universities playing an increasingly
important role in the learning and teaching process. In the UK, Medicine, Dentistry and Veterinary Science have
often been at the forefront of e-learning innovation and in learning environment development in particular. Virtual
learning environments (VLEs) provide an overarching context for e-learning materials: the widespread creation of
sophisticated electronic curricula shows that this has been recognised. Continued development of the electronic
curriculum lies at the heart of this innovation and provides an opportunity to strengthen collaboration between
institutions. A whole new generation of learners is benefiting from the integration of e-learning using more
traditional pedagogical techniques. VLEs are a space in which students can engage with course content in a way
that maps onto their own learning style. For Years 1 and 2 of Medical, Dental and Veterinary courses, VLEs are
very popular and provide a means to inspire and enthuse students in the early stages of their degree. While the
picture is more complex for students out on placements, VLEs make access to more resources easier and this
helps to consolidate knowledge which complements the development of their clinical skills. The integration of
placement students into an on-line community may take longer to achieve, despite the rise of discussion boards
and the like, as this report suggests. Further research into this area would make an important contribution to the
future development of VLEs in relation to their use and impact on learners. The report examines the progress
made in the personalisation of systems and recommends that this area should be prioritised. This will further tailor
the electronic curriculum to the needs of individual learners and presents a real opportunity to engage many more
students more effectively. e-learning then becomes a means for delivering a blended learning experience that
integrates a range of materials and formats across the curriculum.
The 2001 predecessor of this review, the JTAP-623 report, came at a time when Medical, Dental and Veterinary
schools were exploring how VLEs could be made to support the learning and teaching process by using new
technologies, particularly the internet and broadband networks. A pattern emerged of investment in localized
infrastructure and technical staff to develop systems and content to support local needs. While innovation has
been driven by a few champions, a much more systematic approach is required, as this report suggests. Such a
change to the delivery of the electronic curriculum will enable universities to meet the demands of increased
student numbers, and capitalise on their familiarity with new technology.
The present report reflects the change in emphasis in institutions from local to central support for VLEs, from
adoption by individual champions to wider staff uptake, and from locally developed bespoke solutions to the
integration of local systems with commercial tools. We live at a time where it is increasingly difficult for any one
school to develop all the systems and content it needs to support the curriculum. The importance of sharing
experiences, ideas and good practice remains high while VLE innovation is still diverse and rapidly evolving. The
VLE community will be strengthened by this and as progress continues, a more integrated system will emerge. The
challenge however, remains for individual schools to maintain high standards and to invest in the technical
infrastructure necessary to meet the changing needs of the curriculum, teachers, and students. I welcome this
report as a much-needed review of current good practice across our sector and hope that it will be used to
inform the exciting challenges that face us in the future.
Professor William Doe

Dean of the Medical School
University of Birmingham
Page 5
6. Introduction
The JTAP-623 report, completed in 2001, was a survey of the use of VLEs in UK medical education 1. It highlighted
some of the key issues relating to the development of VLEs, and acted as a focal point for the sharing of
experiences, ideas and good practice between those who had already been working in this area for some time and
others who were just beginning. This report updates that work by reporting the results of a more recent survey of
medical, dental and veterinary academics, clinicians and curriculum developers. The term medical VLEs used
throughout this report refers to the use of VLEs in medical, dental and veterinary education. A full
glossary is provided in Appendix Nine: Glossary on page 72 below.
7. Aims
The overall aim was to see how much the overall findings of JTAP-623 still applies in 2004 and to see how the
sector has moved on during the intervening 3 years.
8. Methods
Data was collected on the use of VLEs in medical, dental and veterinary education in the UK. The main data
collection methods and the results are outlined below.
8.1. Participation at face-to-face event. Attendance at the Academy event “Twenty-

Twenty Vision” in Manchester on 6 May 2004.
Data was gathered through:

• Informal discussions.
• A short paper-based questionnaire for the participants of a focus group.
• Notes from a themed group discussion.
8.2. On-line questionnaires.
• An “academics” questionnaire – Forty five responses were received during August / September 2004
from 9 dental schools, 23 medical schools, and 7 veterinary schools. Six anonymous responses were received.
• A “developers” questionnaire – Twenty nine replies were received during August / September 2004,
including 25 medical schools, 1 dental school and 3 veterinary schools.
1 http://www.ltss.bris.ac.uk/interact/23/in23p14.html

9. Summary of results
9.1. Although more schools are now using commercial VLEs (rather than bespoke
systems) than in 2001, mainly because of institutional pressures, there is greater
satisfaction among developers of bespoke systems.
Around half of the responding schools are using a commercial VLE, while the remainder use a bespoke system or
open source system.
Main VLE - Source: Developers Survey (n=28).

The sector is now much more mixed in terms of VLE type used, with 12 respondents using a commercial system,
11 using their own bespoke systems, 2 using a system modified from another institution, and 2 using the Bodington
open source system.
2nd VLE - Source: Developers Survey (n=28).

A few schools are using a second VLE, in some cases where the main institutional VLE supplements the bespoke
system used in the school, and in others bespoke components are retained to supplement the institution’s system
where this has supplanted the school’s original bespoke VLE.
Commercial systems in use - Source: Developers Survey (n=12).

The main commercial system in use is Blackboard, used by 9 responding schools, with another 2 using WebCT,
and 1 using FD Learning’s Learning Environment (now Tribal Technology).
How has the situation changed since 2001? - Source: Developers Survey (n=28).
For 14 of the respondents there had been no change. A quarter (7) had adopted a VLE since 2001 (of these 3
were bespoke or open source systems, and 4 were commercial. Three have changed from using 1 VLE to using 2.
Are the VLEs sufficient in themselves or do they need supplementing with other applications. Source: Developers Survey
(n=25).
The main supplementary applications being used with VLEs are a dedicated assessment system (mostly
QuestionMark Perception), an Optical Mark Reader and an e-portfolio, all used by just under a half of respondents.
There were no statistically significant difference between the bespoke and commercial users.
Users of bespoke systems clearly believe that only these can represent the curriculum adequately, while users of
commercial systems are more ambivalent.
Reasons for choice of VLE - Source: Developers Survey (n=14).

The main reasons selected by users of bespoke systems for not using a commercial system were a) that their own
systems remain the best way of representing the curriculum and b) because of the deficiencies of commercial
systems.
Source: Developers Survey (n=12).

The main reasons selected for their choice by those using commercial systems were: pressure from the hosting
institution; that they were not given a choice in the decision; had insufficient resources to develop their own
system; or that the commercial system met their needs.
Suitability of commercial systems - Source: Developers Survey (n=26).

All but 2 of the bespoke users agreed that the complexity of ‘medical’ curricula made commercial VLEs unsuitable,
whereas half of the commercial users disagreed. However 5 of the 11 commercial users agreed, suggesting some
dissatisfaction with their commercial systems.
Page 7
Satisfaction with VLE’s reflection of curriculum - Source: Developers Survey (n=26).

Bespoke users expressed more satisfaction than commercial users with their own systems’ ability to represent the
curriculum. Only 2 bespoke users were less than “mostly satisfied”, and both represent schools using an “off-the-
shelf” open source system rather than a truly bespoke VLE.
Supplementary tools in commercial systems for managing the curriculum - Source: Developers Survey (n=8).
These include applications to support specific activities (e.g. Special Study unit and placement allocations); linkage
to non-VLE hosted materials, timetable, a bespoke CMS and various VLE-controlled bespoke tools.
9.2. Almost all hosting institutions are using commercial VLEs, but only a minority of
surveyed schools were experiencing pressure to fall into line.
Relationship between school and institutional VLEs - Source: Developers Survey (n=14).
For almost all bespoke users, the hosting institution is using a commercial VLE either exclusively or in conjunction
with a bespoke system. The exceptions are 2 institutions that are using the same open-source VLE as their medical
school.
Institutional attitudes to bespoke medical VLEs - Source: Developers Survey (n=12).

Most bespoke users said their institutions were happy for the two systems to coexist, but 5 schools said that
pressure to adopt the hosting institution’s VLE was either already present or was likely in the future. There are
also moves to make the school and institutional systems more interoperable.
Likely future relationship - Source: Developers Survey (n=12).

Nine bespoke users said the two systems would continue to co-exist – though one or two were unsure, and some
again mentioned a trend towards greater interoperability between systems rather than mutual exclusion.
9.3. There has been progress towards personalisation of systems and data integration,
particularly various forms of single-sign-on.
There has been considerable progress towards individualised user portals, and implementation of survey tools has
also increased.
The 2004 survey asked the bespoke developers how far the intended developments mentioned in the 2001 survey
had actually taken place.
Personalisation features - Source: Developers Survey (n=14).

There is quite a mixed picture, with considerable progress towards user specific portals into the VLE, but other
features, particularly intelligent recommendation of resources and user specified pathways, have not been widely
adopted.
Other features - Source: Developers Survey (n=14).

Author uploading tools, assessment tools and 2-way communication were already established features of most
systems, and survey tools are a newer but very popular feature, while the use of MeSH and RSS feeds is neither
widely adopted nor planned.
Most schools now have some level of integration between the VLE and other systems - Source: Developers Survey (n=24).
At least 24 out of 28 schools have or are planning some level of integration. The most common is a single-sign-on
where the same user details have to be entered separately to login to each system, while a smaller number have
more sophisticated integration such as “true single-sign-on” (where authentication information is passed between
systems) or linking to student record systems. The 11 “other” responses emphasised that fuller integration is on
its way even if it is not present yet in most cases.

9.4. Use of VLEs has increased “significantly” across the sector.
Most schools now use a VLE - Source: Academics Survey (n=38).

Use of a VLE is underway to some extent in 31 of the 38 schools who responded to the relevant question in the
Academics Survey.
In the majority of schools, the VLE is now used by most or all staff and students - Source: Academics Survey (n=38).
Twenty four out of the 38 believe that the VLE is used widely across the school, either by many but not yet all, or
by all as a matter of course.
Broken down into subject disciplines, the greatest level of embeddness appears to be in Medicine, where 9 schools
(almost half of the 19 that responded) see their VLEs used as a matter of course by staff and students. This
represents a step onwards from 2001, where the impression was that in most school VLE uptake was more
sporadic and patchy.
9.5. Levels of use by students and staff have increased “significantly”.
For 20 out of the 23 schools responding, usage has increased either “significantly” or “massively”2. No respondents
said that usage had remained stable or decreased.
Usage statistics - Source: Developers Survey (n=10).

Only 10 respondents were able to give detailed usage figures, and such figures are notoriously unreliable. However
figures given by Edinburgh, Nottingham and Cambridge indicate at least a ten-fold increase in use since 2001.
Years using most heavily (e.g. 1st, 2nd etc.) - Source: Developers Survey (n=19).
The overall picture suggests that usage is heaviest in the first 2 years of the curriculum, although there are schools
where use is heaviest towards the end of the curriculum.
9.6. VLEs are hosted on a mix of Microsoft and Unix platforms; support teams are
mainly small and of mixed composition.
The results presented below represent survey questions that were not directly or systematically addressed in the
2001 survey. It is not possible to link them directly to growth in VLE usage. The results presented simply indicate
the level of IT and support infrastructure in place at the time of the survey.
IT infrastructure
Server descriptions - Source: Developers Survey (n=21).

a) Nine of the schools use “Microsoft” based set-ups mostly using a combination of Win 2000 / 2003, IIS, SQL
Server, and ASP.
b) Six use what could broadly described as “Unix” set-ups, including Linux, OSX, and Solaris with Apache, My
SQL, Zope or PHP.
Few of the responses gave much indication of the hardware power they use to run their VLE, although 8 of the
respondents mentioned that they run their VLE on multiple machines.
Support teams - Source: Developers Survey (n= 22).

Many schools have less than 1 FTE in each support role, except for the roles of Server / site maintenance, VLE
administration, educational development and e-learning development where at least half of the schools have more
than one FTE 3.
A small number of institutions (c. 6) have large development teams (up to 20 people), where another 11 have
around 6 or 7 staff, and the remaining 4 have fewer than this.
2 Acknowledging of course that these terms are very imprecise.

3 However it must be acknowledged that these roles are not very clearly defined and that there may considerable overlap
between them.
Page 9
As a whole most of the support staff are located within the school rather than provided by the institution
centrally, although less so for the roles of server maintenance, VLE administration and user training. For bespoke
users support is much more likely to be provided locally, and in particular all their web development staff are
provided within the schools.
9.7. Although sharing of code and content is only a priority for a few schools, most
schools have shared at least some content, mainly interactive materials and
questions.
The 2001 report recommended that further efforts be made to facilitate the sharing of both VLE components and
content across the sector.
The case for a single medical VLE - Source: Developers Survey (n=14).
There is broad agreement that there is not a justification for a single standard VLE specifically for ‘medicine’
because of variations in curricula between schools.
Interoperability technologies - Source: Developers Survey (n=21).

By far the most common interoperability technology is XML, implemented by a small majority of schools, with
Learning Object metadata and the IMS QTI specification also implemented by a sizeable minority.
Sharing system code and components - Source: Developers Survey (n=23).

Sharing system code and components is only a high priority for 5 out of the schools represented, and for 11 it is
low priority or not on their agenda at all. Eight had shared code either a great deal or somewhat, showing that
some schools have done this even when it is not a high priority.
This is done using a variety of approaches: for example Blackboard users within the sector are beginning to
collaborate (by sharing Blackboard Building Blocks) and there are number of other collaborative projects working
towards code / component sharing.
Sharing content - Source: Developers Survey (n=24).

The level of priority for making content readable by systems at other institutions is about the same as it is for
sharing system code and components and the same schools are enthusiasts for both activities, with one or two
exceptions.
Results suggest rather more actual sharing of content than of system code, with 16 schools having shared some
kind of content somewhat or a great deal. Interactive materials seem to have been shared rather more and
reusable learning objects (RLOs) rather less, despite the relatively large number of schools implementing a
standard for this.
9.8. Although delivery of VLEs to hospital sites is somewhat constrained by IT

infrastructure limitations, VLEs have been successful in facilitating support for and
communication with staff and students at remote sites, and for delivering learning
activities.
Effective uses of VLEs to support students on placements – Source: Paper-based questionnaire (n=7).
Responses included: providing equity across placements; creating a community of learning; allowing rapid
communication; stimulating student / tutor dialogue; improving contact with tutors; facilitating curriculum
coherence; facilitating activities such as e-CPD, formative Computer Assisted Assessment (CAA), video lectures
synchronised with slides.
Limitations of VLE use at remote sites: the effect of hospital IT infrastructure on VLE use - Source: Developers Survey
(n=23).
A considerable majority (16) agree that their user group's ability to take full advantage of their VLE is constrained
by IT infrastructure in hospitals where their students and teachers are located.
Only a minority (4) provide and support IT infrastructure for teaching areas in hospitals, while the majority either
simply make recommendations or have little or no involvement.

9.9. While the single most common use of VLEs remains to deliver programme
information and course documentation, many schools are also using them as a
platform for on-line learning activities – despite some doubts about the efficacy of
these.
The JTAP-623 2001 report found that most schools were using VLEs heavily to i) deliver information about the
programme and less as ii) a platform for on-line learning.
Relative importance of information delivery vs. on-line learning – Source: Paper-based questionnaire (n=7).
Overall respondents see i) as less demanding and consequently it has in many cases been emphasised, but that
there is interest in and in some cases a recognition of the potential for ii) but also some concerns about the cost
and validity of such materials and activities.
How e-learning is used - Source: Academics Survey (n=44).

The single most common use of e-learning selected (mainly but not exclusively delivered via a VLE) is for on-line
management of course literature such as programme / module / study guides, with 41 respondents choosing this
activity. Thus the heaviest use of VLEs is still as a means of delivering information about the programme. However
many schools are engaged in on-line learning activities, the most common of which are formative assessment and
asynchronous discussion.
The continued emphasis on i) is reprised in the response from the Academics who were asked to choose and
prioritise the four main drivers for using a VLE in their school; the two highest scoring drivers were both related
to curriculum management – a result very similar to the 2001 survey.
9.10. Little data is available yet on the impact of VLEs upon learning, teaching and
assessment practices, however there is some evidence that it can release staff time
and facilitate a review of practice.
Source: Paper-based questionnaire (n=7).

Respondents to the paper-based questionnaire describe the effect their VMLE has started to have upon learning,
teaching and assessment practices in their institution. Answers show a mixed picture of schools where use of VLEs
has already had a significant impact (releasing staff time, facilitating review of practice) and where it is too early to
say.
10. Discussion
10.1. Which of the planned developments have actually been implemented and how
successful have they been?
Developing beyond what we understood them to be 3 years ago, VLEs seem to be evolving into aggregations of a
wider set of e-learning tools, for example e-portfolios and assessment tools. Applications such as e-portfolios lead
medical VLEs towards becoming truly personalised. However a limit on this has been placed by the difficulties of
seamless integration between systems. The gradual implementation of true single-sign-on may represent a first step
towards this at least from the user’s perspective, and ultimately towards replacing the single jack-of-all-trades VLE
with more modular suites of integrated applications. However the modular approach risks losing the cohesion and
sense of orientation experienced by users of a well-designed unified on-line environment.
10.2. How much collaboration and sharing of system components and/or content has
there been between institutions developing medical VLEs?
Medical schools seem unlikely to collaborate to the extent of creating a single VLE or even a smaller number of
VLEs for UK medical education. There has been some sharing of system code and components, and of content.
There has been rather more of the latter, perhaps in part due to the high value of good quality content and the
improved interoperability between systems which means that content can be more easily shared across a range of
platforms thus reducing the imperative for sharing of code.
Page 11
Most sharing has been done between groups of schools brought together through formal collaborative projects,
membership of specific national and international associations or through using the same commercial VLE. There
seems to be some inconsistency between reports of content sharing and implementation of standards and
specifications required to enable this, suggesting standards implementation is motivated by local rather than inter-
institutional considerations.
10.3. Given the rapid evolution of commercial VLEs, is there still an overriding case for
the continued use and development of bespoke VLEs?
The answer to this is more mixed than it was in 2001. Many (but not all) schools using commercial VLEs are
satisfied with them, but there is greater satisfaction among bespoke developers. Open source VLEs present a new
rival to bespoke systems as they appear both economical and amenable to local adaptation. However, the
suitability of both commercial and institution-wide open source systems may depend as much on how flexibly they
are implemented institutionally as on their inherent flexibility. With greater integration of systems it will become
easier to assemble VLEs from a range of tools seamlessly joined together. The case for continued dedicated
medical VLEs will depend on their offering unique facilities.
10.4. As institutions have begun to adopt VLEs centrally, has there been any pressure on
schools to fall into line with institution-wide VLE strategies, and how have schools
responded to this?
This has only occurred in a minority of cases and most bespoke developers are optimistic about their future
prospects. However it is worth considering that pressure may come from within schools as well as from the
centre as the facilities and potential benefits of centralised VLEs become more widely known about. Bespoke VLE
developers can add value to their own systems through medicine specific tools and features such as assessment
tools, e-portfolios, case-logging tools.
10.5. How has usage of VLEs by staff and students grown and developed? How much
demand does this place on IT infrastructure?
The growth of VLE use both numerically and in terms of embeddedness indicates that VLEs have been a success
and have an important place in medical education.
The staff teams who support this increased usage vary widely both in terms of size, mix of skills and how they are
distributed. This raises the question of what is really necessary, and which model works best, requiring further
research before a recommendation could made. There seems to be some correspondence between the
composition of the support team according to the type of VLE (bespoke or commercial) and whether the medical
school is new or more established.
10.6. In what ways are VLEs most successful in supporting students and teachers
located at disparate clinical and educational sites?
The limited control exercised by most schools over the IT infrastructure at the sites where many of their users
are located restricts to some extent what they are able to deliver via the VLE. One possible solution is the use of
thin client technology as adopted by the University of Bristol, although this has limitations in its support for
delivering video.
The main benefit of the VLE for students on placement and their teachers was said to be that it improved contact
between remote sites and the central school – but interpretations vary as to whether this means one-way delivery
of information or a multi-way conversation between learners and their teachers. There are suggestions of some
movement towards the latter, although some respondents have reservations.
10.7. Has there been any shift towards using VLEs as a platform for on-line learning
rather than as a means of delivering more traditional documentation?
Is such a shift actually desirable? The key role of VLEs in medicine remains the delivery of traditional course
documentation. While some respondents embrace e-learning activities with enthusiasm – others doubt their value
and are mindful of the cost and effort involved. Arguably while on campus students learn via face-to-face contact
and on-line activities constitute a supplement to this rather than core learning. For students on placement there

may be a greater case for providing on-line learning activities if the intention is to teach them underpinning
knowledge during this time in addition to clinical skills.
10.8. Have the VLEs started to have any effect on learning, teaching and assessment
practices?
Little evidence was collected during this survey but this question is believed to constitute an important area for
research. Since VLEs are now becoming firmly embedded into the curriculum as well as representing significant
ongoing investment by both developers and teaching staff it seems vital to understand their influence upon the
kinds of doctor, dentist or vet that are graduating from these institutions.
11. Recommendations
11.1. Hosting institutions
Where an institution’s medical / dental / veterinary school is using the main institutional VLE, control of the VLE
structure should be devolved as far as possible to the schools to allow schools to modify the VLE for their needs.
11.2. Medical schools
Schools (or hosting institutions) should audit their local support teams and evaluate their suitability for
implementing their e-learning / VLE strategy. Results should be published in order for the community as a whole to
develop models of effective support.
Schools relying on single individuals to develop / support their VLE should consider an expansion of their support
teams and acquiring staff with relevant specialist skills.
Schools are able to share developments and some sharing and exchanging good practice has taken place,
particularly when collaboration is facilitated by external funding such as FDTL or JISC. Sharing can be as effective
between same-course in different institutions, or different-courses in the same institution.
Schools considering their future VLE strategy should keep an eye on developments towards integration and
interoperability and consider the potential benefits of a hybrid solution rather than a single all-encompassing VLE.
11.3. All developers
Developers (particularly those working with or considering hybrid solutions) should prioritise the implementation
of true single-sign-on where this is available locally.
Continue to work to allow modular integration between applications rather than development of all-encompassing
VLEs.
Systems should be developed so that they are linkable at quite a deep level and integrated so that the user is
unaware that they are using a different system.
Developers should analyse the tool integration models that are effective and be willing to share these with the
development community.
11.4. Developers of bespoke systems
Continue to add value to medical VLEs through development of health-education specific applications and tools.
Work towards further development of personalised learning environments.
Demonstrate in detail how bespoke VLEs represent the curriculum better than a commercial or open source VLE
could.
Page 13
11.5. Funders / stakeholders
Evaluation work should be funded into:

• How successful sharing of content has been in terms of how shared content is used and which kind of content
is most popular.
• More specifically, how are RLOs used by different institutions. To what extent are RLOs used across
institutions? How far can they be designed to facilitate local modification?
• Furthermore feasibility studies should be carried out into sharing of learning processes.
Further research should be funded into:
• How communication tools have helped to support students on placement and engender an on-line community.
This would best be done as a set of case studies.
• Give further consideration to the type of RLOs that could be shared.
• Developing models of VLE / e-learning support in medical schools and their hosting institutions, and evaluating
the effectiveness of these.
Further projects should be funded to support and encourage sharing of content.
Finally and most importantly, research is urgently needed into the true impact of VLEs on the practice of health
education and on the experience of students and staff.

Introduction
The early adoption of VLEs by medical schools was a response to a) curriculum changes which had resulted in the
blurring of the preclinical / clinical divide and b) to a need to manage larger numbers of students who now spend
an increased proportion of their study time based away from the host institution engaged in clinical experiential
learning at clinical sites. They were developed to facilitate communication between sites, between students and
teachers, and to find ways to represent and explain an ever more complex curriculum; and in response to quality
assurance requirements particularly as measured in the England / NI 1998-2000 round of QAA visitations.
The JISC JTAP-623 report1, completed in 2001, was a survey of the use of VLEs in UK medical education. It
highlighted some of the key issues relating to the development of VLEs, and acted as a focal point for the sharing of
experiences, ideas and good practice between those who had already been working in this area for some time and
others who were just beginning.
JTAP-623 found that:

• Seventeen out of 21 UK medical schools surveyed were using a VLE, but of these 15 had either developed their
own or adopted / adapted one developed at another UK medical school, rather than commercial VLEs.
• This was mainly because a) it was felt that the latter were unable to represent the complexity of the medical
curriculum, but also b) because commercial systems were in their infancy when development of many of the
medical VLEs began.
• The result was some 12 largely separate developments, representing a considerable replication of effort.
• The main use of VLEs was as a way of providing information about the curriculum and for providing supporting
information rather than to facilitate on-line learning.
• It was too early to get a clear sense of how the VLEs were affecting teaching and learning practices across the
sector.
• All of the developers interviewed intended to continue development of their system, in particular to add
features such as greater personalisation and delivery to mobile devices.
Page 15
Aims
As this is a new and very fast developing field, by late 2003 it was considered that JTAP-623 was in need of
updating. The overall aim of the project was to see how much the overall findings of JTAP-623 still applied and to
see how the sector had moved on during this time. In the meantime several new medical schools had started up,
and it was considered necessary to include them.
The project initially set out to address the following:
1. Given the rapid evolution of commercial VLEs, is there still an overriding case for the continued use and
development of bespoke medical VLEs?
2. As institutions have begun to adopt VLEs centrally, has there been any pressure on medical, dental and
veterinary schools to fall into line with institution-wide VLE strategies, and how have schools responded?
3. Which of the planned developments have actually been implemented and how successful have they been?
4. How has usage of VLEs by staff and students grown and developed? How much demand does this place on
IT infrastructure?
5. How much collaboration and sharing of system components and/or content has there been between
institutions developing VLEs in clinical subjects?
6. In what ways are VLEs most successful in supporting students and teachers located at disparate clinical and
educational sites?
7. Has there been any shift towards using VLEs as a platform for on-line learning rather than as a means of
delivering more traditional documentation?
8. Have the VLEs started to have any effect on learning, teaching and assessment practices?
It was decided that questions 6, 7 and 8 would be difficult to answer reliably and may therefore be beyond the
scope of this study. However ultimately it was decided that 1-5 would be addressed by on-line questionnaire, and
an attempt to get at least some data on questions 6, 7 and 8 would be made through attendance at a Subject
Centre event at which many of their nominated primary contacts (NPCs) would be present.

Methods
12. Data collection

The main data collection methods were participation at face-to-face events and on-line questionnaires.
12.1. Participation at face-to-face events
Attendance at the Subject Centre event “Twenty-Twenty Vision” in Manchester on 6 May 2004 enabled some data
to be gathered addressing questions 5, 6,and 7 (see Aims on page 16 above), as well as individual opinions and
experiences which informed the development of a wider survey (see 12.2 On-line questionnaires below). Although
the theme of the meeting was external clinical placements, there was a significant sub-theme of e-learning
(including a themed breakout group titled - e-learning: How can e-learning support learning on external clinical
placements?). This provided three sets of data:
• Notes from participation in formal and informal discussions and pick up how people are thinking.
• A short open-ended paper-based questionnaire addressing questions 5, 6 and 7 circulated to all participants in
the themed breakout group.
• Detailed notes of the discussion during the themed group taken by a member of the Subject Centre team.
The main use of the first and third items above was to inform the development of the on-line questionnaires.
12.2. On-line questionnaires4
Two on-line questionnaires were created and circulated during August and September 2004 to 2 separate lists of
invited respondents. These were:
• An “academics” questionnaire focussing on the strategic implementation of a VMLE, and the context for its use
was circulated to the nominated primary contacts of the Subject Centre.
• A “developers” questionnaire focussing on the technical aspects of the VLEs. This was circulated to a list of
known VLE developers compiled by the Subject Centre.
While the Academics Survey received responses from 45 of the dental, medical and veterinary schools in the UK
and Ireland, the Developers Survey only received responses from 28 of them. The respondents were from a
limited, invited list and time was not available during this project to extend the list to cover the whole potential
community, or to personally chase up those who did not respond5.
The on-line surveys were created and made available using the SurveyMonkey survey tool (www.surveymonkey.com).
The look and feel of the questionnaire was customised to reflect the look and feel of the Subject Centre website.
A copy of the Developers Survey can be viewed at: http://www.surveymonkey.com/s.asp?u=90194782348 and the
Academics Survey at: http://www.surveymonkey.com/s.asp?u=79224782346
The developers questionnaire was structured so that the questions presented to respondents would depend on
the type of VLE they were using (according to their answer to the first question). Thus respondents who said they
were using a bespoke VLE would see questions relating to bespoke developments and to the merits of bespoke
systems, while those using a commercial VLE did not see questions about system developments but were asked
about the merits of commercial VLEs.
An email was sent to the invitees from the Subject Centre commending the survey to them. Thereafter reminders
were sent out periodically. A lead time of 7 weeks was given. The reminders were sent automatically via the
SurveyMonkey system which enabled reminders to be sent only to those who had not yet responded.
4 Throughout this report the intention is to present all evidence addressing a particular question together under its relevant
heading. The report is organised according to the question being addressed rather than the data source. Hence in places
responses from the different sources are presented side-by-side. Although the report moves back and forth between data
from different sources, the sources are clearly labelled throughout. It is hoped that this is the most illuminating way of giving
the reader an overview of the evidence under each question.
5 See 23 Limitations of this survey on page 60 below for further methodological reflections and caveats.
Page 17
SurveyMonkey allows personalised emails to be sent, including a link to the survey which allows the system to
track who has responded. Where respondents accessed the survey using the URL directly, rather than by
responding to the email, this information was not available.
12.3. Methodology notes
Confidentiality note
In order to avoid sending repeated reminders to people who had already responded, the electronic survey system
tracked who had responded. This also allows the survey authors to map individual responses to specific
institutions. The questionnaire also included an optional question asking for the respondent’s identity. There were
a number of responses where this information was not filled in by the respondent but where the identify of the
respondent was known by the system. In these cases responses will not be identified, either by name or by
institution. The information was only used to indicate which institutions responded to the survey and to identify
quotes.
Terminology notes
Throughout this report, as in the questionnaire both 'VLE' and 'VMLE' are used which are intended to be read
interchangeably for the purpose of this report.
This report represents a survey of departments / schools / colleges involved in undergraduate medical, dental and
veterinary education. It contains instances where the situation in the “school” is contrasted with that in the
“hosting institution”. However a number of the participating “schools” are institutions in their own right. However
for the purpose of this report, the term “school” is used to refer to all participating schools, departments or
whole institutions, and “institution” is used to represent the hosting University or College.
At various points in this report, “medical” or “medicine” is used to refer collectively to medicine, dentistry and
veterinary science / medicine. It is hoped that readers will understand that this is for the sake of brevity.
13. Responses
13.1. Paper-based questionnaire
Seven responses were received, representing the following schools:

• Peninsula Medical School (PMS).
• University of Manchester.
• The Royal Veterinary College, London (RVC).
• Dental Institute, GKT School of Medicine.
• University of East Anglia (UEA).
• and a further 2 who responded anonymously.

13.2. Academics survey
Responses were received from 45 medical, dental and veterinary schools. The following institutions were
represented:
Medical Schools Dental Schools

University of Aberdeen University of Birmingham (Birmingham Dental School)
University of Bath University of Dublin
University of Birmingham University of Dundee
Brighton & Sussex Medical School (BSMS) Eastman Dental Institute
University of Dundee Dental Institute at Guy's, King's College and St
Thomas' Hospitals, King's College, London (GKT
Dental Institute)
University of Durham University of Leeds
University of East Anglia Medical School (UEA) University of Newcastle
University of Glasgow University of Sheffield
Hull York Medical School (HYMS) University of Wales College of Medicine (now Cardiff
University)
School of Hygiene and Tropical Medicine, University
of London (LSHTM)
Imperial College School of Medicine, London
Veterinary Schools / Departments
(Imperial)
Keele University University of Bristol
Guy’s, King’s College and St Thomas’ Hospitals, the University of Cambridge
King’s College School of Medicine, London (GKT
School of Medicine)
University of Leeds University of Dublin
Leicester Warwick Medical School (LWMS) University of Edinburgh
University of Manchester University of Glasgow
6
University of Nottingham University of Liverpool
Queen Mary’s School of Medicine and Dentistry The Royal Veterinary College, London (RVC)
(QMUL)
Queen's University Belfast (QUB)
University of Sheffield
University of Southampton
University of St Andrews
University College London (UCL)
A further 6 institutions posted responses anonymously.
6 Note that this report relates to the undergraduate programme in Nottingham and does not include reference to the
Graduate Entry Programme (Nottingham / Derby).
Page 19
13.3. Developers Survey
Responses were received from 29 medical, dental and veterinary schools. The following institutions are
represented:
Medical Schools Medical Schools (continued)

University of Aberdeen University of Nottingham 7
University of Birmingham University of Oxford
University of Bristol Peninsula Medical School (PMS)
University of Cambridge Royal Free and University College Medical School,
London (UCL)
University of Durham University of Sheffield
University of East Anglia (UEA) University of Southampton
8
University of Edinburgh St George's University of London (SGUL)
University of Glasgow University of Wales College of Medicine (now Cardiff
University)
Guy’s, King’s College and St Thomas’ Hospitals, the
King’s College School of Medicine, London (GKT
School of Medicine)
Hull York Medical School (HYMS)
Dental Schools
Keele University University of Birmingham
University of Leeds
Leicester Warwick Medical School (LWMS)
Veterinary Schools / Departments
University of Liverpool University of Bristol
University of Manchester University of Edinburgh8
University of Newcastle The Royal Veterinary College, London (RVC)
A further 1 institution responded anonymously
7 These results do not represent the University of Derby.

8 The developer’s response from the University of Edinburgh should be read as also covering Edinburgh Veterinary School.

Results
Results will be organised according to the 8 key questions identified in the project proposal (see Aim on page 6
above).
14. Case for Medical VLEs
Given the rapid evolution of commercial VLEs, is there still an overriding case for the continued use and
development of bespoke medical VLEs?
14.1. What type of VLE are schools using now?
Main VLE
Respondents were asked to indicate which of the following is being used as the main VLE at their school:
• Commercial VLE (e.g. Blackboard).
• Bespoke VLE developed at this institution.
• Bespoke VLE developed elsewhere but modified at this institution.
• Bespoke VLE developed elsewhere with no significant local modifications.
• None.
• Other.
Figure 1. Source: Developers Survey (n=28).

As Figure 1 shows, the sector is much more mixed in terms of VLE type used, with just under half using a
commercial system and the remainder using either their own bespoke systems, a system modified from another
institution, or the Bodington open source system (Leeds and Oxford). See Appendix One: VLE types used by
individual schools as main VLE on page 64 below for a full list of schools and the VLE they use.
Page 21
Using more than one VLE

Figure 1 shows that a few respondents said their schools are using a second VLE. Results indicate that where the
first VLE is a bespoke system (and the second VLE is a commercial system), the commercial product that has been
adopted by the hosting institution and is being used to supplement the bespoke system used in the school, possibly
for data exchange purposes, to provide specific functionality (such as CAA), or for specific courses such as
postgraduate programmes (e.g. Newcastle, Aberdeen, GKT School of Medicine). Where the first VLE is commercial
(and the second VLE is listed as a bespoke system), the bespoke developments were described as having been
supplanted by another (commercial) system adopted by the hosting institution, and bespoke components are
retained to supplement what is offered by the institution’s system (e.g. Birmingham Medical School). In both cases
there is some evidence of both VLEs sharing or using common data sources.
Commercial systems in use
VLE No. of schools

Blackboard 9
WebCT 2
FD Learning's LE 1
How has the situation changed since 2001?

Respondents were asked to choose from the following to indicate how their current situation compares to the
situation in 2001.
• We weren't using a VMLE in 2001.
• No change.
• Using a different VMLE.
• We were only using one VMLE in 2001 and now we're using more than one.
• Don't know / I wasn't here then.
Change in type of VLE used 2001 - 2004
Don't know, 3
New since
More than one 2001, 7
VLE now, 3
Different VMLE,
1
No change, 14

For half the respondents there had been no change. A quarter had adopted a VLE since 2001 (of these 3 were
bespoke or open source systems, and four were commercial). See below for a summary of which institutions have
changed or adopted new VLEs since 2001.

New bespoke: Oxford, Glasgow, Manchester
New commercial: LWMS, PMS, RVC, UCL
1 VLE to 2 VLEs: GKT School of Medicine, Birmingham Medical School, Liverpool
Are the VLEs sufficient in themselves or do they need supplementing with other
applications?
The purpose of this question was to get a sense of how satisfied people were with the facilities offered by their
VLE and how far they needed to supplement it with additional applications.
Respondents were asked which of the following they are using to supplement their VLE where they are NOT an
integral part of the basic VLE:
• Content management system.
• e-portfolio / personal academic record.
• Specialist assessment tool (e.g. QuestionMark Perception).
• Courseware authoring tools.
• On-line student survey tool for student feedback.
• Discussion board / chat facilities.
• Optical Mark Reader.
• Other.

Clearly, while all the other tools are being used to some extent, Optical Mark Readers and Assessment tools are
the most widespread supplement, both being used by a majority of respondents, suggesting that although
assessment may not be integrated into the VLE it is nevertheless an activity that is increasingly reliant on
technology.
All of the Blackboard users supplement it with a CAA system, mostly QuestionMark Perception. In general,
commercial users supplement their system more than bespoke users - with the exception of Optical Mark
Readers.
Page 23
14.2. Are bespoke systems still the best way of representing the curriculum or can
commercial systems do this job adequately?
Reasons for choice of VLE

Users of both bespoke and commercial systems were asked to select the reasons for their choice of VLE. The
results are given in Figure 4 and Figure 5.

Users of bespoke (including open source) systems were asked to choose from:
• Legacy.
• Still best way of reflecting curriculum.
• Cost.
• Deficiencies of commercial systems.
• Other.
Other reasons given were:
• Adaptability.
• Extensibility.
• Retains local knowledge.
• Meets local needs.
The reason given in the case of one school using Bodington was that it was used corporately by the hosting
institution.


Users of commercial systems were asked to choose from:
• Cost of software.
• Cost of staff.
• Institutional pressure.
• Commercial system met all our needs.
• Development skills not available.
• Other.
Other responses:
• We didn't choose to buy anything (or haven't yet). We use Blackboard as it is the one supported institutionally.
It's not ideal and we often fit the learning around the product but we have no alternative at the moment. It's
likely that we will continue to use Blackboard as a means to manage and deliver but will increasingly use other
tools for specific things e.g. CAA, content management, etc. (Bristol).
• Flexibility of system. Allows for complexity of curriculum (unlike Blackboard, WebCT). Continuity - allowed us
to continue using QuestionMark and WebBoard - seamlessly integrate with FD Learning LE (Leicester).
• Commercial support, not reliant on developers in house. Ease of academic use (PMS).
• Decision taken at senior management level and we had no say as such (Liverpool).
• Ease of use - after evaluation of products on a TLTP project with Newcastle, we wanted something that staff
would feel confident using, so that they could take ownership of the system. Too small to go it alone, so used
an existing VLE (Durham).
Suitability of commercial systems

The survey asked developers to respond to the statement: ‘Commercial “off the shelf” VLEs are unsuitable for
medical schools because they are unable to represent the complexity of the medical curriculum’.
Page 25
Figure 6. Source: Developers Survey (n=26; 14 Bespoke users, 12 Commercial).

Figure 6 shows a clear (although given the small numbers, not statistically significant) difference between the view
of bespoke and commercial users, with none of the bespoke users disagreeing and half of the commercial users
disagreeing with the question. However nearly half (5 out of 12) of the commercial users agree that commercial
systems are unsuitable for medicine, which suggests some dissatisfaction among users of commercial systems.
Satisfaction with VLE’s reflection of curriculum

The question was then put another way, with respondents asked to indicate how satisfied they are with the way
that their VLE represents the structure and content of their curriculum.
Figure 7. Source: Developers Survey (n=26; 14 Bespoke users, 12 Commercial).

Figure 7 shows that bespoke users are clearly more satisfied than commercial users in this respect (although again
the small numbers mean that the difference is not statistically significant). Furthermore both of the 2 bespoke
users who were less than mostly satisfied represent schools using an “off-the-shelf” open source system rather
than a truly bespoke VLE.

Users of commercial systems were asked in a free text question to indicate what other tools, if any, they use to
help manage the complexity of the medical curriculum. Eight of 12 commercial users posted replies to this
question.
Responses included:
• We are still having the debate as to whether we need any system to manage (or present) the complexity of the
medical curriculum in terms of a VLE for delivery of e-learning. Students study one unit at a time and they
know where to get information on that unit and any e-learning materials. I'm not entirely convinced that we
need a VLE to represent the complexity - this is done adequately well via the course handbook and is then
broken down for each unit. I think those who manage the learning think they need someway to represent this
electronically, but if you asked the students I suspect they don't really care as long as they can access what they
need for what they are studying at the time (i.e. a URL). I think what is important for us is for students (and
staff) to be able to move seamlessly between the various tools and systems e.g. single-sign-on between
Blackboard and our own Intranet (Medici) is really important. Where the complexity requires tools is in the
area of student administration and in particular of placements and assessment. However, it's not the
technology here that is the issue but the underlying processes and guidelines and standards that have to be
agreed and set by consultants / academics i.e. the human / process issues (Bristol Medical School).
• We have had our own in-house system since '97 and since moving to WebCT we have been trying to find ways
in which we can still use the power of our own system to enhance WebCT. Our own system was a web-
services-based XML content management system and to some extent parts of this are still used (Birmingham
Medical School).
• Don't as yet make much use of the CMS - only installed last month . A lot of bespoke Java and MySQL tools
which are controlled via Blackboard (Durham).
• None specifically at present (Liverpool).
• Currently we also use a sophisticated Intranet development in Cold Fusion to provide linkages to Computer
Aided Learning (CAL) packages which cannot be uploaded into the VLE. We are also using the CMIS
timetabling system which is not yet integrated into the VLE (RVC).
• The Clinical Log is the main thing outside the VLE, Also we have had to develop applications to support other
areas i.e. Special Study Unit and Placement allocation (PMS).
14.3. Summary
Altogether a rather more mixed picture than in 2001, with 12 out of 28 respondents using a commercial VLE,
compared to only 2 out of 21 in 2001. This change in proportion is partly accounted for by schools that either did
not exist in 2001 or were not yet using a VLE. However of the 7 schools that have started using a VLE, only 4 are
using commercial VLEs, so this does not account for the full picture. The remainder are probably accounted for by
3 respondents who are currently using a commercial system but didn’t know what was happening in 2001, and the
2 schools which have switched from a bespoke VLE to a commercial system. A further complicating factor is that 2
schools who said they were developing their own system in 2001 have since adopted the open source system
Bodington, which could be described as an off-the-shelf non-commercial system. One of these is Leeds, where
Bodington was developed (though not specifically for clinical subjects).
15. Pressure to standardise
As institutions have begun to adopt VLEs centrally has there been any pressure on schools to fall into line with
institution-wide VLE strategies and how have schools responded to this?
15.1. Relationship between school and institutional VLEs
Users of bespoke systems were asked to say how the situation in their school related to that of the hosting
institution, by indicating whether the hosting institutions was using:
• The same bespoke VLE as the school.
• A commercial VLE.
• A different bespoke VLE to the school.
Page 27
• Both a commercial and bespoke VLE side by side.

The 2 bespoke users whose hosting institutions are using the same bespoke VLE are the 2 who use Bodington
(which should probably be better categorised as a non-commercial “off-the-shelf” system rather than a true
bespoke VLE) (Figure 8). Thus apart from at Leeds where Bodington was developed, there is almost no use of
truly bespoke systems among hosting institutions.
15.2. Institutional attitudes to bespoke VLEs
In the light of reports that some schools who had developed their own VLEs had been put under pressure to
adopt the same system as their hosting institutions, bespoke users were asked to indicate their host institution’s
attitude to their using a different VLE by choosing from the following:
• Applying pressure to adopt main institution VLE.
• Happy for two systems to coexist.
• Tolerant for now but future pressure to standardise is likely.
• Institution interested in adopting an extension of medical school VLE.
• Other.

Attitudes of hosting institutions to school VLEs
Other, 2 Pressure to
adopt main VLE,
Interested in 3
adopting school
VLE, 1
Future pressure
likely, 2
Happy to
coexist, 8

Figure 9 shows that 8 of the 12 respondents said their institutions were happy for the two systems to coexist, but
5 schools said that pressure to adopt the hosting institution’s VLE was either already present or was likely in the
future. Other answers were that they are looking at what elements can be made interoperable with University
level VLE (Manchester), and that the host institution has no VLE (Glasgow).
15.3. Likely future relationship
Bespoke users were asked what they saw as the future relationship between the school’s bespoke system and the
hosting institution’s system over the next 3 years in their school (Figure 10).
What will happen to host institution and med. school

VLEs which currently exist side by side?
Other, 2
Don't know, 1
Commercial
discontinued, 1
Both will
continue, 9
Page 29
The Other answer was that the VLEs would be engineered to allow free movement of data between the 2 systems
(Aberdeen). Clearly most of the respondents to this question believe that both systems will be able to exist side by
side, and the “Other” answers to the last 2 questions suggest that this may be facilitated by greater interoperability
between the 2 systems – possibly with the outcome that they become complementary rather than mutually
exclusive. Nevertheless it does seem that some institutions do not share this view, and some schools were, in
2004, under pressure to adopt institutional approaches (possibly with some aspects of their own independent
systems remaining).
16. Changes to VLEs since 2001
Which of the planned developments have actually been implemented and how successful have they been?
16.1. Features added since 2001
The 2001 report contained quite a long list of additional features and functionality that developers intended to
implement over the coming 2 years or so. The 2004 survey asked the bespoke developers how far this
development had actually taken place, to gauge the overall functionality of the current bespoke systems as well as
plans for further development.
This question was divided into 2 sections:
9. Covering a number of features, mentioned in the 2001 survey, that are intended to make the systems more
personalised: these were sufficient to require a separate dedicated section.
10. Covering other features spanning a range of functions.
Personalisation features
Section i) asked about the following features:
• Separate portals for teaching staff management and students with interface relevant to each group.
• Personalised tools such as “My Timetable” “My Calculator” “My Calendar”.
• Integration of VMLE with a personal academic record system (PARS).
• An on-line learning portfolio management system.
• Intelligent analysis of user's learning style to recommend further suitable resources in the style of Amazon’s
“customers who bought book 'A' also bought books 'B' and 'C' and CD-ROM 'D'”.
• User specified individual pathways through materials9.
For each of these, respondents were asked to say whether this was something they:
• Already had in 2001.
• Had adopted since 2001.
• Planned to adopt.
• Had no plans to adopt.
9 This list is not exhaustive and could also have included: annotations, smart book marking, PPD, discussion, but was developed
from responses to the JTAP-623 2001 survey.


Figure 11 shows quite a mixed picture, with some progress towards personalisation made since 2001, particularly
with user specific portals into the VLE, but some of the features mentioned in the 2001 report, particularly
intelligent recommendation and user specified pathways have not been widely adopted.
Other features
Section ii) asked about the following:
• Author uploading facilities.
• Assessment tools.
• Two-way communication tools (Discussion board / Chat room).
• Student survey tool for student feedback.
• Separate interfaces for authoring different types of materials such as tutorials and assessment e.g. MCQs.
• Use of MeSH headings for indexing and classification of resources10.
• Enhanced use of MeSH allowing users to select headings from pull-down list or expandable tree.
• Incorporation of RSS feeds.
• Increased incorporation of video resources.
10 Other potential purposes for using MeSH, e.g. curriculum mapping, were not specified in the survey.
Page 31

Figure 12 shows that author uploading tools, assessment tools and 2-way communication were already an
established features of most systems, and that survey tools were a newer but very popular feature, while the use
of MeSH and RSS feeds was neither widely adopted nor planned.
The open-ended response option appended to these questions elicited some explanatory comments, particularly
about the use of MeSH11.
• We don't use MeSH in the main VLE, however we do use it in conjunction with our Resource Discovery
System which catalogues 'raw materials' (images, sounds, videos) for developers to create learning packages for
the main VLE (Nottingham).
• We use MeSH in some areas e.g. anatomy but it is not used in e.g. Public Health (Oxford).
• MeSH possibly in the future (Aberdeen).
• MeSH headings are just too complex. We rely on IIS, which seems largely satisfactory. Students and staff can
optionally customise a wide variety of specialist external search engines from their personalised desktop
provided on our MLE (Southampton).
One school reported that they had implemented RSS feeds but since removed them (Aberdeen).
A full breakdown of the features implemented by each of the 14 schools using bespoke systems is given at 30
Appendix Two: How bespoke VLEs have developed since 2001 on page 65 below.
16.2. Integration with other information systems
Another area where developments were ongoing was in the integration of the VLEs with other institutional IT
systems. In 2001 there were few schools that had made much progress in this respect.
The 2004 survey asked all the developers to indicate to what extent their VLE is integrated with other institutional
IT systems, by selecting from the following:
• “Weak” single-sign-on that allows VLE to be accessed with same login details as other institutional systems, but
users have to login separately to each system.
11 The Subject Centre-funded project METRO project also offers a critique of MeSH for education and intends to develop its
own set of subject headings designed for medical education. See
http://www.medev.ac.uk/resources/features/docs/metrofinal_report.pdf

• “True” single-sign-on where login information can be passed between systems so the user only has to login
once to access all participating systems.
• VLE is integrated with Athens systems.
• Students and staff can use all features of the library system from within the VLE interface.
• Staff can build reading lists etc directly from within the VLE interface.
• System integrates with student record system to record assessment scores.
• Student details can be taken directly from student record system and displayed within the VLE.
• Details of student membership of units / modules / groups can be taken from student record system and used
within the VLE.
• Other.

Other responses were:
• Moving towards single-sign-on and passing information between systems - bits of it work e.g. between portal
and Blackboard (and can be for Medicines Intranet and Blackboard - just need to do a bit of development
work). (Bristol Medical School).
• Projects this year will incorporate Shibboleth authentication with bespoke VLE, library sources and institutional
VLE (Blackboard). (Newcastle).
• A number of the above are under consideration but have not been integrated yet. IE integration with MIS and
Library systems (PMS).
• Currently piloting Sentient reading list integration for Medical School (Oxford).
• Working with our institutional portal team towards true single-sign-on. Athens access currently in testing
(Aberdeen).
• above in progress (Cambridge).
• VLE will be integrated with Athens systems eventually (Manchester).
• Athens (and later Shibboleth) authentication will be available later this year. The latter should see SSO
implemented for all University services (Durham).
• Separate sign-on for main system & VALE (medical VLE). Data has to be wangled out of student record system.
VALE keeps a record of all aspects of the medical course (exam marks / groups etc.) plus student details (home
address, term time address etc.). (Glasgow).
Page 33
• We have a separate bespoke student and staff record system which is informed by the central University
system, but which goes further. These systems inform the MLE, e.g. to provide appropriate content for
student's position on course. However, for security reasons, these systems are not designed to be individually
interrogated from within the MLE (Southampton).
• We were sold the idea that WebCT was completely integrated with the institutional systems but this is not the
case. WebCT user database is separate from campus authentication (Active Directory) and gets populated
from data dumps (Birmingham Medical School).
Figure 13 shows that at least 24 out of 28 schools have or are planning some level of integration. The most
common level of integration is a single-sign-on that has to be entered separately for each system, while a relatively
small number have so far implemented more sophisticated integration such as “true single-sign-on” or linking to
student record systems. The 11 “other” responses mainly focus on plans for further integration, suggesting that
fuller integration is on its way even if it is not there yet in most cases.
17. Growth of VLE usage
How has usage of VLEs by staff and students grown and developed?
17.1. Number of schools using a VLE
As Figure 1 shows, 25 out of the 26 schools represented in the Developers Survey are using a VLE (compared to
17 out of 21 surveyed in 2001). The Academics Survey represents a somewhat larger sample of the sector, with 44
schools represented, and including a greater number of dental and veterinary schools.
Where the Developers Survey asked about the type of VLE in use, the Academics Survey contained questions
about the extent to which the VLE has become embedded into everyday practice. Respondents were asked to
choose from:
• Although we are using a range of e-learning applications we are not bringing them together into a VMLE.
• We have only just started using a VMLE.
• We are really just beginning to get our VMLE incorporated into our local teaching and learning culture.
• Our VMLE is used by a lot of our staff and students but there is still some way to go before they are all on
board.
• Our VMLE is now used as a matter of course by most of our staff and students.
• Other12.
12 All3 “other” responses could be incorporated into existing categories, with the exception of 2 for whom the Exploratory
steps category was added.

Figure 14. Source: Academics Survey (n=38).

When asked about e-learning activities in general whether or not they were delivered using a VLE, all of the 44
schools represented said that at least one e-learning activity was taking place to a significant extent at their school
(see below), but as Figure 14 shows, 5 of these were not hosting these activities within a VLE and 2 were only
taking tentative steps towards doing so. Figure 14 shows that use of a VLE is underway to some extent in 31 of the
38 schools who responded to this question.
17.2. Extent to which VLEs are embedded into local practice
Figure 14 also shows that 24 out of the 38 believe that the VLE is used widely across the school, either by many
but not yet all, or by all as a matter of course.
Page 35

Figure 15 shows that, broken down into subject disciplines, the greatest level of embeddedness appears to be in
Medicine, where the largest group of 9 schools (almost half of the 19 that responded) see their VLEs used as a
matter of course by staff and students. This represents a step onwards from 2001, where the impression was that
in most school VLE uptake was more sporadic and patchy.
17.3. Levels of use
Change in level of VLE use over the last 3 years
don't know/not
applicable, 2
increased
slightly, 1 increased
massively, 8
increased
significantly, 12

The picture of increased use is supported by responses to a question from the Developers Survey, which asked
respondents how the level of use of their VMLE(s) has developed over the last 3 years. Figure 16 shows that for

20 out of the 23 schools responding, usage has increased either “significantly” or “massively”13. No respondents
said that usage had remained stable or decreased.
Usage statistics
A more precise measure of the change in usage could be gained from usage statistics. The Developers Survey
asked respondents to provide as much of the following information as they could:
• Number of students in your school.
• Number of students who have used the system.
• Number of hits per year.
• Number of sessions per year.
• Number of hits per day.
• Pages viewed per day.
Only 10 respondents were able to give detailed figures. As pointed out by some respondents, these kinds of
figures are also notoriously unreliable and hard to interpret, particularly for the purpose of comparison between
different systems. Appendix Three: Full system usage data - 2004 on page 67 below presents the full figures
without comment in case any interested reader should wish to draw inferences from them. More reliable might be
a comparison of the same systems at different points in time. For this purpose, the following table presents the
2004 data alongside the 2001 data for the few instances where the equivalent is available.
School Year Students Studen Hits Sessions Hits per Pages

in school ts used per per year day viewed
the year per day
system
Edinburgh 2001 1,900 880 n/a n/a n/a 1,000+
(medics and
vets)
2004 1,900 1900 n/a n/a n/a 30,000+
(medics and
vets)
Nottingham 2001 1,000 650 4,100,000 n/a 9,000 3,500
2004 ~1100 Most 17,000,00 160,000 90,000 27,000
0
Cambridge 2001 246 236 200,000 18,000 (all n/a n/a
years)
2004 475 (across 465 2,400,000 17,000 per 6500 1700
3 years) intake year
(logins)
Birmingham 2001 750 n/a n/a n/a 50,000 n/a
Medical School
2004 2,000 2,000 >12,000, n/a 35,000 n/a
00014
Table 1. Source: Developers Survey and 2001 JTAP-623 Report (n=4).

Table 1 shows that Edinburgh, Nottingham and Cambridge could all be justified in claiming a “massive” increase in
use15. Even the number of schools for which this type of comparison is available is very small, it does allow some
quantitative indication of what may be meant by a “massive” increase.
13 Acknowledging of course that these terms are very imprecise.

14 Itis not clear from Birmingham’s response whether these figures refer to the original bespoke system which is still in use as a
supplementary VLE, or to the medical school’s use of the WebCT VLE adopted by the hosting institution and now officially
used by the medical school as their main VLE.
15 Which Edinburgh and Nottingham did but Cambridge only claimed a “significant” increase.
Page 37
Years using most heavily (e.g. 1st , 2nd, etc.)

Respondents were asked to say which years of the curriculum used their VLE(s) most heavily.
Usage of VLEs across years of curriculum
14
12
10
No. of schools
0
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6

Figure 17 shows the number of schools mentioning each year of the curriculum as seeing the heaviest use 16.
Where respondents mentioned more than one year, each year mentioned is counted. This gives an overall picture
suggesting that usage is heaviest in the first 2 years. This is supported by the fact that 11 of the 19 respondents
mentioned exclusively a combination of the years 1 – 3 of the curriculum (i.e. 1; 2; 1 and 2; 1, 2 and 3) as seeing
the heaviest use, while only 2 mentioned exclusively a combination of years 3 – 5. The remainder either said the
usage was even across years or mentioned other patterns such as years 2 and 4.
18. IT and support infrastructure
How much demand does the growth in VLE usage place on IT and support infrastructure?
The results presented below represent survey questions that were not directly or systematically addressed in the
2001 survey so is not possible to link them directly to growth in VLE usage. The results presented simply indicate
the level of IT and support infrastructure in place at the time of the survey.
18.1. IT infrastructure
Server descriptions
Developers were asked to briefly describe the configuration of the server they use to run their VLE.
16 It was not specified whether Year 6 refers to premed, intercalating students or PRHOs.

VLE server configurations
Part run by Not indicated, 1

hosting insitution,
2
"Microsoft": Win
Unknow n 2000/2003, IIS,
because run by SQL Server, ASP
hosting etc, 9
institution, 3
"Unix": Solaris,
Apache,
Zope/PHP,
MySQL etc, 6

As Figure 18 shows, most of the schools fall quite neatly into one of two types:
• Nine of the schools use “Microsoft” based set-ups mostly using a combination of Win 2000 / 2003, IIS, SQL
Server, and ASP.
• Six use what could broadly described as “Unix” setups including Linux, OSX, and Solaris with Apache, MySQL,
Zope or PHP.
Few of the responses gave much indication of the computing power they use to run their VLE. However 8 of the
respondents mentioned that they run their VLE on multiple machines and 2 schools gave an indication of the
processor type and memory their systems have available. See Appendix Four: Server Descriptions on page 68
below for full responses to this question.
Support teams
Figure 19. Source: Developers Survey (n= 22).
Page 39
Respondents were asked to indicate the number of staff FTEs they have working in support of their school's VLE
in each of the following roles:
• Web development.
• Multimedia development (incl. video / audio).
• Educational development.
• Server / site maintenance and support.
• User training (staff and/or students).
• VLE administration.
• e-learning development.
• Other.
They were asked to place these on the following scale:
• 0 FTEs.
• 0.1 - 0.9 FTEs.
• 1.0 - 1.9.
• 2.0 - 2.9.
• 3.0 - 3.9.
• 4 or more.
Figure 19 shows the result of this analysed by role. This is unfortunately rather hard to interpret, although it does
show that many schools have less than 1 FTE in each role, except for the roles of Server / site maintenance, VLE
administration, Educational development and e-learning development where at least half of the schools have more
than one FTE. It also shows that the approximate total number of FTEs in each role across the sector is fairly
similar, with slightly more e-learning developers, Web developers and Server / site maintainers, and rather fewer
multimedia developers.
More illustrative, but less accurate, is to provide the same data analysed by school. The following graph (Figure 20)
attempts to give an indication of how the support teams are composed at each school. The raw numbers are
actually inaccurate, as in order to create the graph the scale has been rounded up to whole numbers (0.1 – 1 FTE
converted to 1, 1 – 1.9 to 2 etc.). This means that a staff member working 0.1 of an FTE in a particular role is
counted the same as someone working 0.9 in that position. However it does give an indication of the proportions
of each role at each school, and enables comparison between schools. While inaccurate in terms of FTEs, in some
cases it may give a better indication of the actual number of staff employed on whatever basis.
Figure 20. Source: Developers Survey (n= 22). Note that, in some cases, numbers indicated refer to the generic institutional
support available (e.g. Birmingham Medical School, Durham, Leeds, LWMS, Southampton) rather than the VLE support
specific in each school (e.g. Bristol Medical School, Cambridge, Edinburgh, Glasgow, Liverpool, Newcastle, Nottingham, RVC,
Sheffield); and number may include staff on project or ‘soft’ funding. Where institutional figures are quoted there may be no
school-specific support available (e.g. Durham).

Note that, viewing from left to right, Newcastle to Sheffield inclusive represent schools using a bespoke VLE, Oxford and
Leeds use the open source Bodington VLE, and Bristol onwards all use a commercial system. Note that 6 respondents of the
28 did not answer this question, and the 1 that answered anonymously is not included in Figure 20.
In terms of the relative numbers in each role at any institution, no clear pattern emerges. However what is clear is
that a small number of institutions (c. 6) have large development teams (up to 20 people), where another 11 have
around 6 or 7 staff, and the remaining 4 have fewer than this.
Another observation is that many institutions have a fairly equal mix of several roles (e.g. Liverpool, Durham, RVC)
while others emphasise the technical, administrative and content development roles (e.g. GKT School of Medicine,
Manchester, Glasgow) and others the training and educational development roles (e.g. Leeds). Not surprisingly the
schools in which the role of web developer is emphasised tend to be those that have a well-developed bespoke
system (e.g. Newcastle, Southampton, GKT School of Medicine, Cambridge), although at some other bespoke sites the
team members seem to perform a range of roles (e.g. Edinburgh, Aberdeen)17.
This is clearly a complex question, and there is great variability between institutions. Part of this may be accounted
for by the difference between bespoke systems which are developed and maintained by the schools, and
commercial systems run by the hosting institutions.

Respondents were asked to indicate how their support staff were provided. Figure 21 shows that as a whole most
of the support staff are located within the school, although the roles of server maintenance, VLE administration
and user training were just as likely to be provided by the hosting institution.
17 However it must be acknowledged that these roles are not very clearly defined and that there may considerable overlap
between them. A fuller understanding of support roles would require a much larger scale project.
Page 41

Figure 22 shows the same data but only for schools using bespoke VLEs (including the 2 open source systems).
Since bespoke systems are generally created and maintained within the school it is not surprising that the general
picture in these schools is of support being providing locally, and in particular all web development staff are
provided within the schools. Nonetheless 3 institutions provide server support and some of the “softer” support
areas such as educational and e-learning development are provided both centrally and locally.
19. Sharing of system components and content
How much collaboration and sharing of system components and/or content has there been between schools
developing VLEs?
The 2001 report observed that not only were the VLEs being implemented with UK Medical Education generally
those designed specifically for medical education rather than more general “off-the-shelf” systems, but it also
seemed that a large number of medical schools were developing their own systems, resulting in substantial
duplication of effort and increased development cost. Thus the report recommended that further efforts be made
to facilitate the sharing of both VLE components and content across the sector.
The case for a single medical VLE

The 2004 survey also set out to find out whether developers believe that, while the one-size-fits-all solution
offered by commercial systems does not suit medical / clinical education, there is any case for a single or shared
solution for the sector.


Developers of bespoke systems were asked whether they still agreed that even a standard VLE specifically for
medicine and related clinical subjects is not ideal because of variations in curricula between programmes and
schools. Among the 14 bespoke developers who responded to this question, a majority agree that there is not a
case for a single medical VLE (Figure 23).
Interoperability technologies
The first step towards sharing of components and content is arguably the implementation of technologies and
standards into VLEs that will facilitate interoperability. The Developers Survey asked respondents to indicate
which of the following they have implemented:
• RSS external feeds.
• XML.
• SCORM.
• IMS standards.
• IMS QTI specification for questions.
• MeSH indexing system for standardised indexing of resources.
• Learning Object metadata (LOM).
• Other.
Page 43

Figure 24 shows that by far the most common interoperability technology is XML, implemented by a small
majority of schools, with Learning Object metadata and the IMS QTI specification also implemented by a sizeable
minority. See Appendix Five: Interoperability technologies implemented in VLEs on page 69 below for a full
breakdown of interoperability technologies and standards implemented at each school.
“Other” responses included:
• Unfinished beta QTI (Nottingham).
• Only implemented in isolated examples but Bodington is IMS QTI (Oxford).
• Looking at RLOs (Aberdeen).
• Delphi (Glasgow).
• We are making our MLE IMS compliant to support use of IVIMEDS18 RLOs (Southampton).
18 See Discussion on page 54 for more about IVIMEDS.

Sharing system code and components
Sharing system code with other systems
Not on the
Top priority, 1
agenda, 4
High priority, 4
Low priority, 7
Medium priority,
7

Developers were asked what level of priority they gave to sharing system code with other schools. Figure 25
indicates that this is only a high priority for 5 out of the 23 schools represented, and for 11 it is low priority or not
on their agenda at all. Top priority is given to this by Aberdeen, and high priority by Newcastle, RVC, Manchester
and one anonymous school.
System code has been shared with other institutions
A great deal, 3
Not at all, 7
Somewhat, 5
Not very much, 8

Developers were asked how much sharing with other institutions had actually taken place. Figure 26 shows that 8
out of the 23 had shared code either a great deal or somewhat, showing that some schools have done this even
when it is not a high priority. These notably include Leeds and Oxford, the 2 users of the Bodington open source
VLE. However there are others for whom this is a high priority who have not done this at all yet, notably the
RVC.
Page 45
Routes for sharing system code/components
Other, 2
Open source
systems, 6
Formal
collaborative
project, 7
Modular
systems, 4
API
developments, 3

Developers who were interested in sharing and interoperating were asked by what route they were doing this by
choosing from the following:
• Systems will be made available as open source so that other institutions can use components of it.
• Systems will be made more modular which will enable other institutions to adopt parts of it.
• We are developing and/or using shared components written using an Application Program Interface (API) such
as Blackboard Building Blocks.
• We are involved with (an)other institution(s) in a formal project to share system components and tools.
• Other.
Figure 27 shows a variety of approaches – notably that Blackboard users are beginning to collaborate within the
sector and that there are number of collaborative projects running to this end. “Other” responses were “not
sharing, interoperating” (Edinburgh) and “We have talked to other institutions who are interested in sharing our
development” (Glasgow). A number of other schools used the “Other” box to add detail to their response:
• Current FDTL-4 project (e-portfolios) means that we share with three other institutions anyhow. Previous
TLTP3-86 project shared our work with three other institutions. We have recently won funding to turn our
e-portfolio system into a web-service and make this available in an open-source way (Newcastle).
• Bodington consortium – Bodington.org19 (Leeds).
• Through the HEFCE funded OCTAVE project we are developing a MCQ question bank using QuestionMark
perception. We share content developed at the RVC with other veterinary institutions through our veterinary
e-learning portal http://vetschools.ac.uk/. The RVC has developed an e-casebook e-case simulator which is to be
shared with Oxford medical school, and possibly other veterinary schools, but this has not taken place as yet
(RVC).
• Developed building blocks are available free via the Blackboard Catalogue. Some initial talks about sharing code
with PMS, Liverpool, HYMS, etc., held late last year (PMS).
19 http://www.bodington.org/

Sharing Content
Priority given to making content readable between

institutions
Not on our
agenda, 3 Top priority, 2
High priority, 3
Low priority, 8
Medium priority,
8

As Figure 28 shows, the level of priority for making content readable by systems at other institutions is about the
same as it is for sharing system code and components and the same schools are enthusiasts for both activities, with
the exception of Liverpool and Birmingham Medical School who are notably more enthusiastic to share content
than they are to share code.
No. of schools that have shared any kind of VLE content

with other schools
A great deal, 2
Not at all, 4
Not very much, 4
Somewhat, 14

Figure 29 suggests rather more actual sharing of content than of system code. Even though respondents were
asked about several specific types of content (see Figure 30 below), Figure 29 shows the number of schools
sharing of any type of content.
Page 47

Figure 30 shows some difference between types of content; interactive materials seem to have been shared rather
more and RLOs rather less, despite the relatively large number of schools implementing a standard for this (see
Figure 24). When compared with Figure 29 the figures seem lower, this is because most schools have only shared
two or at the most three types of content to any extent.
20. Role of VLEs in supporting student placements
In what ways are VLEs most successful in supporting students and teachers located at disparate clinical and
educational sites?
Effective uses of VLEs to support students on placements

Respondents to the open-ended paper-based questionnaire were asked in what way(s) they believe that VMLEs
have been most successful in their institution in supporting students and teachers located at disparate clinical and
educational sites?
Full responses are given as follows:
11. My institution uses a VMLE (Blackboard) to deliver the entire undergraduate medical curriculum – this is a
new medical school and the VMLE has been used since day 1. It was considered essential because of the
dispersed sites (PMS).
12. Development of e-courses such as veterinary dentistry
Use for delivery of e-CPD 6 week courses for vets
Computer aided assessment for formative assessment
Interactive video lectures synchronised with PowerPoint (RVC).
13. Uniting them into a community of learning (Manchester).
14. Rapid coherent resource for all curricular material (UEA).
15. I don’t think this has really happened yet. We are in our 1st year of rolling it out.
16. Ensures equity across placements. Enabling communication over issues that require immediate response.
Continued communication whilst away from campus. Support for new teachers by having all curricular
information and support materials available. Stimulating dialogue with students through discussion forums.
17. Allow contact with tutors on campus. Have teaching and learning resources at a distance. Can access any
time any place (GKT Dental Institute).

Limitations of VLE use at remote sites: The effect of hospital IT infrastructure on VLE use
One of the views expressed during the Twenty-Twenty Vision event was that hospital / GP practice IT
infrastructure placed constraints on how and how much VLEs could be used by learners on placement, and to the
VLE resources that could be made available to students on placement by their school.
Are hospital-based VLE users constrained by local IT

infrastructure?
Yes very much,

Not at all, 3 1
Not very much, 3
Not sure, 1
Yes somewhat,
15

The Developers Survey accordingly asked: Would you say that your user group's ability to take full advantage of
your VLE is constrained by IT infrastructure in hospitals where your students and teachers are located? Figure 31
indicates that a considerable majority of responding schools believed this to be somewhat the case.
Schools' role re: educational IT infrastructure in

placement hospitals
Provide and
support, 4
Other, 6
Little or no Recommend
involvement, 9 specifications,
11
Page 49
Respondents were also asked to indicate their school's role with respect to IT facilities in the NHS
Trusts / Hospitals where ‘your students are on placement’ by selecting from:
• We provide and support infrastructure.
• We provide infrastructure but the NHS Trusts support it.
• We support infrastructure provided by the NHS Trusts.
• We recommend infrastructure specifications.
• We have little or no involvement.
• Other.
Figure 32 shows that only a minority provide and support infrastructure, while the majority either simply make
recommendations or have little or no involvement. Interestingly 8 of the schools who said they were somewhat
constrained also said they recommended specifications – suggesting perhaps that their recommendations are not
always followed.
“Other” responses included:
• A learning and teaching network for medicine has been established between the NHS Trusts and the University
with clearly defined roles and responsibilities for who delivers and supports which bits (Bristol Medical School).
• We have buildings located at each site with University infrastructure, but within Trusts we have no
involvement (PMS).
• Some we have responsibility, others we don't (Edinburgh).
• Students use learning resources including digital video in the RVC hospitals (RVC).
• The School Director of information and computing sits on main NHS Trust IT committee. A main limitation is
lack of NHS Trust for Java due to security constraints. There is a fibre connecting Trust to the University
(Southampton).
• Dental students are rarely on placement. However, some NHS terminals do not allow web access to the VLE
(Birmingham Dental School).
21. How VLEs are used
Has there been any shift towards using VLEs as a platform for on-line learning rather than as a means of delivering
more traditional documentation?
The 2001 report found that most schools were using VLEs heavily to i) deliver information about the programme
(timetables, clinical placement information, announcements, administrative documentation etc.) and less as ii) a
platform for on-line learning (on-line tutorials, teaching materials, simulations, on-line discussion, on-line
assessment etc.). The 2004 survey set out to find out the extent to which this is still the case.
Relative importance of information delivery vs. on-line learning

Respondents to the paper-based questionnaire were asked to comment on the relative importance of these two
functions in their VMLE.
18. Both equally weighted and will become more important when students are on clinical placement.
Lecturers / tutors need more support for ii) - we will concentrate on this in the future. we have a iii) which
is an electronic library (PMS).
19. Academics are increasingly using ii) although i) is much quicker and less challenging to produce. We see the
future as putting more emphasis on ii (RVC).
20. I see i) as an enormous potential benefit, given the administrative complexity of the curriculum and the high
penalty of administrative failure. ii) is plainly important but we don’t have even good theoretical models as
to how to achieve them, let alone empirically validated methods (Manchester).
21. i) much more than ii) but early days (UEA).
22. We are treating them with equal importance but at the moment we have control over the on-line learning
but we are restricted by the institution about the format of the programme information which is not always
user friendly.

23. We are currently using the i) , with elements of the ii) i.e. repository for teaching material. Would like to
develop more interactive material to support clinical learning on placement.
24. i) is just the calendar, daily course structure information and static information. This is important but
different from a greater interactive series of elements as mentioned. These are essential in any well
designed on-line course (GKT Dental Institute).
The overall impression is that respondents see i) as less demanding and consequently has in many cases been
emphasised, but that there is interest in and in some cases a recognition of the potential for ii) but also some
concerns about the cost and validity of such materials and activities.
How e-learning is used

The academics were asked to indicate all of the e-learning activities that were taking place to any significant extent
in their schools:
• Broadcast of live lectures to sites off-campus.
• On demand delivery of digitally recorded lectures.
• On demand delivery of other video material.
• Live real-time on-line text communication (chat).
• Asynchronous discussion (bulletin boards).
• Interactive tutorials.
• Electronic student portfolios (e-portfolios) / personal learning records.
• On-line presentation of patient cases.
• Signing up on-line to clinical teaching and/or student selected components.
• Managing outcomes.
• On-line management of course literature such as programme / module / study guides.
• On-line formative assessment.
Page 51
• On-line summative assessment.

• Other (please specify).
Other activities mentioned were:
• Study Skills (Dublin Dental School).
• Use of VLE (GKT Dental Institute).
• Details of outreach clinics (Birmingham Dental School).
• Notice-board function (BSMS).
• On-line image-dependent learning resources (SGUL).
• Video linking of live lectures between campuses (HYMS).
• PBL groups prepare and share materials for their own group + lecture and seminar notes are available to all
students (UEA).
While this question focuses on e-learning activities generally and not just VLE usage, the fact that 31 of the 38
schools who responded to the relevant question in the Academics Survey are actively using a VLE (see Figure 14)
suggests that much, if not most, of the activity described in Figure 33 is delivered through a VLE.
The single most common use of e-learning selected is for on-line management of course literature such as
programme / module / study guides, with 41 of the 44 respondents choosing this activity. Thus the heaviest use of
VLEs is still as a means of delivering information about the programme. However many schools are engaged in on-
line learning activities, the most common of which are formative assessment and asynchronous discussion.
For a breakdown of the e-learning activities taking place to any significant extent at each of the schools who
responded to the Academics Survey please see Appendix Six: Breakdown of e-learning activities undertaken to any
significant extent at UK medical, dental and veterinary schools on page 70 below.

Academics were also asked to describe the four main drivers for using a VLE in their school, and to rate them in
importance from 1 to 4 (where 1 is most important). Figure 34 shows the results represented as a % of the
maximum potential score for each driver20. While the result shows a rather mixed picture with no individual
drivers standing out as massively more important, the two highest scoring drivers were both related to curriculum
management – a result very similar to the 2001 survey.
20 Scores were calculated by allocating a score for each item rated 1st in importance, down to a score of 1 for each item rated
4th in importance. These were then added together to get a total score for each driver.

22. Effect of VLEs on learning and teaching practice
Have the VLEs started to have any effect on learning teaching and assessment practices?
Respondents to the paper-based questionnaire were asked to describe what effect, if any, their VMLE has started
to have upon learning, teaching and assessment practices in their institution. Full answers were as follows:
25. Unable to compare with conventional format21 – however, a recent JISC study at our institution identified
that students would prefer to have non-electronic resources to support the VMLE (PMS).
26. Students increasingly using e-learning to support existing courses. Replacement of some taught courses by
e-courses. Use e-learning to deliver PBL (RVC).
27. Raise its22 profile and identify resource demands of placement education (Manchester).
28. Not much – mainly administrative (UEA).
29. Encouraging staff to examine current practices and possible use of VLEs to develop new modes of delivery.
30. We are beginning to look at assessments on line as well as learning. For us it hasn’t happened yet but we
anticipate it will save teachers’ time … eventually.
31. Free up staff. Students like interactive comprehensive resource and can produce very high quality work on-
line. Assessments are easily collated (GKT School of Medicine).
Again these results, from a very small sample of the sector, show a mixed picture of schools where use of VLEs
has already had a significant impact (releasing staff time, facilitating review of practice) and where it is too early to
say.
21 As this is a new medical school that used a VLE from the outset.
22 For example, of learning and teaching and assessment.
Page 53
Discussion
22.1. Which of the planned developments have actually been implemented and how
successful have they been?
In medical, dental and veterinary education, just as across the HE sector as whole, VLEs are no longer “flavour of
the month” as they were three years ago, but are taken for granted much more. This can be seen as a reflection of
their success; many schools now use VLEs “as a matter of course”, and they become less remarked upon, less
visible. Furthermore it is arguable that VLEs have started to become something more than what we understood a
VLE to be 3 years ago – they seem to be evolving into an aggregation of, or gateway into a wider set of tools
including (e-)portfolios and progress files, assessment etc. An important question for the further development of
these systems is how much they will need to include all these tools within them (with all the risks of bloatware we
have seen in many other applications), or will they become something more like a portal, a central point where the
tools from various sources are brought together and used in an integrated way?
The most common developments that have been implemented in bespoke VLEs centre on 3 areas:
32. Personalisation of systems. In medicine, the need to present personalised content is less emphasised
than other subjects as typically the curriculum contains few elective components so all students in a given
year will largely see the same material. A personalised portal simply means showing a different home page
depending on their role (staff, student, etc.) or which year they are in. Where VLEs are becoming truly
personalised is with the addition of tools such as e-portfolios, and as their functionality and importance is
extended VLEs take a major step towards becoming personalised learning environments rather than a
wholesale means of representing the entire curriculum.
33. Integration of institutional IT systems. The main development in this area has been towards
single-sign-on. However few have so far implemented the “true single-sign-on” that will be necessary for
different systems to be linked together seamlessly. Traditionally integration has been understood to mean
passing data between VLEs and other institutional information systems such as library catalogue and student
records systems. However if true single-sign-on does eventually occur, one of the results may be that it
would become feasible to replace the single jack-of-all-trades VLE with aggregations of linked specialist
e-learning tools such as assessment and e-portfolios.
34. Interoperability. The fact that most VLEs, bespoke and commercial are gradually introducing support
for technologies, standards and recommendations that will facilitate interoperability, particularly XML and
IMS QTI adds further weight to the view that it will come to matter less and less what VLE is used as the
core of the system, as it will become easier to link systems together.
One caveat to all this is that by building a VLE from a set of disparate tools one runs the risk of losing any sense
that what you are presenting is a single unifying environment. If each tool within the VLE offers a completely
different look and feel and a separate navigational structure the user may easily become disoriented, as well as
losing any sense of the VLE as the on-line representative of the school or institution. To avoid this, applications
should be linkable at quite a deep level - so that for example you can link a discussion board directly from a
relevant content area – and integrated seamlessly so that the user is unaware that the tools they are using come
from different sources.
The question of how successful the developments have been is difficult to quantify and proved beyond the scope of
this survey to address directly, although some indications of the implications of the developments will be
addressed under other headings (see below).
22.2. How much collaboration and sharing of system components and/or content has
there been between institutions developing VLEs?
The 2001 report noted:

“Given the shortage of resources, this massive duplication of effort, involving only very limited
sharing of systems and components between institutions seems wasteful and illogical. This
seems particularly true given that different systems have developed strengths in different areas,
which if shared could enhance the functionality of all the systems.” JTAP-623 2001.
It is clear from the 2004 survey that medical, dental and veterinary schools are unlikely to set out to collaborate,
to the extent of creating a single or even a smaller number of VLEs supporting clinical education in the UK.

Although there has been a reduction in the number of developments counted since the 2001 survey, this is mostly
due to the adoption of commercial systems at several schools rather than a merging of bespoke systems. Overall
the picture is not one of schools gradually converging to enable greater sharing of code and/or content across the
sector, but of pockets of collaboration working to develop interoperability between systems and sharing of
content.
Most of the code and content sharing between developers seems to have been done between groups of schools
that have become associated via a variety of means, including:
35. as a result of formal collaborative projects for example:
c) the group led by the University of Newcastle resulting from a TLTP3 project - although the number of
schools using the core system created at Newcastle has not expanded beyond the 3 schools doing this in
2001.
d) the FDTL4 UMAP project led by Manchester developing a bank of assessment questions to be shared
between participating medical schools http://www.umap.man.ac.uk/.
e) developments related to the Bodington consortium; Oxford’s medical faculty is coordinating a project to
embed the JISC funded systems LAMS and TOIA within the Bodington VLE.
f) the OCTAVE project to share assessment questions between Veterinary Schools (this is on a costed basis,
where contributors are paid per question – this of course then raises the issue of who then has IPR over
the questions) http://www.rvc.ac.uk/RVC_Life/News_and_Events/OCTAVE.cfm .
g) the ACETS project to investigate the use of third party materials in support of teaching and learning.
ACETS is part of the JISC Exchange for Learning Programme http://www.acets.ac.uk.
36. through membership of specific national and international organisations such as:
h) IVIMEDS, an international consortium of medical schools aiming to “provide an effective means of sharing
digital learning resources among partner institutions” http://www.ivimeds.org/. IVIMEDS focuses on the
creation and sharing of RLOs that can be hosted via any content management system or learning
environment.
i) Universities' Collaboration in e-learning (UCEL) - a multi-institutional collective to collaboratively produce
high quality interactive multimedia resources for health-professional education. Its six founding partners are
Cambridge, Nottingham, Manchester, UEA, Wolverhampton and PMS http://www.ucel.ac.uk/.
j) The Scottish Deans’ Medical Curriculum Group (SDMCG), a collaborative group of all five Scottish medical
schools has developed a common set of outcomes for Scottish medical education. The SDMCG has a
technical group where issues pertaining to VLEs are discussed.
37. through using the same commercial VLE – see above. Ironically the users of commercial VLEs are realising
the benefits of code sharing as much as bespoke developers, assisted partially by the commercial VLEs’
existing procedures for developing and sharing application extensions (for example Blackboard Building
Blocks). A group of schools using Blackboard are beginning to meet to discuss sharing of Building Blocks.
38. there are also a large number of new projects involving medicine, dentistry and veterinary medicine funded
by the JISC through, for example, the Distributed e-learning Programme, Digital Repositories, etc.,
http://www.jisc.ac.uk/.
For projects focussing on the sharing of content, such as OCTAVE and IVIMEDS, universal implementation of the
relevant standards and specifications (e.g. IMS QTI and IEEE Learning Object Metadata) would seem to be vital.
However this survey found that the IMS QTI has not been implemented into the VLE at very many schools,
whereas the standard for learning object metadata has been implemented in the majority23, (although interestingly
not by all the schools who are members of IVIMEDS). The OCTAVE project standardises around the proprietary
standard used by QuestionMark Perception, although QuestionMark does itself support QTI. This project has
demonstrated how third party software such as Respondus can use generic standards to mediate between systems
and facilitate interoperability of materials such as assessment questions – thus further reducing dependence on an
individual platform.
While considering the role of interoperability standards and specifications to facilitate content sharing, there seems
to be little relationship between whether a school prioritises the sharing of a particular content type and whether
it has implemented the technology for doing so. For example few schools say they have shared RLOs but most
23 However the survey question did not stipulate the IEEE P1484.12 Learning Object Metadata standard specifically so there
may have been some variation in how the question was interpreted and in the respondents’ understanding of what is meant
by ‘reusable learning object’ (RLO).
Page 55
have implemented the Learning Object metadata standard for doing this, while conversely assessment questions
have been shared by a greater number of schools but few claim to have implemented the IMS QTI. This suggests
that for the moment at least, interoperability technologies are implemented not so much in order to facilitate
content-sharing between schools, but to interoperate with other or future systems within the same school, or
simply by default where commercial systems come with those standards ready implemented.
It is also interesting to consider possible reasons for the greater sharing of questions than of RLOs. There is still a
great lack of clarity about what a RLO is24 – the term can be used apparently to refer equally to a single image as
to a complete sequence of learning activities. There may be some reluctance to use RLOs created elsewhere
because each teacher will want to present the material within a context and in a style which is their own rather
than the one provided a shared RLO 25. Arguably, questions used for assessment lend themselves more to sharing
because they are habitually presented to students independently of context, and other granular resources such as
images because they impose no ready made context so are highly flexible in how they can be used.
A remaining barrier to convergence is that, just as three years ago medical VLEs were seen as a special case
because of specific subject requirements, some of the specific VLE tools that have been developed to support
medicine, particularly assessment tools, offer functionality that cannot be represented using existing standards,
making it harder for schools who have developed particular strengths to share these.
One perhaps surprising finding is that there seems to have been somewhat more sharing of content than there has
been of system code and components. The 2001 survey suggested that the obstacles to content sharing seemed to
be mainly to do with IPR and ownership issues, which might be expected to be rather more intractable than
technical issues. However the salience of these political and legal obstacles may itself be a reflection of the high
value placed on good quality content. Perhaps the reason that content is shared is because it is worth sharing, but
the experience of the OCTAVE project shows that sharing of content is likely to be successful only if incentives
for doing so reflect the true cost of creating it.
A possible reason for the relatively little sharing of system code and components is that as systems become more
interoperable the cost of diversity is reduced and along with it the imperative towards convergence. It doesn’t
matter if everyone is using different systems if they can all understand each other and work with the same content.
One area that was not investigated in this survey was the potential for sharing learning process rather than just
content. An example of this might be where learners from different institutions take part in a single on-line
discussion. Informal reports indicate that this has been happening on a tentative and unofficial basis in some parts
of the sector. There are clearly contractual and other procedural obstacles to this, but it is an area that merits
further investigation in terms of distributing expertise and learner perspectives.
22.3. Given the rapid evolution of commercial VLEs, is there still an overriding case for
the continued use and development of bespoke medical VLEs?
The 2001 report said

“Even though commercial systems have progressed substantially in the past few years, at the
same time there has emerged a substantial body of highly developed expertise working
specifically in the medical education field. A clear vision is emerging from these people about
how to develop these systems further and to turn them into sophisticated personalised learning
environments that are even better suited to the specific needs of this context. In this respect the
bespoke medical VMLE community is likely to remain ahead of their commercial rivals for the
foreseeable future.” JTAP-623 2001.
Is this still the case? This survey suggests that the answer to this is more mixed than it was in 2001. Some
institutions who are implementing commercial VLEs seem reasonably satisfied with their systems, particularly when
supplemented with other tools. For example an informal phone conversation with an institutional developer from
Dundee (who was not a participant in the Developers Survey) indicated that Dundee has moved to a single
institutional implementation of Blackboard supplemented by the Blackboard Content System – and that this
solution was satisfactory to the medical school.
However other schools that have had a bespoke VLE but have had to shift to a commercial system (e.g.
Birmingham Medical School) are generally critical of the commercial system and want to supplement it with
functionality from the original bespoke system. Overall there was a greater level of satisfaction with the bespoke
24 Aswell as about the appropriate terminology for naming them.

25 However the ACETS project aims to address this to an extent by developing rich descriptions of RLOs and how they can be
used in various contexts. See http://www.acets.ac.uk/node.asp?id=about.

systems’ ability to represent the curriculum compared to the commercial systems (although this difference was not
statistically significant).
But is this simply about commercial vs. bespoke? Whether a VLE is able to represent the curriculum may not be
only about the flexibility of the platform per se, but how it is implemented. Does the institution impose a rigid
structure based on the hosting Institution’s understanding of the curriculum, or can individual Faculties or
Departments create their own areas with the VLE (“Courses” in Blackboard terms) and thus find their own ways
of adapting the VLE to meet their needs?
Another factor is the recent emergence of open source VLEs, for example Moodle (which has already been
adopted at Glasgow and is being used by the Dental School), and Bodington (used by Leeds and Oxford medical
schools). These appear attractive because they can be adopted with little or no software development or licensing
costs, and because their open source nature means they can be adapted for local conditions. However the
response from Oxford suggests that even this may not suit a medical curriculum as well as a specialised bespoke
system: even though their medical faculty has been one of the primary users of the Boddington VLE at Oxford and
have therefore had considerable influence over how it has been implemented institutionally, their response to the
survey indicated that they were “somewhat dissatisfied” with their VLE’s ability to represent the curriculum.
Another of the reasons that schools have needed to develop complete systems has historically been the difficulty
of integrating separate tools together. For example the University of Bristol institutionally uses Blackboard, but the
medical faculty have for several years had their own intranet (Medici) which is structured as a mirror of the
medical curriculum in a way not possible with Blackboard. However it has been considered to be too expensive to
develop this so that it offers the range of tools and integrated environment that Blackboard provides. A proposed
solution has been to link Blackboard directly from the relevant sections of Medici. The obstacle to this has
however been that students have to log in separately each time they do this. As integration improves and true
single-sign-on is implemented the join between the two systems will become less visible and this kind of hybrid
solution will become more practical and the need for complete bespoke VLEs will be reduced. However it should
not be understood from this that tools from any provenance can be bolted together and work seamlessly. Single-
sign-on only means that all participating systems will recognise who you are, not that they will know whether or
not to let you in.
Some institutions prefer to approach this problem through the wholesale adoption of a commercial system and all
its extensions including content management and e-portfolio such as has been done at Durham with Blackboard,
on the basis that the system is already familiar to users. The potential disadvantage is that each tool provided by
the VLE creator is not necessarily the best example of its type or the most suitable for the specific purpose.
However with the opening up of APIs (e.g. Blackboard Building Blocks) and the ability to bolt in additional
functionality commercial systems can now present themselves as a complete e-learning Operating System, where
users can mix and match applications to suit local needs. For example many Universities (and several of the
schools in this survey) are using QuestionMark Perception as a specialist assessment tool integrated into
Blackboard.
Within the open-source / bespoke community this level of integration can occur using an “umbrella” system that
allows specific applications to be developed within it. An open-source product such as Moodle is an example of
this.
To conclude, it seems that the improvements in the flexibility of commercial systems mean that bespoke systems
may have lost some of their advantage, and therefore their ability to justify the cost to their school of local
development, administration, hosting, training etc. Perhaps bespoke developers now need to work even harder to
offer something not available using a central institutional VLE (see below).
22.4. As institutions have begun to adopt VLEs centrally, has there been any pressure on
schools to fall into line with institution-wide VLE strategies, and how have schools
responded to this?
This question was included in this survey in response to various comments from members of the bespoke medical
VLE community and the expectation was that the survey would find many schools using a bespoke VLE
experiencing pressure to standardise their VLE with the central institution. This situation was reported, but only
by a relatively small number of schools. Most bespoke developers claim to enjoy good relations with their hosting
institution and are optimistic that their systems will continue to coexist alongside the institutional VLE. What was
reported however, was that a number of schools who had newly implemented a VLE had adopted their host
institution’s system either because they were given no choice, or because they lacked the resources to develop
their own – suggesting that this may not have been what they would have chosen given a free choice.
The assumption made by this survey was that any pressure would be coming from the institution centrally.
However on further consideration it seems possible that pressure may equally come from within the schools
Page 57
themselves. Senior staff in schools may become aware of the existence of systems run by the institution which may
seem to offer an equivalent service to the medical school at no extra cost, thus saving the additional expense to
the school of running its own VLE 26. Bespoke systems may then have to demonstrate significant added-value in
order to justify their continued existence.
Examples of how this is being done could include assessment tools specifically developed for Medicine (for
example the system developed at Nottingham), e-portfolios again specifically designed for medicine such as those
developed at Newcastle allow students to record progress against a detailed and subject specific set of learning
objectives, and finally as suggested in 22.3 Given the rapid evolution of commercial VLEs, is there still an overriding
case for the continued use and development of bespoke medical VLEs? on page 56 above bespoke systems are still
better at representing the curriculum.
22.5. How has usage of VLEs by staff and students grown and developed? How much
demand does this place on IT infrastructure?
In 2001 the impression was that VLE uptake was growing but was patchy and uneven. In 2004 the picture seems to
suggest that usage has grown a great deal since 2001 (although it has been hard to collect quantifiable evidence of
this) and that VLEs are becoming much more deeply embedded into everyday use. The evidence is that by that
measure VLEs are a success and have an important place in medical education. With that success of course comes
extra demands for support and hardware infrastructure.
This survey has not managed to find out much about the hardware infrastructure implications of this growth. Some
information was gathered about the type of servers used to run the VLEs but little about hardware / software
specifications. However it has gathered detailed data on the teams in place to support VLEs, although since the
same data was not collected systematically in the 2001 survey little can be said about how these have changed.
Support teams seem to vary widely in terms of their size, the mix of skills they contain and how they are
distributed (i.e. whether they are all located in the medical school – within a single unit or various units – or all
located in the institution centrally).
The impression is that at sites using a centrally run commercial system maintenance of hardware and software and
user training is done centrally while development of materials is done within the school, whereas at schools
developing their own system some run their own servers themselves but other have them supported centrally, but
also provide their own user training as well as materials development. Another impression is that new medical
schools, many of whom have decided to make e-learning a central part of their curriculum delivery, have installed
substantial support teams from the word go, whereas some older schools have more of a struggle to make the
case for adequate support and development staff.
Even though the information presented in this report is quite detailed, what is has not addressed is which model is
most effective. What is the most effective mix of skills? How does this correspond to the e-learning and
curriculum development strategies of the school?
What is clear is that some schools have quite large teams while in other institutions the whole operation is the
work of one dedicated individual – and in many cases there is a strong feeling that the support team is too small
and in some cases carried out by clinicians for whom this can hardly represent the best use of their time. The issue
of support teams also impacts closely on the school-based bespoke vs. institutional VLE debate. How much
additional support is required for a bespoke VLE, and how much does this duplicate work that can be done more
efficiently from the centre? 27
22.6. In what ways are VLEs most successful in supporting students and teachers
located at disparate clinical and educational sites?
An issue identified by a large number of schools was the limited amount of control they have over the IT
infrastructure at the sites where many of their users are located, thus also restricting to some extent what they
are able to deliver.
An approach to this issue that has been adopted at the University of Bristol has been to use the Citrix thin client
technology. This allows a virtual desktop to be run on any client PC where all the applications and what appears
on the PC desktop is actually being delivered directly from a server based centrally at the school. When used for
non-medical school business the PCs can use their normal local desktop and applications. This allows complete
26 Although Figure 20 could be interpreted as suggesting that bespoke systems have equal or lower support and development
costs than their commercial equivalents.
27 See footnote 25 on previous page.

control including software and updates to be done centrally and to be done once for all PCs at all participating
sites. However one major limitation encountered has been that this system is very inefficient at delivering video; it
requires much larger network capacity than standard networked video delivery and is unable to use the latest
compression technologies.
The emphasis placed by the small number of respondents to this question in the paper-based questionnaire was
that that benefit of the VLE was that it improved communication between the medical school and students and
staff located at a distance. However this is quite a broad comment and requires much more explanation to
become truly illustrative. What is meant by communication? Between whom and in which direction?
Communication about what and for what purpose? And how does improved communication impact on the
experiences of learners and teachers? The 2001 report suggested that VLEs were used to communicate: 1) from
the medical school to both students and teachers, relatively stable information about timetabling, curriculum,
rotations, locations etc. and 2) using email tools within the VLE to communicate from the centre to students, fast
changing information such as lecture cancellations. There was less emphasis on communication from students –
and findings from the current survey indicate that this function of communication as information provision is still
the main purpose of VLEs.
However the 2004 findings are that most schools now use discussion boards, which are designed for asynchronous
2-way communication. Does this mean that communication is being used to engender “on-line communities” of
learners or teachers that facilitate cohesion among staff and a sense of ownership and shared goals or a reduction
in feelings of isolation for students? At the Twenty-Twenty Vision meeting there were concerns expressed that
communication facilities are often not used, or used inappropriately. However evidence from Edinburgh suggests
that what may be considered inappropriate use (i.e. for social purposes rather than academic) is in fact what gets
students using discussion boards – but once there they then start to use them for academic purposes.
22.7. Has there been any shift towards using VLEs as a platform for on-line learning
rather than as a means of delivering more traditional documentation?
Perhaps the assumption behind this question is that such a shift is both desirable and a natural part of the evolution
of VLEs. But is it? The overall picture remains that the key role of VLEs in medicine is to deliver traditional course
documentation. While some respondents embrace e-learning activities with enthusiasm – others doubt their value
and are mindful of the cost and effort involved in creating materials.
Across higher education as a whole it is arguable that the use of VLEs to deliver on-line learning activities is more
appropriate for distance learners than students based on campus, who can take part in learning activities through
face-to-face contact with their teachers and peers. For them the benefit of a VLE is mainly through electronic
distribution of documentation. To some extent and for some of their course medical students could be described
as distance learners. However it is also arguable that during this time their primary need is still for a system that
delivers the functions traditionally associated with bespoke medical VLEs, such as on-line documentation,
administrative information and curriculum description because:
• The curriculum is complex and needs to be represented in a way that enables learners and teachers to
comprehend it.
• Clinical placements and other clinically based learning requires a good deal of supporting administrative
documentation.
• Medicine etc. is a competency based profession requiring comprehensive coverage of a detailed set of learning
objectives.
The case for on-line learning activities may depend partly on the model of placement practice. Much of the actual
learning still takes place at the bedside and involves practical skills that cannot be taught by e-learning. However, if
while on placement students are taught underpinning knowledge as well as clinical skills, then on-line learning
activities become more relevant. If this is done by clinical teaching staff at the placement then support can be
provided from the centre in order to ensure that students receive a standard quality of learning experience.
e-learning activities can also be appropriate to some extent to fill in gaps in students’ clinical experience. However
where the purpose of clinical placement is exclusively for practical learning, then it may be more appropriate for a
VLE to remain simply the means to deliver course information or other materials designed to be printed out.
Lastly, although the picture is not consistent, the overall trend is for medical VLEs to be used more heavily in the
first two years of the course, when students are not normally on clinical placement and e-learning activities are
presumably seen as supplementary to their face-to-face learning rather than fundamental to it.
Page 59
22.8. Have the VLEs started to have any effect on learning, teaching and assessment
practices?
As stated at the beginning, it was felt that this question was hard to answer within the scope of this study, and
consequently little data has been collected. In 2001 most respondents felt it was too early to say what effect VLEs
were having in this respect. We would suggest that the time is now ripe for some research in this area. It seems
important to know how far VLEs and the ways in which they are used are successful in promoting teaching and
learning practice which we believe to be pedagogically effective, or whether they simply make it more convenient
to do the same things, or even have a detrimental effect. Do they simply affect how learners learn and how
conveniently, or do they also influence what they learn, and as a result what kind of doctors, dentists and vets they
help to create?
Work that has been done so far has mainly focussed on measuring educational impact of VLEs in terms of exam
results, and found little significant difference. Where attention has not been paid is to the broader effect on
learning. Do VLEs simply improve the convenience of learning, or do they also influence what students learn in a
wider sense? Claims are made that VLEs can promote ways of learning that will have an impact on students’ later
professional practice; that they foster greater learner independence and facilitate the development of communities
of practice. These claims need to be tested so that we can see what kind of doctors, dentists and vets VLEs help to
create, and consequently understand their impact upon the quality of patient care.
23. Limitations of this survey

In comparison with the 2001 report, this survey has been necessarily restricted in scope (it represents 10 days
effort compared to 26 days in 2001). At least partly as a result, this report suffers from a number of acknowledged
limitations:
• The vast bulk of the results represented in this report come from the two on-line surveys. This differs greatly
from the 2001 report which mostly relied on site visits and interviews, along with hands-on reviews of the
VLEs. While this has the advantage of getting a greater number of relatively detailed responses, it lacks the
depth and sense of narrative afforded by personal interviews. It also relies on the accuracy of spontaneous
responses given to an electronic questionnaire by hard-pressed staff.
• The reasons for using two separate questionnaires are complex, and it is acknowledged that it may lead to
some inconsistencies. Underlying this is the danger of relying on individual respondents to accurately represent
the situation across a whole school – a problem that could only be avoided by carrying out an in-depth survey
at each school. Where the institution is represented by two individuals as in this report, any inconsistencies or
apparent contradictions between them serve to draw attention to this weakness. The results may then appear
less accurate than if the institution is represented by a single individual, although in reality this is not the case.
The main reason for using two surveys was that the two groups were thought to have detailed knowledge of
separate areas:
• the academics were assumed to have more first-hand experience of issues related to the use of VLEs for
teaching and learning, the reasons for their use and their effect on practice (covered by questions 6-8 listed in
section 3 above). They were also expected to have an overview of the role that the VLE had come to play
within the institution in terms of e-learning activities used and how embedded the VLE is in everyday practice.
• the developers were assumed to have more detailed knowledge of the features of their VLEs, including
technical issues such as implementation of standards, as well as access to detailed quantitative records of usage,
and detailed knowledge of how the system content is structured.
Furthermore a list of “academics” was available covering all UK medical, dental and veterinary schools, whereas
the developers list was more patchy. Thus another reason for the two surveys was to address the at times
incompatible imperatives for both breadth and depth.
• Unlike the 2001 report, the 2004 survey did not include a question about obstacles to VLE implementation – as
one respondent pointed out. This was because it was felt that the obstacles were unlikely to have changed
since 2001 – although in retrospect this could be fruitfully called into question.
• The paper-based questionnaire did elicit a small number of very illuminating responses to the questions about
VLEs’ impact on learning and teaching and supporting students on placement. However it had been agreed that
these questions were hard to research properly and should therefore not be a priority for this survey. In
retrospect however it may have been worthwhile including some of the statements from the paper-based
questionnaire in the Academics Survey in order to gauge their applicability to the sector as a whole.

• There has not been time to carry out a literature review or include any references to support the points made
in the discussion.
Page 61
Recommendations
24. Hosting institutions

Where an institution’s medical / dental / veterinary school is using the main institutional VLE, control of the VLE
structure should be devolved as far as possible to the schools to allow schools to modify the VLE for their needs.
25. Medical, dental and veterinary schools

Schools (or hosting institutions) should audit their local support teams and evaluate their suitability for
implementing their e-learning / VLE strategy. Results should be published in order for the community as a whole to
develop models of effective support.
Schools relying on single individuals to develop / support their VLE should consider an expansion of their support
teams and acquiring staff with relevant specialist skills.
Schools are able to share developments and some sharing and exchanging good practice has taken place,
particularly when collaboration is facilitated by external funding such as FDTL or JISC. Sharing can be as effective
between same-course in different institutions, or different-courses in the same institution.
Schools considering their future VLE strategy should keep an eye on developments towards integration and
interoperability and consider the potential benefits of a hybrid solution rather than a single all-encompassing VLE.
26. All developers

Developers (particularly those working with or considering hybrid solutions) should prioritise the implementation
of true single-sign-on where this is available locally.
Continue to work to allow modular integration between applications to allow greater flexibility, rather than
development of all-encompassing VLEs.
Systems should be developed so that they are linkable at quite a deep level and integrated so that the user is
unaware that they may be using different systems.
Developers should analyse the tool integration models that are effective and be willing to share these with the
development community.
27. Developers of bespoke systems

Continue to add value to medical VLEs through development of medical education specific applications and tools.
Work towards further development of personalised learning environments.
Demonstrate in detail how bespoke VLEs can reflect a range of responding to the needs of the curriculum better
than a commercial or open source VLE could.
28. Funders / stakeholders

Evaluation work should be funded into:
• How successful sharing of content has been in terms of how shared content is used and which kind of content
is most popular.
• More specifically, how are RLOs used by different institutions. To what extent are RLOs used across
institutions? How far can they be designed to facilitate local modification?
• Furthermore feasibility studies should be carried out into sharing of learning processes.

Further research should be funded into:

• How communication tools have helped to support students on placement and engender an on-line community.
This would best be done as a set of case studies.
• The true impact of VLEs on the practice of medical education and on the experience of students and staff.
• Give further consideration to the type of RLOs that could be shared.
• Developing models of VLE / e-learning support in medical schools and their hosting institutions, and evaluating
the effectiveness of these.
• Finally and most importantly, research is urgently needed into the true impact of VLEs on the practice of
medical education and on the experience of students and staff.
Page 63
Appendices
29. Appendix One: VLE types used by individual schools as

main VLE
29.1. Bespoke VLE (either developed locally or elsewhere)28
Aberdeen
Birmingham Dental School
Cambridge
Edinburgh
Glasgow
GKT School of Medicine
Manchester
Newcastle
Nottingham
Southampton
Sheffield
29.2. Bodington Open Source VLE
Leeds
Oxford
29.3. Blackboard29
Bristol Medical School

Bristol Veterinary School
Durham
Liverpool
PMS
RVC
29.4. WebCT
Birmingham Medical School

UCL, London
29.5. FD Learning's LE
LWMS
28 A bespoke VLE is also used by a further 1 school who responded anonymously.

29 Blackboard is also used by a further 3 schools who responded anonymously.

30. Appendix Two: How bespoke VLEs have developed since

2001
Personalisation features adopted or planned
GKT School of
Dental School
Southampton
Anonymous
Nottingham
Birmingham
Manchester
Cambridge
Newcastle
Edinburgh
Aberdeen
Medicine
Glasgow
Sheffield
Oxford
Leeds
User group portals
My Timetable etc.
Integration with
personal and
academic records
e-portfolio
Intelligent
recommendation
User specified
pathways
adopt No
ed plan plan
had in since to to
Key: 2001 2001 adopt adopt
Table 2. Personalisation features adopted (or planned). Source: Developers Survey (n=14).
Page 65
Other features adopted or planned
GKT School of
Dental School
Southampton
Anonymous
Nottingham
Birmingham
Manchester
Cambridge
Newcastle
Edinburgh
Aberdeen
Medicine
Glasgow
Sheffield
Oxford
Leeds
Author uploading
Assessment tools
2-way
communication
Student survey tool
Separate authoring
interfaces
MeSH indexing
MeSH pull-down
lists or tree
RSS feeds
More video
adopt No
ed plan plan
had in since to to
Key: 2001 2001 adopt adopt
Table 3. Other features adopted (or planned). Source: Developers Survey (n=14).

31. Appendix Three: Full system usage data - 2004
School No. of No. of Hits per No. of No. of Pages

students students year sessions hits per viewed
in school used the per year day per day
system
Newcastle 1,400 1,400 12,000,000 230,000 20,000- 33,000
(logins) 50,000
Edinburgh 1,900 1,900 30,000+
Nottingham ~1,100 most 17,000,000 160,000 90,000 27,000
Oxford 900 – 1,000 all 1000 1000
Aberdeen 1,000 1565 51 in the
registered last 24h
users (out of
term)
GKT School of 2,000 2,000 17,000,000 4,000 4,000
Medicine
Cambridge 475 (across 465 2,400,000 17,000 per 6500 1700
3 years) intake
year
(logins)
Sheffield 1,300 1,000 – 1,300 476,000
since
12.09.0330
Birmingham Dental 300 300 130,000 12,000 360 360
School
Birmingham 2,000 2,000 >12,000,00 1M/30
Medical School 0 (1M per
month on
average)
Table 4. Approximate usage data of the VLE systems in place. Source: Developers Survey (n=10).
30 “this is portal homepage only, therefore not true reflection”.
Page 67
32. Appendix Four: Server Descriptions
Institution Server descriptions

Aberdeen Solaris, Apache, PHP, MySQL
Birmingham Dental School Windows IIS. ASP & Access databases.
Birmingham Medical School We've effectively got two separate systems. WebCT is hosted, managed and
run centrally. Our own system runs on two servers, a web server for static
files and an applications / CMS server for dynamic content.
Bristol Medical School Blackboard: don't know as is run by the institution medicine Intranet / web
site (Medici: Linux / Zope and Sun / Oracle) with dynamic links to central
data from institutional systems (data hub) developing multimedia store: Linux
(Redhat?) and Zope (I think)
Cambridge Clinical School Windows 2003 Server, IIS, SQL Server, Access, ASP, ASP.NET
Durham Linux Red Hat AS Series of Twin processor HP boxes set up as load
balanced using round robin
Edinburgh 2 x Win 2k web / app servers and 1 Win 2003 database server
Glasgow Windows 2000 server. We are in the process of moving to 3 servers: 1. a
'live' server 2. a backup server 3. a development server
GKT School of Medicine Windows 2003 IIS 6 SQL Server 2000
Leeds Not known - run by institution
Liverpool Centrally run for us
LWMS 3 Compaq proliant rack servers PIII, 1-2GB SDRAM, RAID
Manchester ASP.NET SQL Server, IIS
Newcastle SunFire V20 dual Opteron (cluster) using load balanced Zope with dedicated
management, MySQL and development nodes.
Nottingham Sun Enterprise 420R (2x 450MHz CPUs) running the VLE using
Apache / MySQL / PHP. Sun Fire V240 (2x 1GHz CPUs) with SSL
accelerator dedicated to formative / summative on-line assessment
(Apache / MySQL / PHP). Removed Zope in 2004 due to performance
issues, memory leaks and lack of institutional support.
Oxford apache, PostgreSQL, Bodington
PMS W2K plus SQL Server, and IIS, Up to this year single server. From Sept two
servers one running the database, the other the web app.
RF & UC, UCL Unknown. WebCT is managed at an institutional level.
The Royal Veterinary College We currently have a single dedicated server running Windows 2000 which
hosts the Blackboard VLE. However, we are now moving to a shared dual
server based at the London School of Hygiene and Tropical Medicine which
will also run Windows 2000. We also have a dedicated server running
QuestionMark Perception.
Sheffield Hosted at Newcastle, don't know full spec. Looking at bringing it to Sheffield
next year, dependant on central!
Southampton 1 web server, 1 file server running MS Server 2003; 1 SQL server. Our MLE
is also the basis for the interfaculty interprofessional education project,
involving 10 schools / departments across the universities of Southampton
and Portsmouth. All are integrated into our underlying database systems.
Table 5. Server specifications. Source: Developers Survey (n=21).

33. Appendix Five: Interoperability technologies implemented in

VLEs
Interoperability technologies implemented in VLEs31
Birmingham Medical School

Birmingham Dental School
Bristol Veterinary School
GKT School of Medicine
Bristol Medical School
Southampton
Anonymous
Nottingham
Manchester
Cambridge
Newcastle
Edinburgh
Aberdeen
Liverpool
Glasgow
Sheffield
Durham
Oxford
LWMS
Leeds
RVC
UCL
PMS
RSS
XML
SCORM
IMS
IMS QTI
MeSH
LOM
Other*
Table 6. Source: Developers Survey (n=23).

*’other’ was not clarified in the Questionnaire, it may refer to IMS LIP, READ or other taxonomy codes, etc.
31 Please
note that in all tables in Appendix Five: Interoperability technologies implemented in VLEs on page 69 below and
Appendix Six: Breakdown of e-learning activities undertaken to any significant extent at UK medical, dental and veterinary
schools on page 70 below the colours have no significance other than to facilitate ease of viewing and comparison across
columns.
Page 69
34. Appendix Six: Breakdown of e-learning activities undertaken

to any significant extent at UK medical, dental and veterinary
schools
e-learning activities: Dental Schools
GKT School of
Bristol Medical
Anonymous
Birmingham
Newcastle
Medicine
Sheffield
Dundee
Cardiff
School
Dublin
UCL
Live lectures
On demand lectures
Other on demand video
Chat
Asynchronous discussion
Interactive tutorials
e-portfolios
Patient cases
Signing up on-line
Managing outcomes
Course literature
Formative assessment
Summative assessment
Other
Table 7. e-learning activities at UK Dental Schools. Source: Academics Survey (n=10).

e-learning activities: Medical Schools
GKT School of
Anonymous
Anonymous
Anonymous
Anonymous
Anonymous
Anonymous
Anonymous
Nottingham
Birmingham
Manchester
Aberdeen
Leicester
Medicine
Glasgow
Sheffield
Durham
Dundee
LSHTM
HYMS
SGUL
Leeds
Keele
BSMS
QUB
UCL
UEA
Live lectures
On demand
lectures
Other on
demand video
Chat
Asynchronous
discussion
Interactive
tutorials
e-portfolios
Patient cases
Signing up on-
line
Managing
outcomes
Course
literature
Formative
assessment
Summative
assessment
Other
Table 8. e-learning activities at UK Medical Schools. Source: Academics Survey (n=25).
Page 71
e-learning activities: Veterinary Schools
Cambridge
Veterinary
Edinburgh
Liverpool
Glasgow
School
Dublin
Bristol
RVC
Live lectures
On demand lectures
Other on demand video
Chat
Asynchronous discussion
Interactive tutorials
e-portfolios
Patient cases
Signing up on-line
Managing outcomes
Course literature
Formative assessment
Summative assessment
Other
Table 9. e-learning activities at UK Veterinary Schools. Source: Academics Survey (n=7).
35. Appendix Seven: On-line academics survey

To: Subject Centre nominated primary contacts (NPCs) in medicine, dentistry and veterinary medicine.
This survey has been commissioned by the Subject Centre to get an up-to-date picture of how Virtual and
Managed Learning Environments (VMLEs) support medical, dental and veterinary education in the UK. It is an
update of a survey carried out in 2001 (which was funded by the JISC under their JTAP programme). You may
remember that this survey was discussed with participants at the Subject Centre event "Twenty-Twenty Vision:
external clinical placements".
This questionnaire focuses on the strategic implementation of a VMLE, and the context for its use (not the
technical issues which are being addressed in a separate survey). As NPC we anticipate that you are the person
most likely to have an overview, however, if that is not the case please forward to an appropriate colleague.
We would be very grateful if you would complete the questionnaire by Monday 27 September. It consists of 16
mainly multiple choice questions and should take no more than 10 - 15 minutes to complete. If you have to pause,
you will be bought back to where you left off when you return. Thank you.
Note: Throughout this questionnaire both 'VLE' and 'VMLE' are used. These are intended to be read
interchangeably for the purpose of this survey.
Copy of Virtual and Managed Learning Environments in UK Medical Education - Academics
2. e-learning activities.
This page focuses on e-learning in general.
1. Please select all of the following e-learning activities taking place to any significant extent at your School / Faculty:
Broadcast of live lectures to sites off-campus.
On demand delivery of digitally recorded lectures.

On demand delivery of other video material.

Live real-time online text communication (chat).
Asynchronous discussion (bulletin boards).
Interactive tutorials.
Electronic student portfolios / personal learning records.
Online presentation of patient cases.
Signing up online to clinical teaching and/or student selected components.
Managing outcomes.
Online management of course literature such as programme / module / study guides.
Online formative assessment.
Online summative assessment.
Other (please specify).
Copy of Virtual and Managed Learning Environments in UK Medical Education – Academics.
3. Focussing on your VMLE and how it is used
One of the aims of this survey is to update the JTAP-623 survey that was carried out in
2001. The following questions are intended to enable direct comparison between 2001 and
2004.
2. Which of the following best describes the extent to which your VMLE is integrated into normal working practices in your
School.
Although we are using a range of e-learning applications we are not bringing them together into a VMLE.
We have only just started using a VMLE.
We are really just beginning to get our VMLE incorporated into our local teaching and learning culture.
Our VMLE is used by a lot of our staff and students but there is still some way to go before they are all on board.
Our VMLE is now used as a matter of course by most of our staff and students.
If you are not using a VLE at your school / faculty, please skip questions 4, 5 and 6 below
and go straight to the next page.
3. From the list below, choose what you consider to be the FOUR main drivers for your use of a VMLE and indicate their
relative order of importance, by selecting a number 1 - 4 (where 1 is most important). To omit a driver from your list simply
leave blank.
Importance 1 – 4.
Need to support students studying at a distance from the institution 1234.
Managing large numbers of teachers 1234.
Accommodating increased student numbers 1234.
Managing changes in the curriculum (problem-based, case studies etc.) 1234.
Managing varied intakes of students (accelerated entry, part-time students etc.) 1234.
Managing complexity of the curriculum 1234.
Demand from staff / students 1234.
Need to keep up with the rest of the sector 1234.
Achieving vertical and horizontal integration of learning resource across the curriculum 1234.
Other (please specify below) 1234.
Page 73
4. If you chose Other for the previous question, please specify here:
5. From the list below, please choose what you consider to be the THREE key function(s) delivered by your VMLE and
indicate their relative order of importance, by selecting a number 1 - 3 (where 1 is most important). To omit a function from
your list simply leave blank.
Importance 1 - 3
Administration of the curriculum (e.g. timetables, clinical attachments, announcements etc.) 123.
Information about course (study guides, curriculum structure etc.) 123.
Supporting materials for teaching integrated into the curriculum structure (lecture notes, timetabled online
tutorials, revision etc.) 123.
Learning resources not integrated into the curriculum but designed as optional extras (links to web sites, CBL
tutorials, quizzes etc.) 123.
Facilitate curriculum design and revision 123.
Other (please specify below) 123.
6. If you chose Other for the previous question, please specify here:
7. If you have any further comments about any of the above, please add them here:
We are also interested to find out how school / faculties are collecting feedback from their students and to what
extent they are doing this electronically.
8. In what format do you collect student feedback on your teaching programme (please select all that apply)
Electronically.
On paper.
Orally (focus groups etc.).
Students can choose paper or electronic.
9. Is completion of surveys:
Optional?
Mandatory?
10. If completion is optional, what strategies do you use, if any, to increase response rates?
Timetabled feedback sessions.
Personalised reminders.
Prize draws for all respondents.
Award for highest responding group.
To say "While not compulsory, we know whether you've responded or not".
11. If you are getting student feedback via electronic surveys, approximately what proportion of students complete these on
average (%)?
12. How are surveys co-ordinated? By:

Unit leaders.
A single person in the School?
Unit administrator.
IT support staff.
Support staff in central institution.

13. If you have any further comments about online student feedback surveys at your school / faculty, please add them here:
Finally we would like to know about the role of educational developers in your
school / faculty VLE, and how they are contributing to curriculum change and e-learning
development.
14. Please indicate the type(s) of educational development staff you have working in your school / faculty by choosing from
the list below. (If you have staff in more than one role please choose both roles).
None
Educational Technologist.
Curriculum Translator (subject expert seconded to fulfil educational development role).
Medical Educationalist (expert in education rather than medicine / dentistry / veterinary science).
15. Please indicate all the roles / activities that are performed by your educational development staff:
Develop our own VMLE.
Provide technical support for our VMLE.
Provide administrative support for our VMLE (enrolling users etc.).
Develop interactive course material.
Put existing paper-based material online.
Evaluate student needs.
Provide face to face teaching to plug gaps in provision provided by main teaching staff.
Provide staff development / training.
16. Please indicate how your educational developers are allocated across the curriculum:
One per year of the programme.
One per unit.
As needed across the curriculum.
17. If you have any comments about the role of educational developers at your school / faculty, please add them here:
Although it is not essential for us to identify you by name if you would prefer to remain
anonymous, it would be invaluable to know which institution you represent so that we can
ensure maximum coverage and compare institutions of a similar type. Thank you.
18. Please enter (optional):

Your name:
Your email:
Your job title:
Your institution:
36. Appendix Eight: On-line developers survey

This survey has been commissioned by the Subject Centre to get an up-to-date picture of how Virtual and
Managed Learning Environments (VMLEs) support medical, dental and veterinary education in the UK. It is an
update of a survey carried out in 2001 (which was funded by the JISC under their JTAP programme).
Page 75
This questionnaire focuses on the more technical aspects and is aimed at VMLE developers in your institutions.
We anticipate that you are the person best placed in your school / faculty to address the questions in this survey.
However if that is not the case please forward the survey URL to an appropriate colleague. We are also sending a
parallel questionnaire to a senior academic on the institutional implementation of a VMLE and the context for its
use.
We would be very grateful if you would complete the questionnaire by Monday 27 September. It consists mainly of
multiple choice questions and should take no more than about 20 - 25 minutes to complete. If you have to pause,
you will be bought back to where you left off when you return. Thank you.
Notes:!
1. Throughout this survey both 'VLE' and 'VMLE' are used which are intended to be read interchangeably for the
purpose of this survey.
2. You may note that some of the page and question numbering is not sequential. This is because there are various
paths through the questionnaire depending on your answers.
1. What VLE are you using? (if you are using more than one, please select your main VLE)
Commercial VLE (e.g. Blackboard, WebCT).
Bespoke VLE developed at this Institution.
Bespoke VLE developed elsewhere but modified at this institution.
Bespoke VLE developed elsewhere with no significant local modifications.
None.
2. If you are using more than one VLE, please indicate any others you are using (otherwise skip)
Commercial VLE (e.g. Blackboard, WebCT).
Bespoke VLE developed at this Institution.
Bespoke VLE developed elsewhere but modified at this institution.
Bespoke VLE developed elsewhere with no significant local modifications.
None.
3. How does this compare to your situation in 2001?

a) We weren't using a VMLE in 2001.
b) No change.
c) Using a different VMLE.
d) We were only using one VMLE in 2001 and now we're using more than one.
e) Don't know / I wasn't here then.
4. Which of these are you also using to supplement your VLE (where they are NOT an integral part of the basic VLE)?
(select all that apply)
Content management system.
e-portfolio / Personal Academic Record.
Specialist assessment tool (e.g. QuestionMark Perception).
Courseware authoring tools.
Online student survey tool for student feedback.
Discussion board / chat facilities.
Optical mark reader.

5. If any of the tools selected above is a COMMERCIALLY AVAILABLE APPLICATION, please write the name of the system in
the appropriate box below (otherwise leave blank):
Content management system.
e-portfolio / Personal Academic Record.
Specialist assessment tool (e.g. QuestionMark Perception).
Courseware authoring tools.
Student survey tool for student feedback.
Discussion board / chat facilities.
Optical mark reader.
Other (please specify):
6. If you wish to add any comments or qualifications, please do so here.
7. What are the main reasons for your continued choice of a bespoke VMLE in preference to a commercial system? (select
all that apply)
Legacy.
Still best way of reflecting curriculum.
Cost.
Deficiencies of commercial systems.
8. In the 2001 survey the main reason that respondents gave for developing their own VLEs was that commercial “off the
shelf” VLEs were unsuitable for medical schools because they were unable to represent the complexity of the medical
curriculum. How much do you now agree with this view?
Completely agree.
Mostly agree.
Unsure.
Mostly disagree.
Completely disagree.
9. A number of respondents also said that each institution would need its own VLE: even a standard VLE specifically for
medicine was not ideal because of variations in curricula between medical schools. How much do you now agree with this
view?
Completely agree.
Mostly agree.
Unsure.
Mostly disagree.
Completely disagree.
10. How satisfied are you with the way that your VLE represents the structure and content of your curriculum?
Wholly satisfied.
Mostly satisfied.
Partly satisfied.
Somewhat dissatisfied.
Very dissatisfied.
Page 77
In recent years there have been reports of HE institutions i.e. Universities trying to impose
the use of a single VLE across the institution, whereas historically many medical schools
within those institutions have developed their own system.
16. How does your school / faculty VMLE relate to any VLEs being used by your hosting institution?
The hosting institution is using:
The same bespoke VLE as ours.
A commercial VLE.
A different bespoke VLE to ours.
Both a commercial and bespoke VLE side by side.
17. If you are using a separate VLE, what is your hosting Institution’s attitude to this:
Applying pressure to adopt main institution VLE.
Happy for two systems to coexist.
Tolerant for now but future pressure to standardise is likely.
Institution interested in adopting an extension of medical school VLE.
Not applicable.
18. If you are using a separate VLE, what do you see as the future relationship between the 2 systems over the next 3 years
in your medical school?
Both will continue to coexist.
Bespoke will be discontinued in favour of commercial.
Commercial will be discontinued in favour of bespoke.
Both will be discontinued.
Don't know.
19. Please write here if you would like to add any further comments on the issues above
20. In the 2001 report, developers described a number of features which they intended to add to their systems in the near
future for making them more personalised. Some of these are listed below. For each feature, please indicate whether this
was something that you:
i) had in 2001.
ii) adopted since 2001.
iii) plan to adopt.
iv) have no plan to adopt.
Separate portals for teaching staff, management and students with interface relevant to each group.
Personalised tools such as “My Timetable”, “My Calculator”, “My Calendar”.
Integration of VMLE with a Personal Academic Record System (PARS).
An online learning portfolio management system.
Intelligent analysis of user's learning style to recommend further suitable resources, in the style of Amazon’s
“customers who bought book 'A' also bought books 'B' and 'C' and CD-ROM 'D'”.
User specified individual pathways through materials.
21. In the 2001 report, developers described a number of other features which they intended to add to their systems in the
near future. Some of these are listed below. For each feature, please indicate whether this was something that you:
i) had in 2001.

ii) adopted since 2001.

iii) plan to adopt.
iv) no plan to adopt.
Author uploading facilities.
Assessment tools.
Two-way communication tools (Discussion board / Chat room).
Student survey tool for student feedback.
Separate interfaces for authoring different types of materials such as tutorials and MCQs.
Use of MeSH headings for indexing and classification of resources.
Enhanced use of MeSH allowing users to select headings from pull-down list or expandable tree.
Incorporation of RSS feeds.
Increased incorporation of video resources.
22. If you would like to mention any other relevant features not in the lists above, please do so here.
23. Which of these features does your system offer for managing students' clinical placements?
Record student hospital / GP etc. placements.
Track student hospital / GP etc. placements.
Automatically allocate student hospital / GP etc. placements according to a set of criteria.
Allow students to choose their hospital / GP etc. placements.
Allow staff to manually allocate student hospital / GP etc. placements.
None of these.
Other.
24. To what extent is your VLE integrated with other institutional IT systems?
Single sign on allows VLE to be accessed with same login details as other institutional systems, but users have to
log in separately to each system.
Single sign on can pass login information between systems so the user only has to login once to access all
participating systems.
VLE is integrated with Athens systems.
Students and staff can use all features of the library system from within the VLE interface.
Staff can build reading lists etc. directly from within the VLE interface.
System integrates with student record system to record assessment scores.
Student details can be taken directly from student record system and displayed within the VLE.
Details of student membership of units / modules / groups can be taken from student record system and used
within the VLE.
25. What level of priority is it for you to share your system code with other systems?
Top priority.
High priority.
Medium priority.
Low priority.
This is not on our agenda.
Page 79
26. To what extent have you shared system code with other institutions?
A great deal.
Somewhat.
Not very much.
Not at all.
27. If you are sharing / intending to share system code, what ways are you using / planning to achieve this?
Systems will be made available as open source so that other institutions can use components of it.
Systems will be made more modular which will enable other institutions to adopt parts of it.
We are developing and/or using shared components written using an Application Program Interface (API) such as
Blackboard Building Blocks.
We are involved with (an)other institution(s) in a formal project to share system components and tools.
28. Please select all of the following technologies and standards that have been implemented in your system:
RSS external feeds.
XML.
SCORM.
IMS standards.
IMS QTI standard for questions.
MeSH indexing system for standardised indexing of resources.
Reusable learning objects.
29. What level of priority do you give to making your content readable by other systems at other institutions?
Top priority.
High priority.
Medium priority.
Low priority.
This is not on our agenda.
30. To what extent have you shared content with other institutions?
A great deal.
Somewhat.
Not very much.
Not at all.
Assessment questions.
Interactive learning materials.
Documentation.
Images.
Reusable learning objects.
Other.

31. If you chose Other from the previous question, please specify here. Also if have any other comments on
Interoperability / Sharing please add them here.
32. For each of the roles listed below, please indicate the number of staff you have working in support of your school's
VMLE. If you have staff performing more than one of these roles, please enter them against their main role only. In the left
hand column, please select the approximate total FTEs available to your school / faculty whether provided by the institution
centrally or by the school / faculty. In the right hand column please indicate whether this is provided by the institution
centrally or by the school / faculty.
The options for each of the following asked for the proportion of FTEs available, and who they were provided by.
Web development.
Multimedia development (incl. video / audio).
Educational development.
Server / site maintenance and support.
User training (staff and/or students).
VLE administration.
e-learning development.
Other.
FTEs:
0
0.1 - 0.9
1 - 1.9
2 - 2.9
3 - 3.9
4 or more.
Provided by:
The institution.
The school.
Both.
N/A.
33. If you chose Other for the previous question, please describe their role here:
34. Please describe here briefly the server configuration you use to run your VMLE
35. Would you say that your user group's ability to take full advantage of your VLE is constrained by IT infrastructure in
hospitals where your students and teachers are located?
Yes very much.
Yes somewhat.
Not sure.
Not very much.
Not at all.
36. What is your school / faculty's role with respect to IT facilities in the NHS Trusts / Hospitals where your students are on
placement?
We provide and support infrastructure.
We provide infrastructure but the Trusts support it.
We support infrastructure provided by the Trusts.
Page 81
We recommend infrastructure specifications.

We have little or no involvement.
We would like to get a sense of how much VMLEs are being used by staff and students.
37. Please complete as many of the boxes as possible below using the most recent data you have available. We appreciate
that you may not have this data readily available but even an informed guess would be very helpful.
No. of students in your school.
No. of students who have used the system.
Years using most heavily (e.g. 1st, 2nd etc.).
No. of hits per year.
No. of sessions per year.
No. of hits per day.
Pages viewed per day.
38. How has the level of use of your VMLE(s) developed over the last 3 years?
Increased massively.
Increased significantly.
Increased slightly.
Remained stable.
Decreased slightly.
Decreased significantly.
Don't know / not applicable.
Although it is not essential for us to identify you by name if you would prefer to remain
anonymous, it would be invaluable to know which institution you represent so that we can
ensure maximum coverage and compare institutions of a similar type. Thank you.
41. Please enter:

Your institution:
Your name (optional):
Your email (optional):
Your job title (optional):
37. Appendix Nine: Glossary

‘Academics’ Academic and clinical teaching staff; curriculum managers
‘Developers’ Curriculum support staff
Access Commercial database software.
ACETS exemplify and share.
API Application program interface.
ASP Active server pages.
CAA Computer Assisted Assessment
CAL Computer Assisted Learning (or Computer Aided Learning)
CD ROM Compact disc read only memory.

Citrix Remote access thin client solution.

CMIS Commercial timetabling software.
CMS Content Management System
ColdFusion A scripting language used for interfacing database (.cmf).
CPD Continuing Professional Development
DEL Department for Employment and Learning (Northern Ireland) (http://www.delni.gov.uk/)
e-learning Electronic or on-line learning.
EMQ Extended matching questions
F2f Face-to-face
FD Learning Commercial learning environment (http://www.fdlearning.com/) (formerly Fretwell Downing
Learning Environment), now Trial Technology (http://www.tribaltechnology.co.uk/)
FDTL Fund for the development of teaching and learning (five phases 1-5).
FTE Full time equivalent.
HEFCE Higher Education Funding Council for England (http://www.hefce.ac.uk/)
HEFCW Higher Education Funding Council for Wales (http://www.elwa.org.uk/)
IEEE Institute of Electrical & Electronic Engineers.
IMS IMS Global (http://www.imsglobal.org/)
IPR Intellectual property rights.
IVIMEDS International virtual medical school.
Java Programming language.
JISC Joint Information Systems Committee
JORUM ‘a large drinking bowl’ and the name of a national repository for RLOs in the UK funded by
the JISC.
JTAP JISC Technologies Application Programme
(http://www.jisc.ac.uk/index.cfm?name=programme_jtap)
LAMS Learning activity management system.
LE Learning environment.
Learning Object Any entity, digital or non-digital, that may be used for learning, education or training (IEEE).
MCQ Multiple choice questions
Medici Promotes the partnership between museums, cultural institutions and industry.
Metadata Metadata provides information about the content, quality, condition, and other characteristics
of data.
METRO Medical education !t axonomy research !organisation.
MLE Managed Learning Environment - definition from the JISC which is widely accepted in higher
education (http://www.jisc.ac.uk/index.cfm?name=issue_vle_mle)
Moodle open source VLE.
MySQL Open source database software.
Nathan Bodington Building (Bodington)
Open source virtual learning environment (http://bodington.org/)
NI Northern Ireland
NLE Medical VLE developed at Newcastle. Used elsewhere.
NPC Subject Centre Nominated Primary Contacts
OCTAVE Optimising computer-aided and traditional assessment in veterinary education.
OSX Unix based operating system.
Page 83
PARS Personal and professional development.

PC Personal computer.
Perl Programming language.
PHP Hypertext pre-processor (was called ‘personal home pages’).
PostgreSQL Commercial database software.
PPD Personal and professional development.
Python Programming language.
QAA Quality Assurance Agency (http://www.qaa.ac.uk/)
QTI IMS Question and Test Interoperability standard / specifications (http://www.imsglobal.org/)
QuestionMark Perception
Commercial assessment software (http://www.questionmark.com/)
RELOAD Reusable elearning authoring and delivery.
SDMCG Scottish deans medical curriculum group.
SHEFC Scottish Higher Education Funding Council (http://www.shefc.ac.uk/)
Single-sign-on Mechanisms for authorising authenticated users access to resources without having to login
separately (authentication comes from elsewhere).
SQL Structured query language.
TLTP Teaching and Learning Technologies Programme
TOIA Advanced online assessment management system.
Tribal Technology Commercial learning environment http://www.tribaltechnology.co.uk/ (formerly FD Learning)
UCEL Universities collaboration in e-learning.
VALE Medical VLE at Glasgow.
VLE Virtual Learning Environment
VMLE Virtual Managed Learning Environment
WebCT Commercial virtual learning environment (http://www.webct.com/)
Win2K Windows
Zope Open source programming application.

The Higher Education Academy:
Medicine, Dentistry and Veterinary Medicine
School of Medical Education Development
Faculty of Medical Sciences
University of Newcastle
Newcastle upon Tyne
NE2 4HH
T: +44 (0)191 2225888
F: +44 (0)191 2225016
enquiries@medev.ac.uk
www.medev.ac.uk
This study was made possible with miniproject funding

from The Higher Education Academy: Medicine,
Dentistry and Veterinary Medicine (formerly LTSN-01)
The Higher Education Academy is a new UK-wide organisation set up to

support quality enhancement in teaching and the student experience in
higher education. It was formed from a merger of the Institute for Learning
and Teaching in Higher Education (ILTHE), the Learning and Teaching
Support Network (LTSN), and the TQEF National Co-ordination Team (NCT).
All 24 previous LTSN subject centres are now part of the Higher Education
Academy subject network. You can find out more at: www.heacademy.ac.uk

Higher Education For Medicine, Using VLE

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Higher Education For Medicine, Using VLE

Încărcat de

Drepturi de autor:

Formate disponibile

Special Report 6

Review of Virtual Learning

Julian Cook August 2005

ISBN 0 7017 0186 2

Review of Virtual Learning Environments in UK

Higher Education Academy Subject Centre for Medicine,

Julian Cook ISBN 0 7017 0186 2 Page 2

Julian Cook ISBN 0 7017 0186 2 Page 4

Professor William Doe

8.1. Participation at face-to-face event. Attendance at the Academy event “Twenty-

Data was gathered through:

8.2. On-line questionnaires.

Julian Cook ISBN 0 7017 0186 2 Page 6

Main VLE - Source: Developers Survey (n=28).

2nd VLE - Source: Developers Survey (n=28).

Commercial systems in use - Source: Developers Survey (n=12).

Reasons for choice of VLE - Source: Developers Survey (n=14).

Source: Developers Survey (n=12).

Suitability of commercial systems - Source: Developers Survey (n=26).

Satisfaction with VLE’s reflection of curriculum - Source: Developers Survey (n=26).

Institutional attitudes to bespoke medical VLEs - Source: Developers Survey (n=12).

Likely future relationship - Source: Developers Survey (n=12).

Personalisation features - Source: Developers Survey (n=14).

Other features - Source: Developers Survey (n=14).

Julian Cook ISBN 0 7017 0186 2 Page 8

9.4. Use of VLEs has increased “significantly” across the sector.

Most schools now use a VLE - Source: Academics Survey (n=38).

9.5. Levels of use by students and staff have increased “significantly”.

Usage statistics - Source: Developers Survey (n=10).

Server descriptions - Source: Developers Survey (n=21).

Support teams - Source: Developers Survey (n= 22).

2 Acknowledging of course that these terms are very imprecise.

Interoperability technologies - Source: Developers Survey (n=21).

Sharing system code and components - Source: Developers Survey (n=23).

Sharing content - Source: Developers Survey (n=24).

9.8. Although delivery of VLEs to hospital sites is somewhat constrained by IT

Julian Cook ISBN 0 7017 0186 2 Page 10

How e-learning is used - Source: Academics Survey (n=44).

Source: Paper-based questionnaire (n=7).

Julian Cook ISBN 0 7017 0186 2 Page 12

11.1. Hosting institutions

11.2. Medical schools

11.3. All developers

11.4. Developers of bespoke systems

11.5. Funders / stakeholders

Evaluation work should be funded into:

Julian Cook ISBN 0 7017 0186 2 Page 14

JTAP-623 found that:

Julian Cook ISBN 0 7017 0186 2 Page 16

12. Data collection

12.1. Participation at face-to-face events

12.2. On-line questionnaires4

12.3. Methodology notes

13.1. Paper-based questionnaire

Seven responses were received, representing the following schools:

Julian Cook ISBN 0 7017 0186 2 Page 18

13.2. Academics survey

Medical Schools Dental Schools

13.3. Developers Survey

Medical Schools Medical Schools (continued)

7 These results do not represent the University of Derby.

Julian Cook ISBN 0 7017 0186 2 Page 20

14. Case for Medical VLEs