The Use of a Collegiate Cyber Defense Competition in
Information Security Education
Art Conklin
The University of Texas at San Antonio
6900 N. Loop 1604 West
San Antonio, TX 78249
210-4-6309
art.conklin@utsa.edu
ABSTRACT
A Collegiate Cyber Defense Competition was conducted this
spring with five universities fielding teams. This event was
designed to provide student led teams a simulated operational
business environment in which they could test their skills at
information security and system operation. This three day event
included external hackers attacking the student networks, as well
as business users placing operational and administrative demands
on the teams. Teams were scored based on their ability to
maintain mandated business services, keeping hackers out and
answering business scenario injects. The objective was to provide
an environment where the teams could exercise their information
technology abilities and do so in an operational mode where most
information was not freely provided, but must be uncovered and
discovered by the students in real time. Issues such as
prioritization, resource allocation and technical skill application
were tested using real world examples on a known instrumented
network. The objective of providing feedback to each team as to
its skill level and abilities was achieved and then some over the
course of this successful event. Lessons were learned which are
being used to move this local event to a national level event.
Categories and Subject Descriptors
K.3.2 Computer and Information Science Education, Computer
science education, Curriculum, Information systems education
General Terms
Security.
Keywords
Information Security, Education, Competition.
1. INTRODUCTION
Computer security is an important issue in today’s environment of
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
Information Security Curriculum Development (InfoSecCD)
Conference '05, September 23-24, 2005, Kennesaw, GA, USA.
Copyright 2005 ACM 1-59593-261-5/05/0009…$5.00.
16
a ubiquitous information technology landscape. This ever
increasing IT environment is driving a need for more college
graduates with significant information technology backgrounds.
Colleges turn out graduates with many different levels of specific
skills related to the proper application of information technology.
Computer science programs are developing the next generation of
professionals that will develop the technical aspects of the IT
systems that will drive our future. Information systems
departments in colleges of business specialize in the business use
of technology to achieve business goals. Between these two
views, there is a continuum of students and skills to support
society’s need for ever increasing technological solutions.
Computer security is one of those areas that have dependencies on
both technical and implementation foundations. Educating
college students to become effective computer security
professionals is a daunting task as the breadth of knowledge
required is immense. Theoretical and technical elements are
typically covered in computer science curriculums. The practical
application of security principles is covered in many CS and IS
programs. However, these two foundational elements have not
proven to be sufficient to produce the professionals desired by
industry today.
Recently, a Collegiate Cyber Defense Competition was conducted
with participants from five colleges. The competition was
designed to challenge the teams in a simulated operational
business environment, to provide basic business related services
and do so in a secure fashion. To test their abilities, a separate
team was used to simulate a series of hacker attacks against their
networks. Additionally, a series of business related challenges
relating to standard system administration were presented, forcing
the teams to react to business requests as well as the external
threat environment. This competition was held at The University
of Texas at San Antonio this spring with interesting results. All
of the teams performed well, with each team eventually securing
their environment from the outside attacks. The teams took
differing approaches in team organization and operational
approach to the problems. There was not a single optimal method
demonstrated, each had advantages and disadvantages. At the
conclusion of the competition, all participants were surveyed as to
the state of their preparedness for the competition.
The answers to the preparedness questions led to some interesting
revelations. The vast majority of participants stated that their
curriculum did not adequately prepare them for the competition.
The reasons were varied, from specific, i.e. no practical
experience setting up a Cisco VPN router and firewall, to general,
i.e. poor preparation to prioritize multiple simultaneous issues.
Technical issues plagued some teams, procedural and
implementation issues challenged others. Over the course of the
three day event, each team had faced numerous challenges of
differing natures. The wide range of these challenges was a
significant contributor to students’ concern over preparation.
The purpose of the competition was to provide an educational
environment for students to critically examine their abilities. This
opportunity is different from a standard examination in a couple
of ways. It is team based; allowing students to work in teams and
capitalize on different team members strengths. It was conducted
over three days with continuous feedback to the teams, enabling
them to make changes in their approaches and activities in
response to the measured effectiveness. The result of the exercise
was that the teams, both students and faculty members, felt that
the event served to increase their education in computer security
and the operational aspects of employing information technology
in a business environment. Based on this result, the expansion of
the competition to a national level event is being planned and
pursued by the initiators of this first regional event..
2. COLLEGIATE CYBER DEFENSE
COMPETITION
The genesis of the idea to conduct a collegiate cyber defense
competition was rooted in two events. First is the annual cyber
defense exercise (CDX) conducted by the United States Military
Service Academies. [1] Second was the NSF sponsored workshop
on developing a national cyber defense exercise. [2] After the
workshop, two faculty members of The University of Texas at
San Antonio initiated actions to field a regional competition.
With the assistance of faculty members from The University of
Texas at Austin and Texas A&M University the regional
collegiate cyber defense exercise was designed. Examining
previous exercise based competitions [3-5], the operational
business environment was chosen to provide a realistic basis for
the event. The use of an operational environment was purposely
chosen, as that is they type of environment the majority of
students will later be involved with in some capacity.
The competition was designed to be defensive only from the
aspect of the student led teams. Each team was given identical
hardware and software, preconfigured into a working business
environment. The systems included web site, e-commerce
activity, email, file servers, database servers and various business
functionalities such as FTP, and user account maintenance. The
systems were built to simulate a working business environment
into which the team was hired to replace an existing IT member.
This was a small business environment, one with a small, couple
person IT staff, and the team was acting in lieu of this small staff.
[6]
The competition involved several taskings for each team. First
was the issue of understanding and securing the existing systems.
Each box in the system had to be examined and brought up to an
appropriate patch level to provide desired functionality and
security. An identical mixture of Microsoft Windows and Linux
based boxes was deployed to simulate a mixed operating system
environment. After a quiet period designed to let each team get
their system under control, a red team was used to conduct
17
penetration tests against each network. This activity was to
simulate an external hacker attempting to disrupt the companies
network operations. The red team had no inside knowledge of
individual systems, or account passwords, nor were any specific
backdoors placed in the systems to facilitate red team actions.
Some limitations were placed on the red team, such as no denial
of service attacks or system destructions to keep the competition
on a positive base. Red teams were also not permitted to use
zero-day or classified exploits or tools against the competing team
systems. In addition to the attacks by the red team, a series of
timed business related injects were periodically presented to the
teams. These injects were designed to act as typical business
related activity and involved items such as equipment changes,
personnel changes and system functionality changes. Lastly, a
scoring engine was used to measure the availability of a series of
business functionality to ensure systems were up and responding
in an appropriate manner.
Each team’s score was a combination of how well they kept their
systems up and functioning as reflected by the scoring engine,
how well they handled the injects, as each inject was scored, and
how well they kept the external hackers at bay. Injects were
multi-level items with a range of points based on the level of
completion. Red team activities were scored based on both
keeping them out, and on reactions to penetrations. The scoring
engine kept a running total of how much time each mandated
system was operational. During the second and third day, service
level agreements were used in conjunction with the scoring
engine. Judges were deployed to each team to assist in the
monitoring of the event and to handle questions and issues.
3. RESULTS OF THE COMPETITION
The University of Texas at San Antonio hosted the
competition on April 15-17, 2005. Teams from five schools
attended the event, representing the University of North Texas,
Delmar College, Texas A&M University, The University of
Texas at Austin and The University of Texas at San Antonio
(UTSA). The participants were two-year college students from
Delmar College and a mix of undergraduate and graduate students
from the other institutions. The only real requirement was that
their institution recognized the student competitors as full-time
students. The student curriculums were a mix of Computer
Science and Information Systems. The competition was hosted
by the Center for Infrastructure Assurance and Security at UTSA
(CIAS) by a team completely separate from the UTSA competing
team. The red team was composed of members from red teams in
industry and government as well as members of the CIAS. White
team members (judges) came from all of the participating schools
and were separate from the competing teams and their faculty
sponsors.
At the beginning of the competition, each team was given
identical equipment, in fully functioning form, but with known
security holes. Each team was free to address the security issues
and equipment as they saw fit, the requirement being to keep the
mandated services up and running. Once the red team began
operation, it was only a matter of time until they had established
root level penetrations against all of the teams. As the teams
learned of the penetrations they all were eventually successful in
securing their systems and keeping the external hackers out, while
maintaining the majority of the required services. Time was
compressed across the three days, with numerous injects
occurring in sequences that would normally occur over months.
For the majority of the injects, adequate time was allocated and
all teams were treated identically, same injects, same time, same
scoring criteria. As many injects were part of long running
sequences, failures in earlier injects that were never corrected by
a team did come back to haunt them later. Each day a list of the
number one and number two team was posted before the
following day began. Points were cumulative and in the end, the
Team from Texas A&M University finished in first place.
The results of the competition were much further reaching than
just crowning of the first winner. One school formed a student
chapter of the ISSA (Information Systems Security Association)
in an effort to improve their computer security related skills and
to connect to industry activity and leadership. Each school went
away with a critical self assessment of their strengths and
weaknesses and a desire to improve their performance. Exit
surveys showed that the event was considered fair, realistic and
valuable to students and sponsoring faculty members. This was
an important finding as the objective was to provide a fair and
realistic test-bed environment where the teams could evaluate and
test their skills in a competitive environment, not unlike a modern
small business environment. One of the take-away items
provided to each team was a tcpdump of all packets for all teams
over the course of the competition, so they could go back and
analyze specific events, activities and responses.
4. BENEFITS OF THE COMPETITION
There are a number of benefits in providing a competition
associated with cyber defense. Computer security is a very
current issue in the eyes of many corporations and government
agencies and jobs exist in this career field. Practical experience is
always an important element in anyone’s education and this event
was built with that in mind. Today’s business environment is a
team based endeavor, and this competition was designed to
require a team based approach. Using student teams to achieve
the results added the issues of team formation and interpersonal
dynamics to the event, something the judges all got a chance to
see as the hours drew on and tempers got short. Ultimately all of
the teams achieved the true desired goal, which was to learn what
they knew, what they didn’t and where they felt they needed to
improve. They use of a competition over three days as a team
based laboratory exercise worked well to produce an environment
that tested each team’s knowledge and ability to employ that
knowledge in an operational environment. They had to use the
knowledge they had garnered in classes and previous experience
to operate the systems, determine the weaknesses and determine
when things went wrong. They were not told when they were
penetrated, they had to find the clues and discover it, all the while
answering executive requests in the form of injects. This opened
many team members’ eyes to the less than perfect information
environment of an operational environment as opposed to an
academic exam where all the relevant facts are typically
enumerated.
18
5. CONCLUSION
Developing the information security professionals of tomorrow is
not simple work. There is a myriad of technical and
implementation information that has to be mastered. Multiple
disciplines are needed working together as a team to solve the
business related information security tasks of a modern e-business
environment, with a mix of technical and non-technical issues.
Finding ways to educate and motivate students to pursue this
demanding set of requirements is challenging. Currently most
curriculums are broken into technical and non-technical tracks
with little overlap or interfacing between them. This competition
was designed to breakdown that academic separation and simulate
the environment the students will find themselves in after
graduation. The inclusion of industry representation, both from
the red team side and through team sponsorship helped increase
awareness among the students of the multi-disciplinary and team
related challenges that lie ahead of them in their future work
environments. Our objective as the implementers of this
competition was to build a better, more realistic learning
environment for students to take an active role in their education.
Based on our success at this first competition, we felt we met our
initial objectives. We felt we came a long way in this inaugural
endeavor and plan to take this event to a national level scale in the
next year.
6. REFERENCES
[1] Ragsdale, D., Welch, D., and Dodge, R. Information
Assurance the West Point Way, IEEE Security and Privacy,
vol. 1, no. 5, Sept/Oct 2003, pp. 64-67.
[2] Hoffman, L. J. and Ragsdale, D. Exploring a National Cyber
Security Exercise for Colleges and Universities, Report No.
CSPRI-2004-08 The George Washington University Cyber
Security and Policy Research Institute Report No. ITOC-TR-
04001 United States Military Academy Information
Technology and Operations Center August 24, 2004
[3] Schepens, W. J., Ragsdale, D. J. and Surdu, J. R. The Cyber
Defense Exercise: An Evaluation of the Effectiveness of
Information Assurance Education, The Journal of
Information Security, Volume 1, Number 2. July, 2002.
[4] Vigna, G. Teaching Hands-On Network Security: Testbeds
and Live Exercises, Journal of Information Warfare, vol. 3,
no. 2, pp. 8-25, 2003.
[5] Welch, D.W. Ragsdale, D.J. and Schepens, W.J. Training for
Information Assurance, IEEE Computer, March 2002.
[6] White, G.B. and D. Williams. The Collegiate Cyber Defense
Competition. in Proceedings of the 9th Colloquium for
Information Systems Security Education. June 2005. Atlanta,
GA.