See

discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/221195622

A Markov Game Theory-Based Risk Assessment

Model for Network Information System.
CONFERENCE PAPER JANUARY 2008
DOI: 10.1109/CSSE.2008.949 Source: DBLP

CITATIONS

DOWNLOADS

VIEWS

70

128

4 AUTHORS, INCLUDING:
Xiaobin Tan
University of Science and Technology of Ch
27 PUBLICATIONS 53 CITATIONS
SEE PROFILE

Available from: Xiaobin Tan

Retrieved on: 14 July 2015

A Markov Game Theory-based Risk Assessment

Model for Network Information System
Cui Xiaolin, Tan Xiaobin, Zhang Yong,Xi Hongsheng
Department of Automation
University of Science and Technology of China
Hefei, Anhui, 230027, P.R.China
E-mail: cuixl@mail.ustc.edu.cn
AbstractRisk assessment is a very important tool to acquire a
present and future security status of the network information
system. Many risk assessment approaches consider the present
system security status, while the future security status, which also
has an impact on assessing the system risk, is not taken into
consideration. In this paper we propose a novel risk assessment
model based on Markov game theory. In this model, all of the
possible risk in the future will impact on the present risk
assessment. The farther away from now, the smaller impact on
the risk assessment it has. After acquiring the system security
status, we proposed an automatic generated reinforcement
scheme which will provide a great convenience to the system
administrator. A software tool is developed to demonstrate the
performance of the risk assessment of a network information
system and a simulation example shows the effectiveness of the
proposed model.

vulnerability, and could discover the hidden risks. But the

model needs to the relationship between every two different
vulnerability. So the relationship is very complex when the
number of vulnerability in network information systems is
large.

Keywords- risk assessment; Markov game theory; threat

transmission

The rest of the paper is organized as follows. In Section 2,

we briefly describe the related work. Section 3 presents our
framework for risk assessment. In Section 4, we describe the
Markov game theory-based risk assessment model for the
network information system. Section 5 shows our experiment
results and discussions. Section 6 concludes the paper.

I.

INTRODUCTION

With the rapid development of the Internet, the amount and

species of malicious codes also ever-increasing, which bring
serious threat to the network security. To counter these threats,
we need to identify those vulnerabilities that are susceptible to
these malicious codes. Vulnerability is the absence or
weakness of a safeguard in some asset or resource. This
absence or shortcoming makes a threat or attack potentially
more harmful or costly and more likely to occur. In order to
understand the affects from these threats to the network
information system security, we need to take a risk assessment
on the network information system and improve the security
situation of the system based on the assessment results.
On the current information security risk assessment
methods can be classified two categories: One is the traditional
risk assessment methods, such as FTA [1], FMECA [2],
HAZOP [3], Markov [4] and so on; The other is the modern
methods of risk assessment, such as CORAS [5], RSDS [6],
CRAMM [7], COBIT [8] and so on. However, these methods
did not give the specific application for the network
information system and not advise how to repair the
vulnerability of the network information system. Zhang et al.
[9] provides a risk propagation model for assessing network
information systems, and give the specific application. This
model considered the relationship between different

This work is supported by the National 863 High-tech Program of China

(No. 2006AA01Z449) and the 42nd National Science Foundation for Postdoctoral Scientists of China (No. 20070420738).

In this paper, we present a Markov game theory-based risk

assessment model for the network information system. In this
model, we use a Markov chain to describe the spreading
process of potential threats so as to assess the system risk. By
using the Markov chain, we can simulate the threat propagation
and discover the hidden risk. And we utilize another Markov
chain to depict the repair process implemented by the system
Administrator aiming at the system vulnerability, so as to make
the amount of vulnerabilities utilized by threats be less and
make the system safer.

II.

RELATED WORK

The purpose of risk assessment is to understand the present

and future system risks, access the security threats and the
degree of influence probably engendered from these risks, and
provide the basis for security strategy identification,
establishment and safe operation of the information system. In
order to achieve the purpose, many countries and organizations
have established the risk assessment audit standards, such as:
CC [10], SSE-CMM [11], ISO/IEC 17799[12], BS 7799 [13],
ISO 13335 [14], IATF [15] and GB/T 20984-2007 [16]. GB/T
[16] puts forward the principle of risk calculation:

RISK = R( A, T , V ) = R ( L(T , V ), F (I a , Va ))

(1)

Where R is the function of security risk calculation; A, T, V

denote asset, threat and vulnerability respectively; Ia denotes
the value of asset acted security event; Va shows the harm
extent of vulnerability; L denotes the probability of security
event induced threats which utilize the vulnerability of asset; F
is the loss took place after safety time.

operation on system vulnerability by system administrator need

to do so as to reduce the system risk to an acceptable range.
III.

Figure 1.

Principle of Risk Assessment

Threat involves malicious code and network attack. In this

paper we primarily consider the threats of malicious code.
Malicious code is divided into five categories: Trojan horses,
worms, viruses, spyware and corpse network client
programmer.
Vulnerability is consisted of management vulnerability and
hardware and software vulnerability.
Asset is described by Confidentiality, integrality and
availability, and includes the importance of network location.
In the network information system, for each asset, the threat
induces risk by utilizing its vulnerability. In order to conduct
risk assessment, first of all we should identify the threat. For an
asset, the threats come from two aspects: the existing threats
and potential threats. The existing threats are the threats which
exist in asset and the potential ones mean that exist in network
information system and the Internet rather than the asset.
Although the potential threat does no harm to system at the
present stage, it will endanger the asset by spreading through
LAN and the Internet.
Now, there are many threat identification methods [17],
vulnerability identification methods [18] and asset
identification methods [19].These methods provide the basic
data for risk assessment. But how these data will integration is
a very important task. Shen et al. [20] puts forward the Markov
game theoretic data fusion approach. The Markov (stochastic)
game method is used to estimate the belief of each possible
cyber attack pattern and give the corresponding defensive
strategy.
Based on the principles of risk assessment and the Markov
(stochastic) game method, this paper have proposed a Markov
game theory-based risk assessment model (MGTBRAM). The
model uses a Markov chain to describe the spreading process
of potential threats so as to assess the system risk, and utilizes
another Markov chain to depict the repair process done by the
system Administrator aiming at the system vulnerability, so as
to make the amount of vulnerabilities utilized by threats be
less and make the system more safe. On the one hand, threats
acting on vulnerability can induce risk and the risk will be
larger and larger by threat spreading; on the other hand, the risk
will be smaller and smaller by the system Administrators
repairing the vulnerability. Thus we can establish a game
relation between threats and vulnerabilities. In this paper we
mainly discuss the following two aspects: 1) the changes of
system risks in the next period of time if the system
administrator doesnt repair any vulnerability. 2) The repair

FRAMEWORK FOR RISK ASSESSMENT

Based on above principle of risk assessment, we suggest a

novel design approach to risk assessment system, illustrated in
Figure 2. This framework gives precise mathematical model to
describe network risk. Especially, it gives a practical security
reinforcement scheme used to guide people to improve network
security. It is composed of five main modules. We will discuss
them in details.
Threat identification module: It detects malicious codes in
each asset of the network system and stores these data in a
database in a certain format. These data include threat name,
threat type, asset IP, related vulnerability, probability of the
threat spread and so on.
Vulnerability identification module: it detects vulnerability
in each asset of the network system and stores these data in a
database in a certain format. These data include vulnerability
name, vulnerability type, asset IP, harm extent of vulnerability
and so on.
Asset identification module: it detects the network system
and evaluates the value of each asset by the importance of
asset. Then it stores the value of assets and the IP of assets in a
database. When assessing risk, the IP of assets will link to
threat and vulnerability.
MGTBRAM module: the data of above three modules will
input into this module. By calculating, the output (max risk and
reinforcement scheme) will input into risk assessment module.
This module is the core of risk assessment and will be
discussed in section 4.
Risk assessment module: in this module, we will calculate
the value at risk for each vulnerability, and recorded as R(i),
i=0,1, , n. We assume that we have repaired each vulnerability
according to the repair scheme, and calculate the value at risk
for each vulnerability again. The new value will be recorded
residual risk as RR(i), i=0, 1,, n. In order to give
administrator a repair scheme list, we recorded eliminate risk
as ER (i), i=0, 1, , n.
ER (i ) = R (i) RR (i) i = 0,1,", n

Figure 2.

The Framework of Risk Assessment

(2)

Let ER(i) list in descending order and we will select several

forefront ER as the suggestion to administrator. The
administrator could use the least operation to minimize the
system risk. The Markov game theory- based risk assessment
model provides a new idea for the automatic generation of a
reinforcement scheme.
IV.

MARKOV GAME THEORY-BASED RISK ASSESSMENT

MODEL
Plays --- Two sides of Game are threat agent and
vulnerability agent. The threat agent increases the risk by threat
spreading and the vulnerability agent decreases the risk by
system administrators repairing the vulnerability.
State Space --- For the threat agent (denoted by t), at the
moment of k, the threat state (TS) of the n-th asset is expressed
as snt(k) and its value is 0 or 1: 0 denotes no threat and 1
denotes a threat. The threat state of the whole system at the
moment of k is:

st (k ) = ( s1t (k ), s2t (k ),", snt (k ))

For the vulnerability agent, we say the system administrator

repairing one vulnerability as its one action. The repair scheme
of system administrator about the vulnerability is described in
the state transition rule. For simplifying the repair process of
vulnerability, we assume that repair one vulnerability of asset
once. At the moment of k, the repair process of the system
administrator about the vulnerability of asset is label as uv(k),
where v is the vulnerability agent.
The State Transition Rule --- As time changes, the state of
each asset in the network system also constantly changes. We
use p(sk+1|sk,ut,uv) to describe the law of changing state, where
sk+1, sk denote the states of the (k+1)-th and k-th moment
separately, ut(k) and uv(k) denotes the actions took by the threat
and vulnerability agent separately. At the moment of k, the
threat and vulnerability agent will adopt corresponding actions
according with their own strategy sets (Transmission Strategy
Sets and Repair Strategy Sets) separately. Figure 3. shows the
Markov game process.

(3)

For example, assume that there are three assets, so (1,0,0)

denotes the first asset has the threat and the second and third
dont have the threat.
Similarity for the vulnerability agent (denoted by v), at the
moment of k, the vulnerability state (VS) of the n-th asset is
expressed as snv(k) and its value is 0 or 1: 0 denotes no
vulnerability and 1 denotes a vulnerability. The vulnerability
state of the whole system at the moment of k is:
(4)

s v (k ) = ( s1v (k ), s2v (k ),", snv ( k ))

The risk state (RS) of network information system at the

moment of k as s(k):
t
1(

v
1(

t
2(

v
2(

t
n(

v
n(

s(k ) = (s k ) * s k ), s k ) * s k ),", s k ) * s k ))

(5)

Figure 3. Markov Game Process

Damage Function --- When a asset has a threat, the threat

will do some damage to the asset and the longer time the threat
exists, the more the damages are. We label the damage value at
the moment of k in the network system as V(s(k)) and in this
system the damage value in unit time of the threat at the
moment of k is R(s(k)). Thus:
V (s(k )) = R(s(k )) +

t
v
p(s(k + 1) | s(k ), u , u )V (s(k + 1)) (6)
s(k + 1)

Action Space --- For the threat agent, we say one spread of
the threat as its one action. The threat can be transmitted to
other assets with certain probability which is given in the state
transition rule. For simplifying the spread process of threat, we
assume that the threat is spread to one asset once. In order to
distinguish the main spread threat, we define source asset s and
destination asset d.

Where s(k+1) is the system state at the moment of k, which

is related with s(k), ut, uv, so we need to sum all possible states
of s(k+1). is a discount factor. p(sk+1|sk,ut,uv) is the state
transition probability. That is probability of the system risk
state changed into s(k+1), when system risk state is s(k) and the
actions of the threat, vulnerability agent are separately.

To one threat, source assets are internet or the assets which

have the threat. And destination assets are ones which have no
this threat. At the moment of k, the action is the process of
threat spreading from source asset to destination asset.

R( s(k )) = VA s (k )

At the moment of k, threats through a variety of ways from

the source of assets to the purpose of dissemination of assets
via various ways is labeled as ut(k) t is the threat agent.
Threats have many spreading modes and we mainly
consider the following several modes: removable Storage, EMail and downloads, shared directories and so on.

(7)

V denotes the harm extent of vulnerability and A is the

value of asset.
In order to facilitate to calculate the risk value of system,
we only consider the changes of system state within n steps.
According to the Damage Function, we can calculate the
system risk induced from each threat and accumulate all these
risks to get the aggregate risk value Vsys of the system.

(8)

Risk Assessment --- When the risk assessment, the system

administrator not to take any repair scheme. The vulnerability
state remains unchanged. Therefore, the process of Markov
game regresses into the process of Markov decision. Given a
speed of threat transmission, we select a transmission strategy
of threat to make the system risk values be largest and utilize
this largest risk value to assess the risk.
V ( s ( k ) ) = m ax V t , v ( s ( k ) )
t T

(9)

Repair Scheme --- When system administrator repair the

vulnerability, the transmission strategy of threat will change
accordingly to maximize the risk affects of system from the
threat. In such a case, the system administrator needs to select
the best repair scheme to minimize the system risk. *(s(k)) is
the best repair scheme.
* ( s (k )) = arg min max Vt ,v ( s (k ))
v V t T

(10)

VDS The vulnerability state (MS07-037) (VS) which is

related to the threat we have detected is sv(k). sv(k)=(1,0,0).
And the harm extent of vulnerability is 3.
ADS: The value of assets divided into five grades. The
small network system has 3 assets and their values are 1, 2 and
3. That is A= (1,2,3).
RAS: In this subsystem, we assume that is 0.8. And we
will calculate the risk value by 4 steps because the impact to
risk value after 4th step can be ignored. The result of
calculation is that the value at risk is 9.72307 and we will
repair the vulnerability at the first asset.
In order to illustrate the superiority of our model, we
compared the two sets of data: the result of calculation by the
Markov game model (MGM) and the result of calculation by
the traditional assessment model (TAM).
VS:(1,0,1)

value at risk

Vsys = Vi

The system risk value will be reduced to the maximum as

long as system administrator operates according with the best
repair scheme.

45
40
35
30
25
20
15
10
5
0

VS:(1,1,1)

(000) (001) (010) (011) (100) (101) (110) (111)

threat state

Figure 4. Traditional Assessment model

V ( s (k )) = min max Vt , v ( s( k ))
v V t T

VS:(1,0,1)

To evaluate our game theoretic approach for risk

assessment, we have constructed a Risk Assessment Platform
(RAP).In the platform, there are four subsystems. They are
Malicious Code Detection Subsystem (MCDS), Vulnerability
Detection Subsystem (VDS), Asset Detection Subsystem
(ADS) and Risk Assessment Subsystem (RAS).
MCDS will detect the malicious code in the network
system. The results of malicious code detection include threat
name, threat type, asset IP, related vulnerability, probability of
the threat spread and so on. VDS will detect the vulnerability in
the network system. The results of vulnerability detection
include vulnerability name, vulnerability type, asset IP, harm
extent of vulnerability and so on. ADS will detect the asset in
the network system. The results of asset detection include asset
name, asset IP, and value of asset. The results of the three
subsystems will save in database for RAS.
We have detected a small network system and gained
following data:
MCDS: We have detected many threats from the small
network system, but we will use a threat (Trojan.Mybot-6307)
as an example. The threat state (TS): st(k)=(1,0,0) and the
transmission probability is 0.2.

value at risk

EXPERIMENTS AND DISSCUSSIONS

45
40
35
30
25
20
15
10
5
0

VS:(1,1,1)

(000) (001) (010) (011) (100) (101) (110) (111)

threat state

Figure 5.

Markov Game model

Traditional Assessment Model

Markov Game Model

30
value at risk

V.

(11)

25
20
15
10
5
0

(000) (001) (010) (011) (100) (101) (110) (111)

threat state

Figure 6.

Two model comparison

From Figure 4, we can see that the two VS have the same
value at risk when threat state is (0,0,1), (0,1,1), (1,0,1). The

traditional model cannot distinguish the risk of different

vulnerability state. But from Figure 5, we can distinguish them.
When the vulnerability state is (1,1,1), there are greater
risks in the small network system. So we think that the Markov
game model can discover the potential risks. From Figure 6, we
can also see the value at risk obtained by MGM is greater than
the value obtained by TAM. It is because the value at risk
obtained by MGM contained the potential risks. From Figure 4,
Figure 5 and Figure 6, it is clear that the performance of
Markov game model is better than the traditional assessment
model.
By the Markov game model, we can not only get more
comprehensive value at risk, but also give the best system
repair scheme. In this experiment, we obtained the repair table
(Figure 7) for all threat state and all vulnerability state. No
matter what threat state and vulnerability state, we can easily
find a repair strategy from the repair table. In Figure 6, 1
denotes that we should repair the first asset. 2 and 3 is similar
to 1. For example, we have detected that the threat state is
(1,1,0) and the vulnerability state is (0,1,1), so we will repair
the vulnerability of the second asset.

This work is supported by the National 863 High-tech

Program of China (No. 2006AA01Z449) and the 42nd National
Science Foundation for Post-doctoral Scientists of China (No.
20070420738).
REFERENCES
[1]
[2]

[3]
[4]
[5]
[6]
[7]
[8]
[9]

Threat state

Vulnerability state

ACKNOWLEDGMENT

000

001

010

011

100

101

110

111

000

001

010

011

100

101

110

111

[10]

[11]
[12]

[13]
[14]

Figure 7.

VI.

Repair table

CONCLUSIONS

In this paper, we present a Markov game theory-based risk

assessment model for the network information system. Using
the model, we can know the risk condition of the network
system, which include the risk from the potential threats. And
we can get the automatic generation of a remedial scheme and
provide network administrators with a convenient. Experiment
results on Risk Assessment Platform approve the effectiveness
of our proposed method. In future, we will research the spread
of the threat (Trojan horses, worms, viruses) and use it in our
model in order to improve our model.

[15]
[16]
[17]
[18]
[19]

[20]

IEC 1025: 1990 Fault Tree Analysis (FTA), 1990

Bouti, A. and Ait Kadi, D., "A State of the Art Review of
FMEA/FMECA", International Journal of Reliability, Quality and
Safety Engineering. January 1949: 515-543.
Redmill F, Chudleigh M, Catmur J, "HAZOP and Software HAZOP",
Wiley, 1999.
B. Littlewood, "A reliability Model for Systems with Markov Structure",
Applied Statistics, 1997, 24(2):172~177.
CORAS IST-2000-25031 Web Site, http://www.nr.no/coras. 24
February 2003.
Reactive System Design Support, "RSDS", http://www.kcl.ac.uk.2002.
Commission of the European Communities Security Investigations
Projects: Project S2014, Risk Analysis, CRAMM Evaluation, 1993.
Control Objectives for Information and Related Technology,
http://www.isaca.org/COBIT, 2003.
J ZHANG Yong-Zheng etc., "Research on Network Node Correlation in
Network Risk Assessment", China Journal of Computers, February
2007: 234~240.
The International Organization for Standardization, Common Criteria
for Information Technology Security Evaluation, ISO/IEC15408:
1999(E), 1999.
SSE-CMM Model Description Document, Version 2.0, 1999,
http://www.sse-cmm.org.
International Organization for Standardization, Code of Practice for
Information Security Management, ISO/IEC 17799:2000, December
2000.
BSI/DISC Committee BDD/2, BS7799 Code of Practice for Information
Security Management, 1999.
J International Organization for Standardization, ISO/IEC TR 13335,
Guidelines for the Management of IT Security (GMITS), 1996-2001.
J National Security Agency, Information Assurance Technical
Framework (IATF), Version 3.0, 2000, http://www.iatf.net.
Information security technology, Risk assessment specification for
information security, GB/Z 20984-2007.
WU Bing etc., "Network-based malcode detection technology", Journal
on Communications, November 2007: 87~91.
Ritchey, R.W etc., "Using model checking to analyze network
vulnerabilities", IEEE Conference on Security and Privacy, May 2000.
Beaudoin, L etc., "Asset Valuation Technique for Network Management
and Security", IEEE Conference on Data Mining Workshops, December
2006.
Dan Shen etc., "Adaptive Markov Game Theoretic Data Fusion
Approach for Cyber Network Defense", IEEE Conference on Military
Communications, October 2007.