Addressing SIEM

vrije universiteit Amsterdam
Postgraduate Opleiding
IT Audit, Compliance & Advisory
Addressing SIEM
Determining focus areas

and their coverage
within IT risk frameworks
Authors Jarno van de Moosdijk vandemoosdijk.jarno@kpmg.nl

Daan Wagenaar wagenaar.daan@kpmg.nl
Status Final
Date 30 August 2015
PGO IT Audit, Compliance & Advisory
1 Contents
2 Management summary ......................................................................................................................... 4
3 Introduction .............................................................................................................................................. 6
3.1 Background ..................................................................................................................... 6
3.2 Research motivation .................................................................................................... 7
3.3 Scope .................................................................................................................................. 8
3.4 Research questions....................................................................................................... 9
3.4.1 Central research question................................................................................................... 9
3.4.2 Sub question 1: SIEM explained ....................................................................................... 9
3.4.3 Sub question 2: SIEM in IT risk frameworks............................................................... 9
3.4.4 Sub question 3: SIEM case study ...................................................................................... 9
3.5 Research approach .....................................................................................................10
3.6 Related work .................................................................................................................11
3.7 Supervision ....................................................................................................................12
4 SIEM explained .......................................................................................................................................13

4.1 The definition of SIEM ...............................................................................................13
4.1.1 For threat management .................................................................................................... 14
4.1.2 For compliance ..................................................................................................................... 14
4.1.3 For forensic support........................................................................................................... 15
4.2 SIEM versus SOC ...........................................................................................................15
4.3 Shortcomings of traditional SIEM implementations ......................................16
4.4 SIEM focus areas ..........................................................................................................16
4.4.1 Knowing what to protect.................................................................................................. 17
4.4.2 Log management ................................................................................................................. 18
4.4.3 Correlation ............................................................................................................................. 19
4.4.4 Alerting .................................................................................................................................... 20
4.4.5 Responding & Evaluating ................................................................................................. 20
5 SIEM in IT risk frameworks...............................................................................................................22

5.1 Finalising SIEM focus areas......................................................................................22
5.1.1 Initial focus areas ................................................................................................................ 22
5.1.2 Validation by field-experts .............................................................................................. 25
5.2 Requirements per focus area ..................................................................................26
5.2.1 Organisational requirements ......................................................................................... 26
2
5.2.2 Operational requirements ............................................................................................... 27

5.2.3 Log management requirements .................................................................................... 28
5.2.4 Correlation requirements ................................................................................................ 30
5.2.5 Alerting requirements ....................................................................................................... 31
5.2.6 Responding requirements ............................................................................................... 31
5.2.7 Evaluation requirements ................................................................................................. 32
5.3 Selected IT risk frameworks ...................................................................................33
5.4 Mapping of selected frameworks ..........................................................................35
5.4.1 Observations of framework mapping ......................................................................... 35
6 SIEM case study ......................................................................................................................................38

6.1 Field research methodology ....................................................................................38
6.2 Company 1: A large financial institution ............................................................39
6.2.1 Background information .................................................................................................. 39
6.2.2 Results...................................................................................................................................... 39
6.2.3 Result discussion ................................................................................................................. 41
6.3 Company 2: A small financial institution ............................................................43
6.3.1 Background information .................................................................................................. 43
6.3.2 Results...................................................................................................................................... 43
6.3.3 Result discussion ................................................................................................................. 46
6.4 Comparison of organisations ..................................................................................48
7 Synthesis ..................................................................................................................................................50
8 Research questions revisited ...........................................................................................................52

8.1 Sub question 1 ..............................................................................................................52
8.2 Sub question 2 ..............................................................................................................52
8.3 Sub question 3 ..............................................................................................................53
8.4 Answering the central research question ..........................................................54
8.5 Future work ...................................................................................................................55
9 Acknowledgments ................................................................................................................................56
10 Bibliography ...........................................................................................................................................57
11 Appendices ..............................................................................................................................................61
11.2 Interview questions....................................................................................................67
11.3 Mapping of selected organisations .......................................................................68
3
2 Management summary
With hacker attacks and data breaches happening each day, organisations see the need for
employing detective security controls instead of solely relying on preventive controls. One such
detective security control is Security Information and Event Management (hereafter: SIEM). A
SIEM can be used by an organisation to detect and alert on predefined unwanted activities within
its IT environment.
Our research focusses on what areas should be addressed when implementing SIEM, how these
areas are covered within selected IT risk frameworks and to what extend this matches current
SIEM implementations. By means of literary research and field-expert input, a model is created
that covers SIEM focus areas. Each focus area contains minimum requirements that are essential
for a successful and effective SIEM implementation. The formulated focus areas are: organisations
requirements, operational requirements, log management, correlation, alerting, responding and
evaluating. Our model is used to assess the coverage of SIEM within the selected IT risk
frameworks and to determine how financial institutions address SIEM.
Frameworks mapped on focus areas

Organisational
requirements
Operational
Evaluation
requirements
Responding Log management
Alerting Correlation
ISO COBIT NIST PCI-DSS ISF
Graph 1: Focus area coverage
The graph above shows to what extend the selected IT risk frameworks address our model. It
clearly shows that while the organisational requirements (i.e. general IT risk management) are
addressed by most of the selected frameworks, the operational requirements are not. The
operational requirements focus area covers specific requirements essential to any SIEM
implementation such as the definition of risk scenarios and use cases.
4
During our case study we assessed the SIEM implementation of two financial institutions. For the
first organisation, threat management was the main driver to implement SIEM. The main driver
for the SIEM implementation of the second organisation was compliance with legislative
requirements from DNB. The first organisation addressed most of the requirements specified
within our model. They defined risk scenarios and use cases, whereas the second organisation only
implemented generic predefined rules due to lack of available resources. Both organisations
acknowledge the fact that one needs to define risk scenarios and use cases in order to increase
SIEM effectiveness.
Based on literature research, field-expert input and the performed case studies, it becomes evident
that risk scenarios and use cases are fundamental for SIEM implementations. Selected frameworks
inadequately cover these aspects. Therefore, we conclude that the selected frameworks do not
match current SIEM implementations.
5
3 Introduction
3.1 Background
We are living in a society where technology plays a more important role each year. Everything is
being connected to the Internet: from cars to household appliances. Not just personal devices are
being connected. Organisations are being interconnected over the Internet as well. With
everything being interconnected, the impact of a security breach in an organisations IT
environment becomes bigger as well. Compromise of a single asset, might lead to an infection
spreading throughout the entire organisations network. Where hackers in the early 1970s used
whistles to make free long-distance phone calls (Times, 2000), nowadays they are targeting
organisations for amongst others: their intellectual property, financial gain or to cause
reputational damage.
One would expect an organisation to expose only the highly

necessary services (i.e. e-mail and corporate website) to the
Internet for use by its customers and employees. However, often it
does not stop with just the bare necessities being exposed.
Knowingly or unknowingly, even ICS/SCADA back-end systems of
national water management systems, power grid management
networks and large industrial organisations are exposed. A recent
attack on a German steel mill resulted in a breach where attackers
caused massive physical damage to the factory and its equipment,
in addition to disrupting the organisations IT infrastructure (BBC,
2014).
According to the annual Verizon data breach investigation report

(Verizon, 2014) and various other data breaches (Krebs, 2014), it
is proven that even the largest of organisations are vulnerable for
cyber attacks. Gartner states that organisations are failing at early
breach detection, with more than 92% of breaches undetected by
the breached organisation (Gartner, 2014). Cisco states, in a recent
cyber security report, that organisations should assume they have
been hacked or assume that they will be at some point in the future
(Cisco, 2014).
In response to the high number of attacks and their increased level

Figure 1: Cyber kill chain
of sophistication, Lockheed Martin developed the cyber kill chain
6
(figure 1) (LockHeed Martin Corporation, 2011). A kill chain, originally described by the American
military, is a structured process that details the components required for an adversary to obtain
the desired results such as espionage or fraud. The cyber kill chain is specifically geared towards
cyber attacks and represents the process that constitutes a successful infiltration. Security
professionals can use the cyber kill chain to identify risks and patterns as well as deploying
appropriate defences for the different stages of an attack more efficiently. Preventing a single
phase of the cyber kill chain disrupts the entire process and causes an attack to fail.
Traditional IT security primarily focuses on the Delivery component of the cyber kill chain by
implementing heavy security measures on the perimeter of the organisations network
(preventive security measures). However, solely focusing on perimeter security is not sufficient
any more. Attackers perform advanced reconnaissance, build specialised weapons, and exploit
zero-day vulnerabilities. Standard anti-virus solutions and Intrusion Detection Systems might not
recognize these specialised weapons, and carefully crafted (spear-) phishing e-mails could pass an
organisations spam filter. As a result, organisations need to implement additional security
measures in order to protect themselves. One such particular (detective) measure has gained
significant popularity over the past years and is named Security Information and Event
Management (hereafter: SIEM).
SIEM allows organisations to monitor status and events from all over their IT landscape. The
collected information can in turn be used to perform detailed analysis and correlation to identify
irregularities and security incidents. Timely identifying security incidents allows for appropriate
measures and actions to be taken. In addition to detection, the information provided by SIEM can
be used to respond to security incidents efficiently. This disrupts the cyber kill chain, stopping
potential breaches to an organisations network. However, implementing an effective SIEM
solution depends on many different aspects and requirements. Both aspects and requirements will
be covered in detail throughout this thesis.
3.2 Research motivation

As covered in the previous section, solely relying on preventive measures is not sufficient to
protect an organisation against security breaches. A popular measure to detect and respond to
security incidents is SIEM. According to research done by (Houten, 2009), many large Dutch
organisations have incident detection on their agendas. The majority of the sampled organisations
succeed in collecting and storing security information in their SIEM tooling. However, few of them
actually succeed at implementing formalized monitoring policies and performing active
monitoring. Furthermore, the research shows the main reasons for failing in this regard is due to
lack of management commitment, lack of sufficient knowledge and the inability to implement the
required technical configuration. (SC Magazine UK, 2012)
7
Multiple whitepapers state that a correctly implemented SIEM solution is crucial for the success
against cyber threats. However, no holistic and structural methodology exists for implementing
SIEM and its required tooling (PvIB, 2011), (Gartner, 2014), (Schinagl & Schoon, 2014). Several
commercial companies like HP, IBM, and McAfee try filling that gap by offering their own SIEM
related software but mainly end up solely pushing their own products. Papers and blog posts
published by these vendors are rather high-level and vague, containing little to no detail regarding
a holistic SIEM implementation approach (HP, 2014), (IBM, 2014).
Our research focuses on how various IT risk frameworks cover SIEM as a holistic subject and how
this subject is addressed in real-world implementations. We aim to achieve this by first providing
a clear definition of SIEM as well as the components and processes it entails. Secondly, we compile
an overview of focus areas, each containing a minimum set of requirements that should be
addressed when implementing SIEM. IT risk frameworks will be plotted to our model, showing
their coverage of SIEM related aspects. The model will also be used as guidance of our empirical
research in which we assess how SIEM is addressed in practise by multiple financial institutions.
3.3 Scope
The scope of this research entails setting up a model containing relevant focus areas and minimum
requirements that need to be addressed when implementing SIEM for the use of threat
management (i.e. detection of security incidents).
Furthermore, our research entails the assessment of IT risk frameworks listed in section 5.3, based
on our model. Additional IT risk frameworks may exist, although have not been selected due to the
time-boxed approach of this research.
Our research does not provide an exhaustive model that can be used to test the effectiveness of a
SIEM implementation. It can however be used as a guideline for determining success factors of a
SIEM implementation, covering the minimum requirements on an organisational-, operational-,
and technical level.
Empirical research is performed in which we assess how multiple financial institutions address
the topic of SIEM. The assessment shows how selected financial institutions address the
requirements of which our model consists. Performed research does not entail a full in-depth audit
of the organisation and its SIEM environment, but will be used to verify the applicability of our
model.
Also, note that since the concept Security Operations Centre (SOC) is closely related to SIEM, our
research covers crossover domain subjects. However, we do not focus on SOC requirements or any
of its areas, nor incident- and response management.
8
3.4 Research questions

Our research motivation and the illustrated problem statement can be translated into specific
research questions. The following central research question and supporting sub-questions are
defined that reflect the objectives of our research.
3.4.1 Central research question

The central question of our research is:
How do IT risk frameworks address SIEM and to what extend does this match current SIEM
implementations?
3.4.2 Sub question 1: SIEM explained

The first supporting sub-question is descriptive and is defined as follows:
What is SIEM and why is it relevant?
3.4.3 Sub question 2: SIEM in IT risk frameworks

The second supporting sub question is analytic and is defined as follows:
What are focus areas of SIEM and how do IT risk frameworks address them?
3.4.4 Sub question 3: SIEM case study

The third supporting sub questions is also analytic and is defined as follows:
How is SIEM addressed in practice?
9
3.5 Research approach

Our research consists out of several components that together outline the process. The visual
representation below shows the complete research approach.
4.
Expert
validation
2.
1. 3, 5 & 7. 8. 9.
Literature
Plan Analyse Conclude Document
research
6.
Empirical
research
Figure 2: Research approach
The plan phase (1) involves gathering background information and determining the final research
questions. Moreover, the research approach is determined and discussed in detail. The planning
phase and the defined research (sub) questions provide the basis for our literature research (2).
During the literature research (2) information is gathered from relevant publications,
whitepapers, testimonials and articles. This phase focusses on two aspects. First, ascertaining a
transparent and comprehensive definition of SIEM. Second, identifying shortcomings, common
issues organisations face and other insights regarding current SIEM implementations. The
information collected during the literature research serves as input for determining key SIEM
focus areas and minimum requirements of which our model comprises (3).
The results of the literature research are analysed (3) and together with our own experience, an
initial version of our model containing key SIEM focus areas and their minimum requirements is
created. Subject matter experts (4) are consulted for rigorously evaluating and validating our
initial model. The feedback provided by the subject matter experts serves as input for the second
round of analysis (5) in which our model is finalised.
With our final model containing SIEM focus areas and minimum requirements, the empirical
research phase (6) commences. This phase consists once again out of two aspects. First, we select
several popular IT risk frameworks that are then mapped to our focus areas. This shows how each
10
selected framework covers our set of minimum requirements that address the implementation of
SIEM. Second, we determine how SIEM is addressed in practice by performing multiple case
studies at financial institutions. The case studies consist out of multiple interviews in which we
match our model to the way each financial institution addresses SIEM.
The results of the empirical research phase (6) are used as input for the final round of analysis (7).
Throughout this phase, we analyse the results of the literature- and empirical research to identify
findings and explanations pertaining to our research questions. We then continue drawing up our
conclusions (8) using the output of the performed analysis.
Finally, the document phase (9) is used for formalizing and documenting our findings and
conclusions.
3.6 Related work

The term SIEM has been around since 2005 (Dorigo, 2012) and has been subject to various papers.
In the recent years, the topic has received more and more attention by vendors who want to push
their products. Papers and blog posts published by these vendors are rather high-level and vague,
containing little to no detail regarding a holistic SIEM implementation approach (HP, 2014), (IBM,
2014).
A recent paper by fellow VU IT Audit, Compliance & Advisory students (Schinagl & Schoon, 2014)
described the importance of an effectively operating SIEM solution for the early detection of
security breaches.
Various papers have been published on subtopics of SIEM. Examples are: The complete guide to
log and event management (Chuvakin, 2010), Effective use case modelling for security information
and event management (SANS, 2009), Attack life cycle use case methodology (HP, 2014). Other
papers are focussed on Security Operations Centre (SOC), and touch upon the subject SIEM (PvIB,
2011).
None of the papers described above provide a holistic view addressing key focus areas of SIEM,
including organisational and operational requirements. In this thesis, we set up a model that
covers focus areas including a minimum set of requirements (focus points) that should be
addressed when implementing SIEM. In addition, we assess multiple IT risk frameworks based on
our model, and look at how financial institutions address the topic of SIEM based on our model.
11
3.7 Supervision
Both our employer KPMG as well as ControlSolutions International and the VU supported and
guided our research, for which we are thankful. The following persons guided us during our
research:
KPMG:
Ing. J.A.M. Hermans RE
Partner, IT Advisory, Risk Consulting, KPMG
Hermans.John@kpmg.nl
ControlSolutions International:
R. Bardoel MSc RE CISA CISSP OSCP
Partner, ControlSolutions International
Ralf.Bardoel@controlsolutions.nl
Vrije Universiteit Amsterdam:

Dr. R. Matthijsse RE
Associate Professor, Vrije Universiteit
matthijsse.rene@gmail.com
12
4 SIEM explained
In this chapter, we cover the definition of SIEM and provide insight why organisations require
SIEM. We then move on to discuss the importance for organisations to know and identify what to
monitor (i.e. their crown jewels). Throughout this chapter, we use expert opinions, surveys and
whitepapers to supplement our own experience and provide factual support. During this chapter
we aim at providing a solid foundation for the conclusion to the first research question:
4.1 The definition of SIEM

The term Security Information and Event Management (SIEM) finds its origin from the
combination of Security Information Management (SIM) and Security Event Management (SEM).
Where SIM focuses on the collection and long-term storage of log files, SEM focuses on real-time
monitoring of (suspicious) behaviour. SEM does this by aggregating and identifying interesting log
entries (events), often collected by a SIM implementation. The term SIEM was coined in 2005 and
was a result of combining the functions SIM and SEM in addition to the ability of correlating
multiple events (Dorigo, 2012).
A SIEM collects log files and security information from internal- (i.e. server-, network- and
application logs) and external sources (i.e. threat intelligence sharing). Event correlation is used
to detect and alert on, by the organisation defined, unwanted activities within the network. An
organisation can use the information within the SIEM to effectively respond to detected security
incidents.
Figure 3: Overview of aspects related to SIEM
13
Figure 3 covers the various aspects of SIEM. The main focus areas that define the fundaments of
every SIEM are: 1) log management, 2) correlation, 3) alerting, 4) responding. These focus areas
will be covered extensively in section 4.4.
There are multiple drivers for an organisation to implement SIEM. Based on our experience in the
field and according to literary study (Netmonastery, 2014), the three main drivers why
organisations implement SIEM are:
1. Threat management (i.e. SIEM as a detective security control);
2. Compliancy;
3. Forensic support.
Each of these drivers will be further discussed in the following sub-sections. In our research, we
focus on organisations that implement SIEM for threat management, as defined in section 3.3.
4.1.1 For threat management

A SIEM focussed on threat management monitors an organisations IT environment and is able to
detect risk scenarios, common attacks as well as attack paths defined by the organisation itself.
The output of a SIEM focussed on threat management is used by the Security Operations Centre
(hereafter: SOC) to respond to detected security incidents.
Using event correlation, relations are established between events from different sources on the
network, such as server logs, IDS/IPS and application. A baseline of normal behaviour is set up for
various events, so that alarms can be raised when deviations on the normal behaviour pattern
occur. Threat intelligence is gathered from various internal and external sources. This enhances
the detection rate of attacks in an ever-changing threat landscape. Incident management functions
are embedded into SIEM to support the IT security department.
Joff Thyer, (Thyer, 2015), believes that in order to successfully defend your network against
hackers and other threats, you need to have the presumption that it is already compromised. This
means that adversaries already found a way around or through existing preventive security
measures and have established a foothold in an organisations environment. He further states that
real-time detective capabilities are required to identify these adversaries and to ensure a holistic
approach for defending an organisations most important assets, also known as an organisations
crown jewels.
4.1.2 For compliance

SIEM focussed on compliance is built for log management, reports and custom compliance
requirements that apply for an organisation. One can think of compliance requirements imposed
by law or sector specific regulators. Examples of this are the Payment Cards Industry Data
Security Standard (PCI-DSS) and Toetsingskader informatiebeveiliging by De Nederlandse Bank
14
(DNB, 2014). PCI-DSS is a security standard imposed on all organisations that handle credit card
information. Organisations are required to implement logging and monitoring mechanisms so they
are able to detect and/or minimise the impact of a data compromise (PCI Security Standards
Council, 2013).
However, when talking about SIEM implementations covering compliance needs, we experience
that many organisations implement SIEM only to be compliant with frameworks such as PCI-DSS.
Our experience is that organisations seldom implement or use SIEM for actual compliance
reporting, an observation further corroborated by a survey done by (EiQ Networks, 2013). This
survey states that 35% of interviewees say that they implemented SIEM to make auditors happy
and comply with regulation.
4.1.3 For forensic support

The third driver for implementing SIEM can be found in the forensics field. Event log information
of a wide variety of devices can be stored tamperproof using SIEM. The information available
within SIEM is very valuable from a forensic perspective and can greatly aid a forensic analyst in
his or her investigation. SIEM allows forensic analysts to search within logs of many systems in a
centralized way, without the need of re-collecting the log files of compromised systems that might
have been tampered with.
4.2 SIEM versus SOC

As mentioned in the previous chapter, vendor branding and marketing resulted in organisations
often viewing SIEM as only the implementation of a tool. In fact, according to (Chuvakin, So you got
that SIEM. NOW what do you do?, 2011), several vendors have gone as far as advertising their SIEM
products as a SOC-in-a-box solution.
A SOC is a collection of people, processes

and technologies that are responsible for
detecting, containing and remediating
security threats and incidents within an
organisation. Many different flavours of
SOCs exist as described by Schinagl &
Schoon (Schinagl & Schoon, 2014). A SOC
uses amongst others, a SIEM as their
source of information. Schinagl & Schoon
conclude that a correctly working SIEM is
crucial for the effective operation of a SOC,
resulting in the early detection of security
breaches. SIEM enables the people and
Figure 4: Position of SIEM
15
processes of a SOC to function correctly. As such, SIEM is an integral part of a SOC but cannot be
considered the SOC itself. Figure 4 illustrates our view on the position of SIEM including the
previously defined aspects and their overlap with a SOC.
4.3 Shortcomings of traditional SIEM implementations

As discussed in the sub section of 4.1: The definition of SIEM, many organisations see the need and
value of implementing SIEM tooling. However, due to the marketing and branding by vendors, the
term SIEM is often associated with a product and has therefore become a synonym for a tool
performing SIEM functions. However, an effectively operating SIEM requires a lot more than just
installing a tool.
It was only after implementing these tools that organisations realized just having the tool installed
is not good enough. According to a survey done by (SC Magazine UK, 2012), organisations are
struggling with effectively managing their SIEM implementations. 59% of interviewees stated that
their implementations failed due to time constraints in monitoring for security incidents and
another 40 percent voiced concern about the time it takes to analyse logs and data.
Another survey by (EiQ Networks, 2013) shows that organisations struggle with the fine-tuning of
their SIEM and that it can take months to obtain useful data. 44% of interviewees state that it took
a few weeks to more than a month to deploy their latest SIEM product. 52% of interviewees require
two or more full-time employees to manage their current SIEM deployment. Another 25% of
interviewees have invested more than a month in professional services since deploying their
current SIEM product.
Netwrix performed a survey (Netwrix, 2014), showing that 61% of interviewees think that their
SIEM reports lack important information. 55% of interviewees even state SIEM reports are hard
to interpret. 75% of interviewees state that SIEM reports contain ample amounts of noise data.
Looking at the results of the surveys, we argue that part of the problem originates in the lack of
adequate preparation by organisations itself. Furthermore, we experience that many
organisations underestimate implementing SIEM. An effective implementation of SIEM requires
an organisation to think about-, and take measures in the areas of: knowing what to protect (i.e.
crown jewels), log management, correlation, alerting, responding and evaluating. We call these
areas the focus areas of SIEM. All focus areas are covered in-depth in the next section.
4.4 SIEM focus areas

This section covers the main focus areas of SIEM that together form our model. The minimum
requirements that need addressing when implementing SIEM will be defined in a later stage. The
focus areas will cover requirements on governance, people, process, and a technical level.
16
4.4.1 Knowing what to protect

When an organisation grows, its IT environment grows as well. Services are added and removed.
If an organisation does not have an up-to-date overview of its IT environment, it is a challenge, if
not impossible, to protect its infrastructure. Inventory management of authorised and
unauthorised devices and software are controls number #1 and #2 in the SANS top 20 model
(SANS, sd). This model covers critical security controls for effective cyber defence.
An organisations network easily consists of hundreds, if not thousands of systems running various
services in often-different locations around the world. All these systems and services generate log
files. It is impossible for an organisation to collect log files of all systems and at the same time
perform real-time analysis and correlation. As such, a more scoped approach is required: a type of
approach that inherently raises several important questions. Namely, how does an organisation
know which services or assets to protect? And, what services or assets should be focussed on?
To answer these questions, an organisation needs to know what its crown jewels are. What is the
most important asset or information that is owned by the organisation? Crown jewels can be
identified by performing a risk analysis on organisational level, in other words: an organisations
strategy. Identified risks that have a negative impact on an organisations strategy or primary
processes need to be addressed, as they have impact on an organisations crown jewels.
Crown jewels of organisations can vary per sector. For example, the crown jewels of a bank differ
from the crown jewels of a tech company. The latter most likely has intellectual property and
detailed designs that need to be protected from being stolen by adversaries and hackers. The
crown jewels of a bank are its customers data, plans on future product changes (i.e. changing
interest rates) as well as their image towards the outside world. A breach might result in bad
publicity and customers moving to competitors.
Once an organisation has defined her crown jewels, one knows what to protect. The next step for
an organisation is defining from what to protect her crown jewels. This is done by defining risk
scenarios. Organisational stakeholders, with support of IT security personnel (i.e. SOC employees),
define risk scenarios. Risk scenarios describe undesirable actions to crown jewels and include
common attacks (i.e. DDoS on online services) and attack paths (i.e. reconnaissance using a port
scan). Some risk scenarios might be imposed by legislative requirements depending on the sector.
In addition, organisations can also choose to implement detection of activities within each step of
the cyber kill chain, as covered in section 3.1.
Risk scenarios are translated into use cases. One risk scenario might result into multiple use cases.
Use cases are defined as a set of rules to which logs are correlated to be able to detect a certain
event. According to (Dorigo, 2012), use cases represent a goal or a task.
17
An organisation knows what logs to collect from what devices based on the information required
by use cases and the rules they consist of. Every rule can require different log sources and events.
For SIEM to work correctly, all logs required by use cases and rules should be gathered, normalised
and available to the SIEM tooling. Log normalisation guarantees that identical fields within
different log sources can be correlated to each other. One can hereby think of the same
representation of an IP address for log files gathered from network devices and servers, or the
usage of the same time stamp format for all log entries that are stored within SIEM tooling.
One example of a use case that many regulatory standards require is tracking the activities of
privileged users, those with administrative privileges. However, before you can implement this
SIEM use case, you will need to know who these users are (including all of their accounts), as well
as collecting the logs of the systems they access. For translating this use case into SIEM
components, models such as the one proposed by (SANS, 2009) can be used. In this example case,
we base our use case model as proposed by (SANS, 2009) and can be seen in table 1.
Privilege escalation
Goal Detect unauthorized user access attempts, including privilege escalation.
Data sources Log data from Active Directory, host-based intrusion detection, file
integrity monitoring, and change management tooling.
Relevant systems Hostnames / IP addresses of relevant systems (i.e. crown jewels).
Importance Critical.
Required speed Real-time.
Trigger Trigger on a privilege escalation (i.e. user added to Administrators
conditions group) with no corresponding change request.
Alert Method E-mail and live dashboard update.
Table 1: example use case
4.4.2 Log management

We argue that log management is an integral part of SIEM because log entries are the single
greatest source of information for SIEM implementations. Though highly crucial, solely collecting
and aggregating logs at a central location is not enough. Figure 5 shows the one-to-many relation
between log files and log entries (events).
18
Figure 5 shows that collecting log entries from

multiple log files and from numerous sources can
quickly escalate into a large number of log entries. An
example of this is when a user logs in to his
workstation. This process generates log entries on
multiple servers such as the Active Directory server
authorised to process the request, the file server
hosting the home directory of a user, SharePoint
server connections, etc.
As the amount of log entries quickly grows, adequate

log management becomes more and more important
to support the integrity, structured handling and
timely processing of both local and centralized log files
and their entries. Note that throughout our research Figure 5: One-to-many relation between log
we discuss log management from a security files and log entries
perspective. As such, we do not focus on the
availability or operational aspects of log management.
4.4.3 Correlation
We argue that correlation is an integral
part of SIEM because it is required to
extract sensible information spread
amongst different log entries. Figure 6
shows the relation between multiple log
entries and correlation. Furthermore, the
figure introduces Use cases, which are
required to make meaningful and
structured correlation decisions.
Correlation of log entries is performed

based on use cases. Every use case
consists of one or multiple rules that
detect an unwanted event that is defined Figure 6: Relation between log entries, correlation and use cases
using risk scenarios by the organisation.
To trigger on use cases, one typically needs to correlate multiple log entries from one or multiple
sources. An example of a use case would be to detect successful brute-force attempts. Hereby one
does not look at one or two single failed login attempts, but multiple failed login attempts on one
19
or more systems by a single user, followed by a successful login attempt. When this happens, it
might indicate that a brute-force attack is in progress. Detecting multiple login attempts on
different systems can only be achieved by correlating log entries from multiple sources.
4.4.4 Alerting
We argue that alerting is an integral part of SIEM because alerting on out of the ordinary actions
(defined in risk scenarios, and implemented by use cases and rules) is the core purpose of a SIEM
focussed on threat management. Figure 7 shows the relation between log files, the correlation of
log entries based on use-case and the resulting alerts that follow.
Figure 7: Alerts generated by correlation rules
Two types of alerting exist, namely passive and active alerting. Passive alerting can be seen as
regular reporting where dashboards contain data about specific areas of interest. The SOC team
monitors dashboards for suspicious behaviour. Active alerting is often used by high-impact use
cases that trigger alerts. Hereby an alert is generated and pushed automatically to a SOC employee
(i.e. by a highlighted message in the SIEM tooling dashboard, a pop-up, or an e-mail/SMS message).
4.4.5 Responding & Evaluating

Figure 8 illustrates that alerts can result into the creation of an incident. There are two ways that
alerts can turn into an incident, namely automatically and manual. Alerts can automatically be
transformed into incidents if they are based on well-defined and mature use cases. For instance,
use cases such as the use of high privileged generic accounts (i.e. the generic Domain
Administrative account that should only be used in disaster recovery or emergencies).
20
Figure 8: Incidents generated by alerts
We argue that responding is an integral part of SIEM as it connects incident information and
alerting with the SOC and its analysts. Most alerts require manual analysis by a SOC analyst. After
analysing the event, the analyst decides the appropriate follow-up for an alert. When an alert is a
true positive, an incident might be created. If an alert is, for instance, generated by a configuration
error, one might choose to register the event as a false positive and link the alert to an existing
change ticket that was created by the employee involved with configuration error.
If analysis by a SOC analyst shows that an alert triggered by a high-impact use case concerns a true
positive, the organisations generic incident management process is followed. This guarantees the
involvement of the right organisational stakeholders as well as the escalation to the appropriate
staff.
Furthermore, we argue that evaluating is also an integral part of SIEM because it allows for lessons-
learned to be fed back into the system. In other words, experiences gained from handling incidents
or false-positives can serve as input for the addition or fine-tuning of use cases and correlation
rules.
21
5 SIEM in IT risk frameworks

We have established a concise and clear definition of SIEM, discussed the different drivers of its
implementation and detailed the importance of knowing what to protect. This chapter aims at
providing a solid foundation for the conclusion to our second research question:
We start with finalising the main focus areas that need to be addressed when implementing SIEM,
as well as minimum requirements for each of the areas. We do this based on literary research, field-
expert opinions as well as our own field experience. This research will result in a model containing
a set of minimum requirements that should be covered when implementing or assessing a SIEM
implementation.
This will be followed by a selection of IT risk frameworks that will be mapped to our model,
assessing to what extent they cover SIEM.
5.1 Finalising SIEM focus areas

This section describes the process of finalising our SIEM focus areas that our model is based upon.
For each of the focus areas a set of minimum requirements that need to be addressed when
implementing SIEM is defined. The model will function as the base to which we map selected IT
risk frameworks listed in section 5.3.
5.1.1 Initial focus areas

In chapter 4 we provided our definition of SIEM as well as its technical focus areas, including: log
management, correlation, alerting, responding and evaluating. We will use these focus areas as
guidance and structure in determining our initial minimum requirements regarding SIEM. In
addition to technical requirements, we will also cover a set of minimum requirements that need to
be addressed on organisational- and operational level. This section covers reasoning why each
focus area is included. Figure 9 expands figure 4, providing a holistic view of all focus areas, and
their relevance to SIEM.
22
Figure 9: Position of SIEM and its focus areas
Literary research and our own experience was used to create the initial version of our model
containing SIEM focus areas and minimum requirements that need to be addressed by an
organisation when implementing SIEM. Below we explain the relevance of each focus area. Section
5.3 contains an overview of minimum requirements per focus area, as well as reasoning why every
requirement is incorporated.
Organisational requirements
The effective working of SIEM tooling depends on different parts within the organisation, as well
as a variety of organisational processes. An organisation needs a certain maturity before it can
start thinking about implementing a SIEM. One can hereby think of the existence of an information
security policy, (IT) risk management, incident management, as well as processes around the SOC
itself. Requirements in this focus area cover an overview of minimum organisational requirements
that need to be addressed by an organisation for a SIEM to function.
Operational requirements
We define operational requirements as prerequisites that directly link to the implementation and
effective working of SIEM tooling. One can hereby think of setting up requirements that SIEM
tooling needs to satisfy, SIEM tooling selection, and the definition of risk scenarios and use cases
on a logical level.
23
Log Management
As discussed in section 4.4.2, log files are the single greatest source of information for SIEM
implementations. Requirements within this focus area are geared towards determining what to
collect, log integrity, from where to collect, how to store logs and doing it all in a structured and
sustainable way.
Correlation
Correlation of log entries is what makes a SIEM strong in only generating true-positive alerts. As
discussed in section 4.4.3, good Use cases are the fundaments of effective correlation rules.
Requirements in this focus area cover requirements regarding the technical implementation of
logical use cases, as well as the management of the use cases.
Alerting
As discussed in section 4.4.4, the core purpose of a security tailored SIEM is alerting on suspicious
events triggered by Use cases. Requirements in this focus area cover requirements regarding the
technical implementation of triggers and alerts within a SIEM.
Responding
As discussed in section 4.4.5, the SOC responds to alerts that are generated by the SIEM. We do not
cover SOC-specific processes, however, we do include generic requirements regarding responding
to alerts. One can hereby think of the automatic creation of an incident when a high-impact use
case triggers an alert.
Evaluating
This focus area consists of evaluating elements that will take place post-alert. One can hereby think
of a root-cause analysis, lessons learned, the adaptation of use cases based lessons learned, and
reporting requirements.
24
5.1.2 Validation by field-experts

By discussing our model of focus areas with minimum requirements with field-experts, we aim to
determine their validity. As such, requirements initially identified but later deemed inadequate
will not be listed. In addition, any gaps or missing requirements as a result of the interviews are
further analysed for potential inclusion.
Interviews have been conducted with multiple field-experts. We have discussed the validity of our
model of minimum requirements with the following persons:
Name Function Organisation

P. K. MSc RE CISA Risk & Compliance, Booking.com
Corporate Security
D. d. G. MSc RE CISA Director KPMG Advisory N.V.
R. B. MSc RE CISA CISSP OSCP Partner Control Solutions International
P. v. H. MSc RE Senior IT auditor SHV Holdings N.V.
J. d. W. MSc Senior Consultant KPMG Advisory N.V.
J. H. MSc CISSP Senior Consultant KPMG Advisory N.V.
All interviewees agreed upon our selection and definition of main SIEM focus areas. Section 5.2
contains our final model built up of the focus areas including minimum requirements that should
be addressed when implementing SIEM. Every requirement contains reasoning why it is
incorporated in our model.
25
5.2 Requirements per focus area

The following subsections list the minimum set of requirements per focus area that should be addressed when implementing SIEM. This list is the result of literary
study, our own professional experience as well as validation by multiple experts in the field. The overview of requirements will function as guideline for our further
research.
5.2.1 Organisational requirements

This section covers the minimum set of organisational requirements that should be addressed by an organisation when implementing SIEM.
# Requirement Explanation References
ORR1.1 The reason for implementing SIEM is understood and supported by C-level
No additional information The leadership of an organisation understands why a SIEM implementation is (SANS, 2010)
needed and supports its implementation by making available budget and
resources.
ORR1.2 Organisational requirements for SIEM are defined
Based on: Organisational requirements are defined for an effective operation of a SIEM. (SANS, 2009)
Crown jewels One can think of requirements that result from the crown jewels that need to be (Gartner, 2014)
Risk appetite monitored (i.e. intellectual property or sensitive processes), the risk appetite of (AccelOps, 2013)
Legislative or regulatory requirements an organisation and legislative requirements by regulators. Examples are
Reporting requirements mandatory sector requirements or log retention requirements.
ORR1.3 Roles and responsibilities regarding SIEM are assigned
This concerns roles and responsibilities within the SIEM Effective operation of a SIEM requires knowing what roles and responsibilities (Gartner, 2012)
team as well as within the organisation itself. (supporting) processes entail, and assigning them to the right people, giving
them the required mandate.
ORR1.4 Security team contains members with relevant skills
No additional information SIEM requires the right amount of people with the right skill sets. One should (Gartner, 2012)
have a clear understanding of what skill sets are required inside the core SIEM
team and what skills can be borrowed from other parts of the organisation.
26
5.2.2 Operational requirements

This section covers the minimum set of operational requirements that should be addressed by an organisation when implementing SIEM.
OPR1.1 SIEM tooling requirements are defined
No additional information Operational SIEM tooling requirements are defined based upon organisational (SANS, 2009)
requirements in which protection of crown jewels, risk appetite and legislative (Gartner, 2014)
or regulatory requirements are covered. Requirements should also cover
technical tooling requirements (i.e. correlation features).
OPR1.2 SIEM tooling is selected based on market research using defined requirements
No additional information Available SIEM tooling is evaluated based on defined set of requirements. SIEM (Gartner, 2014)
tooling is selected based upon this evaluation.
OPR1.3 SIEM tooling is implemented
No additional information SIEM tooling is implemented adhering to vendor and industry good practises,
complying with all organisational and operational requirements.
OPR1.4 Risk scenarios are defined
Based on: Risk scenarios are defined by organisational stakeholders in corporation with (HP, 2014)
Crown jewels and risk appetite the IT security staff, based on categories listed on the left. Risk scenarios define (SANS, 2009)
Legislative and regulatory requirements unwanted actions upon crown jewels and an organisations IT environment. (SANS, 2010)
Previous security incidents
OPR1.5 Use cases are defined

Based on: Use cases are defined on a logical level based on crown jewels that need to be (HP, 2014)
Risk scenarios protected. In addition, the cyber kill chain should be used to define use cases (SANS, 2009)
Common attacks and attack paths that detect common attacks. (SANS, 2010)
Cyber kill chain phases
27
OPR1.6 Threat intelligence (TI) is gathered from (external) sources

No additional information Threat intelligence from external sources is received from external sources. It (Gartner, 2014)
can be of great value in detecting new threats faster. TI gives better context,
making alert triage and incident investigation easier.
OPR1.7 TI and changes in threat landscape are used to create/modify use cases periodically
No additional information A process is in place that ensures periodic creation and evaluation of use cases (HP, 2014)
based upon changes in the threat landscape.
5.2.3 Log management requirements

This section covers the minimum set of requirements regarding log management that should be addressed by an organisation when implementing SIEM.
LM1.1 Log policies are defined
The log policy should at least include: A log policy is required to make sure log files are managed in a structured, (Kent &
Logs should be accurate consistent way across multiple devices throughout the organisations. Only when Souppaya , 2006)
Logs should be stored in a central and tamper-proof log files are handled correctly and uniformly can they be of optimal use within a (Constantine,
location SIEM implementation. 2014)
Log retention period per log type (based on needs & (Chuvakin, The
legislation) Complete Guide
Logs are time stamped to Log and Event
The same time source is used for clock synchronisation Management,
Log permissions are restrictive 2010)
Log format is determined (i.e. syslog, windows event (NSA, 2013)
logs) (Lewis, 2014)
System is not halted through lack of disk space
Logs contain originating source
LM1.2 Technical baselines are defined and implemented based on log policy
No additional information Defined log policies are translated into technical baselines. The actual content of
these baselines depend on the different systems and devices in use throughout
28
an organisation. As such, multiple baselines can exists but should all adhere to
the log policy.
LM1.3 Centralised log requirements are defined
And should at least include: Centralized logging is a key component for supplying your SIEM implementation (Kent &
Centralized logging facilities are implemented with relevant information in a structured and sustainable way. As such, we Souppaya , 2006)
Only authorised users have access to log files recognize the importance of centralized logging facilities and state that it should (Hanley &
Capacity of centralized logging infrastructure is be implemented as described by the log policy. Furthermore, centralized logging Montelibano,
managed helps in securing forensic evidence in case of compromised systems and log 2011)
tampering. (NSA, 2013)
(Lewis, 2014)
LM1.4 Log sources are derived based on use cases

Example log sources: The use cases describing security scenarios or compliance questions determine (Constantine,
Anti-virus which log sources should eventually be connected to the SIEM tooling. Plainly 2014)
Intrusion Detection Systems (IDS) connecting all log sources will result in a flooded SIEM unable to monitor that (Watson & Keary
Vulnerability scans what is important. et al, 2015)
Application logs (SANS, 2010)
Network devices (NSA, 2013)
LM1.5 To be collected log entries are defined based on use cases

No additional information Log sources are identified based on use cases. For each log sources, its parent (Watson & Keary
use-case should dictate the relevant log entries being produced. The actual et al, 2015)
filtering of relevant log entries can be done on the log source itself or within the (Infosec
available SIEM tooling. Institute, 2014)
(NSA, 2013)
LM1.6 Controls for checking if expected logs are received are implemented
No additional information Log sources never unexpectedly stop transmitting their log files to the (Taschler, 2015)
centralized logging facilities. If this happens, it should be noticed and further
investigation should follow.
29
LM1.7 Logs are normalised

No additional information Normalized logs allow for more effective and efficient searching by mapping all (Chuvakin, The
log-entry fields to a universal structure. For instance, Log A has the field Complete Guide
Timestamp for storing time information, log b uses the field time for this to Log and Event
information. Without normalization, searching for time related information had Management,
to happen twice, once for the field Timestamp and once for the field time. 2010)
The normalization process unifies these fields so the SIEM tooling is able to (Constantine,
search and correlate log entries across multiple log files and multiple devices. 2014)
5.2.4 Correlation requirements

This section covers the minimum set of requirements regarding event correlation that should be addressed by an organisation when implementing SIEM.
CO1.1 Use cases are translated into specific correlation rules within the SIEM tooling
No additional information Defined use cases are translated into specific correlation rules before they are (InfoSec Nirvana,
implemented into the SIEM tooling. The correlation rules are highly specialised 2012)
and tuned to the technical fundamentals of the chosen SIEM tooling. (Harrel, (Harrel, 2014)
2014) Describes the process of translating use cases into distinct areas
correlation rules.
CO1.2 Correlation rules are periodically checked on desired result

Checks on: As use cases evolve and expand, correlation rules are updated and reconfigured (InfoSec Nirvana,
Relevance to match these modifications. Furthermore, new attacks could be introduced by 2012)
Desired result hackers or identified by security specialists that require existing correlation (Harrel, 2014)
rules to be updated. As such, it is important that correlation rules are
periodically checked to guarantee that they work as intended and are still
relevant.
30
CO1.3 Modification of correlation rules is limited to authorised users only

No additional information Only authorised users are allowed to modify correlation rules. As correlation
rules are at the heart of detecting unwanted behaviour, unauthorised changes
could allow malicious activity to go unnoticed.
5.2.5 Alerting requirements

This section covers the minimum set of requirements regarding alerting that should be addressed by an organisation when implementing SIEM.
AL1.1 Triggers and alerts defined in use cases are implemented
No additional information Every use case contains triggers that when met, should generate an alert. Alerts (AccelOps, 2013)
can be delivered using various media (i.e. e-mail, SMS). (SANS, 2009)
AL1.2 Alerts contain relevant information
No additional information Alerts contain all relevant information required for a SOC analyst to analyse the (AccelOps, 2013)
event. One can think of, date, time, alert impact level, systems affected. (SANS, 2009)
5.2.6 Responding requirements

This section covers the minimum set of requirements regarding responding that should be addressed by an organisation when implementing SIEM.
RE1.1 Alerts are analysed by the SOC, and sent to incident management process if applicable
No additional information Alerts generated by the SIEM are analysed by the SOC. The SOC checks if alerts (Gartner, 2014)
are false- or true-positives and take appropriate follow-up steps based upon this (AccelOps, 2013)
(i.e. creation of an incident, following the incident management process).
RE1.2 Alerts generated by high impact use cases automatically generate an incident
No additional information Alerts generated by use cases that are marked as high impact, get priority by (Tipton &
analysts above alerts that are generated by use cases with lower impact. This is Krausse, 2008)
covered within the standard incident management process. (Gartner, 2014)
31
5.2.7 Evaluation requirements

This section covers the minimum set of requirements regarding evaluation that should be addressed by an organisation when implementing SIEM.
EV1.1 Adjust use cases if applicable
No additional information Use cases are fine-tuned based on root cause analysis and lessons learned. Use (HP, 2014)
cases that trigger as false-positives, should be fine-tuned to generate only true- (SANS, 2009)
positives in the future.
EV1.2 Reports are defined on the needs of the organisation
No additional information Organisational requirements regarding reporting are translated in (automated) (SANS, 2010)
reports based upon data within the SIEM tooling and other connected systems. (AccelOps, 2013)
32
5.3 Selected IT risk frameworks

Our selection of frameworks is based upon interviews with field experts, our own experience on
frameworks used at organisations, literary research and their relevance. The following
frameworks, in alphabetical order, have been selected:
Framework
2014 Standard of Good Practise for Information Security
COBIT 5
ISO/IEC 27002
NIST Framework for Improving Critical Infrastructure Cybersecurity
PCI DSS 3.0
Table 2: selected frameworks
2014 Standard of Good Practise for Information Security (ISF SoGP)

Information Security Forum (ISF) created The 2014 Standard of Good Practice for Information
Security (ISF, 2014), also dubbed The standard. As input for the standard, ISF used multiple IT
risk frameworks that were selected based on relevance, information security coverage, community
input, popularity and widespread usage. ISF SoGP provides full coverage of COBIT 5, as well as
comprehensive coverage of ISO 27000 series and the SANS top 20 critical security controls. As a
result, this standard comprises an extensive collection of information security control objectives.
Control Objectives for Information and Related Technology 5 (COBIT 5)

COBIT 5 is a framework that covers IT management and IT governance, and is developed by ISACA
(ISACA, COBIT 5 for Information Security, 2013). COBIT 5 provides guidance to help IT and
security professionals understand, utilize, implement and direct important information security-
related activities, and make more informed decisions while maintaining awareness about
emerging technologies and the accompanying threats. It includes requirements regarding logging
and monitoring.
ISO/IEC 27002
ISO/IEC 27002 is an information security standard that is developed by the International
Organisation for Standardisation (ISO) (ISO, 2013). It provides best practise recommendations on
information security management. It is commonly used within organisations in a wide variety of
sectors. It covers a broad spectrum of controls, including requirements regarding logging and
monitoring.
33
NIST Framework for Improving Critical Infrastructure Cybersecurity

NIST Framework for Improving Critical Infrastructure Cybersecurity is a framework developed by
NIST (NIST, 2014). The Framework is a risk-based approach to managing cybersecurity risk. It
reinforced the connection between business drivers and cybersecurity activities. It defines
requirements in the categories: Identify, Protect, Detect, Respond, and Recover. The category
Detect defines requirements regarding monitoring.
Payment Card Industry Data Security Standard 3.0 (PCI DSS 3.0)
PCI DSS 3.0 (PCI Security Standards Council, 2013) is a standard that is developed by the Payment
Card Industry Security Council. This standard is imposed on all organisations that handle credit
card data. It contains controls that should reduce credit card fraud. It also contains requirements
regarding monitoring.
34
5.4 Mapping of selected frameworks

This section covers the results of mapping the IT risk frameworks listed in section 5.1, to our
overview of focus areas and minimum requirements listed in section 5.3. The detailed mapping
that links specific IT risk framework controls to requirements can be found in appendix 10.1:
Mapping of selected frameworks.
Please note that when multiple frameworks map to a requirement, it does not mean that they cover
the exact same load. This due to the IT risk frameworks having a different level of granularity on
topics. We have included matches where wording may vary, although semantic intent is similar.
5.4.1 Observations of framework mapping

The mapping shows that frameworks vary in coverage of our selected focus areas and minimum
requirements. In graph 1 below, the results have been plotted into a spider graph.
Frameworks mapped on focus areas

Organisational
requirements
Operational
Evaluation
requirements
ISO COBIT NIST PCI-DSS ISF
Graph 2: Focus area coverage
ISF SoGP covers the most requirements defined in our focus areas. It scores particularly well in
organisational requirements as well as log management, correlation, alerting and evaluation. As
ISF SoGP is a superset of COBIT 5 and provides comprehensive coverage of ISO 27000 series, we
knew beforehand that it would score better than the two separately.
35
The spider graph shows that all frameworks lack coverage of the operational requirements focus
area. This area covers requirements on an operational level specific to the SIEM implementation.
One can hereby think of specific technical SIEM requirements that result from organisational
requirements, as well as the definition of risk scenarios and use cases.
None of the frameworks cover the setup of risk scenarios in corporation with business
stakeholders, based on crown jewels, legislative requirements and/or previous security incidents.
In addition, none of the frameworks cover the definition of use cases based on risk scenarios,
common attack paths or the cyber kill chain phases. The latter can be seen as the base of correlation
rules to be implemented in SIEM, and can hereby be seen as the intelligence of the SIEM solution.
As such, the lack of coverage of these operational requirements could seriously affect the efficiency
of detecting unwanted activities using SIEM.
The following subsections cover the results of mapping the frameworks to the remaining six areas.
This focus area covers requirements on an organisational level. One can hereby think of
requirements regarding roles and responsibilities concerning SIEM as well as the organisation
setting up specific requirements that should be met by the SIEM implementation. Requirements
differ depending on an organisations risk appetite, crown jewels, legislative and regulatory
requirements as well as reporting requirements.
All frameworks have good coverage within the organisational requirements focus area, where both
COBIT 5 and ISF SoGP cover all our requirements. One can argue that frameworks cover this focus
area well due to the generic nature of its requirements.
Log management
This focus area covers requirements regarding log management. Next to generic log management
requirements, this focus area covers specific SIEM requirements such as the definition of which
log sources should be on boarded, and which log entries should be collected based on defined use
cases.
ISF SoGP has the best coverage of this focus area. ISO 27002 and NIST Cybersecurity perform well
in this area too, where they mostly cover the generic log management requirements.
Correlation
This focus area covers requirements regarding correlation of events within the SIEM. One can
hereby think of the translation of use cases into correlation rules and the periodic checking on
desired results of the implemented rules.
36
Both ISF SoGP and NIST Cybersecurity cover all our defined requirements. PCI-DSS covers none of
the defined requirements.
Alerting
This focus area covers requirements regarding the implementation of triggers and alerts set within
use cases, as well as the information that alerts contain. This focus area contains two requirements.
As such, covering only one requirement drops the line of an organisation to 50% in the spider
graph immediately.
ISF SoGP covers all requirements defined within this focus area. ISO 27002, COBIT 5 as well as PCI-
DSS cover none of the defined requirements.
Responding
This focus area covers requirements regarding the response to alerts that are generated by the
SIEM. Requirements link to the SOC as well as an organisations incident management process.
This focus area contains two requirements. As such, covering only one requirement drops the line
of an organisation to 50% in the spider graph immediately.
COBIT 5 covers all requirements defined within this focus area. One can argue that this is due to
the requirements being defined on process level
Evaluation
This focus area covers requirements regarding the evaluation of alerts and incidents that are
generated by the SIEM. One can hereby think of the adjustment of use cases if applicable. This focus
area contains two requirements. As such, covering only one requirement drops the line of an
organisation to 50% in the spider graph immediately.
COBIT 5, NIST Cybersecurity, and ISF SoGP cover all requirements defined within this focus area.
PCI-DSS covers none of the defined requirements.
37
6 SIEM case study

Now that we have discussed the different aspects of SIEM, defined minimum requirements
regarding SIEM and assessed the coverage of SIEM within IT risk frameworks, we continue by
researching how SIEM is addressed in the field. This chapter covers the results of our field research
and provides insight that will answer our third research question:
6.1 Field research methodology

For our field research, we selected two financial institutions. At each organisation, we conducted
interviews with SIEM experts. Financial institutions on the opposite scale of size were selected in
order to gain diverse results throughout the financial sector. Our motivation for specifically
targeting financial institutions is the result of insights gained during our interviews with SIEM
field-experts. The consensus being that confidentiality, integrity and availability of IT plays a
crucial role for financial institutions. Furthermore, financial institutions aim to invest more time
and money in IT Security (Huang, Emily, & Yadron, 2014).
To establish a common baseline and obtain comparable results, we used a predefined set of
interview questions as well as our SIEM focus-areas as guidance for the interviews. Each interview
was held with one or more subject-matter experts. The interview questions are listed in Appendix
10.2: Interview questions.
The results of the interviews are detailed per financial institution and focus area. Analysis of the
results is preformed based on a 5-point scoring system as detailed in table 3. Based on the
organisations coverage of a requirement, a score is assigned. For each focus area, the total score
will be the average score of all requirements. The results are used to make a comparison between
the organisations.
-- Requirement is not addressed.
- Requirement is not specifically addressed, however is (partially) covered by
other controls within an organisation.
+/- Requirement is addressed in an ad-hoc fashion. These implementations were
born out of necessity and did not go through a proper design and test phase.
+ Requirement is addressed in a structured manner, going through a design and
test phase. Its implementation was unforeseen however, become apparent
during the SIEM implementation process.
++ Requirement is addressed in a structured manner, going through a design and
test phase. Its coverage is the result of defined requirements and/or foresight.
Table 3: Explanation of scoring system
38
6.2 Company 1: A large financial institution
6.2.1 Background information

Financial institution 1 (hereafter named FI1) is a large bank in the Netherlands, serving retail and
business clients. They are also active internationally in a number of specialist activities, such as
Private Banking, Energy, Commodities & Transportation (ECT) and Clearing.
FI1 has a dedicated SOC department that, amongst others, handles security monitoring. The SOC
team consists of seven full-time security professionals. FI1 started using SIEM in 2010, and have
been actively using and developing SIEM and its supporting processes.
6.2.2 Results
This section covers how FI1 addresses SIEM. Results are based on interviews.
General interview
FI1 stated that they did not make use of a framework when implementing their SIEM tooling.
FI1s main reason for implementing SIEM is threat management. FI1 wanted to gain better insights
into ongoing attacks and the overall security of the IT infrastructure. Their SIEM has been pitched
to management as a means for threat management. Compliancy and regulatory requirements were
also a factor during the implementation of SIEM.
The following organisational requirements were defined regarding SIEM:

Confidentiality; all information and correlated data should be kept securely and only
viewable for authorised users.
Integrity; the data stored, displayed and correlated should be correct.
Availability; once implemented and used, SIEM tooling should have high availability.
Compliance; compliance demands are met.
Business needs; if business has specific security monitoring needs, assistance should be
given for their implementation.
FI1 decided to initially focus on the network layer and perimeter monitoring. Doing so allowed FI1
to heavily scope the input sources in order to not flood their SIEM tooling with raw data. Over time,
more log sources were enrolled. A process is in place that enables business departments to define
specific security requirements, in collaboration with the security team. This process covers the on
boarding of crown jewels within SIEM.
The following key roles regarding SIEM have been defined and assigned:
Level 1, 2, 3 analyst
Reporting specialist
System administration
SIEM designer (architect)
Logging specialist
Intelligence analyst
39
The initial operational requirements that led to the implementation of FI1s SIEM tooling are
unclear. A project to select and implement new SIEM tooling is currently ongoing. This project
defines requirements based on gained experiences, future plans and capacity needs.
The business has defined FI1s crown jewels and its main threats. The SIEM designer, in
collaboration with the security team and the business, develops risk scenarios based on these
threats. Each risk scenario is further broken down into use cases. Use cases are defined based on
the phases of the Cyber-kill chain, common attacks, attack paths and threat intelligence.
Furthermore, use cases contain classification, categorisation and prioritizing.
FI1 states that they use three main sources of input for security monitoring. These sources are:
Log files (events).
Intelligence (Threat intelligence, vulnerability scans, netblock information, etc.).
Contextual information.
FI1 states to participate in threat intelligence sharing with other financial institutions, as well as
having a subscription on commercial threat intelligence feeds.
Log management
FI1 designed and implemented log management as part of their SIEM environment. Log
management is not only used to retrieve critical security information but also to perform
availability monitoring and obtain debugging information. FI1 acknowledges that the quality of log
files is of great importance to the successful workings of a SIEM solution.
FI1 states that log management should be well defined and implemented across the IT
environment. The sending of log files should not be interrupted by changes or the reallocation of
systems. As such, log management should be part of operational and technical procedures.
FI1 transmits logs to a central location, from which the SIEM tooling retrieves the log files. The log
files to be retrieved are based on use cases and are specifically selected (i.e. only relevant log files
are retrieved). Logs are automatically normalised by the SIEM tooling.
The SIEM tooling provides mechanisms for checking whether expected log files are still received.
If expected log files are not received, alerts are generated.
Correlation
The SIEM tooling provides authentication and authorisation so that only authorised users are
allowed to modify correlation rules and system settings.
The SIEM designer and specialist are responsible for translating Use cases into specific correlation
rules that can be used in the SIEM tooling. FI1 makes use of a development environment for
designing and testing correlation rules before deploying them into production.
The working of correlation rules is tested by simulating real attacks. Furthermore, periodic red-
and blue teaming exercises are performed to test the overall effectiveness of the rule set.
Alerting
Use cases contain priority, criticality and triggers on when alerts should be generated. The SIEM
tooling provides alerts to the security team, providing all relevant information.
40
Responding
FI1 employs a 3-line responding model
1st line analysts receive alerts and determine the nature of the alert. FI1 states that many
alerts handled by the 1st line analysts are the result of implemented configuration changes.
2nd line analysts take over alerts that cannot be resolved by the 1st line.
3rd line analysts take over alerts that cannot be resolved by the 2nd line. 3rd line analysts
can raise an incident, starting the generic incident management procedure.
Evaluation
The intelligence analyst has the responsibility to update existing use cases if new threats are
identified, new threat-intelligence is received or when scenarios are modified. FI1 states that
updating use cases is an iterative process and takes place throughout the life cycle of a use-case.
FI1 states that use cases are adjusted if applicable. This is mainly done for use cases that generate
a large amount of false positives. No formal process is in place for evaluating the full set of use
cases periodically.
When the business requests to on board a new component in SIEM, reporting requirements are
set up. These reports are then delivered to the business upon agreed interval. No global reporting
requirements have been agreed upon.
6.2.3 Result discussion

This section covers discussion of the information gathered regarding FI1. Appendix 10.3 contains
the mapping of the gathered results against our set of minimum requirements to be addressed
when implementing SIEM.
It is positive to see that the threat management driver for FI1s SIEM implementation is claimed to
be understood and supported by management. This could be due to previous incidents or
experiences that clearly demonstrate the benefit of a threat management focussed SIEM
implementation. FI1 also indicates that legislative and regulatory requirements enforced by
organisations like the De Nederlandsche Bank also increased management support.
Noteworthy is that FI1 scores average when it comes to the definition of organisational
requirements beforehand. FI1 states that they first want to focus on low hanging fruit (i.e.
infrastructure- and perimeter monitoring), before enrolling crown jewels. FI1 states that after
enrolling low hanging fruit, separate projects have been started to on board crown jewels.
Due to the average score on defining organisational requirements regarding SIEM, we also see low
scorings with regard to the definition of SIEM tooling operational requirements, followed by
market research based upon these requirements.
41
When it comes to the technical preparation of shaping the SIEM tooling, FI1 scores very well. FI1
defines risk scenarios for elements to be monitored. Risk scenarios are then translated into one or
more use cases. FI1 makes use of the cyber kill chain when defining use cases. This shows that the
security team and architects have a good understanding of how hackers work, and what to detect
them in an efficient manner. It is therefore not completely unexpected to see that FI1 scores well
on assigning roles and responsibilities and having a security team with relevant skills. Having a
security team with relevant skills also allows for the good scores we observe in the correlation and
alerting focus areas. These areas require highly specialised skills and dedicated resources in order
to score well.
Most organisations already have a well-defined and implemented log policy for availability and
debugging purposes. However, the security aspects of logging are often not very well defined. This
is reflected in FI1s scores regarding the log management. Here we can see that the log policy itself
scores great but that the security aspects with the policy (such as clock synchronisation) are
mostly defined out of necessity (i.e. due to correlation problems or forensic requirements).
The responding focus area contains only two requirements: the analysis of alerts by analysts, and
the automatic creation of incidents for high-impact incidents. FI1 has an extensive three-line
responding structure, which makes it score very well for the first requirement. No automatic
creation of incidents is configured, for which it scores zero on the second requirement. We
understand that FI1 chose to not implement automatic incident creation since it has an extensive
three-line responding structure that eliminates false positives.
FI1 adjusts use cases if applicable. This is mainly done when use cases generate a large amount of
false positives. Reports that have been agreed upon with specific business stakeholders are
delivered on the agreed interval. There are no global reporting requirements defined. This lowers
the average score for the evaluation focus area.
As a final remark, FI1 scores very well overall. That being said, it is worth to mention that FI1 has
already been working extensively with their SIEM environment and tooling for several years.
42
6.3 Company 2: A small financial institution
6.3.1 Background information

Financial institution 2 (hereafter named FI2) is a smaller bank in the Netherlands. FI2 primarily
focusses on corporate clients and to a smaller extend on consumer services. They are active in
specialist activities, such as debt and equity mezzanine, mergers & acquisitions, capitalisation
advisory, leveraged finance and structured finance.
FI2 does not have a dedicated SOC but instead assigns SOC related roles and responsibilities to the
generic IT security team. In addition, security related tasks and responsibilities are an integral part
of all the different IT departments. The IT security team consists of two dedicated security
professionals. FI2 started using SIEM in 2012 and have been developing their SIEM solution and
supporting processes on a best effort basis.
6.3.2 Results
This section covers how FI2 addresses SIEM. Results are based on interviews.
General interview
FI2 stated that they did not make use of a framework when implementing their SIEM tooling.
Furthermore, FI2 states that the daily security operations and security needs are impacted by
having limited resources available. As such, aspects such as 24/7 monitoring, extensive testing,
dedicated use-case designers and formal documentation/implementation process are not feasible.
However, FI2 does mention that they recognize the tremendous value of these aspects.
Due to the relative small size of the IT security team, a decision was made to collaborate with a
Managed Security Service Provider (MSSP). The SIEM tooling is implemented within the
infrastructure of FI2. Management of the technical configuration and its continuous operation is
done by the MSSP. FI2 is negotiating to outsource more services to the MSSP.
FI2s main reason for implementing SIEM is compliance related. The initial DNB self-assessment
revealed the need for a security monitoring solution. This outcome was FI2s driver for
implementing SIEM.
Due to the necessity of SIEM as a result of the DNB self-assessment, management understands and
supports the implementation of SIEM. However, FI2 states that the understanding (and to some
extend support) of management, most likely does not go beyond the original compliance reason.
Due to the compliance driven nature of FI2s SIEM implementation, hardly any organisational
requirements other than compliance and legislative requirements were defined. FI2s strategy was
to have most of their IT infrastructure log everything to their SIEM tooling to filter and sort the
data there.
No real SIEM roles or specific skills are defined and assigned to the members of the IT security
team. However, the IT security team did receive technical training related to the operation and use
of the actual SIEM tooling. Furthermore, the MSSP is responsible for implementing and
43
maintaining the technical SIEM tooling as well as implementing correlation rules and sending out
periodic reports.
Due to the decision to work with a MSSP, requirements were defined aiming more towards vendor
selection than product selection. FI2s official vendor selection process was used.
FI2 states that no top-down risk scenarios are defined but that to some extend the crown jewels
(on business level) are taken into consideration when determining the objects to monitor. An ad-
hoc approach is used for determining what to monitor. Best practices, own experiences and
previous security incidents are used as input. FI2 states that due to the ad-hoc approach, only a
limited amount of use cases are defined. However, the current approach for determining
monitoring requirements does include, to some extent, common attacks, attack paths and cyber
kill chain phases.
FI2 actively participates in gathering threat intelligence. The MSSP supplies FI2 with a threat
intelligence portal. Based on the gathered threat intelligence new monitoring requirements are
defined by FI2. The rules are implemented by the MSSP.
Log management
FI2 states to have a minimum standards document that contains the minimum requirements for
system settings and configuration options. This document covers logging and monitoring in which
specific logging policies are defined. FI2 states that most, if not all, of our listed log policy
requirements are met and configured.
Specific baselines are created, based on the Microsoft implementation baselines. The baselines
contain the logging requirements as detailed in the minimum standards document.
For the on boarding and modification of devices and systems, the generic change management
process is followed. The change management process includes configuring log settings and
connecting new systems to the central logging facility. In addition, FI2 says that most of the IT
engineers are aware of the logging requirements and configure the correct log policies if missing.
The central SIEM logging facility provides enough capacity for a 6-month retention of all received
log files. Log files are transferred from servers via a dedicated agent (built by the SIEM tooling
vendor). Authorisation rules are defined in the SIEM tooling to prevent unauthorised access to the
system and log files.
No formal decision structure (i.e. based on use cases) was followed in determining which log
sources to monitor. However, a decision was made to first gather IT infrastructure logs (i.e.
network devices, servers, anti-virus, etc.) before collecting application logs. Furthermore, FI2s
strategy to have most of their IT infrastructure log everything to their SIEM did not allow for pre-
emptively selecting which log entries to be collected.
FI2s SIEM tooling has built-in controls for checking if expected log sources are still transmitting
their log files. If anomalies are detected, alarms are raised within the SIEM tooling.
The SIEM tooling normalises received log files based on known log structures. For log files with an
unknown layout, the SIEM tooling provides specific modules for manual interpretation. These
modules are configured and implemented by the MSSP based on the requirements of FI2.
44
Correlation
The MSSP is responsible for the implementation of correlation rules within the SIEM tooling. The
implemented rules are based on the input provided by FI2. However, as stated before, providing
monitoring input is mostly done based on an ad-hoc requirements and only to a small extend based
on pre-defined use cases. In addition, the SIEM tooling provides built-in correlation rules for some
common attacks.
No periodic testing of correlation rules is performed. However, FI2 does state that correlation rules
are checked for the desired result upon initial implementation on an ad-hoc basis.
Authorisation rules allow modification of correlation rules by authorised users only. The MSSP has
full administrative permissions and is allowed to modify system settings and change correlation
rules. FI2 has view privileges within the SIEM tooling and can only view the available information
and alerts. FI2 can view changes made by the MSSP to the system, its correlation rules and the
available data.
Alerting
For the limited set of use cases that is implemented, trigger thresholds and alerting conditions are
defined and implemented within the SIEM tooling. In addition, alerting conditions are configured
for the correlation rules that are the result of the ad-hoc monitoring requirements.
Alerts that result in incident tickets contain detailed information such as log snippets, affected
systems and problem descriptions. Alerts that result in e-mails sent only contain high-level
information such as the problem description. These e-mails do contain a reference to the specific
alert within the SIEM tooling for further information.
Responding
Alerts are viewed and handled by FI2s IT security team. FI2 states that due to resource restraints,
timely responding to alerts is sometimes a challenge. FI2 is looking into a model where it works
together with the MSSP more closely. The MSSP would then assist in handling alerts.
For several high-impact and critical correlation rules, automatic incident tickets are created. These
tickets are than part of the regular incident process.
Evaluation
FI2 and MSSP determine new monitoring requirements and refine current correlation rules in a
bi-weekly tuning call. This call is also used for discussing lessons learned and improvements to be
made. Furthermore, an ad-hoc approach is used for fine-tuning correlation rules based on
identified false positives.
The MSSP sends out daily reports on 6 predefined main topics. Periodic reporting regarding
incidents is covered within the generic incident management process. In addition, the
management team discusses major IT incidents including security incidents.
45
6.3.3 Result discussion

This section covers discussion of the information gathered regarding FI2. Appendix 10.3 contains
the mapping of the gathered results against our set of minimum requirements to be addressed
when implementing SIEM.
FI2 has a clear primary driver for its SIEM implementation: compliance, passing SIEM
requirements specific to the DNB self-assessment. The management of the bank values being
compliant to the requirements set by De Nederlandsche Bank. Hereby management support and
understanding is covered.
As the results show, only limited organisational requirements were defined prior to the
implementation of SIEM. We believe this could be due the main implementation driver being
compliant to requirements set in the DNB self-assessment. After all, the main focus was on
becoming compliant rather than building a structured and well-thought solution that is an effective
tool for breach detection.
FI2 makes use of services offered by a MSSP. Outsourcing security related tasks brings its
challenges, however, we can clearly see several of our SIEM requirements being covered by the
services offered by the MSSP. Collaborating with a MSSP requires that roles and responsibilities
are well and assigned. Furthermore, it can be expected of a MSSP to have the required skills for
managing the SIEM tooling, its configuration and the implemented correlation rules. FI2
outsourced tasks related to management of the SIEM environment, implementing correlation rules
and periodic reporting to the MSSP.
While FI2 mentions that they see the benefit of defining risk scenarios and use cases, no risk
scenarios are created. We believe this to be the result of not having strong and clear organisational
requirements in combination with limited resources. In turn, the lack of risk scenarios affect the
structured creation of use cases, and thus less effective breach detection. FI2 states that several
use cases have been created on an ad-hoc basis. No process for structurally defining risk scenarios
and use cases is in place. FI2 employs an ad-hoc approach for defining monitoring needs. In spite
of this, we do see that FI2 strongly uses lessons learned as input for defining new monitoring needs.
The gathering of intelligence is actively performed by both FI2 and their MSSP. The MSSP offers a
threat intelligence portal that can be accessed by FI2. The translation of new threat insights into
monitoring needs and ultimately into correlation rules is done using an ad-hoc approach.
Looking at the results within the log management focus area we can observe that FI2 scores quite
well on the technical requirements. Another focus point FI2 scores well on is having the log policy
being part of operating procedures. We believe these results are the product of FI2s strategy of
46
connecting most of their IT infrastructure to the SIEM tooling. Doing this in a structured and
effective way would be impossible without a log policy, baseline and operating procedures.
FI2 does not score well on the requirements covering the determining of log sources and entries
to be collected based on use cases. This is due to the strategy used by FI2: connecting as much log
sources as possible.
FI2 does not check correlation rules on their desired result periodically. FI2 acknowledges that
this is due to the limited resources available. Another reason for this could be due to budget
constrains (i.e. limited budget for penetration tests in combination with monitoring exercises).
FI2 has bi-weekly tuning calls with the MSSP. Amongst other topics, FI2 discusses the need for
adaptation of implemented use cases. Alerts sent out by the SIEM tooling contain detailed
information. High-impact incidents automatically generate incident tickets. The generic incident
management process is used, helping in the structured and timely handling of incidents.
FI2 struggles with making its SIEM implementation more effective due to limited resources. FI2
acknowledges this, and states that it is exploring the possibility to outsource more tasks to the
MSSP. Examples of tasks are the monitoring of changes in the threat landscape as well as the
creation of new risk scenarios and use cases. In our opinion, FI2 should perform a top-down risk
analysis to determine which elements of their IT infrastructure are considered crown jewels. One
can hereby think of business applications and supporting infrastructure such as FI2s Active
Directory environment. Risk scenarios should be defined for the identified crown jewels to bring
the SIEM implementation to a higher maturity level.
47
6.4 Comparison of organisations

Having used the same framework to assign scorings to both companies, we can now compare the
results. Graph 2 shows the scorings per focus area for both companies. Detailed results per focus
area and requirement can be found in appendix 10.3.
Organisations mapped on focus areas

Organisational
requirements
Operational
Evaluating
requirements
FI1 FI2
Graph 3: Focus area coverage of selected financial institutions
The graph clearly shows that FI1 scores higher in all focus areas, except for responding. The focus
area responding only contains two requirements, of which one is the automatic creation of incident
tickets for high-impact use cases (RE1.2). FI1 chose to manually verify all incidents before creating
incident tickets, whereas FI2 does perform automatic incident ticket creation for high-impact use
cases, resulting in a lower score for FI1.
FI1 has a well-prepared approach, starting with understanding and support of management. FI1s
SIEM implementation is focussed on threat detection, where the main implementation driver of
FI2 is compliance. FI1 identified requirements regarding staffing and required knowledge,
resulting in a well-staffed and layered security team.
FI1 performed a risk analysis, defining its crown jewels, prior to the SIEM implementation. Risk
scenarios are defined for identified crown jewels. The risk scenarios are then used as the base for
defining use cases. This approach makes FI1 score well in the focus areas covering organisational-
and operational requirements.
48
FI2s approach is ad-hoc: no risk scenarios are defined prior to the creation of use cases. Use cases
supplied by the MSSP are used. FI2 provides input on what use cases should be implemented in
addition to the standard use cases. Due to this, FI2s scores on the domains organisational- and
operational requirements are significantly lower than FI1s.
The correlation focus area shows the biggest difference between FI1 and FI2. The scores for
requirements CO1.1 and CO1.2 are at opposite ends of the scale. Requirement CO1.2, which states
that correlation rules should be checked for their desired result periodically, requires FTEs and
budget to be performed well. FI1 has dedicated personnel for this, where FI2 works on a best effort
and ad-hoc basis.
The average score of FI1 and FI2 for the log management focus area is almost identical. That being
said, the individual scores shows some interesting differences. For starters, it is unexpected that
FI1 scores lower on LM1.3, which states that the log policy is part of operating procedures. FI2
loses points for the focus areas that require use cases to be defined. Both companies score very
well on LM1.1, which contains an overview of minimum requirements regarding an organisations
log policy. Log management is not only used for SIEM, as it has links with generic IT operations as
well. Next to input for SIEM tooling, log files are used for a variety of generic IT operations tasks
(i.e. troubleshooting, availability monitoring). This might be one reason that both organisations
have log management well covered.
Both companies score reasonably well in the alerting and evaluation focus area. Where FI2 has an
ad-hoc approach for the adjustment of use cases, FI1 has a more structured approach. Both
companies have high-level periodic reporting, where FI1 has specific reports for on-boarded
business crown jewels. Requirements for these reports are set up during the intake procedure.
49
7 Synthesis
Now that we have set up our model and used it to assess multiple IT risk frameworks as well as
two financial institutions, it is time to combine the outcome of our theoretical and empirical
research.
SIEM and its relevance

The results of our first research sub question show that the relevance of SIEM is acknowledged in
theory as well as by organisations. Where in the past organisations solely relied on preventive
measures (i.e. firewalls at the perimeter of the network), nowadays additional detective measures
like SIEM are being implemented.
Both financial institutions that were assessed during the case study acknowledge the fact that
attackers using advanced reconnaissance and specialised weapons (i.e. zero-day vulnerabilities)
cannot be stopped by solely implementing preventive measures. Standard anti-virus solutions and
Intrusion Detection Systems might not recognize these specialised weapons, and carefully crafted
(spear-) phishing e-mails could pass an organisations spam filter.
Both theoretical and empirical research have shown that SIEM can be used as a detective measure,
allowing organisations to monitor status and events from all over their IT landscape. Collected
information can be used to perform detailed analysis and correlation to identify irregularities and
security incidents. Timely identifying security incidents allows for appropriate measures and
actions to be taken. In addition to detection, the information provided by SIEM can be used to
respond to security incidents efficiently. This disrupts the cyber kill chain, stopping potential
breaches to an organisations network.
Defined focus areas

During our research, we set up a model consisting of the focus areas: operational requirements,
organisational requirements, log management, correlation, alerting, responding and evaluation.
SIEM subject matter experts that assessed our model agreed upon the definition of focus areas.
Both financial institutions that were assessed during the case study also acknowledge that
implementing SIEM requires actions to be taken on various fields, including organisational- and
technical level.
Focus area coverage: theory and practise

Our model provides insight in what aspects should be addressed by an organisation when
implementing SIEM. The minimum requirements defined within each focus area aim at making an
organisation ready to implement SIEM.
50
Mapping selected IT risk frameworks to our model has shown that coverage greatly differs
between frameworks. ISF SoGP covers the most requirements defined in our focus areas. It scores
particularly well in organisational requirements as well as log management, correlation, alerting
and evaluation. As ISF SoGP is a superset of COBIT 5 and provides comprehensive coverage of ISO
27000 series, we knew beforehand that it would score higher than the two separately.
None of the frameworks covers the setup of risk scenarios in corporation with business
stakeholders, based on crown jewels, legislative requirements and/or previous security incidents.
In addition, none of the frameworks covers the definition of use cases based on risk scenarios,
common attack paths or the cyber kill chain phases.
Both financial institutions assessed during our case study do acknowledge the need for defining
risk scenarios and use cases. Risk scenarios and use cases are crucial in determining what- and
how to monitor, and form the fundament of the intelligence that is built in to the SIEM tooling. As
such, the lack of coverage of these operational requirements could seriously affect the efficiency of
detecting unwanted activities using SIEM.
The case study shows that individual SIEM focus area scores vary greatly; one common point being
log management. Adequate log management shows that organisations understand the need for
efficient and effective SIEM input information management. Both financial institutions
acknowledge that turning this information into relevant alerts and incidents is a challenge. Results
show that organisations defining risk scenarios and use cases know better what to protect and
which information to collect. Not adequately defining risk scenarios and use cases results in
collecting to much data and being overwhelmed with information.
One of the financial institutions that we assessed outsourced parts of their SIEM related activities
to a MSSP. If working with a MSSP it becomes even more crucial to define risk scenarios and use
cases. A MSSP is often unaware of organisation particulars and is dependent on the information
provided by the organisation. Risk scenarios and use cases let the MSSP understand what an
organisations crown jewels are and how the organisation wants to protect them.
51
8 Research questions revisited
8.1 Sub question 1

The first supporting sub question is descriptive and is defined as follows:
SIEM stands for Security Information and Event management, as covered in section 4.1. It
combines Security Information Management (SIM), with Security Event Management. (SEM). SIM
focusses on long-term storage of log files, where SEM focusses on real-time monitoring, correlation
of events.
A SIEM collects logs and threat intelligence from internal- (i.e. server-, network- and application
logs) and external sources (i.e. threat intelligence sharing). Event correlation is used to detect and
alert on unwanted activities within the network. Determining what to protect (crown jewels) and
how to monitor this is an integral part of a SIEM solution.
The relevance of SIEM increases by the day as data breaches and company hacks hit the news
regularly. The need for additional security measures grows and focus is shifting from solely
preventive measures to a combination of preventive and detective measures. SIEM is a detective
measure used by an organisations security team (SOC) to detect and respond to security breaches
or compromise of critical assets of (i.e. crown jewels). In some sectors, regulators (i.e. DNB)
enforce detective measures like SIEM. This makes compliance an important driver for SIEM
implementations, next to threat management.
8.2 Sub question 2

The second supporting sub question is analytic and is defined as follows:
We set up a model covering focus areas with minimum requirements that should be addressed
when implementing or assessing a SIEM implementation. The model covers the areas:
organisational requirements, operational requirements, log management, correlation, alerting,
responding and evaluating.
52
We mapped selected IT risk frameworks to our model, assessing to what extent they cover SIEM.
ISF SoGP, being a superset of COBIT 5 and having comprehensive coverage of ISO 27000, has the
best coverage of our focus areas and requirements.
All frameworks lack coverage of SIEM specific requirements such as the ones defined within the
Operational requirements focus area. The most important requirements that are not covered by
the frameworks are the setup of risk scenarios in corporation with business stakeholders, based
on crown jewels, legislative requirements and/or previous security incidents. In addition, none of
the frameworks cover the definition of use cases based on risk scenarios, common attack paths or
the cyber kill chain phases.
Risk scenarios and use cases form the base of the intelligence that is implemented within a SIEM.
As such, the lack of coverage of these operational requirements could seriously impact the
efficiency of detecting unwanted activities using SIEM.
8.3 Sub question 3

The third supporting sub questions is also analytic and is defined as follows:
We assessed the SIEM implementation of two financial institutions and conclude that both
institutions address SIEM very differently. The main driver of the first institution is threat
management (detection of- and response to security incidents), where the second institution
implemented SIEM mainly to be compliant to legislative requirements enforced by DNB. Both
financial institutions state that their SIEM implementations are not based on existing frameworks.
Having a different main driver results in the approach of both organisations being very different.
The first institution applies top-down risk management: determining its crown jewels, setting up
risk scenarios and use cases, having a phased infrastructure on boarding approach. It also has a
dedicated security team of seven FTE.
The driver of the second institution is compliance and in contrast to the first institution, it does not
have a dedicated security team. They collaborate with a MSSP who is responsible for management
of the SIEM environment and implementation of rules. Absence of risk scenarios and use cases
result in an ad-hoc approach creating a difficult to maintain SIEM implementation. In turn, affecting
the effectiveness of the SIEM implementation.
53
The size of an organisation determines the shape of several key aspects. Organisations with fewer
FTEs are required to automate or outsource more tasks. With good preparation, that includes the
identification of crown jewels and definition of risk scenarios and use cases, even a small
organisation can achieve a well working SIEM implementation. However, it might need support of
external resources such as a MSSP.
Finally, it does not matter if one outsources tasks to a MSSP or performs all tasks in-house: risk
scenarios and use cases are key for addressing SIEM in practice. It allows for a structured approach
in defining what to monitor and which information to collect. With risk scenarios and use cases as
a solid basis, a SIEM solution can expanded and grow in a sustainable way.
8.4 Answering the central research question
How do IT risk frameworks address SIEM and to what extend does this match current SIEM
implementations?
The selected IT risk frameworks all address SIEM differently. None of the selected frameworks
covers all minimum requirements we deem essential for a successful SIEM implementation. Most
of the selected IT risk frameworks mention determining ones critical assets (i.e. crown jewels). All
frameworks lack coverage of operational requirements such as the definition of risk scenarios and
use cases. Therefore, we conclude that the selected frameworks do not address the very fundament
of SIEM implementations, namely, how crown jewels need to be monitored.
Our field research shows that both financial institutions did not make use of a framework when
implementing their initial SIEM. The large financial institution assessed during our empirical
research used the knowledge of SIEM subject matter experts. This resulted in covering more
requirements of our model than the small financial institution. Specifically when it comes to
operational requirements such as the definition of risk scenarios and use cases.
The small financial institution does acknowledge the need for risk scenarios and use cases to
improve the overall effectiveness of a SIEM for threat management. The usage of one of the selected
frameworks would not have greatly improved their coverage of operational requirements of our
model they now lack.
As such, we conclude that the selected frameworks inadequately address key aspects (i.e. risk
scenarios and use cases) and that as a result, organisations implement their SIEM solution based
on own expertise and that of third parties.
54
8.5 Future work

Our research comprises a model covering focus areas and minimum requirements for a SIEM that
is used for threat management. Additional research could be done regarding requirements related
to SIEM implementations with a focus on the alternative main drivers: compliance or forensic. Our
model could be used as a base for this research.
A new ISO standard is in the works (ISO 27044) that focusses on SIEM implementations. In our
conclusions, we state that having adequate risk scenarios and use cases are key for successful SIEM
implementations. Based on the subjects in this standard, we see that mainly technical topics are
discussed. As such, it would be interesting to see how the ISO 27044 standard maps to our model.
Furthermore, it would be interesting to see if risk scenarios and use cases are covered and if so, in
how much detail.
One of the financial institutions that was part of our case study partially outsourced SIEM related
tasks to a MSSP. Our model does not cover requirements related to sourcing and MSSPs.
Organisations with only few dedicated security staff might want to partner with a MSSP. Research
regarding MSSP collaboration could provide key insights for specific sourcing requirements,
leading to a more efficient working SIEM.
55
9 Acknowledgments
We would like to express our gratitude to our supervisor Ralf Bardoel whose expertise,
understanding, and patience added considerably to our final result. We appreciate his assistance
and continuous feedback regarding our research and thesis writing. Furthermore, we would like
to thank Ren Matthijsse for the guidance he provided at all levels of the research project. Finally,
we would like to thank Peter Kornelisse to serve as our external reader.
We also acknowledge Erwin Hansen and Stan Hegt of KPMG for providing us with the introductions
at the financial institutions evaluated in this research. Appreciation also goes out to Pieter van
Houten, Jip Hogenboom and Jeroen de Wit for providing valuable and crucial insights regarding
our model of focus areas and minimum requirements.
Finally, we would like to thank all members of the VU faculty staff. Their expertise helped us along
the way and allowed us to perform our research in the first place.
56
10 Bibliography
AccelOps. (2013, February 12). Top 10 SIEM Implementer's Checklist. Retrieved from AccelOps:
http://www.accelops.com/media/1700/Top_10_SIEM_Implementer_Checklist.pdf
Bardoel, R. (2015, June 5). Personal interview. (J. v. Moosdijk, Interviewer)
BBC. (2014). Hack attack causes 'massive damage' at steel works. Retrieved from BBC:
http://www.bbc.com/news/technology-30575104
Chuvakin, A. (2010). The Complete Guide to Log and Event Management. Novell.
Chuvakin, A. (2011, April 22). So you got that SIEM. NOW what do you do? So you got that SIEM.
NOW what do you do? Security Warrior Consulting. Retrieved from
http://www.slideshare.net/anton_chuvakin/so-you-got-that-siem-now-what-do-you-
do-by-dr-anton-chuvakin
Cisco. (2014). 2014 Annual Security Report.
Constantine, C. (2014, March 18). What kind of logs do you need for an effective SIEM
implementation? Retrieved from Alien Vault:
https://www.alienvault.com/blogs/security-essentials/what-kind-of-logs-for-effective-
siem-implementation
DNB. (2014, May 22). Toelichting op Toetsingskader Informatiebeveiliging 2014. Retrieved from
DNB: http://www.toezicht.dnb.nl/binaries/50-230767.pdf
Dorigo, S. (2012). Security Information and Event Management.
EiQ Networks. (2013, March 6). EiQ Networks Survey Reveals Organizations Are Suffering From
SIEM Deployments. Retrieved from EiQ Networks:
http://www.eiqnetworks.com/abouteiqnetworks/news/pressrelease/2013/eIQnetwor
ks-Survey-Reveals-Organizations-Are-Suffering-from-SIEM-Deployments.php
Gartner. (2012, August 9). On People Running SIEM. Retrieved from Gartner:
http://blogs.gartner.com/anton-chuvakin/2012/08/09/on-people-running-siem/
Gartner. (2012, July 30). On SIEM Processes/Practices. Retrieved from Gartner:
http://blogs.gartner.com/anton-chuvakin/2012/07/30/on-siem-processespractices/
Gartner. (2014). 2014 Magic Quadrant for SIEM.
Gartner. (2014, June 30). Evaluation Criteria for Security Information and Event Management.
Retrieved from Gartner: https://www.gartner.com/doc/2785317/evaluation-criteria-
security-information-event
Gartner. (2014, March 26). How to use threat intelligence with your SIEM? Retrieved from
Gartner: http://blogs.gartner.com/anton-chuvakin/2014/03/26/how-to-use-threat-
intelligence-with-your-siem/
Gartner. (2014, September 15). SIEM Technology Assessment and Select Vendor Profiles. Retrieved
from Gartner: https://www.gartner.com/doc/2846817/siem-technology-assessment-
select-vendor
57
Hanley, M., & Montelibano, J. (2011). Insider Threat Control: Using Centralized Logging to Detect
Data Exfiltration Near Insider Termination. Software Engineering Institute.
Harrel, C. (2014, September 1). SIEM Use Case Implementation Mind Map. Retrieved from Journey
Into Incident Response: http://journeyintoir.blogspot.nl/2014/09/siem-use-case-
implementation-mind-map.html
Houten, P. v. (2009). Audit Trail Management. Amsterdam: University of Amsterdam.
HP. (2014, March). HP Attack Life Cycle use case methodology. Retrieved from HP:
http://h20195.www2.hp.com/v2/getpdf.aspx/4aa4-9490enw.pdf
HP. (2014). Security Information & Event Mangement. Retrieved from HP:
http://www8.hp.com/nl/nl/software-solutions/siem-security-information-event-
management/
Huang, D., Emily, G., & Yadron, D. (2014, November 17). Financial Firms Bolster Cybersecurity
Budgets. Retrieved from The Wall Street Journal:
http://www.wsj.com/articles/financial-firms-bolster-cybersecurity-budgets-
1416182536
IBM. (2014). IT Executive Guide to Security Intelligence - Transitioning from SIEM to Total Security
Intelligence.
Infosec Institute. (2014, May 15). Top 6 SIEM Use Cases. Retrieved from Infosec institute:
http://resources.infosecinstitute.com/top-6-seim-use-cases/
InfoSec Nirvana. (2012, June 21). Siem Use Cases - What you need to know? Retrieved from InfoSec
Nirvana: http://infosecnirvana.com/siem-use-cases/
ISACA. (2012). COBIT 5: Business Framework for the Governance and Management of Enterprise IT.
ISACA. (2013, 05 04). COBIT 5 for Information Security. Retrieved from ISACA:
http://www.isaca.org/COBIT/Pages/Information-Security-Product-Page.aspx
ISF. (2014). The Standard of Good Practice for Information Security.
ISO. (2013). ISO/IEC 27001:2013.
Kent, K., & Souppaya , M. (2006). Guide to Computer Security Log Management (SP 800-92). NIST.
Krebs, B. (2014). Sony Breach May Have Exposed Employee Healthcare, Salary Data. Retrieved
from KrebsonSecurity: http://krebsonsecurity.com/2014/12/sony-breach-may-have-
exposed-employee-healthcare-salary-data/
Lewis, R. (2014, September 14). Auvik. Retrieved from Top 7 Success Factors for Setting Up
Centralized Logging: https://www.auvik.com/media/blog/centralized-logging-
checklist/
LockHeed Martin Corporation. (2011). Intelligence-Driven Computer Network Defense Informed
by Analysis of Adversary Campaigns and Intrusion Kill Chains. Annual International
Conference on Information Warfare & Security, (p. 14).
McAfee. (2014). When Minutes Count. Intel Security.
58
Netmonastery. (2014). Key SIEM use cases to ensure you never get compromised.
Netwrix. (2014). SIEM Efficiency Survey Report. Irvine: Netwrix Corporation.
NIST. (2013). Security and Privacy Controls for Federal Information Systems and Organisations (SP
800-53). NIST.
NIST. (2014, 02 12). Framework for Improving Citrical Infrastructure Cybersecurity. Retrieved
from NIST: http://www.nist.gov/cyberframework/upload/cybersecurity-framework-
021214.pdf
NSA. (2013). Spotting the Adversary with Windows Event Log Monitoring. National Security
Agency/Central Security Service.
PCI Security Standards Council. (2013). Payment Card Industry - Data Security Standard 3.0.
PvIB. (2011). Security Operations Center: Een inrichtingsadvies. Retrieved from PvIB:
https://www.pvib.nl/download/?id=17670673
Rorive, K., Beerends, M., Bordewijk, L., Breedijk, F., Cimen, H., Cornelissen, J., . . . Smulders, A.
(2011). Platform voor Informatie Beveiliging, Expertbrief Security Operations Center.
SANS. (2009, September 21). Effective Use Case Modeling for Security Information & Event
Management. Effective Use Case Modeling for Security Information & Event Management.
SANS Institute.
SANS. (2010). Successful SIEM and Log Management Strategies for Audit and Compliance. SANS
Institute InfoSec Reading Room.
SANS. (n.d.). Critical Security Controls fro Effective Cyber Defence. Retrieved from SANS:
https://www.sans.org/critical-security-controls/
SANS, Shackleford. (2012, June). When Breaches Happen: Top Five Questions to Prepare For.
Retrieved from SANS: http://www.sans.org/reading-
room/whitepapers/analyst/breaches-happen-top-questions-prepare-35220
SC Magazine UK. (2012, December 11). Constraints lead to failure to effectively manage SIEM
systems. Retrieved from SC Magazine for IT Security professionals:
http://www.scmagazineuk.com/constraints-lead-to-failure-to-effectively-manage-siem-
systems/article/272068/
Schinagl, S., & Schoon, K. (2014). Security Operations Center: Modelleren en meten van effectiviteit.
Securosis, L.L.C. (2014, February 1). Security Management 2.5: Replacing Your SIEM Yet? Phoenix:
Securosis. Retrieved from mcafee.com: http://www.mcafee.com/nl/resources/white-
papers/wp-replacing-your-siem-yet.pdf
Taschler, S. (2015, April 8). McAffee Community. Retrieved from SIEM Foundations: Verify That
All Data Sources Are Logging as Expected: https://community.mcafee.com/docs/DOC-
6234
Thyer, J. (2015, November 25). Hunt teaming. Let's face it, you are probably compromised. What
next? SANS Institute.
59
Times, S. P. (2000). A history of hacking. Retrieved from St Petersburg Times:

http://www.sptimes.com/Hackers/history.hacking.html
Tipton, H., & Krausse, M. (2008). Information Security Management Handbook, 6th Edition.
Auerbach Publications.
Verizon. (2014). 2014 Data Breach Investigations Report.
Watson, C., & Keary et al, E. (2015, May 19). OWASP. Retrieved from Logging Cheat Sheet:
https://www.owasp.org/index.php/Logging_Cheat_Sheet
60
11 Appendices
This section contains appendices that are referenced within this document.

11.2 Interview questions....................................................................................................67
11.3 Mapping of selected organisations .......................................................................68
61
11.1 Mapping of selected frameworks

This appendix contains the mapping of the selected IT risk frameworks listed in section 5.1, to our set of minimum requirements per focus area that should be
addressed when implementing SIEM.
Please note that when multiple frameworks map to a requirement, it does not mean that they cover the exact same load. This due to the IT risk frameworks having
a different level of granularity on topics. We have included matches where wording may vary, although semantic intent is similar.
# Subject ISO27002 COBIT 5 NIST PCI-DSS ISF SoGP

ORR1.1 The reason for implementing SIEM is understood and supported 16.1.1 EDM01.02 - 1 SG1.1.2
by C-level SG1.1.7
SI2.1.1
ORR1.2 Organisational requirements for SIEM are defined based on:

- Crown jewels 8.1.1 EDM01.01 - 1,2 12.2 SG1.2.2
8.2.1 EDM01.02 - 5 SG2.3
APO02.03 - 1 CF8.3
DSS06.01 - 1 CF10.4.4
- Risk appetite EDM03.01 - 2 ID.RM-2 12.2 SG1.1.6
SG2.3
- Legislative or regulatory requirements 18.1.1 EDM01.01 - 2 ID.GV-3 10.3 SG2.1.2
APO01.03 - 2,3 12.10.6 SG2.3
DSS06.01 - 1 A.1.3 SR2.1
- Reporting requirements EDM05.01 - 1,2 DE.DP-4 12.2 SG1.2.7
APO08.04 - 2 CF10.4.2
SI2.2
62

ORR1.3 Roles and responsibilities regarding SIEM are assigned 6.1.1 APO01.02 - 1, 2 ID.AM-6 12.5 CF2.5
APO01.06 - 1 ID.GV-2 12.10.3
ORR1.4 Security team contains members with relevant skills 6.1.1 APO01.01 - 2, 3 DE.DP-1 12.10.4 CF10.4.3
APO07.03 CF12.2.6
OPR1.1 SIEM tooling requirements are defined 14.1.1 BAI02.01 - 1,2 DE.DP-2 CF9.1.3
CF10.4.2
CF10.4.4
OPR1.2 SIEM tooling is selected based on market research using defined APO10.02 CF16.1.1
requirements BAI02.02 - 1
OPR1.3 SIEM tooling is implemented BAI03.05 CF8.7.3
CF10.4.8
CF11.1.12
OPR1.4 Risk scenarios are defined based on:

- Crown jewels
- Risk appetite
- Legislative or regulatory requirements
- Previous security incidents
OPR1.5 Use cases are defined based on:

- Scenarios
- Common attacks and attack paths
- Cyber kill chain phases
63

OPR1.6 Threat intelligence (TI) is gathered from (external) sources 6.1.4 APO04.03 - 1 ID.RA-2 SG1.2.7
PR.IP-8 CF1.2.6
CF11.2.4
OPR1.7 TI and changes in threat landscape are used to create/modify 6.1.4 DSS05.01 - 4 CF11.2
use cases periodically MEA03.01
Log management
LM1.1 Log policies are defined and should at least include:
- Logs should be accurate 12.4.1 BAI10.01 PR.DS-6 10.5 SR1.4.1
DSS01.03 CF10.4.2
- Logs should be stored in a central and tamper-proof location 12.4.2 PR.DS-1 CF10.4.6
CF10.4.2
- Log retention period per log type (based on needs and 16.1.7 DSS05.07 - 1 PR.PT-1 SR2.1
legislation) CF10.4.10
- Logs are time stamped 12.4.1 PR.PT-1 10.3 CF10.4.5
- The same time source is used for clock synchronisation 12.4.4 PR.PT-1 10.4 CF10.4.5
- Log permissions are restrictive 12.4.2 PR.PT-1 10.4 CF6.1.3, 4
10.5 CF10.4.2
- Log format (i.e. syslog, windows event logs) PR.PT-1 CF10.4.5
- System is not halted through lack of disk space 12.4.2 PR.DS-4 CF10.4.7
- Logs contain originating source 12.4.1 PR.PT-1 10.3 CF10.4.5
LM1.2 Technical baselines are defined and implemented based on log BAI10.02 PR.IP-1 CF10.4.5
policy PR.PT-1
LM1.3 Adhering to the log policy is part of operating procedures, such

as:
64

- On boarding of new devices and systems 14.2.5 DSS01 - 1
DSS01 - 2
- Changing devices and systems 14.2.2 DSS01 - 3
LM1.4 Centralized log requirements are defined, such as:

- Only authorised users have access to log files 9.4.1 DSS05.04 - 1 PR.PT-3 7.1 CF10.4.2
12.4.3 DSS06.03 - 1, 2
- Capacity of centralised logging infrastructure is managed 12.1.3 PR.DS-4 CF10.4.7
PR.MA-1 CF10.5.3
LM1.5 Log sources are derived based on use cases

LM1.6 To be collected log entries are defined based on use cases DSS05.07 - 1 CF10.4.1
(Based on risk, CF10.4.4
not use cases)
LM1.7 Controls for checking if expected logs are received are
implemented
LM1.8 Logs are normalised CF10.4.2
Correlation
CO1.1 Use cases are translated into specific correlation rules within DA.AE-3 CF10.4.8
the SIEM tooling
CO1.2 Correlation rules are periodically checked on desired result PR.IP-7 CF10.4.9
DE.DP-5
CO1.3 Modification of correlation rules is limited to authorised users 9.4.1 DSS05.04 - 1 PR.AC-4
only DSS06.03 - 1, 2
65

Alerting
AL1.1 Triggers and alerts defined in use cases are implemented DE.AE-5 CF10.5.1
CF10.6.4
AL1.2 Alerts contain relevant information
Responding
RE1.1 Alerts are analysed by the SOC, and sent to incident 16.1.1 DSS02.02 - 1 RS.AN-1 12.5.2 CF10.4.10
management process if applicable DSS05.07 - 3 RS.AN-2 12.5.3
RE1.2 Alerts generated by high impact use cases automatically DSS05.07 - 5
generate an incident
Evaluation
EV1.1 Use cases are adjusted if applicable 16.1.6 MEA02.01 DE.DP-5 CF11.1.7
EV1.2 Reports are defined on the needs of the organisation EDM05.01 - 1, DE.DP-4 SI2.1
2, 3
66
11.2 Interview questions

This appendix contains the interview questions that were used as guidance for all interviews
conducted with the selected financial institutions. In addition to the interview questions below,
the SIEM focus areas and defined minimum requirements were used as guidance.
1. What is the structure of your IT security organisation?

2. To whom or what department do you report?
3. When did you start implementing your SIEM environment and/or tooling?
4. Did you use frameworks or best practices during the implementation phase?
a. If so which ones?
5. Did the business or management provide any requirements?
6. What is the size of your security team?
7. Are roles and responsibilities assigned regarding SIEM
8. Are specialized SIEM skills defined?
a. If so, what are these?
9. How do you determine what to monitor?
a. Are crown jewels defined (e.g. based on risk assessments)?
b. Are you using risk scenarios (i.e. based on defined crown jewels)?
c. Did you perform a threat-actor analysis?
d. Do you define and implement use cases? If so:
i. How are use cases structured?
ii. How are use cases defined and based on what?
iii. How are use cases maintained?
10. Which input sources do you use for your SIEM environment?
11. How do you know if your SIEM tooling and correlation rules are working correctly?
67
11.3 Mapping of selected organisations

This appendix contains the mapping of the requirement coverage of selected organisations, to
our set of minimum requirements per focus area that should be addressed when implementing
SIEM.
# Subject FI1 FI2

ORR1.1 The reason for implementing SIEM is understood and supported by C-level ++ +/
ORR1.2 Organisational requirements for SIEM are defined based on: +/

- Crown jewels +/
- Risk appetite +/ +/
- Legislative or regulatory requirements ++ +
- Reporting requirements
ORR1.3 Roles and responsibilities regarding SIEM are assigned ++ +
ORR1.4 Security team contains members with relevant skills ++ +
OPR1.1 SIEM tooling requirements are defined +
OPR1.2 SIEM tooling is selected based on market research using defined
requirements
+
OPR1.3 SIEM tooling is implemented ++ ++
OPR1.4 Risk scenarios are defined based on: ++
- Crown jewels +/
- Risk appetite +
- Legislative and regulatory requirements +
- Previous security incidents +
OPR1.5 Use cases are defined based on: ++
- Scenarios ++
- Common attacks and attack paths ++ +/
- Cyber kill chain phases ++ +/
OPR1.6 Threat intelligence (TI) is gathered from (external) sources ++ +/
OPR1.7 TI and changes in threat landscape are used to create/modify use cases
periodically
++ +/
Log management
LM1.1 Log policies are defined and should at least include: ++ ++
- Logs should be accurate + +
- Logs should be stored in a central and tamper-proof location + +
- Log retention period per log type (based on needs and legislation) + +
68
# Subject FI1 FI2

- Logs are time stamped + +
- The same time source is used for clock synchronisation + +
- Log permissions are restrictive + +
- Log format (i.e. syslog, windows event logs) - +
- System is not halted through lack of disk space + +
- Logs contain originating source + +
LM1.2 Technical baselines are created and implemented based on log policy ++ ++
LM1.3 Adhering to the log policy is part of operating procedures such as: +/ ++
- On boarding of new devices and systems +/ ++
- Changing devices and systems +/ +
LM1.4 Centralized log requirements are defined ++ ++
- Only authorised users have access to log files ++ ++
- Capacity of centralised logging infrastructure is managed +/ +
LM1.5 Log sources are derived based on use cases +
LM1.6 To be collected log entries are defined based on use cases +
LM1.7 Controls for checking if expected logs are received are implemented ++ ++
LM1.8 Logs are normalised ++ ++
Correlation
CO1.1 Use cases are translated into specific correlation rules within the SIEM
tooling
++
CO1.2 Correlation rules are periodically checked on desired result ++
CO1.3 Modification of correlation rules is limited to authorised users only ++ ++
Alerting
AL1.1 Triggers and alerts defined in use cases are implemented ++ +/
AL1.2 Alerts contain relevant information ++ ++
Responding
RE1.1 Alerts are analysed by the SOC, and sent to incident management process if
required
++ +
RE1.2 Alerts generated by high impact use cases automatically generate an
incident
++
Evaluation
EV1.1 Use cases are adjusted if applicable + +/
EV1.2 Reports are defined on the needs of the organisation +/ +/
69

Addressing SIEM

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Addressing SIEM

Încărcat de

Drepturi de autor:

Formate disponibile

vrije universiteit Amsterdam

Determining focus areas

Authors Jarno van de Moosdijk vandemoosdijk.jarno@kpmg.nl

4 SIEM explained .......................................................................................................................................13

5 SIEM in IT risk frameworks...............................................................................................................22

5.2.2 Operational requirements ............................................................................................... 27

6 SIEM case study ......................................................................................................................................38

8 Research questions revisited ...........................................................................................................52

Frameworks mapped on focus areas

Responding Log management

ISO COBIT NIST PCI-DSS ISF

Graph 1: Focus area coverage

One would expect an organisation to expose only the highly

According to the annual Verizon data breach investigation report

In response to the high number of attacks and their increased level

3.2 Research motivation

3.4 Research questions

3.4.1 Central research question

3.4.2 Sub question 1: SIEM explained

What is SIEM and why is it relevant?

3.4.3 Sub question 2: SIEM in IT risk frameworks

3.4.4 Sub question 3: SIEM case study

How is SIEM addressed in practice?

3.5 Research approach

Figure 2: Research approach

3.6 Related work

Vrije Universiteit Amsterdam:

What is SIEM and why is it relevant?

4.1 The definition of SIEM

Figure 3: Overview of aspects related to SIEM

4.1.1 For threat management

4.1.2 For compliance

4.1.3 For forensic support

4.2 SIEM versus SOC

A SOC is a collection of people, processes

4.3 Shortcomings of traditional SIEM implementations

4.4 SIEM focus areas

4.4.1 Knowing what to protect

Table 1: example use case

4.4.2 Log management

Figure 5 shows that collecting log entries from

As the amount of log entries quickly grows, adequate

Correlation of log entries is performed

Figure 7: Alerts generated by correlation rules

4.4.5 Responding & Evaluating

Figure 8: Incidents generated by alerts

5 SIEM in IT risk frameworks

5.1 Finalising SIEM focus areas

5.1.1 Initial focus areas

Figure 9: Position of SIEM and its focus areas

5.1.2 Validation by field-experts

Name Function Organisation

5.2 Requirements per focus area

5.2.1 Organisational requirements

5.2.2 Operational requirements

OPR1.5 Use cases are defined

OPR1.6 Threat intelligence (TI) is gathered from (external) sources

5.2.3 Log management requirements

LM1.4 Log sources are derived based on use cases

LM1.5 To be collected log entries are defined based on use cases

LM1.7 Logs are normalised

5.2.4 Correlation requirements

CO1.2 Correlation rules are periodically checked on desired result

CO1.3 Modification of correlation rules is limited to authorised users only