IBM

QRadar 7.
1 Administration and Deployment

Student’s Training Guide
Course: TXNNNG ERC: X.N
November 2012
© Copyright IBM Corp. 2012. All Rights Reserved.
US Government Users Restricted Rights: Use, duplication or disclosure restricted by GSA ADP
Schedule Contract with IBM Corp.
IBM, the IBM logo, and ibm.com are trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and service names might be trademarks of
IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and
trademark information” at www.ibm.com/legal/copytrade.shtml.
Trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other
countries.
IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications
Agency which is now part of the Office of Government Commerce.
Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon,
Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation
or its subsidiaries in the United States and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in
the United States, other countries, or both.
ITIL is a registered trademark, and a registered community trademark of the Office of Government
Commerce, and is registered in the U.S. Patent and Trademark Office.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle
and/or its affiliates.
Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States,
other countries, or both and is used under license therefrom.
Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM
Corp. and Quantum in the U.S. and other countries.
The information contained in this publication is provided for informational purposes only. While
efforts were made to verify the completeness and accuracy of the information contained in this
publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this
information is based on IBM’s current product plans and strategy, which are subject to change by
IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or
otherwise related to, this publication or any other materials. Nothing contained in this publication is
intended to,nor shall have the effect of, creating any warranties or representations from IBM or its
suppliers or licensors, or altering the terms and conditions of the applicable license agreement
governing the use of IBM software.
References in this publication to IBM products, programs, or services do not imply that they will be
available in all countries in which IBM operates. Product release dates and/or capabilities referenced
in this presentation may change at any time at IBM’s sole discretion based on market opportunities
or other factors, and are not intended to be a commitment to future product or feature availability in
any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or
implying that any activities undertaken by you will result in any specific sales, revenue growth,
savings or other results.
Printed in Ireland
Table of contents
Unit 1: Introduction
Introduction . . . . . . . . . . . . 1-2
Objectives . . . . . . . . . . . . . 1-2
Lesson 1: IBM Security Framework (ISF) . . . . . . . 1-3
ISF maturity categories . . . . . . . . . . . 1-4
Maturity categories and security software types . . . . . . . 1-5
IBM security software portfolio and QRadar SIEM . . . . . . . 1-6
Lesson 2: QRadar brief history . . . . . . . . . 1-7
QRadar key Capabilities . . . . . . . . . . . 1-8
Security Intelligence at work . . . . . . . . . . 1-9
Ease of deployment . . . . . . . . . . . 1-10
Clear leadership position in SIEM Magic Quadrant . . . . . . . 1-11
QRadar main product differentiators . . . . . . . . . 1-12
Lesson 3: QRadar software components and data flow . . . . 1-13
QRadar software components . . . . . . . . . . 1-14
QRadar appliance components and data flow . . . . . . . 1-15
QRadar appliance types . . . . . . . . . . . 1-16
QRadar Log Management versus SIEM . . . . . . . . 1-17
Lesson 4: QRadar distribution . . . . . . . . . 1-18
Summary . . . . . . . . . . . . . 1-19
Unit 2: QRadar deployment and configuration

Introduction . . . . . . . . . . . . 2-2
Objectives . . . . . . . . . . . . . 2-2
Lesson 1: QRadar appliances . . . . . . . . . 2-3
QRadar flow collector appliances . . . . . . . . . 2-4
QRadar Event and Flow Processor Appliances . . . . . . . 2-5
QRadar All in one Appliances . . . . . . . . . . 2-6
Lesson 2: Example small áll in one´ deployment . . . . . 2-7
Example medium distributed deployment . . . . . . . . 2-8
Example: large size company deployment . . . . . . . . 2-9
Lesson 3: Estimating EPS and FPM . . . . . . . . 2-10
Estimating by scope . . . . . . . . . . . 2-11
Lesson 4: QRadar disk utilization parameters . . . . . . 2-12
Retention buckets . . . . . . . . . . . . 2-13
Log Storage Calculation . . . . . . . . . . . 2-14
Lesson 5: Backup storage . . . . . . . . . . 2-15
Usage of Fiber Channel . . . . . . . . . . . 2-16
Technical preparations . . . . . . . . . . . 2-17
Disk performance considerations . . . . . . . . . 2-18
Lesson 6: Distributed deployment across Wide Area Network . . . 2-19
Distributed deployment using a 1501 appliance . . . . . . . 2-21
Student exercise . . . . . . . . . . . . 2-22
•
•
• III
•
•
Table of contents
Summary . . . . . . . . . . . . . 2-23
Unit 3: QRadar Software installation

Introduction . . . . . . . . . . . . . 3-2
Objectives . . . . . . . . . . . . . 3-2
Lesson 1: Prerequisites . . . . . . . . . . . 3-3
Preparations . . . . . . . . . . . . . 3-4
Software Installation of QRadar V7.1 . . . . . . . . . 3-5
Check the media . . . . . . . . . . . . 3-6
RHEL 6.2 installed . . . . . . . . . . . . 3-7
License agreement . . . . . . . . . . . . 3-8
Enter the activation key . . . . . . . . . . . 3-9
Choose the type of setup . . . . . . . . . . . 3-10
Apply the Enterprise tuning template . . . . . . . . . 3-11
Setup the time server . . . . . . . . . . . . 3-12
Time server connectivity . . . . . . . . . . . 3-13
Setup the management network interface . . . . . . . . 3-14
Enter the network information of the appliance . . . . . . . . 3-15
Installation completed . . . . . . . . . . . . 3-16
Configure the network interface for QFlow (optional) . . . . . . . 3-17
Configure the QFlow Network Device . . . . . . . . . 3-18
Specify the IP addresses to use . . . . . . . . . . 3-19
Lesson 2: Basic configuration of the All in One 31xx appliance . . . 3-20
Update the license . . . . . . . . . . . . 3-21
Configure the Flow Collector (optional) . . . . . . . . . 3-22
Add extra interfaces for the Event Collector (optional) . . . . . . . 3-23
Lesson 3: Physical Setup . . . . . . . . . . 3-24
Physical setup X3630M3 . . . . . . . . . . . 3-25
Lesson 4: Post Installation actions . . . . . . . . . 3-26
Update all DSM and Protocols after patching . . . . . . . . 3-27
Configure auto update . . . . . . . . . . . . 3-28
Choose the update policy . . . . . . . . . . . 3-29
Lesson 5: Adding non-Console appliances . . . . . . . 3-30
Adding the managed host . . . . . . . . . . . 3-31
Lesson 6: High Availability Overview . . . . . . . . 3-32
Install a HA appliance with the same role . . . . . . . . . 3-33
Configure the network . . . . . . . . . . . . 3-34
High Availabilty - Management . . . . . . . . . . 3-35
High Availabilty - Details . . . . . . . . . . . 3-36
High Availability - Deployment Options . . . . . . . . . 3-37
High Availability - Disk Synchronization . . . . . . . . . 3-38
High Availability - Shared Storage . . . . . . . . . . 3-39
Summary . . . . . . . . . . . . . 3-41
Unit 4: QRadar architecture

Introduction . . . . . . . . . . . . . 4-2
Objectives . . . . . . . . . . . . . 4-2
Lesson 1: High level Architecture . . . . . . . . . 4-3
Lesson 2: Flow collector architecture . . . . . . . . 4-4
•
•
IV • IBM Tivoli Course ©Copyright IBM Corp. 2012
•
•
Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
Table of contents
Application detection . . . . . . . . . . . 4-5

Superflows . . . . . . . . . . . . . 4-6
Lesson 3: Event Collector architecture . . . . . . . 4-7
FPM and EPS burst handling . . . . . . . . . . 4-8
Log Source parsing uses QID mapping . . . . . . . . 4-9
Mapping definitions are stored in PostgreSQL . . . . . . . 4-10
Log Source Event ID to QIDmap in DSMevent table . . . . . . 4-11
QIDmap to QID mapping in QIDmap table . . . . . . . . 4-12
Custom property extraction . . . . . . . . . . 4-13
Optional Event Coalescing . . . . . . . . . . 4-14
Auto discovery of Log Sources . . . . . . . . . . 4-15
Lesson 4: Event Processor architecture . . . . . . . 4-16
Correlation Rules Engine . . . . . . . . . . . 4-17
Accumulator . . . . . . . . . . . . . 4-18
Accumulator structure . . . . . . . . . . . 4-19
Lesson 5: Console architecture . . . . . . . . . 4-20
Offense management by the Magistrate . . . . . . . . 4-21
Events or flows tagging by matching rules . . . . . . . . 4-22
Offense types . . . . . . . . . . . . 4-23
ADE (Anomaly detection Engine) . . . . . . . . . 4-24
Anomaly Detection Engine Rule Types . . . . . . . . 4-25
New asset and service detection by VIS . . . . . . . . 4-26
Lesson 6: Datastorage Technology . . . . . . . . 4-27
Ariel datastorage in /store/ariel on processors . . . . . . . 4-28
Summary . . . . . . . . . . . . . 4-30
Unit 5: Solution implementation

Introduction . . . . . . . . . . . . 5-2
Objectives . . . . . . . . . . . . . 5-2
Lesson 1: QRadar solution scope . . . . . . . . 5-3
Forensic analysis use case requirements . . . . . . . . 5-4
Compliancy audit use case requirements . . . . . . . . 5-5
Security policy violations use case requirements . . . . . . . 5-6
IT security risk management use case requirements . . . . . . 5-7
Scope of Log Sources and Network Segments . . . . . . . 5-8
QRadar licensing sizing parameters . . . . . . . . . 5-9
QRadar disk storage sizing parameters . . . . . . . . 5-10
Design QRadar appliance topology . . . . . . . . . 5-11
Data source groups examples . . . . . . . . . . 5-12
Lesson 2: Suggested default Log Activity reports . . . . . 5-13
Suggested network activity reports . . . . . . . . . 5-14
Suggested threat analysis network activity reports . . . . . . . 5-15
Lesson 3: Deployment steps . . . . . . . . . 5-16
Lesson 4: Create a QRadar network hierarchy . . . . . . 5-17
Setup Network Hierarchy . . . . . . . . . . . 5-18
Manage Network Hierarchy distribution . . . . . . . . 5-19
Example . . . . . . . . . . . . . 5-20
Accessing the Network Hierarchy in the User Interface . . . . . . 5-21
Example of Network Hierarchy groups . . . . . . . . 5-22
More detailed Network Hierarchy by using sub groups . . . . . . 5-23
•
©Copyright IBM Corp. 2012 IBM Tivoli Course • V
•
•
•
Table of contents
Lesson 5: Populate the Asset profile database . . . . . . 5-25

Search the Asset profile database . . . . . . . . . . 5-26
Commonly edited Building Blocks . . . . . . . . . . 5-27
Lesson 6: Configuring Vulnerability Assessment . . . . . . 5-28
Add Vulnerabilty Scanners . . . . . . . . . . . 5-29
Lesson 7: Collecting eventlogs with ALE . . . . . . . 5-30
License Agreement . . . . . . . . . . . . 5-31
Installation directory . . . . . . . . . . . . 5-32
Installation type . . . . . . . . . . . . . 5-33
Shortcuts and access . . . . . . . . . . . . 5-34
Configure the collection . . . . . . . . . . . 5-35
Configure syslog forwarding . . . . . . . . . . . 5-36
Assign event source to syslog destination . . . . . . . . 5-37
Remote collect Windows events . . . . . . . . . . 5-38
Configure remote collect Windows events . . . . . . . . 5-39
Check access to remote Windows events . . . . . . . . . 5-40
Use WinCollect instead of ALE . . . . . . . . . . 5-41
Prepare QRadar Console . . . . . . . . . . . 5-42
Install WinCollect with Token . . . . . . . . . . 5-43
Create a Log Source . . . . . . . . . . . . 5-44
Check collection . . . . . . . . . . . . 5-45
Lesson 8: Configure authentication method . . . . . . . 5-46
Select authentication method . . . . . . . . . . 5-47
Creating users in the remote directory . . . . . . . . . 5-48
Add the user to QRadar . . . . . . . . . . . 5-49
Lesson 9: Troubleshooting Connectivity . . . . . . . . 5-51
Troubleshooting Data collection . . . . . . . . . . 5-52
Analyze event collection . . . . . . . . . . . 5-53
Check for Dropped Events & Flows (pre V7.1) . . . . . . . . 5-54
Logs and Messages . . . . . . . . . . . . 5-55
QRadar Setup reconfiguration . . . . . . . . . . 5-56
Additional Command Line Tips.. . . . . . . . . . . 5-57
Summary . . . . . . . . . . . . . . 5-59
Unit 6: Custom Log Sources

Introduction . . . . . . . . . . . . . 6-2
Objectives . . . . . . . . . . . . . 6-2
Lesson 1: Create Custom Log Sources . . . . . . . . 6-3
Required tools . . . . . . . . . . . . . 6-4
7 Stages to get to full Custom Log Source support . . . . . . . 6-5
Lesson 2: Obtain the sample (from remote location) . . . . . 6-6
Lesson 3: Upload the LSX_Template.xml file . . . . . . . 6-7
Create a Universal DSM Log Source . . . . . . . . . 6-8
Test the Universal DSM Log Source . . . . . . . . . 6-9
Lesson 4: Start mapping the unknown logrecords . . . . . . 6-11
Example of a LEEF log record event ID . . . . . . . . . 6-12
Create RegEx to extract the Log Source EventID . . . . . . . 6-13
Lesson 5: Creating appropriate regular expressions . . . . . 6-14
•
•
VI • IBM Tivoli Course ©Copyright IBM Corp. 2012
•
•
Table of contents
Common regular expressions . . . . . . . . . . 6-15

Use of capture groups . . . . . . . . . . . 6-16
regular expression recommendation . . . . . . . . . 6-17
Lesson 6: Apply RegEx patterns to the LSX . . . . . . 6-18
Cleanup the LSX Template . . . . . . . . . . 6-19
Rerun the test . . . . . . . . . . . . 6-20
Lesson 7: The value of QIDs . . . . . . . . . 6-22
Create a new QID entry using qidmap_cli.sh . . . . . . . 6-23
Lesson 8: Map the Log Source ID to the Custom QID . . . . 6-24
Map the Log Source Event IDs to existing QIDs . . . . . . . 6-25
Test the mapping . . . . . . . . . . . . 6-26
Final remarks . . . . . . . . . . . . 6-27
Summary . . . . . . . . . . . . . 6-29
Summary . . . . . . . . . . . . . 6-30
Unit 7: Rules creation and fine tuning

Introduction . . . . . . . . . . . . 7-2
Objectives . . . . . . . . . . . . . 7-2
Lesson 1: QRadar Rules reminder . . . . . . . . 7-3
QRadar Building Block reminder . . . . . . . . . 7-4
Linked tests . . . . . . . . . . . . . 7-5
Linking tests in the right order . . . . . . . . . . 7-6
Custom Rule Engine - CRE . . . . . . . . . . 7-7
Lesson 2: Using Building Blocks . . . . . . . . 7-8
Combine Building Blocks to capture specific events or flows . . . . . 7-9
Lesson 3: Rule creation . . . . . . . . . . 7-10
Rule to capture account creation . . . . . . . . . 7-11
Rule to capture access to sensitive data . . . . . . . . 7-12
Rule to capture account deletion . . . . . . . . . 7-13
Combine the rules to the Offense Rule . . . . . . . . 7-14
Lesson 4: Offense analysis . . . . . . . . . 7-15
Analyse the Offense summary . . . . . . . . . . 7-16
Check the rules that fired for the Offense . . . . . . . . 7-17
View events or flows contributing to the Offense . . . . . . . 7-18
Example of an Attack scenario . . . . . . . . . . 7-19
Could be detected by this rule . . . . . . . . . . 7-20
Lesson 5: False positive management . . . . . . . 7-22
Example 1 . . . . . . . . . . . . . 7-23
Example 2: Botnet access - determine the rule . . . . . . . 7-24
Create a search for the contributing events . . . . . . . . 7-25
Use the False Positive wizard . . . . . . . . . . 7-26
Example 3: Capture the events first and decide later . . . . . . 7-27
Finde the rules with the highest offense counts . . . . . . . 7-28
Use the Rule to capture the events in a report . . . . . . . 7-29
Analyze the report . . . . . . . . . . . . 7-30
Lesson 6: Tuning Methodology . . . . . . . . . 7-31
Commonly Edited Building Blocks (reminder) . . . . . . . 7-32
•
©Copyright IBM Corp. 2012 IBM Tivoli Course • VII
•
•
•
Table of contents
Summary . . . . . . . . . . . . . 7-34
•
•
VIII • IBM Tivoli Course ©Copyright IBM Corp. 2012
•
•
Unit 1 Introduction
© 2 012 IBM Corp .
•
•
• 1-1
•
•
Introduction
This unit describes the QRadar product and its components and functions.
Objectives
Objectives
When you complete this unit, you can perform the following tasks:
 Understand the QRadar positioning
 Know the QRadar software and appliances
IBM Sof tware Group | Securit y Division

© 2012 I BM Corp.
Overview
•
•
1-2 • IBM QRadar Implementation ©Copyright IBM Corp. 2012
•
•
Lesson 1: IBM Security Framework (ISF)
Lesson 1:IBM Security Framework (ISF)
ISF recognises 6 security

domains.
Software and appliances

for each of these doamins
can either be of the
security enablers or
security controllers type.
Depending on the maturity

of the security framework
implementation one will
find either of these types in
the domains.

© 2012 I BM Corp.
•
•
©Copyright IBM Corp. 2012 IBM QRadar Implementation • 1-3
•
•
ISF maturity categories
ISF maturity categories
d
te
a
m
o
t
u
A
Optimized
Or ga ni za ti on s u se
p re di ctive a n d
a u to ma ted se cur ity
a n al ytics to d riv e to wa rd
s ecu ri ty i nte lli ge n ce
Basic l
a
O rga ni za tio ns u
em pl oy p eri me ter n
a Proficient
pr ote ctio n, w hi ch M Se cu rity is l aye re d
re g ul ate s ac ces s an d in to the IT fa br ic a nd
fe ed s man u al re po rtin g bu si ne ss op e rati on s
Reactive Proactive
© 2012 I BM Corp.
•
•
•
•
Maturity categories and security software types
Maturity categories and security software types

Security In telligen ce:
Information and event management
Secu rity Advanced correlation and deep analytics
In te ll ig ence External threat research
Optimized Ad vance d ne tw ork
Rol e b ased a nal ytics Sec ure ap p mo nitor ing
Id entity g ove rnan ce D ata fl ow anal ytic s e ngi nee ring
pr oces ses For ensi cs / da ta
Privi le ged user Data g overn ance min ing
con trols Frau d d etection
Sec ure s ystems
Vir tu ali zation secu rity

U ser p rovi sio nin g Ap pli catio n fire wal l
Acces s mon itori ng Asset mg mt
P rofic ie nt Ac cess mgmt Sourc e cod e
Data los s pre ventio n Endp oin t / n etwork
Stro ng a uthenti catio n sca nni ng
se curi ty manag eme nt
Enc rypti on Pe rimeter secu rity

Ba s ic Ce ntral ized dir ectory Ap pli cati on sc anni ng
Acces s co ntrol Anti-v iru s
People Data Applications Infrastructur e

© 2012 I BM Corp.
•
•
•
•
IBM security software portfolio and QRadar SIEM
IBM security software portfolio and QRadar SIEM

© 2012 I BM Corp.
•
•
•
•
Lesson 2: QRadar brief history

• 1999 University of New Brunswick, Fredericton, Canada, introduces
high speed network. Gradually updating to Gigabit Ethernet. Network
Operations required a tool for network information, surveillance, and
analysis.
• 2001 This resulted in a product called QVision positioned for the

Network Behaviour and Anomaly Detection (NBAD) products space.
• 2002 Log files from network devices are pulled into QRadar to
correlate flows with log records. Early SIM functionality (V4.3)
• 2005 V5.0 LogSource Extensions, DSMs, Rules are added,

including the introduction of Ariel datastructure.
• 2011 V7.0 introduces optimized event and flow pipelines.
•2012 IBM acquires Q1Labs. QRadar bluewashed and runs only on

RHEL 6.2 IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
QRadar key Capabilities
Qradar Key Capabilities

 Correlation, normalization and secure storage of events,
flows, assets, topologies, vulnerabilities and external data
 Reporting, searching, forensics against this data
 Layer 7 flow capture and analysis for deep application insight
 Offense (incident) scoring & management, with linked
forensic data
 Full workflow management to track and resolve threats
 Compliance-driven report templates for regulatory reporting
and auditing – PCI, SOX, FISMA, HIPAA, NERC, etc.
 Reliable, tamper-proof log storage for forensic investigations
and evidentiary use
 Scalable architecture to support the largest deployments
© 2012 I BM Corp.
•
•
•
•
Security Intelligence at work
Security intelligence at work
2 Bn security records per day 25 security offenses per day
•Reliable, secure and scalable log data storage

•Advanced security data correlation turning data into information
•Advanced and easy to use rule based security event correlation
engine to extract the real security offenses
© 2012 I BM Corp.
•
•
•
•
Ease of deployment
Ease of deployment: QRadar has many automations
Automated Automated offense

discovery of log prioritization
sources, Automated update
Monitor Analyze of threats
applications and
assets Automated
Automated asset response
type grouping Directed
Automated self Act remediation
audits
Auto-tuning of rules
Auto-detect threats
using VIS
Best practice rules and
role based reports IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Clear leadership position in SIEM Magic Quadrant
Clear leadership position in SIEM Magic Quadrant
IBM-ers check
Product sales Kit for
more information on
differentiators.

© 2012 I BM Corp.
http://w3-103.ibm.com/software/xl/portal/content?synKey=B885344U22687H00#assets
•
•
•
•
QRadar main product differentiators
QRadar main product differentiators

 Ease of deployment
 Scalability
 Mimimal customization required. Many predefined (compliance)
reports. Many predefined correlations rules
 Native Layer 7, netw ork package contents, capture and analysis
 Proofen data storage technology

© 2012 I BM Corp.
•
•
•
•
Lesson 3: QRadar software components and data flow
Lesson 3: QRadar software components and

data flow
Lesson 3: QRadar software components and data

flow
Network Logs
Flow Colle ctor Eve nt Colle ctor • Network information

collected from 3rd party
network flows, and from
onboard network card
Event Pr ocess or
interfaces.
• Logs collected from

devices, applications,
U ser Cons ole databases, or operating
systems.
© 2012 I BM Corp.
•
•
•
•
QRadar software components
QRadar software components

 Central User Console
• Magistrate (manages offense creation and magni tude)
• Global correlation across flow and event processors
• Offense management
• Assets and identity management
 Event Processor
• Rule Proce ssor
• Storage for events, accumul ated me ta data
• Storage for flows, accumulated meta d ata
 Event Collector
• Log event collection, coalescing, and normalization
• 3rd party Flow col lection J-Flow, NetFl ow, S-Fl ow, dedupl ication, and
recomb ination
 Flow Collector
• QFlow and Superflow creation, an d applicati on detection
© 2012 I BM Corp.
•
•
•
•
QRadar appliance components and data flow
QRadar appliance components and data flow
Network Network
Logs Logs • Network flows preferably
packages flows
are processed by a seperate
flow processor appliance
Flow Colle ctor Event c ollect or • Event logs preferably are

processed by a seperate
event processor appliance
Flow Pr oc essor Event Pr oc essor • A flow collector is required

if layer 7 data analisys is
required.
Console
•An event collector is used
where bandwith between log
sources and log storage is
critical IBM Sof tware Group ©| Securit y Division
2012 I BM Corp.
•
•
•
•
QRadar appliance types
QRadar appliance types

 Console only. Must be combined with event or flow
processors, and flow collectors.
 Server. For medium and enterprise environments. Combines
a flow processor and event processor in one
 Event Processor . Processes and stores event logs.
 Flow Processor. Processes 3rd party network flows, QFlow
and stores flows.
 Flow Collector. Receives 3rd party network flows and
packages. Normalizes and forwards them as QFlows.
 Event Collector. Receives log records, normalizes and
forwards them to an event processor. Temporary storage of
normalized log events and payload.
© 2012 I BM Corp.
•
•
•
•
QRadar Log Management versus SIEM
QRadar Log Management versus SIEM

 QRadar Log Management solution collects, normalizes,
stores and reports on security log data.
 QRadar SIEM adds to QRadar Log Management advanced
data and security information correlation functionality and
network data capture, analysis and storage.
 Security offense generation and management is not available
in QRadar Log Management
 Automated asset discovery and vulnerability information
integration is not available in QRadar Log Management
 Network data analysis is not available in QRadar Log
Management

© 2012 I BM Corp.
•
•
•
•
Lesson 4: QRadar distribution

 Current version is Qradar V7.1
 Software install on RHEL 6.2 on Intel hardware.
 Or xSeries Appliance (recommended)
Scale
$
© 2012 I BM Corp.
•
•
•
•
Summary
Summary
Summary
Now that you have completed this unit, you can perform the following tasks:
 Understand the QRadar positioning
 Know the QRadar software and appliances

© 2012 I BM Corp.
•
•
•
•
Summary
•
•
•
•
Unit 2: QRadar deployment and
configuration
Unit 2 QRadar Appliances
© 20 12 IBM C orp.
•
•
• 2-1
•
•
Introduction
This unit decribes QRadar appliances and how tPut your introduction text here. Do not delete the
anchor because it is the anchor for the cross-reference to the description in the preface.
Objectives
Objectives
 Explain the QRadar appliances offering
 Estimate the number of Events per second and Flows per minute for a
QRadar deployment.

© 2012 I BM Corp.
•
•
2-2 • IBM Tivoli Course ©Copyright IBM Corp. 2012
•
•
Lesson 1: QRadar appliances
Lesson 1: QRadar Appliances
The following appliance types are available. The table maps them
on the QRadar appliance components.
Flow Flow E vent E vent Se rver Console

Colle ctor Processor Collector P rocessor
1201, 1202, 1705, 1724 1 501 160 5, 1624 1805 2100, 3105,
1301,1310 3124
Hardware used are IBM XSeries: X3550M3 and X3630M3

© 2012 I BM Corp.
•
•
©Copyright IBM Corp. 2012 IBM Tivoli Course • 2-3
•
•
QRadar flow collector appliances
QRadar Flow Collector Appliances
Model Max. network speed

1201 200 Mbps Base-T
1202 2 Gbp s Base-T
1301 2 Gbp s Base SX Fiber
1310 10 Gb ps XFP
QRadar Event Collector Appliances

Model Max EPS sustained Storage
1501 2 500 1.3 TB

© 2012 I BM Corp.
•
•
•
•
QRadar Event and Flow Processor Appliances
QRadar Event and Flow Processor Appliances
EP Model Max EPS sustained S torage

1605 2 0000 6.2 TB
1624 2 0000 16 TB
FP Model Max FPM s ustained S torage

1705 6 00000 6.2 TB
1724 1 200000 16 TB
Server Model Max EPS Max FPM Storage

sustained sustained
1805 5000 20 0000 6.2 TB

© 2012 I BM Corp.
•
•
•
•
QRadar All in one Appliances
QRadar All in one Appliances
All in one Model Max EP S Max FPM Storage

sustained sustained
2100 1000 500 00 1.3 TB
3105 5000 200 000 6.5 TB
3124 5000 200 000 16 TB
There are also console only versions of the 31xx series

© 2012 I BM Corp.
•
•
•
•
Lesson 2: Example small áll in one´ deployment
Lesson 2: Example small áll in one´

deployment
Lesson 2: Example small ‘all in one’ deployment
Single 2100 console
-Event log collection and

processing
-Flow and network
package collection, and
processing

© 2012 I BM Corp.
•
•
•
•
Example medium distributed deployment
Example medium distributed deployment
3100 series i/c with two

1201 flow collectors
-3100 series for event log

collection, processing,
flow processing and
console
-1201 for flow and

network package
collection

© 2012 I BM Corp.
•
•
•
•
Example: large size company deployment
Example enterprise distributed deployment
3100 series i/c with 1201

flow collector, 1705 flow
processor and 1605 event
processor
1705 -3100 series for console

-1201 for flow collection
1605
-1605 for event log
collection and processing
-1705 for flow processing

© 2012 I BM Corp.
•
•
•
•
Lesson 3: Estimating EPS and FPM

Figures based on 10 years of field experience in network monitoring:
• every user generates 15 FPM and 3 EPS

• every server generates 200 FPM and 15 EPS
• 100 Mbps line speed fully utilized, equals roughly to 50000 FPM
Best Practice Est imates per server type:

AD Servers 15 EPS
IIS/Exchange 10 EPS
DNS/DHCP Servers 15 EPS
Firewall 150 EPS
Pro xy Servers 25 EPS
IDS/IPS/ IDP 5 EPS
VPN 5 EPS
Ro uters and Switches .25 EPS

© 2012 I BM Corp.
•
•
•
•
Estimating by scope
Estimating by scope
 FPM and EPS sizing depends on the parts of the network and the
machines that need to be made visible in QRadar.
• Assume an organization with 15.000 users, 10 di re ctory servers an d 1000
servers.
• Assume that the scope is determined by the PCI regulati on
• Assume 10 PCI servers a nd 2 directory servers in scope
• Assume 250 users with access to PCI machines in scope
• EPS sizing includes only the 250 users and the 2 directo ry servers.
Th erefore we can assume an approxi mate event rate of 780 EP S
• FP M sizing includes the 250 u sers and 10 PCI servers. Therefore we can
assu me a flow rate of 5 750 FPM
• Try to use appliances that would NOT be utilized more than 75% of the sustained EPS or
FPM capacity
• A single 2100 appliance would be sufficient to monitor this PCI

environment. But log retentio n time may become a critical factor.
© 2012 I BM Corp.
•
•
•
•
Lesson 4: QRadar disk utilization parameters
Lesson 4: QRadar disk utilization

parameters

 Record retention time is defined by retention buckets.
 A retention bucket defines a filter and contains matching records that are
subject to the compression and deletion policy defined for the retention
bucket.
 The default retention bucket captures ´left overs´. Records that are not
captured by any custom made retention bucket.
 Disk size is determined by the regulatory requirements regarding
retention time in combination with the amount of logs received in 1 hour.
 Smart retention policy means delete when storage space is required.
(Default configuration)
 Default configuration: Record deletion starts at 87% and stops at 82%
disk utilization. At 95% disk utilization the Qradar system will stop
receiving data.

© 2012 I BM Corp.
•
•
•
•
Retention buckets
Retention buckets
Default: records older than 6 hours are compressed and when disk
space is required (87%), the oldest records are deleted untill either
the 30 days of data is kept on disk or disk utilization reaches 82% .
© 2012 I BM Corp.
•
•
•
•
Log Storage Calculation
Log Storage Calculation

 Assume the 780 EPS are received by a 2100 appliance
 Assume that the average log record is approximately 600 Bytes.
 Stored in Ariel the size increases to 150% on average. ~900 Bytes
 780 * 900 * 3600 ˜ 2.4 GB/hour
 O r approximately 56 GB/day
 87% of 1.3TB = 1.1 TB which means that the 2100 appliance can keep
up to 20 days of uncompressed log records before it will start deleting
records.
 The compression factor is between .1 and .15 . Only payload
information is compressed. Record indexes remain uncompressed
 Essentially 1 GB of uncompressed log record Ariel storage per day is
approximately equal to 13 EPS.
 The actual storage needed per day depends on the deviation around
the 13 EPS mean value on a normal labor day.
© 2012 I BM Corp.
•
•
•
•
Lesson 5: Backup storage
Lesson 5 : Backup storage

 Start making daily backups from day 1.
 Daily copy the backup from the appliance to external storage
• IPSAN (IP Storage A rea Ne twork) through ISCSI
o Fiber Channel option (see next sli de)
o Offli ne storage for backups
o S hould not be used fo r online storage because QRadar i s I/O inten sive
• NAS (Network A ttached Storage) NFS
o Offli ne storage for backups
o Cannot be used for online storage; hi gh insertion rates

© 2012 I BM Corp.
•
•
•
•
Usage of Fiber Channel
Usage of Fiber Channel

• Fiber Channel support available as of 7.0 MR1
• What is Fiber Channel?
• Fiber optic network technology for storage networks
• Allows direct connections to SAN, supports transfer rates up to 5.1GB/s

© 2012 I BM Corp.
•
•
•
•
Technical preparations
Technical preparations
• Fiber Channel PCI card must be installed in the QRadar appliance
• Existing appliances can have cards added
• New orders can specify to have Fiber Channel cards pre-installed
• QRadar can be configured to use the Fiber Channel storage in different
ways:
• Store all online event & flow data (not recommended)
• Store only backups
• Fiber Channel is transparent to QRadar
• Drivers and configuration utilities included for convenience
• HA with Fiber Channel is supported if latency time between machines is
less then 10Ms and assuming that the Fibre Channels bandwith is at least
1 Gbps. These two requirements also apply to non Fiber Channel
connections in a HA environment.
© 2012 I BM Corp.
•
•
•
•
Disk performance considerations
Disk performance considerations
• Fiber Channel throughput is extremely fast, but the storage array

on the other end may not be
• QRadar performance will be entirely dependent on the speed of
the SAN
• Same considerations with iSCSI, NFS, etc
• QRadar appliance on-board storage is generally cheaper and
faster than SAN over Fiber Channel
• 3105, 1605, and XX24 appliances have up to 16TB onboard
storage. There shouldn’t be a need for Fiber Channel with these
platforms.

© 2012 I BM Corp.
•
•
•
•
Lesson 6: Distributed deployment across Wide Area Network
Lesson 6: Distributed deployment across

Wide Area Network
Lesson 6: Distributed deployment across Wide Area

Network
• In general it is recommended to restrict a QRadar deployment to
a single Local Area Network.
• In case the QRadar deployment must cover a WAN, use a
compressed ssh tunnel between the appliances. (ssh –C)
• In case the bandwith between the log sources and the event
storage is critical, deploy the 1501 appliance to collect events and
forward on schedule.
• To learn about ports used between QRadar appliances, refer to
appendix A of the student guide.

© 2012 I BM Corp.
Q: why is port 23 (rdate) mentioned when rdate is completely deprecated and ntp (port 123/udp)
should be used instead? Is it really still used?
A: indeed, we do use rdate between qradar components and the console, rather than NTP. NTP can
be configured on each host to reach out to an NTP server, but if used, all systems should point to
the exact same NTP service, regardless of their location. That said, all managed hosts, regardless
of NTP settings, will also reach out to the console every 10 minutes to sync up their time to the
console. RDATE is used over NTP, as a built in service on the console will service those requests
(the time service in xinetd), and saves the (admittedly minimal) overhead of running an ntpd service
on the console.
Q: are the ports related to flows used at all when no flows are analyzed?
A:Assuming we are talking about netflow ports, such as 2055, 9995, etc. While the qflow collectors
are listening for these ports, if nothing is being send to the flow collectors on these ports, then no,
there will not be any traffic on them, we won't generate netflow or any other external flow data. If
we are talking about the ports used between a flow collector and a flow processor (32xxx), then if
there are no flow collectors they will not be used.
•
•
•
•
Q: why is port 80 (http) used and not only 443 (https)?
A: Part of the deployment editor download uses port 80 instead of 443, which had something to do
with java security checks and download jar files via http versus https and being required to verify
the ssl host when it was self signed. For that reason, part of the downloads in the deployment editor
are via port 80.
Q: why does the Used Ports document mention a lot of ports related to nfs (111/tcp&udp, 762/
tcp&udp, 2049/tcp, …) and how are they relevant/necessary for QRadar?
A: these are only important if they are used. IF NFS is not being used, then these ports are not used
by qradar. This is only listed because for customers who use NFS.
Q: why does the used ports document mention snmp ports (161/udp, 199/tcp) and how are they
relevant/necessary for QRadar?
A: we accept inbound data from SNMP devices into our appliances for event sources, so we need
to accept data on that port. The other port, 199, we accept SNMP polls from other devices, so that
they can check on the health status of the qradar appliances.
Q: why is port 6514/tcp used for syslog with TLS instead of the standard port 514/tcp?
A: 6514 for TLS syslog is a design restriction, ECS normally listens on port 514, and syslog-ng is
moved to 1514. The additional port was necessary because ECS does not handle the TLS directly
but an intermediate process does. We change the syslog-ng port. Since ECS is what we want
processing syslog, ECS listens on port 514 and we move syslog-ng to port 1514 for our own internal
logging. We use JSVC as a wrapper in order to easily change process owners and catch the output
of stdout/stderr and send to our local syslog.
Q: what kind of protocol is used on ports 7676/tcp and 7677/tcp?
A:These are messaging ports used by the "imq" service between managed hosts.
Q:what are the ports 7777-8080/tcp and 32004-32011 used for?
A:ports 32004 and above are randomly chosen (in order, starting at 32004) for general
communication purposes between qradar components. qflow to ec, ec to ep, ep to ep on the console,
etc. These ports are used to transit collected data between QRadar components, and may be
assigned as required, as each process on a host will require it's own listen port. Ports 7777-778x are
debug & monitoring ports on the java processes on the managed hosts. These are only used for local
(on the host itself) debugging purposes, and are not available off the machine itself. Ports 8005-
>8080 are only used between the HTTPD & tomcat services on the console itself, and would not be
exposed to any processes outside the console.
Q: what is port 23111/tcp used for?
A: They are used for managed hosts to report event & flow related information back to the console
ecs service.
•
•
•
•
Distributed deployment using a 1501 appliance
Distributed deployment using a 1501 appliance
A ny data collecte d by a 150 1 must be forwarded to a

1 600 seri es applian ce for analysis and storage. Once
e vents are col lected b y a 1501 they are bu ffered locally
u ntil they can b e forwarded to an upstream 1600 series
a ppliance. This forwarding behavior is configurable and
can be specified as a function of time of day.

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
Open your Student Exercises book and perform the exercises for this unit.
•
•
•
•
Summary
Summary
Summary
 Explain the QRadar appliances offering
 Estimate the number of Events per second and Flows per minute for a
QRadar deployment.

© 2012 I BM Corp.
•
•
•
•
Summary
•
•
•
•
Unit 3 QRadar Software Installation
© 20 12 IBM Corp .
•
•
• 3-1
•
•
Introduction
This unit describes he steps to install the Redhat operating system and to install, upgrade and
configure QRadar.
Objectives
Objectives
 Install QRadar V7.1 with 31xx appliance role on Redhat 6.2
 Bring up QRadar appliance hardware
 Perform the basic configurations

© 2012 I BM Corp.
•
•
3-2 • IBM QRadar Administration ©Copyright IBM Corp. 2012
•
•
Lesson 1: Prerequisites
 At least 9 GB of RAM.
 At least 50 GB of storage device space.
 64 bit architecture
 RHEL 6.2
 QRadar V7.1 ISO image.

© 2012 I BM Corp.
•
•
©Copyright IBM Corp. 2012 IBM QRadar Administration • 3-3
•
•
Preparations
Preparations
 Obtain the fixed IP addresses for the QRadar appliance interfaces to be
used.
 Determine which interface should be used for the flow collector.
Preferably the high speed interface.
 Obtain the necessary user IDS and passwords for the QRadar appliance
OS and QRadar admin account.
 Communicate the ports used by QRadar appliance in case the
appliances will be separated by firewalls. See appendix A.
 Download the QRadar V7.1 ISO and burn to DVD if necessary.
 The QRadar host will be hardened during the installation. Only the base
OS will be installed.

© 2012 I BM Corp.
•
•
•
•
Software Installation of QRadar V7.1
Software Installation of QRadar V7.1
Power on the
machine.
You will need to

choose one of the
‘Install’ types.

© 2012 I BM Corp.
•
•
•
•
Check the media
Check the media
You may skip this step using the Tab key and
Enter.
© 2012 I BM Corp.
•
•
•
•
RHEL 6.2 installed
RHEL 6.2 installed

After skipping several additional media tests,
RHEL 6.2 will be installed and when finished
rebooting, present the next screen. It takes
about 20 minutes to install RHEL6.2.
Type SETUP and at localhost login: use the

‘root’ account

© 2012 I BM Corp.
•
•
•
•
License agreement
License agreement
Read the license

agreement carefully.
Scroll down through the
text by pressing the
spacebar. If you accept
the license conditions,
type ‘yes’ at the end of
the license agreement
text.

© 2012 I BM Corp.
•
•
•
•
Enter the activation key
Enter the activation key

Enter the activation
key belonging to
the role this
appliance will have
in the QRadar
deployment.
Shown is an
activation key for
the 31xx appliance
role software install.

© 2012 I BM Corp.
•
•
•
•
Choose the type of setup
Choose the type of setup

Choose ‘Normal’ to
setup the initial
QRadar appliance
deployment.

© 2012 I BM Corp.
•
•
•
•
Apply the Enterprise tuning template
Apply the Enterprise tuning template

Currently we only
have one template.

© 2012 I BM Corp.
•
•
•
•
Setup the time server
Setup the time server

Time server must
be setup because
time
synchronization is
regulatory
compliance
requirement.
But in the class

environment we
setup the time
manually

© 2012 I BM Corp.
•
•
•
•
Time server connectivity
Time server connectivity

Make sure that the
timeserver can be
reached by the
QRadar appliance

© 2012 I BM Corp.
•
•
•
•
Setup the management network interface
Setup the management network interface

Used for the
QRadar web
interface and log
source collection.
W e will use the 2nd

interface for QFlow

© 2012 I BM Corp.
•
•
•
•
Enter the network information of the appliance
Enter the network information of the appliance

All fields are
mandatory except
for the Public IP.
Do not use
common QRadar
terms in the
Hostname, like:
QRadar, EP, EC,
FC, FP, Console,
etc.

© 2012 I BM Corp.
•
•
•
•
Installation completed
Installation completed
It takes about
another 30 minutes
for QRadar
appliance to install
and start.

© 2012 I BM Corp.
•
•
•
•
Configure the network interface for QFlow

(optional)
Configure the network interface for QFlow (optional)

Logon as root and
start ‘setup’.
Choose the Network configuration tool

© 2012 I BM Corp.
•
•
•
•
Configure the QFlow Network Device
Configure the QFlow Network Device

In this example we
configure the extra
network interface
for the QFlow
collector on the
31xxx ‘All in one’
appliance.

© 2012 I BM Corp.
•
•
•
•
Specify the IP addresses to use
Specify the IP addresses to use
Use the IP address provided to you for the flow collector. After
exiting the tool, reboot the system.

© 2012 I BM Corp.
•
•
•
•
Lesson 2: Basic configuration of the All in One 31xx appliance
Lesson 2: Basic configuration of the All in

One 31xx appliance
Lesson 2: Basic configuration of the All in One 31xx

appliance
Open a webbrowser and connect to https://192.168.177.143

Logon with Username admin and Password object00

© 2012 I BM Corp.
•
•
•
•
Update the license
Update the license

Go to the admin tab and
select System and License
management.
Browse to the license key

file and save it
Don’t forget to deploy the

changes IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Configure the Flow Collector (optional)
Configure the Flow Collector (optional)

Go to the admin tab and
select Flow sources
Make sure that you assign

the right interface to the
right flow collector
Don’t forget to
deploy the changes

© 2012 I BM Corp.
•
•
•
•
Add extra interfaces for the Event Collector

(optional)
Add extra interfaces for the Event Collector (optional)

 The management interface is mainly used for the user interface.
 The Event Collector will use any interface in the same subnet as the
management interface.
 You can use additional interfaces for log record collection.
 Make sure that you communicate the Event Collector interfaces IP
addresses to the log source administrators who want to forward the log
rec ords to the Event Collector(s).

© 2012 I BM Corp.
•
•
•
•
Lesson 3: Physical Setup
Lesson 3: Physical Setup X3550M3

 The IBM X3550M3 includes four network interfaces.
The IMM provides the following functions:

•Around-the-clock remote access and management of your server
•Remote management independent of the status of the managed
server
•Remote control of hardware and operating systems
•Web-based management with standard web browsers
© 2012 I BM Corp.
•
•
•
•
Physical setup X3630M3
Physical Setup X3630M3

 The IBM X3630M3 includes two network interfaces.
Check the 00d2490.pdf chapters 2 and 3 to learn more on how

to use the Integrated Management Module

© 2012 I BM Corp.
•
•
•
•
Lesson 4: Post Installation actions
Lesson 4: Post installation actions

• Download the latest updates.
 Patches and major releases (MR)
 DSM and protocols
DSM Updates
Pro tocol Upd ates
Scan ner Updates
Note: QRa da r Auto Upda tes prov ide update d conf iguration information for QRadar
deployme nts. Informa tion update d include De vice Eve nt ma pping s ( for
DS Ms), Geogra phic da ta (for the G eographicVie w), and Re mote N etwork
update s(for bogon lists). Deploy ments tha t do not hav e direct interne t
a cce ss (https conne ction from your QRa da r console to qmmunity. q1labs.com)
will require tha t you se tup a n inte rnal upda te se rv er for your cons ole to
download the fil es from.
Follow the instructions in the readme text to install the packages.

© 2012 I BM Corp.
•
•
•
•
Update all DSM and Protocols after patching
Update all DSM and Protocols after patching

 Before starting to receive log data and flows, you should update all DSM
and protocols.
 Download the latest archive with DSMs or Protocols available.
 Use WinScp or a similar tool to copy the file to the QRadar Console
appliance.
Tar zxvf the file and run the following command:

for FILE in *Common* .rpm DSM-*.rpm PROTOCOL-*.r pm VIS-*.rpm; do
rpm -Uvh "$FILE";
done
The command wil update DSM, PROTOCOL and VIS if all are uploaded and prepared.
The upgrade will take long to execute. IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Configure auto update
Configure auto update

 System includes a dataset of predefined traffic types,
vulnerabilities/exploits, and event types
 This dataset is continuously updated as new information becomes
available
 The updates, located on the Qmmunity web site, are sourced from
various network security organizations
• https://qmmunity.q1labs.com
 To access the dataset updates, the Console requires Internet access
 Your system can either replace the existing data set completely, or
integrate updates to help prevent unwanted changes
• For example, a custom application port definition will not be overwritten
if a new application is released that uses the same port

© 2012 I BM Corp.
•
•
•
•
Choose the update policy
Choose the update policy

The schedule lets
you choose when
and how the
update will be
applied.
Auto integrate
means that
customized
configurations will
be maintained.

© 2012 I BM Corp.
•
•
•
•
Lesson 5: Adding non-Console appliances
 Install the QRadar appliance as shown before

 Use deployment editor to add the appliance as a Managed host.
• Make sure you have jre-6u29-windows-i 586-s and/or jre-6u29-windows-x64
i nstalled on the browser machine before you try to use the deployment editor.
 Assign and configure EP,FP or QFlow role to Managed Host.

© 2012 I BM Corp.
•
•
•
•
Adding the managed host
Adding the managed host
In case of problems, try to ssh to from the console to the managed host first and scp a file from
the managed host to the console. This might help to synchronise the passphrases.
© 2012 I BM Corp.
•
•
•
•
Lesson 6: High Availability Overview

 The QRadar High Availability (HA) system allows appliances to have a backup
appliance take over their duty when in trouble
 The system was designed to have one shared virtual IP.
 The rest of the deployment sees no difference between an HA cluster and a

single managed host. The only exception is the console for configuration and
User Interface for Management.
 There are two types of setup for data synchronization:
 Shared Storage
 Disk Replication
 QRadar appliances by default
 Hardware RAID 10 provides redundancy across drives
 Dual Redundant power

© 2012 I BM Corp.
•
•
•
•
Install a HA appliance with the same role
Install a HA appliance with the same role

You will need a
seperate activation
key for HA
configuration
The Master
appliance’s IP
address will become
the virtual IP later
on.

© 2012 I BM Corp.
•
•
•
•
Configure the network
Configure the network
It will take about 45

minutes to install

© 2012 I BM Corp.
•
•
•
•
High Availabilty - Management
High Availability - Management
Start the HA wizard and

assign the new IP address to
the Primary
After a while you will see the

secondary machines appear
in the System and License
Management page

© 2012 I BM Corp.
•
•
•
•
High Availabilty - Details
High Availability - Details

 Heartbeat between primary and secondary makes sure that the
secondary will take over duties in timely fashion.
 The secondary is installed as a HA appliance, but once coupled with a
primary, it will look exactly like the primary once it is responsible for the
duties of the cluster.
 When patching a primary, the secondary will automatically get the same
patches, even if down for long periods of time.
 When upgrading, the primary needs to be active. Once the primary is
upgraded, the secondary will not be accessible until it is also upgraded.
 QFlow collector needs hardwiring and therefore HA configuration should
be applied with reservations

© 2012 I BM Corp.
•
•
•
•
High Availability - Deployment Options
High Availability – Deployment Options

Disk Synchronization Shared Storage

© 2012 I BM Corp.
•
•
•
•
High Availability - Disk Synchronization
High Availability – Disk Synchronization

 Uses system resources, appliances can scale to their full potential
• A linux kernel module (DRBD) is used to make sure that all data is
synchronized and correct.
 Latency time between master and slave appliance must be less then 10
ms.
 Network connection between master and slave appliance must be at
least 1 Gbps with latency time less then 10 ms.
 Very easy to deploy
 Low Cost

© 2012 I BM Corp.
•
•
•
•
High Availability - Shared Storage
Lesson 6: High Availability – Shared Storage

 Doesn’t use system resources, appliances can scale to their full
potential
 HA system will automatically leverage Shared Storage
 Not as simple to deploy
 IP SANs have many built in HA functions such as network RAID.
 Higher Cost
 Scalable Storage (which may be required anyway)
 3rd Party
 Minimum I/O requirements per appliance connecting to the SAN.
• Reads: 4000 IOPS, sustaining 900 MB/sec.
• Writes: 600 or more IOPS writes, sustaining 100 MB/sec .

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise
IB M Sof tware Group | Securit y Division

3-46 © 2012 IB M Corp.
•
•
•
•
Summary
Summary
Summary
 Install QRadar V7.1 with 31xx appliance role on Redhat 6.2
 Bring up QRadar appliance hardware
 Perform the basic configurations

© 2012 I BM Corp.
•
•
•
•
Summary
•
•
•
•
Unit 4 Architecture V7.x

IBM confidential
© 20 12 IBM C orp.
•
•
• 4-1
•
•
Introduction
This unit explains the architecture of QRadar and each component.
Objectives
Objectives
 Explain the Qradar architecture

© 2012 I BM Corp.
•
•
4-2 • IBM QRadar Administration IBM CONFIDENTIAL ©Copyright IBM Corp. 2012
•
•
Lesson 1: High level Architecture
Lesson 1: High level Architecture
Lesson 1: High Level architecture
Ident ity Asset Off ense

• Flow and event data is stored
on the Ariel datastorage on the
processors. If accumulation is
Console Services
required, then accumulated
UI
Magistrate
data will be stored in the
Reporting accumulation database
Flo w
• Offenses, asset, and identity
Assets I dentity
information is stored in the
EventProcessor
master PostgreSQL database
Event
on the console.
Accumulations • SSH between appliances in a

distributed environment is
Flow Collector Event Colle ctor supported.
Packet Events from

Int erface, Log Sources
SFLOW and 3 rd
party IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
IBM CONFIDENTIAL ©Copyright IBM Corp. 2012 IBM QRadar Administration • 4-3
•
•
Lesson 2: Flow collector architecture

To flo w c oll ector e very 60 sec onds
C o l e cto r
Flo w Re p or tin g a nd R o uti n g
- Cr ea te Su p e rFl ow s
•Superflows are created
Ap pl i ca ti on D ete c tio n M od u l e
a p p Id = e v e ntId
•(Custom) applications are

detected
Flo w A gg re g a to r P ac k et Ag g re g a tor
-En fo rc e Li c en s e Li m ti - En for ce L ci e n se L m
i it
V5 F o
l w V5 Flo w
Ca c he Ca c h e
•If the flow license limit is
exceeded, an overflow
N e tFlo w
V 1,5 ,7 ,9
Pa ck e t e e r NIC D AG Na p a te c h SFL O W record is created with src/dst
Q FL OW
address 127.0.0.4/5.
Fl o w d a ta SFL O W
FD R
fro m d ata
Re co rd s
J un ip e r fro m
fro m
C si co Fo u n dr y
Pa c ke te e r
De v ci e s

© 2012 I BM Corp.
•
•
•
•
Application detection
Application detection
Four methods of determining the application of the flow. In order of priority:
1. User defined. This method is mainly used when a user has a proprietary
application running on their network. For example: All traffic going to host
10.100.100.42 on port 443 is recognized to be MySpecialApplication. Uses
user_application_mapping.xml
2. State based decoders. This method is implemented in the source code and
determines the application by analyzing the payload for multiple markers.
For example. If we see A followed by B then application = X, If we see A
followed by C, then application = Y.
3. Signature matching. Uses /opt/qradar/conf/signatures.xml. Basic string

matching in the payload. Custom signatures are allowed. (See Application
Configuration Guide for signature customization)
4. Port based matching. Port 80 = http...etc

© 2012 I BM Corp.
•
•
•
•
Superflows
Superflows
• Types of superflows
• Type A: Single SRC, Multiple DST – Same DST Port (tcp/udp),
byte count, SRC flags/ICMP Codes. Network Sw eeps
• Type B: Multiple SRC, Single DST – Same DST Port (tcp/udp),
byte count, SRC flags/ICMP Codes. DDos attacks
• Type C: Single SRC and DST, TCP/UDP Only, Changing
SRC/DST ports. Portscans
• Only store the single flow with the collection of IPs
• Specific rule tests can leverage the flow type to determine if an
offense needs to be created
• Creation of Superflows can be disabled.

© 2012 I BM Corp.
•
•
•
•
Lesson 3: Event Collector architecture

Colle ctor( s)
(OffSit e Tar ge st
fo r FL OWS)
Eve nt Proce ss or rec ei ves th e Ne two rk Eve nts
Fu l
Fo rward ed
Q ue ue
Store
Onl y
Postg res i s up da ted wi th
disco ve red Devi ce s
C ol e
l ctor(s)
(Off Site Ta rge ts fo r
• Log sources are
Fl ows EVENTS)
automatically detected
Fu l
Que ue
Store
RPC
Sta t Fi tl er
after record analysis
Onl y
Forwa rda bl e
Eve nts
Flo w Fo rwa rdi ng
Fi tl er Traffic An al ysi s Fil te r
Flo wSu pp ort Fil te r

Pay o
l ad In
SEC Ev en t Norm al iz ed Eve nt
Eve nt Forwa rding Filt er
• Similar eventsare
Su pp o rt Qu eu e
Fo rward ed Ev en t
(Fo rward ed)
Co alesc n
i g Fil ter
Ful l Qu eu e
Store Onl y coalesced
Fil te r
Qu eu e
• Custom properties are
Fl ow Go vern er
(L ci en si ng )
extracted from the events
Parse r
Fl ow Statis ti cs
DSM
Pa rser Th rea ds
Que ue
and flows
Fi tl er
DSM Norm aliz e Fil te r
No n-Forwa rd ed Eve nts

• Eventsare parsed by Log
Flo wDe -Dup Fi tl e r
Lice nc e
Forwa rde d Eve nts
EC Entry Ro utin g Fil ter
Source parser
Asymm etri c Flow
Re com bi na tio n
Fi tl er
Mon ti o rs Li ce nc e Ov erflo w Fi tl e r (Li cen si ng ) • EPS license is checked
Sou rce Mo ni tor
• Flows are subject to
Flo w Sou rces Mo ni tors (Wire ) M on ti o rs Eve nt Sou rce s
recombination and
COL LECT OR deduplication.
Flo w Sou rces Lo g So urce s
© 2012 I BM Corp.
•
•
•
•
FPM and EPS burst handling
FPM and EPS burst handling

•Flows or Events are temporarly stored in an overflow buffer if the FPM or
EPS license is exceed.
•Every log source protocol has an overflow buffer of about 100000 events.
•If the overflow buffer fills up, the additional flows and events are dropped.
•In general a FC or EC may be able to handle an event burst for up to 15

seconds. (depending on the size of EPS or FPM burst)
•There are also event and flow queues for the normalization and custom
property extraction process. If any of these queues fill up because of slow
parsing, the raw flows and events will be written to the Ariel datastorage.
(HLC,LLC) eq (Unknow, Stored)

© 2012 I BM Corp.
•
•
•
•
Log Source parsing uses QID mapping
Log Source parsing uses QID mapping

• The Log Source p arser extracts the Lo g Source Event ID from the log record
• QIDMap is a un ique id that links the extracte d Log Sou rce Event ID to a QID
• A QID is an a lmost arbitrary integral number
• Each QID number relates to an (custom) Event Name and descripti on, as well as
severity and event category information
• QRadar ‘s even t category information i s structured in several High Le vel
Category (HLC) and Low Level Categories (LLC). Every QID is li nked to one of
these l ow level categories. For examp le “Authen ticati on (HLC) – Admin Logi n
Successful (LLC)” is a categ ory combination
• Log source records that are not assigned to a Log Source parser, are mapped to HLC=
Unknown, LLC=Unknown log event
• Log source records that are manually assigned to a Log Source parser but not
recognized, are categorized as HLC= Unknown, LLC=Unknown
• Log source records that are automatically assigned to a Log Source parser but not
recognized, are categorized as HLC= Unknown, LLC=Stored This may imply that a
parser needs to be updated
• Check ´Writing a Log Source Extension.pdf´ to learn more about QID

mapping IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Mapping definitions are stored in PostgreSQL
Mapping definitions are stored in PostgreSQL

 Accessing the postgreSQL database
• Open an ssh conne ction to qradar:eth0 (e.g.using Putty)
• User role in postgres is qradar
• L ist all databases oid2name –U qradar
• Connect to qradar config database psql –U q radar
• L ist all tables psql select \d
• L ist table schema psql select \d+ tabl ename
• L ist table colu mns only psql sel ect \d tablena me
• E xit psql \q
 Interesting tables: dsmevent, qidmap, and category_type

 Caution: Do not modify tables using psql

© 2012 I BM Corp.
•
•
•
•
Log Source Event ID to QIDmap in DSMevent table
Log Source Event ID to QIDmap in DSMevent table

Refers to an entry in
QIDmap table
Log Source Event ID

extracted by the parser
© 2012 I BM Corp.
•
•
•
•
QIDmap to QID mapping in QIDmap table
QIDmap to QID mapping in QIDmap table

This is the actual QID assigned
The Event Name assigned to the Log Record
Links to LLC entry in category_type table

This is the LLC
Refers to the HLC

record
This is the HLC

© 2012 I BM Corp.
•
•
•
•
Custom property extraction
Custom property extraction

• Any data element can be parsed from event or flow and accessed
from the pipeline
• May be Optimized for rules, reports and searching
• Custom property created today also apply to data stored in the past.
• Simple GUI editor for adding and testing custom properties for events
and flows using Java regex.
• Custom property extraction can be done at event collection or after
event storage.
• Optimized properties may be indexed for better property search
performance. But this may have impact on overall performance.

© 2012 I BM Corp.
•
•
•
•
Optional Event Coalescing
Optional Event Coalescing

• Events with similar properties and same values for these properties,
are coalesced.
• Properties tested are:
• Src/dst port
• Src/dst IP
• Username
• First 3 similar log events are stored individually, following similar
events are coalesced.

© 2012 I BM Corp.
•
•
•
•
Auto discovery of Log Sources
Auto discovery of Log Sources

• Essential module for Automating a successful evaluation or depl oyment
• Categorizes traffic from devices that are unknown to the system
• If de tection is successful on an IP a ddress a new log source created
• Lock-in Thre sholds are con figure on a per DSM basis. See
/opt/qradar/conf/TrafficAnalysisConfig.xml
<Threshold name="MinNumEvents" value="25"/>
<Threshold name="MinSuccessRate" value="35"/>
<Threshold name="MaxEventsBeforeFail" value="1000"/>
<Threshold name="Aban donAfterSuccessiveFailures" value="50"/>
• Detection can only be carri ed out on event protocols which are “pushed” to
the Event Collector. (ie Syslog)
• Discover multiple devi ces on the same IP add ress as long as there are
unknowns events from that IP address and auto detecti on has not been
abandone d for the IP address
© 2012 I BM Corp.
•
•
•
•
Lesson 4: Event Processor architecture

Co nso e
l Ari el Proxy
TCP TO CENTRAL An oma yl Detec ti on Que rie sa nd
MAGISTRATE EP Eng ni e Re ceives
Acc umu al te d Data
VIS (Pa ssiv e)
New Hos t/ Port Eve nts
Offe nse Eve nts
• New offenses are created by the
magistrate (see console)
P3 : EPExit Fil e
tr
• If a new port or host is detected, an
RPC
P3: Even t Stre ami ng Fi tl er Accu mul ator Ac cumu a
l te d
asset profile is updated or created in
Eve nts
Da ta
P ostgreSQL. (see console)
From PS: Ho st Profil in g Fil te r
Distribute d
EP
Arie l
Rou ted
Strai gh t
To
Norm ail ze d Ev ents Que ry Se rver
• Events are accu mulated every
MPC
P3: Eve nt Stora ge Fil ter
Even ts
(Arie l )
Fl ows
(Arie)l
Flow Eve nts minute and stored i n the accumulator
Ful l
Qu eue
Sto re
A ri el datastorage
On yl
P3: Off en se Type An alyzer • Events and flows are stored in the
STORAGE
ONL Y events or flows Ariel datastorage
(Ful l Que ue DROP
CRE Res pon se Ru e
l Testi ng n
i EC) * No Match o n
Que ue Eve nts fro m
Offb oa rd EP*
P2: Cu stom Rul e Eng n
i e (CRE) • Events and flows are queued for
rule matchi ng and tagged if they
ERS: Entry Routi ng Fil te r
match a rul e. A rule that is matched
P1 : Overflo wFilt er (Lic ens n
i g) L ci e nse
may result in a respon se
EPMo nito r
TCPFrom EP Native fro m CRE Nativ e Fro m EC TCPFrom EC

CRE
Ev ent Mo nti o rsIn comi ng
Network Even t Sou rces Even t Ra te
EVENT PROCESSOR
Offb oa rd
L oca l EC Offbo ard EC
EP
© 2012 I BM Corp.
•
•
•
•
Correlation Rules Engine
Correlation Rules Engine (CRE)

• A single event or flow is tested against all enabled rules
• Rules matched may have an response or result
• Rules matched may trigger the creation of an offense or create a CRE

event that triggers the creation of an offense
• Multiple events, flows and rules matched may correlate into a single
offense
• A single event or flow can be correlated into multiple offenses
• By default Rules are tested against events or flows received by a

single EP or FP. (Local Rules)
• Global Cross Correlation (GCC) allows Rules testing across multiple

EPs and FPs in the QRadar deployment.
© 2012 I BM Corp.
•
•
•
•
Accumulator
Accumulator
• Accumulations defined by ´grouped by´ searches
• Creates time series statistical meta data (counts) that are used in
• Dashboards
• Event/Flow forensics and searching
• Reporting
• Anomaly and behavior Alerts
• Accumulated intervals are 1 minute, 1 hour and 1 day
• Distributed component that operates on each Event Processor
• The Ariel Query Server gathers the distributed data for QRadar to use.

© 2012 I BM Corp.
•
•
•
•
Accumulator structure
Accumulator structure
I nterval Queue Timer
Tim er Event
Inte rval T ask
Main Processor
Ariel Reader
A ri el
Preprocessor
Acce ssors
Recor d Pack
Record P reprocessor
Config table contains the search
Conf ig
T hread P ool
Preprocessor
definitions for the accumulator (the
O bject[ ] pack Preprocessor
“results grouped by” searches)
View Co nfigurat o
i n
A ggregation Aggrega tion T ask

A ggregation
T hread P ool
Aggregat ion
Aggregat ion
Writer The Ariel datastorage contains both

Writer
T hread P ool
events, flows and the accumulated
Writer data as the result of the configured
Writer aggregation
Ari el © 2012 I BM Corp.
•
•
•
•
Lesson 5: Console architecture

CO L L ECTO R
(Sy sl o g TCP )
•Offen ses are stored in the

Po stgreSQL database by
O FFEN SE
CR E OFFEN SE
the Magistrate
DA TA
•The Magistrate ca n
Ev e nts
Th a t e
l ad
U p to Offe n s e M AG ISTR ATE
MP C Ev e nt
Qu e ry Pro ce s so r i nstruct the Ariel proxy to
gather all events, that
triggered the creation of an
O ve rfl o w Fi l te r (L i ce n s n
i g)
Ev e nts Th a t Le a d offense, on search.
To Offe n s e
•The Anomaly Detection

En gine (A DE) searches the
M PC
Eve n t
Qu e ry S o ur ce
NATIVE Fro m E P TC P Fro m EP AR IEL
PR OXY
V IS (Pa ss i ve )
An om a l y
De te c tio n
En g ni e
Accumul ator database for
N e two rk Eve n t So u rc es
SE RVER
Ne w Ho st/Po rt
E ve n ts
( ADE) anomalies
CONS OL E - MPC
•The Vulnerabili ty
RP C Informati on S erver (V IS),
create s new assets o r adds
open ports to existing
L oc a l E P O ffb o a rd EP
AR IEL
QU ERY
SER VER( S)
HO ST
PR OF L
I ER Acc u m ula to r(s ) assets.
© 2012 I BM Corp.
•
•
•
•
Offense management by the Magistrate
Offense management by the Magistrate
• Rules can correlate both events and flows into a single offense
• A single event or flow can belong to multiple offenses
• While rules are tested, they may lead to the creation of an offense.
• These pending offenses tag the events or flows as long as the rule
that may trigger the creation of the offense, remains partially matched.
• Offenses that have been created after rules completely matched, are
back filled with the tags from all events or flows leading up to the
offense creation.
• A maximum of 100000 offenses can be stored

© 2012 I BM Corp.
•
•
•
•
Events or flows tagging by matching rules
Events or flows tagging by matching rules
CRE Rule X Rule triggers the Magistrate

creation of an
offense
Partial matches
tag the flows or
events
Query for all tags of the events
or flows on Offense Creation.
Ariel
Offense is created with all tags to
events or flows that lead up to the
offense.
© 2012 I BM Corp.
•
•
•
•
Offense types
Offense types
• An Open Offense that is created, remains an Active Offense as long as the rules that
triggered the offense creation are matched by events or flows within 30 minutes after
the last match has been found. Tags of events or flows are added to the Active Offense.
• If an Open Offense did not find additional matches for more than 30 minutes, it
becomes a Dormant Offense
• Dormant Offense can become active again when additional matches are found within 5
days after the offense became dormant. Thus becoming a Recalled Offense. Tags of
events or flows are added to the Recalled Offense.
• Once a Dormant Offense did not receive any matches within 5 days after it became
dormant, it will turn to an Inactive Offense
• Open Offenses can manually be turned to Closed Offenses
• If events or flows are matched for an Inactive Offense or Closed Offense, a new Open
Offense will be created.
• A maximum of 2500 Active Offense are allowed and a maximum of 500 Recalled
Offenses are allowed
• Closed and Inactive Offenses are subject to retention management

© 2012 I BM Corp.
•
•
•
•
ADE (Anomaly detection Engine)
ADE (Anomaly Detection Engine)
 Monitors Accumulated Data

created by EP
 ADE implements Behavior
and Anomaly rules
 Threshold rules are
implemented directly in CRE
A ccumu lator generates accumulated data for

reports and views based on time series saved
searches or reports. IBM Sof tware Group |©Securit y Division
2012 I BM Corp.
•
•
•
•
Anomaly Detection Engine Rule Types
Anomaly Detection Engine Rule Types

Three categorie s to choose from
• Threshold: g reat than, less than and range
• Band width of an applicati on
• Failed service
• Number of users connected to a V PN
• Large o utbound transfer
• Anoma ly: Change in short term when compari ng against a longer time
frame.
• New service activi ty
• Change in the bandwidth vol ume on a link
• Behavioral: Ch ange from the same ti me yesterday or last week
• IPS Changes in “Alert only alarms”
• Mail traffi c - Incre ase on external SMTP server (could be a relay)
• Backup monitoring (backup failed)
• Just about anything with a repetitive pattern.
© 2012 I BM Corp.
•
•
•
•
New asset and service detection by VIS
New asset and service detection by VIS

 As new hosts, services and vulnerabilities are discovered events are
generated
 Uses flow information to detect new or modified assets and automatically
checks the asset information against uploaded vulnerability information
Building
blocks are
used to
categorize the
assets based
on identified
ports by IP
address

© 2012 I BM Corp.
•
•
•
•
Lesson 6: Datastorage Technology

• Ariel: Purpose build database
• A s soon as data stored it can no t be changed
• File locks and transactions are not used
• S tored data can be filtered and placed i nside differen t files based on gi ven
filters results (retention bucketing)
• Data indices are written in parallel and asynchronously
• PostgreSQL
• QRadar SIEM: Configuration, Assets, Offenses
• Scalability and Performance are managed through bulk insert/update
transactions and populating memory caches to avoid numerous round
trips to the database
• One master database with copies on each processor for backup and
automatic restore

© 2012 I BM Corp.
•
•
•
•
Ariel datastorage in /store/ariel on processors
Ariel datastorage in /store/ariel on processors

Files are sto red in date/time structured directories
Access to Ariel through Ariel Query Server (AQS) using Ariel Query
Language (AQL)
AQS runs in the separate process and can be accessed by clients

through TCP/IP based proprietary protocol in a distributed environment
Query results are send back or stored and cached by the AQS inside
C ursors

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

4-42 © 2012 IB M Corp.
•
•
•
•
Summary
Summary
Summary
 Explain the QRadar architecture

4-43 © 2012 IB M Corp.
•
•
•
•
Unit 5:Solution implementation
Unit 5 Solution implementation
© 20 12 IBM Corp .
•
•
• 5-1
•
•
Introduction
This unit decribes the tasks to implement a QRadar solution.
Objectives
Objectives
 Scope a QRadar solution
 Explain the QRadar basic deployment steps
 Setup a QRadar Network Hierarchy
 Use Asset Profiles
 Integrate Vulnerability information
 Use QRadar windows agents to collect Windows event logs
 Integrate QRadar authentication with Windows AD
 Perform basic QRadar troubleshooting

© 2012 I BM Corp.
•
•
•
•
Lesson 1: QRadar solution scope

 Determine the QRadar use case.
• IT Log Management. Collect and securily archive log
records for forensic analysis.
• IT Regulatory Compliance. Collect and securily archive log
records a for audit and compliancy. Generate reports
required by internal or external regulations to succesfully
pass compliancy audits.
• IT Internal monitoring. Frequently collect, correlate and
analyze data to alert on security policy violations
• Security breach detection. Analyze data to detect and alert
on IT security risk management related issues.

© 2012 I BM Corp.
•
•
•
•
Forensic analysis use case requirements
Forensic analysis use case requirements

 Data archived is used to analyze incidents and gather
evidence
 Data must be collected and stored reliably in original format
 Data must be archived for several years
 Data must be searchable

© 2012 I BM Corp.
•
•
•
•
Compliancy audit use case requirements
Compliancy audit use case requirements

 Data archived is used to proof that relevant audit information
has been collected and securily stored
 Data must be used to create reports required by the
regulation
 Regulatory compliance reports must be stored for a period of
time.
 Reports may be required to support the execution of IT
security controls defined by the regulation
 Regulations may require data masking in reports

© 2012 I BM Corp.
•
•
•
•
Security policy violations use case requirements
Security policy violations use case requirements

 Requires an IT Security Policy or IT Code of Conduct that
define appropriate use of the IT environment
 High risk offenses to the policy must be identified and
reported upon
 Offenses must be managed
 IT usage not in compliance with the policy must be reported
upon

© 2012 I BM Corp.
•
•
•
•
IT security risk management use case

requirements
IT security risk management use case requirements

 The requirements for Security policy violations apply
 The requirements for forensic analysis apply
 Knowledge of the IT environment and the (network) threats
to which it is exposed
 Understanding of data patterns captured to perform anomaly
detection

© 2012 I BM Corp.
•
•
•
•
Scope of Log Sources and Network Segments
Scope of Log Sources and Network Segments

 Forensic analysis: Determine the number and types of Log Sources that
need to be integrated with QRadar
 Compliancy audit: Determine the number and types of Log Sources that
need to be integrated with QRadar and have the minimal audit
configuration determined for these log sources to produce the data
needed to generate the reports. (See appendix B)
 Security policy violation: Determine the number and types of Log Sources
that need to be integrated with QRadar and have the minimal audit
configuration determined for these log sources to produce the data
needed to check for policy violations. In addition determine if network
segments may be used to check network data for policy violations
 IT security risk management: All of the above and include the network
segments to identify possible IT security threats

© 2012 I BM Corp.
•
•
•
•
QRadar licensing sizing parameters
QRadar licensing sizing parameters

 The number and types of Log Sources that must be supported by
QRadar, relate to the Events Per Second QRadar must handle in
general.
 The audit configuration of the Log Sources relate to the Events Per
Second QRadar must handle in general.
 The type of assets in the network segments integrated, relate to the
Flows Per Minute QRadar must be able to handle.
• E very server generates b etween 200 an d 300 FPM,
• E very workstation generates between 15 and 60 FPM
• E very switch or router generates around 12 FP M
 The bandwidth of the network is an indication of the maximum Flows Per
Minute.
• E very 1 Mbps bandwid th all ows ˜ 170 000 NetFlows per minute
• E very 1 Mbps bandwith all ows ˜ 500 Flows per minute

© 2012 I BM Corp.
•
•
•
•
QRadar disk storage sizing parameters
QRadar disk storage sizing parameters

 The time log data must be stored online (retention time), determines the
minimum size of the diskspace required on the QRadar appliances
 QRadar log data compression ˜ 85%
 Assume that the average log message is around 600 bytes
 Log Source record parsing adds ˜ 50% to the original log record size
 Assume that the average flow record size is 15 KB per minute.
 Disk space required is affected by the amount of bytes stored from the
QFlow records. Default is to store 256 bytes from the payload
 Flow parsing adds ˜ 50 % to the original flow record size.

© 2012 I BM Corp.
•
•
•
•
Design QRadar appliance topology
Design QRadar appliance topology

 Determine how many QRadar Consoles (aka deployments) are required.
(Driven by overall network topology in scope) Then repeat for every
deployment:
 Group the data sources in scope by security profiles
• Place all applications and machines with similar reporting and alerting
requirements in a single logsource group
• Put network segments with similar similar network monitoring
requirements together in a single logical network hierarchy group
 Apply the licensing and disk sizing parameters to each of these groups
 Determine the minimal EP, FP or FC appliance type for each group
 Make sure that you are aware of possible bandwidth issues between log
sources and EPs. In case of slow lines, a 1501 EC appliance might be
the solution
• In case of 1501 EC usage, make sure that no real time analyses is
required for the log sources connected to the 1501
© 2012 I BM Corp.
•
•
•
•
Data source groups examples
Data source groups examples
Log source groups
Network hierarchy groups IBM Sof tware Group | Securit y Division

© 2012 I BM Corp.
•
•
•
•
Lesson 2: Suggested default Log Activity reports
Lesson 2: Suggested default Log Activity

reports
Lesson 2: Suggested default log activity reports

 If no specific reports are required, one can suggest to use the following
types of reports:
• Privileged User Monitoring and Audit reports
o Usage of Privi leged Users
o P asswo rd resets
o A ccount management
o S ecurity Facili ty and Audi t subsystem modificatio ns
 See “Compliance Audit Settings.xls” for examples of audit

configuration

© 2012 I BM Corp.
•
•
•
•
Suggested network activity reports
Suggested network activity reports

 Common Clear-Text Applications
• Application is any of DataTransfer.FTP or Mail.IMAP or Mail.POP or
RemoteAccess.Telnet
• Source Payload Matches Regular Expression equals .+
 Local P2P Activity by Source IP
• Application equals P2P
• Flow Direction equals L2R
• Group by Source IP
 Local P2P Activity by Application
• Application equals P2P Flow
• Direction equals L2R
• Group by Application
 Top Internet Usage by Source IP
• Group by Source IP
 Top Internet Usage by Application
• Group by Application © 2012 I BM Corp.
•
•
•
•
Suggested threat analysis network activity reports
Suggested threat analysis network activity reports

 Check for smtp outbound traffic to port 25 from non email servers.
(Malware)
 Check for outbound IRC communication to ports 6660-6669. (Almost
retired Botnet technique. HTTP botnet requires more advanced
signature detection)
 DNS activity from non DNS servers. (Trojans)
 Use regular expressions to check the source payload for possible SQL
injections
• /(\%27)|(\')|(\-\-)|(\%23)|(#)/ix
• /((\%3D)|(=))[^\n]*((\%27)|(\')|(\-\-)|(\%3B)|(;))/i </TD< tr>
• /\w*((\%27)|(\'))((\%6F)|o|(\%4F))((\%72)|r|(\%52))/ix </TD< tr>
• /((\%27)|(\'))union/ix
• /exec(\s|\+)+(s|x)p\w+/ix </TD< tr>
 See “siem-based-intrusion-detection-q1labs-qradar_33278.pdf” for more
details IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Lesson 3: Deployment steps

1. Create a QRadar network hierarchy
2. Populate the QRadar Asset Profile Database
3. Adjust the QRadar Asset Profile Database or Asset Profile Building
Blocks
4. Import vulnerability information
5. Disable the responses of rules that generate too many Offenses and
create a report to capture these offenses instead. Finetune them later
6. Create Universal DSM when required
7. Create custom properties, optimize and index if recommended
8. Create the required rules to capture the offenses to watch for
9. Create the necessary reports.
10. Configure backup and retention schedules
© 2012 I BM Corp.
•
•
•
•
Lesson 4: Create a QRadar network hierarchy
Lesson 4: Create a QRadar network

hierarchy

 QRadar network hierarchy defines which IP addresses are internal and
therefore all others will be considered external.
 Create an object for any routable address ranges. It can include entire
/16 and /8 networks if necessary. This can be made more granular later
on.
 Create a detailed network hierarchy for the internal network locations that
are in scope of the QRadar solution.
 Group the remaining internal network locations to the highest possible
hierarchy level.

© 2012 I BM Corp.
•
•
•
•
Setup Network Hierarchy
Setup Network Hierarchy

 When you develop your network hierarchy, you should consider the most
effective method for viewing network activity.
 Additional Notes:
• The network you confi gure i n QRa dar does not have to resemble the physical
d eployment of your network.
• QRadar supports any network hierarchy that can be d efined b y a range o f IP
a ddre sses.
• Y ou can create your network based on many different variables, including
g eographica l or busine ss units.

© 2012 I BM Corp.
•
•
•
•
Manage Network Hierarchy distribution
Manage Network Hierarchy distribution

 To move the Network Hierarchy from one QRadar deployment to another
follow these steps:
• Copy /opt/qradar/conf/net.conf to a temp directory on your new QRadar
console, then run in the same directory /opt/qradar/bin/convertNet.pl
which will produce the following three files:
 Net.conf.new
 Netid.conf.new
 DOTn etid.new
• Perform a chown nobody:nobody *.new
• Perform a chmod 664 *.new
• Copy those files to the following location:
/ store/configservices/staging/globalconfi g/
• Rename DOTnetid to .n etid
• Remove the .new postfix from the file names.
• Do a deploy from the QRadar user interface.
© 2012 I BM Corp.
•
•
•
•
Example
Example
Create security
relevant network
groups

© 2012 I BM Corp.
•
•
•
•
Accessing the Network Hierarchy in the User

Interface
Accessing the Network Hierarchy in the User

Interface
1. Navigate to the Admin tab

2. Select the System Configuration section
3. Click on Network Hierarchy
You must have admin rights

© 2012 I BM Corp.
•
•
•
•
Example of Network Hierarchy groups
Example of Network Hierarchy groups
The ´global´ group

captures the left overs.
Use http://ip2cidr.com
to calculate CIDR
ranges

© 2012 I BM Corp.
•
•
•
•
More detailed Network Hierarchy by using sub

groups
More detailed Network Hierarchy by using sub groups

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Lesson 5: Populate the Asset profile database
Lesson 5: Populate the Asset profile

database

 If a flow collector is part of the deployment, let the flow collection run for
a day to automatically populate the asset profile database with machine
communicating on the network segments covered by the flow collectors
in the deployment.
 Import asset profile manually, if no flow information is available for the
network segment that is covered by QRadar
Logsource
events with
identity
information also
add asset profile
information

© 2012 I BM Corp.
•
•
•
•
Search the Asset profile database
Search the Asset profile database

 Assets automatically discovered are automatically categorized by server
type. The Building Blocks used for categorization might need fine –
tuning.
 Let the system collect asset profiles for at least 24 hours before tuning
the Building Blocks

© 2012 I BM Corp.
•
•
•
•
Commonly edited Building Blocks
Commonly edited Building Blocks
Proxy servers and virus servers can generate high volumes of traffic. To reduce
the offenses created by these server types, edit the following building blocks to
reduce the number of offenses:
•BB:HostDefinition: VA Scanner Source IP

•BB:HostDefinition: Network Management Servers
•BB:HostDefinition: Virus Definition and Other Update Servers
•BB:HostDefinition: Proxy Servers
•BB:NetworkDefinition: NAT Address Range
•BB:NetworkDefinition: TrustedNetwork
Check Table 3-1 of the TuningGuidexxx.pdf document to get a complete

list of Building Blocks that might help to fine tune QRadar

© 2012 I BM Corp.
•
•
•
•
Lesson 6: Configuring Vulnerability Assessment
Lesson 6: Configuring Vulnerability

Assessment

 Configuring vulnerability assessment requires two steps:
• Configure one or more scanners from the Admin tab
• Schedule vulnerability assessment scans from either the
Admin tab or the VA scan tool in the Assets interface
 The scan results automatically create new asset profiles for
discovered hosts, or add detail to existing asset profiles
 Scan results only update profiles of local hosts
 QRadar connects to the configured scanners via APIs or ssh
depending on the scanner

© 2012 I BM Corp.
•
•
•
•
Add Vulnerabilty Scanners
Add Vulnerability Scanners
The scanner result gathering must

also be schedulled.

© 2012 I BM Corp.
•
•
•
•
Lesson 7: Collecting eventlogs with ALE
QRadar collects logs from

Microsoft W indows
machines preferably using
the syslog protocol. An
agent is required to send the
events as syslog messages
to the QRadar EC. QRadar
native agent is called
Adaptive Log Exporter
(ALE).

© 2012 I BM Corp.
•
•
•
•
License Agreement
License Agreement
After starting the setup,

accept the license
agreement.

© 2012 I BM Corp.
•
•
•
•
Installation directory
Installation directory
Choose the installation

directory.

© 2012 I BM Corp.
•
•
•
•
Installation type
Installation type
Choose packages. The UI

makes configuration easy.

© 2012 I BM Corp.
•
•
•
•
Shortcuts and access
Shortcuts and access
Startup the service after

installation.

© 2012 I BM Corp.
•
•
•
•
Configure the collection
Configure the collection

Start
configuration
Right click and add device
Configure the device

© 2012 I BM Corp.
•
•
•
•
Configure syslog forwarding
Configure syslog forwarding
Point ALE to a QRadar EC

© 2012 I BM Corp.
•
•
•
•
Assign event source to syslog destination
Assign event source to syslog destination
Finally assign the device(s) to the syslog destination being

the QRadar EC. (Don´t forget to deploy the changes.)

© 2012 I BM Corp.
•
•
•
•
Remote collect Windows events
Remote collect Windows events

 Remote collection requires you to run the ALE service under an account
with appropriate user credentials to read the event logs from the remote
machine
 ALE reads the remote event logs using the Admin$ share. NETbios
library must be installed on the target machine. Open a webbrowser and
connect to \\<remotemachine>\admin$ to see if the admin share is
enabled on the <remotemachine>
 The ALE service account must have read access to the remote eventlog.
Login to the ALE machine using the ALE service account credentials and
try to access the eventlog on the remote machine

© 2012 I BM Corp.
•
•
•
•
Configure remote collect Windows events
Configure remote collect Windows events

© 2012 I BM Corp.
•
•
•
•
Check access to remote Windows events
Check access to remote Windows events
•Open the eventviewer on the machine

that hosts ALE
•Connect to the remote machine
•You should be able to access the event
log

© 2012 I BM Corp.
•
•
•
•
Use WinCollect instead of ALE
Use WinCollect instead of ALE

 ALE has limitations. For example the maximum number of remote
collections by one ALE agent is 20 Log Sources or W2k8 not supported.
 WinCollect replaces ALE as from QRadar 7.1
 WinCollect installation currently only supported through commandline.
 WinCollect requires:
• 8GB of RAM (2GB reserved for the WinCollect agent)
• Intel Core 2 Duo processor 2.0 GHz or better
• 3 GB of available disk space for software and log files
• At minimum, 20% of the available processor resources
• The physical or virtual host system for the WinCollect agent must be installed with one of
the following operating systems:
o Windows Server 2003
o Windows Server 2008
o Windows 7
o Windows Vista
 Administrative privileges to install the WinCollect agent

© 2012 I BM Corp.
•
•
•
•
Prepare QRadar Console
Prepare QRadar Console

 Install DSM-WinCollect-xxx.noarch.rpm on the console first (7.0)
 Continue with installing PROTOCOL-WinCollect-xxxx.noarch.rpm (7.0)
 Create an authorized service and copy token with the WinCollect setup
executable to the windows host machine.

© 2012 I BM Corp.
•
•
•
•
Install WinCollect with Token
Install WinCollect with Token
Check the Setup Log dddd #n file created in the users temp directory
Agent should be detected automatically

© 2012 I BM Corp.
•
•
•
•
Create a Log Source
Create a Log Source
Specify the collection

parameters
and the W incollect agent to

be used
© 2012 I BM Corp.
•
•
•
•
Check collection
Check collection
WinCollect will also inform if it is running

without errors and in general what it is
doing

© 2012 I BM Corp.
•
•
•
•
Lesson 8: Configure authentication method

 Determine how new QRadar users should be authenticated:
• System
• Radius
• TACACS
• Active Directory
• LDAP
 If a method other than System is choosen, password management will be

handled by the external directory.
• For example if a new Active Directory user is created specifically for QRadar, don´t
create the account with Úser must change password at next logon´ option selected,
because QRadar can not change the password of the account in the external directory.

© 2012 I BM Corp.
•
•
•
•
Select authentication method
Select authentication method
Typical settings for Active

Directory:
• ldap://<hostname or ip>:389
• OU=ids,OU=ACC,
DC=<the>,DC=<domain>
• <the>.<domain>

© 2012 I BM Corp.
•
•
•
•
Creating users in the remote directory
Creating users in the remote directory

2
Example for Active Directory

© 2012 I BM Corp.
•
•
•
•
Add the user to QRadar
Add the user to QRadar

Make sure that you ALWAYS have a QRadar ad min de fined
before you change the authentication method from S yste m to
any other. That will ensure access to QRada r even after the
remote directory ti mes out.

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Lesson 9: Troubleshooting Connectivity
 Consult the QRadar_xx_TroubleshootingGuide.p df

 Connectivity Issues - i.e. “Events no t reaching the QRadar”
• e thtool
o This tool can be used to get information on a network interface, such as
speed, and dupl ex settings.
o e thtool eth0
• Check basic IP conn ectivity from source device to QRadar
o P ing devi ce from QRadar command l ine
o P ing QRadar from device
 Note: iptables is enabled on QRadar by defaul t so the managed host is
not ping’able
 To be able to pi ng QRadar you should type the following command:
service iptables stop
 To switch iptables back on type the fol lowing command: service iptables
start
© 2012 I BM Corp.
•
•
•
•
Troubleshooting Data collection
Troubleshooting Data collection
 If IP connectivity is good, then test that data being received

• Use the following commands on the QRadar shell:
o tcpdump –i eth0 looks for traffic to QRadar on eth0
o tcpdump –i eth0 port 514 checks to see if data is arriving at the
syslog port on the QRadar appl iance
o tcpdump –n src host x.x.x.x looks to see if traffic is arri ving to the
QRadar box from the x.x.x.x IP address
o tcpdump –A –s 0 –I eth0 –n port 514 and host x.x.x.x
l ooks to see if traffic is arriving to the QRadar box on syslog from the x.x.x.x
IP address.
o tcpdump –n –A –s 0 src host x.x.x.x provi des verbose output on
traffic going or coming from the specified host
• Note, by default you can on ly ssh to the distributed compon ents from the
console.

© 2012 I BM Corp.
•
•
•
•
Analyze event collection
Analyze event collection

 Change directory to /var/log and type:
• tail –f qradar.log | grep ‘Events per second‘
/var/l og/qradar.log
Events per second: 1s:41,41 (peak 8379,83 79) (compression: 0%)

5s:49,52 (peak 1702,1722) (compression: 4%) 10s:54,65 (peak 888,900)
(compression: 18%) 30s:46,58 (peak 332,344) (comp ression: 20%)
60s:100,112 (peak 264,277) (compression: 11%)
 Log statistics every sixty seconds when there is event traffic

 Includes raw and coalesced counters
 Sixty seconds most accurate
 Troubleshoot whether or not events are making it into the
system
© 2012 I BM Corp.
•
•
•
•
Check for Dropped Events & Flows (pre V7.1)
Check for Dropped Events & Flows (pre V7.1)

 Determining the cause of dropped events/flows requires a review of the
event processing pipeline.
• This is typically done via a shared session with support, reviewing
the details shown via the "jmanage" application. This is however not
always possible, convenient, or permitted.
• To generate such report:
o Change directory to /opt/qradar/bin and type:
./dumpMBeanSummary.sh
o Once the process is complete, you can download the .tgz file to
your desktop and review it. You should also forward this to
support for review, by email or attaching it to your open case.
 Troubleshoot whether (or not) events/flows are being dropped, and find
out why!
• Note that such report is very complete, and includes plenty of
statistics (and information on the QRadar system) that can be used
in many other troubleshooting situations. IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Logs and Messages
Logs and Messages

 Logs & Messages specific to QRadar
• /var/log/qradar.log
• QRadar system messages
• /var/log/qradar.error
o Where to troubleshoot items such as log sources…
• System Specific
• /var/log/messages
o Hardware message log. dmesg can be run as root for the same sort of output.
• /var/log/boot.log
o Contains system startup and shutdown messages – excellent for troubleshooting
power-outage times and impact.
• /var/log/audit.log
o Contains authentication requests and failures to system
• ALE Specific (When logging an ALE include these files)
• C:\Program Files\QRadar Adaptive Log Exporter\config
• C:\Program Files\QRadar Adaptive Log Exporter\logs
o Raw log file being monitored

© 2012 I BM Corp.
•
•
•
•
QRadar Setup reconfiguration
QRadar Setup reconfiguration

 /opt/qradar/bin/qchange_netse tup
• This script can be used to set the IP ad dress of a managed host. If you are
on a managed host, you will need to stop hostcontext in order to run thi s
script.
o S ervice ho stcontext stop
 /opt/qradar/bin/qradar_netse tup
• Will run the initi al configuration
 To restart processes
o S ervice tomcat start (consol e on ly)
o S ervice tomcat stop (console only)
o S ervice i mq start (console only)
o S ervice i mq stop (conso le only)
o S ervice po stgresql restart (console only)
o S ervice ho stcontext start
o S ervice ho stcontext stop © 2012 I BM Corp.
•
•
•
•
Additional Command Line Tips..
Additional Command Line Tips..

 To examine current license
• Type cat /opt/qradar/conf/license
 To get serial no. of system
• Type /opt/qradar/bin/getserial
• or type dmidecode | grep serial
 If you lock yourself out of the Console (too many password
attempts for example) –
• Login to QRadar as root
• Type service tomcat restart

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Summary
Summary
 Scope a QRadar solution
 Explain the QRadar basic deployment steps
 Setup a QRadar Network Hierarchy
 Use Asset Profiles
 Integrate Vulnerability information
 Use QRadar windows agents to collect Windows event logs
 Integrate QRadar authentication with Windows AD
 Perform basic QRadar troubleshooting

© 2012 I BM Corp.
•
•
•
•
•
•
•
•
Unit 6 Custom Log Sources
© 20 12 IBM C orp.
•
•
• 6-1
•
•
Introduction
Put your introduction text here. Do not delete the anchor because it is the anchor for the cross-
reference to the description in the preface.
Objectives
Objectives
 Obtain a test log file
 Create a Custom Log Source using a Universal DSM
 Create a LSX log parser document
 Create custom QIDs
 Map the custom log records to HLC and LLC categories

© 2012 I BM Corp.
•
•
•
•
Lesson 1: Create Custom Log Sources

 Challenges:
• Some customers require DSMs for devices not yet integrated with
QRadar
• Customer needs to have knowledge of and access to the device they
want to send logs from
• Need to be familiar with regular expressions (regex)
• Manual mapping of events required
 Benefits:
• Can quickly create DSMs for currently unsupported devices
• Can use any protocols in UDSM log source list

© 2012 I BM Corp.
•
•
•
•
Required tools
Required tools
• A U niversal DSM (UDSM ) can be used to integrate logs that have no
official DSM.
• A L og Source E xtension (LSX ) is applied to the UDSM to provide
parsing logic. The LSX uses Java regular expressions java/util/regex
• Obtain sample logs in the same format as logs that will be sent to
QRadar by the logsource.
• Syslog, FTP/SFTP/SCP protocol are the most commonly used
protocols

© 2012 I BM Corp.
•
•
•
•
7 Stages to get to full Custom Log Source support
7 Stages to get to full Custom Log Source support

1. Obtain a log sample in original format.
2. Create a Universal DSM to receive the log sample in QRadar
3. Create regular expressions to capture the Log Source EventID from the
log records
4. Create additional regular expressions to capture any other information
from the log records
5. Update the LSX template with these regular expressions and map the
Log Source EventIDs to Event Names
6. Create QID entries for the new Log Source EventIDs
7. Map the Event Names to the appropriate QIDs

© 2012 I BM Corp.
•
•
•
•
Lesson 2: Obtain the sample (from remote location)
Lesson 2: Obtain the sample (from remote

location)
Lesson 2: Obtain the sample (from remote location)

 Locate the unknown event by log source SIM Generic Log DSM-7
 Export the search result to a xml file and copy to your local QRadar
console
 Extract the base64 payload using /labfiles/xml2logfile.pl

© 2012 I BM Corp.
•
•
•
•
Lesson 3: Upload the LSX_Template.xml file
Obtain the LSX_Template.xml and upload it to the QRadar console

(C:\labfiles\QRadar on the W indowsAM machine)

© 2012 I BM Corp.
•
•
•
•
Create a Universal DSM Log Source
Create a Universal DSM LogSource
Choose an easy to recognize identifier
Disable event coalescing

Attach the LSX Template
Don’t forget to deploy the changes

© 2012 I BM Corp.
•
•
•
•
Test the Universal DSM Log Source
Test the Universal DSM LogSource
Use /opt/qradar/bin/logrun.pl to send the logrecords to QRadar.

Make sure to use the correct log source identifier

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Lesson 4: Start mapping the unknown logrecords
Lesson 4: Start mapping the unknown

logrecords

 Open the summary page for any logrecord and open the Custom Event
Properties window.

© 2012 I BM Corp.
•
•
•
•
Example of a LEEF log record event ID
Example of a LEEF log record event ID
Log Event Extended

Format is an IBM Log
record standard. The key
‘cat’ in the record contains
the Log Source EventID

© 2012 I BM Corp.
•
•
•
•
Create RegEx to extract the Log Source EventID
Create RegEx to extract the LogSource EventID
Just test the RegEx in the

Custom Event Property
window. Don’t save or create
a property
The sample RegEx is

.*cat\=(\w+)\t

© 2012 I BM Corp.
•
•
•
•
Lesson 5: Creating appropriate regular expressions
Lesson 5: Creating appropriate regular

expressions

 Use any regular expression tester based on java.regex.util
• http://www.regexplanet.com/advanced/java/index.html
• Or use the Extract Property utility.
The match is highlighted

© 2012 I BM Corp.
•
•
•
•
Common regular expressions
Common regular expressions

IP Address: \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}
Port Number: \d{1,5}
MAC Address: (?:[0-9a-fA-F]{2}\:){5}[0-9a-fA-F]{2}
Protocol: (tcp|udp|icmp|gre)
Device Time: \w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}
White Space: \s
Tab: \t
Match Anything: *? //Lazy * //Greedy
Additionally, the escape character, or "\", is used to denote a literal character. For example, in regex the "."
character means "any single character" and would match A, B, 1, X, etc. To specify a literal match, you
w ould use "\." instead. This would only match the "." character itself. Escaping any non-digit or non-alpha
character is usually the best way to ensure you do not accidentally match another character.

© 2012 I BM Corp.
•
•
•
•
Use of capture groups
Use of capture groups

 A capture group isolates a certain value in the regular expression.
 The value in the capture group is what is passed to the re levant fiel d in QRadar.
S impl y place parenthesis around the values you would like to capture. (Alternates
u ses two ;-) Be aware that alternates are p rocesse d as separate capture groups.
 Capture groups are expensi ve. There is almost no need to specify more than one
capture g roup in the regul ar expression.

© 2012 I BM Corp.
•
•
•
•
regular expression recommendation
Regular expression recommendation

 Try to use literal expression as much as possible. For example to match
“Windows EventLog” or “Windows DHCP”, use ( \” Windows \s (
EventLog | DHCP ) \” )
 Try to avoid the usage of alternation ( | ) as much as possible. The Java
engine (java.util.regex) is a NFA engine and backtracking is expensive.
 If possible indicate the location in the string to which the expression
applies to. Using ( ^ ) or ( $ ) for example.
 Apply lazy or greedy quantifiers wisely. In general if what you are trying
to match is in the beginning of the string, use the lazy quantifier.
Otherwise use the greedy.

© 2012 I BM Corp.
•
•
•
•
Lesson 6: Apply RegEx patterns to the LSX

 The next step is to assign the regex patterns with optional capture groups
to the appropriate fields in the LSX template.
In the example edit the EventName field:
[CDATA[]] become s [CDATA[.*cat\=(\w+)\t]]
 After this is done for the required fields, the LSX is ready to test.
 Also try to capture at least the following fields:
<pattern id="SourceIp" xmlns=""><![CDATA[]]></pattern>
<pattern id="SourcePort" xmlns=""><![CDATA[]]></pattern>
<pattern id="DestinationIp" xmlns=""><![CDATA[]]></pattern>
<pattern id="DestinationPort" xmlns=""><![CDATA[]]></pattern>
<pattern id="DeviceTime" xmlns=""><![CDATA[]]></pattern>
<pattern id="UserName" xmlns=""><![CDATA[]]></pattern>
<pattern id="HostName" xmlns=""><![CDATA[]]></pattern>

© 2012 I BM Corp.
•
•
•
•
Cleanup the LSX Template
Cleanup the LSX Template
If you do not need all the fields in the LSX template,

then delete them
Learn more about the LSX syntax from the

‘LogSources-7.xMRx_x.pdf ’ document

© 2012 I BM Corp.
•
•
•
•
Rerun the test
Rerun the test

 Load the adjusted LSX template to the Universal DSM LogSource
 Rerun the logrun.pl script with the test log file.
In any Event Summary, click on Map Event.

You MUST see a Log Source Event ID.
Otherwise something is wrong with your LSX

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Lesson 7: The value of QIDs

 A Log Source Event ID must be mapped to High Level Category and Low
Level category (HLC/LLC pair)
 This is arranged by creating record in the qidmap table, linking the Log
Source Event ID to a QID
 Every QID record contains an Event Name field value.
 This value is used as the QRadar Event Name for the Log Source ID
linked to the QID
 Sometimes existing QIDs are good enough to map the custom log
records
 Sometimes new QIDs are needed to map very specific custom log
records

© 2012 I BM Corp.
•
•
•
•
Create a new QID entry using qidmap_cli.sh
Create a new QID entry using qidmap_cli.sh

/opt/qradar/bin/qidmap_cli.sh -c --qname “My Custom Logon Event
Name" --qdescription “logon event from Custom LEEF log" --severity 3
--lowlevelcategoryid 3004
Use the –f option to import multiple QIDmappings from a file

© 2012 I BM Corp.
•
•
•
•
Lesson 8: Map the Log Source ID to the Custom QID
Lesson 8: Map the Log Source ID to the

Custom QID
Use the newly created

QID value to search for
the correct HLC/LLC
Clicking on OK will
create a QIDmap
record, linking the Log
Source Event ID to an
Event Name and QID.

© 2012 I BM Corp.
•
•
•
•
Map the Log Source Event IDs to existing QIDs
Map the Log Source Event IDs to existing QIDs
You can choose to

map the event to an
existing QID and be
done with it.
Remains the fact that

you should have a
clear understanding
what the Log Source
Event ID really means.
Check the Admin guide to get a list of existing HLC/LLC

combinations in the Event categories chapter
© 2012 I BM Corp.
•
•
•
•
Test the mapping
Test the mapping

 Rerun /opt/qradar/bin/logrun.pl with the same test log and check if the
‘Unknown’ Event Names are replaced by the Event Names you created.

© 2012 I BM Corp.
•
•
•
•
Final remarks
Lesson 9: Final remarks

1.At this point all new incoming log data will be mapped as designated.
There is no method to map old data in the system.
2.You can not add your own fields in the LSX. If you need to use a new
field, use a custom property instead. They can also be optimized (and
indexed.) (7.1+)
3.Custom Event properties / LSX can be used in reports, correlation rules
(7.0+), for dashboards, saved searches, etc
4.Custom Flow Properties (7.0+) can also be used in a similar way, learn
this and be ready for Custom Flow Properties (jitter, ttl, etc)

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Summary
Summary
Summary

© 2012 I BM Corp.
•
•
•
•
Summary
Summary
Summary

© 2012 I BM Corp.
•
•
•
•
Unit 7: Rules creation and fine
tuning
Unit 7 Rules creation and fine

tuning
© 20 12 IBM Corp .
•
•
• 7-1
•
•
Introduction
in this unit you learn how to create offense rules and fine tune false possitives
Objectives
Objectives
 Create effective rules leading to a minimal set of Offenses
 Fine-tune rules to minimize false positives

© 2012 I BM Corp.
•
•
•
•
Lesson 1: QRadar Rules reminder

 The basic components of rules are tests.
 Tests are onto functions defined on the domain space spanned by:
• Log activity events
• Network activity events
• Rules
• Offenses
To the set {TRUE,FALSE}
 Rules may have a Response or Action

© 2012 I BM Corp.
•
•
•
•
QRadar Building Block reminder
QRadar Building Block reminder

 The basic components of building blocks are tests.
 Tests are onto functions defined on the domain space spanned by:
• Log activity events
• Network activity events
• Rules
• Offenses
To the set {TRUE,FALSE}
 Building Blocks never have a Response or Action

© 2012 I BM Corp.
•
•
•
•
Linked tests
Linked tests
 Multiple tests can be linked to a single rule or building block using the
logical AND or NOT operators on the set {TRUE, FALSE} onto itself
 When linking tests, put the tests using the smallest subset of flows or
records at the bottom.
 Visualize the linked tests in a decision tree where each test narrows
down the result set that is used as search set for the next test.
 Logical OR is constructed by defining a test on rules or building blocks.
© 2012 I BM Corp.
•
•
•
•
Linking tests in the right order
Linking tests in the right order

 Tests are evaluated in top down order
 Put the test that narrows down the result search set of events or flows at
most, at the top
 Put the test that applies to the smallest set of events or flows, at the
bottom
 WRONG order:
1. Test for cl eartext appli cation usage
2. Test payload for creditcard numbers
3. Test if logsourcegroup is PCI critical
4. Test if network segment i s P CI network
 CORRECT order:
1. Test if network segment i s P CI network
2. Test if logsourcegroup is PCI critical
3. Test for cl eartext appli cation usage
4. Test payload for creditcard numbers IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Custom Rule Engine - CRE
Custom Rule Engine – CRE

 The CRE is the correlation engine of QRadar
 It executes the tests defined by Rules or Building Blocks
 Rules may have rule responses or actions
 Building Blocks do not have responses or actions
 All rule types except for Offense Rules, may trigger an Offense.
 Offenses are indexed by an event or flow property
 Events or flows matching a rule that triggers an offense AND where the
rule index type and value, matches the index type and value of an active
or recalled offense, are added to that offense.
 Events or flows matching a rule that trigger an offense AND where the
rule index type and value, DOES NOT match the index type or value of
any active or recalled offense, create a new offense.
 Events created by CRE have Log Source type “Custom Rule Engine-*”
© 2012 I BM Corp.
•
•
•
•
Lesson 2: Using Building Blocks

 Building blocks are used to categorize the properties of events or flows
 For example start with creating Building Block categories for the
properties:
• Destination IP, IPv6, MAC A ddress or Port
• Source IP , IPv6, MAC Address or Port
• Event Name, Event Category or IP Protocol
• Username

© 2012 I BM Corp.
•
•
•
•
Combine Building Blocks to capture specific

events or flows
Combine Building Blocks to capture specific events or

flows
 Implement the policy rule “Root or Administrator account must be used to
modify the audit subsystem configuration.”
 This translates into a rule combining the building blocks:
 And possible result:

© 2012 I BM Corp.
•
•
•
•
Lesson 3: Rule creation
Lesson 3:Rule creation

 CRE Rules can be used to capture relatively complex sequences of
events or flows and capture them in a single Offense
 Assume that the following sequence of events must be captured as a
single offense:
• Administrator creates an account
• Account is used to access sensitive data sets
• Administrator del etes the account
 Next slides show how to create the rules and actions to capture this
sequence of events
 First start with creating the rules to capture the individual events of the
sequence

© 2012 I BM Corp.
•
•
•
•
Rule to capture account creation
Rule to capture account creation
Store the name of

the account
recently created in
a Reference Set

© 2012 I BM Corp.
•
•
•
•
Rule to capture access to sensitive data
Rule to capture access to sensitive data
Created a Building Block

to categorize the
sensitive data files and
directories

© 2012 I BM Corp.
•
•
•
•
Rule to capture account deletion
Rule to capture account deletion
Created an additional property
EventID=(624|630).*?Message=(\w+)
© 2012 I BM Corp.
•
•
•
•
Combine the rules to the Offense Rule
Combine the Rules to capture the sequence of events
The rules defined earlier, are now

combined. Caution: the order in
which the rules are created
determines the order in which they
appear in the offense annotations.
In case this sequence is important,

try to build the capture rules in the
same sequence as the events/flows
are expected to appear.

© 2012 I BM Corp.
•
•
•
•
Lesson 4: Offense analysis

 Properly created offenses, indexes all triggered rules onto a single
offense
 The annotations are used to describe the offense
 The previous example indexed the triggered by the source ip address
• In this example thi s works because only a single windows machine was used to
forward al l win dows events in the domain. This machine assigns the source IP
address i n the wi ndows syslog message
 Locate the offense in the All Offenses page.

© 2012 I BM Corp.
•
•
•
•
Analyse the Offense summary
Analyse the Offense summary
The events annotations show up in the description in order of capture
This offense is indexed by source IP
The offense annotations appear at the bottom

© 2012 I BM Corp.
•
•
•
•
Check the rules that fired for the Offense
Check the rules that fired for the Offense
Sort Last Event/Flow column descending to see last triggered rule on top

© 2012 I BM Corp.
•
•
•
•
View events or flows contributing to the Offense
View events or flows contributing to the Offense
By indexing the offenses, you will see all contributing events and
flows in a single report

© 2012 I BM Corp.
•
•
•
•
Example of an Attack scenario
Example of an attack scenario

 A PC in user network attacks a server (web1) in DMZ network using know exploit.
 The server (web1) is vulnerable and the PC successfully exploits it.
 The server (web1) starts communication with attacker’s server (C&C) using port 80 with
non-HTTP protocol
Assu ming undete ctable AD/ DNS/ DHCP

mal ware down loa d her e 192.168.30. 11
User 192.168.30. 0/24

In ternet 192.168.40. 0/24
Q Rad ar App Scan

web Attacker Wi ndo ws wi ndow s w indo ws 192. 168. 10. 10 192. 168. 10. 30
192. 168. 40.100 192. 168. 40.44 192.168.30.33 192. 168. 30. 102 192.168.30. 103
Mg mt 192.168.10.0/24
Attac k Interna l Serv er(MS08 -67)

sno rt scann er xgs
Wi ndo ws proxy
R evers e 192. 168. 10. 11 192. 168. 10. 40 192.168.10. 88
192. 168. 30. 103 192. 168. 20.11
co nnec ti on
us ing por t 80
DMZ 192. 168. 20.0/24
w eb 1 w eb 2 web3
192. 168. 20. 40 192. 168. 20. 50 192.168.20. 60
(win) (alt oro) (wi n)
Vul nerab le
© 2012 I BM Corp.
•
•
•
•
Could be detected by this rule
Could be detected by this rule

 Detect unknown protocol on port 80
• Detect a flow a t start of HTTP session by checking S YN/ACK/Push packets
• If applicati on is Web.Web.Misc then assume th is is suspicious.

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Lesson 5: False positive management

 Most rules need to be fine tuned to the actual events and flows captured
 Customize the BB: FalsePositive: All Default False Positive BBs first.
 Building Blocks make use of pre-defined Asset Profiles , Network
Hierarchy Groups, Super user accounts, Event Names, IP protocols,
Remote Network Groups, etc…
 They all need to be reviewed and adjusted to the actual values that
should be used for the BB categories used in rules.
 It may also be the case that the rules triggering Offenses are too tight.
For example it might be that the current rule defines that 5 logon failures
within 5 minutes should trigger an offens. Although it should be noticed if
10 logon failures within 5 minutes occur.
 There is no single method to fine tune rules for false positive
management.
 Each case needs to be examined seperately
© 2012 I BM Corp.
•
•
•
•
Example 1
Example 1
 Take the example of suspicious access to sensitive data.
 Currently the test check for at least 1 account deletion within 5 days after
data is accessed and an account is created.
 But within 5 days many accounts are created and deleted while the
reference set only expands.
 Potentially this may lead to many false positives.
 Suggestion decrease the timewindow to 1 day and install a process that
cleans up the reference every day
 Instead of capturing access to the labfiles directory and subdirectories,
capture access to the subdirectories of the labfiles directory. This will
drop denied access to the labfiles directory.

© 2012 I BM Corp.
•
•
•
•
Example 2: Botnet access - determine the rule
Example 2: Botnet access – determine the rule
Start at the offense summary page

© 2012 I BM Corp.
•
•
•
•
Create a search for the contributing events
Create a search for the contributing events

A search helps to locate the events/flows that trigger the rule
In this sample
a Dutch IP
address has
been
connected to
and this IP
address is
listed in the
QRadar Botnet
group of
remote
networks IBM Sof tware Group | Securit y Division
© 2012 I BM Corp.
•
•
•
•
Use the False Positive wizard
Use the False Positive wizard
Use the false positive wizard

to accept similar events
under certain conditions

© 2012 I BM Corp.
•
•
•
•
Example 3: Capture the events first and decide

later
Example 3: Capture the events first and decide later

 Frequently many offenses will be created during the tuning phase of
QRadar.
 Sometimes it is impossible to address them al at once but you will need
to keep the number of active offenses as low as possible.
 Solution:
• Find the rule that cau ses the most offenses
• Capture the events/flows triggering tthe rule in a repo rt an d let schedul e the
report generati on
• Deci de offline which events/fl ows should be consid ered false positives.

© 2012 I BM Corp.
•
•
•
•
Finde the rules with the highest offense counts
Find the rules with the highest offense counts
Create a search and maybe even a report to capture all events and
flows

© 2012 I BM Corp.
•
•
•
•
Use the Rule to capture the events in a report
Use the Rule to capture the events in a report
Also adjust the rule to

temporarly not to trigger
Offenses

© 2012 I BM Corp.
•
•
•
•
Analyze the report
Analyze the report
You might decide that the source IPs in this sample should have
access, and use the false positive wizard for fine-tuning
© 2012 I BM Corp.
•
•
•
•
Lesson 6: Tuning Methodology

Single Target Multi ple Targets
One attacker, on e event Use Fals e Positive Wizard t o tune U se False Pos itive W iz ard t o t une
spec ific event s pecific event
One attacker, man y uni que Use Fals e Positive Wizard t o tune U se False Pos itive W iz ard t o t une
events in the same category category c at egory
Man y attackers, on e even t Use Fals e Positive Wizard t o tune Edit Building Bloc ks , using C ustom
spec ific event R ules Edit or, -t o t une s pec ific
event
Man y attackers, m any events in Use Fals e Positive Wizard t o tune Edit Building Bloc ks , using C ustom
the same catego ry category R ules Edit or, t o tune c at egory
One attacker, man y uni que Inves tigat e t he offense and det ermine Invest igat e the offense and
events in di fferen t categ ories the nature of t he att acker. If the det ermine the nature of the
off ense(s) c an be t uned out , edit at tacker. If the of fens e(s) can be
Building B locks, using Cus tom R ules t uned out , edit B uilding B locks,
Edit or, to t une c at egories f or the hos t us ing Cust om Rules Edit or, to t une
IP. c at egories for the host IP.
Man y attackers, m any uniq ue Edit B uilding Blocks, us ing Cust om Edit Building Bloc ks , using C ustom
events in di fferen t categ ories Rules Editor, t o t une categories R ules Edit or, t o tune c at egories
Check Chapter 3 of the TuningGuidexxx.pdf for detailed information

© 2012 I BM Corp.
•
•
•
•
Commonly Edited Building Blocks (reminder)
Commonly Edited Building Blocks (reminder)
Proxy servers and virus servers can generate high volumes of traffic. To reduce
the offenses created by these server types, edit the following building blocks to
reduce the number of offenses:
•BB:HostDefinition: VA Scanner Source IP

•BB:HostDefinition: Network Management Servers
•BB:HostDefinition: Virus Definition and Other Update Servers
•BB:HostDefinition: Proxy Servers
•BB:NetworkDefinition: NAT Address Range
•BB:NetworkDefinition: TrustedNetwork
Check Table 3-1 of the TuningGuidexxx.pdf document to get a complete

list of Building Blocks that might help to fine tune QRadar

© 2012 I BM Corp.
•
•
•
•
Student exercise
Student exercise

© 2012 I BM Corp.
•
•
•
•
Summary
Summary
Summary
 Create effective rules leading to a minimal set of Offenses
 Fine-tune rules to minimize false positives

© 2012 I BM Corp.
•
•
•
•

IBM

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

IBM

Încărcat de

Drepturi de autor:

Formate disponibile

QRadar 7.

1 Administration and Deployment

Unit 2: QRadar deployment and configuration

Unit 3: QRadar Software installation

Unit 4: QRadar architecture

Application detection . . . . . . . . . . . 4-5

Unit 5: Solution implementation

Lesson 5: Populate the Asset profile database . . . . . . 5-25

Unit 6: Custom Log Sources

Common regular expressions . . . . . . . . . . 6-15

Unit 7: Rules creation and fine tuning

© 2 012 IBM Corp .

IBM Sof tware Group | Securit y Division

Lesson 1: IBM Security Framework (ISF)

Lesson 1:IBM Security Framework (ISF)

ISF recognises 6 security

Software and appliances

Depending on the maturity

IBM Sof tware Group | Securit y Division

ISF maturity categories

ISF maturity categories

Maturity categories and security software types

Maturity categories and security software types

Vir tu ali zation secu rity

Enc rypti on Pe rimeter secu rity

People Data Applications Infrastructur e

IBM security software portfolio and QRadar SIEM

IBM security software portfolio and QRadar SIEM

IBM Sof tware Group | Securit y Division

Lesson 2: QRadar brief history

Lesson 2: QRadar brief history

• 2001 This resulted in a product called QVision positioned for the

• 2005 V5.0 LogSource Extensions, DSMs, Rules are added,

• 2011 V7.0 introduces optimized event and flow pipelines.

•2012 IBM acquires Q1Labs. QRadar bluewashed and runs only on

QRadar key Capabilities

Qradar Key Capabilities

Security Intelligence at work

Security intelligence at work

2 Bn security records per day 25 security offenses per day

•Reliable, secure and scalable log data storage

Ease of deployment: QRadar has many automations

Automated Automated offense

Clear leadership position in SIEM Magic Quadrant

Clear leadership position in SIEM Magic Quadrant

IBM Sof tware Group | Securit y Division

QRadar main product differentiators

QRadar main product differentiators

IBM Sof tware Group | Securit y Division

Lesson 3: QRadar software components and

Lesson 3: QRadar software components and data

Flow Colle ctor Eve nt Colle ctor • Network information

• Logs collected from

QRadar software components

QRadar software components

QRadar appliance components and data flow

QRadar appliance components and data flow

Flow Colle ctor Event c ollect or • Event logs preferably are

Flow Pr oc essor Event Pr oc essor • A flow collector is required

QRadar appliance types

QRadar appliance types

QRadar Log Management versus SIEM

QRadar Log Management versus SIEM

IBM Sof tware Group | Securit y Division