The 10th International Conference for Internet Technology and Secured Transactions (ICITST-2015)

RAT-based Malicious Activities Detection

on Enterprise Internal Networks

Masahiro Yamada, Masanobu Morinaga, Yuki Unno, Satoru Torii

Managed Security Division, Fujitsu Limited
Kawasaki, Japan

Masahiko Takenaka
Advanced Network Technology Division, Fujitsu Limited
Kawasaki, Japan

Abstract— The detection of APT has recently become an urgent
problem needing to be resolved. Attackers use Remote Access
Trojan/Remote Administration Tools (RATs), which often bypass
general security measures, and the traditional detection
techniques don't consider reconnaissance activities after RAT
infections. We analyzed the behavior of the reconnaissance for
this paper so that RAT-based malicious activities on internal
networks can be divided from the operations of normal users.
Based on the features of their behaviors, we propose a detection
technique that monitors the communications on internal
networks and extracts the communication sequences of the
reconnaissance. The result from our evaluation showed that the
proposed technique can detect 99.26 % of the experimental
reconnaissance communications by using the real 34 RATs (29
families) and 4 SMB-based remote management methods, and
also work without false-positive on an actual organization's
internal network.
Keywords-RAT; detection; internal network; reconnaissance;
I. INTRODUCTION
The risk of cyber-attacks has risen, and notably many
companies had already experienced incidents of APT [1]. APT
wreaks serious damage on the target despite the general
security measures that are taken. The purpose of APT is to gain
confidential information from a targeted company. In many
cases, attackers use a Remote Access Trojan/Remote
Administration Tool (RAT) for remotely controlling the hosts
on the targeted company's internal network. Attackers often
proceed to attack in stages to get deep into the internal
networks [2]. A typical attack scenario is as follows. At first,
an attacker encourages a host in the targeted company to get
infected with a RAT by using social engineering such as spear-
phishing or water-holing. After RAT infection, the attacker
seeks user accounts that have high privileges. If the attacker
succeeds to gain the administrative privileges, he or she orders
the infected host to intrude into another host on the internal
network, and then executes a RAT, hacking tool, command or
other program on the host. In this paper, we defined this
intrusion and execution activity as reconnaissance. The
reconnaissance activities are repeated through an increasing
number of compromised hosts until the attack is detected.
Therefore, developing new detection techniques to help prevent
this reconnaissance is an urgent task needing immediate
attention.
The many techniques against cyber-attacks have been
proposed. The most common techniques are Anti-Virus
software (AV) [3] and Intrusion Detection System (IDS) [4],
[5], [6]. Because host-based approach and network-based
approach catch up on the weaknesses each other, many
organizations use the both as security measures against cyber-
attacks. In our research, we focused the network-based
approach by monitoring and analyzing network traffic. The
traditional network-based techniques detect known malware
based on their signatures defined by each malicious packet.
Although this signature detection is suitable to detect known
malware and exploits, they cannot effectively detect variants of
known malware families, which send different messages or
encrypt messages with different keys. Behavior detection
approach, which analyzes the features of malicious connections
such as packet numbers, transferred bytes, protocols, and time
intervals of packets, is a possible way for detecting malware
variants. However, RAT behaviors are similar to that of push-
based information delivery services, and the existing behavior
detection techniques cannot accurately distinguish RAT
connections from normal users' communication.
We propose a novel network-based approach in this paper
for real-time detection of the RAT-based reconnaissance
communication sequences on enterprise internal networks. The
proposed technique extracts RAT connection candidates from
the inbound and outbound communications and the
combinations of the packets defined as administrative
operations of Windows such as the SMB-based remote
management. It also integrates the RAT connection candidates
and the related administrative communications. In other words,
it detects the combination use of a RAT and administrative
operation as reconnaissance. We implemented one specific
model of the proposed technique into a prototype network
sensor for evaluating our technique. The result from our
evaluation showed that the prototype can detect 99.26 % of
experimental reconnaissance communications including 34

978-1-908320-52/0/$31.00 ©2015 IEEE 321

 The 10th International Conference for Internet Technology and Secured Transactions (ICITST-2015)

RATs (29 families) and 4 SMB-based remote management III. ATTACK ANALYSIS
methods, and work without false-positive in an actual In the reconnaissance of APT, the attackers use RATs to
organization's internal network. remotely control the infected hosts on internal networks. They
The rest of this paper is organized as follows. Section 2 also steal the account information by using the pass-the-hash
gives an overview of the related works. Section 3 presents our and repeat unauthorized access to other hosts on the internal
analysis of the RAT-based reconnaissance used for APT. networks [2].
Section 4 presents our novel approach to detect the activities of
attacks on enterprise internal networks. An experiment and A. RAT
actual network practice for evaluating the detection accuracy of Analyzing the behavior of RAT communications, we
the proposed technique are presented in Section 5. Finally, we consider how to extract it from a large amount of normal users'
conclude in Section 6. communications. For this reason, we obtained approximately
200 RATs distributed on the internet. We found that there are
II. RELATED WORKS two types of RATs, direct connect type and reverse connect
In this section, we define the issues with the existing type. The direct connect type directly connects to the infected
network-based techniques for detecting reconnaissance. We host from the attacker's host, and the reverse connection type
categorized the existing techniques into three types, signature reversely connects from the infected host to the attacker's host.
detection, anomaly detection and behavior detection, and we Since general companies block direct accesses from the
gives an overview of three categories in this order. For example, internet to their internal networks, we analyzed 43 reverse
they can detect the communication of an infection or a keep- connection type RATs including 37 families. Each of the
alive between the attacker's host and the host infected with the analyzed RATs is composed of a RAT server and a RAT client.
known RAT. However, attackers can implement encrypted The RAT server runs on an infected host and provides remote
communication into a RAT or change the encryption key. Such control ability to the RAT clients. We analyzed the main
customized RATs can thus bypass the signature detection. remote control functions implemented into a RAT listed in the
Table I.
Anomaly detection is a contrary approach to signature
detection, and there are two typical types of network-based
TABLE I. REMOTE CONTROL FUNCTIONS OF RATS
anomaly detection techniques, machine learning and statistical
analysis. The machine learning type generates a whitelist or Function Type Percentage
threshold that distinguishes anomaly from the normal users' Kill process environment management 97.67 % (42/43)
communications [7]. The statistical analysis type finds anomaly Start/stop service environment management 76.74 % (33/43)
connections, hosts, or packets as outliers [8]. Both possibly Edit registry environment management 69.77 % (30/43)
detect attacks as anomalies even if the method of attack or the
List active window environment management 88.37 % (38/43)
malware is unknown. On the other hand, the anomaly detection
Search/download file environment management 86.05 % (37/43)
approach cannot define what the detected attack is, and should
be combined with another approach for practical use. URL Capture screen environment management 83.72 % (36/43)
Reputation is a hybrid approach of signature and anomaly Execute program cooperation with other tool 100.0 % (43/43)
detection [9]. However, the existing URL reputation approach Command prompt cooperation with other tool 79.07 % (34/43)
doesn't completely overcome the problems with the signature- Upload file cooperation with other tool 86.05 % (37/43)
based and anomaly approaches. For example, when an attacker Uninstall server server management 93.02 % (40/43)
launches the RAT client on a regular cloud Infrastructure as a Update server server management 60.47 % (26/43)
Service (IaaS), the URL of the host is difficult to judge as an
attack.
We found that the functions are classified into the three
Behavior detection is a possible approach for detecting the types. The first type is environment management, which
reconnaissance of APT. Some existing behavior detection provides the ability to change the environment and steal
techniques for finding the remote control of malware have been information from the infected host. For example, the attacker
proposed [10], [11]. S.Li et al. proposed a detection technique, kills and uninstalls an Anti-Virus software process when he/she
Manto [11], for finding Trojan horses based on the features of finds it in the running processes. The second type is in
their communications, such as the directions and intervals of cooperation with the other tools. Attackers use this type of
their packets on a connection and the number of connections function when they want to use the commands or programs that
between a server and a client. Since RATs are modern type of are not implemented into the RAT. For example, the attacker
Trojan horse, Manto should be able to extract RAT connections uploads and executes tools for attacking the other hosts on the
based on the features. However, whether or not a RAT also has internal network. The third type is server management.
all of the features of Trojan horses and is distinguishable from Attackers update or delete their RAT servers from their remote
the normal users' communications by using Manto still remains RAT clients to hide their own activities. These three types of
to be verified. functions are based on an interactive remote control. It is
believed that attackers choose the best way to stealthily attack
depending on the environment of the infected host. Assuming
the remote control of RATs is interactive, the unauthorized
access from the infected host follows shortly after receiving the

978-1-908320-52/0/$31.00 ©2015 IEEE 322

 The 10th International Conference for Internet Technology and Secured Transactions (ICITST-2015)
order from the RAT client. The sending of the results to the
RAT client also follows shortly after the completion of
unauthorized access.
TABLE II. FEATURES OF RAT COMMUNICATION
Number Feature of RAT communication Percentage
1 Connect form server to client 100.0 % (43/43)
2 Encrypted communication 53.49 % (23/43)
3 Push communication 100.0 % (43/43)
4 Sustain remote control connection 100.0 % (43/43)
5 Establish sub-connection 41.86 % (18/43)
We also analyzed the behavior of the RAT communications.
Many of its features are shared, as shown in Table II. Manto
[11], the existing detection technique for Trojan
communications, defines them as features 1, 2, 3, 4, and 5.
However, roughly half of RATs do not show the features 2 and
5 in our analysis. All of RATs sustain their remote control
connections until the RAT servers stop. Some of RATs use
keep-alive communication that is sent at constant time intervals
for checking that the connections exist. Thus, RAT servers
receive commands from the RAT clients and respond every
time on the sustained connections. The intervals between each
command are usually longer than that of many automated
programs, because the attackers manually send commands.
Normal users' accessing to some general web services on
the HTTP or HTTPS protocol also shows the features 1, 3 and
5. For example, automatic information distribution services for
stock prices keep the connection established for the client and
send information to the client at constant time intervals. We
cannot distinguish the RAT communications from such
services' communications using only these features. However,
we can use the features to narrow down the candidate
connection of a RAT from the normal users' communications.
We implemented a test program and installed it in an internal
network containing more than 1000 hosts. The test program
measures the rate of the candidate connections, which meet the
following conditions based on the features of the RAT
communication behavior.
1) The destination TCP port of the connection is allowed
to connect to the internet. (Feature 1)
2) In the case of the connection through a HTTP proxy
server, the connection uses CONNECT method for
interactive communication. (Feature 3)
3) The connection contains push communications
defined by the packet from the internet server after a
certain interval. (Feature 3)
4) The connection is sustained for a certain time.
(Feature 4)
978-1-908320-52/0/$31.00 ©2015 IEEE
Figure 1. Narrowing down candidate connections
The test program monitored 50-60 millions of TCP
connections over a 22-hour period. We assumed that the
interval of push communications for this measurement is 10
seconds. The graph in Figure 1 shows that 0.0159 %
connections were sustained for 10 seconds and only 0.0005 %
of the connections were sustained more than 240 seconds. This
result confirmed that the features of RAT communication
behavior is useful to narrow down the candidate connections of
RAT from the normal users' communication.
B. SMB-based Remote Management
This section considers the way attackers intrude internal
Windows hosts after the RAT infection. Pass-the-hash is a
method to steal information of Windows user accounts. The
pass-the-hash tools steal the user name and its password hash
from the Security Account Manager (SAM) Database and the
lsass.exe (LSASS: Local Security Authority Subsystem
Service) process [12]. The SAM Database records the account
information of all the local users. The Lsass.exe process
manages the account information of the users currently logged
in. The network logon and many other Windows services
authenticate the account by using the challenge and response
generated from the password hash. Once the attackers get a
password hash of administrative users, they can access the
other hosts on the internal network by using regular tools or
commands instead of exploits. Remote access of the regular
tools and commands is usually based on the common protocols
implemented into Windows, such as the Server Message Block
(SMB), Distributed Computing Environment/Remote
Procedure Call (DCE/RPC) and Remote Desktop Protocol
(RDP). Since SMB and DCE/RPC support many Windows
administrative services, we focused them in this paper.
PsExec[13] is a standard remote management tool in the
SMB and DCE/RPC. It is distributed by Windows Sysinternals
as a freeware. It provides access to a remote Windows host and
executes commands or programs without needing a dedicated
server program on the target host. The details of its behavior
are unclear, since the source code is not available. We
examined the behavior of PsExec and its communications, and
showed it is suitable for APT.
The behavior of PsExec consists of the following 10 steps.
Attackers can execute any commands and programs on the
target host when the target host allows remote hosts to access
the resources of its administrators based on the SMB by
sending and starting the service from the infected host. In other
323
 The 10th International Conference for Internet Technology and Secured Transactions (ICITST-2015)
words, the reconnaissance is the combination of a RAT and
remote management.
1) Establish an SMB session with target host.
2) Request a resource that requires the privileges of
administrators on the target host.
3) Send a service program file to the target host.
4) Access the remote service management service on the
existing SMB or DCE/RPC session established at the
time.
5) Register the sent file as a service on the target host.
6) Start the service.
7) Send the command or the program designated as a
PsExec argument.
8) Receive the results of the command or program from
the service.
9) Stop and delete the service.
10) Disconnect from the resource.
There are some PsExec-like tools, such as PAExec[14] and
xCmd[15]. Although they show the same behaviors, PAExec is
different from PsExec because it is a redistributable tool. xCmd
is different in only the Step 9, which is done by the service
itself. The function of these SMB-based remote management
tools is also reproducible by combining a customized service
program and some of the Windows APIs or commands, such as
"net use", "copy" and "sc". We focused on the combination of
the following three operations as the behavioral features of the
administrative communications on SMB and DCE/RPC.
x Access to a remote host as an administrator.
x Send a file to the remote host.
x Start a service on the remote host.
We can classify each operation by analyzing the
information in SMB or DCE/RPC protocol headers of packets.
By extracting the combination of the operations, we can divide
communications of the SMB-based remote management from
that of the other SMB-based operations. Additionally, the
remote management by RAT-based attacks immediately starts
after that the infected host receive the RAT command. Thus,
we can divide communications of the remote management by
attackers and administrators from existence of the related RAT
connection candidate.
IV. PROPOSAL
In this paper, we propose a novel approach for detecting
communication sequences of the reconnaissance behaviors of
APT. We focused RATs and the administrative
communications on SMB and DCE/RPC. The proposed
technique detects the distinctive communication sequences of
the attacks from the large amount of normal users'
communications on internal networks based on the two factors.
The first factor is a temporal relationship between RAT
978-1-908320-52/0/$31.00 ©2015 IEEE
connections and administrative operations because of
interactive remote control. The second factor is that each RAT
connection candidates and SMB-based administrative
operations can be individually extracted from communications
of the normal users when focusing on the features of their
behaviors.
Figure 2. Sequence of SMB-based reconnaissance
Figure 2 shows a simplified communication sequence of
the SMB-based reconnaissance. In this sequence, the infected
host with the RAT first receives an order from the RAT client
on the attacker's host. After that, the infected host accesses the
target host and executes a command or a program on it using
SMB-based remote management. SMB-based remote
management includes accessing a resource allowed for only
administrators, sending a service program file, starting a
service remotely on the same connection. Then, the infected
host sends the results of the command or the program to the
attacker's host. The relationship between the RAT command
from the attacker’s host, the SMB-based administrative
operations, and the result of RAT command forms into a nest
structure. This nest structure, which has the both behavioral
features of RAT connection and administrative operations, is a
distinctive behavior model of reconnaissance.
V. EVALUATION
In order to evaluate the proposed detection technique, we
implemented a prototype of it to be used as a network sensor. It
extracts the communication sequences of the SMB-based
remote management and the related candidates of the RAT
connection. Our experimental environment and actual network
practice on an actual organization's internal network were used
to evaluate and validate how effectively the prototype extracts
the reconnaissance communications from that of normal users.
A. Experiment
For the evaluation, we validated that the prototype exactly
detects the combination of RATs and SMB-based remote
management tools. The 34 RATs (29 families) of the reverse
connection type RATs analyzed in Section 3.1 have the
command prompt function for cooperation with the remote
management tools. We evaluated the detection accuracy of our
prototype against the combinations of these RATs and some
SMB-based remote management tools.
324
 The 10th International Conference for Internet Technology and Secured Transactions (ICITST-2015)
TABLE III. DETECTION RESULTS FOR EXPERIMENTAL VALIDATION
SMB-based remote management Detection accuracy
PsExec 100.0 % (34/34)
PAExec 97.06 % (33/34)
xCmd 100.0 % (34/34)
Original batch script 100.0 % (34/34)
Total 99.26 % (135/136)
The prototype distinguishes 99.26 % of the combinations of
the RATs and SMB-based remote management tools with only
one false-negative as shown in Table III. In the false-negative
case, the RAT server failed when it executed PAExec. We
assumed that this is caused by a bug in the RAT server.
Although the RAT client received the output of PAExec, the
remote control connection was broken and the attacker could
not continue the attack. This result confirms that the proposed
technique accurately detects the RAT-based reconnaissance of
APT when the attackers use SMB-based remote management.
B. Actual Network Practice
For evaluating the actual level of detection accuracy of the
proposed technique, we used the prototype sensor to monitor a
given point in an internal network of an actual organization.
The prototype monitored 50-60 millions of TCP connections
over a 22-hour period. The practice results included zero false-
positive communications. This also confirms that the proposed
technique works without any false-positive in an actual
organization.
C. Discussion
The proposed approach is based on the combination of a
RAT and SMB-based remote management to accurately
distinguish malicious communications from that of normal
users. The proposed technique focuses on the behavioral
features of modern RATs. Although their behavior may
drastically change in the future, we assume that it takes at least
few years. The term is enough to enhance the proposed
technique or develop another technique.
When the attackers use other protocols for reconnaissance
such as RDP or SSH, the current proposed technique misses
the attacks. We assumed that current APT trend is SMB-based
remote management because many Windows administrative
services work on SMB and DCE/RPC. However, we have to
expand the proposed technique for detecting reconnaissance on
the other protocols used for remote management in the future.
VI. CONCLUSION
In this paper, we analyzed the RAT-based reconnaissance
behaviors on internal networks, and also proposed a detection
technique for their communication sequences that is based on
the behavioral features. It extracts RAT connection candidates
from the inbound and outbound communications and the
administrative operations analyzed from the protocol headers
of SMB and DCE/RPC packets. It also integrates and detects
the RAT connection candidates and the related administrative
communications. Our evaluation of the proposed technique
prototype showed that the proposed technique can detect the
978-1-908320-52/0/$31.00 ©2015 IEEE
reconnaissance of 99.26 % of the 34 real RATs (29 families)
on the experimental environment, and also accurately
distinguish between the behaviors of the reconnaissance of
APT from that of the normal users on an actual organization's
internal network.
ACKNOWLEDGMENT
This research is supported by R&D for analysis and
detection of cyber-attacks, administered by the Ministry of
Internal Affairs and Communications of Japan.
REFERENCES
[1] C. Tankard, “Advanced Persistent Threat and How to Monitor and Deter
Them,” Network Security, vol.2011, issue.8, pp.16-19, 2011
[2] MANDIANT, “M-Trends: The Advanced Persistent Threat,” Technical
Report, 2010
[3] O. Erdogan, P. Cao, “Hash-AV: Fast Virus Signature Scanning by
Cache-Resident Filters,” International Journal of Security and Networks,
vol. 2, Issue. 1, pp.50-59, 2007
[4] K. Ilgun, R. A. Kemmerer, P. A. Porras, “State Transition Analysis: A
Rule-Based Intrusion Detection Approach,” IEEE Transactions on
Software Engineering, vol.21, pp.181-199, 1995
[5] M. Roesch, “Snort - Lightweight Intrusion Detection for Network,”
Proceedings of The 13th USENIX Conference on System
Administration, pp.229-238 ,1999
[6] V. Paxon, “Bro: A System for Detecting Network Intruders in Real-
Time,” Computer Networks: The International Journal of Computer and
Telecommunications Networking, vol.31, no. 23-25, pp.2435-2463,
1999
[7] S. Yun Lim, A. Jones, “Network Anomaly Detection System: The State
of Art of Network Behaviour Analysis,” Proceedings of The 2008
International Conference on Convergence and Hybrid Information
Technology, pp.459-465, 2008
[8] T. Yen, M. K. Reiter, “Traffic Aggregation for Malware Detection,”
Proceedings of The 5th International Conference on Detection of
Intrusions and Malware, and Vulnerability Assessment, pp.207-227,
2008
[9] J. Ma, L. K. Saul, S. Savage, G. M. Voelker, “Beyond Blacklists:
Learning to Detect Malicious Web Sites from Suspicious URLs,”
Proceedings of The 15th ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining, pp.1245-1254, 2009
[10] G. Gu, P. Porras, V. Yegneswaran, M. Fong, W. Lee, “BotHunter:
Detecting Malware Infection Through IDS-driven Dialog Correlation,”
Proceedings of 16th USENIX Security Symposium, pp.167-182, 2007
[11] S. Li, X. Yun, Y. Zhang, J. Xiao, Y. Wang, “A General Framework of
Trojan Communication Detection Based on Network Traces,”
Proceedings of The 2012 IEEE Seventh International Conference on
Networking, Architecture, and Storage, pp.49-58, 2012
[12] SANS Institute, “Pass-the-hash Attacks: Tools and Mitigation,”
Technical Report, SANS Institute, 2010
[13] Microsoft, Windows Sysinternals "PsExec", available at:
https://technet.microsoft.com/ja-jp/sysinternals/bb897553.aspx
(accessed Nov. 20, 2015)
[14] Power Admin LLC, PAExec - The Redistributable PsExec, available at:
http://www.poweradmin.com/paexec (accessed Nov. 20, 2015)
[15] Google Code, linkwithweb - Revision 111:
/trunk/Utilities/RemoteExecution/xCmd, available at:
https://linkwithweb.googlecode.com/svn/trunk/Utilities/RemoteExecutio
n/xCmd (accessed Nov. 20, 2015)
325