Securing Serviceguard

March 2009

Table of Contents
Abstract ................................................................................................................................ 3 1. 2.
2.1.

Introduction .................................................................................................................. 4 Threat Analysis ............................................................................................................. 4

Security Threats...........................................................................................................................4

3.
3.1.

Threats which Serviceguard installations commonly face..................................................... 4

External threats ...........................................................................................................................5

3.1.1.

Ports Used by Serviceguard......................................................................................... 5

It is not recommended to utilize host-based configuration to achieve isolation from potentially hostile activity. ..5

3.1.1.1.
3.2. 3.3. 3.4. 3.5.

inetd.sec ...........................................................................................................6

Internal threats ............................................................................................................................6 Inside the firewall protected security perimeter: the trusted network ...................................................6 Summary of threats which must be environmentally defended: ..........................................................6 A future hardened Serviceguard ...................................................................................................7

4.
4.1. 4.2. 4.3. 4.4. 4.5. 4.6.

The Serviceguard Security patch of 2004 ......................................................................... 7

Serviceguard authentication .........................................................................................................7 Authentication using identd ..........................................................................................................7 Weaknesses with auth ..............................................................................................................7 Is identd a security threat? ............................................................................................................7 Stronger alternatives to identd ......................................................................................................7 Weaker alternatives to identd .......................................................................................................8
/.rhosts ............................................................................................................................................8 cmclnodelist, RBA ..............................................................................................................................8

4.6.1. 4.6.2.

5.
5.1. 5.2. 5.3. 5.4.

Considerations for Serviceguard Manager........................................................................ 8

Firewall configuration ..................................................................................................................8 Cluster Object Manager ..............................................................................................................8 Sniffing concerns.........................................................................................................................8 Spoofing concerns.......................................................................................................................8

6.
6.1.

Considerations for Quorum Server................................................................................... 9

Quorum server is not a threat, but is inside the security domain ........................................................9

7. 8. 9.

Considerations for Continentalclusters .............................................................................. 9 Appendix A: required firewall protection.......................................................................... 9 Ports Used by Serviceguard......................................................................................... 9 Appendix B: requirements of a trusted network................................................................ 11
Authentication: ................................................................................................................................................11 Authorization: .................................................................................................................................................11 Denial of Service: ............................................................................................................................................11 Root Exploit: ...................................................................................................................................................11 Root User:.......................................................................................................................................................11

Glossary.............................................................................................................................. 11

Abstract
Security Analysis for Serviceguard consists of first doing a threat analysis and then by implementing defenses against existing threats. Serviceguard must always be protected against various threats, and requires, for example, firewall protection from a hostile internet. In this paper, we propose highly effective, low cost solutions that address the threats commonly faced by Serviceguard customers. This document is intended for anyone who is analyzing or configuring security for a Serviceguard cluster. We hope to confirm to most customers that their existing security analysis and protection is correct and sufficient, while warning others that they may have unprotected vulnerabilities.

1.

Introduction

Security Analysis for Serviceguard consists of first doing a threat analysis and then by implementing defenses against existing threats. Serviceguard must always be protected against various threats, and requires, for example, firewall protection from a hostile internet. In this paper, we propose highly effective, low cost solutions that address the threats commonly faced by Serviceguard customers.

2.

Threat Analysis

Consider three levels of security: 1. Bathroom/bedroom locks 2. Outside door deadbolts. 3. Defended compound perimeter For most people, threat analysis in daily life results in appropriate usage of levels of security. At the inner level, we expect that people inside a house will behave socially, and will not exploit the potential to crack the security provided by bed/bath locks which usually can be opened using a paper clip. The outer door deadbolt offers security against anti-social behavior, but no real defense against determined, armed criminal intent. The local police are expected to provide a secure community, and defense against criminals, but if the police cannot be expected to provide that level of security (such as when a government official faces an assassination threat), then greater measures are appropriate. Observe that the defenses match the threat. Inside the home, there is a threat to privacy which is defended by the bed/bath lock. Outside there is the threat of forcible entry defended by the deadbolt. The defended compound perimeter defends against the assassin. It can be observed that: 1. Each level of security comes with increased cost. 2. Its often appropriate to protect an outer perimeter more aggressively than an inner perimeter. 3. Too weak of security exposes one to threats, while overly aggressive security can be highly disruptive to daily life. Thus, we must analyze the threats correctly, and select defenses appropriately. In particular, maximum security is observed to be both expensive and intrusive to daily life.

2.1. Security Threats

Security threats in the abstract reduce to two classes: 1. Root exploits, meaning that some weakness can be exploited to gain control of a computer with no intentional authorization. 2. Denial of Service, meaning that some weakness can be exploited to cause a computer or program to crash, or otherwise be prevented from providing the service it normally provides.

3.

Threats which Serviceguard installations commonly face

Most Serviceguard clusters lead a sheltered life. Theyre owned by commercial enterprises and benefit from the good perimeter defenses which the rest of the IT infrastructure requires. The people who are allowed access to them are employees only, who reliably conform to written and unwritten constraints against anti-social behavior.

3.1. External threats

This sheltered life exists because of a strong perimeter defense. Serviceguard (repeated in Appendix A): Here are the IP ports used by

Ports Used by Ports Used by Serviceguard on HP-UX Serviceguard on HP-UX and Linux Before A.11.19 and Linux - At A.11.19
Ident 113/tcp hacl-cs 1238/tcp clvm-cfg 1476/tcp hpux only hacl-hb 5300/tcp hacl-hb 5300/udp hacl-gs 5301/tcp hacl-gs 5301/udp hacl-cfg 5302/tcp hacl-cfg 5302/udp hacl-probe 5303/tcp hpux only hacl-probe 5303/udp hpux only hacl-local 5304/tcp hacl-test 5305/tcp hacl-dlm 5408/tcp Ident 113/tcp hacl-qs 1238/tcp clvm-cfg 1476/tcp hpux only hacl-hb 5300/tcp hacl-hb 5300/udp removed removed hacl-cfg 5302/tcp hacl-cfg 5302/udp hacl-probe 5303/tcp hpux only hacl-probe 5303/udp hpux only hacl-local 5304/tcp removed removed hacl-poll 5315/udp - added icmp 8/icmp added wbem-https 5989/tcp snmp 161/udp snmp 162/udp

These rules should be established for both IPv4 and IPv6 network addressing. Serviceguard uses ports in the dynamically allocated range. On Suse Linux Suse distribution the dynamic range is 1024 29999. All of these ports must be isolated, using an external firewall, from any hostile activity. Hostile activity includes nuisance connections to the port, connections to the port from foreign nodes masquerading as cluster nodes, and attempts to crack into the system. Crucially, the firewall must also never pass traffic which originates on the outside of the firewall but which claims to be from inside the firewall.

3.1.1. It is not recommended to utilize host-based configuration to achieve isolation from potentially hostile activity.
We do not recommend the use of local IP filtering, sometimes called a software firewall, to achieve any of our security requirements. IP filtering, configured on the node being defended, requires great care to defend against spoofing and denial of service attacks which are relatively simple to defend at the external firewall. While some sites may find a combination of internal configuration and an external firewall or formally authenticated communications to be optimal, most sites will find that the external firewall is the correct platform to implement defence against potential exposure. Its far simpler, and thus easier to analyze and to
5

maintain. Observe that the external firewall cannot detect some kinds of spoofs either, but it is possible for it to prevent external computers from spoofing internal ones. The use of IP filtering to meet non-Serviceguard security requirements is fully supported. 3.1.1.1. inetd.sec

inetd can be configured to reject selected transactions based on ip addresses. This is configured via the file inetd.sec. However, inetd requires roughly the same kind of protection that Serviceguard requires in order for this to be useful. It is vulnerable to spoofing. Inetd.sec should never be seen as protection from hostile threats.

3.2. Internal threats

The nature of Serviceguard clustering implies that all nodes in a cluster must reside inside of a common security domain (see 3.3 for the definition of security domain); root (refer to the glossary for the Serviceguard non-standard definition of root user) on one node inside a cluster cannot be prevented from achieving root on another node in the same cluster. Its pointless to try to isolate one cluster node from another, and the security analysis should accept this basic assumption. In addition to the actual cluster nodes, the security domain of a cluster includes all machines which can place arbitrary packets onto the networks connected to the cluster. The Serviceguard security model prevents non-root users inside the security domain from exploiting Serviceguard to gain root access on a node running the Serviceguard daemons. Serviceguard is exposed to denial of service attacks from internal, non-root users. Were deliberately not including any details here, but the security analysis should take into account this potential. We expect that system administrators will not grant any access or accounts to users who might feel hostile or are inclined to disruptive pranks.

3.3. Inside the firewall protected security perimeter: the trusted network
One or more hardware firewalls should be used to create a security perimeter surrounding more than one cluster. The inside of such a security perimeter we refer to as a security domain. Within a security domain, security between clusters (more literally, between root users on clusters) is compromised. Generally, it is required that any computer not isolated by an external defense must have a trusted root user. Root users have unlimited access to the networks, and no security measure would prevent them from engaging in various attacks based on forged network packets or spying on network traffic. It is also required that the security domain be physically protected from untrusted computers. It would be a serious breach to allow an untrusted user to connect a laptop PC to a network inside the perimeter. Were not including any details here, but such a user, given appropriate knowledge, skill, and motivation, could eventually exploit Serviceguard to gain root on all Serviceguard nodes in the cluster.

3.4. Summary of threats which must be environmentally defended:

Within the security domain defined by the security perimeter, root on one computer must be trusted by all Within the security domain, every computer must be competently administered such that an untrusted user cannot gain root. Within the security domain, even non-root users must be trusted to not launch disruptive attacks.

3.5. A future hardened Serviceguard

Conceivably, Serviceguard could be provided with security options allowing it to operate securely in a very hostile environment. The Serviceguard security patch of 2004, however, is based on the defenses described here.

4.

The Serviceguard Security patch of 2004

The Serviceguard Security patch of 2004 hardens the Serviceguard authentication mechanism. One enhancement is the use of the Unix/Linux auth service, and its TCP daemon, identd, to identify users. This patch is not motivated by customer reported issues, rather its based on internal analysis that predicts that a small portion of our customer base may be exposed to some threats which we can do better to defend against. HP strongly recommends this patch to be installed, and that identd be configured to achieve greater security.

4.1. Serviceguard authentication

The essence of this patch is a hardening of the SG authentication mechanism. Pre-patch, Serviceguard used a method described as self-authentication. The patch replaces self-authentication with reliable methods to determine which user is attempting to use Serviceguard functionality.

4.2. Authentication using identd

UNIX variants provide a standard caller id service, called, variously, ident, or auth. This service implements RFC1413, a formal specification of the auth protocol. The daemon which implements this service, identd is shipped standard on all variants of UNIX, including HP-UX and Linux (on some variants of Linux, it is called pidentd). The Serviceguard patch requires identd to be configured on all cluster nodes, on any node that intends to use Serviceguard commands, and on any node running the Cluster Object Manager. See the patch installation documentation for details. For sites that face no security threat, the use of identd is optional.

4.3. Weaknesses with auth

Identd and the auth protocol it implements are no more reliable than the computer they are running on. If that computer has been compromised, the computer can report any name it chooses. Thus, trusting the information reported by identd is dependent upon trusting the root user on that computer. It can and should also be observed that the information provided by identd is no less reliable than the computer supplying the information.

4.4. Is identd a security threat?

Security experts sometimes recommend that identd be disabled, because it gives out information about users on that computer. The information given out is login name (as recorded in /etc/passwd, or whatever local technology is replacing /etc/passwd). If two login names share the same UID, the name reported is usually, but not always the first one in /etc/passwd. Given the nature of the trust required inside the firewall, the Serviceguard development team sees no compromise to security by enabling identd to identify users who hold TCP connections to other computers. Serviceguard does not require the external firewall to pass any identd traffic, either incoming or outgoing. For sites which do not wish to expose to the external internet the limited information which identd provides, we recommend blocking, at the external firewall, all incoming packets destined for auth on any protected node (port 113).

4.5. Stronger alternatives to identd

Stronger authentication mechanisms than identd exist, based on credentials which must be configured and, depending on the level of security required, themselves certified by contact with a trusted third party.
7

Such measures require significant administrative load at configuration and maintenance time. Serviceguard has selected the use of identd as more appropriate for our customers. Serviceguard is evaluating the future potential to supply an option to use stronger security for those customers who seek to use Serviceguard in a more hostile context.

4.6. Weaker alternatives to identd

Serviceguard strongly recommends the use of identd. However, a weaker alternative, not requiring identd, exists which is suitable for installations where all users are trusted not to be attempting hostile activity. Please see the Serviceguard manual and patch documentation for details.

4.6.1. /.rhosts
Serviceguard has always allowed the use of /.rhosts (more precisely, ~root/.rhosts) to expose cluster nodes to each other. However, the use of /.rhosts exposes the nodes to other threats, and has long been discouraged as a security weakness. Since very early Serviceguard releases, it is not required or even recommended to use /.rhosts. Note that after Serviceguard Release A.11.19 usage of .rhosts and host equivalency will not be supported for access control policies.

4.6.2. cmclnodelist, RBA

Previous to Serviceguard A.11.16, we have provided a Serviceguard only method, the cmclnodelist file, as a replacement for using /.rhosts. $SGCONF/cmclnodelist works much like /.rhosts, but was only used by Serviceguard, and thus is not a general exposure. When $SGCONF/cmclnodelist exists, /.rhosts is not used by Serviceguard at all. Previous to Serviceguard A.11.16 the cmclnodelist file was used to configure all inter-host access for Serviceguard commands. For A.11.16 A.11.19, this file is used only to configure the cluster. Once the A.11.16 cluster is configured, an internal access control list is used. This internal list defaults to allowing root from any cluster node manage any other cluster node, but rejects all external nodes. See the A.11.16 Serviceguard manual for a more complete description of Role Based Access.

5.

Considerations for Serviceguard Manager

5.1. Firewall configuration

Serviceguard Manager, the Serviceguard GUI, is sometimes desired to run outside the firewall, outside the security domain. Serviceguard Manager functionality uses connects to port 5303, hacl-probe, using TCP. To enable such use, the firewall must be configured to allow such connection. Serviceguard recommends restricting such access as tightly as possible, and never to a hostile address space.

5.2. Cluster Object Manager

The Cluster Object Manager can be used from cluster nodes, or from nodes of another cluster, or from nodes not in any cluster. When used from any node not in the cluster, the node containing the Cluster Object Manager must be inside the same security domain as the cluster being managed.

5.3. Sniffing concerns

Sniffing means examining network packets as they stream by or through ones computer. If Serviceguard Manager is being used outside the firewall, great concern must be taken to protect, from sniffing, the traffic sent by Serviceguard Manager to the Serviceguard cluster. Again, it is not acceptable to simply connect Serviceguard Manager to a hostile network and begin using it to administer a Serviceguard cluster.

5.4. Spoofing concerns

A computer can generally be configured with any IP address its administrator desires. Forging anothers IP address is easy, and a significant threat outside the trusted network.
8

6.

Considerations for Quorum Server

The quorum server must exist within the same security domain as the cluster(s) it provides quorum services to. The quorum server must be protected from attack similarly to any Serviceguard node.

6.1. Quorum server is not a threat, but is inside the security domain
The root user on any machine inside the security domain has exploit opportunities on other machines inside the security domain, and therefore must be trusted. A running quorum server does not create new exploit paths.

7.

Considerations for Continentalclusters

Continentalclusters is our disaster tolerant solution. It uses multiple Serviceguard clusters, separated from one another by as far as thousands of kilometers. The nature of Continentalclusters requires that all participating clusters must share a security domain, and thus they must share the same firewall protected network perimeter. This security domain may be very geographically distributed, but the clusters cannot be firewalled off from each other. The communications paths between the clusters must not be exposed beyond this security perimeter.

8.

Appendix A: required firewall protection.

Ports Used by Ports Used by Serviceguard on HP-UX Serviceguard on HP-UX and Linux Before A.11.19 and Linux - At A.11.19
Ident 113/tcp hacl-cs 1238/tcp clvm-cfg 1476/tcp hpux only hacl-hb 5300/tcp hacl-hb 5300/udp hacl-gs 5301/tcp hacl-gs 5301/udp hacl-cfg 5302/tcp hacl-cfg 5302/udp hacl-probe 5303/tcp hpux only hacl-probe 5303/udp hpux only hacl-local 5304/tcp hacl-test 5305/tcp hacl-dlm 5408/tcp Ident 113/tcp hacl-qs 1238/tcp clvm-cfg 1476/tcp hpux only hacl-hb 5300/tcp hacl-hb 5300/udp removed removed hacl-cfg 5302/tcp hacl-cfg 5302/udp hacl-probe 5303/tcp hpux only hacl-probe 5303/udp hpux only hacl-local 5304/tcp removed removed hacl-poll 5315/udp - added icmp 8/icmp added wbem-https 5989/tcp snmp 161/udp snmp 162/udp

These rules should be established for both IPv4 and IPv6 network addressing. Serviceguard uses ports in the dynamically allocated range. On Suse Linux Suse distribution the dynamic range is 1024 29999.

All of these ports should be isolated, using an external firewall, from any hostile activity. Hostile activity includes nuisance connections to the port, connections to the port from foreign nodes masquerading as cluster nodes, and attempts to crack into the system. The firewall must also never pass traffic which originates on the outside of the firewall but which claims to be from inside the firewall.

10

9.

Appendix B: requirements of a trusted network

1. The root user on any node inside the trusted network must be trusted by all nodes. This includes any Windows computer. 2. The network must be physically defended against any spoof or sniff attacks. 3. The network firewalls must be correctly configured to defend against external attacks.

Glossary
Authentication:
How to know who a user is. A special case is the verification that a process claiming to be owned by root really is owned by root. What a user is authorized, or allowed to do. A security vulnerability that allows a non-root user to crash a computer or otherwise disrupt its intended purpose. A security vulnerability that allows a non-root user to achieve UNIX superuser status by means other than knowing the password of a root enabled account. Any user on a UNIX system with an effective userid of 0. Also includes any user account on any system running Microsoft Windows. The relevant issue for this document is whether or not the user can generate arbitrary network data (including spoofed packets) and can listen on any port, including low numbered ports. If a user meets either of these criteria, they will be considered a root user for purposes of this document.

Authorization:

Denial of Service:

Root Exploit:

Root User:

Copyright 2005 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice and is provided as is without warranty of any kind. The warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 10/04

11