Sunteți pe pagina 1din 224

PATROL Security

User Guide

Supporting
PATROL Central Console 7.5

February 28, 2005

Contacting BMC Software


You can access the BMC Software website at http://www.bmc.com. From this website, you can obtain information about the company, its products, corporate offices, special events, and career opportunities.

United States and Canada


Address BMC SOFTWARE INC 2101 CITYWEST BLVD HOUSTON TX 77042-2827 USA Telephone 713 918 8800 or 800 841 2031 Fax 713 918 8000

Outside United States and Canada


Telephone (01) 713 918 8800 Fax (01) 713 918 8000

Copyright 2005 BMC Software, Inc., as an unpublished work. All rights reserved. BMC Software, the BMC Software logos, and all other BMC Software product or service names are registered trademarks or trademarks of BMC Software, Inc. IBM is a registered trademark of International Business Machines Corporation. All other trademarks belong to their respective companies. BMC Software considers information included in this documentation to be proprietary and confidential. Your use of this information is subject to the terms and conditions of the applicable End User License Agreement for the product and the proprietary and restricted rights notices included in this documentation.

Restricted rights legend


U.S. Government Restricted Rights to Computer Software. UNPUBLISHED -- RIGHTS RESERVED UNDER THE COPYRIGHT LAWS OF THE UNITED STATES. Use, duplication, or disclosure of any data and computer software by the U.S. Government is subject to restrictions, as applicable, set forth in FAR Section 52.227-14, DFARS 252.227-7013, DFARS 252.227-7014, DFARS 252.227-7015, and DFARS 252.227-7025, as amended from time to time. Contractor/Manufacturer is BMC SOFTWARE INC, 2101 CITYWEST BLVD, HOUSTON TX 77042-2827, USA. Any contract notices should be sent to this address.

Customer support
You can obtain technical support by using the Support page on the BMC Software website or by contacting Customer Support by telephone or e-mail. To expedite your inquiry, please see Before Contacting BMC Software.

Support website
You can obtain technical support from BMC Software 24 hours a day, 7 days a week at http://www.bmc.com/support_home. From this website, you can
I I I I I I I

read overviews about support services and programs that BMC Software offers find the most current information about BMC Software products search a database for problems similar to yours and possible solutions order or download product documentation report a problem or ask a question subscribe to receive e-mail notices when new product versions are released find worldwide BMC Software support center locations and contact information, including e-mail addresses, fax numbers, and telephone numbers

Support by telephone or e-mail


In the United States and Canada, if you need technical support and do not have access to the web, call 800 537 1813 or send an e-mail message to support@bmc.com. Outside the United States and Canada, contact your local support center for assistance.

Before contacting BMC Software


Before you contact BMC Software, have the following information available so that Customer Support can begin working on your problem immediately:
I

product information product name product version (release number) license number and password (trial or permanent)

operating system and environment information machine type operating system type, version, and service pack or other maintenance level such as PUT or PTF system hardware configuration serial numbers related software (database, application, and communication) including type, version, and service pack or maintenance level

I I I

sequence of events leading to the problem commands and options that you used messages received (and the time and date that you received them) product error messages messages from the operating system, such as file system full messages from related software

PATROL Security User Guide

Contents
Chapter 1 Introduction 15 16 16 16 17 18 20 21 21 22 23 24 24 25 25 25 26 26 27 28 29 29 30 30 31 32 34 34 35 36 36 37 37 37 38 38 38 39
5

Overview of Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Protection Provided by PATROL Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interoperability of Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security versus Usability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Password Requirements for Levels 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of Costs and Benefits of Various Security Levels . . . . . . . . . . . . . . . . . . PATROL Security Installation Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Startup Modes: Unattended and Attended . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unattended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Attended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations For Choosing a Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mode Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Startup Modes for PATROL Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keys, Key Databases, and Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Communications-Level Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anonymous Communications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Authenticated Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SSL Communications Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Default Key Databases and Certificate Authorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . PATROL Knowledge Module Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 2 Planning

Setting Up and Configuring Security Content. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of the Setup and Configuration Process . . . . . . . . . . . . . . . . . . . . . . . . . Preparing to Install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Security Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents

Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Setup Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Maintaining Security Content and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Maintenance and Management Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Verifying Security Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Test Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Performing Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Troubleshooting Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Chapter 3 Installation 47

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Installation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Over-the-Top Installation and Policy Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Compatibility with the Previous Version of PATROL Security . . . . . . . . . . . . . . . 49 Customizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Selecting the Level of Security and Overwriting of Existing Security . . . . . . . . . . 53 Selecting Advanced Security Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Selecting Connection Type for Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Location and Storage of Security Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Directories, Files Types, and Registry Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Chapter 4 Keys and Certificates 59

Authentication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Types of Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Concepts and Components of Authentication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Default Key Databases and Certificate Authorities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Workflow for Configuring PKI-Based Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Utilities for Key Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 sslcmd Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 bmckeycli Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Management of Keys and Key Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Key Databases Shipped with PATROL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Creating an SSL Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Changing the Password for the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Transferring a Keyfile.kdb from Unix to Windows Environment . . . . . . . . . . . . . 71 Generating Public and Private Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Listing PublicPrivate Key Pairs in the Key Database . . . . . . . . . . . . . . . . . . . . . . . 73 Changing the Label of a Key Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Deleting Private and Public Key Pairs and Certificates . . . . . . . . . . . . . . . . . . . . . . 75 Exporting Key Pairs and Assigned Certificates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Importing Key Pairs and Assigned Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Management of User Credential (Labeled Password) . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Purpose and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Adding User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Listing User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6 PATROL Security User Guide

Deleting User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of Certificate Authority. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Establishing a CA Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing a CA Certificate in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verifying Trusted Root Authority Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Field Information for CA Certificates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting Trusted Root Authority Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of User Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Certificate Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a Certificate Signing Request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing a User Certificate in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . Listing Certificates in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting a Certificate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of Certificate Revocation Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Description of a Certificate Revocation List (CRL). . . . . . . . . . . . . . . . . . . . . . . . . . Missing Certificate Revocation List Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acquiring a Certificate Revocation List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing a Certificate Revocation List. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 5 Security Policies

82 83 83 85 86 87 88 89 89 89 92 93 93 94 94 95 95 95 97

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Site Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Application Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Policy Roles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Policy Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Inheritance and Precedence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 PATROL Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Format and Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Microsoft Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Utilities for Policy Testing and Password Encryption . . . . . . . . . . . . . . . . . . . . . . . . . 111 esstool Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 plc_password Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 bmcryptpw Utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 signFile and verifyFile Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Policy and Role Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Creating a Security Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Viewing the Policies and Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Viewing Version Information for Security Modules . . . . . . . . . . . . . . . . . . . . . . . 117 Authentication and Encryption Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Specifying an Authentication Provider and Service . . . . . . . . . . . . . . . . . . . . . . . 119 Testing Authentication Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Selecting an Encryption Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Listing the Encryption Algorithms Supported by the Encryption Module . . . . 130 Testing Encryption Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Contents 7

Key Database and Password Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Designating a Key Database for an Applications Role . . . . . . . . . . . . . . . . . . . . . 133 Setting the Attended or Unattended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Adding or Editing a Password Stored in a Policy. . . . . . . . . . . . . . . . . . . . . . . . . . 135 Encrypting a Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Signer and Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Operation of Signing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Operation of Verifying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Testing Digital Signing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Testing the Verification of a Digital Signature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Client-Server Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Testing a Secure TCP/IP Channel for the Client and Server. . . . . . . . . . . . . . . . . 145 Policy Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 PATROL Security versus Extended Security System . . . . . . . . . . . . . . . . . . . . . . . 151 ESS 3.0.00 and ESS 3.0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Migration Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Migrate or Overwrite. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Migration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Chapter 6 Configuration Files 155

PATROL Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 patrol.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 config.default . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Working with Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Configuring the SSL access File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Operating System and Application-Specific Configurations. . . . . . . . . . . . . . . . . . . . 168 Configuring the dlls.conf for PATROL for Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 Using PATROL Event Manager Applications with PATROL Security . . . . . . . . 170 Appendix A Changing the Security Level 171

Changing the Security Level for the Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Appendix B Troubleshooting 177

Issues and Workarounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Character @ Interpreted as Kill Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Attempt to Generate a Key Results in Extended Error Message . . . . . . . . . . . . . 179 Defaults to Security Level 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Missing bindir Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Missing securitydir Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Password Prompter Canceled Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Password Attribute Requires 2 Fields Error Message . . . . . . . . . . . . . . . . . . . . . . 182 Key File Cannot Be Reached Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Decrypting Stored Password Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Identity Missing from Key Database Error Message . . . . . . . . . . . . . . . . . . . . . . . 183 Unexpected Password Prompt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

PATROL Security User Guide

Installation Fails. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Uninstallation Fails to Remove Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Key Database Will Not Open With Correct Password . . . . . . . . . . . . . . . . . . . . . 185 No Key for Negotiated Cipher Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Cannot Install a Certificate into a Key Database. . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Cannot Install a CRL into a Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Windows CA Rejects a CSR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Password Not Configured . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Password Prompt Does Not Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Typed Password Does Not Appear in Password Dialog Box. . . . . . . . . . . . . . . . 188 Password Dialog Prompt Does Not Appear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 PATROL KM for Microsoft Cluster Server Does Not Support Attended Mode at Level 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Cannot Find Shared Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Discovery Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Password for 64-bit Key Files Is Not Validated . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Password Dialog Prompt Does Not Appear When Running at Level 4 . . . . . . . 191 Error Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Invalid Policy Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Invalid Policy Keyfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Incorrect Encrypted Password Used During Security Bootstrap . . . . . . . . . . . . . 193 Invalid Policy Identity Field (Non-Existing Key) . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Mutual Authentication Nominal Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Missing Key On Level 4 Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Missing Trusted Root (client). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Missing Certificate (Server) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Expired Certificate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Appendix C Glossary Index Valid Country Codes 203 211 217

Contents

10

PATROL Security User Guide

Figures
Unattended Mode Settings in Policy File on Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Unattended Mode Settings in Registry Key on Windows . . . . . . . . . . . . . . . . . . . . . . 27 Select Level of Security Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Select Advanced Level of Security Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 sslcmd Example keyfile.kdb not found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Example of a CRL Stored in a Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Sample Site Policy File (site.plc) for Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Edit String Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Sample Site Policy Registry Key for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Regedit View of Site Policy Registry Key for Windows . . . . . . . . . . . . . . . . . . . . . . . 110 esstool policy Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 esstool policy Example on Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 esstool policy Example Output on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 esstool query Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 esstool query Result Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 pam.conf Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Authenticator Role of Site Policy on Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Example of Reference to pam_krb5 in pam.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 krb5.conf Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 esstool authenticator Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 esstool authentication Results Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . 127 Sample List of Cipher Types for bmcpwk.dll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 esstool encryptor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 esstool encryptor Example of Encryption Command and Output . . . . . . . . . . . . . . 132 esstool encryptor Example of Decryption Command and Output . . . . . . . . . . . . . . 132 plc_password Example Setting Mode to Unattended . . . . . . . . . . . . . . . . . . . . . . . . . 134 plc_password Example Setting Mode to Unattended . . . . . . . . . . . . . . . . . . . . . . . . . 135 plc_password Example of Policy File Contents on Unix . . . . . . . . . . . . . . . . . . . . . . 137 bmcryptpw Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 bmcryptpw Results Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 bmcryptpw Test Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 bmcrypt Test Results Example on Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 signFile Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 signFile Example of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 verifyFile Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 verifyFile Example of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 esstool server Example Command on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 esstool server Example Startup Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 esstool client Example Command on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 esstool client Example Startup Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Figures 11

esstool server Example of Message Received from esstool client . . . . . . . . . . . . . . . 150 Result of the Migration of the pamservice Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . 152 patrol.conf File Example of the ESI Section on Unix . . . . . . . . . . . . . . . . . . . . . . . . . . 159 patrol.conf Example of the ESI Section on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . 160 access File Example Restricting Access to Two Users . . . . . . . . . . . . . . . . . . . . . . . . . 166 access File Example Allowing Access to a Group and Denying Access to an Individual User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 ESI Variable Configured for PATROL Event Manager Applications . . . . . . . . . . . . 170 ESI Library Location for PATROL Event Manager Applications . . . . . . . . . . . . . . . . 170 Registry Keys for PATROL Agent and PATROL Security . . . . . . . . . . . . . . . . . . . . . 170 p7_change_security_level Example on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 p7_change_security_level Example on Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 p7_change_security_level Script Sample Log on Windows . . . . . . . . . . . . . . . . . . . . 175 Generate a Key Extended Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Password Prompter Canceled Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Password Attribute Requires 2 Fields Error Message . . . . . . . . . . . . . . . . . . . . . . . . . 182 Identity Missing from Key Database Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Key Database Will Not Open With Correct Password Error Message . . . . . . . . . . . 185 Invalid Policy Password Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Invalid Policy Keyfile Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Incorrect Encrypted Password Used During Security Bootstrap Error Message . . 193 Invalid Policy Identity Field (Non-existing Key) Error Message, Client Log . . . . . . 193 Invalid Policy Identity Field (Non-existing Key) Error Message, Server Log . . . . . 194 Mutual Authentication Nominal Case Error Message, Client Log . . . . . . . . . . . . . . 195 Mutual Authentication Nominal Case Error Message, Server Log . . . . . . . . . . . . . . 195 Missing Key on Level 4 Client Error Message, Client Log . . . . . . . . . . . . . . . . . . . . . 196 Missing Key on Level 4 Client Error Message, Server Log . . . . . . . . . . . . . . . . . . . . . 197 Missing Trusted Root (client) Error Message, Client Log . . . . . . . . . . . . . . . . . . . . . . 198 Missing Trusted Root (client) Error Message, Server Log . . . . . . . . . . . . . . . . . . . . . . 199 Missing Certificate, Client Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Missing Certificate, Server Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Expired Certificate, Client Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Expired Certificate, Server Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

12

PATROL Security User Guide

Tables
Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 PATROL 3.x Security Level Interoperability Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Usability versus Security for the Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Password Usage in PATROL Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Default Modes (Unattended and Attended) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Anonymous Communications and Security Levels 0, 1, and 2 . . . . . . . . . . . . . . . . . . 31 Authenticated Communications and Security Levels 3 and 4 . . . . . . . . . . . . . . . . . . . 31 Overview of Preinstallation Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Overview of Installation Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Overview of Configuration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Overview of Maintenance Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Overview of Testing Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Policy installation location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Installation location for versions of PATROL Security . . . . . . . . . . . . . . . . . . . . . . . . . 50 Installation Paths of Security Files and Registry Keys . . . . . . . . . . . . . . . . . . . . . . . . . 57 Default Certificate Expiration Dates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Order of Configuration Tasks for Authentication Security . . . . . . . . . . . . . . . . . . . . . 66 sslcmd Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 bmckeycli Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Distinguished Name Prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Policy Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 PATROL Applications and Their Corresponding Application Policy Names . . . . 101 Policy Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Policy Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Order of Precedence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 esstool Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 plc_password Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 bmcryptpw Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 signFile and verifyFile Installation Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 esstool policy Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Security Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Location of the PAM Configuration File by Operating System . . . . . . . . . . . . . . . . . 121 IBM Updates for AIX 5.2 or Later . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 esstool authenticator Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Supported Encryption Algorithms and Their Cipher Values . . . . . . . . . . . . . . . . . . 128 esstool encryptor Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 plc_password Utility Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 bmcryptpw Utility Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 signFile Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 verifyFile Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Tables 13

esstool server Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 esstool client Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Installation and Migration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 PATROL Configuration Files That Contain Security Information . . . . . . . . . . . . . . . 156 Location of patrol.conf File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Security Configuration Data of patrol.conf File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 ESI Variables in patrol.conf File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Location of config.default File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Agent and Console Features in config.default File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 Location of access File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Configuration Data in access File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Location of the dlls.conf File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 p7_change_security_level Script Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 p7_change_security_level Script Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Valid Country Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

14

PATROL Security User Guide

Chapter

1
16 16 16 17 18 20 21 21 22 23 24 24 25 25 25 26 26 27 28 29 29 30 30 31 32 34 34

Introduction
This chapter provides an overview of security concepts that will help you understand the issues involved in securing your PATROL environment. This chapter contains the following topics: Overview of Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Protection Provided by PATROL Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interoperability of Security Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security versus Usability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Password Requirements for Levels 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of Costs and Benefits of Various Security Levels . . . . . . . . . . . . . . . . . . PATROL Security Installation Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Startup Modes: Unattended and Attended . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unattended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Attended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations For Choosing a Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mode Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Startup Modes for PATROL Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keys, Key Databases, and Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Communications-Level Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anonymous Communications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Authenticated Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SSL Communications Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Default Key Databases and Certificate Authorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . PATROL Knowledge Module Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 1

Introduction

15

Overview of Security

Overview of Security
All systems in any environment are susceptible to potentially harmful events if the proper security is not in place. Both internal and external users can instigate critical events, either maliciously or unintentionally. Careful implementation of security controls and restrictions minimizes the occurrences of security violations. To protect against these violations, PATROL Security uses a combination of
I

security roles to categorize the functions of an application and the security challenges that each function presents security levels to determine the amount of security applied to the applications as they operate in different roles and interact with other applications

Security Roles
PATROL Security associates the potential security violations with the types of applications that interact within the PATROL environment. To address these concerns, PATROL provides security roles. The PATROL Security roles are as follows:
I I I I I I I

authenticator encryptor keystore client server signer verifier

When properly configured, these roles can address any security problem posed by applications fulfilling these roles. For more information about security roles, see Policy Roles on page 101.

Security Levels
PATROL employs a graduated security system that is divided into security levels. This approach enables you to configure the security of your PATROL environment to your security needs and usability requirements. You can install the level of security that you want and configure the chosen security level to your specifications.

16

PATROL Security User Guide

Protection Provided by PATROL Security

For more information about security roles, see Levels of Security on page 18.

NOTE
Throughout this document, discussion of PATROL Security refers to the security employed by PATROL to address those security issues introduced by PATROL components; PATROL Security does not provide comprehensive network security.

Protection Provided by PATROL Security


Protecting data within PATROL means that the data delivered by PATROL users is private and secure. After you install PATROL, the implementation of security within PATROL delivers the following protection:
I

Communications privacy and encryptionPATROL Security provides message privacy for communications between PATROL components. AuthenticationAdditional security is provided for communications between PATROL components by verifying the identity of each component. Password privacy and configurationWith password encryption inherent in PATROL, you can control and change encryption keys, thus helping to prevent unwanted access to your PATROL environment. You can change your password to prevent infiltration of your accounts and to secure your personal environment and settings. Digital signingTo ensure the integrity of data that you create in PATROL, PATROL implements digital signing and verification tools. Signature verification proves that the signer's certificate is valid and was granted by a trusted certificate authority (CA). After a signer's certificate has been verified, the certificate's public key is used for signature verification. The method of digital signature is only as secure as the signing keys that that CA uses. Digital signing maintains the integrity of product-related files by helping to ensure that the PATROL Knowledge Module (KM) file or PATROL configuration file is original and unaltered. User privilegesPATROL Security administrators can define an account or group and can assign specific permissions and access rights to that account or group. As an administrator, you can add privileges to or remove them from a user or group of users. Access control lists (ACLs)To stop unwanted connections to a server that is running the PATROL Agent, the communication to and from the agents and consoles might require restricted access. PATROL ACLs specify which user names are granted access, from which hosts access will be granted, and what type of

Chapter 1

Introduction

17

Levels of Security

access will be granted to the PATROL Agent. PATROL Security administrators can manipulate the access control for any object in the PATROL namespace. Administrators can grant or deny access on a hierarchical basis for any object and for any user or group. In addition to ACLs maintained for PATROL consoles and agents, security level 4 also maintains its own set of ACLs in the access file. For more information about the file, see access on page 164.
I

Impersonation controlTo facilitate seamless functioning of PATROL across multiple hosts and security domains, PATROL incorporates an impersonation table, which a PATROL Security administrator can use to specify user names and passwords for connecting to a new host.

For more specific information about how to use PATROL to modify or create these security functions, see the PATROL Security information in the PATROL Agent Reference Manual.

Levels of Security
Each security level is defined by a specific set of configuration variables residing in configuration files (as described in Chapter 5, Security Policies). Table 1 defines the five principal levels of security policy. Table 1
Level 0 (basic)

Security Levels (Part 1 of 2)


Description Level 0 (basic security) is the default level of security. Basic security provides neither cryptographic protection of network traffic (messages traverse the network in plain text) nor authentication of peers (the user providing the user name and password is not required to prove that he or she is the originator or owner of that information). This level offers the lowest level of security in exchange for greater ease of use. This level does not provide communication security or authentication security. Access control lists (ACLs) are minimized in favor of usability and performance.

Level 1 provides communications privacy using anonymous Diffie-Hellman public keys exchange and triple data encryption standards-cipher block chain (DES-CBC) encryption to ensure privacy. It does not require that the user perform additional configuration or store a secret key. Additional cryptographic services include communication channel privacy, data integrity, and audit logging. As with basic security, PATROL ACLs are relaxed. Level 1 provides communication security, but it does not provide SSL authentication of the console or agent.

18

PATROL Security User Guide

Levels of Security

Table 1
Level 2

Security Levels (Part 2 of 2)


Description Levels 2 and higher use Secure Socket Layer (SSL) protocol, the essential ingredient in providing high-level communications security. The agent provides a certificate to the console as part of the SSL handshake. The console accepts this certificate unconditionallywhether or not it can verify the certificate to a trusted root authority. Because there is no SSL authentication of the client (console) or the server (agent), the actual degree of security is no greater than that of level 1. The difference lies in the use of the SSL protocol for encryption to ensure message privacy, making communications more secure. Level 2 defaults to unattended mode, allowing the agent to restart without requiring manual password entry.

Level 3 uses SSL protocol for message privacy and authentication on the server (agent side) only. Level 3 security assures that the server is not an imposter by requiring the agent to provide a certificate to the client (console), which must then authenticate the servers certificate to a trusted root authority. The certificate of the trusted root authority must be present in the clients encrypted database. The client opens this database by supplying a password. Level 3 defaults to unattended mode, allowing the agent to restart without requiring manual password entry, but can be configured to run in attended modea. Level 3 provides communication security and authentication of the server (agent) to verify that the client (console) is receiving valid data from a legitimate server.

Level 4 uses SSL for message privacy and authentication of both the server (agent) and the client (console). The server and the client provide each other a certificate that proves their respective identities. Each certificate must be verified to a trusted root authority present in the server and client databases. This level of security has the most configuration requirements and provides the most rigorous form of security available. Level 4 defaults to unattended mode, allowing the agent to restart without requiring manual password entry, but can be configured to run in attended modea. Level 4 provides communication security and both client (console) and server (agent) authentication.

See PATROL Security Installation Options on page 23 and Startup Modes: Unattended and Attended on page 25.

Chapter 1

Introduction

19

Interoperability of Security Levels

Interoperability of Security Levels


Interoperability of PATROL applications varies based on the PATROL architecture and the PATROL Security level implemented on each component.

PATROL 7.3.x or Earlier


Security level interoperability is restricted to the same level. For example, clients running at level 2 can interact only with servers running at level 2.

PATROL 7.3.x or Later and PATROL 3.x


Table 2 describes the interoperability between security levels in the PATROL 3.x architecture. Table 2 PATROL 3.x Security Level Interoperability Matrix
Server Level 2 Anonymous SSL Trusted Root CA, Server Key no Level 3 Server Authentication Trusted Root CA, Server Key, certificate no Level 4 Mutual Authentication Trusted Root CA, Server Key, certificate, ACL no

Level 1 Anonymous Diffie-Hellman Level 1 Anonymous Diffie-Hellman Level 2 Anonymous SSL Trusted Root CA, Server Key Level 3 Server Authentication Trusted Root CA, Server Key, certificate Level 4 Mutual Authentication Trusted Root CA, Server Key, certificate, ACL
a

yes

no

yes

yes

yesa

Client

no

yes

yes

yesa

no

yes

yes

yes

Interoperabililty in this instance assumes that a key database has been set up for the client so that the client supports mutual authentication.

20

PATROL Security User Guide

Security versus Usability

Security versus Usability


The security policies allow you to install the least secure (basic security) up to the most secure (level 4) features of PATROL Security, depending on your system needs and how much complexity you can allow in securing your systems. High levels of security require additional configuration of the communicating components (the agent and console), are more difficult to use, and may affect network performance. Lower levels of security are much easier to configure and use, but provide less security. Before installing PATROL, decide how much of a trade-off you are willing to make between security and usability by examining the differences among the policy levels, as described in Levels of Security on page 18. Basic security provides a minimal level of security with no configuration requirements. At the highest security level (4), all communicating components must authenticate with each other, and key databases must validate connection requests.

Password Requirements for Levels 3 and 4


Both levels 3 and 4 require a password to open the encrypted key and certificate database. The key database stores private and public key pairs, certificates, and trusted roots required for SSL operations. Triple DES-CBC cipher with a userprovided password protects the database. A key material file computes the password encryption key. The key material file is a file that is provided and protected by the user and can be virtually any binary file. To ensure optimal security protection, users can carry this on their person, such as on a floppy disk. Chapters 2 and 3 provide detailed information about password encryption and policy definitions. To allow greater usability at levels 2, 3, and 4 and support autonomous agent operations, PATROL Security gives you the option to store the encrypted password in the policy. This feature enables an application to start up without a user in attendance and is referred to as running in unattended mode. For more information, see Startup Modes: Unattended and Attended on page 25.

Chapter 1

Introduction

21

Summary of Costs and Benefits of Various Security Levels

Summary of Costs and Benefits of Various Security Levels


Table 3 summarizes usability issues and configuration requirements for each of the policy levels. Table 3 Usability versus Security for the Security Levels
Description
I

Security Level Basic security

does not introduce any overhead in performance or configuration does not provide any additional security beyond basic PATROL Security features (see Protection Provided by PATROL Security on page 17) introduces no overhead in usability or maintenance provides message confidentiality and integrity by using Diffie-Hellman key exchange and 3DES encryption introduces a minimal amount of overhead associated with performance and disk space preconfigured with demo keys out of the box; they have expiry dates as indicated in Table 16 on page 65 can use SSL but it requires a set of CA and configuration; use your own CA and certificates preconfigured upon installation and does not require any additional configuration efforts increases performance overhead and maintenance costs because of the use of SSL and X.509 certificates introduces SSL-based authentication provides substantially increased traffic security and general data integrity requires more configuration due to the requirement of a certificate for each authenticating agent. provides mutual authentication, which requires a certificate for each authenticating agent and console can use SSL but requires you to acquire a set of CA and configuration; use your own CA and certificates requires the most configuration due to the requirement of a certificate for each authenticating agent and console

Level 1

I I

Level 2

Level 3

I I

Level 4

22

PATROL Security User Guide

PATROL Security Installation Options

PATROL Security Installation Options


Installing security levels 3 and 4 affects PATROL operations in the following ways:
I

Default operation for the PATROL Console version 3.x is in operator mode. If you choose levels 3 or 4 during installation, a screen will prompt you to select TCP or UDP or both for the Network connection allowed option. (For details about installation, see Selecting the Level of Security and Overwriting of Existing Security on page 53.) If you select the TCP option only, traffic defaults to TCP instead of UDP on the PATROL Agent. To use the pconfig utility at levels 3 and 4, you must specify pconfig ...+tcp to connect to an agent. (If you selected the UDP option, pconfig defaults to UDP). In order to use xpconfig, you can connect to the agent only by selecting the TCP connection mode.

Chapter 1

Introduction

23

Passwords

Passwords
Passwords provide authentication security. A user proves its identity by supplying a password that only that user should know. PATROL Security implements this type of authentication to prove the identity of users trying to establish communications from one PATROL application to another. It also uses passwords to protect some of its own components such as key databases.

Usage
Table 4 provides a list of the different components and usages of passwords and reference the section in this manual where you can learn more about managing passwords in that context. Table 4
Protected Component PATROL applications local key database remote key database

Password Usage in PATROL Security


Location in key database attached to key database policy, password attribute Usage access PATROL application Information Adding User Credentials (Labeled Passwords) on page 80 Changing the Password for the Key Database on page 71 Adding or Editing a Password Stored in a Policy on page 135

access key database access key databases that supports the role under which an application is operating

24

PATROL Security User Guide

Utilities

Utilities
PATROL Security provides several utilities that you can use to encrypt passwords and distribute them. They include
I

plc_passwordenables you to encrypt a password using a key material file and

insert the password in a policy file; see plc_password Utility on page 112 bmcryptpwenables you to encrypt and verify a password using a key material file; see bmcryptpw Utility on page 113 sslcmdenables you to manage the following types of passwords; user passwords (referred to as Labeled Passwords) stored in a key database by PATROL applications; you can apply a label to these user passwords to help you identify and manage them key database passwords, which are required to manage key databases; see sslcmd Utility on page 67

Startup Modes: Unattended and Attended


The distinction between attended and unattended modes refers to how much user intervention an application requires to startup and run. The start-up mode is determined by the presence or absence of an encrypted password in the password parameter of either the site or application policy. The startup mode affects applications running at levels 2, 3, and 4. Applications with security level 0 and 1 start up in unattended mode exclusively.

Unattended Mode
In unattended mode, a password entry is present in the policy file or registry key depending upon the operating system. You launch the application, the application retrieves the encrypted password from attribute in the policy, verifies it against the password to access the key database, and if the password is correct, then the application starts up and runs.

Chapter 1

Introduction

25

Attended Mode

Attended Mode
In attended mode, password information is missing from the policy. You launch the application, the application attempts to retrieve the encrypted password from the policy. When it does not find the password, it presents a user name and password dialog box to the user. The user types in the information and submits it to the application. The application verifies it, and if the password is correct, then the application starts up and runs.

Considerations For Choosing a Mode


When deciding whether to employ attended or unattended mode, consider the following factors.

Security versus Usability


Attended mode is more secure than unattended operation using an encrypted password; however, attended mode requires you to enter a password every time you restart the application. For example, if you ran the PATROL Agent in attended mode, you or another system administrator would have to physically attend the restart of an agent so that you could type in the account name and password.

Physical Security
The degree of physical security in your network environment is relevant deciding whether to run a server in unattended mode. A server that is not physically secured from unauthorized users is inherently more vulnerable to unauthorized access if it is running in unattended mode.

Virtual Security
The degree of virtual security in your network environment is also relevant when deciding whether to run a server in unattended mode. Storing a password on a computer makes it vulnerable to discovery by intruders that gain ownership of a service. To secure your computer, shut down unnecessary services such as inetd, telnet, netbios, ftp, and other similar services that can be exploited by intruders.

26

PATROL Security User Guide

Mode Settings

Mode Settings
Attended and unattended mode settings depend upon the presence or absence of the following policy parameters in the policy file for Unix or registry key for Windows. These parameters must be specified as described in Setting the Attended or Unattended Mode on page 134.
I I

password, key material file key database

Figure 1 illustrates the attribute settings for unattended mode in a Unix environment. Figure 1 Unattended Mode Settings in Policy File on Unix

[client] logfile = console_client.log password = 82a153ecffbc901bb73fefe0c23c84b8b76d422a1e6ed83d, /opt/Tuscany/JA/011016/common/security/keys/tree.bin keyfile = /home/patrol/patrol.kdb

Figure 2 illustrates the attribute settings for unattended mode in a Windows environment. Figure 2 Unattended Mode Settings in Registry Key on Windows

Changing Between Unattended and Attended Mode


To set a component to run in unattended or attended mode, use the plc_password utility as described in plc_password Utility on page 112.

Chapter 1

Introduction

27

Startup Modes for PATROL Components

Startup Modes for PATROL Components


Table 3 shows the default mode for various PATROL components. Table 5 Default Modes (Unattended and Attended)
Default Mode (Security levels 3 and 4) attended attended unattended unattended unattended unattended

PATROL Component PATROL 3.x console PATROL Central Microsoft Windows Edition PATROL Central Web Edition PATROL Agent 3.5 and 3.4 PATROL Console Server PATROL Event Manager 3.5 PATROL CLI PATROLLink pconfig utility xpconfig utility wpconfig utility client applications

NOTE
For level 4 security, the client section in the site policy does not contain a password. Therefore, if the application policy (client, server, and so forth) does not exist or cannot be loaded, the site policy will be used and the mode will default to attended.

28

PATROL Security User Guide

Keys, Key Databases, and Certificates

Keys, Key Databases, and Certificates


Keys, key databases, and certificates are used to protect data as it travels from one point in a network to another. They are also used to prove the origin of that data. As the name implies, keys are used to lock or encrypt data. The process of locking data ensures that it is safe from being tampered with, either by being altered or being read. Key databases (also referred to as keystores) store keys that are used to unlock or decrypt data. Certificates are documents that confirm the identity of a user and they are issued by a Certificate Authority (CA, also referred to as trusted root authority). The CA certificate is installed into the key database and used to verify a user certificate. The user certificates are linked to private and public key pairs and used to digitally sign data and to verify digitally signed data. Certificate Revocation Lists (CRL) are periodically retrieved from the CA and installed into the key database. If a user certificate appears on the CRL, the certificate is compromised and any requests with which it is associated are denied. For more information about keys, key databases, and certificates, see Chapter 4, Keys and Certificates.

Policies
Security policies contain setup and configuration information for implementation of PATROL Security, which addresses potential security violations. A security policy consists of roles and attributes. Roles categorize applications according to their functions and the potential security threats that they pose. Attributes define the security behavior. When roles are properly configured through attributes, the roles can address any security problem posed by applications that fulfill these roles. Policy roles link PATROL applications to key databases, which provide them with the means for encryption and authentication. Each roles can reference a different key database or all roles can reference the same key database. For more information about keys, key databases, and certificates, see Chapter 5, Security Policies.

Chapter 1

Introduction

29

Communications-Level Security

Communications-Level Security
The term security covers a wide territory, even in the restricted domain of computer networks. Conventional logon passwords, for example, ensure that only authorized users can access computing resources. Just as access security protects access to computing resources, communications security protects information that is transmitted over a communications channel. Communications security protects such information only in the context of a transaction between communicating parties. After that information is received, it moves from being a transaction requiring communications security into some other format (such as data stored on a disk), where it must be protected by other forms of security. This section covers how security is implemented at the communications level. It discusses in detail the ways in which the transactions between PATROL components are secured so that message privacy is secured and the communicating components are authenticated. These two aspects of security (privacy and integrity) are addressed by communications-level security. Verification of the rights and privileges of communicating components (authorization) is addressed by user administration. The level of security that you install determines whether or not the communicating components are authenticated with SSL communications security. Security levels 0, 1, and 2 do not employ SSL to authenticate the communicating components. Levels 3 and 4 do provide SSL-authentication: level 3 authenticates the client to the server, and level 4 authenticates both the client and the server to each other.

Anonymous Communications
Anonymous communications are exactly what they claim: communications between two applications that have no means of verifying that the other application is what or who it says it is. Anonymous communications are vulnerable to impersonation attacks. Levels 1 and 2 encrypt these communications, which prevents eavesdropping. Level 0 does not encrypt communications, sending clear text messages back and forth. Table 6 describes in detail the differences in security levels with regard to anonymous communications. For more information about differences between security levels, see Levels of Security on page 18.

30

PATROL Security User Guide

Authenticated Communications

Table 6

Anonymous Communications and Security Levels 0, 1, and 2

Security Level Description 0 1 Security level 0 (basic security, the default level) does not employ either DiffieHellman or SSL for message privacy. Security level 1 is based on anonymous Diffie-Hellman public key exchange, which provides a high degree of privacy protection but no authentication. DiffieHellman key exchange does not require any configuration and thus has no configuration cost and no configuration vulnerabilities. This protocol is a desirable choice for environments where only message privacy is required. Security level 2 employs SSL for message privacy, but does not use SSL to authenticate the client or the server. SSL is considered more secure because it is more difficult to decrypt.

Authenticated Communications
Authenticated communications are communications between two applications, that can verify the authenticity of the other. At level 3, the client verifies that the server application is what it claims to be. At level 4, both the client and server verify each others authenticity. Both levels provide for SSL encryption of communications to prevent eavesdropping. Table 7 describes in detail the differences in security levels with regard to authenticated communications. Table 7 Authenticated Communications and Security Levels 3 and 4

Security Level Description 3 Security level 3 employs SSL communications security to authenticate the server to the client (for example, the agent to the console). Authentication requires that a trusted third party, the certificate authority (CA), verify the servers certificate. The integrity of this authentication process relies on the integrity of the key database that stores the trusted CA certificate. Security level 4 provides mutual client-server SSL authentication. This level requires the proper maintenance of the authentication credentials on each of the communicating peers (clients and servers).

Chapter 1

Introduction

31

SSL Communications Security

SSL Communications Security


Security levels 2 and higher use SSL communications security for
I I

message privacy (levels 2, 3, and 4) authentication (levels 3 and 4)

To provide communication security at level 2 or higher, you must set up an SSL key database for each user or PATROL component that presents or verifies certificates. To maintain communications security, an SSL key database contains
I I I I

public and private cryptographic keys trusted authority certificates user certificates certificate revocation lists

PATROL components require a naming convention for both the key database filename and for the SSL identity that the database contains. To operate at levels 2, 3, and 4, the agent requires a key database named server.kdb, which must also contain an SSL identity named server. This identity provides the agent with its own keypair (one public key and one private key) and corresponding certificate. The bmcuser.kdb file contains default keys and certificates (security content) for the agent and client with the default user name bmcuser. This default configuration enables PATROL Security to run without further configuration at level 3, but is not secure because the default is publicly available. At level 3, the agent sends its certificate to the console for validation. The certificate contains the name of an issuer (the trusted root authority); the console searches for this name in the console database in order to verify the agents authenticity. For level 4, the database must also contain an SSL identity key for the user name (for example, user1). This design permits a separate key database and password for each user. The console must provide its certificate so that the agent can verify the authenticity of the console. At this level, the console database must contain the signed certificate for the user while the agent database must also contain the certificate for the signing authority that signed the console's certificate. To operate at security level 4, the SSL communications console requires a key database corresponding to the user name of the person starting the console. For example, user1 requires a database named user1.kdb.

32

PATROL Security User Guide

SSL Communications Security

NOTE
To operate at level 4, server.kdb must contain certificates for all signing authorities that have signed the certificates of valid users. If multiple signing authorities sign user certificates, server.kdb must contain certificates for each of these signing authorities in order to grant all users access to the agent.

Chapter 1

Introduction

33

Default Key Databases and Certificate Authorities

Default Key Databases and Certificate Authorities


WARNING
This product is delivered with default keys and certificates that are not unique to you. BMC Software highly recommends that you promptly change these default keys and certificates to ones that are unique. If you do not make these changes, there is a higher risk that any third party who gains physical access to your network, or to the data that you send over the internet, might have a better opportunity to use these default keys and certificates to gain unauthorized access to your system. BMC Software is not responsible for any damage or liability associated with your use of default keys and certificates.

Default keys and certificate authority (CA) certificates supplied by BMC and stored in keyfiles with .kdb extensions are provided only to demonstrate a turnkey security configuration, for purposes such as demonstrations and trial installations. Before using BMC Software products, replace the default keys and certificates with your own unique entities. For instructions, see Setting Up and Configuring Security Content on page 36 and Establishing a CA Certificate on page 83.

PATROL Knowledge Module Security


PATROL Knowledge Modules (KMs), along with configuration files, directly control the behavior of PATROL, and thus play an integral role in preserving the integrity of the PATROL user environment. Because KMs require the PATROL infrastructure to deliver their functionality, KMs already include many security components that are handled by the infrastructure, such as traffic encryption, execution of the script in the proper accounts, and auditing. However, older KMs may offer less security. Presently, all KMs use DES-encrypted storage in agent configuration. In addition, BMC Software is adding a set of basic rules for KM security that future KM development and certification processes will require.

34

PATROL Security User Guide

Chapter

2
36 36 37 37 37 38 38 38 39 39 40 41 41 42 44 44 44 45 45 45

Planning
This chapter describes the process of planning the installation and it discusses the considerations that you must make relating the actual tasks that you need to perform to set up and configure your PATROL Security environment. It references the sections in this document that describe how to perform the individual tasks. This chapter presents the following topics: Setting Up and Configuring Security Content. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of the Setup and Configuration Process . . . . . . . . . . . . . . . . . . . . . . . . . Preparing to Install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Security Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setup Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintaining Security Content and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintenance and Management Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verifying Security Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Test Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performing Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshooting Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 2

Planning

35

Setting Up and Configuring Security Content

Setting Up and Configuring Security Content


WARNING
This product is delivered with default keys and certificates that are not unique to you. BMC Software highly recommends that you promptly change these default keys and certificates to ones that are unique. If you do not make these changes, there is a higher risk that any third party who gains physical access to your network, or to the data that you send over the internet, might have a better opportunity to use these default keys and certificates to gain unauthorized access to your system. BMC Software is not responsible for any damage or liability associated with your use of default keys and certificates.

Because PATROL Security is shipped with and installs keys and certificates for demonstration purposes only (see Default Key Databases and Certificate Authorities on page 34), to create a secure environment, you must acquire your own unique keys and certificates and then modify key databases, passwords, and policies to use this unique content. This section lists all the tasks that you must perform and refers you to instructions and additional information about how to complete those tasks.

Overview of the Setup and Configuration Process


The tasks that you have to perform to set up PATROL Security depend upon the security level at which you want to run your PATROL environment and the security roles of the PATROL applications that you run and install. The tasks can be categorized and ordered as follows: 1. preparation for installation 2. installation of PATROL Security 3. configuration of PATROL Security 4. verification of PATROL Security 5. diagnosis of problems if they occur

36

PATROL Security User Guide

Preparing to Install

Preparing to Install
PATROL Security is packaged with a demonstration set of key databases, certificates and certificate authorities, policies, and passwords. While this information is extremely valuable in setting up demonstrations of PATROL Security, it is publicly available to all PATROL customers and therefore is insecure. To create a secure environment, you must identify a certificate authority from which you can acquire the certificates and trusted root certificate and certificate revocation list (CRL).

Considerations
When preparing to install PATROL Security, consider the following questions:
I

Do you want to use a third party as your Certificate Authority (CA)? If so, does the CA provide certificates in an ASCII text file in version 3 of the X.509 PEM (PrivacyEnhanced Mail) Base64 format? Do you want to manage your own CA in-house?

Tasks
Table 8 lists tasks to perform when preparing to install PATROL. Table 8
Order 1.

Overview of Preinstallation Tasks


Task Establish a Certificate Authority. Procedural Instructions Establishing a CA Certificate on page 83 Applicable Security Level all

Chapter 2

Planning

37

Installing

Installing
PATROL Security is automatically installed with such PATROL applications as the PATROL consoles, the PATROL Agent, and the Console Server. BMC Software does not provide an independent installation of PATROL Security.

Considerations
Before you install PATROL Security, consider the following questions:
I

Does PATROL Security already exist on this computer? Which version is it? Do you want to overwrite customizations to security contents such as unique keys, certificates, and modified policies?

What level of security do you want to install and will it operate with the rest of your PATROL environment?

Installation Tasks
Table 9 lists tasks to perform during the installation of PATROL. Table 9
Order 1.

Overview of Installation Tasks


Task Choose basic or advanced security. Procedural Instructions Selecting the Level of Security and Overwriting of Existing Security on page 53 same as the previous action Selecting Advanced Security Level on page 55 Appendix B, Installed Files, Directories, and System Changes Applicable Security Level all

2. 3.

Specify whether to overwrite existing security. Depending on what you choose in the first task, select an advanced security level. Verify that the security components were properly installed.

all 14

4.

all

38

PATROL Security User Guide

Configuring Security Content

Configuring Security Content


After PATROL Security is installed, you can run PATROL with the default security contents. However, this content is for demonstration purposes only and is NOT secure. This section outlines the procedures that you must perform to configure PATROL Security to be secure. The content that you need to modify, replace, or add includes
I I I I I I I I

key databases passwords public and private key pairs certificates Certificate Authority (trusted root authority) authentication provider encryption algorithm security ACL (level 4 only)

Considerations
Before configuring your security content, consider the following questions:
I

Do you want to create your own SSL key databases or modify the ones packaged with PATROL Security? Do you want to run your applications in attended or unattended mode? What naming conventions will you use for key databases and labeled key pairs? Do you want to use the default authentication method for your operating system or set up Pluggable Authentication Module (PAM) on Unix or UserLogon( ) on Windows? Which applications polices are being used by which PATROL applications? (See PATROL Applications and Their Policies on page 100.)

Chapter 2

Planning

39

Setup Tasks

Setup Tasks
Table 10 lists tasks to perform after installation. Table 10
Order 1. 2. 3.

Overview of Configuration Tasks


Task Create an SSL key database. Install a CA certificate (trusted root certificate) into the key database. Verify CA certificate (trusted root certificate) is installed into the key database Generate public and private keys. Create a certificate signing request (CSR). Install a key pair certificate into the key database. List signed certificates in the key database. Procedural Instructions Creating an SSL Key Database on page 70 Installing a CA Certificate in the Key Database on page 85 Verifying Trusted Root Authority Certificates on page 86 Applicable Security Level required for 3 4, optional for 2 required for 3 4, optional for 2 required for 3 4, optional for 2

4. 5. 6. 7. 8. 9. 10. 11.

Generating Public and Private Keys on required for 3 4, page 72 optional for 2 Creating a Certificate Signing Request on page 89 Installing a User Certificate in the Key Database on page 92 required for 3 4, optional for 2 required for 3 4, optional for 2

Listing Certificates in the Key Database required for 3 4, on page 93 optional for 2 all required for 3 4, optional for 2 required for 3 4, optional for 2 4

Discover which polices are employed PATROL Applications and Their by your PATROL applications. Policies on page 100 Designate a key database for an applications. Set the Attended or Unattended Mode. Edit the security access control list (ACL). Designating a Key Database for an Applications Role on page 133 Setting the Attended or Unattended Mode on page 134 Configuring the SSL access File on page 166

40

PATROL Security User Guide

Maintaining Security Content and Configuration

Maintaining Security Content and Configuration


PATROL Security provides a number of utilities that help you manage and maintain security in your PATROL environment. This section groups these procedures by the security content that they affect.

Considerations
After you have performed the minimal configuration for your security content, consider the following ways in which you can manage and maintain security.
I

Do you want to be able to change the password to your key databases? If so, how often?

How will you manage and distribute passwords to key databases and policies? How often will you update your certificate revocation list, which prevents compromised certificates from being accepted? How will you manage key content over the network? by generating individual keys for each computer and distributing them by exporting one key pair and importing it into key databases on all other computers

Are you satisfied with the security level you chose during installation or would you like to change the security level for one or more computers?

Chapter 2

Planning

41

Maintenance and Management Tasks

Maintenance and Management Tasks


Table 11 lists tasks to manage and update various aspects of security. Table 11
Order 1.

Overview of Maintenance Tasks (Part 1 of 2)


Task Change the password for the key database. Export and import key pairs and certificates. Procedural Instructions Key Database Changing the Password for the Key Database on page 71 Key Pair and Certificates Exporting Key Pairs and Assigned Certificates on page 76 Importing Key Pairs and Assigned Certificates on page 78 24 Applicable Security Level

1.

24

2. 3. 4. 5.

Change label (identity) of an imported key pair.

Changing the Label of a Key Pair on page 74

24 24 24 24

Delete private and public key pairs and Deleting Private and Public Key Pairs and certificates. Certificates on page 75 List a certificate in the key database. Delete a certificate from the key database. Listing Certificates in the Key Database on page 93 Deleting a Certificate on page 93

User Credentials (Labeled Passwords) in Key Databases 1. Adding User Credentials (Labeled Add, list, and delete user credentials (labeled passwords), which are stored Passwords) on page 80 in key databases and used by PATROL Listing User Credentials (Labeled applications to authenticate users. Passwords) on page 81 Deleting User Credentials (Labeled Passwords) on page 82 Certificate Authority\Trusted Root 1. 2. View field information for CA certificate (trusted root certificate). Delete CA certificate (trusted root certificate). Acquire a CRL. Install a CRL. Viewing Field Information for CA Certificates on page 87 Deleting Trusted Root Authority Certificates on page 88 Acquiring a Certificate Revocation List on page 95 Installing a Certificate Revocation List on page 95 24 24 24

Certificate Revocation List (CRL) 1. 2. 24 24

42

PATROL Security User Guide

Maintenance and Management Tasks

Table 11
Order 1. 2. 3. 4.

Overview of Maintenance Tasks (Part 2 of 2)


Task Learn the policy content and role. List version information for security modules. Procedural Instructions Policies Viewing the Policies and Roles on page 115 Viewing Version Information for Security Modules on page 117 24 24 24 24 Applicable Security Level

Specify an authentication provider and Specifying an Authentication Provider service. and Service on page 119 Select an encryption algorithm. Selecting an Encryption Algorithm on page 128 Passwords in Policies Adding or Editing a Password Stored in a Policy on page 135 Password Encryption Encrypting a Password on page 138

1.

Add or edit a password stored in a policy. Encrypt a password for use as a key database password in a policy or user credential (labeled password). Adjust security level for PATROL applications in your enterprise.

24

1.

24

Security Level 1. Changing the Security Level for the Enterprise on page 172 all

Chapter 2

Planning

43

Verifying Security Configuration

Verifying Security Configuration


After you have customized the security content to suit your environment, verify that the changes have been implemented and that they were successful.

Considerations
Changes are opportunities for error, which can leave your environment exposed to security risk. Test any security configuration changes that you make.

Test Tasks
Table 12 lists tasks to perform when confirming the integrity of the configuration for PATROL Security. Table 12
Order 1. 2. 3. 4. 5. 6.

Overview of Testing Tasks


Task Test authentication provider. Test encryption algorithm. Test clientserver communication. Verify that the security components were properly installed. Test digital signing. Test the verification of a digital signature. Procedural Instructions Testing Authentication Configuration on page 126 Testing Encryption Algorithm on page 131 Testing a Secure TCP/IP Channel for the Client and Server on page 145 Viewing the Policies and Roles on page 115 Testing Digital Signing on page 141 Testing the Verification of a Digital Signature on page 143 Applicable Security Level 24 24 24 24 24 24

44

PATROL Security User Guide

Performing Diagnostics

Performing Diagnostics
If, even after you have verified your customization of security content, you encounter problems with PATROL Security, you can review your setup for the most common problems.

Considerations
When you encounter problems, check
I

the rights and privileges of the accounts under which you are running PATROL applications the logs for the application policies and roles under which the application is running; the location of the log file for each role is specified in logdir and logfile attributes of that role section

TIP
The log files provide multilevel tracing of all conditions and the root cause of the condition.

Troubleshooting Tasks
Appendix B, Troubleshooting lists some common problems that can arise with regard to security. This appendix provides symptoms and their causes to assist you in diagnosing the problems and offers solutions to resolve the problems.

Chapter 2

Planning

45

Troubleshooting Tasks

46

PATROL Security User Guide

Chapter

Installation
PATROL Security and the Extended Security System (ESS) are installed during the installation process of the PATROL products with which ESS is integrated. Most of the installation options are determined by the product with which it is installed. However, you do get to specify the level of security, whether an existing security configuration is overwritten, and indirectly, where security components are installed. This chapter presents the following topics: Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Over-the-Top Installation and Policy Migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . Compatibility with the Previous Version of PATROL Security . . . . . . . . . . . . . . . Customizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selecting the Level of Security and Overwriting of Existing Security. . . . . . . . . . Selecting Advanced Security Level. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selecting Connection Type for Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Location and Storage of Security Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Directories, Files Types, and Registry Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 48 48 49 51 53 55 56 56 56

Chapter 3

Installation

47

Overview

Overview
The PATROL Security and the Extended Security System components are packaged with all software infrastructure pieces. These components compose the foundation of the 3-tier architecture and include console systems such as the PATROL Central Windows, common services such as the Console Server, and managed systems such as PATROL Agents. The necessary security components are installed when you install one or more of the infrastructure components.

Installation Process
The Extended Security System (ESS) is integrated with a number of PATROL products and does not have its own, separate installation process. The ESS components are installed during installation of these products. The position of the few security screens within the installation process varies depending upon the product with which it is packaged.

Over-the-Top Installation and Policy Migration


PATROL Security 3.0.05, which contains Extended Security System 3.0 (ESS3.0), can be installed over the previous version of PATROL Security 1.2.07, which contains Extended Security System 2.0 (ESS2.0). You do not have to uninstall the old version. The first time that you perform an over-the-top installation, the installation utility creates separate directories for the ESS3.0 files and then copies any customizations of the ESS2.0 configuration to the new ESS3.0 configuration. The goal of the migration process during installation is to preserve the customizations that you have made to existing security content such as key databases, acquired key pairs, unattended passwords and other aspects. The migration process preserves your ESS2.0 configuration while copying changes to your security content and configuration to the ESS3.0 configuration. This approach enables PATROL applications that use the ESS2.0 version (such as PATROL Agent 3.5.20.x) and other applications (such as the Console Server 7.5.x) that use ESS3.0 to operate on the same computer. For information about migration, see Policy Migration on page 151.

48

PATROL Security User Guide

Compatibility with the Previous Version of PATROL Security

Compatibility with the Previous Version of PATROL Security


PATROL Security 3.0.05, which contains Extended Security System 3.0 (ESS3.0), is fully compatible PATROL Security 1.2.07, which contains Extended Security System 2.0 (ESS2.0). This compatibility enables PATROL applications that use the ESS2.0 version, such as PATROL Agent 3.5.20.x, and other applications that use ESS3.0, such as the Console Server 7.5.x, to operate simultaneously on the same computer. Compatibility is achieved by means of
I I I I

separating policy locations separating runtime environments and directory structures sharing security contents providing network security protocols that are compatible over the wire

Separate policy locations


To preserve your custom configuration of the security policies in PATROL Security 1.2.07, the installation process creates a separate location in which to store PATROL Security 3.0.05 policy information. During migration, policy information is copied from PATROL Security 1.2.07 to the PATROL 3.0.05 location. Table 13 lists the location of policies for both PATROL Security 1.2.07 (ESS2.0) and PATROL Security 3.0.05 (ESS3.0). Table 13 Policy installation location
Version of PATROL Security Location 3.0.05 HKEY_LOCAL_MACHINE\Software\BMC Software\Patrol\SecurityPolicy_v3.0 (ESS3.0) 1.2.07 HKEY_LOCAL_MACHINE\Software\BMC Software\Patrol\SecurityPolicy (ESS2.0) Unix 3.0.05 /etc/patrol.d/security_policy_v3.0 (ESS3.0) 1.2.07 /etc/patrol.d/security_policy (ESS2.0)

Operating system Windows

Chapter 3

Installation

49

Compatibility with the Previous Version of PATROL Security

Separate directory structures


To enable both PATROL Security 3.0.05 and PATROL Security 1.2.07 to run simultaneously on the same computer, the runtime environments have been separated into two distinct locations. Table 14 lists the installation path for both PATROL Security 1.2.07 (ESS2.0) and PATROL Security 3.0.05 (ESS3.0). Table 14 Installation location for versions of PATROL Security
PATROL PATROL Security 1.2.07 Security 3.0.05 (ESS2.0) (ESS3.0) File Types bin_v3.0 libraries and executables (.dll, .exe, .so) log file (.log) libraries (.a and .lib) scripts, templates, and executables (.cmd, .conf, .exe, .plc, .reg, .sh) key databases (.kdb) created at install or by user key databases (.kdb) created by PATROL applications

Installation path

%BMC_ROOT%\common\security bin (Windows) $BMC_ROOT/common/security (Unix) log lib config

log_v3.0 lib_v3.0 config_v3.0

keysa

sksa

The keys directory and the sks directory are shared between PATROL Security 1.2.07 and PATROL Security 3.0.05.

Shared security contents


To preserve customized security contents, such as unique key pairs, acquired certificates, and modified key databases, both PATROL Security versions use the same locations (keys and sks directories) to retrieve and store their security contents, as listed in Table 14.

Network protocols
All network security protocols modules are backward compatible to assure on-thewire compatibility.

50

PATROL Security User Guide

Customizations

Customizations
You can define the following aspects of the security system:
I I I I

overwriting existing security settings in an over-the-top installation choosing the level of security level choosing connection type determining the installation path of the security components

Overwriting Existing Security Content


The installation process for PATROL Security enables you to overwrite existing security content and configuration.

WARNING
BMC Software does not recommend selecting the Overwrite check box during the installation process to modify existing security content and configuration. This option will erase all changes to existing security content (acquired Certificate Authority certificates, updated key databases, custom-generated key pairs and certificates, modifications to policies) and require you to begin the security configuration process from the beginning.

Upgrading from PATROL Security 1.2.07 to 3.0.05


When you select the overwrite option during the installation, the installation process will replace the security content in the keys directory with the BMC Software default security content. If you have made any changes to that content such, as installed a Certificate Authority, generated unique key pairs, or install a certificate, you will lose those changes.

Installing PATROL Security 3.0.x over an existing 3.0.x installation


When you select the overwrite option during the installation, the installation process will replace the security content in the keys directory as described in Upgrading from PATROL Security 1.2.07 to 3.0.05. The process will also replace your security policies with the default policies. If you have made any changes to the security policies, such as configured authentication, changed the key databases used by a particular role, or added or changed a password, you will lose those changes.

Basic or Advanced Security


During installation, you are prompted to select the level of security that you want to implement for your PATROL installation. If you select Advanced Security, the installation utility presents a second screen that prompts you to select the level of security that you want.
Chapter 3 Installation 51

Customizations

NOTE
If you select Advanced Security (levels 1-4), you must configure various security components to create a secure environment.

Location
The installation location of the security components is determined by the installation path that you choose for the product. Relative to the product, the installation utility always installs the security components in the same directory structure: BMC_ROOT\common\security. For more information about the security components directory structure, see Directories, Files Types, and Registry Keys on page 56.

52

PATROL Security User Guide

Selecting the Level of Security and Overwriting of Existing Security

Selecting the Level of Security and Overwriting of Existing Security


The first screen in the installation process that pertains to security is the Select Level of Security screen. As illustrated in Figure 3, the screen contains two user options: the security option and the overwrite option. Figure 3 Select Level of Security Screen

Chapter 3

Installation

53

Selecting the Level of Security and Overwriting of Existing Security

Security Option
By using the security option, you can determine the type of security that you want to use in your PATROL installation:
I

Advanced security optionsencompasses levels 1 through 4, which provide varying

degrees of encryption and authentication but also require varying degrees of post installation configuration to make them secure, such as identifying a Certificate Authority, generating key pairs, acquiring certificates and so forth
I

Basic securityencompasses level 0, which does not require any additional configuration

Selecting the advanced option invokes the Select Level of Security Screen screen.

Overwrite Current Security Configuration


If you are installing into an environment that already has an existing PATROL installation, this option specifies whether to overwrite the existing security configuration information. Overwriting your security configuration means that you will have to reinstall
I I I I I

keys key databases certificates Certificate Authority certificate (also referred to as trusted root authority certificates) certificate revocation list

It may also involve re-entering changes or customizations to policy files and policy registry entries, patrol.conf, and config.default.

54

PATROL Security User Guide

Selecting Advanced Security Level

Selecting Advanced Security Level


The installation process presents the screen that is show in Figure 3 to you only if you selected the Advance security options on the Select Level of Security Screen screen. Figure 4 Select Advanced Level of Security Screen

Advanced Security Level


The Advanced security level option enables you to select the level of security at which you want PATROL to run. For information about the levels of PATROL Security, see Levels of Security on page 18.

Chapter 3

Installation

55

Selecting Connection Type for Security

Selecting Connection Type for Security


If you select advanced security levels 3 or 4, the installation process presents the Select Connection Type for Security screen. Choose the network protocol for security communication, either
I I

TCP UDP

Location and Storage of Security Information


The components of the security infrastructure are installed in a dedicated subdirectory structure. This structure contains executables, shared libraries, key files, and logs related to security. The policy files, which define the level of security and store crucial information about how the security system is implemented, are stored in a subdirectory of /etc on Unix systems and in the Registry on Windows systems.

WARNING
To ensure the integrity of your security components, limit access to the security directory structure and ownership of the security files. Group access should be allowed only in environments where membership to groups is strictly defined, tightly controlled, and routinely monitored.

Directories, Files Types, and Registry Keys


Table 15 outlines the directory structure and registry paths (for Windows) in which the installation utility installs security information.

56

PATROL Security User Guide

Directories, Files Types, and Registry Keys

Table 15

Installation Paths of Security Files and Registry Keys


Directory or Registry Path Unix $BMC_ROOT/common/security ../common/security/bin_v3.0/Unix_platform ../common/security/lib_v3.0/Unix_platform shared libraries (*.so) executables (*.*) key database files (*.kdb) File Types or Registry Keys

Location of Information BMC security directory Shared Libraries and Utilities Key databases Configuration scripts and templates Java Log files PATROL Security directory Policy files security directory Shared Libraries and Utilities

../common/security/keys ../common/security/config_v3.0 ../common/security/java/v.r.mm ../common/security/log_v3.0 /etc/patrol.d /etc/patrol.d/security_policy_v.30. Windows %BMC_ROOT%\common\security ..\common\security\bin_v3.0\Windows-x86 ..\common\security\lib_v3.0\Windows-x86

Java archive (*.jar) text file (*.log) policy file (*.plc)

shared libraries / dynamically linked libraries (*.dll) executables (*.exe)

Key databases Configuration scripts and templates

..\common\security\keys ..\common\security\config_v3.0

key database files (*.kdb) registry entries (*.reg) registry entry template (*.reg_tmpl) Windows command scripts (*.cmd) executables (*.exe)

Java Log files Registry Entries policy keys

..\common\security\java\v.r.mm ..\common\security\log_v3.0 My Computer\HKEY_LOCAL_ MACHINE\SOFTWARE\BMC Software\PATROL\Security Policy_v3.0

Java archive (*.jar) text file (*.log) esi site signer verifier

Chapter 3

Installation

57

Directories, Files Types, and Registry Keys

58

PATROL Security User Guide

Chapter

Keys and Certificates


This chapter describes the roles that keys and certificates play in securing your environment and how to set up and configure them based upon the level of security that you select. This chapter provides both conceptual and practical information. It discusses the concept of authentication and explains how it uses keys and certificates for implementation. The tasks provide step-by-step instructions for configuring your security environment to support authentication by means of keys and certificates. This chapter presents the following topics: Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Types of Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concepts and Components of Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Default Key Databases and Certificate Authorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . Workflow for Configuring PKI-Based Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Utilities for Key Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . sslcmd Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . bmckeycli Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of Keys and Key Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Key Databases Shipped with PATROL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating an SSL Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Password for the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transferring a Keyfile.kdb from Unix to Windows Environment . . . . . . . . . . . . . Generating Public and Private Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Listing PublicPrivate Key Pairs in the Key Database. . . . . . . . . . . . . . . . . . . . . . . Changing the Label of a Key Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting Private and Public Key Pairs and Certificates. . . . . . . . . . . . . . . . . . . . . . Exporting Key Pairs and Assigned Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . Importing Key Pairs and Assigned Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of User Credential (Labeled Password) . . . . . . . . . . . . . . . . . . . . . . . . . . Purpose and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . . Listing User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting User Credentials (Labeled Passwords) . . . . . . . . . . . . . . . . . . . . . . . . . . . Management of Certificate Authority. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Establishing a CA Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 61 62 65 66 67 67 68 69 69 70 71 71 72 73 74 75 76 78 80 80 80 81 82 83 83

Chapter 4 Keys and Certificates

59

Installing a CA Certificate in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Verifying Trusted Root Authority Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Viewing Field Information for CA Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Deleting Trusted Root Authority Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Management of User Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Certificate Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Creating a Certificate Signing Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Installing a User Certificate in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Listing Certificates in the Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Deleting a Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Management of Certificate Revocation Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Description of a Certificate Revocation List (CRL) . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Missing Certificate Revocation List Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Acquiring a Certificate Revocation List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Installing a Certificate Revocation List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

60

PATROL Security User Guide

Authentication

Authentication
Keys, key databases, and certificates facilitate the type of security referred to as authentication. In the context of PATROL, authentication is a means of security by which software components (consoles, servers, agents, and so forth) can programmatically verify that a component or the user of a component is who it states it is. For a component to authenticate the certificate of another component, the original component must trust the Certificate Authority (CA) of the component attempting the communication. In terms of keys and certificates, the original component must have in its key database a copy of the communicating components CA certificate.

Types of Authentication
Communication and the transfer of information or data involves two parties or components: a client and a server. SSL protocol authentication enables PATROL applications to positively authenticate the identity of the server, and optionally, of a client. The different types of authentication protect against different susceptibilities in an enterprise.
I

SSL Protocol Server AuthenticationA server, such as a PATROL Agent, presents a certificate to a client, such as a PATROL console 3.x or Console Server, so that the client can authenticate the server. In this type of authentication, the server is required to prove its identity. Security level 2 and 3 supports server authentication; however, at level 2, the client does not discontinue communications if it cannot prove the servers identity.

SSL Protocol Mutual AuthenticationBoth clients and servers present certificates to each other so that each can verify the identity of the other. Mutual authentication requires that both components, client and server, have the others CA certificate installed in its key database. In this type of authentication, both the server and the client are required to prove their identities. Security level 4 supports mutual authentication.

Chapter 4 Keys and Certificates

61

Concepts and Components of Authentication

Concepts and Components of Authentication


This section defines some concepts basic to authentication and describes the various components need to implement authentication.

Concepts
The following concepts apply to keys and certificates.

Chain of Trust
This is a principle of security by which a software component verifies the identity of an unknown party by accepting the assurance of a third party whose identity it knows is genuine. It is possible that this partys identity is trusted because of the assurance of yet another party. This series of verifications by a trusted party continues (in a chain) until it is traced back to a trusted root (also known as a Certificate Authority) that the software component knows is trustworthy because it is provided by its own company or an approved vendor.

Digital Signing
This is the process of generating a hash value or check sum by applying an algorithm to a file. The check sum is then used by the recipient of the file to verify that the contents of the file have not been altered during transmission from the sender to the receiver. The check sum is protected by its being encrypted with the signers private key. The resulting value is called a signature. BMC Software generates digital signatures in compliance with the Public-Key Cryptography Standard # 1 (PKCS# 1) standard.

Digitial Verification
This is the process of decrypting a signature with the public key of the signer. The signers public key resides in the signers certificate, which must be stored in the key database used by an application operating in the verifier role.

Components
Certificate
This is a digital document containing a public key and a name used to authenticate the identity of the source of the data accompanying the certificate.

62

PATROL Security User Guide

Concepts and Components of Authentication

Certificate Authority
This is an issuer of an x509 certificate used in Secure Socket Layer (SSL) connections. It is also referred to as a trusted root authority.

Key
A key is a number (large) or set of numbers that possess mathematical properties that support both
I I

encryption with a private key and decryption with a public key encryption with a public key and decryption with a private key

Key Database
Also referred to as a key file and designated by the extension *.kdb, this file contains all the information necessary to verify a certificate. The file is encrypted with 3DES-CBC encryption and protected by a password. Its contents include
I I I I

public keys for the software application and for the trusted roots private keys for the software application user certificates and trusted roots Certificate Revocation Lists (CRL)

Depending upon the various roles of a computer, more than one key database can exist on a single computer. The key database can contain any number of CAs, private and public keys, and user certificates.

Label
This is a descriptive, alphanumeric text string that is assigned to a key pair or password in the key database to help an administrator identify and manage the key and/or password. In the sslcmd utility, a label is also referred to as identity.

Labeled Password
Sometimes the need arises for some means of securely storing the passwords to other systems in the key database. The sslcmd utility provides a means to assign to a password or other string of bytes a descriptive text string to help identify and manage the password.

Chapter 4 Keys and Certificates

63

Concepts and Components of Authentication

Public Key Infrastructure (PKI)


This infrastructure provides the means for performing public and private key cryptography. PKI-based security includes Secure Socket Layer (SSL), digital signing, and verification.

Self-Signed Certificate
A self-signed certificate is a certificate issued directly by the Certificate Authority (CA). It is also referred to as a trusted root authoritys certificate.

sslcmd
This is the key management utility used to create, set up, and manage key databases and certificates.

User Credentials
This is the user name and password used by an application to verify the identity of a user. Some PATROL applications store user credentials in a key database. User credentials can be added to, viewed, or deleted from a key database using the sslcmd key management utility. User credentials are also referred to as Labeled Password.

64

PATROL Security User Guide

Default Key Databases and Certificate Authorities

Default Key Databases and Certificate Authorities


WARNING
This product is delivered with default keys and certificates that are not unique to you. BMC Software highly recommends that you promptly change these default keys and certificates to ones that are unique. If you do not make these changes, there is a higher risk that any third party who gains physical access to your network, or to the data that you send over the internet, might have a better opportunity to use these default keys and certificates to gain unauthorized access to your system. BMC Software is not responsible for any damage or liability associated with your use of default keys and certificates.

Default keys and Certificate Authority (CA) certificates supplied by BMC Software and stored in key database files with .kdb extensions are provided only to demonstrate a turnkey security configuration, for purposes such as demonstrations and trial installations. Before using BMC Software products, replace the default keys and certificates with your own unique entities. A password is required to open the default key files. The password for all default key files is password. PATROL installs the BMC Software-provided default CA certificates in the key database in keys directory. The default certificates will expire on the date specified in the certificate, as shown in Table 16. Table 16
Certificate server

Default Certificate Expiration Dates


Expiration Valid Begin: Fri Dec 17 11:08:29 2004 Valid End: Sun Dec 17 11:08:29 2006

bmcuser

Valid Begin: Fri Dec 17 11:02:36 2004 Valid End: Sun Dec 17 11:02:36 2006

signer

Valid Begin: Fri Dec 17 11:15:08 2004 Valid End: Sun Dec 17 11:15:08 2006

Chapter 4 Keys and Certificates

65

Workflow for Configuring PKI-Based Security

Workflow for Configuring PKI-Based Security


Configuration consists of the following key management operations:
I I I I I I I I I

selecting a Certificate Authority (or choosing to implement your own) obtaining a root authoritys certificate (a certificate from your chosen CA) creating a key database file inserting the root authoritys certificate (CA certificate) into the key database generating a public and private key generating a certificate signing request (CSR) for the public key obtaining a certificate viewing and deleting a certificate distributing key pairs and certificates throughout an enterprise

NOTE
This process applies to security level 2 or greater. If your PATROL installation runs at security levels 0 or 1, you do not need to perform these tasks.

Table 17 suggests an order in which you may perform the configuration tasks for PKIbased security. In this chapter, the documented tasks have been organized according to the security entity (key database, certificate authority, certificate) that they affect. Table 17
Order 1. 2. 3. 4. 5. 6. 7.

Order of Configuration Tasks for Authentication Security


Task Creating an SSL Key Database Documentation Section Key and Key Database Management Page 70 85 86 72 89 92 93

Installing a CA Certificate in the Key Certificate Authority Database Management Verifying Trusted Root Authority Certificates Certificate Authority Management

Generating Public and Private Keys Key and Key Database Management Creating a Certificate Signing Request Installing a User Certificate in the Key Database Listing Certificates in the Key Database Certificate Management Certificate Authority Management Certificate Authority Management

66

PATROL Security User Guide

Utilities for Key Management

Utilities for Key Management


This section briefly describes the key management utilities installed with PATROL Security.

sslcmd Utility
The sslcmd utility is the key management utility with which you manage the key database and certificates to enable authentication.

Capabilities
This utility enables you to perform the following tasks:
I I I I I I

generating, listing, and deleting keys adding, listing, viewing, and deleting a Certificate Authority adding, listing, and deleting certificates generating a Certificate Signing Request adding a Certificate Revocation List changing a password for the key database

Location
Table 18 provides the installation path of the sslcmd utility based upon the operating system. Table 18
Windows Unix

sslcmd Installation Location


Path %BMC_ROOT\..\common\security\bin_v3.0\OS $BMC_ROOT/../common/security/bin_v3.0/OS

Operating System

Chapter 4 Keys and Certificates

67

bmckeycli Utility

bmckeycli Utility
The bmckeycli utility is a noninteractive version of the key management utility sslcmd. bmckeycli supports key management commands to be executed by CGI scripts, batch files or other scripts.

Capabilities
This utility enables you to perform the following tasks:
I I I I I I I I I I I I

generating RSA/DSA key pair with selectable key length of 512 or 1024 installing Certificate Authority (CA) certificate (trusted root authoritys certificate) generating certificate signing request (CSR) installing certificates listing and removing keys and certificates listing certificates only listing, viewing, and deleting trusted roots installing a new CRL from file importing and exporting key pairs in PKCS# 12 format adding a password to password storage listing and deleting an applications labeled passwords retrieved from storage changing the label of a key pair

For more information about bmckeycli, use the -h option.

Location
Table 19 provides the installation path of the bmckeycli utility based upon the operating system. Table 19
Windows Unix

bmckeycli Installation Location


Path %BMC_ROOT%\..\common\security\bin_v3.0\OS $BMC_ROOT/../common/security/bin_v3.0/OS

Operating System

68

PATROL Security User Guide

Management of Keys and Key Databases

Management of Keys and Key Databases


This section describes key and key database tasks that you will perform using the sslcmd utility, including the configuration tasks listed in Workflow for Configuring PKI-Based Security on page 66 and the following tasks:
I I I I I

listing public and private key pairs in the key database exporting key pairs and assigned certificates importing key pairs and assigned certificates deleting private and public key pairs and certificates changing the key database file password

You can perform these tasks on the key databases shipped with PATROL or on the key databases that you create.

Key Databases Shipped with PATROL


During the installation process, depending upon the products that you select, the Common Installation utility may install the following key database files:
I I I I I

bmcuser.kdb server.kdb signer.kdb verifier.kdb trustedroots.kdb

In the client policy, the keyfile attribute is left blank. It defaults to bmcuser.kdb. The client requires a key database when running at security levels 2 through 4.

WARNING
Do not delete or replace the trustedroots.kdb. This file is used by PATROL to verify the integrity of PATROL applications. If you are concerned about the presence of the BMC Software Demo Certificate Authorities (CN = Demo Certificate Authority and CN = WWWQA Testing Certificate Authority) in this database, you can remove them from the database. For information about how to remove a CA, see Deleting Trusted Root Authority Certificates on page 88.

Chapter 4 Keys and Certificates

69

Creating an SSL Key Database

Creating an SSL Key Database


This procedure describes how to create a custom key database, also referred to as a keyfile.

To Create Your Own SSL Key Database NOTE


These procedures are written using Microsoft Windows conventions. Users of Unix should make the appropriate substitutions where necessary.

1 At a command-line prompt, change to the directory that contains the sslcmd


utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by entering sslcmd -k path\keyfile.kdb, where keyfile can be
any alphanumeric string, except trustedroots. To create the file in a directory (such as keys in which the default *.kdb files are installed), you must to provide the relative path, such as ..\..\keys\keyfile.kdb, because the keyfile does not exist. Figure 5 displays the sslcmd utility message. Figure 5 sslcmd Example keyfile.kdb not found

File <keyfile> not found. Enter new key file <keyfile> password (at least 8 characters):

3 Enter a password (at least eight characters and a combination of letters, numbers,
or special characters).

4 Re-enter the password when prompted.


The system creates a new key database in the location that you specified (or in the default location, bin_v3.0, if you did not provide one) and displays the sslcmd menu. For information about how to change the password, see Changing the Password for the Key Database on page 71.

NOTE
BMC Software recommends that you back up your key database file on a regular basis and keep the backup copy in a secure location.

70

PATROL Security User Guide

Changing the Password for the Key Database

Where to go from here


After you have created a key database, you must edit the appropriate policys keyfile attribute so that the software application will use the correct key database. For information about how to edit a policy, see Chapter 5, Security Policies.

Changing the Password for the Key Database


This procedure describes how to change the password for a key database.

To Change the Password 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 12 for Change KDB Password to change the
current password, and then press Enter.

5 At the Enter new key file password (key_database_filename) prompt, type the new
password and press Enter. The password must a minimum of eight printable characters and a maximum of 255.

6 At the Retype password prompt, retype the new password and press Enter.
If the password change is successful, sslcmd displays the message, Command successful: Change KDB Password.

Transferring a Keyfile.kdb from Unix to Windows Environment


If you set a key database file using a Unix computer, you can use file transfer protocol (FTP) to transfer the file to a Windows computer. It is important to transfer the file using the bin command so that the file is transferred as a binary file rather than an ASCII file.
Chapter 4 Keys and Certificates 71

Generating Public and Private Keys

Generating Public and Private Keys


Before you can request a certificate from the Certificate Authority, you must generate a public and private cryptographic key pair as described below. A cryptographic key pair is a set of two cryptographic keys (one publicly shared and one private) used to start an SSL session. Next, assign that key pair to the new certificate for a BMC Software product user or product component.

To Generate Public and Private Keys (Key Pair) 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 1 for Generate Key to generate a publicprivate
key pair to be assigned to a new certificate, and then press Enter.

5 At the Enter identity prompt, enter an identity (alias) name for the key pair, and
then press Enter. The identity name is the ID that identifies the publicprivate key pair. The identity is usually the same as the name of the key database file.

6 At the Enter keypair type, D for DSA, <other> for RSA prompt, select an RSA
algorithm by pressing Enter. (DSA, otherwise known as DSS, the USA's federal Digital Signature Standard, is not implemented).

7 At the Enter key length 512|1024 prompt, enter the size for the publicprivate key
pair that you want to create, and then press Enter. You can specify either a 512-bit or a 1024-bit key. If the key generation is successful, sslcmd generates the publicprivate key pair and displays the message, Command successful: Generate key.

Where to go from here


To verify that the key pair has been created, see Listing PublicPrivate Key Pairs in the Key Database on page 73.

72

PATROL Security User Guide

Listing PublicPrivate Key Pairs in the Key Database

Listing PublicPrivate Key Pairs in the Key Database


This procedure describes how to display the identity (alias name) of the generated key pair and any certificates that use the key pair.

To List Keys 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 5 for List keys to list the publicprivate key pairs
in the SSL key database, and then press Enter. For each key pair, the utility displays the label assigned to the certificate that uses the publicprivate key pair. If no certificate exists, the label has a value of 0 and the name of the generated key pair is displayed under the label value. After all key pairs are listed, the utility displays the message, Command successful: List keys.

Chapter 4 Keys and Certificates

73

Changing the Label of a Key Pair

Changing the Label of a Key Pair


Labels are assigned to key pair to make them easily identifiable and manageable. This procedure describes how to change an existing label assigned to a key pair, even an imported key pair.

To Change the Key Pair Label 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 18 for Change Label of Key Pair, and then press
Enter.

5 At the (-) Enter identity prompt, type the alphanumeric text string (alias name) used
to identify the key pair. For information about how to view the identity of a key pair, see Listing Public Private Key Pairs in the Key Database on page 73.

6 At the (+) Enter identity prompt, type the alphanumeric text string (alias name) to
which you want to change the label, and then press Enter. If the label change is successful, sslcmd displays the message, Command successful: Change Label of Key Pair.

74

PATROL Security User Guide

Deleting Private and Public Key Pairs and Certificates

Deleting Private and Public Key Pairs and Certificates


This procedure describes how to remove a key pair (private and public keys) from a key database. Removing the key pair also removes any certificates that are assigned to and thus encrypted\decrypted by that key pair.

To Remove a Key Pair and Certificate 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 6 for Delete key, and then press Enter. 5 At the Enter identity prompt, enter the identity (alias) name of the key pair that you
want to delete from the SSL key database, and then press Enter.

6 At the Confirm deletion of prompt, enter y for yes, and then press Enter.
Enter n if you do not want to delete. sslcmd deletes the key pair and its associated certificates from the key database and displays the message, Command successful: Delete key.

Chapter 4 Keys and Certificates

75

Exporting Key Pairs and Assigned Certificates

Exporting Key Pairs and Assigned Certificates


Exporting key pairs from one key database and importing them into another is a means to distribute keys and certificates. The export process provides two forms of protection: key encryption and Message Authentication Code (MAC), commonly referred to as check sum. The encryption of the key pair allows you to transfer the pair to another computer by a insecure method. The check sum feature ensures that the encrypted key pair has not been altered or corrupted. This procedure describes how to export a privatepublic key pair and its related certificates so that they can be easily transferred to another computer and imported into a second key database.

To Export a Key Pair 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 17 for Export Key Pair, and then press Enter. 5 At the Export File Name prompt, type a filename, and press then Enter. 6 At the Enter identity prompt, type the identity of the key pair, and then press Enter. 7 At the Encryption password prompt, type a password, and press then Enter.
The password must be a minimum of eight printable characters and a maximum of 255, and should consist of printable characters.

NOTE
The encryption password is the key to the encryption algorithm that is used to encode the exported key. Because only the authorized recipient is supposed to know the password, only the authorized recipient will be able to decrypt the exported key.

8 At the Retype password prompt, re-enter the password, and then press Enter.

76

PATROL Security User Guide

Exporting Key Pairs and Assigned Certificates

9 At the MAC password, type a password, and then press Enter.


The password must be a minimum of 8 printable characters and a maximum of 255.

NOTE
Message Authentication Code (MAC) protection, also referred to as a check sum, is incorporated into the exported key file. During importation of the exported key, the MAC provides a means of verifying that the file containing the exported key was not altered in any way during transit. This check prevents an intruder from changing the imported key value from that which was exported. Frequently, the encryption password and the MAC password are the same value.

10 At the Retype password prompt, re-enter the password, and then press Enter.
sslcmd generates a PKCS# 12 formatted file with the name that you supplied in step 5 and displays the message, Command successful: Export Key Pair.

Chapter 4 Keys and Certificates

77

Importing Key Pairs and Assigned Certificates

Importing Key Pairs and Assigned Certificates


Importing key pairs and assigned certificates is an efficient way to distribute public private keys throughout an enterprise. This procedure describes how to import a key pair and a certificate assigned to it that has been exported into a Public Key Cryptography Standard number 12 (PKCS# 12) formatted file.

Before you begin


I

For importing, sslcmd expects a file containing a private key and its associated certificate in PKCS# 12 format. You must acquire the encryption password and the MAC password from the user that created the exported key pair file. The root authority (CA) of the private keys associated certificate must already be present in the key database. Otherwise, the database cannot authenticate the keys and certificate and will not import them into the database. For information about how to add a CA certificate, see Installing a CA Certificate in the Key Database on page 85.

To Import a Key Pair and Certificate 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 16 for Import Key Pair, and then press Enter. 5 At the Import File Name prompt, type a filename, and then press Enter. 6 At the Encryption password prompt, type the password, and then press Enter.
The user that exported this key pair assigned the encryption password to it. You must get the password from that user.

78

PATROL Security User Guide

Importing Key Pairs and Assigned Certificates

7 At the MAC password prompt, enter a password and press Enter.


The user that exported this key pair assigned the encryption password to it. You must get the password from that user. sslcmd imports the key pair and certificate and displays the message, Command successful: Import Key Pair.

Where to go from here


For information about how to view the key pair in the key database, see Listing PublicPrivate Key Pairs in the Key Database on page 73.

Chapter 4 Keys and Certificates

79

Management of User Credential (Labeled Password)

Management of User Credential (Labeled Password)


To protect user credentials (user passwords), PATROL applications store them in key databases. To be able to identify the passwords and extract them from the key database when needed, the PATROL applications assign a text string to them. This text string is called a label and passwords with an assigned label are referred to as Labeled Passwords. This section describes the following labeled password tasks that you can perform using the sslcmd utility:
I I I

adding user credentials (labeled passwords) to a key database listing user credentials that are stored in a key database deleting user credentials from a key database

Purpose and Usage


The sslcmd utility provides a means to assign an identity or tag to a password in the same manner as a publicprivate key pair. The labeled password can then be stored securely in the key database, along with the rest of the private data belonging to the security system. Applications can retrieve the password from the key database, using the label to identify it.

Adding User Credentials (Labeled Passwords)


This procedure describes how to add a labeled password into the key database.

To Add User Credentials (Labeled) Password 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

80

PATROL Security User Guide

Listing User Credentials (Labeled Passwords)

4 At the Enter a choice prompt, enter 13 for Add Labeled Password, and then press
Enter.

5 At the Enter identity prompt, type a description text string for the password, and
then press Enter.

6 At the Password (identity_name) prompt, type a password, and then press Enter. 7 At the Retype Password prompt, type the password, and then press Enter.
sslcmd displays the message Command successful: Add Labeled Password.

Listing User Credentials (Labeled Passwords)


This procedure describes how to list labeled password stored in a key database.

To List Labeled Passwords Stored in a Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 14 for List Labeled Password, and then press
Enter.

sslcmd lists all the labeled passwords in the key database and displays the message Command successful: List Labeled Password.

NOTE
The key management utility list the labels but does not display the values of the passwords.

Chapter 4 Keys and Certificates

81

Deleting User Credentials (Labeled Passwords)

Deleting User Credentials (Labeled Passwords)


This procedure describes how to delete a labeled password from a key database.

To Delete a Labeled Password from a Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 15 for Delete Labeled Password, and then press
Enter.

5 At the Enter identity prompt, type the identity (also referred to as the label) of the
password, and then press Enter.

6 At the Confirm deletion of identity_name (y/n) prompt, type y, and then press Enter.
sslcmd deletes the labeled password that you specified from the key database and displays the message, Command successful: Delete Labeled Password.

82

PATROL Security User Guide

Management of Certificate Authority

Management of Certificate Authority


This section describes certificate tasks that you will perform using the sslcmd utility, including the configuration tasks listed in Workflow for Configuring PKI-Based Security on page 66 and the following tasks:
I I

viewing field information for CA certificates deleting trusted root authority certificates

Establishing a CA Certificate
After you have generated a Certificate Signing Request (CSR) by using the sslcmd utility, you can submit the CSR to one of several public companies that serve as Certificate Authorities (CA) or your company can acquire the necessary software and credentials and become its own Certificate Authority. Examples of Certificate Authorities are
I I I I I I I I

Certiposte Serveur Deutsche Telekom Root CA 1 Entrust.net Secure Server Cerification Authority GTE Cyber Trust Root IPS SERVIDORES Microsoft Root Authority SecureNet VeriSign Trust Network

NOTE
BMC Software does not make any recommendations for the companies listed as examples. These companies are listed only to demonstrate the prevalence and diversity of companies that provide Certificate Authority service.

The CA certificate should be obtained from the Certificate Authority by a secure means. Using the sslcmd utility, this certificate can then be loaded into the security modules key database. After the certificate is loaded, it can be presented to any peer. This certificate contains a genuine copy of the CAs public key.

Chapter 4 Keys and Certificates

83

Establishing a CA Certificate

Secure Manner
A certificate should be obtained in a secure manner from a trusted CA. Failure to do so undermines the endeavor to provide security. In this context, a secure manner is defined as a manner in which the certificate is transferred (physically or electronically) from the CA to a key database without being intercepted and altered by a third party.

WARNING
Obtaining a Certificate Authority certificate from the internet is not considered a secure manner.

Certificate Format
A certificate that you obtain must be an ASCII text file in version 3 of the X.509 PEM (Privacy-Enhanced Mail) Base64 format. The key management utility (sslcmd) uses the X.509 ASCII string format to import certificates. You can obtain CA certificates from your chosen Certificate Authority.

84

PATROL Security User Guide

Installing a CA Certificate in the Key Database

Installing a CA Certificate in the Key Database


This procedure describes how to install a CA certificate, also referred to as a trusted root authority certificate, into a key database.

Before you begin


You must have already obtained a CA certificate in the required format, as described in Establishing a CA Certificate on page 83.

NOTE
The CA certificate that you are installing in this task differs from the publicprivate keypair certificate that you install in Installing a User Certificate in the Key Database on page 92.

To Install a Root Authority Certificate in the Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 2 for Add CA, and then press Enter to add a CAs
certificate to the key database.

5 At the Enter CA certificate file name prompt, enter the path relative to the current
directory and the file name of the CA certificate, and then press Enter. The system installs the specified CA certificate in the SSL key database and displays a verification message.

Chapter 4 Keys and Certificates

85

Verifying Trusted Root Authority Certificates

Verifying Trusted Root Authority Certificates


This procedure describes how to list all the CA certificates, also referred to as the trusted root authority certificates, that have been installed in a key database.

To Verify Trusted Root Authority Certificates 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 Enter 8 for List CA, and then press Enter.


sslcmd displays a list of CA certificates in the key database. After the information is displayed, the utility displays the message, Command successful: List CA.

86

PATROL Security User Guide

Viewing Field Information for CA Certificates

Viewing Field Information for CA Certificates


This procedure describes how to display the information stored within the certificate such as label name, country, and encryption method.

To View CA Certificate Information 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 9 for View CA, and then press Enter. 5 At the Enter CA number to view prompt, enter the number for the CA certificate that
you want to view. sslcmd displays the information about the CA certificate in the key database is displayed. After the information is displayed, the utility displays the message, Command successful: View CA.

Chapter 4 Keys and Certificates

87

Deleting Trusted Root Authority Certificates

Deleting Trusted Root Authority Certificates


If you learn that a CAs certificate has been compromised or you discontinue using the services of a particular CA, you should remove the CAs certificate from your key database.

TIP
BMC Software recommends that you remove the Demo Certificate Authorities (CN = Demo Certificate Authority and CN = WWWQA Testing Certificate Authority) from the trustedroots.kdb.

This procedure describes how to remove the certificate.

To Remove CA Certificate from a Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 10 for Delete CA, and then press Enter. 5 At the Enter CA number prompt, enter the number of the CA certificate that you
want to delete and press Enter. For information about how to view a list of CA certificates, see Verifying Trusted Root Authority Certificates on page 86.

6 At the Confirm deletion of <number> (y/n) prompt, enter y.


sslcmd deletes the CA certificate and displays the message, Command successful: Delete CA.

88

PATROL Security User Guide

Management of User Certificates

Management of User Certificates


The difference between a Certificate Authority (CA) certificate and a user certificate is that a user certificate is associated through a label or identity with a privatepublic key pair stored in the key database. The CA certificate contains only the public key. This section describes certificate tasks that you will perform using the sslcmd utility, including the configuration tasks listed in Workflow for Configuring PKI-Based Security on page 66 and the following tasks:
I I

listing signed certificates in the key databases deleting certificates

Certificate Format
The certificates that you obtain must be an ASCII text file in version 3 of the X.509 PEM (Privacy-Enhanced Mail) Base64 format. The key database administrator utility (sslcmd) uses the X.509 ASCII string format to import certificates. You can obtain these certificates with Microsoft Certificate Server, Netscape Certificate Server, and OpenSSL.

Creating a Certificate Signing Request


This procedure describes how to create a certificate signing request that is associated with a key pair that your generated and that can then be submitted to your Certificate Authority.

Before you begin


You must generate a public and private cryptographic key pair (as described in Generating Public and Private Keys on page 72. Next, you generate a certificate signing request that is associated with the key pair that you generated.

To Create a Certificate Signing Request 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb.

Chapter 4 Keys and Certificates

89

Creating a Certificate Signing Request

3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 3 for Generate CSR, and then press Enter. 5 At the CSR output file name prompt, enter the file name for the generated CSR, and
then press Enter. Unless you provide a path relative to the executables working directory along with the file name, sslcmd creates the CSR output file in the directory in which the executable resides.

6 At the Enter alias name prompt, enter the alias (identity) name for the key pair that
you generated, and then press Enter. The alias is the alphanumeric string that identifies the publicprivate key pair. The alias is usually the same as the name of the key database file.

7 When prompted to supply the information of the distinguished name (DN)


associated with your new certificate, see Table 20 to help you determine what information is needed for each prompt. Table 20
Prompt Country

Distinguished Name Prompts


Description of Requested Value the 2-character country code of the certifieds resident address. For a list of codes, see Appendix C, Valid Country Codes. For example, the country code for the United States is US.

State

the 2-character abbreviation of the state of the certified residents address For example, the state code for Texas is TX.

Locality Name Unit Common name E-mail address

the address of the certified resident the organization to which the certified person belongs the body within an organization to which the certified person belongs the name of the entity that you are certifying the return e-mail address for the certified person This value is used by the ESS connection profile ACL_Deny and ACL_Allow configuration variables, which are stored in the access file. At level 4 security, the e-mail address must match the value (literal string or expression with wildcards) set in the configuration variables.

90

PATROL Security User Guide

Creating a Certificate Signing Request

After you respond to all of the prompts, a CSR is generated and is ready for you to submit to the trusted CA for signing. If the generation of the signing request is successful, the message Command successful: Generate CSR appears and the system writes the certificate signing request (CSR) to the file that you specified in step 5 on page 90.

Where to go from here


Your next task is to present the certificate signing request (CSR) to your Certificate Authority (CA). Depending upon how your CA accepts CSRs, you will either have to use a text editor to extract the contents of the CSR and copy and paste it into a form or you can import the CSR file. Your CA will respond by giving you a signed certificate that you can then install in your key database, as described in Installing a User Certificate in the Key Database on page 92.

Chapter 4 Keys and Certificates

91

Installing a User Certificate in the Key Database

Installing a User Certificate in the Key Database


Installing a user certificate in the key database makes the certificate available to the BMC Software security subsystem for use in secure product communications.

WARNING
You must install the CA (trusted root authority) certificate in the key database before you install a user certificate in the database. If you do not, the key management utility fails to install the user certificate and returns a -45 error code. This procedure describes how to install the certificate that you received from your Certificate Authority into the key database from which you generated the Certificate Signing Request (CSR).

Before you begin


I

You must have installed the CA (trusted root authority) certificate from the vendor site to the database, as described in Installing a CA Certificate in the Key Database on page 85. You must have generated a CSR and submitted it to the vendor site, as described in Creating a Certificate Signing Request on page 89. You must have generated and downloaded the signed certificate from the vendor.

To Install a User Certificate in the Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 4 for Add cert to add a digital certificate to the
key database, and then press Enter.

5 At the Enter certificate file name prompt, enter the file name for the digital
certificate that you downloaded from the vendor site, and then press Enter. If the certificate is added, sslcmd displays the message: Command successful: Add Cert.
92 PATROL Security User Guide

Listing Certificates in the Key Database

Listing Certificates in the Key Database


This procedure describes how to view a list of signed certificates in a key database.

To List Certificates in a Key Database 1 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.

2 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 3 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

4 At the Enter a choice prompt, enter 7 for List certs to list the digital certificates
installed in the SSL key database, and then press Enter. For each signed certificate, sslcmd displays the label assigned to the certificate and the information that was assigned using the distinguished name prompts (see Table 20 on page 90). After the certificates, the utility displays the message Command successful: List Cert.

Deleting a Certificate
Deleting a privatepublic key pair also removes from the key database all the certificates associated with that key pair. For information about how to delete keys, see Deleting Private and Public Key Pairs and Certificates on page 75.

Chapter 4 Keys and Certificates

93

Management of Certificate Revocation Lists

Management of Certificate Revocation Lists


If the private key is given to an unauthorized entity or otherwise compromised, use a Certificate Revocation List (CRL) to invalidate it. The CRL is maintained by the Certificate Authority (CA). Similar to user certificates, CRLs are time stamped and signed by the issuing CA. When the private key associated with the public key contained in the certificate is compromised, the owner of the compromised private key should immediately notify the CA that signed the certificate. The CA then publishes a CRL, which lists the certificate as having been revoked. Each user of a particular CA should obtain the CRL of that CA on a regular basis and install the CRL in the key database, so that if the revoked certificate is presented at a later date, the software will detect it as a revoked certificate and the chain of trust will be broken. The frequency with which you acquire updated CRLs should be determined by the sensitivity of the application.

Description of a Certificate Revocation List (CRL)


CRLs are signed by the CA that issues them. The key database will not accept a CRL that has not had the certificate of the CA previously installed; therefore, a chain of trust for a CRL is established in the same manner as one for a certificate that is to be installed. A CRL is obtained from the CA by e-mail or over the web.

CRL Format
In the PATROL environment, the CRL is stored in Base64 encoding, as shown Figure 6. Figure 6 Example of a CRL Stored in a Key Database

-----BEGIN CERTIFICATE REVOCATION LIST----MIIBVDCBvjANBgkqhkiG9w0BAQQFADB5MQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVGV4 YXMxEDAO BgNVBAcTB0hvdXN0b24xEjAQBgNVBAoTCUNvcnBvcmF0ZTEVMBMGA1UECxMMQk1DIFNv ZnR3YXJl MR0wGwYDVQQDExRCTUMgU29mdHdhcmUgQ0EgUm9vdBcNMDExMDA0MTkzNTQ3WhcNMDEx MDA0MTk1 NTQ3WjAUMBICAQkXDTAxMDkxNzIxMDkzM1owDQYJKoZIhvcNAQEEBQADgYEACh2SCmVh nnYXz95G SHQ2WJbMBgjYkGvC4w/FF+c+4Q66ONbEZGmSFec3WfgW53Xb9C5RwKSDwU3ORPYkH2yV haUSDZkF 7M2AQdShu3K9fh3gs4pO1EBF/fOW4Frrc39w9fYML/3Jqp+9IOspJw9Ymx3S0bub9Q+n nS6YofkM Up0= -----END CERTIFICATE REVOCATION LIST-----

PATROL does not display the contents of the CRL.

94

PATROL Security User Guide

Missing Certificate Revocation List Warning

Precedence of New CRLs


As new CRLs are issued by the CA, they are installed. New CRLs overwrite any previous ones pertaining to that CA. Once a CRL is installed in the key database, you cannot remove it independently of the CA certificate. CRLs can only be updated.

Missing Certificate Revocation List Warning


The PATROL warning message REVOCATION UNKNOWN indicates that a chain of trust for a certificate has been established, but no CRL can be found for the CA. That is, the certificate is validated, but there is no list present to see if it has been revoked. PATROL does not accept a missing list as meaning no certificates have been revoked. Only a signed CRL that is empty can accomplish this.

WARNING
PATROL does not regard REVOCATION UNKNOWN as a fatal error. When this error occurs, a PATROL component does not prevent another component from establishing a connection and communicating with it. To ensure the security of your PATROL environment and to prevent this message from occurring, install a CRL (regardless of its contents) for the CA that signed the certificate.

Acquiring a Certificate Revocation List


Contact your Certificate Authority to see how you can obtain its CRLs on a regular and timely basis.

Installing a Certificate Revocation List


This procedure describes how to install a CRL.

To Install a CRL 1 Obtain the new CRL from the trusted CA. 2 At a command-line prompt, change to the directory that contains the sslcmd
utility. The path to the sslcmd utility is given in Location on page 67.
Chapter 4 Keys and Certificates 95

Installing a Certificate Revocation List

3 Start the sslcmd utility by enter sslcmd -k path\keyfile.kdb. 4 Enter the password for the SSL key database file that you want to access.
The system displays the menu.

5 At the Enter a choice prompt, enter 11 for Add CRL, and then press Enter. 6 At the Enter crl file name prompt, enter the file name of the CRL that you want to
install in the key database, and then press Enter. sslcmd installs the CRL into the key database and displays the message Command successful: Add CRL.

96

PATROL Security User Guide

Chapter

Security Policies
This chapter describes what security policies are, what part they play in PATROL Security, what kind of information they contain, and how that information is organized, formatted, and stored. This chapter provides both conceptual and practical information. It discusses the concepts of roles and explains how they are implemented by using files on Unix and registry entries on Windows. The tasks provide step-by-step instructions for how to create, configure, manage, test, and trouble-shoot these configurations. Finally, this chapter covers the automated migration process used to upgrade earlier versions of PATROL Security to the most current version. This chapter presents the following topics: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Site Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Application Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Policy Roles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Policy Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Inheritance and Precedence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 PATROL Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Format and Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Unix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Microsoft Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Utilities for Policy Testing and Password Encryption . . . . . . . . . . . . . . . . . . . . . . . . . 111 esstool Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 plc_password Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 bmcryptpw Utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 signFile and verifyFile Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Policy and Role Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Creating a Security Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Viewing the Policies and Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Viewing Version Information for Security Modules . . . . . . . . . . . . . . . . . . . . . . . 117 Authentication and Encryption Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Specifying an Authentication Provider and Service . . . . . . . . . . . . . . . . . . . . . . . 119 Testing Authentication Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Selecting an Encryption Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Chapter 5

Security Policies

97

Listing the Encryption Algorithms Supported by the Encryption Module . . . . 130 Testing Encryption Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Key Database and Password Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Designating a Key Database for an Applications Role . . . . . . . . . . . . . . . . . . . . . 133 Setting the Attended or Unattended Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Adding or Editing a Password Stored in a Policy. . . . . . . . . . . . . . . . . . . . . . . . . . 135 Encrypting a Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Signer and Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Operation of Signing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Operation of Verifying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Testing Digital Signing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Testing the Verification of a Digital Signature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Client-Server Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Testing a Secure TCP/IP Channel for the Client and Server. . . . . . . . . . . . . . . . . 145 Policy Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 PATROL Security versus Extended Security System . . . . . . . . . . . . . . . . . . . . . . . 151 ESS 3.0.00 and ESS 3.0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Migration Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Migrate or Overwrite. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Migration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

98

PATROL Security User Guide

Introduction

Introduction
A collection of data that defines and controls how security is implemented is referred to as a security policy. Security policies contain set up and configuration information for implementation of PATROL Security, which addresses potential security violations. The policies associate the potential security violations and the capability to prevent them with the types of applications that interact within the PATROL environment. The functions of these applications are termed roles within PATROL Security. PATROL provides security roles that, when properly configured, can address any security problem posed by applications fulfilling these roles. Security roles for PATROL applications include
I I I I I I I

authenticator encryptor keystore client server signer verifier

In a PATROL environment, the security policies define each role by storing in a series of policy attributes the details of how much security is implemented for an application operating in that role. These attributes define such aspects of security as
I I I I I

the PATROL Security level which key database to use the encrypted password required to access a key database the amount of security information written to the security log and its location the mode setting, which determines whether a password must be manually entered to start an application the location of security information such as key databases and key material files used to generate unique keys

At startup, each PATROL component attempts to load two security policies to determine its security configuration: a general site policy and a specific application policy. In the Unix environment, policies are implemented as *.plc files. In Windows environments, policies are implemented as registry entries.

Chapter 5

Security Policies

99

Site Policy

Site Policy
The default security policy is the site policy (site.plc on Unix; site registry entry on Windows). It is the only required security policy. The site policy defines the security configuration shared by PATROL services and provides the minimal amount of information that a PATROL component needs to load and run the Extended Security System (ESS) module. The site policy contains the default attributes for all security roles. The site policy attributes can be overridden by optional application policy attributes.

Application Policy
Application policies define roles, which contain attributes used by specific applications, such as agents or consoles. Application policies can override and augment the basic security policy for all PATROL services. As each application is initialized, the attributes specified for its role are loaded from the site policy. Selected attributes of the site policy are then modified by the application policy. Any attributes specified in the application policy take precedence over or override the attributes of the site policy.

TIP
The application policy name used by a PATROL application is built into that application. You cannot change which application policy a PATROL application loads. To change the policy information for a PATROL application, you must edit the policy that the PATROL application references. For information about which PATROL applications employ which policies, see Table 22 on page 101.

For more information about how policies operate, see Inheritance and Precedence on page 106.

PATROL Applications and Their Policies


For PATROL applications, security policies are stored in different locations on Unix and Windows. Table 21 provides the location based upon the operating system. Table 21
Windows Unix 100 PATROL Security User Guide

Policy Installation Location


Type of Storage registry key directory/file Location HKEY_LOCAL_MACHINE\Software\BMC Software\Patrol\SecurityPolicy_v3.0 /etc/patrol.d/security_policy_v3.0

Operating System

Policy Roles

Table 22 shows the policy configuration files or Windows registry entries that correspond to each PATROL application. The site policy is not listed because it is the default policy of every PATROL application and is required. Table 22 PATROL Applications and Their Corresponding Application Policy Names
Application Policy Application Policy Files Registry Entry Keys (Windows) (Unix) console.plc not applicable agent.plc cserver.plc client.plc esi.plc console pcentral agent cserver client esi

PATROL Application PATROL 3.5 console PATROL Central Microsoft Windows Edition 7.x PATROL Agent 3.5 or later PATROL Console Server PATROL Event Manager 3.5a, PATROL Console 3.4.11, PATROL Agent 3.4.11 and earlier, PATROL Command Line Interface, pconfig PATROL Configuration Managerb SignFile utility (digital signature signing CLI utility) VerifyFile utility (digital signature verification CLI utility)
a

pcm.plc signer.plc verifier.plc

pcm signer verifier

All applications that run in a PATROL 3 session or interact with PATROL using the PEM API will use the esi policy by default. PATROL Configuration Manager employs this application policy for use with only its reporting function. The key database specified by the policy stores user names and passwords of PATROL Agents for which the manager generates reports.

Policy Roles
The roles specify security capabilities of the application process. An application process that acts as a client will adopt the security constraints defined by the security policys client role. A server application will adopt the server role. Within the context of security, application processes can have multiple roles. Besides the communication roles, a process can also operate in an authentication role. An application can also operate in either a signer or verifier role by applying a signature to a file, by verifying a signature of a file, or by both signing and verifying signatures.

Chapter 5

Security Policies

101

Policy Roles

To define these roles for each application, policies consist of sections. Each section describes a role. In Unix, sections are designated by square brackets [ ] around a rolename: [role_name]. In Windows, sections are registry keys. Table 23 lists all the possible policy roles and describes each ones purpose. Table 23
Roles common

Policy Roles (Part 1 of 2)


Description specifies the shared configuration for all applications This role consolidates into one section all the attributes essential to the implementation of security. The Common policy section of a site security policy provides default values for the following policy attributes: bindir 32-bit binaries location bindir64 64-bit binaries location logdir location of log files logfile default log file name loglevel default log level securitydir location of private files such as key databases and key files

client

specifies security configuration of a client application At a minimum, the Client section should specify the security level, log level, and log file name and log file location. It supports the following attributes: keyfile, logdir, logfile, loglevel, password, and security_level.

server

specifies security configuration of a server application At a minimum, the Server section should specify the security level, log level, and log file name and location. It supports the following attributes: keyfile, logdir, logfile, loglevel, password, and security_level.

authenticator

specifies security configuration of an authentication application The Authentication section enables a user to specify an authentication provider and service parameters. It supports the following attributes: provider and service.

encryptor

specifies security configuration of a bulk encryption module The Encryptor security section specifies the encryption algorithm. It has one attribute, cipher_type.

keystore

specifies the configuration of a keystore security application A keystore application provides integrity and protection to confidential user data. It supports the following attributes: keyfile the path to the key database (*.kdb) for this policys application password the password required to access the key database. This attribute is optional. Include it only if you want to run in unattended mode.

102

PATROL Security User Guide

Policy Attributes

Table 23
Roles signer

Policy Roles (Part 2 of 2)


Description specifies which keystore (and thus user-created keys) that the application uses when signing data The Signer section lists attributes provided in both the Common and Keystore sections such as keyfile, password, and log attributes. It supports the following attributes: identity, keyfile, logdir, logfile, loglevel, password, and security_level.

verifier

specifies which keystore (and thus user-created keys) the application uses when verifying signed data The Verifier section lists attributes provided in both the Common and Keystore sections such as keyfile, password, and log attributes. It supports the following attributes: identity, keyfile, logdir, logfile, loglevel, password, and security_level.

Policy Attributes
To define each role policies contain attributes, which are assigned to roles. Attributes define the contents of security policy by defining specific actions that an application can and cannot perform with regards to security. Attributes also define characteristics of the application (role) within the context of PATROL. In the policies installed by PATROL Security, the default set of attributes assigned to each role is considered the optimal configuration. Table 24 lists all the possible policy attributes and describes each ones purpose. Table 24
Attribute bindir
a a

Policy Attributes (Part 1 of 3)


Description specifies the absolute path of the security 32-bits library specifies the absolute path of the security 64-bits library specifies the encryption algorithm cipher type used by the encryption module For a list of values, see Table 35 on page 128.

bindir64

cipher_type

identity keyfilea logdira logfilea

specifies the name or label under which the keypair is stored in the SSL keystore specifies the location of a SSL keystore database specifies the absolute path to a subdirectory where the log file will be written specifies the log file path When only a file name is provided, the log file is created in the current working directory. Chapter 5 Security Policies 103

Policy Attributes

Table 24
Attribute loglevel

Policy Attributes (Part 2 of 3)


Description ERROR, WARNING, INFO, TRACE specifies the log level desired. Any comma-separated combination of these tokens can be used to generate error, warning, or diagnostics log messages. To generate security-level information in the log file, you must include the INFO token.

password

specifies an unattended service encrypted password, key material, and optional lock mode required to retrieve the master password from the SSL key databases The encrypted password can be generated offline using bmcryptpw. The value consists of the following parameters separated by commas: encrypted_password, keymaterialfile location, [optional lock mode].
I

encrypted _password specifies the encrypted password generated by the offline bmcryptpw or plc_password password encoding program For more information, see bmcryptpw Utility on page 113 or plc_password Utility on page 112. The password encryption method is based on Triple DES PCBC cipher and CBC checksum.

keymaterialfile is the user-supplied file used for 3 DES key computation Any file can be used as a key material. You are responsible for administrative protection of the file. In operational environment, limit file exposure to the service startup only, and physically remove a file (for example, from a floppy disk drive) after the service is running. You are responsible for the protection and security risk taken due to the selection of such an unattended service startup. For a discussion of the security risks inherent in unattended operation, see Setting the Attended or Unattended Mode on page 134.

lock_mode field - user or ip specifies additional data, inserted during 3 DES key computation When user lock mode is specified, only the user specified at the time when password was encrypted using bmcryptpw or plc_password (-u user option) can decode the password. When ip lock mode is specified, the local hosts IP address is inserted during the key computation. The password can be decoded only on a computer with the same IP address as the one on which the password was encrypted.

104

PATROL Security User Guide

Policy Attributes

Table 24
Attribute provider

Policy Attributes (Part 3 of 3)


Description specifies an authentication security mechanism and overrides the default mechanism that is provided by the operating system A common provider for Unix systems is Pluggable Authentication Modules (PAM). A common provider for Microsoft Windows is LogonUser. The service attribute specifies to which application, such as rlogin or telnet on Unix and LOGON32_LOGON_ INTERACTIVE on Windows, the provider supplies security.

service securitydira

specifies additional details of security mechanism listed in the provider attribute specifies the directory where sensitive key information is stored (for example, SSL keystore or key material files)

security_level is a security grade (0-4) that specifies the methodology and security strengths of the application 0 is weakest; 4 is strongest. If the security_level field is deleted or contains an empty string, the level of security defaults to 4. This differs from the PATROL installation process, which defaults to 0 (basic security level).
a

Ensure that all file path conventions comply with operating system naming conventions.

Chapter 5

Security Policies

105

Inheritance and Precedence

Inheritance and Precedence


PATROL applications load two policies: the site policy and an application specific policy. When starting up a PATROL application first loads the site policy. After loading the site policy, PATROL loads the application policy. The resulting configuration
I

supplements the security established by the site policy with application policy attributes that the site policy does not contain overwrites the site policy roles with corresponding application roles inherits the site policy roles that the application policy does not contain

EXAMPLE
The Console Server plays the role of server (using the server policy) when communicating with a PATROL Central console. At the same time, the Console Server plays the role of client (using the client policy) when communicating with a PATROL Agent.

Table 25
Order 1. 2. 3.
a

Order of Precedence

Action Commona section of a site security policy is read in. Role section of a site security policy provides or overrides previously read parameters Role section of an application security policy provides or overrides previously read parameters

To ensure backward compatibility with ESS2.0 policy, the common role section of the policy is optional.

PATROL Configuration Files


In PATROL, security configuration and setup information are stored in PATROL configuration files (patrol.conf, config.default, dlls.conf, and access) and security policies. These files are discussed in Chapter 6, Configuration Files.

106

PATROL Security User Guide

Format and Implementation

Format and Implementation


Due to the differences in how the Windows and Unix operating systems manage and store sensitive information, the implementation of security differs. This section describes those differences in detail.

Unix
In Unix environments, a security policy is implemented as an ASCII text file. The format of the file is the standard .ini format.

Implementation of Roles
Roles are implemented as stanzas, which are indicated by the role name enclosed in square brackets ([ ]). Attributes are implemented as attribute/value pairs, which are formatted as attribute_name1 = value1, value 2, value N and ended by a new-line character.

Location of Policy Files


Separate files store the individual policies. On Unix computers, .plc files store the policies. They are located in the following directory:
/etc/patrol.d/security_policy_v3.0

Example of Unix Policy File with Stanzas and Attributes


Figure 7 displays a typical site policy (site.plc) for a Unix operating system. Figure 7 Sample Site Policy File (site.plc) for Unix (Part 1 of 2)

[common] bindir = /local/xyz/common/security/bin_v3.0/solaris-2-9-sparc bindir64 = /local/xyz/common/security/bin_v3.0/solaris-2-9-sparc64 securitydir= /local/xyz/common/security/keys logdir = /local/xyz/common/security/log_v3.0 sksdir = /local/xyz/common/security/sks [client] security_level=0 loglevel = ERROR,WARNING logfile = site_client.log

Chapter 5

Security Policies

107

Unix

Figure 7

Sample Site Policy File (site.plc) for Unix (Part 2 of 2)

[server] security_level=0 loglevel = ERROR,WARNING logfile = site_server.log [signer] security_level=0 password = 17fa9e37f011ec79ef0b32d00cbc98c4f4ca367272714f6b, /local/xyz/common/security/keys/sample.bin keyfile = /local/xyz/common/security/keys/signer.kdb identity = signer loglevel = ERROR logfile = site_signer.log [verifier] security_level=0 password = 17fa9e37f011ec79ef0b32d00cbc98c4f4ca367272714f6b, /local/xyz/common/security/keys/sample.bin keyfile = /local/xyz/common/security/keys/trustedroots.kdb loglevel = ERROR logfile = site_verifier.log [authenticator] loglevel = ERROR logfile = site_authenticator.log [keystore] password = 17fa9e37f011ec79ef0b32d00cbc98c4f4ca367272714f6b, /local/xyz/common/security/keys/sample.bin loglevel = ERROR logfile = site_keystore.log [encryptor] password = 17fa9e37f011ec79ef0b32d00cbc98c4f4ca367272714f6b, /local/xyz/common/security/keys/sample.bin loglevel = ERROR logfile = site_encryptor.log

108

PATROL Security User Guide

Microsoft Windows

Microsoft Windows
In Windows environments, a security policy is defined by registry keys placed in the Windows registry.

Implementation of Roles
Roles are implemented as one or more registry keys. Attributes are implemented as attribute/string values assigned to a registry key. Attribute/string values are entered in the Edit String dialog box, an example of which is displayed in Figure 8. Figure 8 Edit String Dialog Box

Location of Policy Registry Keys


Separate registry keys store the individual policies. On Windows, a .reg file sets the registry entries. After being set, the policy information is stored in the following Registry location.
HKEY_LOCAL_MACHINE\Software\BMC Software\Patrol\SecurityPolicy_v3.0

Chapter 5

Security Policies

109

Microsoft Windows

Example of Windows Policy with Registry Keys and String Values


Figure 9 displays a typical site policy implemented in the form of a registry key for a Windows operating system. Figure 10 demonstrates what a security policy looks like when viewed from the Windows Registry Editor. Figure 9 Sample Site Policy Registry Key for Windows

My Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Bmc Software\ PATROL\SecurityPolicy_v3.0\site

Figure 10

Regedit View of Site Policy Registry Key for Windows

110

PATROL Security User Guide

Utilities for Policy Testing and Password Encryption

Utilities for Policy Testing and Password Encryption


This section briefly describes the policy testing tool and the password encryption tools installed with PATROL Security.

esstool Utility
The esstool utility is a command-line, diagnostic tool that provides information about the security configuration.

Capabilities
This utility enables you to perform the following tasks:
I I I I I

discovering which policies and roles have been implemented testing authentication testing encryption methods (algorithm) testing client-server communication viewing utility version information

Location
Table 26 provides the installation path of the esstool utility based upon the operating system. Table 26
Windows Unix

esstool Installation Location


Path %BMC_ROOT\..\common\security\bin_v3.0\<OS> $BMC_ROOT/../common/security/bin_v3.0/<OS>

Operating System

Chapter 5

Security Policies

111

Usage

Usage
The esstool is used to perform the following tasks:
I I I I I

Viewing the Policies and Roles on page 115 Viewing Version Information for Security Modules on page 117 Testing Authentication Configuration on page 126 Testing Encryption Algorithm on page 131 Testing a Secure TCP/IP Channel for the Client and Server on page 145

plc_password Utility
The plc_password utility is the key management utility with which you encrypt and manage (store) key database passwords in the security policies.

Capabilities
This utility enables you to perform the following tasks:
I I I I I I

encrypt passwords using a user-specified file as a unique key assign an encrypted password to a role store encrypted password into a site or application policy change the password for a key database set the mode to attended or unattended restrict who or from what computer an encrypted password can be decrypted

Location
Table 27 provides the installation path of the plc_password utility based upon the operating system. Table 27
Windows Unix

plc_password Installation Location


Path %BMC_ROOT\..\common\security\bin_v3.0\<OS> $BMC_ROOT/../common/security/bin_v3.0/<OS>

Operating System

112

PATROL Security User Guide

Usage

Usage
The plc_password is used to perform the following tasks:
I I

Setting the Attended or Unattended Mode on page 134 Adding or Editing a Password Stored in a Policy on page 135

bmcryptpw Utility
The bmcryptpw utility is the command-line utility with which you can encrypt and verify passwords. The results can be used as a key database password and entered into a policy or as user credentials (labeled password) and entered into the key database of a PATROL application.

Capabilities
This utility enables you to perform the following tasks:
I I I

encrypt passwords using a user-specified file as a unique key verify that a text string is an encrypted password restrict who or from what computer an encrypted password can be decrypted

Location
Table 28 provides the installation path of the bmcryptpw utility based upon the operating system. Table 28
Windows Unix

bmcryptpw Installation Location


Path %BMC_ROOT\..\common\security\bin_v3.0\<OS> $BMC_ROOT/../common/security/bin_v3.0/<OS>

Operating System

Usage
The bmcryptpw is used to perform the following task:
I

Encrypting a Password on page 138

Chapter 5

Security Policies

113

signFile and verifyFile Utilities

signFile and verifyFile Utilities


The signFile and verifyFile utilities are the command-line utilities that can be used to test the process of digitally signing and verifying digital signatures.

Capabilities
These utilities enables you to perform the following tasks:
I I

sign a file verify a file

Location
Table 29 provides the installation path of the signFile and verifyFile utility based upon the operating system. Table 29
Windows Unix

signFile and verifyFile Installation Location


Path %BMC_ROOT\..\common\security\bin_v3.0\<OS> $BMC_ROOT/../common/security/bin_v3.0/<OS>

Operating System

Usage
The signFile and verifyFile utilities are used to perform the following tasks:
I I

Testing Digital Signing on page 141 Testing the Verification of a Digital Signature on page 143

114

PATROL Security User Guide

Policy and Role Information

Policy and Role Information


Policies establish the configuration of security. This section enables you to manage that configuration by describing
I I I

what aspects of security you can control through a policy how to exercise control through changes to policies how to test the effectiveness and success of those changes

Creating a Security Policy


Security policies are created and installed by the installation utility during the installation of PATROL components. During the installation process, the installation utility asks you questions about security. It uses those answers to configure your the security policies on that computer for the applications that you are installing.

WARNING
Creating a security policy is an automated process. BMC Software strongly recommends against your creating a customized policy or modifying a policy through means other than those provided by the installation utility.

Viewing the Policies and Roles


This procedure describes how to view what security an application will use when it starts up.

To Learn Which Policies Are Being Used and Which Roles Are Being Played 1 At a command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

2 Enter esstool policy and desired options. Figure 11 and Figure 12 provide examples
of how to enter the esstool policy command. Figure 11 esstool policy Example on Windows

esstool policy -r server -a -S PATROL\SecurityPolicy_v3.0\site -P PATROL\SecurityPolicy_v3.0\agent

Chapter 5

Security Policies

115

Viewing the Policies and Roles

Figure 12

esstool policy Example on Unix

esstool policy -r server -a -S ../config_v3.0/patrol.plc -P ../config_v3.0/agent.plc

Table 30 lists the options and their arguments. Table 30


Option -r

esstool policy Options


Argument role Description designates the role whose security policy information you want to view Possible roles include client, server, signer, verifier, keystore, encryptor, and authenticator.

-a -b -S -P -? path path

prints out all the security policy for the specified role checks to see that the security module required for the current security level is present and loads it supplies the path of the site policy if it is stored in a location other than the default location supplies the path of the site policy if it is stored in a location other than the default location (Optional) prints usage information and exits

3 Press Enter.
esstool displays the information that you requested in the command. Figure 13 provides sample output for the esstool policy function on Windows. On Unix, the content of the output is the same but the format differs. Figure 13 esstool policy Example Output on Windows

security role: server security level: 2, SSL anonymous site policy: SOFTWARE\BMC\PATROL\SecurityPolicy_v3.0\site application policy: SOFTWARE\BMC\PATROL\SecurityPolicy_v3.0\agemt log level: ERROR,WARNING log file: esi_server.log keyfile: C:\Program Files\BMC\common\security\keys\server.kdb identity: server sksdir: C:\Program Files\BMC\common\security\sks securityDir: C:\Program Files\BMC\common\security\keys logdir: C:\Program Files\BMC\common\security\log_v3.0 bindir: C:\Program Files\BMC\common\security\bin_v3.0\Windows-x86 network security module: C:\Program Files\BMC\common\security\bin_v3.0\Windows-x86\bmcssl.dll BCA API version BCA Version 1.0|ess3.0.5.21|win32|Jan 26 2005|11:42:43 last error:

116

PATROL Security User Guide

Viewing Version Information for Security Modules

Viewing Version Information for Security Modules


PATROL Security supports the following security modules:
I I I

Spyrus Secure Socket Layer (SSL) Diffie-Hellman PATROL ESI

This procedure describes how to view the security module capabilities including version, build dates, identification information, and operating system.

To View the Supported Encryption Algorithms 1 At a command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

2 Enter esstool query security_module. Figure 14 provides an example of how to enter


the esstool query command. Figure 14 esstool query Example on Windows

esstool query bmcssl.dll

Table 31 lists the security modules and their associated files for the supported platforms. Table 31 Security Modules
Windows bmcssl.dll bmcdh.dll bmcesi.dll Unix bmcssl.so bmcdh.so bmcesi.so

Security Module Spyrus Secure Socket Layer (SSL) Diffie-Hellman PATROL ESI

Chapter 5

Security Policies

117

Viewing Version Information for Security Modules

3 Press Enter.
The esstool utility lists the security module capabilities. Figure 15 provides sample output for the esstool query function on Windows. On Unix, the content of the output is the same but the format differs. Figure 15 esstool query Result Example on Windows

doing BMC_LoadModule bmcssl.dll Module bmcssl.dll loaded by user Security Module capabilities (bmcssl.dll) ----------------------------------------version: SSL Module, Version 1.0|ess3.0.5.21|win32|Jan 26 2005|11:43:21 (Domestic) authentication: MUTUAL ciphers v1: des_64_cbc_with_md5, des_192_ede3_cbc_with_md5, rc4_128_with_md5, rc2_128_cbc_with_md5, rc4_128_export40_with_md5, rc2_128_cbc_export40_with_md5 ciphers v2: rc4_128_with_md5, rsa_with_rc4_128_md5, rsa_with_rc4_128_sha, rsa_with_3des_ede_cbc_sha, dhe_rsa_with_3des_ede_cbc_sha, dhe_dss_with_3des_ede_cbc_sha, dhe_rsa_with_des_cbc_sha, dhe_dss_with_des_cbc_sha, des_192_ede3_cbc_with_md5, rc2_128_cbc_with_md5, rsa_with_des_cbc_sha, des_64_cbc_with_md5, dhe_dss_with_rc4_128_sha, rsa_with_rc4_40_md5, rsa_with_rc4_56_sha, rsa_with_des_64_sha, rsa_with_rc2_40_md5, rsa_with_des40_cbc_sha, rc4_128_export40_with_md5, dhe_dss_with_des_64_sha, dhe_dss_with_rc4_56_sha, dhe_dss_export_with_des40_cbc_sha, dhe_rsa_export_with_des40_cbc_sha, rc2_128_cbc_export40_with_md5 ciphers v3: rsa_with_rc4_128_md5, rsa_with_rc4_128_sha, rsa_with_3des_ede_cbc_sha, rsa_with_des_cbc_sha, dhe_rsa_with_3des_ede_cbc_sha, dhe_dss_with_3des_ede_cbc_sha, dhe_rsa_with_des_cbc_sha, dhe_dss_with_des_cbc_sha, dhe_dss_with_rc4_128_sha, rsa_with_rc4_40_md5, rsa_with_rc4_56_sha, rsa_with_des_64_sha, rsa_with_rc2_40_md5, rsa_with_des40_cbc_sha, dhe_dss_with_des_64_sha, dhe_dss_with_rc4_56_sha, dhe_rsa_export_with_des40_cbc_sha, dhe_dss_export_with_des40_cbc_sha authorization: SERVER_ACL

118

PATROL Security User Guide

Authentication and Encryption Information

Authentication and Encryption Information


This section describes how to override the default authentication method supplied by the operating system and select an encryption algorithm. It also discusses how to test these changes.

Specifying an Authentication Provider and Service


Operating systems support different authentication providers and services. Because this information is stored in the policy, the procedures for specifying an authentication provider and service are different for different the platforms. This procedure is divided into two procedures:
I I

how to specify an authentication provider and service in a Windows environment how to specify an authentication provider and service in a Unix environment

NOTE
Regardless of the platform, you can specify only one authentication provider and only one service per policy.

Microsoft Windows
The Microsoft Windows operating system provides a default authentication method, LogonUser, used by ESS3.0. Therefore, when specifying authentication, you can select which authentication service is used by the LogonUser function.

To Specify an Authentication Service 1 Access the Site policy. 2 Navigate to the Authenticator role. If this role does not exist in the site policy, add
it.

Chapter 5

Security Policies

119

Specifying an Authentication Provider and Service

3 Edit the service attribute. If this attribute does not exist, add it. The supported
service values are
LOGON32_LOGON_BATCHis for batch servers, where processes can execute on

behalf of a user without their direct intervention. This type is also for higher performance servers that process many plaintext authentication attempts at a time, such as mail or Web servers. For this logon type, the LogonUser function does not store credentials.
LOGON32_LOGON_INTERACTIVEis for users who will be interactively using the computer, such as a user being logged on by a terminal server, remote shell, or similar process. This logon type has the additional expense of caching logon information for disconnected operations; therefore, it is inappropriate for some client/server applications, such as a mail server. LOGON32_LOGON_SERVICE is for a service-type logon. The service privilege must be enabled for the logon account. LOGON32_LOGON_NETWORKis for high performance servers to authenticate plaintext passwords. For this logon type, the LogonUser function does not store credentials.

4 Save the policy and exit the application that you used to edit the policy.

120

PATROL Security User Guide

Specifying an Authentication Provider and Service

Unix
Some variants of the Unix operating system automatically use either shadow passwords or NIS account databases for authentication. The PATROL Security component enables you to configure it to use Pluggable Authentication Module (PAM) services. To do so, you must determine which PAM services your system uses. Then, in the site policy, you must specify the PAM service that you want to employ.

To Determine Which PAM Services Are in Use


On Unix, PAM Services are specified in a system file stored in the /etc directory. Table 32 lists the platforms on which PATROL Security supports PAM services and the name and location of the PAM configuration file on those platforms. Table 32
Platform AIX HP Linux Solaris
a a

Location of the PAM Configuration File by Operating System


Version 5.2 or later 11.0 or later all 5.6 or later Example of File Location/Type /etc/pam.conf /etc/pam.conf /etc/pam.d /etc/pam.conf

AIX requires operating system patches and the manual installation of the Kerberos PAM module library. For instructions, see AIX Kerberos Support on page 123.

Navigate to the appropriate directory and perform a cat operation on the file, cat pam.conf. Figure 16 provides an example of the files contents. For details about PAM Services, see the documentation for your operating system. Figure 16 pam.conf Example

# #ident @(#)pam.conf 1.16 01/01/24 SMI # # Copyright (c) 1996-2000 by Sun Microsystems, Inc. # All rights reserved. # # PAM configuration # # Authentication management # login auth required /usr/lib/security/$ISA/pam_unix.so.1 login auth required /usr/lib/security/$ISA/pam_dial_auth.so.1 # rlogin auth sufficient /usr/lib/security/$ISA/pam_rhosts_auth.so.1 rlogin auth required /usr/lib/security/$ISA/pam_unix.so.1 ...

Chapter 5

Security Policies

121

Specifying an Authentication Provider and Service

To Set Up Authentication Provider and Service NOTE


On Unix, attribute values are case-sensitive. Enter the value as it appears within the operating systems. For example, if a file lists a service in mixed case such as TelNet, enter the attribute value in mixed case, TelNet.

1 Access the Site policy. The path to the policy is


cd /etc/patrol.d/security_policy_v3.0/site.plc

2 Navigate to the Authenticator role. If this role does not exist in the policy, add it. 3 Edit the provider attribute. If this attribute does not exist, add it.
The only supported authentication providers for Unix that you can specify for this attribute is pam, Pluggable Authentication Module (PAM)

4 Edit the service attribute.


For PAM, the supported service values are listed in /etc/pam.d/pam.conf file. Such services include login, ftp, telnet, passwd, pop, and many others. Figure 17 provides an example of the contents of a Site policy file on Unix. Figure 17 Authenticator Role of Site Policy on Unix

[authenticator] provider = pam service = login

5 Save the policy and exit the application that you used to edit the policy.

122

PATROL Security User Guide

Specifying an Authentication Provider and Service

AIX Kerberos Support


PATROL Security 3.0.05 supports Kerberos authentication on AIX 5.2 or later. However, you must ensure that certain IBM updates have been applied to the operating system. You will also need to configure some files.

Required IBM Updates (APAR)


To support Pluggable Authentication Module (PAM) on AIX 5.2 or later, you must install IBM updates (APARs). Table 33 lists the version and the required updates. Table 33 IBM Updates for AIX 5.2 or Later
APAR none IY66349 IY66349 none Run-Time Environment (RTE) none none bos.rte.libc 5.3.0.2 bos.rte.security 5.3.0.2 bos.rte.libc 5.3.0.2 bos.rte.security 5.3.0.2

Operating System AIX 5.2 32bit AIX 5.2 64bit AIX 5.3 64bit AIX 5.3 32bit

Configuring AIX to Support Kerberos


PATROL Security supports Pluggable Authentication Module (PAM) on AIX 5.2 or later. To configure PATROL Security, you must manually update the pam.conf file with the location of the pam_krb5.so file and update the krb5.conf. For more information about how to select this authentication provider, see Specifying an Authentication Provider and Service in Chapter 5 Security Policies of the PATROL Security User Guide.

To Support AIX NOTE


This procedure involves modifying files in the /etc directory. To perform this procedure, you must have root access.

1 Access /etc/patrol.conf. Copy or record the location of pam_krb5.so. 2 Navigate to the directory that contains pam_krb5.so. 3 Change the file permissions of pam_krb5.so to 755. 4 Access /etc/pam.conf.

Chapter 5

Security Policies

123

Specifying an Authentication Provider and Service

5 Add the service and location information to pam.conf. Figure 18 provides an


example of what the entry will look like. Your services and the path to pam_krb5 may differ from this example. Figure 18
klogin klogin klogin klogin

Example of Reference to pam_krb5 in pam.conf


auth account password session required required required required /auth/lib/security/bmc/pam_krb5 /auth/lib/security/bmc/pam_krb5 /auth/lib/security/bmc/pam_krb5 /auth/lib/security/bmc/pam_krb5

6 Save and exit the file. 7 Navigate to /etc. 8 Access krb5.conf. If it does not exist, create it. 9 Edit the realms stanza to reference the Kerberos Key Distribution Center (KDC)
server and the Kerberos administration server. Figure 19 provides an example of a krb5.conf in which the KDC server and the Kerberos administration server is installed on the server kdc.bmc.com. Figure 19 krb5.conf Example

[libdefaults] default_realm = BMC.COM [realms] BMC.COM = { kdc = kdc.bmc.com admin_server = kdc.bmc.com default_domain = bmc.com } [domain_realm] .bmc.com = BMC.COM bmc.com = BMC.COM [logging] default=FILE:/var/log/krb5lib.log

10 Save and exit the file.

124

PATROL Security User Guide

Specifying an Authentication Provider and Service

Where to go from here


To test the configuration of this authentication, see Testing Authentication Configuration on page 126.

Chapter 5

Security Policies

125

Testing Authentication Configuration

Testing Authentication Configuration


This procedure describes how to verify a user name and password and test the authentication module. The password can be verified by checking the password file, checking the shadow password file, or using the password authentication module (PAM).

To Verify a User Name and Password Using Authentication 1 At a command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

2 Enter esstool authenticator and desired options. Figure 20 provides an example how
to enter the esstool command to test the PAM Login Service. Figure 20 esstool authenticator Example on Windows

esstool authenticator -d my.company.com -u Admin1 -p adminpwd -S Patrol\SecurityPolicy_v3.0\site -P Patrol\SecurityPolicy_v3.0\console

Table 34 list the options and their arguments. Table 34


Option -d

esstool authenticator Options (Part 1 of 2)


Argument domain Description lists the Windows domain authenticates against If no domain is specified, the default is null.

-u

user_name

specifies the user name of the account to be verified This option is required.

-p

password

specifies the password to be verified This option is required.

-n

number

repeat the test the number of times specified by the argument The default is 1.

-w -S -P path path

checks the supplied password against the shadow password supplies the path of the site policy if it is stored in a location other than the default location supplies the path of the site policy if it is stored in a location other than the default location

126

PATROL Security User Guide

Testing Authentication Configuration

Table 34
Option -r

esstool authenticator Options (Part 2 of 2)


Argument provider Description specifies the provider of authentication that is being tested Use this option to test a provider other than the one specified in the authentication role.

-s

service

specifies the service whose authentication is being tested Use this option to test a service other than the one specified in the authentication role.

-?

prints usage information and exits The help option is optional.

3 Press Enter.
esstool displays the results of the authentication test. Figure 21 provides sample output for the esstool authentication function on Windows. On Unix, the content of the output is similar. Figure 21 esstool authentication Results Example on Windows

Site policy = 'SOFTWARE\BMC\Patrol\SecurityPolicy_v3.0\site' Appl policy = 'SOFTWARE\BMC\Patrol\SecurityPolicy_v3.0\console' BAA Version:BAA Module, Version 1.0|ess3.0.5.21|win32|Jan 26 2005|11:44:24 caps: BAA Module, Version 1.0|ess3.0.5.21|win32|Jan 26 2005|11:44:24 caps: NTLM |LogonUser |LOGON32_PROVIDER_DEFAULT |LOGON32_LOGON_BATCH |LOGON32_LOGON_INTERACTIVE |LOGON32_LOGON_SERVICE |LOGON32_LOGON_NETWORK ** Authentication successful **

Chapter 5

Security Policies

127

Selecting an Encryption Algorithm

Selecting an Encryption Algorithm


The default cipher used by ESS 3.0 encryptor is Data Encryption Standard (DES) cipher des-cbc. It is backwards compatible with the encryption method used by ESS2.0 release. You can overwrite the default cipher by using the cipher_type attribute in the Encryptor role of the Site policy. PATROL supports the following encryption algorithms operating in cbc, cfb, ecb, and ofb modes.
I I I I I I

Blowfish CAST Data Encryption Standard (DES) Triple Data Encryption Standard (3DES) RC2 RC4

This procedure describes how to specify which supported encryption algorithm, other than the default, PATROL uses.

To Select an Encryption Algorithm 1 Access the Site policy. 2 Navigate to the Encryptor role. If this role does not exist in the site policy, add it. 3 Edit the cipher_type attribute. If this attribute does not exist, add it.
Table 35 lists the supported encryption algorithms and their corresponding cipher types. Table 35
Blowfish

Supported Encryption Algorithms and Their Cipher Values (Part 1 of 2)


Cipher Type Values bf-cbc bf bf-cfb bf-ecb bf-ofb cast-cbc cast cast5-cbc cast5-cfb cast5-ecb cast5-ofb

Encryption Algorithm

CAST

128

PATROL Security User Guide

Selecting an Encryption Algorithm

Table 35
DES

Supported Encryption Algorithms and Their Cipher Values (Part 2 of 2)


Cipher Type Values des-cbc des des-cfb des-ofb des-ecb des-ede-cbc des-ede des-ede-cfb des-ede-ofb des-ede3-cbc des-ede3 des3 des-ede3-cfb des-ede3-ofb desx rc2-cbc rc2 rc2-cfb rc2-ecb rc2-ofb rc2-64-cbc rc2-40-cbc rc4 rc4-40

Encryption Algorithm

3DES

RC2

RC4

To generate a complete list of supported ciphers types from the encryption module, see Listing the Encryption Algorithms Supported by the Encryption Module on page 130.

4 Save the policy and exit the application that you used to edit the policy.

Chapter 5

Security Policies

129

Listing the Encryption Algorithms Supported by the Encryption Module

Listing the Encryption Algorithms Supported by the Encryption Module


This procedure describes how to view the ciphers types supported by the encryption module.

To View The Supported Encryption Algorithms 1 At a command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

2 Enter esstool query encryption_module, where the encryption module for Windows
is bmcpwk.dll and the encryption module for Unix is libbmcpwk.so.

3 Press Enter.
The esstool utility displays the supported cipher types. Figure 22 provides sample output for the Windows encryption module, bmcpwk.dll. Figure 22 Sample List of Cipher Types for bmcpwk.dll

Security Module capabilities (bmcpwk.dll) ----------------------------------------version: BPW Module, Version 1.0|ess3.0.5.12|win32|MMM dd CCYY|HH:MM:SS crypto: OpenSSL 0.9.7c 30 Sep 2003 bf-cbc|bf|bf-cfb|bf-ecb|bf-ofb| cast-cbc|cast|cast5-cbc|cast5-cfb|cast5-ecb|cast5-ofb| des-cbc|des|des-cfb|des-ofb|des-ecb|des-ede-cbc|des-ede|des-ede-cfb| des-ede-ofb|des-ede3-cbc|des-ede3|des3|des-ede3-cfb|des-ede3-ofb|desx| rc2-cbc|rc2|rc2-cfb|rc2-ecb|rc2-ofb|rc2-64-cbc|rc2-40-cbc| rc4|rc4-40

130

PATROL Security User Guide

Testing Encryption Algorithm

Testing Encryption Algorithm


This procedure describes how to verify that the encryption algorithm and cipher type that you are using is functioning correctly.

To Verify an Encryption Algorithm 1 At a command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

2 Enter esstool encryptor and desired options. Figure 23 provides an example of


testing encryption. Figure 23 esstool encryptor Example

esstool encryptor -c rc2-64-cbc -p mypassword -e test_string

Table 36 list the options and their arguments. Table 36


Option -S -P -c

esstool encryptor Options (Part 1 of 2)


Argument path path cipher name Description supplies the path of the site policy if it is stored in a location other than the default location supplies the path of the site policy if it is stored in a location other than the default location specifies the encryption algorithm. The default is des-cbc. For a list of supported encryption algorithms and cipher types, see Table 35 on page 128.

-p -l -e

password lock_string string

specifies a temporary password for the encryption\decryption test lock string used to perturb password encrypt the string provided as argument Either this option or -d is required.

-d

string

decrypt the string provided as argument Either this option or -d is required.

-D

string

decrypt the results of the encryption option This option requires the -e option.

Chapter 5

Security Policies

131

Testing Encryption Algorithm

Table 36
Option -n

esstool encryptor Options (Part 2 of 2)


Argument number Description repeat the test the number of times specified by the argument The default is 1.

-?

prints usage information and exits The help option is optional.

3 Press Enter.
esstool displays the information that you requested in the command. Figure 24 provides an example of testing encryption. Figure 25 provides an example of testing decryption. Figure 24 esstool encryptor Example of Encryption Command and Output

esstool encryptor -p mypassword -e test_string Creating new policy ... Site policy='..\SecurityPolicy_v3.0\site' Appl policy='..\SecurityPolicy_v3.0\agent' ModuleName C:\..\security\bin_v3.0\Windows-x86\bmcpwk.dll BPW Module Version 1.0|ess3.0.5.13|win32|Nov 10 2004|11:33:36 test_string->pGkxdmT3nGI+YoAzXk30l2EXIZX ... Anticipated decryption error -1.

Figure 25

esstool encryptor Example of Decryption Command and Output

esstool encryptor -p mypassword -d pGkxdmT3nGI+YoAzXk30l2EXIZX Creating new policy ... Site policy='..\SecurityPolicy_v3.0\site' Appl policy='..\SecurityPolicy_v3.0\agent' ModuleName C:\..\security\bin_v3.0\Windows-x86\bmcpwk.dll BPW Module Version 1.0|ess3.0.5.13|win32|Nov 10 2004|11:33:36 pGkxdmT3nGI+YoAzXk30l2EXIZX->test_string ... Anticipated decryption error -1.

The Anticipated decryption error in both the examples results from an internal esstool test that is not expected to succeed. Disregard it.

132

PATROL Security User Guide

Key Database and Password Information

Key Database and Password Information


This section describes how to designate a key database for different policy roles. It also explains how to encrypt passwords and insert them into policy roles. Finally, this section discusses how to test these changes.

Designating a Key Database for an Applications Role


The key database that an application uses is specified in the role section of a security policy. For each role of the policy, a different key database can be specified. This procedure describes how to designate in a site or application policy a key database for the role which an application plays.

To Designate a Key Database for an Application 1 Access the Site or an application policy for the application that runs on the current
computer. For example, if you are running a PATROL Agent on this computer, you should access the server security policy.

2 Navigate to the role of the application for which you want to designate a key
database. If the desired role does not exist in the policy, add it.

NOTE
You can specify a key database in the role section for the application that runs on this computer. You can also specify a key database in the respective role section of each application with which this application interacts.

3 Edit the keyfile attribute by setting the value equal to the key database filename
(*.kdb). If this attribute does not exist, add it.

4 Repeat step 2 and step 3 for the role of each application with which the application
that owns this policy file interacts.

5 Save the policy and exit the application that you used to edit the policy. Where to go from here
If you want the to interact in unattend mode, you must add a password for each keyfile attribute that you set. For information about how to add passwords to policies, see Adding or Editing a Password Stored in a Policy on page 135.

Chapter 5

Security Policies

133

Setting the Attended or Unattended Mode

Setting the Attended or Unattended Mode


WARNING
To guarantee the security of an application running in unattended mode, you must maintain physical security of the computer by placing it in an area with restricted access. You must also insure the operational security of its operating system by closely controlling those ports which are open to the outside and shutting down unnecessary services that can be exploited. These precautions are necessary because the password is stored on the computer. Although the password is encrypted, no means of password encryption is indecipherable, and thus the computer is at risk of being compromised.

The mode determines whether a user must manual type in a password to start up an application. Setting the mode involves storing a password in a security policy or removing a password from one. Figure 26 provides an example of adding a password to an application policy and setting the mode to unattended using the plc_password utility. Figure 26 plc_password Example Setting Mode to Unattended

plc_password -r server -P \etc\Patrol.d\SecurityPolicy_v3.0\agent -m unattended -n Da$h4ca$h -f scramble.bin -k agent.kdb

For information about how to encrypt a password and save it to a security policy, see Adding or Editing a Password Stored in a Policy on page 135.

NOTE
The PATROL Agent on OpenVMS and PATROL Agent on iSeries (AS400) run in unattended mode only.

134

PATROL Security User Guide

Adding or Editing a Password Stored in a Policy

Adding or Editing a Password Stored in a Policy


Policy files can contain passwords for many different roles. The usage and storage of passwords determines the security and ease-of-use of a PATROL installation. This procedure describes how to create an encrypted password using a secret key for a key database and then install the password and a reference to the secret key in the appropriate role section of a policy file.

To Add or Change a Password in a Policy 1 At a command-line prompt, change to the directory that contains the
plc_password utility. The path to the plc_password is given in Location on page 112.

2 Type in the plc_password command string for your operating system with the
desired options and arguments. Figure 27 provides an example of adding a password to a site policy and setting the mode to unattended. Figure 27 plc_password Example Setting Mode to Unattended

plc_password -r server -S \etc\Patrol.d\SecurityPolicy_v3.0\site -m unattended -n pa$$3word -f padlock.jpg -k agent.kdb

Table 37 lists the available options and their arguments. Table 37


Option -r

plc_password Utility Options (Part 1 of 2)


Argument role Description designates the role within the security policy whose password you want to edit Policy sections (Roles) that support the password attribute are client, server, signer, verifier, and keystore.

-P

policy

the security policy whose password you want to edit For the location of security polices, see Location of Policy Files on page 107.

Chapter 5

Security Policies

135

Adding or Editing a Password Stored in a Policy

Table 37
Option -m

plc_password Utility Options (Part 2 of 2)


Argument mode Description specifies whether you want to force a user to enter a password when starting\restarting the component with the associated role specified by the role option attended removes policy password field; forcing users to enter a password every time a process that uses this policy must be restarted unattended creates policy password field; allowing any processes that use this policy to start up without user intervention, in other words, a user entering a password through the keyboard Unattended mode options -f key_material_file used as a secret key to encrypt the password; this file can be any format (*.jpg, *.txt, *.bin, and so forth) but should be larger than 1024 bytes and is required to decrypt the password -H hostname or ip_address lock by IP address; creates a password that is readable (can be decrypted) only from the IP address -u username --- lock by user name; creates a password that is readable (can be decrypted) only by user -w unchange key database password; if a role password and the key database password are the same, this option allows you to change the role password without changing the key database password -o old_key_database_password the old password in plain text -n new_password the new password, in plain text, to be encrypted -k key_database_file the key database whose password you want to change The mode option is optional.

-v

prints the version of the utility and exits The version option is optional.

-h

prints usage information and exits This help option is optional.

136

PATROL Security User Guide

Adding or Editing a Password Stored in a Policy

3 Press Enter.
The utility performs the specified action and displays the message. Figure 28 displays a probable result from a command similar to the one in Figure 27. Figure 28 plc_password Example of Policy File Contents on Unix

... [server] keyfile = agent.kdb password = kljf;lji9u8yu39miu-u3, padlock.jpg [signer] ...

Chapter 5

Security Policies

137

Encrypting a Password

Encrypting a Password
Policy files can contain passwords for many different roles. The usage and storage of passwords determines the security and ease-of-use of a PATROL installation. This procedure describes how to encrypt a password using the bmcryptpw utility. It also describes how to verify that a password in encrypted format is valid.

To Encrypt a Password 1 At a command-line prompt, change to the directory that contains the bmcryptpw
utility. The path to the bmcryptpw is given in Location on page 113.

2 Type in the bmcryptpw command with the desired options and arguments.
Figure 29 provides an example of encrypting a password. Figure 29 bmcryptpw Example on Windows

bmcryptpw -m ..\..\keys\company_logo.jpg -e

Table 38 lists the available options and their arguments. Table 38


Option -m

bmcryptpw Utility Options


Argument key_material_file Description used as a secret key to encrypt the password; this file can be any format (*.jpg, *.txt, *.bin, and so forth) but should be larger than 1024 bytes and is required to decrypt the password a password in encrypted format lock by IP address; creates a password that is readable (can be decrypted) only from the IP address lock by user name; creates a password that is readable (can be decrypted) only by user prompt user for password to encrypt generate key material prints the version of the utility and exits The version option is optional.

-V -H -u -e -g -v

encrypted_pswd hostname or ip_address user_name

-h

prints usage information and exits This help option is optional.

138

PATROL Security User Guide

Encrypting a Password

3 Press Enter.
bmcryptpw prompts you to enter the password.

4 Type the password that you want to encrypt and press Enter.
bmcryptpw encrypts the password and prints it out in encrypted form. Figure 30 displays a probable result from a command similar to the one in Figure 29. Figure 30 bmcryptpw Results Example on Windows

Enter password: ******** Encoded passwd: 4a2b5466c070e2b43bc4290eb66558b8d801dc3fbd513949

To Verify that a Password in Encrypted Format is Valid 1 At a command-line prompt, change to the directory that contains the bmcryptpw
utility. The path to the bmcryptpw is given in Location on page 113.

2 Type in the bmcryptpw command with the desired options and arguments.
Figure 29 provides an example of verifying that an encrypted password is valid. Figure 31 bmcryptpw Test Example on Windows

bmcryptpw -m ..\..\keys\company_logo.jpg -V 4a2b5466c070e2b43bc4290eb66558b8d801dc3fbd513949

Table 38 lists the available options and their arguments.

3 Press Enter.
bmcryptpw verifies that the string that you pass is a password that was encrypted using the key material file. Figure 32 bmcrypt Test Results Example on Window

password decoded: valid

Chapter 5

Security Policies

139

Signer and Verifier

Signer and Verifier


This section discusses the need for digitally signing files and verify digital signatures. It also describes how to test the digital signing and digital verification of a signature using the command-line utilities signFile and verifyFile.

Purpose
When important data (file, BLOB, etc.) is stored in insecure locations or transported by insecure means, it is useful to have a method of verifying that the documents have not been changed in any way in the interim. Signing a file and then verifying it when it arrives at its destination is one such method.

Operation of Signing
Digitally signing a file involves generating a checksum, or hash, of the entire file from top to bottom. A properly designed hashing algorithm produces a checksum value of the document which has two properties.
I I

If even a single bit of the document is changed, the checksum value is changed. It is very difficult to compose a separate document that produces the same hash value. (Such documents exist. However, one cannot identify them all by working backwards starting with the hash value).

The checksum value of the document is then encrypted with the private key of a trusted entity creating a digital signature. The process of signing does not change the signed file but rather creates an additional file, called a signature file. The signature (*.sgn) file contains the following data:
I I

a signature (the encrypted check sum) a certificate that corresponds to private key of the signer

The file and its signature file are kept together as a pair. The signer's certificate contained in the signature file is used during verification to procure the public key to the signature. To ensure that the public key is genuine, the user first establishes a chain of trust between the signer's certificate and the certificate of a trusted Certificate Authority.

140

PATROL Security User Guide

Operation of Verifying

Operation of Verifying
The integrity of the document can be verified by the receiver of the signed file by 1. decrypting the checksum value with the public key of the trusted entity 2. generating a checksum value of the file 3. comparing the receivers checksum value to the decrypted checksum value that accompanied the file The two checksum values should be equal. If they do not, the document has been altered in some way. Verifying a file ensures that the content is unchanged and that the owner of the private key signed the content.

Testing Digital Signing


This procedure describes how to test digital signing using the command-line utility signFile. Files signed by this utility should have their signatures verified by the command-line utility verifyFile.

To Sign a File 1 At a command-line prompt, change to the directory that contains the signFile
utility. The path to the signFile utility is given in Location on page 114.

2 Enter signFile and desired options. Figure 33 provides an example of digitally


signing a file. Figure 33 signFile Example

signfile ..\..\financial_strat\secretplan.txt -s ..\..\digisigs\ -V

Table 39 list the options and their arguments.

Chapter 5

Security Policies

141

Testing Digital Signing

Table 39
Option file -s

signFile Options
Argument file_name path Description the name and location of the file that you want to digitally sign specifies the location where the utility creates the signature (*.sgn) file If you do not specify the signature directory, the utility creates the signature file in the same directory as the file to be signed.

-S -P -a

path path

supplies the path of the site policy if it is stored in a location other than the default location supplies the path of the application\signer policy if it is stored in a location other than the default location signs the file according to the PKCS# 1 standards The default signature format is a legacy, BMC Software proprietary format.

-v -V -h

displays the version of the utility prints out the options and arguments that it uses in the signing process prints usage information and exits The help option is optional.

3 Press Enter.
The utility creates the digital signature. Figure 34 displays the results from a command similar to the one in Figure 33. Figure 34 signFile Example of Results

Site policy :SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\site Apps policy :SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\signer Object to sign:..\..\financial_strat\secretplan.txt Signature path:..\..\digisigs\ Object signed.

142

PATROL Security User Guide

Testing the Verification of a Digital Signature

Testing the Verification of a Digital Signature


When any digitally signed file is verified, the utility uses the certificate and its public key contained within the signature file to perform the verification. This procedure describes how to test digital signature verification using the command-line utility verifyFile. Signatures verified by this utility should be generated by the command-line utility signFile.

To Verify a Digital Signature 1 At a command-line prompt, change to the directory that contains the verifyFile
utility. The path to the verifyFile utility is given in Location on page 114.

2 Enter verifyFile and desired options. Figure 35 provides an example of digitally


signed file being verified. Figure 35 verifyFile Example

verifyfile ..\..\financial_strat\secretplan.txt -s ..\..\digisigs\ V

Table 40 list the options and their arguments. Table 40


Option file -s

verifyFile Options (Part 1 of 2)


Argument file_name path Description the name and location of the file whose digital signature that you want to verify specifies the location of the signature (*.sgn) file to be used for verification of the file If you do not specify the signature directory, the utility looks in the same directory as the file to be verified.

-S -P -a

path path

supplies the path of the site policy if it is stored in a location other than the default location supplies the path of the application\verifier policy if it is stored in a location other than the default location verifies signature files that were created according to the PKCS# 1 standards only Omitting this option permits the utility to verify signature files that conform to either PKCS# 1 format or the legacy, BMC Software proprietary format. Omitting this option is recommended.

-v

displays the version of the utility

Chapter 5

Security Policies

143

Testing the Verification of a Digital Signature

Table 40
Option -V -h

verifyFile Options (Part 2 of 2)


Argument Description prints out the options and arguments that it uses in the verifying process prints usage information and exits The help option is optional.

3 Press Enter.
The utility checks the digital signature and displays the message, Verified OK. Figure 36 displays the results from a command similar to the one in Figure 35. Figure 36 verifyFile Example of Results

Site policy :SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\site Apps policy :SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\verifier Object to verify:..\..\financial_strat\secretplan.txt Signature path:..\digisigs Verified OK

144

PATROL Security User Guide

Client-Server Communication

Client-Server Communication
This section describes how to test the SSL secure communication channel that is established for an application operating in client role to communicate with an application operating in a server role. The primary benefit of this test is to verify that the security policy is properly configured to be used by PATROL applications. This test cannot be performed at security level 0 because that level does not employ secure channel communication.

Testing a Secure TCP/IP Channel for the Client and Server


This procedure tests the communication modules shipped with PATROL Security. They include
I I

Spyrus Secure Socket Layer (SSL) Diffie-Hellman

This procedure describes how to start a client and server using the esstool and then send messages from the client to the server to demonstrate that the connection works.

To Start an esstool Server 1 Access a command-line prompt. 2 At the command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

3 Enter esstool server and desired options. Figure 37 provides an example of starting
a test server. Figure 37 esstool server Example Command on Windows

esstool server -h Tron -L 2 -S Patrol\SecurityPolicy_v3.0\site -P Patrol\SecurityPolicy_v3.0\agent

Table 41 lists the options and their arguments.

Chapter 5

Security Policies

145

Testing a Secure TCP/IP Channel for the Client and Server

Table 41
Option -h

esstool server Options


Argument hostname Description specifies a computers host name if you are not starting an esstool server a computer other than the one from which you are issuing the command specifies the port on which the server listens The default is 4443.

-p

portnumber

-s -L

service_name security_level

assigns a service name to the esstool server process other than the default, esstool specifies the security level at which to run the esstool server The esstool server must run at a security level greater than 0.

-S -P -V -n

path path version_number

specifies the path to the site policy specifies the path to the application policy displays the version number of the esstool server module sets communication to nonblocking input\output mode, which allows the computer to service other connections This option is for developing testing and should not be employed.

-?

prints usage information and exits This help option is optional.

If you specify level 3 or 4 and you have not set the server role to run in unattend mode, the esstool process prompts you for the password to its key database. Enter the password and click OK.

146

PATROL Security User Guide

Testing a Secure TCP/IP Channel for the Client and Server

Figure 38 displays startup messages. Figure 38 esstool server Example Startup Messages

C:\Program Files\BMC Software\common\security\bin_v3.0\Windowsx86>esstool server -L 1 host: localhost, port 4443, sprinc mysprinc Creating new policy ... Site policy = 'SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\site' Appl policy = 'SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\agent' ModuleName C:\Program Files\BMC Software\common\security\bin_v3.0\Windows-x86\bmcdh.dll client doing connect on 664 socket accepted new fd 656 ********************Starting Accept******************** **************New session established sid: 512**************** session established , doing BCA_Read ---recv: 11, Message 1

Chapter 5

Security Policies

147

Testing a Secure TCP/IP Channel for the Client and Server

To Start an esstool Client 1 Access a command-line prompt. It must be a separate command prompt/shell
than the one used for the esstool server.

2 At the command-line prompt, change to the directory that contains the esstool
utility. The path to the esstool utility is given in Location on page 111.

3 Enter esstool client and desired options. Figure 39 provides an example of starting a
test client. Figure 39 esstool client Example Command on Windows

esstool client -L 2 -S Patrol\SecurityPolicy_v3.0\site -s 2

Table 42 lists the options and their arguments. Table 42


Option -h

esstool client Options (Part 1 of 2)


Argument hostname Description specifies a computers host name if you are not starting an esstool server a computer other than the one from which you are issuing the command specifies the port on which the server listens The default is 4443.

-p

port_number

-s -L

time_in_seconds security_level

specifies the amount of time (in seconds) that the client waits before trying to connect to the server specifies the security level at which to run the esstool server The esstool server must run at a security level greater than 0.

-S -P -V -r -u -k

path path version url string path

specifies the path to the site policy specifies the path to the application policy displays the version number of the esstool server module performs an HTTP GET request assigns user-defined text string (also referred to as an identity) to the process specifies the path to the key file

148

PATROL Security User Guide

Testing a Secure TCP/IP Channel for the Client and Server

Table 42
Option -n

esstool client Options (Part 2 of 2)


Argument Description sets communication to nonblocking input\output mode, which allows the computer to service other connections This option is for developing testing and should not be employed.

-?

prints usage information and exits This help option is optional.

4 If you specify level 3 or 4 and you have not set the client role to run in unattend
mode, the esstool process prompts you for the password to its key database. Enter the password and click OK. Figure 40 displays startup messages. Figure 40 esstool client Example Startup Messages

C:\Program Files\BMC Software\common\security\bin_v3.0\Windowsx86>esstool server -L 1 host: localhost, port 4443, sprinc mysprinc Creating new policy ... Site policy = 'SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\site' Appl policy = 'SOFTWARE\BMC Software\Patrol\SecurityPolicy_v3.0\agent' ModuleName C:\Program Files\BMC Software\common\security\bin_v3.0\Windows-x86\bmcdh.dll client doing connect on 664 socket accepted new fd 656 ********************Starting Accept******************** **************New session established sid: 512****************

Chapter 5

Security Policies

149

Testing a Secure TCP/IP Channel for the Client and Server

To Test Communication Between Client and Server 1 Access the command-line prompt where you started the esstool client. 2 Type a text string, such as Message 1, and press Enter. 3 Access the command-line prompt where you started the esstool server and
observer the message. Figure 41 displays an example of what the esstool server would display. Figure 41 esstool server Example of Message Received from esstool client

session established , doing BCA_Read ---recv: 11, Message 1

To Stop the Client 1 Access the command-line prompt where you started the esstool client. 2 Press CTRL + C. To Stop the Server 1 Access the command-line prompt where you started the esstool server. 2 Press CTRL + C.

150

PATROL Security User Guide

Policy Migration

Policy Migration
The goal of the migration process is to preserve the customizations that you have made to existing security content such as, key databases, acquired key pairs, unattended passwords, and other aspects. PATROL Security 3.0.05 further extends the deployment of ESS3.0 by providing migration capability for PATROL Security 1.2.07, which contained ESS2.0 policy information. This fundamental policy migration requirement preserves existing security configuration and transfers certain configuration attributes into a new ESS3.0 policy. The migration process copies ESS2.0 policy attributes into the ESS3.0 policy configuration.

PATROL Security versus Extended Security System


PATROL Security 3.0.05 contains the Extended Security System 3.0.05 (ESS3.0.05). PATROL Security 1.2.07 contains the Extended Security System 2.0 (ESS2.0). This section refers to the ESS version rather than the PATROL Security versions.

ESS 3.0.00 and ESS 3.0.05


ESS 3.0.05 release is an extension of the ESS 3.0 release and will be installed as 3.0 version. Due the common feature set of 3.0 and 3.0.05 ESS versions, these releases utilize the common ESS3.0 security policy version, designated by the suffix _v3.0. All ESS 3.0.x libraries and binaries are compatible and can be replaced by later versions of 3.0.x.

Chapter 5

Security Policies

151

Migration Process

Migration Process
As part of the installation process, PATROL Security performs the following steps to migrate information from version 2.0 to 3.0. 1. Detects a ESS2.0 policy. 2. Scans ESS2.0 policy for the replicated parameters. The duplicate parameters are replaced by the parameters stored in a common role section of the site policy. The ESS3.0 policy template file will supply required ESS 3.0 configuration information.

NOTE
The migration process will run only in the absence of ESS3.0 policy. After an ESS3.0 policy has been created, the migration will not be initiated.

3. ESS3.0 products will reside in versioned directories. The following policy attributes and their corresponding values are associated with ESS2.0 and are will not be migrated and are not referenced in ESS 3.0 policy.
bindir bindir64 logdir lib config

To ensure independent operation of ESS2.0 and ESS3.0.x releases, the ESS3.0 product components use locations with versioned suffixes. The suffix for ESS3.0 is _v3.0.
bindir_v3.0 bindir64_v3.0 logdir_v3.0 lib_v3.0 config_v3.0

If the migration process detects the existence of the ESS2.0 pamservice attribute in either the client or server roles, the process will create an Authenticator role with a provider attribute and a service attribute and transfer the value from the pamservice attribute to the provider and service attributes as shown in Figure 42. Figure 42 Result of the Migration of the pamservice Attribute

[authenticator] provider = pam service = service_name

152

PATROL Security User Guide

Migrate or Overwrite

Migrate or Overwrite
The installation process provides you with the ability to control whether it overwrites an earlier version of security. Through the use of the Overwrite checkbox, you can choose whether to
I

preserve and migrate the existing, customized security configuration or

create new security configuration and overwrite any existing configuration

For more information about the installation process and the ability to overwrite or preserve security configuration, see Installation Process on page 48.

WARNING
Overwrite your existing PATROL Security content and configuration only if you want to start over with demo certificates. BMC Software does not recommend overwriting existing security.

Migration Scenarios
The following scenarios describe some common conditions and the behavior of the migration process.

Choose Not to Overwrite Existing Security


During the installation process, if you choose not to overwrite the existing security, the migration process looks for the ESS2.0 policy information. If the 2.0 policy information exists and there is no 3.0 policy, then the process migrates the 2.0 policy information during the installation of ESS3.0.05 and the creation of 3.0 policy. If the 3.0 policy information already exists, then the process does not attempt to migrate the 2.0 information.

Choose to Overwrite Existing Security


During the installation process, the 3.0 policy is created using the default information specified by the installation package.

Chapter 5

Security Policies

153

Migration Scenarios

Common Scenarios
Table 43 describes some common installation/migration scenarios. Table 43
Scenario Installing ESS3.0 on a new computer

Installation and Migration Scenarios


Overwrite Checkbox checked not checked Results ESS3.0 is installed with default settings ESS3.0 is installed with default settings ESS3.0 is installed with default settings. Security customizations are discarded. ESS2.0 policy information is migrated to ESS3.0 ESS3.0 is installed with default settings. Security customizations are discarded. No change is made to the current settings. ESS3.0 is installed with default settings. Security customizations are discarded. No change is made to the current settings.

Installing ESS3.0 on a computer with ESS2.0 for the 1st time

checked

not checked Installing ESS3.0 on a computer with ESS2.0 subsequent installations (not the 1st time) Installing ESS3.0 on a computer with ESS3.0 for the 1st time checked

not checked checked

not checked

154

PATROL Security User Guide

Chapter

6
156 157 161 164 166 166 168 168 170

Configuration Files
This chapter describes the PATROL configuration files that store additional security information not contained in the certificates, key databases or security policies. This chapter presents the following topics: PATROL Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . patrol.conf. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . config.default . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Working with Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the SSL access File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Operating System and Application-Specific Configurations . . . . . . . . . . . . . . . . . . . Configuring the dlls.conf for PATROL for Unix. . . . . . . . . . . . . . . . . . . . . . . . . . . Using PATROL Event Manager Applications with PATROL Security. . . . . . . .

Chapter 6

Configuration Files

155

PATROL Configuration Files

PATROL Configuration Files


The PATROL configuration files listed in Table 44 contain parameters that pertain to security implementation. Table 44
Filename patrol.conf

PATROL Configuration Files That Contain Security Information


Description contains the Extended Security Interface (ESI) configuration stanzas When you install security (and choose a security level), the installation process updates patrol.conf and backs up the original version of the file. For more information, see patrol.conf on page 157.

config.default

is the default configuration file for the PATROL Agent When you install security (and choose a security level), the installation process updates config.default and backs up the original version of the file. For more information, see config.default on page 161.

access

is a file that stores SSL access control list information for applications operating in the server role and running at security level 4 For more information, see Configuring the SSL access File on page 166.

dlls.conf

lists the .dll files necessary for KMs to work with the PATROL Agent For more information, see Configuring the dlls.conf for PATROL for Unix on page 168.

156

PATROL Security User Guide

patrol.conf

patrol.conf
The patrol.conf file contains the Extended Security Interface (ESI) configuration stanza. (For further information, see Extended Security Interface (ESI) on page 159.) When you install PATROL and choose a security level, the installation process updates patrol.conf and backs up the original file. The security features controlled by patrol.conf include
I I I I

prevent or permit the execution of PSL commands from SNMP monitor allow or deny commits from the PATROL Console 3.x running in developer mode allow or deny a PATROL Console 3.x running in developer mode to connect prevent or permit the system output window from executing operating system commands

Location
Table 45 provides the location of the patrol.conf file for each operating system. Table 45
Windows Unix

Location of patrol.conf File


Path %PATROL_HOME%\..\common\patrol.d /etc/patrol.d

Operating System

Chapter 6

Configuration Files

157

patrol.conf

Security-Related Contents
Table 46 lists an example of configuration data and a description of each section of data in patrol.conf. (Please note the legend at the end of the table.) For more detail on the patrol.conf file, see the PATROL Agent Reference Manual. Table 46 Security Configuration Data of patrol.conf File
Level 0 (Basic Security) C I A I Level 1 Level 2 Level 3 Level 4

Stanza and Parameter Name [ESI] esi_lib

C I

A I

C I

A I

C I

A I

C I

A I

Description ESI stanza name


I

specifies the path to the PATROL ESI library for the console and agent install location example is /home/seqqa/PATROL3.3/ Solaris25-sun4/bin/bmcesi.so

[AGENT] allowsnmpexecute T T T T T T F F F F

agent stanza name permits or prevents the ability to run PSL commands from an SNMP network monitor console stanza name T T T T T T F F F F permits or prevents the console from committing any KM changes to any connected agents (if you remove this right, PATROL disables menus that provide access to KM commit operations) permits or prevents a console from establishing a developer mode connection to an agent permits or prevents a user from entering operating system commands into the PATROL system output window

[CONSOLE] allowcommit

allowdeveloper

allowsysoutputexec

C = console A = agent I = installed T = true (value) F = false (value)

158

PATROL Security User Guide

patrol.conf

Extended Security Interface (ESI)


The ESI pluggable security component specifies the PATROL Security plug-in used to ensure privacy of communications between PATROL components. The installation process installs the libraries and sets the appropriate variables in patrol.conf. In the patrol.conf file, the esi_lib32 and esi_lib64 variables specify the ESI library location for 32-bit or 64-bit installed products, respectively. Table 47 describes these variables. Table 47
Variables esi_lib

ESI Variables in patrol.conf File


Valid Values the default path of the ESI library or none if no ESI library is being used The default value for the esi_lib variable is none. Description specifies an ESI library to use for authentication and encryption

esi_lib32

the path of the 32-bit ESI library or none if no specifies a 32-bit ESI library to use for authentication and encryption ESI library is being used The default value for the esi_lib32 variable is none.

esi_lib64

the path of the ESI library or none if no ESI library is being used The default value for the esi_lib64 variable is none.

specifies a 64-bit ESI library to use for authentication and encryption

Values and Security Levels


At security level 0, the installation process sets all the ESI variables to none. At security levels 1-4, the installation process sets the ESI variables to the location of the libraries: bmcesi.so on Unix and bmcesi.dll on Windows, which is the secure channel provider for PATROL.

Unix
On Unix, the ESI variables appear in the patrol.conf file as shown in Figure 43. Figure 43 patrol.conf File Example of the ESI Section on Unix

#[ESI] esi_lib = $PATROL_HOME/common/security/bin/OS/bmcesi.so esi_lib32= $PATROL_HOME/common/security/bin/OS/bmcesi.so esi_lib64= $PATROL_HOME/common/security/bin/OS/bmcesi.so

Chapter 6

Configuration Files

159

patrol.conf

Windows
On Windows, the ESI variables appear in the patrol.conf file as shown in Figure 44. Figure 44 patrol.conf Example of the ESI Section on Windows

#[ESI] esi_lib = %PATROL_HOME%\common\security\bin\<OS>\bmcesi.dll esi_lib32= %PATROL_HOME%\common\security\bin\<OS>\bmcesi.dll esi_lib64= %PATROL_HOME%\common\security\bin\<OS>\bmcesi.dll

For more information about using an ESI pluggable security component, see the PATROL API Reference Manual and the PATROL Agent Reference Manual.

160

PATROL Security User Guide

config.default

config.default
The config.default file is the default configuration file for the PATROL Agent. When you install PATROL and choose a security level, the installation process updates config.default and backs up the original file. The security features controlled by config.default include
I I I

sets the access control list determines the communication protocol enables the ESI library

Location
Table 48 provides the location of the config.default file for each operating system. Table 48
Windows Unix

Location of config.default File


Path %PATROL_HOME%\lib $PATROL_HOME/lib

Operating System

Chapter 6

Configuration Files

161

config.default

Security-Related Contents
Table 49 describes security-related parameters in the config.default file. Table 49
Description /AgentSetup/accessControlList This variable lists which user names may */*/CDOPS */*/CDOPS */*/CDOPS */*/COP R R R be used by which consoles when connecting to an agent. The format is a comma-separated list of entries, with each entry being of the form UserName/HostName/Mode.
I

Agent and Console Features in config.default File (Part 1 of 2)


Basic Security Level 1 Level 2 Level 3 Level 4 */*/COP

UserName is the name of a local account that the connecting console may request to use. UserName may be either a single asterisk (*) (meaning that any user name is allowed, assuming the account exists), or the actual name of the account. HostName is the console that is authorized to connect to the agent. HostName may be a single asterisk (*)(meaning that all hosts are allowed to connect), the actual name of a host (indicating that this entry is for that host only), or a wildcard specification, in which the first character is a single asterisk (*) with other characters following.

Mode is a list of zero or more of the characters C, D, O, P, and A (see legend). Mode indicates that the host is authorized to connect to the agent in a particular mode and log on as that user. /AgentSetup/PortConnectType This variable allows you to select the communication protocols UDP, TCP, or both when binding to a port. UDP/ TCP UDP/ TCP UDP/ TCP TCP TCP

162

PATROL Security User Guide

config.default

Table 49
Description

Agent and Console Features in config.default File (Part 2 of 2)


Basic Security Level 1 Level 2 Level 3 Level 4

/AgentSetup/BindToAddress blank a This variable allows you to bind the PatrolAgent to a specific network card on a machine with more than one network card. /AgentSetup/security/ExtendedSecurityEnabled This variable indicates when the ESI is yes enabled. If this variable is set to yes, but the ESI library could not be found or loaded, the agent will exit. PATROL Roles Used by ACL C = Configure D = Developer O = Operator P = PATROL Event Manager S = System Output R = Operator Overwrite A = Anonymous Communication Protocols TCP = Transmission Control Protocol UDP = User Datagram Protocol
a

yes

yes

yes

yes

The unfilled entries in config.default are empty strings.

Chapter 6

Configuration Files

163

access

access
The SSL access file stores access control list (ACL) information for PATROL applications operating in the server role and running at security level 4. For security levels 3 or lower, it is not used. Within the file, users are identified by e-mail address.

WARNING
The purpose of this access control list is to determine which user can connect to the server by means of an SSL connection. When a user is denied access by this file, that user is completely locked out of that computer. The user cannot even establish a connection to the server.

Operation
The server application determines whether to grant or deny access by comparing the values in the allow and deny parameters of the access file with the Distinguished Name associated with the users certificate. For more information about the Distinguished Name, see Table 20 on page 90.

Location
Table 50 provides the location of the access file for each operating system. Table 50
Windows Unix

Location of access File


Path %PATROL_HOME%\..\common\security\keys $PATROL_HOME/../common/security/keys

Operating System

164

PATROL Security User Guide

access

Security-Related Contents
Table 51 describes security-related parameters in the access file. Table 51 Configuration Data in access File
Description SSL server stanza name designates which users are allowed access This parameter supports the wildcards asterisk (*) for many characters and question mark (?) for a single character. DENY_ACL designates which users are denied access This parameter supports the wildcards asterisk (*) for many characters and question mark (?) for a single character.

Stanza and Parameter [SSL_SERVER] ALLOW_ACL

Precedence
The DENY_ACL parameter takes precedence over the ALLOW_ACL parameter. If a user meets the criteria specified in both parameters, the user will be denied access.

Defaults
The installation process installs an access file in which the parameters are set to allow access to all users (ALLOW_ACL = *) and deny access to no one (DENY_ACL = ). This file overrides the default behavior of PATROL Security on a server at level 4, which is to deny access to all add allow access to no one.

WARNING
If the access file is deleted from a computer running a PATROL application in the server role such as the PATROL Agent, no other PATROL applications (PATROL Console, Console Server, PATROL Agent) will be able to connect to the application with the missing file.

Chapter 6

Configuration Files

165

Working with Configuration Files

Working with Configuration Files


This section describes tasks that you may need to perform in the configuration files described in the previous sections.

Configuring the SSL access File


This procedure describes how to edit the access file for use by a PATROL application operating as a server and running at security level 4.

To Edit the SSL access File 1 At a command line prompt, change to the keys directory, which contains the access
file. The path to the file is given in Location on page 164.

2 Open the SSL access file in the text editor of your choice. 3 Navigate to the ALLOW_ACL parameter and enter the e-mail address or addresses
of users to whom you want to grant access.
I

If you want to greatly restrict access, list only the users who require access to the server. Figure 45 provides an example. access File Example Restricting Access to Two Users

Figure 45

[SSL_SERVER] ; ALLOW_ACL = me@company.com, mysupervisor@company.com DENY_ACL =


I

If you want to provide access to a group or range of users with similar e-mail addresses, use patterns with wild cards: * and ?. Figure 46 provides an example. If you want to allow everyone access, enter an asterisk *.

4 Navigate to the DENY_ACL parameter and enter the e-mail address or addresses of
users to whom you want to explicitly deny access. Otherwise, leave this field blank. Figure 46 provides an example.

166

PATROL Security User Guide

Configuring the SSL access File

Figure 46

access File Example Allowing Access to a Group and Denying Access to an Individual User

[SSL_SERVER] ; ALLOW_ACL = *@company.com, *@subsidiary.com DENY_ACL = Johnny-not_B_Goode@company.com

5 Save and exit the file.

Chapter 6

Configuration Files

167

Operating System and Application-Specific Configurations

Operating System and Application-Specific Configurations


The different operating systems and certain applications require unique configuration changes. This sections describes how and in which configuration files you must make the necessary changes.

Configuring the dlls.conf for PATROL for Unix


The PATROL Knowledge Module for Unix requires a file called apidll.dll. In order to load the apidll.dll file to the PATROL Agent, the apidll.dll file must be authenticated in dlls.conf. The dlls.conf file lists the .dll files necessary for KMs to work with the PATROL Agent. The dlls.conf file must be modified prior to loading the KM either by the PATROL Agent preload list or by a PATROL console. Modifying this file enables all functionality of the KM and makes it available to the agent. After installing PATROL Security components and before starting the agent, perform the following steps to modify the dlls.conf file.

To Modify the dlls.conf File 1 At the command line prompt, change to the directory that contains the dlls.conf
file. Table 52
Windows Unix

Location of the dlls.conf File


Path %PATROL_HOME%\..\common\patrol.d /etc/patrol.d

Operating System

2 Create the following entries in the dlls.conf file:


DLLDIR = $PATROL_HOME/lib/psl/$TARGET DLL = apidll.dll

or
DLL = $PATROL_HOME/lib/psl/$TARGET/apidll.dll

168

PATROL Security User Guide

Configuring the dlls.conf for PATROL for Unix

3 Specify any other dll files as needed:


DLLDIR = $OTHER_DIR DLL = otherdll.dll

or
DLL = $OTHER_DIR/otherdll.dll

4 Save and close the file.

Chapter 6

Configuration Files

169

Using PATROL Event Manager Applications with PATROL Security

Using PATROL Event Manager Applications with PATROL Security


When using PATROL Security at levels 1 through 4, the PATROL Event Manager (PEM) applications need to be recompiled with the PEM library only if with the applications are to be compatible with the 3.5 PEM APIs.

To Configure PATROL Security 1 Configure the following parameter in the config.default file as shown Figure 47.
Figure 47 ESI Variable Configured for PATROL Event Manager Applications

"/AgentSetup/security/ExtendedSecurityEnabled" = { REPLACE="yes"}

2 Configure the following parameter in the patrol.conf file as shown Figure 48.
Figure 48 ESI Library Location for PATROL Event Manager Applications

esi_lib = location_of_the_esi_library

3 On Windows, set the registry HKEY_LOCAL_MACHINE\SOFTWARE\BMC


Software\PATROL\ as shown Figure 47.

Figure 49

Registry Keys for PATROL Agent and PATROL Security

PATROL Agent = esi_lib_path PATROL Security = path_for_patrol.conf

At security level 4, the PEM client application can be launched by a system user rather than a login user. In this case, a certificate with identity = system must exist in the client key database for the authentication with the server application (for example, the PATROL Agent). If the application is launched by a login user rather than a system, then use identity = user_name. Using attended mode on Windows, in which the user is required to enter a password for the keyfile to get certificate information, the PEM service needs to be able to interact with Desktop. See Setting the Attended or Unattended Mode on page 134.

170

PATROL Security User Guide

Appendix

Changing the Security Level


This appendix describes how to change the security level of a computer after the installation process. This appendix presents the following topics: Changing the Security Level for the Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Appendix A Changing the Security Level

171

Changing the Security Level for the Enterprise

Changing the Security Level for the Enterprise


The security level is set during the installation process. However, you can change the level security on an individual computer. PATROL Security provides a script that enables you to run the script locally and change the level of the PATROL applications installed on that computer. Those applications include
I I I I I I

PATROL Agent 3.x PATROL Console 3.x Console Sever PATROL Central Web Edition PATROL Central Microsoft Windows Distribution Server: server, client, and command line interface

NOTE
If multiple PATROL applications are installed on the same computer (for example, a PATROL Agent and a Console Server), you must execute the script for each individual component to ensure that the security settings are consistent among applications.

Before you begin


The p7_change_security_level script is a command line utility that enables you to change the security level and configuration of a PATROL application running on the local computer. To run this utility, on Unix, you need to log on as root. On Windows, you need Administrator privileges.

To Change the Security Level of All PATROL Applications on Computer 1 Access a command prompt. 2 Navigate to the config_v3.0 directory. Table 53 provides the installation path of the
script based upon the operating system. Table 53
Windows Unix

p7_change_security_level Script Location


Path %BMC_ROOT\..\common\security\config_v3.0 $BMC_ROOT/../common/security/config_v3.0

Operating System

3 Type in the p7_change_security_level command string for your operating system


with the desired options and arguments. Figure 50 provides an example of a script setting the security level to 2 for PATROL Central - Web Edition installed on Windows. Figure 51 provides an example of a script that sets the Distribution Servers server to run at level 3 security on Unix.

172

PATROL Security User Guide

Changing the Security Level for the Enterprise

Figure 50

p7_change_security_level Example on Windows

p7_change_security_level.cmd -c PCWEB -l 2 -d Web Central

Figure 51

p7_change_security_level Example on Unix

p7_change_security_level.sh -c DS_SERVER -l 3

Table 54 lists the available options and their arguments. The order of the options is unimportant; however, a space must be inserted between an option and its argument. Table 54
Option -h -c component

p7_change_security_level Script Options


Argument Description displays the online help information. designates the PATROL application whose security level you want to change; components are AGENT_CON PATROL Agent and Console 3.x CSERVER Console Server DS_CLIENT Distribution Server, client DS_CLI Distribution Server, command line interface DS_SERVER Distribution Server, server PCWEB PATROL Central Web Edition PCWIN PATROL Central Microsoft Windows Edition

-l

security_level

sets the new security level Valid values range from 0 to 4. For information about PATROL Security levels, see Levels of Security on page 18.

-n

protocol(s)

determines which network communication protocols are supported. Security Levels 0, 1 and 2 require both TCP and UDP protocols. Security Levels 3 and 4 permit one protocol or both. TCP supports the Transmission Control protocol UDP supports the User Datagram Protocol BOTH supports TCP and UDP communication This parameter applies to only PATROL Agent 3.x and PATROL Console 3.x.

-d

Patrol3_subdir or Patrol 3 subdira

provides the name of the Patrol3 subdirectory; no path is needed This parameter applies to only PATROL Agent 3.x and PATROL Console 3.x.

Appendix A Changing the Security Level

173

Changing the Security Level for the Enterprise

Table 54
Option -d

p7_change_security_level Script Options


Argument Description PCWeb_subdir or provides the name of the PATROL Central Web PC WEB subdira subdirectory; no path is needed This parameter applies to only PATROL Central - Web Edition.

-v

version_number

indicates the version of PATROL Security whose security level will be changed ESS version 2.0 (no flag) _v3.0 ESS version 3.0

Double quotes are required if the path\directory names contain spaces such as \Program Files\BMC Software\PATROL Central.

The script attempts to change the security level. In the process, it updates the policy two configuration files: patrol.conf and config.default. For more information about these files, see Chapter 6, Configuration Files. The script also writes its results to the command prompt. Figure 52 displays a selection form the results log.

174

PATROL Security User Guide

Changing the Security Level for the Enterprise

Figure 52

p7_change_security_level Script Sample Log on Windows

[LOG] [LOG] p7_change_security_level.cmd execution begins... [LOG] Parameters passed in: [LOG] 1. Component: AGENT_CON [LOG] 2. BMC Installation Base: "C:\Program Files\BMC Software" [LOG] 3. Security Level: 1 [LOG] 4. Version: Default to 2.0 - i.e. no _v3.0 extensions was specified. [LOG] 4. Protocol: BOTH [LOG] 5. Patrol 3 Directory: PATROL3 [LOG] 1 file(s) copied. [LOG] Using config.default from "C:\Program Files\BMC Software"\PATROL3\lib [LOG] policy_install.cmd execution begins... << Log entries have been deleted from this example. >> [LOG]config_install.cmd execution begins... [LOG]Parameters passed in: [LOG]1. Path of new config.default = "C:\Program Files\BMC Software"\common\patr ol.d\config.default [LOG]2. Path of destination = "C:\Program Files\BMC Software"\PATROL3\lib\config .default [LOG]3. Path of bak destination = "C:\Program Files\BMC Software"\PATROL3\lib\ba [LOG]4. Overwrite flag = TRUE 1 file(s) copied. [LOG]Copied new file over existing file. [LOG]config_install.cmd completed successfully.

Appendix A Changing the Security Level

175

Changing the Security Level for the Enterprise

176

PATROL Security User Guide

Appendix

Troubleshooting
This appendix briefly describes common problems that can occur. This appendix presents the following topics: Issues and Workarounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Character @ Interpreted as Kill Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Attempt to Generate a Key Results in Extended Error Message . . . . . . . . . . . . . 179 Defaults to Security Level 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Missing bindir Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Missing securitydir Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Password Prompter Canceled Error Message. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Password Attribute Requires 2 Fields Error Message . . . . . . . . . . . . . . . . . . . . . . 182 Key File Cannot Be Reached Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Decrypting Stored Password Error Message. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Identity Missing from Key Database Error Message . . . . . . . . . . . . . . . . . . . . . . . 183 Unexpected Password Prompt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Installation Fails. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Uninstallation Fails to Remove Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Key Database Will Not Open With Correct Password . . . . . . . . . . . . . . . . . . . . . 185 No Key for Negotiated Cipher Error Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Cannot Install a Certificate into a Key Database. . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Cannot Install a CRL into a Key Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Windows CA Rejects a CSR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Password Not Configured . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Password Prompt Does Not Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Typed Password Does Not Appear in Password Dialog Box. . . . . . . . . . . . . . . . 188 Password Dialog Prompt Does Not Appear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 PATROL KM for Microsoft Cluster Server Does Not Support Attended Mode at Level 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Cannot Find Shared Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Discovery Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Password for 64-bit Key Files Is Not Validated . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Password Dialog Prompt Does Not Appear When Running at Level 4 . . . . . . . 191 Error Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Invalid Policy Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Appendix B

Troubleshooting

177

Invalid Policy Keyfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Incorrect Encrypted Password Used During Security Bootstrap . . . . . . . . . . . . . 193 Invalid Policy Identity Field (Non-Existing Key) . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Mutual Authentication Nominal Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Missing Key On Level 4 Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Missing Trusted Root (client) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Missing Certificate (Server) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Expired Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

178

PATROL Security User Guide

Issues and Workarounds

Issues and Workarounds


This section identifies errors that you may encounter when working with PATROL Security components, identifies the corresponding causes, and provides diagnostic solutions, if applicable.

Character @ Interpreted as Kill Command


The stty terminal settings can affect the operation of the sslcmd command. For terminal setting stty -a, the following error might appear when you choose sslcmd command option 2 (Generate CSR): The @ character in the e-mail address field may be read as a kill character, so that the e-mail address is truncated.

Cause
Terminal setting stty -a is not supported.

Solution
Enter an alternative terminal setting, such as stty kill^U.

Attempt to Generate a Key Results in Extended Error Message


On some platforms, extended error messages as shown in Figure 53 may appear when choosing sslcmd command option 1 (Generate Key). Figure 53 Generate a Key Extended Error Message

HP-UX hppcoqs6 B.11.00 A 9000/785 2015255295 two-user license /local_home/patqa1/classicrc1/common/security/bin_v3.0/hpux-11-00pa20-64 $ getconf KERNEL_BITS 64

After the error messages appear the key is generated successfully; therefore, no action is required.

Appendix B

Troubleshooting

179

Defaults to Security Level 4

Cause
The code generates a stream of unpredictable bytes by performing a list of system calls to selected utilities, for example, ps, netstat, and vmstat. If a certain platform does not to support one or more of the utilities, an error message may appear.

Solution
Because the command successfully generates the key, no user action is required and the error message can be ignored.

Defaults to Security Level 4


The security policy level defaults to level 4 unexpectedly.

Cause
In the site policy file, if the security_level field is deleted or contains an empty string, the default level of security is set to 4. Note that this differs from the default 0 (basic security level) that is set during PATROL installation if no security level is specified.

Solution
Specify the desired security policy level in the security_level field.

Missing bindir Error Message


An error message appears in the command prompt stating that there is a missing bindir.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the bindir field was deleted.

Solution
Include the bindir field in the file.

180

PATROL Security User Guide

Missing securitydir Error Message

Missing securitydir Error Message


An error message appears in the log file stating that there is a missing securitydir.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the securitydir field was deleted.

Solution
Include the securitydir field in the file.

Password Prompter Canceled Error Message


An error message appears in the log file stating that the password prompter was cancelled.

Cause
The user canceled the password dialog box. Figure 54 Password Prompter Canceled Error Message

1:Mon Feb 7 14:46:57 2005:pid=11974:ERR:bmccfg_role.c:399:Password prompter canceled 2:Mon Feb 7 14:46:57 2005:pid=11974:ERR:bmccfg.c:649:PolicyLoad failed -1

Solution
Restart and enter the password for attended mode or set the mode to unattended by placing the password in the policy file for the product that you are starting.

Appendix B

Troubleshooting

181

Password Attribute Requires 2 Fields Error Message

Password Attribute Requires 2 Fields Error Message


An error message appears in the log file stating that the password entry requires 2 fields (password and keymaterial) followed by an optional lock string.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the password field requires an entry for both a password and key material file, which may have been deleted in order to run in attended mode. Figure 55 Password Attribute Requires 2 Fields Error Message

1:Mon Feb 7 14:53:40 2005:pid=11999:ERR:bmccfg.c:1966:invalid policy start up, 'password' entry requires 2 fields (password and keymaterial) followed by an optional lock string 2:Mon Feb 7 14:53:48 2005:pid=11999:ERR:bmccfg_role.c:399:Password prompter canceled 3:Mon Feb 7 14:53:48 2005:pid=11999:ERR:bmccfg.c:649:PolicyLoad failed -1 "BUT PASSWORD PROMPTER WILL POP UP"

Solution
Include a valid entry in the password field.

Key File Cannot Be Reached Error Message


An error message appears in the log file stating that the key file cannot be reached.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the keyfile field may contain an invalid entry or may reference an invalid directory.

Solution
Include a valid entry in the keyfile field.

182

PATROL Security User Guide

Decrypting Stored Password Error Message

Decrypting Stored Password Error Message


An error message appears in the log file stating that there was an error decrypting the stored password.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the password field may contain an invalid password.

Solution
Include a valid entry in the password field.

Identity Missing from Key Database Error Message


An error message appears in the log file stating that the identity does not appear in the key database.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the identity: field may contain an invalid entry. Figure 56 Identity Missing from Key Database Error Message

1:Mon Feb 7 18:15:01 2005:pid=12507:ERR:bmccfg_role.c:316:Security policy entry server does not contain identity entry 2:Mon Feb 7 18:15:01 2005:pid=12507:ERR:bmccfg.c:649:PolicyLoad failed -1

Solution
Include a valid entry in the identity field.

Appendix B

Troubleshooting

183

Unexpected Password Prompt

Unexpected Password Prompt


A password prompt unexpectedly appears.

Cause
In the site file (site.plc on Unix, site.reg on Windows), the password field have been deleted, or the keyfile or password entry in this field may have been deleted. This can occur when a user wishes to run in attended mode, and thus intentionally deletes the password field in order to enable the password prompt to appear.

Solution
To revert to unattended mode, include valid entries in the password field.

Installation Fails
Installation of PATROL Security fails.

Cause
Installation may fail due to lack of privilege. On Unix, you may lack the privilege for modifying the /etc directory; therefore you cannot create the /etc/patrol.d/security_policy or place the policy files in /etc/patrol.d. On Windows, you may lack administrator privilege to modify registry entries; therefore you cannot create the necessary registry entries.

Solution
Obtain an account with the requisite privilege.

184

PATROL Security User Guide

Uninstallation Fails to Remove Security Policies

Uninstallation Fails to Remove Security Policies


Uninstallation fails to remove the security registry entries (Windows) or policy files (Unix).

Cause
The uninstallation process is unable to remove policies.

Solution
Manually remove the security registry entries and/or policy files only if another PATROL application does not use them. Otherwise, leave them.

Key Database Will Not Open With Correct Password


The key database will not open although you entered the correct password.

Causes
I

A 32-bit platform cannot use a key database generated on a 64-bit platform. In an international context, a key database generated in one locale cannot be opened in another locale. Key Database Will Not Open With Correct Password Error Message

Figure 57

1:Mon Feb 7 17:14:38 2005:pid=12351:ERR:bmccfg_role.c:316:Security policy entry server does not contain keyfile entry 2:Mon Feb 7 17:14:38 2005:pid=12351:ERR:bmccfg.c:649:PolicyLoad failed -1

Appendix B

Troubleshooting

185

No Key for Negotiated Cipher Error Message

No Key for Negotiated Cipher Error Message


An SSL server or client opens the key database, but refuses an SSL connection due to a no key for negotiated cipher error.

Cause
The key associated with the SSL Identity was found in the key database, but a certificate guaranteeing its authenticity was not found. The SSLV2CipherSuite or the SSLV3CipherSuite attribute limits the list of possible cipher suites that can be used by the server or client. A common cipher suite supported by both cannot be found.

Cannot Install a Certificate into a Key Database


This certificate cannot be installed into the key database.

Causes
I

This certificate does not pertain to any key pair contained in the key database. The CA certificate used to sign this certificate has not been previously installed in the key database. A CA certificate has been installed in the key database, but it is not the CA certificate used to sign this certificate. This certificate has (mistakenly) previously been installed in the key database as a CA certificate.

Cannot Install a CRL into a Key Database


A certificate revocation list (CRL) cannot be installed into the key database.

Cause
The CA certificate of the CA to which this CRL pertains has not been previously installed into the key database.

186

PATROL Security User Guide

Windows CA Rejects a CSR

Windows CA Rejects a CSR


A Windows Certificate Authority rejects a certificate signing request (CSR) which contains the public key of a DSA key pair.

Cause
The Windows Certificate Authority will not generate a certificate for a DSA key pair.

Password Not Configured


The default ASCII Password Dialog prompts for the keyfile and password selection on operating systems s390/Linux and Siemens Sinix 5.43, or for systems where the X11 runtime environment is not configured.

Cause
Using the keyboard-based ASCII prompt is possible only by a foreground process.

Solution
For non-GUI configuration requiring an attended password entry, the PatrolAgent service must run directly from a user shell. The PatrolAgent script should not be used. Run PatrolAgent from the PATROL3/OS/bin directory.

Password Prompt Does Not Display


The password prompt fails to display when starting the agent or console for Unix.

Solution
Specify your computer as the hostname:
$ DISPLAY=hostname:0.0

Appendix B

Troubleshooting

187

Typed Password Does Not Appear in Password Dialog Box

Typed Password Does Not Appear in Password Dialog Box


When using the scripts file in PATROL3 to start the PATROL Console and Agent on Unix in attended mode (password required), the password dialog does not receive standard input.

Solution
Perform one of the following actions:
I

If you wish to run in attended mode, start the PATROL Console and Agent from the PATROL3/OS/bin directory. Be aware that starting the console and agent from the bin directory prevents the Perform Agent, dcm, and bgscollect services from running. If you wish to run the Perform Agent, dcm, and bgscollect services, you can start the PATROL Console and Agent from the PATROL3 directory, but you must run in unattended mode.

Password Dialog Prompt Does Not Appear


When running PEM-based services, including the console, agent, and any other PEMbased applications, as a service under the domain account, the Password Dialog prompt does not appear.

Solution
Perform one of the following actions:
I

If you wish to use attended mode, run the PEM-based service at the command line under the local system account or run it as a service using the system account and allowing the service to interact with the desktop. If you wish to run the PEM-based service as a service under the domain account, use unattended mode.

188

PATROL Security User Guide

PATROL KM for Microsoft Cluster Server Does Not Support Attended Mode at Level 4

PATROL KM for Microsoft Cluster Server Does Not Support Attended Mode at Level 4
The PATROL KM for Microsoft Cluster Server does not operate in attended mode with Level 4 security.

Solution
Attended mode does not support the use of services running under a domain account. Since the Cluster Service runs only under a domain account, you are unable to run this service in attended mode.

Cannot Find Shared Library


PATROL fails to discover the shared library API km.

Solution
For security levels 1 through 4, all dll files must have signature files (api.dll.sgn) in order to load. To create signature files, use the signFile utility located in $BMC_ROOT/common/security/bin_v3.0/target (Unix) or %BMC_ROOT%\common\security\bin_v3.0\target (Windows). After you have created signature files for the dll files, perform either of the following actions:
I

In the patrol.conf file, in the 'agentrights' section under the [AGENT] stanza, change the allowalldlls attribute to allowalldlls=true. In /etc/patrol.d/dlls.conf, list all DLL file(s) and directories that the agent is authorized to load. A template file for dlls.conf is loaded during installation and resides in /etc/patrol.d.

Appendix B

Troubleshooting

189

Discovery Fails

Discovery Fails
Discovery performs a UDP ping and fails.

Cause
Discovery using a UDP fails when during installation you select TCM network connection only.

Solution
In the config.default file, comment out the following line:
"/AgentSetup/PortConnectType"= {REPLACE="TCP"}

Password for 64-bit Key Files Is Not Validated


When entering a newly created .kdb file and password in the Password Dialog box, the system fails to validate the correct password for 64-bit key files and an Incorrect password message appears.

Cause
A key database created by a 64-bit sslcmd application cannot be opened by a 32-bit application. Similarly, a key database created by a 32-bit sslcmd application but opened and subsequently modified by a 64-bit sslcmd application can no longer be opened by the 32-bit application. In short, key databases are, in general, not transportable between 32-bit and 64-bit platforms.

190

PATROL Security User Guide

Password Dialog Prompt Does Not Appear When Running at Level 4

Password Dialog Prompt Does Not Appear When Running at Level 4


When running PATROL Console Server (or any service which must run in attended mode at level 4) as a service under the domain account, the password prompt dialog box is not displayed.

Solution
Perform one of the following actions:
I

If you want to use attended mode, run Console Server at the command line under the local system account or run it as a service using the system account and allowing the service to interact with the desktop. If you want to run Console Server as a service under the domain account, use unattended mode.

Appendix B

Troubleshooting

191

Error Conditions

Error Conditions
If you are experiencing a problem with PATROL Security, review the following error conditions. These conditions are the most frequently experienced problems.

Invalid Policy Password


Cause
The server encrypted password text was modified. Figure 58 Invalid Policy Password Error Message

1:Wed Mar 6 11:14:15 2002:pid=27584:ERR:ess_policy.c:586:error decrypting stored password

Invalid Policy Keyfile


Cause
The incorrect key database path and/or filename was entered in the keyfile attribute. Figure 59 Invalid Policy Keyfile Error Message

unable to bootstrap policy /etc/patrol.d/security_policy/agent.plc, unable to set /home/mpetkevi/bmc/ess2.0/cert/server1.kdb keystore: key store /home/mpetkevi/bmc/ess2.0/cert/server1.kdb cannot be reached

192

PATROL Security User Guide

Incorrect Encrypted Password Used During Security Bootstrap

Incorrect Encrypted Password Used During Security Bootstrap


Cause
The server encrypted password differs from the actual key database password. Figure 60 Incorrect Encrypted Password Used During Security Bootstrap Error Message

1:Wed Mar 6 11:20:35 2002:pid=27733:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 11:20:36 2002:pid=27733:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 11:20:37 2002:pid=27733:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 4:Wed Mar 6 11:20:37 2002:pid=27733:ERR:ssl_tsw.c:344:Cannot initiate with TSW_crypt_init, invalid password, key file /home/mpetkevi/bmc/ess2.0/cert/server.kdb, CORE: Wrong version 5:Wed Mar 6 11:20:38 2002:pid=27733:ERR:../bcm/bcm_api.c:522:BCM_Option: unable to execute option

Invalid Policy Identity Field (Non-Existing Key)


Cause
The server identity field is set to an invalid value.

Client Log
Figure 61 Invalid Policy Identity Field (Non-existing Key) Error Message, Client Log

1:Wed Mar 6 11:47:08 2002:pid=28431:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 11:47:09 2002:pid=28431:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 11:47:09 2002:pid=28431:INF:ess_policy.c:770:client security system at level 2, application security at level 2, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 4:Wed Mar 6 11:47:12 2002:pid=28431:ERR:ssl_tsw.c:1133:caught SSL alert from 127.0.0.1 , 40, Handshake failure, level 2, SSL: Caught alert 5:Wed Mar 6 11:47:13 2002:pid=28431:ERR:../bcm/bcm_api.c:230:unable to connect secure sessionfor user: , service 6:Wed Mar 6 11:47:14 2002:pid=28431:INF:bcm_profile.c:334:session 512 was shutdown 7:Wed Mar 6 11:47:15 2002:pid=28431:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 8:Wed Mar 6 11:47:16 2002:pid=28431:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Appendix B

Troubleshooting

193

Invalid Policy Identity Field (Non-Existing Key)

Server Log
Figure 62 Invalid Policy Identity Field (Non-existing Key) Error Message, Server Log

1:Wed Mar 6 11:47:04 2002:pid=28430:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 11:47:05 2002:pid=28430:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 11:47:06 2002:pid=28430:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 4:Wed Mar 6 11:47:07 2002:pid=28430:INF:ess_policy.c:770:server security system at level 2, application security at level 2, site policy /etc/patrol.d/security_policy/site.plc/server, application policy /etc/patrol.d/security_policy/agent.plc/server 5:Wed Mar 6 11:47:10 2002:pid=28430:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 6:Wed Mar 6 11:47:11 2002:pid=28430:TRC:bcm_sslsess.c:157:BCM_ModuleServer(): done 7:Wed Mar 6 11:47:12 2002:pid=28430:ERR:key_hook.c:93:identity: server1 is not found in key database /home/mpetkevi/bmc/ess2.0/cert/server.kdb 8:Wed Mar 6 11:47:12 2002:pid=28430:ERR:ssl_tsw.c:1149:Error initiating handshake as server with 127.0.0.1 , errno 0, SSL: Operation Cancelled 9:Wed Mar 6 11:47:13 2002:pid=28430:ERR:../bcm/bcm_api.c:279:unable to establish session side, handle: 0007b0e8, service: mysprinc 10:Wed Mar 6 11:47:14 2002:pid=28430:INF:bcm_profile.c:334:session 512 was shutdown 11:Wed Mar 6 11:47:15 2002:pid=28430:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 12:Wed Mar 6 11:47:16 2002:pid=28430:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locatesession

194

PATROL Security User Guide

Mutual Authentication Nominal Case

Mutual Authentication Nominal Case


Cause
Nominal level 4.

Client Log
Figure 63 Mutual Authentication Nominal Case Error Message, Client Log

1:Wed Mar 6 14:41:19 2002:pid=1962:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 14:41:20 2002:pid=1962:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 14:41:20 2002:pid=1962:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 4:Wed Mar 6 14:41:21 2002:pid=1962:INF:ess_policy.c:770:client security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 5:Wed Mar 6 14:41:22 2002:pid=1962:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 6:Wed Mar 6 14:41:24 2002:pid=1962:INF:ssl_tsw.c:1039:creating cert list of 2 7:Wed Mar 6 14:41:25 2002:pid=1962:WRN:auth_hook2.c:226:REVOCATION UNKNOWN certificate discovered --> subject: CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 07:6D:34:8F:00:00:05:13 8:Wed Mar 6 14:41:28 2002:pid=1962:INF:ssl_tsw.c:1112:connection with 127.0.0.1 established

Server Log
Figure 64 Mutual Authentication Nominal Case Error Message, Server Log (part 1 of 2)

1:Wed Mar 6 14:41:07 2002:pid=1959:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 14:41:08 2002:pid=1959:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 14:41:09 2002:pid=1959:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 4:Wed Mar 6 14:41:09 2002:pid=1959:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 5:Wed Mar 6 14:41:10 2002:pid=1959:INF:ess_policy.c:770:server security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/server, application policy /etc/patrol.d/security_policy/agent.plc/server 6:Wed Mar 6 14:41:22 2002:pid=1959:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 7:Wed Mar 6 14:41:23 2002:pid=1959:TRC:bcm_sslsess.c:157:BCM_ModuleServer(): done 8:Wed Mar 6 14:41:23 2002:pid=1959:INF:ssl_tsw.c:1098:Client authentication enabled

Appendix B

Troubleshooting

195

Missing Key On Level 4 Client

Figure 64

Mutual Authentication Nominal Case Error Message, Server Log (part 2 of 2)

9:Wed Mar 6 14:41:26 2002:pid=1959:INF:ssl_tsw.c:1039:creating cert list of 2 10:Wed Mar 6 14:41:26 2002:pid=1959:WRN:auth_hook2.c:226:REVOCATION UNKNOWN certificate discovered --> subject: CN=TrialUserPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 09:AC:8F:E2:00:00:05:22 11:Wed Mar 6 14:41:27 2002:pid=1959:INF:auth_hook2.c:260:access granted for client patrol_security@bmc.com 12:Wed Mar 6 14:41:28 2002:pid=1959:INF:ssl_tsw.c:1112:connection with 127.0.0.1 established 13:Wed Mar 6 14:41:32 2002:pid=1959:INF:bcm_profile.c:334:session 512 was shutdown 14:Wed Mar 6 14:41:33 2002:pid=1959:INF:../bcm/bcm_api.c:430:BCM_Terminate: session 512 terminated

Missing Key On Level 4 Client


Cause
The bmcuser key was removed from bmcuser.kdb.

Client Log
Figure 65 Missing Key on Level 4 Client Error Message, Client Log (part 1 of 2)

1:Wed Mar 6 14:59:04 2002:pid=2198:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 14:59:04 2002:pid=2198:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 14:59:05 2002:pid=2198:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 4:Wed Mar 6 14:59:06 2002:pid=2198:INF:ess_policy.c:770:client security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 5:Wed Mar 6 14:59:06 2002:pid=2198:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 6:Wed Mar 6 14:59:09 2002:pid=2198:INF:ssl_tsw.c:1039:creating cert list of 2 7:Wed Mar 6 14:59:09 2002:pid=2198:WRN:auth_hook2.c:226:REVOCATION UNKNOWN certificate discovered --> subject: CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 07:6D:34:8F:00:00:05:13 8:Wed Mar 6 14:59:10 2002:pid=2198:ERR:ssl_tsw.c:1141: no key, while handshaking with 127.0.0.1 , SSL: No key available for negotiated cipher 9:Wed Mar 6 14:59:11 2002:pid=2198:ERR:../bcm/bcm_api.c:230:unable to connect secure session for user: , service 10:Wed Mar 6 14:59:11 2002:pid=2198:INF:bcm_profile.c:334:session 512 was shutdown

196

PATROL Security User Guide

Missing Key On Level 4 Client

Figure 65

Missing Key on Level 4 Client Error Message, Client Log (part 2 of 2)

11:Wed Mar 6 14:59:13 2002:pid=2198:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 12:Wed Mar 6 14:59:14 2002:pid=2198:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Server Log
Figure 66 Missing Key on Level 4 Client Error Message, Server Log

1:Wed Mar 6 14:59:04 2002:pid=2198:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 14:59:04 2002:pid=2198:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 14:59:05 2002:pid=2198:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 4:Wed Mar 6 14:59:06 2002:pid=2198:INF:ess_policy.c:770:client security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 5:Wed Mar 6 14:59:06 2002:pid=2198:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 6:Wed Mar 6 14:59:09 2002:pid=2198:INF:ssl_tsw.c:1039:creating cert list of 2 7:Wed Mar 6 14:59:09 2002:pid=2198:WRN:auth_hook2.c:226:REVOCATION UNKNOWN certificate discovered --> subject: CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 07:6D:34:8F:00:00:05:13 8:Wed Mar 6 14:59:10 2002:pid=2198:ERR:ssl_tsw.c:1141: no key, while handshaking with 127.0.0.1 , SSL: No key available for negotiated cipher 9:Wed Mar 6 14:59:11 2002:pid=2198:ERR:../bcm/bcm_api.c:230:unable to connect secure session for user: , service 10:Wed Mar 6 14:59:11 2002:pid=2198:INF:bcm_profile.c:334:session 512 was shutdown 11:Wed Mar 6 14:59:13 2002:pid=2198:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 12:Wed Mar 6 14:59:14 2002:pid=2198:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Appendix B

Troubleshooting

197

Missing Trusted Root (client)

Missing Trusted Root (client)


Cause
WWWQA trusted root removed from bmcuser.kdb

Client Log
Figure 67 Missing Trusted Root (client) Error Message, Client Log

1:Wed Mar 6 15:13:18 2002:pid=2470:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 15:13:19 2002:pid=2470:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 15:13:19 2002:pid=2470:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 4:Wed Mar 6 15:13:20 2002:pid=2470:INF:ess_policy.c:770:client security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 5:Wed Mar 6 15:13:21 2002:pid=2470:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 6:Wed Mar 6 15:13:23 2002:pid=2470:WRN:verify.c:89:No trusted CA for the last certificate in the chain: Subject CN=WWWQA Testing Certificate Authority,OU=WEBDEV,O=BMC Software,L=Houston,ST=Texas,C=US Issuer CN=WWWQA Testing Certificate Authority,OU=WEBDEV,O=BMC Software,L=Houston,ST=Texas,C=US Certificate Serial=2cc4384b1000128f11d2e2e0a91681d4 Valid Begin:Thu Mar 25 12:44:14 1999 Valid End: Thu Mar 25 12:44:14 2004 Status: UNVERIFIED, verification required - certificate rejected 7:Wed Mar 6 15:13:24 2002:pid=2470:WRN:verify.c:89:No trusted CA for the last certificate in the chain: Subject CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com Issuer CN=WWWQA Testing Certificate Authority,OU=WEBDEV,O=BMC Software,L=Houston,ST=Texas,C=US Certificate Serial=076d348f00000513 Valid Begin:Thu Jul 1923:15:54 2001 Valid End: Sat Jul 19 23:15:54 2003 Status: UNVERIFIED, verification required certificate rejected 8:Wed Mar 6 15:13:25 2002:pid=2470:ERR:ssl_tsw.c:1149:Error initiating handshake as client with 127.0.0.1 , errno 2, SSL: Required certificate not provided 9:Wed Mar 6 15:13:25 2002:pid=2470:ERR:../bcm/bcm_api.c:230:unable to connect secure session for user: , service 10:Wed Mar 6 15:13:26 2002:pid=2470:INF:bcm_profile.c:334:session 512 was shutdown 11:Wed Mar 6 15:13:27 2002:pid=2470:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 12:Wed Mar 6 15:13:27 2002:pid=2470:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

198

PATROL Security User Guide

Missing Trusted Root (client)

Server Log
Figure 68 Missing Trusted Root (client) Error Message, Server Log

1:Wed Mar 6 15:12:57 2002:pid=2459:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 15:12:58 2002:pid=2459:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 15:12:59 2002:pid=2459:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 4:Wed Mar 6 15:12:59 2002:pid=2459:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 5:Wed Mar 6 15:13:00 2002:pid=2459:INF:ess_policy.c:770:server security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/server, application policy /etc/patrol.d/security_policy/agent.plc/server 6:Wed Mar 6 15:13:21 2002:pid=2459:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 7:Wed Mar 6 15:13:22 2002:pid=2459:TRC:bcm_sslsess.c:157:BCM_ModuleServer(): done 8:Wed Mar 6 15:13:23 2002:pid=2459:INF:ssl_tsw.c:1098:Client authentication enabled 9:Wed Mar 6 15:13:28 2002:pid=2459:ERR:ssl_tsw.c:1149:Error initiating handshake as server with 127.0.0.1 , errno 0, SSL: IO error 10:Wed Mar 6 15:13:28 2002:pid=2459:ERR:../bcm/bcm_api.c:279:unable to establish session side, handle: 0007b0e8, service: mysprinc 11:Wed Mar 6 15:13:29 2002:pid=2459:INF:bcm_profile.c:334:session 512 was shutdown 12:Wed Mar 6 15:13:30 2002:pid=2459:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 13:Wed Mar 6 15:13:30 2002:pid=2459:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Appendix B

Troubleshooting

199

Missing Certificate (Server)

Missing Certificate (Server)


Cause
The server policy specifies a key with no certificate.

Client Log
Figure 69 Missing Certificate, Client Log

1:Wed Mar 6 15:20:31 2002:pid=2629:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 15:20:32 2002:pid=2629:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 15:20:32 2002:pid=2629:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 4:Wed Mar 6 15:20:33 2002:pid=2629:INF:ess_policy.c:770:client security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/client, application policy /etc/patrol.d/security_policy/console.plc/client 5:Wed Mar 6 15:20:33 2002:pid=2629:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmcuser.kdb 6:Wed Mar 6 15:20:37 2002:pid=2629:ERR:ssl_tsw.c:1133:caught SSL alert from 127.0.0.1 , 40, Handshake failure, level 2, SSL: Caught alert 7:Wed Mar 6 15:20:38 2002:pid=2629:ERR:../bcm/bcm_api.c:230:unable to connect secure session for user: , service 8:Wed Mar 6 15:20:39 2002:pid=2629:INF:bcm_profile.c:334:session 512 was shutdown 9:Wed Mar 6 15:20:39 2002:pid=2629:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 10:Wed Mar 6 15:20:40 2002:pid=2629:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Server Log
Figure 70 Missing Certificate, Server Log (part 1 of 2)

1:Wed Mar 6 15:20:10 2002:pid=2623:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:27:23 2:Wed Mar 6 15:20:11 2002:pid=2623:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|sunos_5.5.1_sun4u|Mar 1 2002|14:29:09 (Domestic) 3:Wed Mar 6 15:20:12 2002:pid=2623:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 4:Wed Mar 6 15:20:13 2002:pid=2623:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 5:Wed Mar 6 15:20:13 2002:pid=2623:INF:ess_policy.c:770:server security system at level 4, application security at level 4, site policy /etc/patrol.d/security_policy/site.plc/server, application policy /etc/patrol.d/security_policy/agent.plc/server 6:Wed Mar 6 15:20:33 2002:pid=2623:INF:ssl_tsw.c:299:key file name is: /home/mpetkevi/bmc/ess2.0/cert/server.kdb 7:Wed Mar 6 15:20:35 2002:pid=2623:TRC:bcm_sslsess.c:157:BCM_ModuleServer(): done

200

PATROL Security User Guide

Expired Certificate

Figure 70

Missing Certificate, Server Log (part 2 of 2)

8:Wed Mar 6 15:20:35 2002:pid=2623:INF:ssl_tsw.c:1098:Client authentication enabled 9:Wed Mar 6 15:20:36 2002:pid=2623:ERR:key_hook.c:93:identity: mike is not found in key database /home/mpetkevi/bmc/ess2.0/cert/server.kdb 10:Wed Mar 6 15:20:37 2002:pid=2623:ERR:ssl_tsw.c:1149:Error initiating handshake as server with 127.0.0.1 , errno 0, SSL: Operation Cancelled 11:Wed Mar 6 15:20:38 2002:pid=2623:ERR:../bcm/bcm_api.c:279:unable to establish session side, handle: 0007b2f0, service: mysprinc 12:Wed Mar 6 15:20:39 2002:pid=2623:INF:bcm_profile.c:334:session 512 was shutdown 13:Wed Mar 6 15:20:39 2002:pid=2623:ERR:../bcm/bcm_api.c:536:BCM_CleanUp: Unable to locate session 14:Wed Mar 6 15:20:40 2002:pid=2623:ERR:../bcm/bcm_api.c:423:BCM_Terminate: Unable to locate session

Expired Certificate
Cause
The key certificate has expired.

Client
Figure 71 Expired Certificate, Client Log (part 1 of 2)

1:Sun Mar 07 15:01:06 2004:pid=4016:WRN:auth_hook2.c:226:EXPIRED certificate discovered --> subject: CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 07:6D:34:8F:00:00:05:13 2:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:1149:Error initiating handshake as client with peer , errno 2, SSL: Permission denied by auth hook 3:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:962:Unable to rehandshake 4:Sun Mar 07 15:01:06 2004:pid=4016:ERR:..\bcm\bcm_api.c:437:esi_Read, unable to process BCA event for context 01E905B0 5:Sun Mar 07 15:01:06 2004:pid=4016:WRN:auth_hook2.c:226:EXPIRED certificate discovered --> subject: CN=TrialServerPrincipal,OU=TestAndVerification,O=BMC Software,L=Houston,ST=TX,C=US,EM=patrol_security@bmc.com, serial: 07:6D:34:8F:00:00:05:13 6:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:1149:Error initiating handshake as client with peer , errno 2, SSL: Permission denied by auth hook 7:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:962:Unable to rehandshake 8:Sun Mar 07 15:01:06 2004:pid=4016:ERR:..\bcm\bcm_api.c:437:esi_Read, unable to process BCA event for context 01E90530 9:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:1002:error on TSW_SSL_Read, status = -13, unknown, SSL: Internal error 10:Sun Mar 07 15:01:06 2004:pid=4016:ERR:..\bcm\bcm_api.c:437:esi_Read, unable to process BCA event for context 01E905B0

Appendix B

Troubleshooting

201

Expired Certificate

Figure 71

Expired Certificate, Client Log (part 2 of 2)

11:Sun Mar 07 15:01:06 2004:pid=4016:ERR:ssl_tsw.c:1002:error on TSW_SSL_Read, status = -13, unknown, SSL: Internal error 12:Sun Mar 07 15:01:06 2004:pid=4016:ERR:..\bcm\bcm_api.c:437:esi_Read, unable to process BCA event for context 01E90530

Server Log
Figure 72 Expired Certificate, Server Log

1:Sun Mar 07 15:09:43 2004:pid=2396:INF:ess_policy.c:692:BCA Version 1.0|ess2.0.d.5.7|win32|Feb 1 2002|12:42:15 2:Sun Mar 07 15:09:43 2004:pid=2396:INF:ess_policy.c:694:SSL Module, Version 1.0|ess2.0.d.5.7|win32|Feb 1 2002|12:44:33 (Domestic) 3:Sun Mar 07 15:09:43 2004:pid=2396:INF:ssl_tsw.c:299:key file name is: D:\Program Files\BMC Software\common\security\keys\server.kdb 4:Sun Mar 07 15:09:43 2004:pid=2396:INF:ssl_tsw.c:299:key file name is: D:\Program Files\BMC Software\common\security\keys\server.kdb 5:Sun Mar 07 15:09:43 2004:pid=2396:INF:ess_policy.c:770:server security system at level 4, application security at level 4, site policy SOFTWARE\BMC Software \Patrol\SecurityPolicy\SITE\server, application policy SOFTWARE\BMC Software\Patrol\SecurityPolicy\AGENT\server 6:Sun Mar 07 15:12:57 2004:pid=2396:TRC:..\bcm\bcm_api.c:437:esi_FreeCtx, deallocating module security context 7:Sun Mar 07 15:12:57 2004:pid=2396:ERR:..\bcm\bcm_api.c:423:BCM_Terminate: Unable to locate session 8:Sun Mar 07 15:12:57 2004:pid=2396:INF:..\bcm\bcm_api.c:437:esi security context deallocated for 015C05A0, session termination status is -7 9:Sun Mar 07 15:12:58 2004:pid=2396:TRC:..\bcm\bcm_api.c:437:Entering esi_Read with len 99, ctx 015C05A0 10:Sun Mar 07 15:12:58 2004:pid=2396:INF:ssl_tsw.c:299:key file name is: D:\Prog

202

PATROL Security User Guide

Appendix

Valid Country Codes


Table 55 provides the valid two-letter ISO country codes for the Distinguished Name prompt Country, as used by the sslcmd utility. For more information, see Creating a Certificate Signing Request on page 89. Table 55
Country Afghanistan Albania Algeria American Samoa Andorra Angola Anguilla Antarctica Antigua And Barbuda Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belgium Belize Benin Bermuda

Valid Country Codes (part 1 of 7)


Code AF AL DZ AS AD AO AI AQ AG AR AM AW AU AT AZ BS BH BD BB BE BZ BJ BM

Appendix C

Valid Country Codes

203

Table 55
Country Bhutan Bolivia

Valid Country Codes (part 2 of 7)


Code BT BO BA BW BV BR IO BN BG BF BI BY KH CM CA CV KY CF TD CL CN CX CC CO KM CG CK CR CI HR CU CY CZ CS DK DJ DM

Bosnia Hercegovina Botswana Bouvet Island Brazil British Indian Ocean Territory Brunei Darussalam Bulgaria Burkina Faso Burundi Belarus Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos (Keeling) Islands Colombia Comoros Congo Cook Islands Costa Rica Cote D'ivoire Croatia Cuba Cyprus Czech Republic Czechoslovakia Denmark Djibouti Dominica

204

PATROL Security User Guide

Table 55
Country

Valid Country Codes (part 3 of 7)


Code DO TP EC EG SV GQ EE ET FK FO FJ FI FR GF PF TF GA GM GE DE GH GI GR GL GD GP GU GT GN GW GY HT HM HN HK HU IS

Dominican Republic East Timor Ecuador Egypt El Salvador Equatorial Guinea Estonia Ethiopia Falkland Islands (Malvinas) Faroe Islands Fiji Finland France French Guiana French Polynesia French Southern Territories Gabon Gambia Georgia Germany Ghana Gibraltar Greece Greenland Grenada Guadeloupe Guam Guatemala Guinea Guinea-bissau Guyana Haiti Heard And Mc Donald Islands Honduras Hong Kong Hungary Iceland

Appendix C

Valid Country Codes

205

Table 55
Country India Indonesia

Valid Country Codes (part 4 of 7)


Code IN ID IR IQ IE IL IT JM JP JO KZ KE KI KP KR KW KG LA LV LB LS LR LY LI LT LU MO MG MW MY MV ML MT MH MQ MR

Iran (Islamic Republic Of) Iraq Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Kiribati Korea, Democratic People's Republic Of Korea, Republic Of Kuwait Kyrgyzstan Lao People's Democratic Republic Latvia Lebanon Lesotho Liberia Libyan Arab Jamahiriya Liechtenstein Lithuania Luxembourg Macau Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Martinique Mauritania

206

PATROL Security User Guide

Table 55
Country Mauritius Mexico Micronesia

Valid Country Codes (part 5 of 7)


Code MU MX FM MD MC MN MS MA MZ MM NA NR NP NL AN NT NC NZ NI NE NG NU NF MP NO OM PK PW PA PG PY PE PH PN PL PT PR QA

Moldova, Republic Of Monaco Mongolia Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands Netherlands Antilles Neutral Zone New Caledonia New Zealand Nicaragua Niger Nigeria Niue Norfolk Island Northern Mariana Islands Norway Oman Pakistan Palau Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Puerto Rico Qatar

Appendix C

Valid Country Codes

207

Table 55
Country Reunion Romania

Valid Country Codes (part 6 of 7)


Code RE RO RU RW SH KN LC PM VC WS SM ST SA SN SC SL SG SK SI SB SO ZA ES LK SD SR SJ SZ SE CH SY TW TJ TH TG

Russian Federation Rwanda St. Helena Saint Kitts And Nevis Saint Lucia St. Pierre And Miquelon Saint Vincent And The Grenadines Samoa San Marino Sao Tome And Principe Saudi Arabia Senegal Seychelles Sierra Leone Singapore Slovakia Slovenia Solomon Islands Somalia South Africa Spain Sri Lanka Sudan Suriname Svalbard And Jan Mayen Islands Swaziland Sweden Switzerland Syrian Arab Republic Taiwan, Province Of China Tajikistan Thailand Togo

Tanzania, United Republic Of TZ

208

PATROL Security User Guide

Table 55
Country Tokelau Tonga

Valid Country Codes (part 7 of 7)


Code TK TO TT TN TR TM TC TV UG UA AE GB US

Trinidad And Tobago Tunisia Turkey Turkmenistan Turks And Caicos Islands Tuvalu Uganda Ukraine United Arab Emirates United Kingdom United States

United States Minor Outlying UM Islands Uruguay Ussr Uzbekistan Vanuatu Vatican City State (Holy See) Venezuela Viet Nam Virgin Islands (British) Virgin Islands (U.s.) Wallis And Futuna Islands Western Sahara Yemen, Republic Of Yugoslavia Zaire Zambia Zimbabwe UY SU UZ VU VA VE VN VG VI WF EH YE YU ZR ZM ZW

Appendix C

Valid Country Codes

209

210

PATROL Security User Guide

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Glossary
access control list A list that is set up by using a PATROL Agent configuration variable and that restricts PATROL Console access to a PATROL Agent. A PATROL Console can be assigned access rights to perform console, agent configuration, or event manager activities. The console server uses access control lists to restrict access to objects in the COS namespace. authentication A method of proving a person's identity. certificate This is a digital document containing a public key and a name used to authenticate the identity of the source of the data accompanying the certificate. certificate authority (CA) This is an issuer of an x509 certificate used in Secure Socket Layer (SSL) connections. It is also referred to as a trusted root authority. See trusted root certificate authority. certificate revocation list (CRL) The CRL is maintained by the Certificate Authority (CA). When the private key associated with the public key contained in the certificate is compromised, the owner of the compromised private key should immediately notify the CA that signed the certificate. The CA then publishes a CRL, which lists the certificate as having been revoked. Each user of a particular CA should obtain the CRL of that CA on a regular basis and install the CRL in the key database, so that if the revoked certificate is presented at a later date, the software will detect it as a revoked certificate and the chain of trust will be broken. If the certificate revocation list of a CA is missing from the key database, PATROL will issue the warning REVOCATION UNKNOWN. chain of trust This is a principle of security by which a software component verifies the identity of an unknown party by accepting the assurance of a third party whose identity it knows is genuine. It is possible that this partys identity is trusted because of the assurance of yet another party. This series of verifications by a trusted party continues (in a chain) until it is traced back to a trusted root (also known as a Certificate Authority) that the software component knows is trustworthy because it is provided by its own company or an approved vendor. console server A server through which PATROL Central and PATROL Web Central communicate with managed systems. A console server handles requests, events, data, communications, views, customizations, and security. Diffie-Hellman public key A public key algorithm that allows participants to generate public-private keys, exchange public and private keys, and commute a common session key.

Glossary

211

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
digital signature Digitally signed hash of a user data. digital signing This is the process of generating a hash value or check sum by applying an algorithm to a file. The check sum is then used by the recipient of the file to verify that the contents of the file have not been altered during transmission from the sender to the receiver. The check sum is protected by its being encrypted with the signers private key. The resulting value is called a signature. digital verification This is the process of decrypting a signature with the public key of the signer. The signers public key resides in the signers certificate, which must be stored in the key database used by an application operating in the verifier role. distinguished name (DN) The fully-qualified hierarchical names that uniquely identify a specific entity that is authenticated by a digital certificate. DSA A type of public-key algorithm used to encrypt and decrypt a signature passed from a private key to a public key. It is also known as DSS, which is stands for the USA's federal Digital Signature Standard. For other key types, see RSA. DSS See DSA. encryption key A key that is used by an encryption algorithm to encrypt a message or data. key A key is a number (large) or set of numbers that possess mathematical properties that support both
I I

encryption with a private key and decryption with a public key encryption with a public key and decryption with a private key

key database Also referred to as a key file and designated by the extension *.kdb, this file contains all the information necessary to verify a certificate. The file is encrypted with 3DES-CBC encryption and protected by a password. Its contents include
I I I I

public keys for the software application and for the trusted roots private keys for the software application user certificates and trusted roots Certificate Revocation Lists (CRL)

212

PATROL Security User Guide

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Depending upon the various roles of a computer, more than one key database can exist on a single computer. The key database can contain any number of CAs, private and public keys, and user certificates. key file See key database. key pair A set of two cryptographic keys, one public and freely shared, one private and kept secret, used to encrypt and decrypt data. Synonym: public/private key pair. KM See Knowledge Module (KM). Knowledge Module (KM) A set of files from which a PATROL Agent receives information about resources running on a monitored computer. A KM file can contain the actual instructions for monitoring objects or simply a list of KMs to load. KMs are loaded by a PATROL Agent and a PATROL Console. KMs provide information for the way monitored computers are represented in the PATROL interface, for the discovery of application instances and the way they are represented, for parameters that are run under those applications, and for the options available on object pop-up menus. A PATROL Console in the developer mode can change KM knowledge for its current session, save knowledge for all of its future sessions, and commit KM changes to specified PATROL Agent computers. label This is a descriptive, alphanumeric text string that is assigned to a key pair or password in the key database to help an administrator identify and manage the key and/or password. In the sslcmd utility, a label is also referred to as identity. labeled password Sometimes the need arises for some means of securely storing the passwords to other systems in the key database. The sslcmd utility provides a means to assign to a password or other string of bytes a descriptive text string to help identify and manage the password. PATROL Agent The core component of PATROL architecture. The agent is used to monitor and manage host computers and can communicate with the PATROL Console, a stand-alone event manager (PEM), PATROL Integration products, and SNMP consoles. From the command line, the PATROL Agent is configured by the pconfig utility; from a graphical user interface, it is configured by the xpconfig utility for Unix or the wpconfig utility for Windows. PATROL Command Line Interface (CLI) An interface program that you can access from the command line of a monitored computer and through which you can run some PATROL products and utilities. With the CLI, you can monitor the state of PATROL Agents remotely, execute PSL functions, and query and control

Glossary

213

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
events. The CLI is used in place of the PATROL Console when memory and performance constraints exist. PATROL Console The graphical user interface from which you launch commands and manage the environment monitored by PATROL. The PATROL Console displays all of the monitored computer instances and application instances as icons. It also interacts with the PATROL Agent and runs commands and tasks on each monitored computer. The dialog is event-driven so that messages reach the PATROL Console only when a specific event causes a state change on the monitored computer. A PATROL Console with developer functionality can monitor and manage computer instances, application instances, and parameters; customize, create, and delete locally loaded Knowledge Modules and commit these changes to selected PATROL Agent computers; add, modify, or delete event classes and commands in the Standard Event Catalog; and define expert advice. A PATROL Console with operator functionality can monitor and manage computer instances, application instances, and parameters and can view expert advice but not customize or create KMs, commands, and parameters. PATROL roles In PATROL 3.x and earlier, a set of permissions that grant or remove the ability of a PATROL Console or PATROL Agent to perform certain functions. PATROL roles are defined in the PATROL User Roles file, which is read when the console starts. PATROL Script Language (PSL) A scripting language (similar to Java) that is used for generic system management and that is compiled and executed on a virtual machine running inside the PATROL Agent. PSL is used for writing application discovery procedures, parameters, recovery actions, commands, and tasks for monitored computers within the PATROL environment. Pluggable Authentication Mode (PAM) PAM is a library for authentication-related services. This library enables a system administrator to add new authentication methods by installing new PAM modules and to modify authentication policies by editing configuration files. PSL See PATROL Script Language (PSL). Policy See security policy. public key infrastructure (PKI) This infrastructure provides the means for performing public and private key cryptography. PKI-based security includes Secure Socket Layer (SSL), digital signing, and verification.

214

PATROL Security User Guide

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Public-Key Cryptography Standard (PKCS) A set of specifications produced by RSA Laboratories and developers worldwide for the purpose of standardizing public-key cryptography. RSA A type of public-key algorithm used to encrypt and decrypt a signature passed from a private key to a public key. RSA is an acronym for Rivest, Shamir, and Adelman. For other key types, see DSA. secure socket layer (SSL) protocol A standard protocol created by Netscape for secure message transmission in a network. It provides a cryptographic protocol for both mutual authentication and data protection of Internet communications. security policy A centralized location in which configuration data regarding security is stored. A security policy enables a user to easily manage and apply common administrative rules of protection to its computer environment. Security policy information is stored as registry entries on computers running Microsoft Windows operating systems and *.plc files on computers running supported variations of Unix. self-signed certificate A self-signed certificate is a certificate issued directly by the certificate authority (CA). It is also referred to as a trusted root authority certificate. setup command A command that is initiated by the PATROL Console and run by the PATROL Agent when the PATROL Console connects or reconnects to the agent. For example, a setup command can initialize an application log file to prepare it for monitoring. PATROL provides some setup commands for computer classes. Only a PATROL Console with developer functionality can add or change setup commands. signing The actions that a certificate authority (CA) takes to create a valid digital certificate by first hashing the certificate contents and then signing the hash with the CA's private key. This process is also referred to as digital signing. sslcmd This is the key management utility used to create, set up, and manage key databases and certificates. startup command See setup command. trusted root certificate authority The final certificate authority whose digital signature and certificate completes the validation of a digital certificate.

Glossary

215

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
user credentials This is the user name and password used by an application to verify the identity of a user. Some PATROL applications store user credentials in a key database. User credentials can be added to, viewed, or deleted from a key database using the sslcmd key management utility. User credentials are also referred to as labeled password. user profile The PATROL Web Central specific information that is associated with a particular user. It corresponds directly to the user and is defined by the user back-end. The groups to which certain users belong are properties of that user. user profile template What you use to create your profile. It contains the default information in your user profile, but does not map to the user group.

216

PATROL Security User Guide

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Index
A
access control lists (ACLs) 17 access file ALLOW_ACL 165 DENY_ACL 165 anonymous communications 30 apidll.dll 168 application policy 100 locations 107, 109 attended setting mode 134 attended mode 19, 25 attributes 103 authenticated communications 31 authentication 17, 19, 32 creating key database 70 customer support 3

D
default certificates 34, 65 default keys 34, 65 default modes (attended, unattended) 28 default password 65 deleting private and public key pair 75, 93 Diffie-Hellman key exchange 18, 31 digital signing 17 directory structure 56 distinguished name (DN) 90 dlls.conf 168 DSA 72

B
BMC Software, contacting 2 bmckeycli 68 bmcryptpw 113, 138, 139

E
error messages 179, 180, 181, 183 esi_lib32 159 esi_lib64 159 esstool 111 Extended Security Interface (ESI) 157, 159

C
certificate revoking 94 certificate authority (CA) 31 certificate authority (CA) certificate 83 certificate revocation list (CRL) 95 certificate revocation list warning 95 certificate signing request (CSR) 83 certificates 84, 92 deleting 88 viewing field information 87 changing label of key pair 74 password for key database 71 communications privacy 17 compatibility versions 49 config.default 161 configuration 21 configuration files 156 PATROL 106 connection type 56

F
files configuration 106 install location 56

G
generating public and private Keys 72

I
identity 32 impersonation 18 installation

Index

217

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
directories 56 files 56 migration 48 over-the-top 48 installing a certificate in the SSL key database 92 installing new certificate revocation lists 95 issues and workarounds 179 overview of security 16 overwriting warning against 51

P
PAM support 121 password 25 changing for key database 71 password privacy and configuration 17 password prompt 184 PATROL Event Manager (PEM) applications 170 PATROL Knowledge Modules (KMs) 34 patrol.conf 157, 159 plc_password 112, 114, 135 Pluggable Authentication Module AIX 123 Pluggable Authentication Module support 121 policy application 100 site 100 policy attributes 103 policy implementation windows 109 private-public keys listing 73 product support 3 public and private key pair 72 public and private keys 32, 72

K
Kerberos supported by AIX 123 key database 21, 31, 32, 70, 73, 85, 93 changing password for 71 creating 70 shipped with PATROL 69 key databases management of 69 key material file 21 key pair 32, 72 changing label 74 keys management of 69

L
label changing of for key pair 74 level of security selecting 54 listing private-public keys 73 listing signed certificates 93

R
registry entries 184 registry key 110 revoking user certificates 94 root authority certificate installing 85 verifying 86 RSA 72

M
managing keys 69 managing key databases 69 message privacy 32 mode setting 134 modes (attended, unattended) 25

S
Secure Socket Layer (SSL) protocol 19, 32 security selecting level of 54 security contents overwriting 51 security levels 18 costs and benefits 22 level 1 18 security overview 16 security policies 106 selecting level of security 54

N
naming conventions 32 network protocol selecting 56

O
overhead 22

218

PATROL Security User Guide

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
network protocol 56 server.kdb 32 setting attended mode 134 mode 134 unattended mode 134 setting mode 134 signFile 141 site policy 100 locations 107 sslcmd 67, 70, 71, 72, 73, 74, 75, 76, 78, 80, 81, 82, 84, 85, 86, 87, 88, 89, 92, 93, 95, 115, 117, 126, 130, 131, 145, 148 support, customer 3

T
technical support 3 transaction 30 trusted root authority 19, 32 trusted third party 31

U
unattended 134 unattended mode 19, 25 usability 21, 22 user certificate revoking 94 user rights and privileges 17 utility bmckeycli 68

V
verifyFile 143 version compatibility 49

W
Windows registry 109

Index

219

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

220

PATROL Security User Guide

Third-Party Product Terms


The following terms apply to third-party products that are included with or in a BMC Software product as described in the BMC Software, Inc., License Agreement that is applicable to the product.

Frank Cusack License


Copyright (c) Frank Cusack, 1999-2000. fcusack@fcusack.com All rights reserved 1. Redistributions of source code must retain the above copyright notice, and the entire permission notice in its entirety, including the disclaimer of warranties. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

OpenSSL License
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/). Copyright (c) 1998-2003 The OpenSSL Project. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgment:: "This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)" 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact openssl-core@openssl.org. 5. Products derived from this software may not be called "OpenSSL" nor may "OpenSSL" appear in their names without prior written permission of the OpenSSL Project. 6. Redistributions of any form whatsoever must retain the following acknowledgment: "This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/)" THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ==================================================================== This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). (tjh@cryptsoft.com). This product includes software written by Tim Hudson

Original SSLeay License


Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) All rights reserved. This package is an SSL implementation written by Eric Young (eay@cryptsoft.com). The implementation was written so as to conform with Netscapes SSL. This library is free for commercial and non-commercial use as long as the following conditions are aheared to. The following conditions apply to all code found in this distribution, be it the RC4, RSA, lhash, DES, etc., code; not just the SSL code. The SSL documentation included with this distribution is covered by the same copyright terms except that the holder is Tim Hudson (tjh@cryptsoft.com). Copyright remains Eric Young's, and as such any Copyright notices in the code are not to be removed. If this package is used in a product, Eric Young should be given attribution as the author of the parts of the library used. This can be in the form of a textual message at program startup or in documentation (online or textual) provided with the package. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. All advertising materials mentioning features or use of this software must display the following acknowledgement:"This product includes cryptographic software written by Eric Young (eay@cryptsoft.com)" The word 'cryptographic' can be left out if the rouines from the library being used are not cryptographic related ). 4. If you include any Windows specific code (or a derivative thereof) from the apps directory (application code) you must include an acknowledgement: "This product includes software written by Tim Hudson (tjh@cryptsoft.com)" THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

*53017* *53017* *53017* *53017*


*53017*

S-ar putea să vă placă și