Design and Deployment Guide For Windows HPC Server 2008

Design and Deployment Guide for Windows
HPC Server 2008

Microsoft Corporation
Published: September 2008
Abstract
This guide provides detailed information and step-by-step procedures for designing and installing
a high performance computing cluster using Windows® HPC Server 2008. You can use this guide
to first plan the deployment of your HPC cluster, and when you are ready to deploy you can follow
the procedures to configure the head node, add compute nodes to the cluster, and verify that your
cluster deployment has been successful.
Copyright Information
Information in this document, including URL and other Internet Web site references, is subject to
change without notice. Unless otherwise noted, the companies, organizations, products, domain
names, e-mail addresses, logos, people, places, and events depicted in examples herein are
fictitious. No association with any real company, organization, product, domain name, e-mail
address, logo, person, place, or event is intended or should be inferred. Complying with all
applicable copyright laws is the responsibility of the user. Without limiting the rights under
copyright, no part of this document may be reproduced, stored in or introduced into a retrieval
system, or transmitted in any form or by any means (electronic, mechanical, photocopying,
recording, or otherwise), or for any purpose, without the express written permission of Microsoft
Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual
property rights covering subject matter in this document. Except as expressly provided in any
written license agreement from Microsoft, the furnishing of this document does not give you any
license to these patents, trademarks, copyrights, or other intellectual property.
© 2008 Microsoft Corporation. All rights reserved.
Microsoft Active Directory, Windows, Windows PowerShell, Windows Server, and Windows Vista
are trademarks of the Microsoft group of companies.
All other trademarks are property of their respective owners.

Contents
Design and Deployment Guide for Windows HPC Server 2008......................................................4
Checklist: Deploy an HPC Cluster (Overview)................................................................................4
Step 1: Prepare for Your Deployment.............................................................................................5
Step 2: Deploy the Head Node......................................................................................................11
Step 3: Configure the Head Node.................................................................................................13
Step 4: Add Compute Nodes to the Cluster..................................................................................20
Step 5: Run Diagnostic Tests on the Cluster.................................................................................26
Step 6: Run a Test Job on the Cluster...........................................................................................26
Additional Resources....................................................................................................................30
Appendices...................................................................................................................................30
Appendix 1: HPC Cluster Networking........................................................................................31
Appendix 2: Creating a Node XML File......................................................................................42
Appendix 3: Node Template Tasks and Properties....................................................................50
Appendix 4: Job Template Properties........................................................................................58
Appendix 5: Scriptable Power Control Tools..............................................................................64
Appendix 6: Using HPC PowerShell..........................................................................................65
Design and Deployment Guide for Windows
HPC Server 2008
This guide provides conceptual information for planning the deployment of a high performance
computing cluster using Windows® HPC Server 2008. It also provides step-by-step procedures
for deploying the head node in your cluster, adding compute nodes, and for verifying that your
deployment was successful.
Note
You can configure your HPC cluster for high availability by installing the head node in the
context of a failover cluster. If the server that is acting as the head node fails, the other
server in the failover cluster automatically begins acting as the head node (through a
process known as failover). For more information about running an HPC cluster with
failover clustering, see http://go.microsoft.com/fwlink/?LinkId=123894.
Checklist: Deploy an HPC Cluster (Overview)

The following checklist describes the overall process of designing and deploying a Windows HPC
Server 2008 cluster. Each task in the checklist is linked to the section in this document that
describes the steps to perform the task.
Task Description
Step 1: Prepare for Your Deployment Before you start deploying your HPC cluster,
review the list of prerequisites and initial
considerations.
Step 2: Deploy the Head Node Deploy the head node by installing Windows
Server 2008 and HPC Pack 2008.
Step 3: Configure the Head Node Configure the head node by following the steps
in the configuration to-do list.
Step 4: Add Compute Nodes to the Cluster Add nodes to the cluster by deploying them
from bare metal, by importing an XML file, or by
manually configuring them.
Step 5: Run Diagnostic Tests on the Cluster Run diagnostic tests to verify that the
deployment of the cluster was successful.
Step 6: Run a Test Job on the Cluster Run some basic jobs on the cluster to verify that
the cluster is operational.
4
Step 1: Prepare for Your Deployment
The first step in the deployment of your HPC cluster is to make important decisions, such as
deciding how you will be adding nodes to your cluster, and choosing a network topology for your
cluster. The following checklist describes the steps involved in preparing for your deployment.
Checklist: Prepare for your deployment

Task Description
1.1. Review initial considerations and system Review the list of initial considerations and
requirements system requirements to ensure that you have all
the necessary hardware and software
components to deploy an HPC cluster.
1.2. Decide how to add compute nodes to your Decide if you will be adding compute nodes to
cluster your cluster from bare metal, as preconfigured
nodes, or using an XML file.
1.3. Choose the Active Directory domain for Choose the Active Directory® domain to which
your cluster you will join the head node and compute nodes
of your HPC cluster.
1.4. Choose a user account for installation and Choose an existing domain account with
diagnostics enough privileges to perform installation and
diagnostics tasks.
1.5. Choose a network topology for your cluster Choose how the nodes in your cluster will be
connected, and how the cluster will be
connected to your enterprise network.
1.6. Prepare for multicast (optional) If you will be deploying nodes from bare metal
and want to multicast the operating system
image that you will be using during deployment,
configure your network switches appropriately.
1.7. Prepare for the integration of scriptable If you want to use your own power controls
power control tools (optional) tools to start, shut down, and reboot compute
nodes remotely, obtain and test all the
necessary components of your power control
tools.
5
1.1. Review initial considerations and system
requirements
The following sections list some initial considerations that you need to review, as well as
hardware and software requirements for Windows HPC Server 2008.
Initial considerations
Review the following initial considerations before you deploy your HPC cluster.
Compatibility with previous versions

The following list describes compatibility between Windows HPC Server 2008 and Windows
Compute Cluster Server 2003:
• Windows HPC Server 2008 provides application programming interface (API)-level
compatibility for applications that are integrated with Windows Compute Cluster Server 2003.
These applications might, however, require changes to run on Windows Server® 2008. If you
encounter problems running your application on Windows Server 2008, you should consult
your software vendor.
• Windows HPC Server 2008 supports job submission from Windows Compute Cluster
Server 2003 clients, including jobs that are submitted through the use of the command-line
tools, the Compute Cluster Job Manager, and the COM APIs.
• The Windows HPC Server 2008 client tools, including the cluster administration console
(HPC Cluster Manager), the job scheduling console (HPC Job Manager), the command-line
tools, and the APIs cannot be used to manage or submit jobs to a Windows Compute Cluster
Server 2003 cluster.
• Clusters that have both Windows Compute Cluster Server 2003 nodes and Windows
HPC Server 2008 nodes are not supported.
• A side-by-side installation of Windows HPC Server 2008 and Windows Compute Cluster
Server 2003 on the same computer is not supported. This includes the Windows HPC Server
2008 client utilities.
• The upgrade of a Windows Compute Cluster Server 2003 head node to a Windows HPC
Server 2008 head node is not supported.
Server roles added during installation

The installation of HPC Pack 2008 adds the following server roles to the head node:
• Dynamic Host Configuration Protocol (DHCP) Server, to provide IP addresses and
related information for compute nodes.
• Windows Deployment Services, to deploy compute nodes remotely.
• File Services, to manage shared folders.
• Network Policy and Access Services, which enables Routing and Remote Access so that
network address translation (NAT) services can be provided to the cluster nodes.
6
Hardware requirements
Hardware requirements for Windows HPC Server 2008 are very similar to those for the 64-bit
editions of Windows Server 2008.
Note
For more information about installing Windows Server 2008, including system
requirements, see Installing Windows Server 2008 (http://go.microsoft.com/fwlink/?
LinkID=119578).
Processor (x64-based):
• Minimum: 1.4 GHz
• Recommended: 2 GHz or faster
RAM:
• Minimum: 512 MB
• Recommended: 2 GB or more
Available disk space:
• Minimum: 50 GB
• Recommended: 80 GB or more
Drive:
• DVD-ROM drive
Network adapters:
• The number of network adapters on the head node and on the compute nodes depends
on the network topology that you choose for your cluster. For more information about the
different HPC cluster network topologies, see Appendix 1: HPC Cluster Networking.
Software requirements
The following list outlines the software requirements for the head node and the compute nodes in
a Windows HPC Server 2008 cluster:
• Windows Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008
• Microsoft HPC Pack 2008
Important
Microsoft HPC Pack 2008 cannot be installed on any edition of Windows Server 2008 R2.
It can only be installed on Windows Server 2008 HPC Edition, or another 64-bit edition of
Windows Server 2008.
To enable users to submit jobs to your HPC cluster, you can install the utilities included with
Microsoft HPC Pack 2008 on client computers. Those client computers must be running any of
the following operating systems:
• Windows XP Professional with Service Pack 3 or later (x86- or x64-based)
7
• Windows Vista® Enterprise, Windows Vista Business, Windows Vista Home, or Windows
Vista Ultimate
• Windows Server 2003 Standard Edition or Windows Server 2003 Enterprise Edition with
Service Pack 2 or later (x86- or x64-based)
• Windows Server 2003, Compute Cluster Edition
• Windows Server 2003 R2 Standard Edition or Windows Server 2003 R2 Enterprise
Edition (x86- or x64-based)
1.2. Decide how to add compute nodes to your

cluster
There are three ways to add compute nodes to your cluster:
• From bare metal. The operating system and all the necessary HPC cluster components
are automatically installed on each compute node as it is added to the cluster. No manual
installation of the operating system or other software is required.
• Add preconfigured compute nodes. The compute nodes are already running Windows
Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008, and Microsoft
HPC Pack 2008 is manually installed on each node.
• Import a node XML file. An XML file that contains a list of all the nodes that will be
deployed is used. This XML file can be used to add nodes from bare metal or from
preconfigured nodes. For more information about node XML files, see Appendix 2: Creating a
Node XML File.
The following is a list of details to take into consideration when choosing how to add nodes to
your HPC cluster:
• When deploying nodes from bare metal, Windows HPC Server 2008 automatically
generates computer names for your compute nodes. During the configuration process, you
will be required to specify the naming convention to use when automatically generating
computer names for the new nodes.
• Compute nodes are assigned their computer name in the order that they are deployed.
• If you want to add compute nodes from bare metal and assign computer names in a
different way, you can use a node XML file. For more information about node XML files, see
Appendix 2: Creating a Node XML File.
• If you want to add preconfigured nodes to your cluster, you will need to install Windows
Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008 on each node (if
not already installed), as well as Microsoft HPC Pack 2008.
8
1.3. Choose the Active Directory domain for your
cluster
The head node and the compute nodes in your HPC cluster must be members of an Active
Directory domain. Before deploying your cluster, you must choose the Active Directory domain
that you will use for your HPC cluster.
If you do not have an Active Directory domain to which you can join your cluster, or if you prefer
not to join an existing domain, you can install the Active Directory Domain Services role on the
head node and then configure a domain controller on that node. For more information about
installing the Active Directory Domain Services role on a computer that is running Windows
Server 2008, see the AD DS Installation and Removal Step-by-Step Guide
(http://go.microsoft.com/fwlink/?LinkID=119580).
Caution
If you choose to install and configure an Active Directory domain controller on the head
node, consult with your network administrator about the correct way to isolate the new
Active Directory domain from the enterprise network, or how to join the new domain to an
existing Active Directory forest.
1.4. Choose a user account for installation and

diagnostics
During the configuration process of your HPC cluster, you must provide credentials for a domain
user account that will be used for installation and diagnostics. You must choose an existing
account or create a new account, before starting your cluster deployment.
The following is a list of details to take into consideration when choosing the user account:
• The user account that you choose must be a domain account with enough privileges to
create Active Directory computer accounts for the compute nodes. Alternatively, you can
create the computer accounts manually or ask your domain administrator to create them for
you.
• If part of your deployment requires access to resources on the enterprise network, the
user account must have the necessary permissions to access those resources—for example,
installation files that are available on a network server.
• If you want to restart nodes remotely from the cluster administration console (HPC
Cluster Manager), the account must be a member of the local Administrators group on the
head node. This requirement is only necessary if you do not have scriptable power control
tools that you can use to remotely restart the compute nodes.
9
1.5. Choose a network topology for your cluster
Windows HPC Server 2008 supports five cluster topologies. These topologies are distinguished
by how the compute nodes in the cluster are connected to each other and to the enterprise
network. The five supported cluster topologies are:
• Topology 1: Compute Nodes Isolated on a Private Network
• Topology 2: All Nodes on Enterprise and Private Networks
• Topology 3: Compute Nodes Isolated on Private and Application Networks
• Topology 4: All Nodes on Enterprise, Private, and Application Networks
• Topology 5: All Nodes on an Enterprise Network
For more information about each network topology, see Appendix 1: HPC Cluster Networking.
When you are choosing a network topology, you must take into consideration your existing
network infrastructure:
• Decide which network in the topology that you have chosen will serve as the enterprise
network, the private network, and the application network.
• Do not have the network adapter that is connected to the enterprise network on the head
node in an automatic configuration (that is, the IP address for that adapter does not start with:
169.254). That adapter must have a valid IP address, dynamically or manually assigned
(static).
• If you choose a topology that includes a private network, and you are planning to add
nodes to your cluster from bare metal:
• Ensure that there are no Pre-Boot Execution Environment (PXE) servers on the
private network.
• If you want to use an existing DHCP server for your private network, ensure that it is
configured to recognize the head node as the PXE server in the network.
• If you want to enable DHCP server on your head node for the private or application
networks and there are other DHCP servers connected to those networks, you must disable
those DHCP servers.
• If you have an existing Domain Name System (DNS) server connected to the same
network as the compute nodes, no action is necessary, but the compute nodes will be
automatically deregistered from that DNS server.
• Contact your system administrator to determine if Internet Protocol security (IPsec) is
enforced on your domain through Group Policy. If IPsec is enforced on your domain through
Group Policy, you may experience issues during deployment. A workaround is to make your
head node an IPsec boundary server so that compute nodes can communicate with the head
node during PXE boot.
1.6. Prepare for multicast (optional)

If you will be deploying nodes from bare metal and want to multicast the operating system image
that you will be using during deployment, we recommend that you prepare for multicast by:
10
• Enabling Internet Group Management Protocol (IGMP) snooping on your network
switches, if this feature is available. This will help to reduce multicast traffic.
• Disabling Spanning Tree Protocol (STP) on your network switches, if this feature is
enabled.
Note
For more information about these settings, contact your network administrator or your
networking hardware vendor.
1.7. Prepare for the integration of scriptable power

control tools (optional)
The cluster administration console (HPC Cluster Manager) includes actions to start, shut down,
and reboot compute nodes remotely. These actions are linked to a script file (CcpPower.cmd) that
performs these power control operations using operating system commands. You can replace the
default operating system commands in that script file with your own power control scripts, such as
Intelligent Platform Management Interface (IPMI) scripts that are provided by your vendor of
cluster solutions.
In preparation for this integration, you must obtain all the necessary scripts, dynamically linked
library (DLL) files, and all other components of your power control tools. After you have obtained
all the necessary components, test them independently and ensure that they work as intended on
the computers that you will be deploying as compute nodes in your cluster.
For information about modifying CcpPower.cmd to integrate your own scriptable power control
tools, see Appendix 5: Scriptable Power Control Tools.
Step 2: Deploy the Head Node

The next step in the deployment of your HPC cluster is to deploy the head node. The following
checklist describes the steps involved in deploying the head node.
Checklist: Deploy the head node

Task Description
2.1. Install Windows Server 2008 on the head Install Windows Server 2008 HPC Edition, or
node computer another 64-bit edition of Windows Server 2008
on the computer that will act as the head node.
2.2. Join the head node computer to a domain Join the computer that will act as the head node
to a Microsoft Active Directory Domain.
2.3. Install Microsoft HPC Pack 2008 on the Install Microsoft HPC Pack 2008 on the
11
Task Description
head node computer computer that will act as the head node, using
the installation media or from a network
location.
2.1. Install Windows Server 2008 on the head node

computer
To deploy the head node of your HPC cluster, you must start by installing Windows Server 2008
HPC Edition, or another 64-bit edition of Windows Server 2008 on the computer that will act as
the head node. For more information about installing Windows Server 2008, including system
requirements, see Installing Windows Server 2008 (http://go.microsoft.com/fwlink/?
LinkID=119578).
Important
We strongly recommend that you perform a clean installation of Windows Server 2008
before installing Microsoft HPC Pack 2008. If you want to install Microsoft HPC
Pack 2008 on an existing installation of Windows Server 2008, remove all server roles
first and then follow the procedures in this guide.
Note
It is recommended that you obtain the latest device drivers for your head node computer
from the Web site of your hardware vendors.
2.2. Join the head node computer to a domain

As described in the Step 1: Prepare for Your Deployment section, the head node must be a
member of an Active Directory domain. After you have installed Windows Server 2008 on the
head node, manually join the head node to an existing Active Directory domain.
2.3. Install Microsoft HPC Pack 2008 on the head

node computer
After Windows Server 2008 is installed on the head node computer, and the head node is joined
to an Active Directory domain, you can install Microsoft® HPC Pack 2008 on the head node.
To install Microsoft HPC Pack 2008 on the head node computer
1. To start the Microsoft HPC Pack 2008 installation wizard on the computer that will act
as the head node, run setup.exe from the HPC Pack 2008 installation media or from a
network location.
12
2. On the Getting Started page, click Next.
3. On the Microsoft Software License Terms page, read or print the software license
terms in the license agreement, and accept or reject the terms of that agreement. If you
accept the terms, click Next.
4. On the Select Installation Type page, click Create a new HPC cluster by creating
a head node, and then click Next.
5. Continue to follow the steps in the installation wizard.
Step 3: Configure the Head Node

After you have deployed the head node of your HPC cluster, you must configure the head node
by following the configuration to-do list in HPC Cluster Manager.
Checklist: Configure the head node

The following checklist includes the items in the configuration to-do list in HPC Cluster Manager
that you need to complete in order to configure your head node.
Task Description
3.1. Configure the HPC cluster network Configure the cluster network by using the
Network Configuration Wizard.
3.2. Provide installation credentials Specify which credentials to use for system
configuration and when adding new nodes to
the cluster.
3.3. Configure the naming of new nodes Specify the naming convention to use when
generating names automatically for new
compute nodes.
3.4. Create a node template Create a template that defines the steps to
follow when configuring a compute node.
3.5. Add drivers for the operating system If you will be deploying compute nodes from
images (optional) bare metal and those nodes require special
device drivers, add drivers for the operating
system images that you created for your node
template on the previous task.
3.6. Add or remove users (optional) If you will be giving access to the cluster to
other members of your organization, add or
remove users or administrators for your cluster.
13
3.1. Configure the HPC cluster network
The HPC cluster network configuration is the first step in the configuration process of your head
node. The HPC cluster network is configured by following the Network Configuration Wizard in
HPC Cluster Manager. When configuring the HPC cluster network, you must choose the network
topology that you have selected for your cluster, as described in “1.5. Choose a network topology
for your cluster”, in Step 1: Prepare for Your Deployment.
Important
Before you start configuring the HPC cluster network in HPC Cluster Manager, ensure
that the head node and the computers that you will add as compute nodes to the cluster
are physically connected according to the network topology that you have chosen for
your cluster. Also, ensure that you are able to identify to which network each one of the
network adapters in the head node is connected.
To configure the HPC cluster network
1. If HPC Cluster Manager is not already open on the head node, open it. Click Start,
point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager.
2. In the To-do List, click Configure your network. The Network Configuration
Wizard appears.
3. On the Network Topology Selection page, click the topology that you have chosen
for your cluster, and then click Next.
4. On the Enterprise Network Adapter Selection page, in the Network adapter list,
click the name of the network adapter that is physically connected to your enterprise
network, and then click Next.
Important
To ensure that you are selecting the correct network adapter, use the
information displayed on this wizard page after you select a network adapter
from the list. Use the IP address, domain information, and Media Access
Control (MAC) address of each adapter as a reference.
5. If you chose topology number 5 for your cluster, jump to step 9 in this procedure.
Otherwise, repeat step 4 for the private network adapter.
6. On the Private Network Configuration page, type a static IP address and a subnet
mask for the head node. Optionally, select network services for that network:
a. To give access to resources on the enterprise network to compute nodes that are
connected to this network, select the Enable network address translation (NAT) on
the head node check box.
b. To enable DHCP services for the nodes connected to this network, select the
Enable DHCP and define a scope check box, and then type the starting and ending
IP addresses for the DHCP scope. If the Gateway and DNS server IP addresses
have not been automatically detected, type each of these addresses.
14
Note
For more information about enabling NAT and DHCP on your cluster
network, see “HPC network services” in Appendix 1: HPC Cluster
Networking.
7. Click Next after you are done configuring the private network.
8. Repeat steps 4, 6, and 7 for the application network adapter. Click Next after you are
done configuring the application network.
9. On the Firewall Setup page, select the firewall setting for the cluster:
a. To apply firewall settings automatically to head nodes and compute nodes on
each network, click ON for that network.
b. To disable the firewall on a network, click OFF.
c. If you do not want to change any firewall settings, click Do not manage firewall
settings.
Note
For more information about firewall settings for your cluster, see “Windows
Firewall configuration” in Appendix 1: HPC Cluster Networking.
10. On the Review page, verify your settings and click Configure. If you want to change
any of the settings, navigate to the appropriate wizard page by clicking it on the
navigation pane or by clicking Previous.
11. After the network configuration process is completed, on the Configuration
Summary page, review the list of configuration items. If you want to save a report of the
network configuration, click Save the configuration report.
12. To close the wizard, click Finish.
3.2. Provide installation credentials

Installation credentials must be provided in order to configure new compute nodes. These
credentials will be used when installing the operating system, applications, and when adding
nodes to the Active Directory domain. Also, these same credentials will be used when running
diagnostic tests on the cluster nodes. For more information, see “1.4. Choose a user account for
installation and diagnostics” in Step 1: Prepare for Your Deployment.
To provide installation credentials
1. In the To-do List, click Provide installation credentials. The Installation

Credentials dialog box appears.
2. Type the user name, including the domain (DOMAIN\User), and then the password
for the domain user account you will use to deploy compute nodes and to run diagnostic
tests.
Important
15
The account must be a domain account with enough privileges to create
Active Directory computer accounts for the compute nodes. Alternatively, you
can create the computer accounts manually or ask your domain administrator
to create them for you.
Important
If part of your deployment requires access to resources on the enterprise
network, the account should have the necessary permissions to access
those resources.
Important
If you want to restart nodes remotely from the cluster administration console
(HPC Cluster Manager), the account must be added as an HPC cluster
administrator on the head node. This requirement is only necessary if you do
not have scripted power control tools that you can use to remotely restart the
compute nodes.
3. To save the specified credentials, click OK.
3.3. Configure the naming of new nodes

If you deploy compute nodes from bare metal, and you are not using a node XML file to import
nodes to the cluster, Windows HPC Server 2008 will automatically generate computer names for
the new nodes that are being deployed. You need to specify how those names will be generated,
by defining a naming series.
The naming series is defined by selecting a root name and the starting number that will
accompany that name. The starting number is enclosed in percentage signs (%). For example:
ClusterNode%1000%.
When you deploy compute nodes from bare metal, nodes will be named in sequence, as they
become available. For example, if you deploy three nodes after specifying the following naming
series: ClusterNode-%100%, those nodes will be assigned these names:
• ClusterNode-100
• ClusterNode-101
• ClusterNode-102
Important
Compute node names are limited to 15 characters. When specifying the compute node
naming series, take into account the number of compute nodes in your deployment and
ensure that the series that you specify will not generate names that exceed 15
characters. For example, if your deployment will consist of 1,000 compute nodes, and
your starting number is 1, your root name cannot have more than 12 characters;
otherwise, your node number 1,000 will need a name that consists of 16 characters.
16
To specify the compute node naming series
1. In the To-do List, click Configure the naming of new nodes. The Specify
Compute Node Naming Series dialog box appears.
2. Type the naming series that you want to use. The preview helps you to see an
example of how the naming series will be applied to the names of the compute nodes.
Note
You cannot specify a compute node naming series that consists only of
numbers.
3. To save the compute node naming series that you have specified, click OK.
3.4. Create a node template

Node templates are new in Windows HPC Server 2008. They define the necessary tasks for
configuring and adding compute nodes to your cluster. With a node template, you can deploy an
operating system image, add specific drivers and software to compute nodes, or simply add a
preconfigured node to your cluster. Because you might have more than one type of compute
node, or you may be adding compute nodes to your cluster in different ways, you can create
different templates that apply to different nodes or situations.
You can create two types of node templates:
• With an operating system image. This type of template includes a step to deploy an
operating system on the compute nodes. Use this type of template when adding compute
nodes from bare metal.
• Without an operating system image. This type of template is used to add preconfigured
compute nodes to the cluster, or to update existing nodes.
The type of template that you create for the initial deployment of your HPC cluster depends on
how you decided to add compute nodes to your cluster. For more information, see “1.2. Decide
how to add compute nodes to your cluster” in Step 1: Prepare for Your Deployment.
Important
If you will create a node template with an operating system image, you will need the
installation media for Windows Server 2008 HPC Edition or another 64-bit edition of
Windows Server 2008, or you must have the installation files available on a network
location that is accessible from the head node computer.
To create a node template
1. In the To-do List, click Create a node template. The Create Node Template
Wizard appears.
2. On the Specify Template Name page, type a descriptive name for the template, and
then click Next.
17
3. If you will be adding compute nodes to your cluster from bare metal:
a. On the Select Deployment Type page, click With operating system, and then
click Next.
b. If the operating system image that you want to use for your deployment is
already listed in the Image Name list, click that image and then jump to step 3.f. If
you want to use a different operating system image, on the Select Operating
System Image page, click Add Image.
c. On the Add Operating System Image window, click Create a new operating
system image, and then type or browse to the location of the Windows setup file for
one of the 64-bit editions of Windows Server 2008.
d. Type a descriptive name for the new operating system image, and then click OK.
e. After the image is added, in the Image Name list, click the image that you want
to use with the template.
f. Optionally, specify if you want to multicast the operating system image during
deployment. For more information, see “1.6. Prepare for multicast (optional)” in Step
1: Prepare for Your Deployment.
g. Optionally, specify if you want to include a product key to activate the operating
system on the compute nodes, and then type the product key that should be used.
h. Click Next to continue.
i. On the Specify Local Administrator Password for Compute Node page, click
Use a specific password, and then type and confirm the password that you want to
use.
j. Click Next to continue, and then jump to step 5 in this procedure.
4. If you will be adding preconfigured compute nodes to your cluster, on the Select
Deployment Type page, click Without operating system, and then click Next.
5. On the Specify Windows Updates page, specify if you want to add a step in the
template to download and install updates using Microsoft Update or the enterprise
Windows Server Update Services (WSUS). Also, you can specify specific updates to be
added to the template. Click Next to continue.
6. On the Review page, click Create.
Note
The node templates that you create with the Create Node Template Wizard include
the most common deployment and configuration tasks. You can add more tasks to
the node templates that you create by using the Node Template Editor. For more
information, see Appendix 3: Node Template Tasks and Properties.
18
3.5. Add drivers for the operating system images
(optional)
If you will be deploying compute nodes from bare metal and those nodes require special device
drivers, you will need to add those drivers during the configuration process of your head node.
Drivers must be in the .inf format, and must be accessible from the head node.
Note
The device drivers that you add will be available to all operating system images in the
image store.
Note
It is recommended that you obtain the latest device drivers for your compute nodes from
the Web site of your hardware vendors.
To add drivers for the operating system images
1. In the To-do List, click Manage drivers. The Manage Drivers dialog box appears.
2. To add a driver, click Add.
3. Type or browse to the location of the setup information file for the driver that you want
to add (.inf format), and then click Open.
4. Repeat the two previous steps for all the drivers that you want to add.
5. After you are done adding drivers, click Close.
3.6. Add or remove users (optional)

If you will be giving access to the cluster to other members of your organization, you need to add
them as HPC cluster users or HPC cluster administrators. Also, you can remove users or
administrators that were added by default during installation.
Important
The Domain Users group is added as an HPC cluster user during installation. If you do
not want all users in the domain to have access to your cluster, you can remove the
Domain Users group from the list of HPC cluster users, and add a different domain
group specifically created for users of your HPC cluster, or you can add individual domain
users.
To add or remove users for the cluster
1. In the To-do List, click Add or remove users.

2. To add a user to the cluster:
a. In the Actions pane, click Add User. The Select Users or Groups dialog box
19
appears.
b. Type the user name of the user that you want to add, and then click Check
Names. For more information, on the Select Users or Groups window, click
examples.
c. Repeat the previous step for all users that you want to add.
d. After you are done adding users, click OK.
3. To add an administrator to the cluster:
a. In the Actions pane, click Add Administrator. The Select Users or Groups
dialog box appears.
b. Type the user name of the administrator that you want to add, and then click
Check Names. For more information, on the Select Users or Groups window, click
examples.
c. Repeat the previous step for all administrators that you want to add.
d. After you are done adding administrators, click OK.
4. To remove a user or administrator, select it on the Users list, and then click Remove.
Note
You cannot remove the domain Administrator account from the list of cluster
administrators.
Step 4: Add Compute Nodes to the Cluster

Windows HPC Server 2008 simplifies the deployment process of compute nodes by providing
automatic node imaging, automatic naming of nodes, and other capabilities to streamline
deployment tasks. Also, it provides tools that you can use to monitor the progress of your
deployment.
Important
Unlike previous versions of Windows HPC Server 2008, the default in Windows HPC
Server 2008 is to respond only to Pre-Boot Execution (PXE) requests that come from
existing compute nodes. This default setting is automatically changed when you use the
Add Node Wizard to add nodes from bare metal. Also, you can manually change this
setting in the Options menu, under Deployment Settings.
After creating a node template, you can use the Add Node Wizard to add compute nodes to your
HPC cluster. There are three ways by which you can add compute nodes to your cluster:
• Deploy compute nodes from bare metal
• Add compute nodes by importing a node XML file
• Add preconfigured compute nodes
20
For more information about each of these three node deployment options, see “1.2. Decide how
to add compute nodes to your cluster” in Step 1: Prepare for Your Deployment.
In this section:
• 4.1. Deploy Compute Nodes from Bare Metal
• 4.2. Add Compute Nodes by Importing a Node XML File
• 4.3. Add Preconfigured Compute Nodes
• 4.4. Monitor deployment progress
• 4.5 Cancel the deployment of a node
4.1. Deploy compute nodes from bare metal

The following procedure describes how to add compute nodes to your HPC cluster from bare
metal, by using a node template that includes a step to deploy an operating system image.
Important
To complete this procedure, you must have a template that includes a step to deploy an
operating system image. If you do not have a template that includes a step to deploy an
operating system image, create one by following the steps in “3.4. Create a node
template”, in Step 3: Configure the Head Node.
Important
Before turning on a compute node for this procedure, verify in the configuration of the
BIOS of that computer that the compute node will boot from the network adapter that is
connected to the private network, instead of booting from the local hard drive or another
device, and that Pre-boot Execution Environment (PXE) boot is enabled for that network
adapter.
To deploy compute nodes from bare metal
2. In Node Management, in the Actions pane, click Add Node. The Add Node Wizard
appears.
3. On the Select Deployment Method page, click Deploy compute nodes from bare
metal using an operating system image, and then click Next.
4. On the Select New Nodes page, in the Node template list, click the name of a node
template that includes a step to deploy an operating system image.
5. Turn on the computers that you want to add as compute nodes to your cluster.
Computers will be listed on the Add Node Wizard as they contact the head node during
PXE boot. They will be named using the naming series that you specified when you
configured the head node. For more information, see “3.3. Configure the naming of new
21
nodes” in Step 3: Configure the Head Node.
6. When all computers that you have turned on are listed, click Select all, and then click
Deploy. If you see a node that you do not want to deploy at this time, you can unselect it.
7. On the Completing the Add Node Wizard page, if you will be deploying more
nodes, click Continue responding to all PXE requests. If you will not be deploying
more nodes, click Respond only to PXE requests that come from existing compute
nodes.
8. To monitor deployment progress, select the Go to Node Management to track
progress check box, and then click Finish. For more information, see 4.4. Monitor
deployment progress.
4.2. Add compute nodes by importing a node XML

file
The following procedure describes how to add compute nodes by importing a node XML file.
Important
To complete this procedure, you must have a valid node XML file. For more information,
see Appendix 2: Creating a Node XML File.
To add compute nodes by importing a node XML file
appears.
3. On the Select Deployment Method page, click Import compute nodes from a
node XML file, and then click Next.
4. On the Select Node XML File page, type or browse to the location of the node XML
file, and then click Import.
5. To monitor deployment progress, on the Completing the Add Node Wizard page,
select the Go to Node Management to track progress check box, and then click
Finish. For more information, see 4.4. Monitor deployment progress.
4.3. Add preconfigured compute nodes

A preconfigured compute node is a computer that has HPC Pack 2008 already installed and that
is connected to the HPC cluster network according to the network topology that you have chosen
for your cluster. After HPC Pack 2008 is installed on all the compute nodes that you want to add
to your cluster, you can use the Add Node Wizard on the head node to add the preconfigured
nodes to your cluster.
22
The following procedures describe how to add preconfigured compute nodes to your HPC cluster.
The first procedure describes how to install HPC Pack 2008 on the computers that will act as
compute nodes, and the second procedure describes how to add the preconfigured compute
nodes to the cluster.
Important
The computers that you will add to your cluster as preconfigured compute nodes must
already be running Windows Server® 2008 HPC Edition, or another 64-bit edition of the
Windows Server 2008 operating system.
Important
We strongly recommend that you perform a clean installation of Windows Server 2008
before installing HPC Pack 2008. If you want to install HPC Pack 2008 on an existing
installation of Windows Server 2008, remove all server roles first and then follow the
procedures in this guide.
Important
To complete this procedure, you must have a node template that does not include a step
to deploy an operating system image. If you do not have a node template that does not
include a step to deploy an operating system image, create one by following the steps in
“3.4. Create a node template”, in Step 3: Configure the Head Node.
To install HPC Pack 2008 on a compute node computer
1. To start the HPC Pack 2008 installation wizard on the computer that will act as a
compute node, run setup.exe from the HPC Pack 2008 installation media or from a
network location.
2. On the Getting Started page, click Next.
3. On the Microsoft Software License Terms page, read or print the software license
terms in the license agreement, and accept or reject the terms of that agreement. If you
accept the terms, click Next.
4. On the Select Installation Type page, click Join an existing HPC cluster by
creating a new compute node, and then click Next.
5. On the Join Cluster page, type the computer name of the head node on your cluster,
and then click Next.
6. On the Select Installation Location page, click Next.
7. On the Install Required Components page, click Install.
8. On the Installation Complete page, click Close.
After HPC Pack 2008 is installed on all the compute nodes that you want to add to your cluster,
follow the steps in the Add Node Wizard on the head node to add the preconfigured nodes to
your cluster.
23
To add preconfigured compute nodes to your cluster
appears.
3. On the Select Deployment Method page, click Add compute nodes that have
already been configured, and then click Next.
4. Turn on all the preconfigured nodes that you want to add to your cluster.
5. After all the preconfigured nodes are turned on, on the Before Deploying page, click
Next.
6. On the Select New Nodes page, in the Node template list, click the name of a node
template that does not include a step to deploy an operating system image.
7. Select the preconfigured compute nodes that you want to add to your cluster. To
select all the preconfigured compute nodes, click Select all.
8. To add the selected compute nodes to your cluster, click Add.
9. On the Completing the Add Node Wizard page, if you will be deploying more
nodes, click Continue responding to all PXE requests. If you will not be deploying
more nodes, click Respond only to PXE requests that come from existing compute
nodes.
10. To monitor deployment progress, select the Go to Node Management to track
progress check box, and then click Finish. For more information, see 4.4. Monitor
deployment progress.
4.4. Monitor deployment progress

During the deployment process of a compute node, its state is set to Provisioning. After the
deployment process is complete, the state changes to Offline. You must bring compute nodes
online before they can process jobs.
You can monitor the progress of the deployment process of compute nodes in Node
Management, and bring online nodes that have finished deploying. You can also see detailed
information for each deployment operation, and any errors that may have occurred.
To monitor deployment progress
2. To view information about the deployment operations:
a. In Node Management, in the Navigation Pane, click Operations.
b. To view more information about a specific operation, click that operation. The
Detail Pane will list the log entries for that operation.
24
3. To view the list of compute nodes that are currently being deployed:
a. In Node Management, in the Navigation Pane, under Nodes, under By State,
click Provisioning.
b. To view the list of operations related to the deployment of a specific node,
double-click that node, and then click the Operations tab.
4. To bring online the nodes that have finished deploying:
a. In Node Management, in the Navigation Pane, under Nodes, under By State,
click Offline.
b. Select all the nodes that you want to bring online. To select all nodes that are
currently offline, on the list of offline nodes, click any node and then press CTRL+A.
c. In the Actions pane, click Bring Online.
5. If the deployment process of a compute node fails, the state of that node is set to
Unknown and the health is set to Provisioning Failed. To determine the reason of the
failure, review the provisioning log for that node and the list of operations that were
performed:
a. In Node Management, in the Navigation Pane, under Nodes, under By Health,
click Provisioning Failed.
b. To review the provisioning log for a node, in the views pane, click the node, and
then in the Detail Pane, click the Provisioning Log tab
c. To view the list of operations related to the deployment failure, on the Properties
tab, click View operations. The pivoted view in Node Management will list all the
operations related to that node.
d. To view more information about a specific operation, click that operation. The
Detail Pane will list the log entries for that operation.
4.5 Cancel the deployment of a node

You can stop the deployment process of a compute node from HPC Cluster Manager by
canceling the provisioning operations.
To cancel the deployment of a node
1. To view only compute nodes that are currently being deployed, in Node
Management, in the Navigation Pane, under Nodes, under By State, click
Provisioning.
2. In the views pane, click the node that you want to stop deploying.
3. To cancel the provisioning operations, in the Detail Pane, on the Properties tab,
click Cancel Operations. The deployment process will stop, the node will be moved to
the Unknown state, and the health for that node will be changed to Provisioning Failed.
25
Step 5: Run Diagnostic Tests on the Cluster
After you have configured your head node and added all compute nodes to the cluster, you
should run diagnostic tests to validate cluster functionality and troubleshoot any configuration
issues.
To run diagnostic tests on the cluster
2. In Configuration, in the Navigation Pane, click To-do List.
3. In the To-do List, click Validate your cluster (under Diagnostics).
4. In the Run Diagnostics dialog box, ensure that the Run all functional tests and All
nodes options are selected, and then click Run.
5. To view the progress of the diagnostic tests and the test results, in Diagnostics, in
the Navigation Pane, click Test Results.
6. To view detailed information about a test, double-click the test. To expand the
information in a section of the test results, click the down arrow for that section.
Step 6: Run a Test Job on the Cluster

After you have finished deploying your cluster, you can run a simple test job to verify that your
cluster is fully functional. The following checklist describes the steps involved in running a simple
test job on your cluster.
Checklist: Run a test job on the HPC cluster

Task Description
6.1. Create a Job Template Create a job template by running the Generate
Job Template Wizard in HPC Cluster Manager.
6.2. Create and Submit a Job Create and submit a basic job in HPC Cluster
Manager.
6.3. Create and Submit a Job Using the Create and submit a basic job by using the HPC
Command-Line Interface (Optional) command-line tools.
6.4. Create and Submit a Job Using HPC Create and submit a basic job by using the
PowerShell (Optional) cmdlets in HPC PowerShell.
26
6.1. Create a job template
Job templates simplify the job management of your HPC cluster by helping you to limit the kinds
of jobs that can be submitted to your cluster, what resources are assigned to jobs, and which
users can submit jobs. HPC Cluster Manager includes the Generate Node Template Wizard to
help you create basic job templates
To create a simple job template
2. In Configuration, in the Navigation Pane, click Job Templates.
3. In the Actions pane, click New. The Generate Node Template Wizard appears.
4. On the Enter Template Name page, type Test Template for the name of the new job
template, and optionally a description. Click Next to continue.
5. On the Limit Run Time page, select the Run jobs no longer than check box, and
then click Next without changing any settings. This will limit all jobs that are submitted
using this template to run for no longer than one minute.
6. On the Set Priorities page, click Next without changing any settings. This will run
jobs that are submitted using this template with Normal priority.
7. On the Set Project Names page, click Next without changing any settings. This will
allow jobs from any project to be submitted using this template.
8. On the Limit Node Groups page, click Next without changing any settings. This will
allow jobs that are submitted using this template to run on any node group.
9. On the Finish page, click Finish.
6.2. Create and submit a job

This section describes how to submit a job in HPC Cluster Manager that:
• Displays a directory list of the files in the C:\Program Files folder of a compute node in
your cluster.
• Uses the job template that you created in the previous section, which limits to 1 minute
the maximum duration of time that a job can run.
• Runs at low priority.
To create and submit a job
1. In Job Management, in the Actions pane, click New Job.

2. In Job Details, specify the following job parameters:
a. In the Job template list, click Test Template (the template that you created in
section “6.1. Create a job template”).
b. When you are prompted if you want to change the job template for the job, click
27
Yes.
c. In the Job name box, type Folder Contents.
d. In the Priority list, click Lowest.
3. To add a task, click Task List, and then specify the following task parameters:
a. To add a new basic task to the job, click Add.
b. In the Task name box, type a name for the new task.
c. In the Command line box, type dir.
d. In the Work directory box, type c:\Program Files.
e. To add this task, click Save.
4. To limit the job so that it only runs on a specific compute node in your HPC cluster,
click Resource Selection, and then specify the following resource parameters:
a. Select the Run this job only on nodes in the following list check box.
b. Select the check box for one of the nodes in your HPC cluster.
5. To submit the job, click Submit.
6. If you are prompted to enter your credentials, type your user name and password,
and then click OK.
7. To view the progress and the results of the job that you submitted:
a. In Job Management, in the Navigation Pane, click All Jobs.
b. In the views pane, click the job that you submitted.
c. When the state of the job is Finished, in the Details Pane, double-click the task
that you created in step 3.
d. In the Task Properties window, in the Results tab, the Output box will display
the directory list of c:\Program Files for the compute node that you selected in step
4.
e. If you want to copy the results to the clipboard, click Copy output to clipboard.
6.3. Create and submit a job using the command-

line interface (optional)
You can create and submit a job similar to the job that you created and submitted in the previous
section, using the command-line interface tools that are included with Windows HPC
Server 2008.
To create and submit a job using the command-line interface
1. Open a Command Prompt window. Click Start, point to All Programs, click
Accessories, and then click Command Prompt.
2. To create a new job, type the following command:
job new /jobname:"Folder Contents" /priority:"Lowest" /RunTime:0:0:1
28
/requestednodes:”<ComputeNodeName>”
Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

3. To add a task to the job, type the following command:
job add <JobID> /workdir:"C:\Program Files" dir
Where <JobID> is the identification number for the job, as displayed on the command-
line interface after typing the command in step 2.
4. To submit the job, type the following command:
job submit /id:<JobID>
Where <JobID> is the identification number for the job, as displayed on the command-
line interface after typing the command in step 2.
5. If you are prompted to enter your credentials, type your password, and then type
ENTER.
6.4. Create and submit a job using HPC

PowerShell (optional)
You can also create and submit the same job that you created and submitted in the previous
section, using HPC PowerShell.
Note
For more information about HPC PowerShell, see Appendix 6: Using HPC PowerShell.
To create and submit a job using HPC PowerShell
1. On the head node, click Start, point to All Programs, and then click Microsoft HPC
Pack.
2. Right-click HPC PowerShell, and then click Run as administrator.
3. If you are prompted by Windows PowerShell if you want to run the
ccppsh.format.ps1xml script, type A, and then press ENTER.
4. To create a new job, type the following cmdlet:

$j = New-HpcJob -Name "Folder Contents" -Priority Lowest -RunTime "0:0:1"
-RequestedNodes “<ComputeNodeName>”
Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

5. To add a task to the job, type the following cmdlet:
$j | Add-HpcTask -WorkDir "C:\Program Files" -CommandLine "dir"
6. To submit the job, type the following cmdlet:

$j | Submit-HpcJob
7. If you are prompted to enter your credentials, type your password, and then type
ENTER.
29
Notes
You can also type all three cmdlets in one line:
New-HpcJob -Name "Folder Contents" -Priority Lowest -RunTime "0:0:1"
-RequestedNodes “<ComputeNodeName>” | Add-HpcTask -WorkDir "C:\Program
Files" -CommandLine "dir" | Submit-HpcJob
Where <ComputeNodeName> is the name of a compute node in your HPC

cluster.
Related documents
• For more information about creating and submitting jobs, see the Submitting Jobs in
Windows HPC Server 2008 Step-by-Step Guide (http://go.microsoft.com/fwlink/?
LinkId=121887).
• For more information about configuring job submission and scheduling policies, see the
Configuring Job Submission and Scheduling Policies in Windows HPC Server 2008 Step-by-
Step Guide (http://go.microsoft.com/fwlink/?LinkId=121888).
Additional Resources
• Additional online resources, including step-by-step guides, technical reference
documentation and troubleshooting guides are available on the Windows HPC Server 2008
Technical Library (http://go.microsoft.com/fwlink/?LinkId=119594).
• HPC Cluster Manager includes comprehensive Help documentation. This documentation
is available from the user interface by clicking any of the in-context help links or by pressing
F1.
Appendices
In this section:
• Appendix 1: HPC Cluster Networking
• Appendix 2: Creating a Node XML File
• Appendix 3: Node Template Tasks and Properties
• Appendix 4: Job Template Properties
• Appendix 5: Scriptable Power Control Tools
• Appendix 6: Using HPC PowerShell
30
Appendix 1: HPC Cluster Networking
Windows HPC Server 2008 supports five cluster topologies designed to meet a wide range of
user needs and performance, scalability, manageability, and access requirements. These
topologies are distinguished by how the compute nodes in the cluster are connected to each
other and to the enterprise network. Depending on the network topology that you choose for your
cluster, certain network services, such as Dynamic Host Configuration Protocol (DHCP) and
network address translation (NAT), can be provided by the head node to the compute nodes.
You must choose the network topology that you will use for your cluster well in advance of setting
up an HPC cluster.
This section includes the following topics:
• HPC cluster networks
• Supported HPC cluster network topologies
• HPC network services
• Windows Firewall configuration
HPC cluster networks

The following table lists and describes the networks to which an HPC cluster can be connected.
Network Name Description
Enterprise network An organizational network to which the head

node is connected and optionally the compute
nodes. The enterprise network is often the
network that most users in an organization log
on to when performing their job. All intra-cluster
management and deployment traffic is carried
on the enterprise network unless a private
network (and optionally, an application network)
also connects the cluster nodes.
Private network A dedicated network that carries intra-cluster

communication between nodes. This network
carries management, deployment, and
application traffic if no application network
exists.
Application network A dedicated network, preferably with high

bandwidth and low latency. These
characteristics are important so that this
network can perform latency-sensitive tasks,
such as carrying parallel Message Passing
Interface (MPI) application communication
between compute nodes.
31
Supported HPC cluster network topologies
There are five cluster topologies supported by Windows HPC Server 2008:
• Topology 1: Compute Nodes Isolated on a Private Network
• Topology 2: All Nodes on Enterprise and Private Networks
• Topology 3: Compute Nodes Isolated on Private and Application Networks
• Topology 4: All Nodes on Enterprise, Private, and Application Networks
• Topology 5: All Nodes on an Enterprise Network
Topology 1: Compute nodes isolated on a private network

The following image illustrates how the head node and the compute nodes are connected to the
cluster networks in this topology:
The following table lists and describes details about the different components in this topology:
Component Description
Network adapters • The head node has two network

adapters.
• Each compute node has one network
adapter.
• The head node is connected to both an
enterprise network and to a private network.
• The compute nodes are connected only
to the private network.
Traffic • The private network carries all

communication between the head node and
the compute nodes, including deployment,
management and application traffic (for
example, MPI communication).
Network services • The default configuration for this

32
topology is NAT enabled on the private

network in order to provide the compute
nodes with address translation and access
to services and resources on the enterprise
network.
• DHCP is enabled by default on the
private network to assign IP addresses to
compute nodes.
• If a DHCP server is already installed on
the private network, then both NAT and
DHCP will be disabled by default.
Security • The default configuration on the cluster

has the firewall turned ON for the enterprise
network and turned OFF for the private
network.
Considerations when selecting this topology • Cluster performance is more consistent

because intra-cluster communication is
routed onto the private network.
• Network traffic between compute nodes
and resources on the enterprise network
(such as databases and file servers) pass
through the head node. For this reason, and
depending on the amount of traffic, this
might impact cluster performance.
• Compute nodes are not directly
accessible by users on the enterprise
network. This has implications when
developing and debugging parallel
applications for use on the cluster.
Topology 2: All nodes on enterprise and private networks

33
Network adapters • The head node has two network

adapters.
• Each compute node has two network
adapters.
• All nodes in cluster are connected to
both the enterprise network and to a
dedicated private cluster network.
Traffic • Communication between nodes,

including deployment, management, and
application traffic, is carried on the private
network in this topology.
• Traffic from the enterprise network can
be routed directly to a compute node.

topology has DHCP enabled on the private
network, to provide IP addresses to the
compute nodes.
• NAT is not required in this topology
because the compute nodes are connected
to the enterprise network, so this option is
disabled by default.

network and turned OFF for the private
network.
Considerations when selecting this topology • This topology offers more consistent
cluster performance because intra-cluster
34
communication is routed onto a private

network.
• This topology is well suited for
developing and debugging applications
because all compute nodes are connected
to the enterprise network.
• This topology provides easy access to
compute nodes by users on the enterprise
network.
• This topology provides faster access to
enterprise network resources by the
compute nodes.
Topology 3: Compute nodes isolated on private and application networks

Network adapters • The head node has three network

adapters: one for the enterprise network,
one for the private network, and a high-
speed adapter that is connected to the
application network.
• Each compute node has two network
adapters, one for the private network and
another for the application network.
35
Traffic • The private network carries deployment

and management communication between
the head node and the compute nodes.
• Jobs running on the cluster use the
high-performance application network for
cross-node communication.

topology has both DHCP and NAT enabled
for the private network, to provide IP
addressing and address translation for
compute nodes. DHCP is enabled by
default on the application network, but not
NAT.
• If a DHCP is already installed on the
private network, then both NAT and DHCP
will be disabled by default.

network and turned OFF on the private and
application networks.
communication is routed onto the private
and application networks.
• Compute nodes are not directly
accessible by users on the enterprise
network in this topology.
Topology 4: All nodes on enterprise, private, and application networks

36
Network adapters • The head node has three network

adapters.
• All compute nodes have three network
adapters.
• The network adapters are for the
enterprise network, the private network, and
a high speed adapter for the high
performance application network.
Traffic • The private cluster network carries only

deployment and management traffic.
• The application network carries latency-
sensitive traffic, such as MPI
communication between nodes.
• Network traffic from the enterprise
network reaches the compute nodes
directly.

topology has DHCP enabled for the private
and application networks to provide IP
addresses to the compute nodes on both
networks.
• NAT is disabled for the private and
application networks because the compute
nodes are connected to the enterprise
network.
37

network and turned OFF on the private and
application networks.
communication is routed onto a private and
application network.
• This topology is well suited for
developing and debugging applications
because all cluster nodes are connected to
the enterprise network.
network.
compute nodes.
Topology 5: All nodes on an enterprise network

Network adapters • The head node has one network

adapter.
• All compute nodes have one network
adapter.
• All nodes are on the enterprise network.
Traffic • All traffic, including intra-cluster,
38
application, and enterprise traffic, is carried

over the enterprise network. This
maximizes access to the compute nodes by
users and developers on the enterprise
network.
Network services • This topology does not require NAT or

DHCP because the compute nodes are
connected to the enterprise network.

network.
Considerations when selecting this topology • This topology offers easy access to
network.
• Access of resources on the enterprise
network by individual compute nodes is
faster.
• This topology, like topologies 2 and 4, is
well suited for developing and debugging
applications because all cluster nodes are
connected to the enterprise network.
network.
compute nodes.
• Because all nodes are connected only
to the enterprise network, you cannot use
Windows Deployment Services to deploy
compute node images using the new
deployment tools in Windows HPC
Server 2008.
HPC network services

Depending on the network topology that you have chosen for your HPC cluster, the following
network services can be provided by the head node to the compute nodes connected to the
different cluster networks:
39
• Network Address Translation (NAT)
• Dynamic Host Configuration Protocol (DHCP) server
This section describes these HPC network services.
Network address translation (NAT)

Network address translation (NAT) provides a method for translating Internet Protocol version 4
(IPv4) addresses of computers on one network into IPv4 addresses of computers on a different
network.
Enabling NAT on the head node enables compute nodes on the private or application networks to
access resources on the enterprise network. You do not need to enable NAT if you have another
server providing NAT or routing services on the private or application networks. Also, you do not
need NAT if all nodes are connected to the enterprise network.
DHCP server
A DHCP server assigns IP addresses to network clients. Depending on the detected configuration
of your HPC cluster and the network topology that you choose for your cluster, the compute
nodes will receive IP addresses from either the head node running DHCP, or from a dedicated
DHCP server on the private network, or via DHCP services coming from a server on the
enterprise network.
Windows Firewall configuration

Windows HPC Server 2008 opens firewall ports on the head node and compute nodes to enable
internal services to run. By default, Windows Firewall is enabled only on the enterprise network,
and disabled on the private and application networks to provide the best performance and
manageability experience.
Important
If you have applications that require access to the head node or to the cluster nodes on
specific ports, you will have to manually open those ports in Windows Firewall.
Firewall ports required by Windows HPC Server 2008

The following table lists all the ports that are opened by Windows HPC Server 2008 for
communication between cluster services on the head node and the compute nodes.
Port Number (TCP) Required By
5969 Required by the client tools on the enterprise

network to connect to the HPC Job Scheduler
Service on the head node.
9892, 9893 Used by the HPC Management Service on the

compute nodes to communicate with the HPC
40
System Definition Model (SDM) Service on the

head node.
5970 Used for communication between the HPC

Management Service on the compute nodes
and the HPC Job Scheduler Service on the
head node.
9794 Used for communication between

ExecutionClient.exe on the compute nodes and
the HPC Management Service on the head
node. ExecutionClient.exe is used during the
deployment process of a compute node. It
performs tasks such as imaging the computer,
installing all the necessary HPC components,
and joining the computer to the domain.
9087, 9088, 9089 Used for communication between the client

application on the enterprise network and the
services provided by the Windows
Communication Foundation (WCF) broker node.
1856 Used by the HPC Job Scheduler Service on the

head node to communicate with the HPC Node
Manager Service on the compute nodes.
8677 Used for communication between the HPC MPI

Service on the head node and the HPC MPI
Service on the compute nodes.
6729 Used for management services traffic coming

from the compute nodes to the head node or
WCF broker node.
5800 Used for communication between the HPC

command-line tools on the enterprise network
and the HPC Job Scheduler Service on the
head node.
5801 Used by the remote node service on the

enterprise network to enumerate nodes in a
node group, or to bring a node online or take it
offline.
5999 Used by HPC Cluster Manager on the

enterprise network to communicate with the
HPC Job Scheduler Service on the head node.
41
443 Used by the clients on the enterprise network to

connect to the HPC Basic Profile Web Service
on the head node.
Appendix 2: Creating a Node XML File

A node XML file contains a list of compute nodes that you want to add to your cluster. This list
includes:
• When adding compute nodes from bare metal, a hardware identification parameter for
each compute node, such as the System Management BIOS (SMBIOS) GUID or the Media
Access Control address (MAC) address.
• When adding preconfigured nodes that are already running one of the 64-bit editions of
the Windows Server 2008 operating system, and HPC Pack 2008 has been installed, a
computer name for identification purposes.
• Other properties, such as the physical location of each compute node and the Windows
product key that should be used to activate the operating system.
Benefits of using a node XML file for deployment
The following list outlines some of the benefits of using a node XML file when adding compute
nodes to your cluster:
• You can pre-stage a PXE deployment of compute nodes for your HPC cluster by
importing a node XML file with a list of all the computers that you will be adding to the cluster.
The compute nodes can be deployed both from bare metal or as preconfigured nodes.
• Preconfigured nodes that are added to your HPC cluster using a node XML file do not
need to be manually approved into the cluster. This makes the deployment process more
efficient and streamlined.
• Importing a node XML file is a simple and efficient way for you to associate properties
with compute nodes. Examples of properties that can be associated with compute nodes are:
location (including data center, rack, and chassis), a Windows product key, node templates,
or tags that are used to automatically create node groups.
• You can give specific computer names (NetBIOS names) to compute nodes that are
deployed from bare metal, without having to worry about powering them on in a specific
order. Using a node XML file, computer names will already be associated with a specific
SMBIOS GUID or MAC address (or both).
The node XML file schema

The node XML file is based on an XML Schema Definition (XSD) language file:
NodeConfigurationFile.xsd. This XSD file is available on the head node, in the Bin folder of the
installation path for HPC Pack 2008. For example, if you are using the default installation path,
the XSD file is available here:
42
C:\Program Files\Microsoft HPC Pack\Bin\NodeConfigurationFile.xsd
The following table lists and describes the attributes and elements that are defined in the node
XML file schema:
Attribute, Element, or Required Description

Element:Attribute
Location No • Optional element.

• Contains attributes with
information about the location
of the compute node.
Location:DataCenter No • Optional attribute of the

Location element.
• Specifies the name of the
data center where the
compute node is located.
Location:Rack No • Optional attribute of the

Location element.
• Specifies the name or
number of the server rack
where the compute node is
located.
Location:Chassis No • Optional attribute of the

Location element.
• Specifies the name or
number of the chassis that is
used for the compute node.
Template No • Optional element.

• This element is required
when deploying compute
nodes from bare metal.
• Contains attributes with
information about the node
template that will be used to
deploy the compute node.
Template:Name Yes • Required attribute of the

Template element.
• This attribute is required
only when a Template
element is included.
43
Element:Attribute

node template that will be
used to deploy the compute
node.
• If the specified node
template name does not exist
on the head node, the
deployment will fail.
• If you are deploying
compute nodes from bare
metal, this attribute must
specify the name of a node
template that includes a step
to deploy an operating system
image, or your deployment
will fail.
Template:Provisioned No • Optional attribute of the

Template element.
• Specifies if the node is a
preconfigured node
(Provisioned=“true”, or
Provisioned=“1”), or not
(Provisioned=“false”, or
Provisioned=“0”).
• If this attribute is not
specified, the node is not
considered a preconfigured
node, and the node template
will be applied to the node
when the node is added to
the cluster.
• If Provisioned=“true”, or
Provisioned=“1” is
specified, the node template
will not be applied to the node
the cluster, but the node will
be imported with that node
template associated with it.
• If Provisioned=“false”,
44
Element:Attribute
or Provisioned=“0” is
specified, the node template
will be applied to the node
the cluster.
metal, this attribute must be
Provisioned=“false”,
Provisioned=“0”, or must
not be specified. Also, the
node template must include a
step to deploy an operating
system image.
MacAddress No • Optional element.

• Specifies the MAC
address of the network
adapter that will be used by
the compute node.
metal, you must specify this
element or the MachineGuid
parameter, or the deployment
will fail. You must also specify
this element if the cluster
nodes in your system have
SMBIOS GUIDs that are not
unique (that is, two or more
nodes in the node XML file
have the same value for the
MachineGuid parameter).
• There can be multiple
instances of this element, if
the compute node uses more
than one adapter.
• Ensure that you specify
only those MAC addresses
that exist in the compute
node. Specifying a MAC
45
Element:Attribute
address that does not exist in

a compute node, might cause
the import of that node to fail.
Important
When you specify a MAC
address in the node XML
file, do not include any
blank spaces, colons (“:”),
hyphens (“-”), or dots (“.”).
Include only the twelve
hexadecimal digits for the
MAC address. For
example, the following
MAC address is correctly
specified: 00301B445F02.
Tag No • Optional element.

node group to which the
compute node should be
added during deployment.
• There can be multiple
instances of this element, if
the compute node should be
added to more than one node
group.
Name Yes • Required attribute.

• Specifies the computer
name (NetBIOS name) of the
compute node.
metal, this attribute specifies
the computer name that will
be assigned to the node
during deployment.
preconfigured nodes, this
attribute specifies the current
computer name of the
46
Element:Attribute
compute node.
• If the specified name is
that of a preconfigured node
that has already been added
to the cluster (that is, it is not
in the Unknown state), the
node XML file will fail to
import.
Domain No • Optional attribute.

• Specifies the Active
Directory® domain to which
the compute node should be
added.
• If this attribute is not
specified, the Active Directory
domain of the head node is
used.
ManagementIpAddress No • Optional attribute.

• Specifies information that
is required for the integration
of scriptable power control
tools like Intelligent Platform
Management Interface (IPMI)
scripts.
• You only need to specify
this attribute if you are using
scriptable power control tools
to manage power on your
cluster.
MachineGuid No • Optional attribute.

• Specifies the SMBIOS
GUID of the computer where
the compute node is
deployed.
metal, you must specify this
parameter or the
47
Element:Attribute
MacAddress element, or the

node XML file will fail to
import.
ProductKey No • Optional attribute.

• Specifies the Windows
product key that will be used
to activate the operating
system on the compute node.
• The product key is used
during the activation task of a
node template that includes a
step to deploy an operating
system image.
• The product key that you
specify must match the
edition of the operating
system in the image that is
used by the node template.
Important
You must specify a
Windows product key if
you are using an
operating system image
of a retail version of
Windows Server 2008, or
the evaluation version of
Windows Server 2008
HPC Edition.
Creating a node XML file for deployment from bare metal

The node XML file can be created in any XML editor or text editor, but it must follow the node
XML file schema. Also, a node XML file can be created from an HPC cluster that is already
configured, by exporting it from HPC Cluster Manager.
48
Note
For detailed information about creating a node XML file, see the Creating a Node XML
File in Windows HPC Server 2008 Step-by-Step Guide (http://go.microsoft.com/fwlink/?
LinkId=139371).
When creating a node XML file for a deployment from bare metal, you will need a hardware
identification parameter for each compute node. This parameter can be the SMBIOS GUID or the
MAC address of the computer.
When creating a node XML file for deployment from bare metal:
• Specify the MAC address of a compute node in the MacAddress attribute for that
compute node.
• Specify the SMBIOS GUID of a compute node in the MachineGuid attribute for that
compute node.
• If both the SMBIOS GUID and MAC address of a compute node are specified, the
SMBIOS GUID is used.
• If for some reason you do not have access to the SMBIOS GUID of a node, you can use
only the MAC address.
• Ensure that you specify only those MAC addresses that exist in each compute node.
Specifying a MAC address that does not exist in a compute node, might cause the import of
that node to fail.
• You must specify a node template for each compute node listed, and that node template
must include a step to deploy an operating system image. If yosu do not specify a node
template or if you specify a node template that does not include a step to deploy an operating
system image, the deployment will fail.
• Ensure that the node template names that are specified in the node XML file match the
names of the node templates listed on the head node.
• Specify any location information that you want to be attached to the node.
• If you want nodes to be automatically added to specific node groups during deployment,
specify the Tag attribute with the name of the node group for each compute node.
• You must include a Windows product key if you are using an operating system image of a
retail version of Windows Server 2008, or the evaluation version of Windows Server 2008
HPC Edition
• If your integration of scriptable power control tools requires a BMC IP address for each
compute node, it can be added to the node XML file.
Sample node XML file

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<Nodes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns="http://schemas.microsoft.com/HpcNodeConfigurationFile/2007/12">
49
<Node
Name="ComputeNodeName1"
Domain="CONTOSO"
MachineGuid="{4c4c4544-0038-5710-804b-c6c04f464331}">
<Location
DataCenter="Data Center 1"
Rack="2"
Chassis="1" />
<Template
Name="Default ComputeNode Template" Provisioned="true" />
<MacAddress>00301B445F02</MacAddress>
<MacAddress>001B2104EDF5</MacAddress>
<Tag>ComputeNodes</Tag>
<Tag>Rack2</Tag>
</Node>
</Nodes>
Appendix 3: Node Template Tasks and Properties

You can use the Node Template Editor in HPC Cluster Manager to add deployment,
configuration, and maintenance tasks to a node template. Some of these tasks are added to new
templates when you create them using the Create Node Template Wizard.
To add tasks to a node template using the Node Template Editor
2. In Configuration, click Node Templates.
3. In the Views pane, click the node template to which you want to add tasks.
4. In the Actions pane, click Edit. The Node Template Editor appears.
5. To add a task, click Add Task, and then click the task that you want to add from the
list of available tasks. For more information about each node task and its parameters, see
Available node template tasks.
6. Repeat the previous step for all tasks that you want to add to the node template. After
you are done adding tasks, click Save.
50
Available node template tasks
There are four types of node template tasks:
• Provisioning
• Configuration
• Deployment
• Maintenance
Provisioning
Provisioning tasks are performed on the head node before the deployment process of the
compute nodes takes place. The following table lists the provisioning task that you can add or
modify on a node template, and the properties that are associated with it.
Task Name Task Description Properties
Create Computer Account Creates a computer account • Domain (optional):

in Active Directory for the specifies the name of the
compute node. domain on which the
computer account will be
created. If this property is not
specified, the domain of the
head node is used.
• ComputerPath
(optional): specifies the path
in Active Directory where the
computer account will be
created. The default path is
cn=Computers. If this
property is not specified, the
computer path of the head
node is used.
Important
When you specify one of
these two properties, you
must also specify the
other one. If you only
specify one property, it
will be ignored and the
domain and computer
path of the head node will
be used.
51
Configuration
Configuration tasks are performed on a compute node after the compute node is booted into
Windows Preinstallation Environment (Windows PE) at the beginning of the deployment process.
The following table lists the configuration tasks that you can add or modify on a node template
and the properties that are associated with each.
Run Windows PE Command Runs a command in • ContinueOnFailure

Windows PE. (optional): if True is selected,
the configuration task will not
fail if the command fails to run
successfully. If False is
selected, the configuration task
will fail if the command fails to
run successfully.
• ErrorWhiteList (optional):
specifies the return error codes
that should be ignored for the
command. The default return
code that is expected from the
command when it runs
successfully is zero (0). If the
command returns a success
code other than zero, then you
must add that return code to
the list of error codes that
should be ignored, or it will be
interpreted as an error code
and the task will fail.
• Command (required):
specifies the Windows PE
command that you want to run.
Multicast Copy Copies a file from the head • UnicastFallback (optional):

node using the multicast if True is selected and multicast
protocol. fails, the file will be copied
using the Server Message
Block (SMB) protocol. If False
is selected and multicast fails,
the task will fail.
• DestFile (required):
specifies the absolute path to
the drive of the compute node
52
to which the file will be copied.

If you have added the Partition
Disk task to the node template,
ensure that you specify a path
that is valid for the partitions
that will be created with that
task.
• SourceFile (required):
specifies the name and path of
the file to copy, relative to the
Microsoft HPC
Pack\Data\InstallShare folder.
Unicast Copy Copies a file from the head • Directory (optional):

node using the Server specifies if a file (False) or a
Message Block (SMB) folder (True) is being copied.
protocol. • Destination (required):
specifies the absolute path to
to which the file will be copied.
If you have added the Partition
Disk task to the node template,
ensure that you specify a path
that is valid for the partitions
that will be created with that
task.
• Source (required):
Microsoft HPC
Partition Disk Partitions the disk on the • DiskPartScript (Required):

compute node using a script specifies the name and path of
for Diskpart. the script to use with Diskpart.
Mount Share Shares a folder during the • DriveLetter (optional):

Windows PE phase of the specifies the drive letter where
operating system installation. the folder will be shared.
• User (optional): specifies
the user name to use when
sharing the folder.
• UserPassword (optional):
53
specifies the password to use

when sharing the folder.
• Path (required): specifies
the name and path of the folder
that will be shared.
Install Windows Installs the Windows Server • Custom Unattend File

operating system on the (optional): specifies the
compute node. absolute path to the custom
unattend file to use for
installation.
• Installation Drive
(optional): specifies the drive
letter where the Windows
Server operating system will be
installed. If you have added the
Partition Disk task to the node
template, ensure that you
specify a drive letter that is
valid for the partitions that will
be created with that task.
• Local Administrator
password (optional): specifies
the password for the local
Administrator account on the
compute node.
• Product Key (optional):
specifies the product key to use
with this node template for the
activation of the operating
system.
• Autogenerate Local
Admin Password (required): if
True is selected, the password
for the local Administrator
account is automatically
generated. After a password is
automatically generated, it is
secret and cannot be
recovered. If False is selected,
then you should specify a
password using the Local
54
Administrator password
attribute.
• Image (required): specifies
the image to use for the
installation of the operating
system.
Apply WIM Image Extracts the files in a WIM • DestinationPath

file to a local disk on the (required): specifies the path on
compute node. the compute node where the
files in the Windows Imaging
Format (WIM) file will be
extracted. If you have added
the Partition Disk task to the
node template, ensure that you
specify a path that is valid for
the partitions that will be
created with that task.
• WimPath (required):
specifies the path on the
compute node where the WIM
file that will be extracted is
stored.
Restart Restarts the compute node. • None
Deployment
Deployment tasks are performed on a compute node after the operating system has been
installed. The following table lists the deployment tasks that you can add or modify on a node
template and the properties that are associated with each.
Unicast Copy Copies a file from the head • Directory (optional):

node using the Server specifies if a file (False) or a
Message Block (SMB) folder (True) is being copied.
protocol. • Destination (required):
specifies the absolute path on
where the file will be copied.
• Source (required):
55

Microsoft HPC
Run OS command Runs a command as the local • ContinueOnFailure

Administrator. (optional): if True is selected,
deployment will not fail if the
command fails to run
selected, deployment will fail if
the command fails to run
successfully.
• ErrorWhiteList (optional):
specifies the return error codes
that should be ignored for the
command. The default return
code that is expected from the
command when it runs
successfully is zero (0). If the
command returns a success
code other than zero, then you
must add that return code to
the list of error codes that
should be ignored, or it will be
interpreted as an error code
and the task will fail.
specifies the command that
you want to run as
Administrator.
Install HPC Pack Installs HPC Pack on the • Setup Source Directory
compute node. (required): specifies the
location of the installation files
for the HPC Pack.
Mount Share Shares a folder on the • DriveLetter (optional):

compute node. specifies the drive letter where
the folder will be shared.
• User (optional): specifies
the user name to use when
sharing the folder.
56
• UserPassword (optional):
specifies the password to use
when sharing the folder.
• Path (required): specifies
the name and path of the folder
that will be shared.
Join Domain Joins the compute node to an • Domain (optional):

Active Directory domain. specifies the name of the
domain to which the compute
node will be joined. If this
property is not specified, the
domain of the head node is
used.
Restart Restarts the compute node. • None
Log Off Logs off the compute node. • None
Maintenance
Maintenance tasks are performed on a compute node when you select a node in Node
Management and then click Maintain. The following table lists the maintenance tasks that you
can add or modify on a node template and the properties that are associated with each.
Post Install Command Runs a command on the • ContinueOnFailure

compute node after HPC (optional): if True is selected,
Pack has been installed. the maintenance task will not
fail if the command fails to run
selected, the maintenance task
will fail if the command fails to
run successfully.
• Timeout (optional):
specifies the number of
seconds before the command
times out. If this property is not
specified, the default timeout
value is 60 seconds.
• WorkingDirectory
(optional): specifies the folder
57
where the command runs.

specifies the command to run.
This command runs using the
installation credentials that
were provided during the
configuration process of the
head node.
Activate Operating System Activates the operating • None

system on the compute
node.
Apply Updates Applies updates to the • Patches (optional):

compute node from Microsoft specifies the list of updates that
Update or Windows Server will be applied to the compute
Update Services (WSUS). node.
• Categories (required):
specifies the type of updates
that will be applied to the
compute node.
Appendix 4: Job Template Properties

You can use the Job Template Editor in HPC Cluster Manager to add properties to a job
template. Some of these properties are added to new templates when you create them using the
Generate Job Template Wizard.
To add properties to a job template using the Job Template Editor
2. In Configuration, click Job Templates.
3. In the Views pane, click the job template to which you want to add properties.
4. In the Actions pane, click Edit. The Job Template Editor appears.
5. To add a property, click Add, and then click the property that you want to add from
the list of available properties. For more information about each job template property,
see Available job template properties.
6. Repeat the previous step for all properties that you want to add to the job template.
After you are done adding properties, click Save.
58
Available job template properties
The following table lists all the job properties that you can set in a job template. The properties
that you set in a job template constrain the properties that a cluster user can choose when
submitting a job to the cluster using that template.
Job Property Description
Auto Calculate Maximum If True is selected as the only valid value, the cluster
user cannot specify the maximum number of resources
(cores, sockets, or nodes) assigned to the job, and
resources will be automatically calculated based on the
tasks in the job. If False is selected as the only valid
value, the cluster user must specify the maximum
number of resources assigned to the job. If True and
False are selected as valid values, the cluster user can
choose to specify the maximum number of resources, or
to have them automatically calculated.
Auto Calculate Minimum If True is selected as the only valid value, the cluster
user cannot specify the minimum number of resources
(cores, sockets, or nodes) assigned to the job, and
resources will be automatically calculated based on the
tasks in the job. If False is selected as the only valid
value, the cluster user must specify the minimum
number of resources assigned to the job. If True and
choose to specify the minimum number of resources, or
to have them automatically calculated.
Exclusive If True is selected as the only valid value, no other jobs

can run on a compute node at the same time as the job
being submitted by the cluster user. If False is selected
as the only valid value, the cluster user cannot select
this property when submitting the job. If True and False
are selected as valid values, the cluster user can select
or unselect this property when submitting the job.
Fail on Task Failure If True is selected as the only valid value, the failure of
any task in the job that is submitted by the cluster user
will cause the entire job to fail immediately. If False is
selected as the only valid value, the cluster user cannot
select this property when submitting the job. If True and
select or unselect this property when submitting the job.
Job Name Specifies a list of names that the cluster user can select
59
for the job. If you set this property, the cluster user
cannot specify a job name that is not on the list.
Licenses Specifies a list of licenses that the cluster user can

select for the job. If you set this property, the cluster user
cannot specify a license that is not on the list. Licenses
in this list can be validated by a job activation filter that is
defined by the cluster administrator.
Maximum Cores Specifies a range of values for the maximum number of

cores that the cluster user can assign to the job. This
property has no effect if any of these conditions is true:
• The Auto Calculate Maximum property has
been added to the template with True selected as
the only valid value
• The Unit Type property has been added to the
template and the list of the type of resources that
can be assigned to the job does not include Core
• The user selects a different type of resource to
assign to the job (that is, sockets or nodes)
Maximum Nodes Specifies a range of values for the maximum number of

cluster nodes that the cluster user can assign to the job.
This property has no effect if any of these conditions is
true:
can be assigned to the job does not include Node
assign to the job (that is, sockets or cores)
Maximum Sockets Specifies a range of values for the maximum number of

sockets that the cluster user can assign to the job. This
property has no effect if:
can be assigned to the job does not include Socket
60

assign to the job (that is, nodes or cores)
Minimum Cores Specifies a range of values for the minimum number of

cores that the cluster user can assign to the job. This
• The Auto Calculate Minimum property has
can be assigned to the job does not include Core
assign to the job (that is, sockets or nodes)
Minimum Nodes Specifies a range of values for the minimum number of

cluster nodes that the cluster user can assign to the job.
This property has no effect if:
can be assigned to the job does not include Node
assign to the job (that is, sockets or cores)
Minimum Sockets Specifies a range of values for the minimum number of

sockets that the cluster user can assign to the job. This
can be assigned to the job does not include Sockets
assign to the job (that is, nodes or cores)
Node Groups Specifies a list of node groups that the cluster user is
required to select for the job. If you set this property, the
cluster user can still specify a node group that is not on
the list, but cannot remove any of the node groups that
61
are listed as required.
Node Ordering Specifies the ordering to use when assigning nodes to

run the job. This property gives preference to nodes with
specific attributes. The node ordering options are:
• Memory Size (Ascending). The job will be
assigned first to nodes that have the smallest
amount of memory.
• Memory Size (Descending). The job will be
assigned first to nodes that have the largest amount
of memory.
• Processor Number (Ascending). The job will
be assigned first to nodes that have the smallest
number of cores.
• Processor Number (Descending). The job will
be assigned first to nodes that have the largest
number of cores.
Preemptable If True is selected as the default value (Default Value

parameter for this property), the job being submitted by
the cluster user can be preempted by another job that
has a higher priority, if there are not enough resources to
run the higher priority job. If False is selected as the
default value, the job being submitted by the cluster user
cannot be preempted by another job.
Note
This property has no effect if the preemption
policy for the HPC Job Scheduler Service is set
to No pre-emption. To review or configure the
preemption policy for the HPC Job Scheduler
Service, in HPC Cluster Manager, from the
Options menu, click Job Scheduler
Configuration.
Note
If True and False are selected as valid values
for this property (Valid Value parameter), a
cluster user that is using the job template can
submit a job that cannot be preempted by using
the HPC API, independently of the value that
was selected as the default value. It is not
possible to specify that a job cannot be
62
preempted by submitting it using HPC Cluster

Manager, HPC Job Manager, the HPC
PowerShell, or the HPC command-line tools. It
is only possible to do this by using the HPC API.
For more information about the HPC API, see
the Microsoft HPC Pack
(http://go.microsoft.com/fwlink/?LinkID=123849).
Note
If you want to allow only certain cluster users to
submit jobs that cannot be preempted, create a
job template that includes this property with
False selected as the default and the only valid
value. Then, select which users can submit jobs
with the job template by setting permissions for
the template. Permissions for a template can be
set in HPC Cluster Manager, in Configuration,
under Job Templates (Set Permissions
action).
Priority Specifies a list of priority values that the user can select
for the job. If you set this property, the cluster user
cannot specify a priority value that is not on the list.
Project Specifies a list of project names that the cluster user can
select for the job. If you set this property, the cluster user
cannot specify a project name that is not on the list.
Requested Nodes Specifies a list of nodes that the cluster user can select
to run the job. If you set this property, the cluster user
cannot select a node that is not on the list.
Run Time Specifies a range of values for the amount of time that
the cluster user can specify the job is allowed to run. If a
task in the job is still running after the specified run time
is reached, the task is stopped and the job is
automatically canceled by the HPC Job Scheduler
Service.
Run Until Canceled If True is selected as the only valid value, the job runs
until it is canceled or until its run time expires. If False is
selected as the only valid value, the cluster user cannot
select this property when submitting the job. If True and
select or unselect this property when submitting the job.
63
Service Name Specifies a list of services names that the cluster user
can select for a Service-Oriented Application (SOA) job.
If you set this property, the cluster user cannot specify a
service name that is not on the list.
Unit Type Specifies a list of the type of resources (cores, sockets,

or nodes) that can be assigned to the job. If you set this
property, the cluster user cannot specify a resource type
that is not on the list.
Appendix 5: Scriptable Power Control Tools

The cluster administration console (HPC Cluster Manager) includes actions to start, shut down,
and restart compute nodes remotely: Start, Reboot, and Shut Down in the Actions pane in
Node Management. These actions are linked to the CcpPower.cmd script, which performs these
power control operations, with the exception of the start action that is not enabled, using
operating system commands.
You can replace the default operating system commands in CcpPower.cmd with custom power
control scripts, like Intelligent Platform Management Interface (IPMI) scripts.
CcpPower.cmd is available in the Bin folder of the installation path for HPC Pack 2008. For
example, if you are using the default installation path, the file is available here:
C:\Program Files\Microsoft HPC Pack\Bin\CcpPower.cmd
The default CcpPower.cmd file has the following code:

@setlocal
@echo off
if L%1 == Lon goto on
if L%1 == Loff goto off
if L%1 == Lcycle goto cycle
echo "usage:CcpPower.cmd [on|off|cycle] nodename [ipaddress]"
goto done
:on
exit /b 1
goto done
:off
shutdown /s /t 0 /f /m \\%2
64
goto done
:cycle
shutdown /r /t 0 /f /m \\%2
goto done
:done
exit /b %ERRORLEVEL%
endlocal
To enable scriptable power control tools for the Shut Down and Reboot actions in HPC Cluster
Manager, replace the entries of the shutdown command in CcpPower.cmd with the name and path
of your tool or tools for shutting down and restarting the node. To enable tools for the Start action,
replace the exit entry in the :on section with the name and path of your tool for this action.
Also, you must associate a management IP address with each compute node in the cluster (for
example, the IP address for the Base Management Controller (BMC) of the compute node). The
management IP address is the third string (%3) that is passed to the CcpPower.cmd script by HPC
Cluster Manager, and should be provided to your power control tools when you add them in
CcpPower.cmd. A management IP address can be associated with each compute node in the
cluster in the following ways:
• When compute nodes are deployed using a node XML file, by specifying the
ManagementIpAddress attribute for each node. For more information, see Appendix 2:
Creating a Node XML File.
• By using the Set-HpcNode cmdlet in the HPC PowerShell, with the
ManagementIpAddress parameter. For more information about this cmdlet, see the
Windows HPC Server 2008 PowerShell Reference (http://go.microsoft.com/fwlink/?
LinkID=120725). Alternatively, in the HPC PowerShell, type Get-Help Set-HpcNode.
Appendix 6: Using HPC PowerShell

HPC PowerShell is built on Microsoft Windows PowerShell™ technology, and provides a
powerful command-line interface and a scripting platform to enable the automation of
administrative tasks. HPC PowerShell is installed by default on the head node, and can also be
installed on a client computer as part of the utilities available with Windows HPC Server 2008.
To start HPC PowerShell on the head node
1. Click Start, point to All Programs, and then click Microsoft HPC Pack.
2. Right-click HPC PowerShell, and then click Run as administrator.
3. If you are prompted by Windows PowerShell to choose if you want to run the
65
To start HPC PowerShell on a client computer
1. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC
PowerShell.
2. If you are prompted by Windows PowerShell to choose if you want to run the
You can also add the HPC PowerShell snap-in from Windows PowerShell.
To add HPC PowerShell from Windows PowerShell
1. Click Start, point to All Programs, click Windows PowerShell 1.0, and then click
Windows PowerShell.
2. In Windows PowerShell, type the following cmdlet:
Add-PsSnapin Microsoft.HPC
Add the HPC PowerShell snap-in to your Windows PowerShell

profile
If you have a Windows PowerShell profile, you can add the HPC PowerShell snap-in to it so that
it is available in every PowerShell session under your user name. For more information about
Windows PowerShell profiles, see Windows PowerShell Profiles (http://go.microsoft.com/fwlink/?
LinkID=119587).
To add the HPC PowerShell snap-in to your Windows PowerShell profile
1. Open Windows PowerShell. Click Start, point to All Programs, click Windows
PowerShell 1.0, and then click Windows PowerShell.
2. To edit your profile in Notepad, type:
notepad $profile
3. Type the following cmdlet as a new line in the profile:

Add-PsSnapin Microsoft.HPC
4. To save the profile, in the File menu, click Save.

5. To close Notepad, in the File menu, click Exit.
View Help in HPC PowerShell

In-context help is available for HPC PowerShell cmdlets:
• To view a list of the cmdlets that are available in HPC PowerShell, type the following
cmdlet:
Get-Command –PSSnapin Microsoft.HPC
• To view basic help information for a specific cmdlet, type:
66
Get-Help <cmdlet>
Where <cmdlet> is an HPC PowerShell cmdlet.

• To view detailed information for a specific cmdlet, type:
Get-Help <cmdlet> -Detailed
Where <cmdlet> is an HPC PowerShell cmdlet.

• To view the help information on the screen one page at a time, type | More at the end.
For example, if you type:
Get-Help New-HpcJob –Detailed | More
Only the first page of the detailed information for the New-HpcJob cmdlet will be initially
displayed. As you press SPACE or ENTER, more information will be displayed. To stop
viewing the help information, press Q.
67

Design and Deployment Guide For Windows HPC Server 2008

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Design and Deployment Guide For Windows HPC Server 2008

Încărcat de

Drepturi de autor:

Formate disponibile

Design and Deployment Guide for Windows

HPC Server 2008

© 2008 Microsoft Corporation. All rights reserved.

All other trademarks are property of their respective owners.

Checklist: Deploy an HPC Cluster (Overview)................................................................................4

Step 1: Prepare for Your Deployment.............................................................................................5

Step 2: Deploy the Head Node......................................................................................................11

Step 3: Configure the Head Node.................................................................................................13

Step 4: Add Compute Nodes to the Cluster..................................................................................20

Step 5: Run Diagnostic Tests on the Cluster.................................................................................26

Step 6: Run a Test Job on the Cluster...........................................................................................26

Checklist: Deploy an HPC Cluster (Overview)

Checklist: Prepare for your deployment

Compatibility with previous versions

Server roles added during installation

1.2. Decide how to add compute nodes to your

1.4. Choose a user account for installation and

1.6. Prepare for multicast (optional)

1.7. Prepare for the integration of scriptable power

Step 2: Deploy the Head Node

Checklist: Deploy the head node

2.1. Install Windows Server 2008 on the head node

2.2. Join the head node computer to a domain

2.3. Install Microsoft HPC Pack 2008 on the head

To install Microsoft HPC Pack 2008 on the head node computer

Step 3: Configure the Head Node

Checklist: Configure the head node

To configure the HPC cluster network

3.2. Provide installation credentials

To provide installation credentials

1. In the To-do List, click Provide installation credentials. The Installation

3.3. Configure the naming of new nodes

3.4. Create a node template

To create a node template

To add drivers for the operating system images

3.6. Add or remove users (optional)

To add or remove users for the cluster

1. In the To-do List, click Add or remove users.

Step 4: Add Compute Nodes to the Cluster

4.1. Deploy compute nodes from bare metal

To deploy compute nodes from bare metal

4.2. Add compute nodes by importing a node XML

To add compute nodes by importing a node XML file

4.3. Add preconfigured compute nodes

To install HPC Pack 2008 on a compute node computer

4.4. Monitor deployment progress

To monitor deployment progress

4.5 Cancel the deployment of a node

To cancel the deployment of a node

To run diagnostic tests on the cluster

Step 6: Run a Test Job on the Cluster

Checklist: Run a test job on the HPC cluster

To create a simple job template

6.2. Create and submit a job

To create and submit a job

1. In Job Management, in the Actions pane, click New Job.

6.3. Create and submit a job using the command-

To create and submit a job using the command-line interface

Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

6.4. Create and submit a job using HPC

To create and submit a job using HPC PowerShell

4. To create a new job, type the following cmdlet: