Documente Academic
Documente Profesional
Documente Cultură
white papers are now part of the Schneider Electric white paper library
produced by Schneider Electrics Data Center Science Center
DCSC@Schneider-Electric.com
Contents
Click on a section to jump to it
12
13
Real-time management of
high-density pods
16
Conclusion
17
Resources
18
Appendix A
19
Introduction
High-density equipment such as blade servers, 1U servers, and multi-core, high-end servers
provide more computing-per-watt compared to previous generation servers. However, when
consolidated, this equipment requires concentrated power and cooling resources. Data
center operators and IT executives are often uncertain about the capability of their existing
data center and whether a new data center must be built to support higher rack densities. A
simple solution exists that allows for the rapid deployment of high-density racks within a
traditional low-density data center. A high-density pod, as illustrated in Figure 1, allows data
center managers to support a mixed-density data center environment for a fraction of the cost
of building an entirely new data center.
Low-density room
Figure 1
Basic concept of a highdensity pod
High-density
pod
Hot/cool air
circulation is
contained within
the pod
In this paper a high-density pod is defined as one or more rows of racks containing highdensity equipment all clustered together with dedicated row-based cooling that is deployed as
a unit. A high-density pod resides within the borders of a larger, low-density data center.
The high-density pod is not the same as a high-density data center, which is a data center
dedicated to supporting nothing but, or mostly, high-density racks. Managing for the deployment and operation of a high-density data center is not the subject of this paper.
Rev 2
Consumption of additional floor space, which may be difficult to justify or simply not
possible
Reduced electrical efficiency of the data center, because cooling-system air paths are
longer and less well targeted. The longer the air path in an uncontained system, the
more chance there is for hot and cold air to mix. This mixing results in the lowering of
the return temperature to the air conditioner which means the system will be less efficient in removing heat energy. For more information regarding the containment of air
streams, see White Paper 135, Hot-Aisle vs. Cold-Aisle Containment for Data Centers.
For these reasons, it is expected that data enter operators will begin to deploy IT equipment
at its full density capability in pods rather than try to stay within an overall room power
density by spreading out the load. With new power and cooling technologies, there is now a
significant efficiency entitlement from concentrating high-density equipment into pods.
Related resource
White Paper 46
Cooling Strategies for UltraHigh Density Rack and Blade
Servers
The problem:
unmanaged high
density
This paper assumes the choice has been made to deploy high-density IT racks in a lowdensity data center. Row-based cooling, as a technique to implement these high-density
pods, is presented as a simple solution for addressing high-density power and cooling issues
in both existing and new data centers. For more information regarding alternatives for
deploying high-density equipment, including the option of spreading out IT equipment, see
White Paper 46, Cooling Strategies for Ultra-High Density Racks and Blade Servers.
Traditional data center design uses a raised floor to distribute cooling to low-density IT
equipment (Figure 2a) and air streams are uncontained. However, when high-density
equipment is randomly installed throughout a low-density data center the cooling stability is
upset and hot spots begin to appear (Figure 2b).
Low-density room
Low-density room
Stable
cooling
Unpredictable
cooling
Data centers designed for low-density racks (typically 1-3 kW / rack) vary dramatically in
construction. Ceiling heights, raised floor depths, room geometry, power distribution, and
raised floor obstructions are all quite different. In addition, IT managers vary in how they
define a high-density rack. This paper defines a high-density rack as 6 kW or higher.
Regardless of which number is used to denote a high-density rack, the following deployment
issues need to be considered:
Rev 2
Unplanned downtime due to overloaded power distribution circuits or thermal shutdown of IT equipment
Related resource
Unpredictable cooling throughout the data center no certainty that every high-density
server will be properly cooled after every move, add, or change (see White Paper #121,
Airflow Uniformity Through Perforated Tiles in a Raised-Floor Data Center).
Loss of cooling redundancy as more high-density racks are added, air conditioning
units that were once redundant are now required to supply the concentrated airflows.
Some subsystems are extremely impractical or costly to instrument for power consumption (for example, PDUs due to number of output connections, or switchgear)
The solution:
high-density
pods
Fortunately a solution exists that can neutralize these issues and is discussed in the following
sections. Placing high-density racks in an isolated, standardized, and self-contained area of
the data center provides a low cost, viable solution to the challenges mentioned above. This
high-density pod avoids dependence on the unpredictable nature of raised floor cooling and
would not require complex computational fluid dynamics (CFD) analysis prior to installation.
Figure 3 illustrates three high-density pod implementation methodologies all of which are
capable of supporting independent power distribution, UPS, and cooling systems. This dropin solution eliminates the hot spots in Figure 2b by simply moving high-density equipment
into the pod that contains dedicated row-oriented cooling units. The heat generated from the
high-density IT equipment within this pod is rejected to the outdoors with no negative impact
to the existing data center cooling system or the surrounding low-density IT racks. In fact,
the pod acts as its own high-density data center within an existing low-density data center.
This self-sufficient pod is, at a minimum, thermally invisible or, more likely, is a net exporter
of cooling to the rest of the room.
Low-density room
Figure 3
Isolated, self-sufficient
high-density pod
High-density
zone
Hot/cool air
circulation is
localized within
the zone
HEAT OUT
to buildings heat
rejection system
Uncontained
HOT-AISLE
containment
RACK
containment
Rev 2
Figure 4
IT
rack
IT
rack
IT
rack
IT
rack
Front-view of a standardized
modular multi-rack highdensity pod (no containment
in this example)
Integrated row-based
air conditioners
Rev 2
Related resource
Pod
containment
methods
In contrast, high-density pods use standardized hot / cold aisle widths, rack height, and air
path distances to the rack. Row-based cooling also eliminates the variability introduced by
the raised floor. These simplifications make it possible to design predictable high-density
pods using standardized tools. These design tools provide the confidence that any design
will capture and neutralize the expected amount of hot exhaust air. For more information on
the row-based cooling architecture, and how it compares to room-based cooling, see White
Paper 130, The Advantages of Row and Rack-Oriented Cooling Architectures for Data
Centers.
Server exhaust heat can be diverted back to the air conditioners in three ways: uncontained,
hot aisle containment, and rack air containment (see Figure 5). All of these methods
leverage a row-based cooling concept (e.g., the air conditioner is brought within a few feet of
the IT rack).
Rev 2
Low-density room
Uncontained
HOT-AISLE
RACK
Figure 5
High-density pod
containment methods
containment
Hot/cool air
circulation is
localized within
the zone
High-density zone
Room-neutral island
in a low-density room
containment
1. Uncontained
> Importance of
blanking panels
Effective row-based cooling
depends on the isolation of hot
and cold air streams. If any of
the vertical space in a rack is
not filled by equipment, the
gaps between equipment allow
hot exhaust air to flow through
the rack and to the front of
equipment such as servers.
This mixing between the hot
and cold air streams reduces
the effectiveness of row-based
cooling. For more information
see White Paper 44, Improving
Rack Cooling Performance
Using Airflow Management
Blanking Panels (link in
Resource section)
Uncontained pods rely on the standard layout and widths of the common hot aisle and cold
aisle arrangement to keep hot and cold air streams from mixing. For this reason, uncontained pods depend on multiple racks in a row and are not effective in cooling stand-alone IT
racks. The hot and cold aisles formed by rows of racks (and in some cases walls) are what
isolate the hot and cold air streams as illustrated in Figure 6. The closer an IT equipment
rack is to a row-based air conditionec, the greater the amount of exhaust air that is captured
and cooled. As the distance between the IT rack and the row-based air conditioner increases
in an uncontained system, the more the hot exhaust air mixes with the surrounding air in the
data center.
When to use this method:
When IT racks designated for the pod are moved and relocated frequently
When IT racks are used from a variety of different vendors
Trade-offs:
More row-based air conditioners required at lower densities in order to properly capture
hot exhaust air from all IT racks.
Figure 6
Rack
CRAC
Rack
CRAC
Rack
CRAC
REAR
High-density pod
with no containment
Rack
Hot aisle
FRONT
Rev 2
In cases where floor space must be conserved. This method is popular because it
consumes the same space as two rows of low-density racks.
CRAC
Rack
Hot aisle
Rack
CRAC
REAR
Contained
hot aisle
Rack
Rack
Rack
CRAC
High-density pod
with hot aisle
containment
Rack
CRAC
Rack
CRAC
Figure 7
CRAC
FRONT
Rack
FRONT
3. Rack containment
Rack containment (also called rack air containment) is similar to hot aisle containment except
that the hot exhaust air is contained using the back frame of the equipment racks and a
series of panels to form a rear air channel. This channel can be attached to a single IT rack
or to a row of racks (Figure 8). The panels used to create the hot exhaust air channel
increase the depth of a normal rack by 20 cm (8 in). An optional series of front panels may
be used on rack containment arrangements that require complete containment of hot and
cold air streams as shown in Figure 9. This optional front containment adds an additional 20
cm (8 in) to the depth of the rack.
Rev 2
In cases where hot aisle containment is the preferred method, but a single odd row is
left uncontained
In wiring closets that lack any form of cooling, exposing high-density equipment to high
temperatures only when optional front containment is used
When sound attenuation is required only when optional front containment is used
Trade-offs:
Rack
re
ar
do
or
s
Rack
Rack
CRAC
So
lid
CRAC
CRAC
FRONT
Figure 8
Rack
REAR
Rev 2
Single rack
Figure 9
Solid rear
doors
CRAC
FRONT
Rack
CRAC
Solid front
doors
Rack
Optional front
containment
REAR
fron
t do
or s
FRONT
Rack
re
ar
do
or
s
Rack
CRAC
So
lid
CRAC
Multiple racks
Rack
REAR
Rev 2
10
Table 1
Comparison of pod
containment methods
Selection
criteria
No
containment
Hot aisle
containment
Rack air
containment
Comments
Minimize
footprint
Good
Good
Moderate to poor
Ease of change
management
Good
Moderate to poor
Moderate to poor
Moderate
Good
Good
Moderate to poor
Moderate to good
Good
Minimize energy
consumption
Ease of
redundancy
Minimize # of
row-based
CRACs
(particularly at
low density)
Sound
attenuation
Installation in
thermally
unstable or
non-data center
space
Cost
Moderate
Poor to
moderate
Poor
Poor
Good
Good
Moderate to poor
Poor
Good
Dependent upon variables such as rack power density and number racks
Rev 2
11
The decision on whether to move forward with deployment of a high-density pod should also
consider the following benefits:
Related resource
Rev 2
12
Attribute
Traditional
approach
Pod
approach
Comments
Difficult
Easier
Very difficult
Easy
Table 2
Just-in-time IT
deployments
Predictability of
performance
High
Low
High
Very low
Cooling efficiency
Poor
Excellent
Ability to plan
Poor
Excellent
Standardization / predictability
facilitate what-if scenarios before
moves, adds, and changes
In-house vs.
vendor-assisted
deployment
The data center owner has two options for the deployment of high-density pods: in-house
deployment or vendor-assisted deployment. In both cases a solid project plan is required.
More specific information regarding data center projects and system planning is available in
white papers 140, Data Center Projects: Standardized Process and 142, Data Center
Projects: System Planning.
Rev 2
13
In-house deployment
Related resource
Related resource
IT managers can easily deploy smaller sized pods or smaller data centers (less than 20
racks) with no previous experience. A worksheet and checklist is provided in Appendix A.
This worksheet can serve as a helpful guide and facilitates the collection of information
required to specify and deploy a high-density pod. The worksheet assumes the project owner
has knowledge of the IT equipment associated with the planned high-density pod (e.g. total
power requirements, plug requirements, rack U-height requirements and communications
cabling requirements).
If the worksheet is properly filled out, an educated decision can be made on which pod
containment method to choose. APC TradeOff Tool, Data Center InRow Containment
Selector, (see Figure 10) can help select the most appropriate pod containment method.
The results generated by the tool are based on typical scenarios and in some cases the
recommended containment option may differ from the actual final design.
Once a containment type is chosen, a decision must be made on which components the pod
will include. The worksheet helps data center staff determine whether to include a dedicated
UPS, PDU, or chiller. In some cases, certain preferences and constraints dictate which
components are included in a pod and which are not. Table 3 provides a list of possible
constraints that could affect the ultimate configuration of the high-density pod.
Constraint
None
No spare power
distribution positions
Racks, row-based cooling unites, and rowbased power distribution unit (PDU)
Table 3
High-density pod components under various
constraints
No spare cooling capacity on existing Racks, row-based cooling units, and packaged
chiller
chiller
No spare power or cooling capacity
on existing UPS and chiller
Rev 2
14
Figure 10
Interactive tool for
containment
method selection
Even with the constraint of no spare UPS, chiller, or power distribution capacity, it is still
possible to extend the life of an existing data center by installing a high-density pod with its
own dedicated power and cooling resources. For example, the high-density pod in
Figure 11 includes its own chiller plant, UPS, and power distribution. It is assumed that the
data centers electrical service entrance has sufficient spare capacity to supply power to this
packaged solution. In cases where a data center has run out of spare electrical service
capacity, a decision must be made to install an additional utility feed(s) or build a new data
center. Other factors beyond the scope of this paper such as available floor space, virtualization potential, business objectives, leasing contracts, and future growth plans factor into the
buy-or-build decision.
Figure 11
Packaged standalone highdensity pod
Packaged chiller
Rev 2
15
Vendor-assisted deployment
Although it is possible for data center staff to deploy high-density pods without outside
assistance, projects involving data centers with 20 or more racks can be considerably more
complex. In such cases consultation with design experts and project managers is recommended.
Vendor-assisted deployment usually begins with an assessment of the existing data center or
the design plans for a new data center. In either case an assessment provides the design
experts with valuable information, including preferences and constraints, which allows
optimum design decisions. Assessments help answer questions such as:
Can an existing row be retrofit with row-based air conditioners to avoid downtime?
If spare chilled water capacity is unavailable should a self-contained air conditioning unit
be used as opposed to a packaged chiller?
What steps can be taken to increase the speed of deployment of a future high-density
pod?
An effective assessment (such as Schneider Electrics Blade Server Readiness Assessment)
measures spare bulk power and cooling capacity as well as spare distribution capacity. Bulk
cooling capacity is measured at the chiller while the distribution capacity is measured at the
CRAH units on the data center floor. This data provides an estimate of cooling capacity and
compares constraints against current and future requirements. Ultimately this will help
answer the question, When will I run out of cooling capacity and require a high-density pod?
After measuring and analyzing the data, a plan is created to meet future high-density needs.
In the end, an effective design plan for mixed-density data centers should incorporate power,
cooling and floor space utilization efficiency. An effective design plan allows a data center to
use up its power, cooling, and space resources all at the same point in the future, thereby
avoiding stranded resources.
Real-time
management
of high-density
pods
Related resource
Rev 2
16
Conclusion
In the past it was a major challenge for IT personnel to successfully deploy a mix of highdensity and low-density equipment in the same data center space. Traditional data centers
were specified to cool a uniform rack power density and were not capable of predictably
cooling a large number of high-density racks. Now architectures such as row-based cooling
allow for the rapid deployment of high-density pods within an existing or new low-density data
center. Modular row-oriented power and cooling can be added where and when high-density
racks are required, without any negative effect on the existing room-level infrastructure. In
combination with capacity and change management systems, pods offer a high-density
deployment solution capable of maintaining predictable operation even after moves, adds,
and changes.
Rev 2
17
Resources
Click on icon to link to resource
Browse all
white papers
whitepapers.apc.com
Browse all
TradeOff Tools
tools.apc.com
Contact us
For feedback and comments about the content of this white paper:
Data Center Science Center
DCSC@Schneider-Electric.com
If you are a customer and have questions specific to your data center project:
Contact your Schneider Electric representative
Rev 2
18
Power
The room
The pod
Data / input
Value
Comments
Criticality level:
1, 2, 3, or 4
10
11
12
13
Rev 2
19
Cooling
Data / input
Value
Comments
14
15
16
17
18
19
20
21
22
23
24
Rev 2
20
Monitoring / management
Data / input
Value
Comments
25
26
27
28
29
Rev 2
21