Global Load Balancing For SSL VPN

GLOBE Data Center Network Design
5 Global Load Balancing (GLB)
5.1 Introduction
In distributed data center architectures, global load balancing (GLB) allows to virtualize a corporate
service at the global level, by providing a unique URL to users, for accessing content from any data
center. The benefit is performance through traffic distribution and, of course, business continuity
through multiple centers capable of providing the same content.
Global load balancing is a DNS based technology that processes user DNS requests for a specific
domain, by globally selecting the best data center for a specific host name, using pre-defined
criteria. Typical selection criteria are:
• Data center operational status (Up / Down)

• Round trip time (RTT) between the client and the data center
• Real time portal load condition in each data center
Once the best data center has been selected, the global load balancer answers to the user DNS
request with the best VIP address (Virtual IP address) that corresponds to the requested service.
Then, a local load balancer in the chosen data center will select the best real server for the user
request in the real server farm.
Although traditional DNS provides an efficient and scalable system for users to be matched with
the address of servers that contain the data they seek, the end user may not always be directed to
the best site. For example, traditional DNS has no way of knowing whether the host whose
address it receives is on-line, in which case the data returned may be an error message that the
server is down. In addition, in a distributed content architecture, the selected site is not always the
most optimal in terms of network delay or servers load condition.
In the GLOBE infrastructure, global load balancing technology is used for extranet access as well
in the intranet :
• Extranet : SSL VPN for employees, partners and 3rd parties

• Intranet : Global SAP Portal
In the next paragraphs, we will first explain the details of the Cisco GLB technology and then, we
will address the specific implementation for GLOBE extranet and intranet.
Network Design Guide 3.0.doc Page 182 of 311 28 March 2006

5.2 GLB from Cisco Systems

This section is a tutorial summary on Cisco Global Load Balancing technology that should help you
to understand the implementation description in the next two sections.
Basically, three elements make-up the Cisco GLB architecture:
• One or multiple global load balancers (Global Site Selectors)

• Multiple distributed local load balancers (Content Switches)
• Proximity measurement mechanism (DRP protocol & Agents)
In the following paragraphs, we will describe the exact role of each element in the global load
balancing solution. The following figure should help you to understand the role of each element :
(2) DNS request Global Site

(6) DNS response Selector (GSS)
Client
(3)
DR
(1) DNS request (4) RTT probing sg P
M Ms
RP g
)D
(3
g
(3)
s
RP M
e
DR
iv
(7) DNS response
VI
l
pa
P
PM
Ke
ee
(3) D
e
(5
K
ep
lu
VIP K
sg
)R
iv e
P
va
al
VI
TT
iv
pal
(8) HTTP Connection

T
e
RT
va
eepali
Kee
lu
)
e
(5
VI P
e v
(5) R
value
CSS/ CSS/
CSM CSM
TT v
(5) RTT
DRP DRP
luea
Agent Agent
Server farm Server farm
Site 1 Site n
CSS/ CSS/
CSM CSM
DRP DRP
Agent Agent
Server farm Server farm
Site 2 Site 3
Figure 74: Global Load Balancing Elements
To introduce this principles, here is a summary of what happens in a global load balancing
network, when a user wants to establish a connection :
• The requesting client sends a DNS A-record query to its local DNS proxy (1)
• The local DNS proxy determines that the GSS is authoritative for the domain and forwards
the DNS query to the GSS (2)
• If the DNS proxy server is requesting DNS resolution for the first time, the GSS proximity
database does not contain any proximity information for it. Therefore, the GSS sends the
configured default VIP answer to the D proxy (6), which forwards the answer to the client
(7)

• At the same time, the GSS sends a DRP message to each DRP agent, which initiates an
RTT probing process from the agents towards the D proxy (4).
• After the RTT has been determined, the measured values are sent to the GSS (5), which
stores the results in its proximity database. These entries will be used for subsequent DNS
requests form the same DNS proxy.
• In parallel, the GSS uses the keep alive messages to query local load balancers (CSS or
CSM) for portal status and load information
5.2.1 Global Load Balancer

On the global level, a Global Site Selector (GSS 4480 or 4490) acts as an intelligent DNS server
for the global domain. The global load balancer is also sometimes known as a “content router”.
This device is responsible to sense the availability of each VIP, their operational status and to
collect proximity information, in order to return the best answer to the user’s DNS lookup. The best
answer depends on service requirements, but this is typically based on the network latency and
real servers load. As we will see later, network latency (RTT) is the preferred criterion for the SSL
VPN connection while server CPU load is the main decision criterion for the Global SAP Portal.
5.2.1.1 GSS Roles

Basically, a GSS can be configured for 3 different roles in the global network :
• GSS Manager Primary: The primary GSSM is a GSS performing content routing as well
as centralized management functions for the GSS network. The primary GSSM serves as
the organizing point of the GSS network, hosting the embedded GSS database that
contains configuration information for all your GSS resources, such as individual GSS and
DNS rules. Other GSS devices report their status to the primary GSSM. Configuration
changes initiated on the primary GSSM using the graphical user interface (GUI) are
automatically communicated to each device that the primary GSSM manages. Any GSS
device can serve as a GSSM.
• GSS Manager Standby: The standby GSSM is a GSS performing DNS functions for the
GSS network even while operating in standby mode. In addition, the standby GSSM can be
configured to act as the GSSM should the primary GSSM go offline or become unavailable
to communicate with other GSS devices. As with the primary GSSM, the standby GSSM is
configured to run the GSSM GUI and contains a duplicate copy of the embedded GSS
database that is currently installed on the primary GSSM. Any configuration or network
changes affecting the GSS network are synchronized between the primary and the standby
GSSM so that the two devices are never out of step.
• GSS Slave: It is a slave GSS is performing routing of DNS queries based on DNS rules and
conditions configured using the GSSM. Each GSS is known to and synchronized with the
primary GSSM, but individual GSS do not report their presence or status to one another.
Each GSS on the network must delegate authority to the parent domain GSS DNS server
that serves the DNS requests. Each GSS is managed separately using the Cisco CLI. GUI
support is not available on a GSS device in slave mode. However, a GSS device may also
be serving as the primary GSSM if configured so.

5.2.1.2 GSS Availability

Up to 8 GSS can be deployed in a GSS network to provide high availability. Recall that only one
GSS in the network has the role of GSS Manager Primary. Each GSS (whatever role it has) is
authoritative for the domain and is able to answer to client DNS lookups. Upstream DNS (i.e
nestle.biz for the domain connect.nestle.biz) should be configured with multiple NS-records and
corresponding A-Records, one for each GSS in the domain. Upon reception of DNS lookups for the
global domain, the upstream DNS will return one of the listed GSS IP address (round-robin) to the
requesting client proxy DNS.
As an example, here is a DNS Dig lookup in one of the upstream DNS server for the domain
connect.nestle.biz for which two GSS devices are active (other GSS will be deployed as you can
see in chapter 5.3) :
; <<>> DiG 8.3 <<>> connect.nestle.biz NS @141.122.67.67 (the upstream DNS server)
;; QUERY SECTION:
;; connect.nestle.biz, type = NS, class = IN
;; ANSWER SECTION:
connect.nestle.biz. 1D IN NS ctrgss1.nestle.com. (The 1st GSS name)
connect.nestle.biz. 1D IN NS ctrgss2.nestle.com. (The 2nd GSS name)
;; ADDITIONAL SECTION:
ctrgss1.nestle.com. 1H IN A 141.122.188.18 (The 1st GSS IP address)
ctrgss2.nestle.com. 1H IN A 141.122.188.19 (The 2nd GSS IP address)
5.2.1.3 GSS Configuration and GLB Algorithm

It would be out of scope of this document to describe the details of the GSS configuration.
However, it is important to understand the basic concepts before reading the next paragraphs.
The GSS can be configured a multiple ways to handle receives DNS lookups. However, in our
implementation, we will always consider the GSS returning an IP address corresponding to the
best VIP to use for a specific client.
Global load balancing strategy is configured in a GSS by specifying “DNS rules”. To make it
simple, a DNS rule is a set of three balance clauses that are evaluated in order. Each clause
specifies a collection of possible VIP addresses (answers) and how to select the best one to be
returned to the client. If all sites fail to comply with the clause conditions (i.e all sites exceed a
configured load threshold or RTT cannot be measured), the next clause is evaluated. The last
clause is often used to return a default answer to the client if both other clauses failed.

The following figure shows all relevant configuration elements and their relationship:
VIP with VIP with VIP with

associated associated associated
Keep-Alive Keep-Alive Keep-Alive
Method Method Method


Answer Group Answer Group Answer Group
RTT Balance RTT Balance RTT Balance

on/off method on/off method on/off method
Clause #1 Clause #2 Clause #3
Address List DNS Rule Domain List
Figure 75: GSS Configuration Concepts
The DNS rule is the connection point of all configuration objects that make up the global load-
balancing configuration. A DNS rule accepts an address list that specifies source IP addresses
valid for this DNS rule, a domain list that specifies for which domain name the DNS rule is destined
and up to three load-balancing clauses, together with whether or not RTT probing should be
enabled and which load balancing method should be used. Each answer group specifies a list of
possible VIP answers to be use by the load balancing clause.

Here is an attempt to explain the GSS Global Load Balancing algorithm:
DNS lookup
received
Is the lookup
coming from a
configured src
address list ?
N Y
Select the DNS Select the DNS

rule with no src rule for this src
address list address list
Can Clause #1
be used ?
N Y
Evalue the best

Can Clause #2
site for the
be used ?
request
N Y
Evalue the Evalue the best Return the best

Clause #3 as a site for the site VIP address
default answer request to user
Return the Return the best

default VIP site VIP address
address to user to user
Figure 76: Generic GSS Algorithm
5.2.2 VIP Keep-Alive

The GSS maintains a list of available sites by periodically scheduling keep-alive packets to each
VIP address hosted by data Centres. Different types of keep-alive are used by the GSS, depending
on what information has to be retrieved from the local load balancers. Basically, we consider two
types of keep-alive:
TCP Keep-Alive: This keep-alive is a periodic TCP session establishment attempt to the VIP
address, using the conventional three-way handshake exchange (SYN - SYN/ACK - ACK). The
TCP port used by the keep-alive is the port used by the real service. Once a session has been
successfully established with the VIP, the GSS closes the session either with a graceful sequence
(FIN - FIN/ACK - FIN/AK - ACK) or with a RST packet. This is a configurable option. If the keep-
alive fails after three consecutive retries, the corresponding VIP is marked inactive by the GSS,
which removes the site for its DNS routing decisions. After 3 successful keep-alive (configurable),
the site is declared back on-line.

KAL-AP Keep-Alive: KAL-AP (Keep-Alive Access Protocol) is a more advanced method to test a
VIP address. KAL-AP is a UDP based protocol (UDP port 5002) used between the GSS and
distributed CSS/CSM. KAL-AP is not only able to retrieve availability of a VIP, but also the
averaged load information for real portal servers. Load of each individual real server is measured
by the local load balancer (see chapter 5.4 for a detailed explanation on how server load is
measured) and an averaged value for the entire server farm is sent to the GSS via KAL-AP. Load
is a number between 2 and 255 where a load of 255 represents an offline site.
5.2.3 Proximity
Proximity is an important aspect of a global load balancing solution, especially if the access is has
to be established through a network where latency is highly variable, like the Internet. In such a
case, one would prefer to connect to the most proximate data centre, if it can provide the same
content, to minimize response time. Proximity is determined by the DRP protocol (Director
Response Protocol), which used by the GSS and DRP agents running on Cisco routers.
In each data centre, an agent running in a Cisco router is responsible to measure the round trip
time (RTT) between itself and the requesting DNS proxy.
5.2.3.1 DRP Protocol and DRP Agents

DRP is a query/response mechanism using UDP port 1974 (reserved by IANA). The GSS sends
DRP-echo messages every 60 seconds to all DRP agents to keep track of their availability. The
GSS can anytime initiate a proximity probe request by sending a DRP message to all DRP agents,
asking them to start probing the DNS proxy IP address (the IP address of the DNS proxy is
included in the DRP request message)
Currently, a DRP agent running in an IOS router uses two methods to probe the DNS proxy:
• ICMP Echo
• TCP SYN/ACK
The first probe method is a simple PING packet. If the first method fails (most probably due to
packet filtering), the DRP agent tries the second method, which is a TCP SYN/ACK packet, sent to
the DNS proxy on port 53. As defined in TCP RFC #793, if a host receives a packet with both SYN
and ACK bits set, and no SYN packet has been sent before by the host, the host should respond
with a TCP reset (RST) packet back to the DRP agent. The delay between the SYN/ACK and the
RST packets is measured in milliseconds by the DRP agent and sent to the GSS.
5.2.3.2 Proximity Database

Once RTT values have been collected by the DRP agents and communicated to the GSS, the
GSS maintains a local proximity database for each individual DNS proxy. This database will be
used by the GSS for subsequent proximity decisions. Over time, the GSS proximity database gets
populated with proximity information. 500’000 entries can be stored in the GSS proximity database
and the database is periodically dumped on the GSS disk. It can be uploaded or downloaded from
or to another GSS if necessary (each GSS maintains its own independent proximity database).
RTT values for DNS proxies are updated every hour (configurable) by requesting the DRP agents
to start a new RTT probe. If an entry in the proximity database is not used during 168 hours (1
week), the entry is removed from the database.

5.2.4 DNS Caching

When a DNS server returns an IP address to a client, the time-to-live (TTL) value in the answer
informs the client to keep the A-record during this time period. The client will then use the returned
IP address for subsequent content requests, which avoids launching a new DNS lookup. The client
decrements this TTL value each second and when it reach zero, a new DNS lookup is sent if the
same content is requested. The TTL value is often set to several hours. Here is an example of a
lookup answer where the TTL value is set to more than 15 hours :
QUESTIONS:
www.cisco.com, type = A, class = IN
ANSWERS:
-> www.cisco.com
internet address = 198.133.219.25
ttl = 55650 (15 hours 27 mins 30 secs)
In a global load-balancing environment, the GSS should indicate a much lower TTL value, in order
to allow the user to re-assess the best site, in order to adapt to the Internet connection
performance variations. Therefore the GSS can be configured to set the TTL to a lower value. 60
second is a good choice
Here is an example applied to the domain connect.nestle.biz , where the TTL value is set to 20
seconds :
QUESTIONS:
connect.nestle.biz, type = A, class = IN
ANSWERS:
-> connect.nestle.biz
internet address = 160.213.122.146
ttl = 20 (20 secs)
DNS cache & Microsoft Explorer :
Microsoft Internet Explorer ignores DNS TTL value returned in an A-record. It means that even if
the A-record returned by the GSS contains a short TTL value, the client browser caches the DNS
answer for some fixed amount of time, typically 30 minutes. In other words, the Windows operating
system use the returned TTL value, but the browser uses an internal DNS cache of 30 minutes !
5.2.5 Global Persistence

Persistence (stickiness) enables the GSS to remember the DNS A-record returned to each
individual client DNS proxy. Hence, even if the proximity condition changed (i.e another site would
be selected), the same VIP address is returned to the client, provided the data center is available
and not overloaded. An inactivity timer ages out unused entries and to remove an entry pointing to
an unavailable VIP. Global persistence tables can be shared between all GSS in the network, in
order for the client to receive the same answer, disregarding which GSS received the DNS lookup.

5.2.6 Local Load Balancers

In each data centre, a local load balancer (CSS or CSM) evenly distributes user http requests
among real portal servers. Local load balancers monitor availability and load of each individual
server in the server farm. Load balancing decisions are based on configurable algorithms, from a
simple round robin to more complex algorithms that consider actual server load.
Local load balancers receive keep-alive messages from all GSS in the network and respond to
them according to the keep-alive type :
TCP Keep-Alive : If there is at least one active real server in the server farm, the CSS/CSM
completes the session establishment sequence. If all servers are down, the keep alive fails
KAL-AP Keep-Alive : If there is at least one active real server in the server farm, the CSS/CSM
returns the averaged server load for each active VIP in the CSS/CSM. The load is locally
measured on each individual real server.

5.3 GLB for connect.nestle.biz

The SSL VPN connect.nestle.biz infrastructure allows Nestlé employees to securely access their
local applications and mail from any place in the Internet, without needing to install a VPN client on
its client station. The user connects to the SSL infrastructure using the SSL (Secure Socket Layer)
capabilities of its browser. Special Java programs will be downloaded from the SSL infrastructure
to the client station, hence creating a secure runtime environment in memory (no direct disk
access, no files kept on the local station). The SSL VPN infrastructure is only destined for Internet
access, not from the Intranet.
This section is a detailed explanation of Global Load Balancing applied to the SSL VPN Access.
Note that SSL VPN GLB implementation is applied to the access of the service and not to the data
the user can access. Internet users get connected with data centre presenting the lowest network
latency but access their local applications through the GWAN. As the response time in the GWAN
is managed (which is not the case in the Internet), an optimal response time should be observed.
5.3.1 GLB Elements and Placement

The following figure is a global representation showing placement of key GLB elements :
EUR
2 x GSS on VLANK0
1 x VIP on VLANK0
2 x DRP Agts on VLANO0
2 x SSL Access Devices
AMS AOA
1 x GSS on VLANK0 1 x GSS on VLANK0
1 x VIP on VLANK0
Internet 1 x VIP on VLANK0
2 x DRP Agts on VLANO0 2 x DRP Agts on VLANO0
2 x SSL Access Devices 2 x SSL Access Devices
Figure 77 : SSL VPN Elements Placement
The three regional data centres participate in the global load balancing architecture. A user
connected to the Internet requests access to the infrastructure using the unique URL
https://connect.nestle.biz. In the three data centres, a full set of GLB elements are installed. This
includes Cisco-GSS, DRP agents, Cisco-CSS and SSL Appliances.

• Cisco GSS (G1 & G2) : Provide DNS based global load balancing functions
• DRP Agents : Perform RTT probing on request from the GSS devices
• Cisco CSS (C3 & C4) : Perform local load balancing functions
• Juniper SSL IVE : Perform SSL tunnel terminations and application relay
5.3.1.1 GSS Roles and Availability

DC-EUR hosts both GSSM-Primary and GSSM-Standby, while DC-AMS and DC-AOA only have
one GSS-Slave installed. In the global network however, all four GSS devices are used to process
client DNS lookups (recall that upstream DNS servers use a round robin algorithm for their NS-
record answers). GSS devices in AOA and AMS report their status to the GSSM-Primary and
GSSM-Standby located in EUR. Configuration changes done on the GSSM-Primary are
immediately replicated in the GSSM-standby and in the GSS-Slave devices. This Inter-GSS
communication will flow through GWAN, to avoid any security concerns.
Important notice:
At the time of this writing, CTR (Vevey) is still providing an SSL VPN access with two GSS devices
and two SSL Appliances, as any other regional data centre. However, this description corresponds
to the strategic deployment of GLOBE, where RDC and CDC are being moved to DC-EUR.

5.3.1.2 Elements Placement

The following diagram shows the logical connections of each element in the WAN Block :
VLANG1 (101) SCZ-3
VLANG2 (102) SCZ-3
R5 A B R6
VLANI0 (501) SCZ-3
VLANSYN78 (603) Firewall synchronization only
VLANJ0 (502) SCZ2
F7 R7 C3 L3 G1 IP55 V1 V2 IP56 G2 L4 C4 R8 F8
SCA GSS SSL SSL GSS SCA

VLANR1 (509) SCZ3
VLANR2 (510) SCZ3

IP44 IP49 IP53 IP54 IP50 IP45
VLANK0 (503) SCZ2
F5 VPN VLANQ0 (508) SCZ1 VPN F6
VLANO0 (507) SCZ1
F3 VLAN_SYN34 (601) Firewall synchronization only F4
VLANN0 (506) SCZ2
IP1 IP2
DRP DRP
Agent Agent
GWAN 1 LNS 1 DIA 1 DIA 2 LNS 2 GWAN 2
Figure 78: GLB Elements Placement
This layout corresponds to the EUR implementation as two GSS devices are connected. In AOA
and AMS data centres, only one GSS is installed. Please refer to the key IP address assignment
table on page 109 for GSS and SSL appliances IP addresses.
5.3.2 Proximity
Proximity is measured between the 3 regional data centres (EUR, AOA and AMS) and any
requesting DNS proxy in the Internet
5.3.2.1 DRP Agents

Each regional data centre provides two redundant DRP agents, configured in the DIA Internet
routers. One DIA router is acting as primary DRP agent, while the second router is a backup DRP
agent. Each GSS in the network checks availability of DRP agent by sending them encrypted
keep-alive messages at regular intervals.

Here is a generic DRP agent configuration to apply on all DIA routers:
!
ip drp server
!
ip access-list 2 permit <IP49 in EUR>
ip access-list 2 permit <IP50 in EUR>
ip access-list 2 permit <IP49 in AOA>
ip access-list 2 permit <IP49 in AMS>
ip drp access-group 2
!
ip drp authentication key-chain gss
key-chain gss
key 1
key-string cdcgss
!
Figure 79 : DRP Agent configuration for SSL VPN
5.3.2.2 Issue with non-responding DNS proxies

When DRP is used in the Internet, we observe about 45% of unsuccessful probing, mostly due to
firewalls or other filtering mechanisms, installed by ISPs to protect their DNS servers. This causes
an issue as no RTT can be measured for some DNS proxies and hence, the closest data centre
cannot be determined.
Note :
At the time of this writing, Nestlé is in discussion with Cisco Systems, requesting more probing
methods to be implemented in the GSS to improve the ratio of successful RTT probing
Therefore, static entries can be entered in the GSS proximity database to direct users, for which no
RTT is available, to the most proximate data centre. Different IP to location databases are
available in the market and on-line mapping services exist as well (www.ip2location.com/ or
www.maxmind.com/app/ip_locate). This method is however somehow cumbersome as the static
entries have to be configured via the GSS CLI and static entries must be manually copied from one
GSS to another.
The designed solution is more straightforward. We configure a specific DNS rule for each region
(AMS, AOA, EUR) to which we assign an address list with IP prefixes assigned by Regional
Internet Registries (RIR). The GSS will then statically return the regional VIP address to the client,
providing the VIP is available and active. Hence, we can consider the static VIP definition as a
backup answer, in case of non-responding DNS proxies.
ISPs obtain allocations of IP addresses from a Local Internet Registry (LIR) or National Internet
Registry (NIR), or from their appropriate Regional Internet Registry (RIR). IANA (Internet Assigned
Numbers Authority) is the organism that assigns addresses to these registries.
In a first step, we will configure static entries for addresses assigned to the following regional
registries (RIR):
• ARIN (America) Mapped to AMS

• LACNIC (Latin American and Caribbean) Mapped to AMS
• RIPE (Europe) Mapped to EUR

• AFRINIC (Africa) Mapped to EUR

• APNIC (Asia Pacific) Mapped to AOA
You can see a list of all assigned IP prefixes assigned to these registries at the following web site:
(http://www.iana.org/www.iana.org/assignments/ipv4-address-space)
While address ranges assigned to these registries can be easily mapped to GLOBE data centers,
other address ranges are flagged by IANA as assigned to “various registries”, or to commercial
companies and other organizations. For instance, addresses between 128.0.0.0/8 to 172.0.0.0/8
are assigned to different registries worldwide, and for those addresses, a clear geographical
location cannot be determined. Therefore, for those addresses, a “wildcard” DNS rule is configured
with no address list assignment.
5.3.3 GSS Global Load Algorithm

The following figure shows the decision tree for connect.nestle.biz, as configured in the GSSM:
DNS lookup
received
What is the
address of the
D-proxy ?
An address An address An address

Another
assigned by assigned by assigned by
addresse
ARIN / LACNIC RIPE / AFRINIC APNIC
Evalue
Evalue Evalue Evalue
Wildcard
AMS DNS Rule EUR DNS Rule AOA DNS Rule
DNS Rule
Is RTT probing Is RTT probing Is RTT probing Is RTT probing

successful to this successful to this successful to this successful to this
DNS-proxy ? DNS-proxy ? DNS-proxy ? DNS-proxy ?
Y N Y N Y N Y N
Return the most Is the AMS Return the most Is the EUR Return the most Is the AOA Return the most Return another
proximate active regional VIP proximate active regional VIP proximate active regional VIP proximate active active VIP
VIP to client active ? VIP to client active ? VIP to client active ? VIP to client (Round-Robin)
Y N Y N Y N
Return another Return another Return another

Return the AMS Return the EUR Return the AOA
active VIP active VIP active VIP
VIP to client VIP to client VIP to client
(Round-Robin) (Round-Robin) (Round-Robin)
Figure 80: GLB Algorithm for SSL VPN
First, the GSS checks if the lookup request comes from a configured address list. If it is the case,
the appropriate DNS rule is selected. Else, a wildcard DNS rule is used which does not specify any
address list. Once the appropriate DNS rule is selected, the GSS then evaluates the balance
clauses in order they are defined.
All DNS rules use the same principle: The GSS tries to use proximity information (RTT) to select
the best available site. If no proximity information is available because the DNS proxy cannot be
probed, then the GSS returns the data centre VIP address mapped to the address list, if an

address list is specified in the DNS rule. If not (wildcard DNS rule), or if the regional data centre
VIP is not available, the GSS selects one other active data centre VIP in a round robin fashion.
5.3.4 DNS Rules for SSL VPN

This section explains the details of regional DNS rules and the wildcard DNS rule configuration.
5.3.4.1 Regional DNS Rules

Here is a sample GSS DNS rule configuration for DC-EUR :
Figure 81 : SSL VPN Regional DNS Rule
The Source Address List “PREFIXES-EUR” is a pointer on an address list that contains IP prefixes
assigned to RIPE and AFRINIC.
• In the Balance Clause #1, the answer group “CONNECT-RTT-ANSGRP” contains VIP
addresses for EUR, AOA and AMS. Because “Proximity enable” is checked on this clause,
RTT probing results will be evaluated to select the closest active VIP in the network.

• In the event where RTT probing was unsuccessful, Balance Clause #2 is executed. In this
clause, the GSS returns the EUR VIP address. This is the best decision as the source IP
address of the received DNS lookup came form a RIPE assigned prefix.
• If the EUR VIP is not available (i.e both SSL appliances are down), the GSS selects
another available data centre VIP, in a round robin fashion.
5.3.4.2 Wildcard DNS Rule
The wildcard DNS rule is used to process DNS lookups coming from IP addresses outside any
static prefixes definition:
Figure 82 : SSL VPN General DNS Rule
As you can see, the Source Address List field indicates “Anywhere”.
• In the Balance Clause #1, the answer group “CONNECT-RTT-ANSGRP” contains VIP
addresses for EUR, AOA and AMS. Because “Proximity enable” is checked on this clause,
RTT probing is used and results will be evaluated to select the closest VIP in the network.
The associated balance method (Round Robin) has no significance, as only one VIP will be
returned to the client.

• In the event where RTT probing was unsuccessful, Balance Clause #2 is executed. ), the
GSS selects another available data centre VIP, in a round robin fashion.
5.3.5 VIP Keep-Alive

The GSS maintains a list of available VIP by periodically scheduling keep-alive packets to each
CSS VIP7 addresses in EUR, AOA and AMS. This keep-alive is a periodic TCP session
establishment attempt (once every minute) to TCP port 443, using the conventional three-way
handshake exchange (SYN - SYN/ACK - ACK). Once a session has been successfully
established, the GSS closes the session with a RST packet. If the keep-alive fails after three
consecutive retries, the corresponding VIP is marked inactive by the GSS, which removes the site
for its DNS proximity routing decisions. After 3 successful keep-alive (configurable), the VIP is
declared back on-line.
5.3.6 DNS Caching

GSS answers indicate a TTL value of 20 seconds. As we explained in paragraph 5.2.4, this value
has no real significance, as the browser does not use the TTL value returned by the GSS.
5.3.7 Global Persistence

Persistence is not required for SSL VPN. The reason is that the best site is only selected to
establish the SSL tunnel. Later, the client uses SSL tunnel (that remains open as long as the user
uses its connection) for all subsequent DNS lookups for internal content. In other words, there is no
new DNS lookup for connect.nestle.biz after the tunnel has been established. Therefore global
persistence is not enabled in the GSS configuration
5.3.8 Local Load Balancing

The CSS C3 and C4 on VLANK0 provide local load balancing on both SSL IVE devices on the
same subnet. There is no SSL processing on the CSS itself. Upon reception of a new TCP-443
session establishment request (SYN packet) on VIP7, the CSS selects an IVE device fro the
session using a round robin algorithm. The CSS provides HTTPS redirection if the user omits to
specify it in the URL.

5.3.9 Dataflow Analysis

This section shows the different flows within the WAN Block logical layout for the SSL VPN global
load balancing solution.
5.3.9.1 DRP & DNS Communication

The following layout shows the communication path between GSS devices and DRP agents, as
well as DNS request/answer path between GSS and DNS proxies in the Internet:
VLANG1 (101) SCZ-3
VLANG2 (102) SCZ-3
R5 HSRP2
A B R6
VLANI0 (501) SCZ-3
FO4
VLANJ0 (502) SCZ2

VLANR1 (509) SCZ3
FO3
VLANR2 (510) SCZ3

HSRP6 VIP7 FO10 IP49 IP53 IP54 IP50
VLANK0 (503) SCZ2
FO2
FO1
VLANO0 (507) SCZ1
VLANN0 (506) SCZ2
IP1 HSRP1 IP2
DRP DRP
Agent Agent
Figure 83: DRP & DNS Dataflow
DRP packets sourced by the GSS and destined to DRP agents on VLANO0 in each data centre
are routed through the GSS’s default gateway HSRP6 on R7/R8. Then, the packets are routed
through firewalls F5/F6 towards the Internet using the R7/R8 default gateway FO2. DRP reply
packets are directly to the GSS without passing through R7/R8.
Similarly, DNS lookup packets from Internet users are directly delivered to the GSS on VLANK0.
DNS reply packets however, are routed through R7/R8 via the GSS’s default gateway HSRP6.

5.3.9.2 VIP Keep-Alive

Each GSS in the network sends Keep-Alive probes (TCP session sequence on TCP port 443) to
each data centre VIP7. The following figure shows this traffic:
VLANG1 (101) SCZ-3
VLANG2 (102) SCZ-3
R5 HSRP2
A B R6
VLANI0 (501) SCZ-3
FO4
VLANJ0 (502) SCZ2

VLANR1 (509) SCZ3
FO3
VLANR2 (510) SCZ3

VLANK0 (503) SCZ2
FO2
FO1
VLANO0 (507) SCZ1
VLANN0 (506) SCZ2
IP1 HSRP1 IP2
DRP DRP
Agent Agent
Figure 84: VIP Keep-alive dataflow
This VLANK0-to-VLANK0 communication must pass through the Internet in order for the GSS
devices, to keep track on the real VIP availability, as seen by an Internet user. By default, VLANK0
to VLANK0 communication is routed through F7/F8 to the internal network. Therefore, static routes
have to be configured in R7/R8, C3/C4 and F5/F6, to force GSS keep-alive inbound and outbound
packets to use the Internet connection.
The GSS uses its default gateway HSRP6 to send its keep-alive probes. R7/R8 then uses the
configured static routes (see paragraph 5.3.12) to remote VIP7 addresses in other data centres, to
send the probes via F5/F6. On the return path, the static routes configured in C3/C4 force the
keep-alive return packets to pass through F5/F6. On F5/F6, a static route to remote VLANK0
subnets forces the keep-alive packets to flow through the Internet.

5.3.9.3 Inter-GSS Communication

• GSS devices communicate together and exchange status reports. More precisely, GSS
slave devices (AMS and AOA) communicate their status to both GSSM primary and GSSM
standby in EUR. In addition, they synchronize configuration changes done on the GSSM
primary GUI interface.
VLANG1 (101) SCZ-3
VLANG2 (102) SCZ-3
R5 HSRP2
A B R6
VLANI0 (501) SCZ-3
FO4
VLANJ0 (502) SCZ2

VLANR1 (509) SCZ3
FO3
VLANR2 (510) SCZ3

VLANK0 (503) SCZ2
FO2
FO1
VLANO0 (507) SCZ1
VLANN0 (506) SCZ2
HSRP3 IP1 HSRP1 IP2
DRP DRP
Agent Agent
Figure 85: Inter-GSS Communication
Inter-GSS communication is a VLANK0-to-VLANK0 communication that can use the internal

network (GWAN) for this purpose. Hence, the GSS uses its default gateway HSRP6 on R7/R8,
which forward the traffic through F7/F8 to the target VLANK0 destination. Remember that R7/R8
receive routes to other VLANK0 subnets from R5/R6 via BGP.
GSS use the following UDP/TCP ports for inter-GSS communication:
• UDP 2000 : GSS status reports

• TCP 2001-2009 : Inter-GSS communication
• TCP 3001-3009 : Inter-GSS communication

5.3.9.4 User Traffic

The following figure shows the SSL VPN user traffic path in the WAN Block :
VLANG1 (101) SCZ-3
VLANG2 (102) SCZ-3
R5 HSRP2
A B R6
VLANI0 (501) SCZ-3
FO4
VLANJ0 (502) SCZ2

VLANR1 (509) SCZ3
FO3
VLANR2 (510) SCZ3

VLANK0 (503) SCZ2
FO2
FO1
VLANO0 (507) SCZ1
VLANN0 (506) SCZ2
IP1 HSRP1 IP2
DRP DRP
Agent Agent
Figure 86: SSL VPN User Dataflow
SSL session establishment requests coming from Internet users are directly received by the CSS
on VIP7, which represents the SSL VPN connect.nestle.biz service. Then, the CSS uses a Layer-4
content rule to load balance these sessions on both SSL appliances on IP53 and IP54. No source
address NAT is performed by the CSS. Therefore, a virtual redundant interface (FO10) is
configured on both CSS and used as default gateway by the SSL appliances for the return
packets. This ensures that the return traffic flows through the same CSS as the incoming flow. For
the return flow towards the client, the CSS uses its default gateway HSRP6, which then routes the
packets through F5/F6 by using its own default gateway (FO2). Remember that R7/R8 are acting
as route servers on VLANK0, and therefore decide if a packet has to be routed in the internal
network or through the Internet.
SSL appliances appear as proxy devices for user traffic. New HTTP sessions are established
between the SSL appliances and the target server (Outlook Web Access, Documentum etc..).
5.3.10 In-Band GSS Administration

In-Band GSS administration is exclusively performed from the Management Plane. Packets are
routed through R5/6 and F7/F8 towards the GSS and the GSS uses its default gateway HSRP6 on
R7/R8, to route the packets back to the Management Plane. No specific static routes are required

for this traffic; recall that R7/R8 receive Production Plane and Management Plane routes from
R5/R6 via BGP (see section 4.6.4)
Administration traffic for the GSS includes the following protocols:
• HTTPS : Graphical configuration tool

• Syslog : Logging messages
• SNMP : Device management
• TACACS : User authentication and authorization
5.3.10.1 Access to GSS GUI

To access the GSSM GUI, the administrator connects to a Windows Terminal Server on the
Management Plane and then, starts the HTTPS connection to the GSSM. Terminal Service has to
be enabled on CiscoWorks servers in all GLOBE data centre Management Plane. Note that the
GSS is not integrated in the current version of CiscoWorks, but administrators should use
CiscoWorks servers as terminal server platform to access the GSS GUI. All data centres should
provide terminal services in their CiscoWorks server for this purpose.
5.3.10.2 Syslog and SNMP Trap

Syslog messages and SNMP Trap alerts generated by the GSS are collected by the ZoneRanger
appliance (http://www.tavve.com), acting a proxy for SCZ-2 managed devices. A secure SSL
connection is then established between the local appliance and their software counterpart on the
Management Plane. Then, the software ZoneRanger receiver on the Management Plane will
dispatch the messages to their final destination (Netview, CiscoWorks, etc.)
5.3.10.3 TACACS+
GSS devices send authentication requests to the TACACS+ server, connected to
VLANX6/VLANY6 on the Management Plane. In order to guarantee that packets from the GSS are
routed through FM1/FM2, the static NAT address defined in section 6.5.6.3 on VLANP1/VLANP2 is
used as target address.
5.3.11 Out-of-Band GSS Administration

GSS devices are connected to VLANK0, which sits on security zone SCZ-2. As there is no
dedicated management LAN interface on the GSS device, access to the Command Line Interface
(CLI) is allowed through the reverse Telnet infrastructure of the Management Plane (see paragraph
6.7.4).

5.3.12 Static Routes Definition

Some static routes are required to implement the SSL VPN solution. The following list is generic
and valid for EUR, AOA and AMS.
Device Destination Next Hop Notes
G1 Default HSRP6 GSS default gateway

G2
R7 Default FO2 Points to F5/F6 towards the Internet

R8
Remote-VIP7 FO2 Used for VIP7 Keep-alive from local GSS to
VIP7 in other data centres. This forces keep-
alive through the Internet
C3 Remote-IP49 FO2 Used for VIP7 Keep-alive responses, sent

C4 Remote-IP50 FO2 back to the GSS, which sent the keep-alive.
This forces packets to flow through the
Internet.
Note: Without these routes, return keep-alive

packets would be routed by R7/R8 through
the internal network
Remote-VIP7 HSRP1 Used for VIP Keep-alive from local GSS to

F5 VIP7 in other data centres.
F6
Remote-VLANO0 HSRP1 Used to reach DRP agents from the local
GSS
Figure 87: Static routes for SSL VPN

5.3.13 Firewall Rules in F5/F6

In order to permit keep-alive and probing communication, the GLB solution for connect.nestle.biz
requires the following firewall rules to be configured in F5/F6 in EUR, AOA and AMS :
Src IP Dst IP Src Port Dst Port Notes
IP49 IP1 EUR UDP-1974 UDP-1974 DRP communication from

IP50 IP1 AOA local GSS (G1/G2) to DRP
IP1 AMS agents. This includes access
to the local DRP agents on
IP2 EUR the local VLANO0
IP2 AOA
IP2 AMS
IP49 Remote-VIP7 ANY TCP-443 Keep-alive from local GSS to

IP50 VIP7 in other data centres
ANY VIP7 ANY TCP-443 Permit user SSL traffic from

the Internet and VIP7 keep-
alive to from remote data
centres
ANY VIP7 ANY TCP-80 Allows CSS C3/C4 to

automatically redirect users to
HTTPS
ANY IP49 ANY UDP-53 DNS request/response from

IP50 the Internet to the GSS
G1/G2
Figure 88: F5/F6 Firewall Rules for connect.nestle.biz


The following firewall rules are to be configured in F7/F8 for internal communication :
IP33 on IP49-EUR ANY TCP-443 GSS GUI Access

VLANP1-EUR IP50-EUR TCP-21 FTP
IP49-AOA TCP-20 FTP
IP34 on IP50-AOA
VLANP2-EUR IP49-AMS
IP50-AMS
IP49 Remote-IP49 ANY UDP-2000 GSS Status reporting

IP50 Remote-IP50
TCP-2001- Inter-GSS communication
TCP-2009

TCP-3009
Bidirectional Rule
R5 loopback0 IP49 UDP-123 UDP-123 NTP, Time synchronization

R6 loopback0 IP50 with local R5/R6
IP33 on IP49 ANY UDP-161 SNMP requests from the local

VLANP1 IP50 Management Plane (NAT by
FM1/FM2)
IP34 on
VLANP2


For inter-GSS communication, specific ports must be permitted on F3/F4 :
IP33 on IP49-EUR ANY TCP-443 GSS GUI Access

VLANP1-EUR IP50-EUR TCP-21 FTP
IP49-AOA TCP-20 FTP
IP34 on IP50-AOA
VLANP2-EUR IP49-AMS
IP50-AMS
IP49 Remote-IP49 ANY UDP-2000 GSS Status reporting

IP50 Remote-IP50
TCP-2009

TCP-3009
Bidirectional Rule

Global Load Balancing For SSL VPN

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Global Load Balancing For SSL VPN

Încărcat de

Drepturi de autor:

Formate disponibile

GLOBE Data Center Network Design

5 Global Load Balancing (GLB)

• Data center operational status (Up / Down)

• Extranet : SSL VPN for employees, partners and 3rd parties

Network Design Guide 3.0.doc Page 182 of 311 28 March 2006

5.2 GLB from Cisco Systems

Basically, three elements make-up the Cisco GLB architecture:

• One or multiple global load balancers (Global Site Selectors)

(2) DNS request Global Site

(7) DNS response

(8) HTTP Connection

Server farm Server farm

Server farm Server farm

Figure 74: Global Load Balancing Elements

Network Design Guide 3.0.doc Page 183 of 311 28 March 2006

5.2.1 Global Load Balancer

5.2.1.1 GSS Roles

Network Design Guide 3.0.doc Page 184 of 311 28 March 2006

5.2.1.2 GSS Availability

5.2.1.3 GSS Configuration and GLB Algorithm

Network Design Guide 3.0.doc Page 185 of 311 28 March 2006

VIP with VIP with VIP with

VIP with VIP with VIP with

VIP with VIP with VIP with

Answer Group Answer Group Answer Group

RTT Balance RTT Balance RTT Balance

Clause #1 Clause #2 Clause #3

Address List DNS Rule Domain List

Figure 75: GSS Configuration Concepts

Network Design Guide 3.0.doc Page 186 of 311 28 March 2006

Here is an attempt to explain the GSS Global Load Balancing algorithm:

Select the DNS Select the DNS

Evalue the best

Evalue the Evalue the best Return the best

Return the Return the best

Figure 76: Generic GSS Algorithm

5.2.2 VIP Keep-Alive

Network Design Guide 3.0.doc Page 187 of 311 28 March 2006

5.2.3.1 DRP Protocol and DRP Agents

5.2.3.2 Proximity Database

Network Design Guide 3.0.doc Page 188 of 311 28 March 2006

5.2.4 DNS Caching

DNS cache & Microsoft Explorer :

5.2.5 Global Persistence

Network Design Guide 3.0.doc Page 189 of 311 28 March 2006

5.2.6 Local Load Balancers

Network Design Guide 3.0.doc Page 190 of 311 28 March 2006

5.3 GLB for connect.nestle.biz

5.3.1 GLB Elements and Placement

Figure 77 : SSL VPN Elements Placement

Network Design Guide 3.0.doc Page 191 of 311 28 March 2006

5.3.1.1 GSS Roles and Availability

Network Design Guide 3.0.doc Page 192 of 311 28 March 2006

5.3.1.2 Elements Placement

VLANG1 (101) SCZ-3

VLANG2 (102) SCZ-3

VLANI0 (501) SCZ-3

VLANSYN78 (603) Firewall synchronization only

VLANJ0 (502) SCZ2

SCA GSS SSL SSL GSS SCA

VLANR2 (510) SCZ3

VLANK0 (503) SCZ2

VLANSYN56 (602) Firewall synchronization only