Ha Scalability Elb Rout3e53

Architecting for Availability & Scalability with
Elastic Load Balancing and Amazon Route 53

David Brown (Elastic Load Balancing)
Sean Meckley (Amazon Route 53)
Paul Kearney (InfoSpace)
November 15, 2013
2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Thursday, November 21, 13

welcome
!2
Everything fails all the time
Werner Vogels, CTO, Amazon.com
!2
Avoid single points of failure.
!4
Elastic Load Balancing and Amazon Route 53 are
critical components when building scalable and
highly-available applications.
!5
Load Balancer Elastic Secure Integrated Cost-Effective
[ What is Elastic Load Balancing? ]
!6
Availability Zone 1a
EC2 Instances EC2 Instances

Elastic
Elastic
Load Balancing
Load Balancing
(Internal)
Client
Availability Zone 1b
[ What is Elastic Load Balancing? ]
!7
3
Levels
of
Availability
!7
1
Instance
Availability
!8
1 2
Instance Zonal
Availability Availability
!8
1 2 3
Instance Zonal Regional
Availability Availability Availability
!8
1
Instance
Availability
!9
First step in increasing the availability
of a system or application.
!10
Load balancer used to
route incoming
requests to multiple
EC2 Instance
Client EC2 instances
[ Instance Redundancy ]
!12
EC2 Instance Load balancer used to
route incoming
Elastic
Load Balancing requests to multiple
EC2 Instance
Client EC2 instances
EC2 Instance
[ Instance Redundancy ]
!13
Incoming request load shared
by all instances behind the load balancer.
!13
EC2 Instance
Elastic Leastconns used to spread

Load Balancing
request across healthy
EC2 Instance
Client instances
EC2 Instance
[ Request Routing ]
!15
EQUAL UTILIZATION
ON EACH INSTANCE
EC2 Instance

Load Balancing
EC2 Instance
Client instances
EC2 Instance
[ Request Routing ]
!15
EQUAL UTILIZATION
TARGETS INSTANCES WITH
ON EACH INSTANCE
FEWEST OUTSTANDING REQUESTS
EC2 Instance

Load Balancing
EC2 Instance
Client instances
ADJUSTS TO REQUEST SMOOTHS REQUEST LOAD
RESPONSE TIMES ACROSS ALL INSTANCES
EC2 Instance
[ Request Routing ]
!15
Instances that fail can be replaced seamlessly
while other instances continue to operate.
!15
EC2 Instance
Application level
Elastic health checks ensure that
Load Balancing
EC2 Instance
request traffic is shifted
Client
away from a failed instance
EC2 Instance
[ Health Checks ]
!17
FAILURE DETECTED
X
EC2 Instance
Application level
Load Balancing
EC2 Instance
Client
EC2 Instance
[ Health Checks ]
!17
TRAFFIC SHIFTED
FAILURE DETECTED
X X
EC2 Instance
Application level
Load Balancing
EC2 Instance
Client
EC2 Instance
[ Health Checks ]
!17
TRAFFIC SHIFTED
FAILURE DETECTED
X X
EC2 Instance
Application level
Load Balancing
EC2 Instance
Client
EC2 Instance HEALTHY INSTANCES CARRY

ADDITIONAL REQUEST LOAD
[ Health Checks ]
!17
TRAFFIC SHIFTED
FAILURE DETECTED
USED TO DETERMINE THE HEALTH OF
THE INSTANCE X
AND APPLICATION
X TCP AND HTTP
EC2 Instance
Application level
Load Balancing
EC2 Instance
Client
CONSIDER THE DEPTH AND CUSTOMIZE FREQUENCY
ACCURACY OF YOUR AND FAILURE THRESHOLDS
EC2 Instance HEALTHY INSTANCES CARRY
HEALTH CHECKS
ADDITIONAL REQUEST LOAD
[ Health Checks ] 503 ERRORS RETURNED IF

NO HEALTHY INSTANCES
!17
Auto Scaling can be used to automatically adjust
instance capacity up or down depending on
conditions you define.
!18
Elastic
Load Balancing
EC2 Instance EC2 Instance EC2 Instance
[ ELB & Auto Scaling ]
!19
Elastic
Load Balancing
LOAD INCREASES
!19
Elastic
INSTANCES ADDED FOR Load Balancing
INCREASED LOAD
EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance
!19
Elastic
Load Balancing
EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance
LOAD DECREASES
!19
Elastic INSTANCES REMOVED
Load Balancing
AS LOAD DECREASES
!19
Elastic INSTANCES REMOVED
Load Balancing
AS LOAD DECREASES
AUTOMATICALLY SCALES
INSTANCES UP OR DOWN CUSTOM SCALING METRICS
REDUCES COSTS
AUTOMATICALLY REPLACES
FAILED INSTANCES
!19
2
Zonal
Availability
!19
Availability Zones are distinct geographical
locations that are engineered to be insulated from
failures in other zones.
!20
Region
Availability Zone
!21
It is important to run application
stacks in more than one zone.
!22
Avoid unnecessary dependencies
between zones.
!23
Zone 1a
EC2 Instances
Load balancer used to
Elastic
Load Balancing balance across instances in
multiple Availability Zones.
Client
EC2 Instances
Zone 1b
[ Availability Zone Redundancy ]
!25
Each load balancer will contain one or more
DNS records, one for each load balancer node.
!25
Client
Elastic
192.0.2.1 Load Balancing 192.0.2.2
EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance
[ Understanding DNS ]
!27
Client
Elastic
192.0.2.1 Load Balancing 192.0.2.2
DNS ROUND ROBIN USED TO

EXPECT DNS RECORDS
BALANCE TRAFFIC BETWEEN
TO CHANGE OVER TIME
AVAILABILITY ZONES
EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance
[ Understanding
EACH LOAD BALANCER DOMAIN NAME
DNS ]
MAY CONTAIN MULTIPLE A RECORDS
!27
Using multiple Availability Zones does
bring a few challenges.
!27
requests / minute
Availability Zones may

see traffic imbalances
due to clients caching
DNS records.
time
[ Multiple Zone Challenges ]
!28
Zone 1a
2
EC2 Instances An unequal number of
Elastic instances per zone can
Load Balancer
lead to over utilization of
Client
3 instances in a zone.
EC2 Instances
Zone 1b
[ Multiple Zone Challenges ]
!30
Problem solved.
!30
Cross-Zone Load Balancing distributes
traffic across all healthy instances,
regardless of Availability Zone.
!31
Zone 1a
EC2 Instances Effectively balances the

Elastic request load across all
Load Balancing
instances behind the load
Client
3 balancer.
EC2 Instances
Zone 1b
[ Cross-Zone Load Balancing ]
!33
requests / minute
Traffic is spread evenly

across each of the active
Availability Zones.
time
!33
requests / minute
Availability
ELIMINATES Zones may
IMBALANCES IN
NO BANDWIDTH CHARGE see traffic
INSTANCE imbalances
UTILIZATION
FOR CROSS-ZONE TRAFFIC
due to clients caching
DNS records.
REDUCES IMPACT OF CLIENTS

REQUESTS DISTRIBUTED EQUALLY TO
CACHING DNS RECORDS
ALL INSTANCES REGARDLESS OF ZONE
time
!33
3
Regional
Redundancy
!35
Elastic Load Balancing and Amazon Route 53 have
been integrated to support a single application
across multiple regions.
!36
Region
Availability Zone
!37
AWSs authoritative Domain Name Service (DNS)
ROUTE
53
Health checking service
Highly available and scalable
Offers tools that provide flexible, high-performance, and

highly available architectures on AWS
[ What is Amazon Route 53? ]
!39
Improves availability by
ROUTE
53
health checking load balancer nodes and rerouting
traffic to avoid failures
supporting multi-region and backup architectures for
high-availability
[ What is Amazon Route 53? ]
!40
Health Checks
Automated requests sent over the
Internet to your application to verify
that your application is reachable,
+ Failover
Only returns answers for resources
that are healthy and reachable from
the outside world, so end users are
available, and functional. routed away a failed application.
[ What is DNS failover? ]
!40
Work on Failure Constant Work
System activity System activity
Time to react Time to react
time time
When nothing is failing, volume of API Health checkers and edge locations
calls is zero. When failure occurs, perform the same volume of activity
volume of API calls spikes. whether endpoints are healthy or
unhealthy.
[ How does it work? ]
!41
Amazon Route 53
conducts health checks
from within each AWS
region
[ Global Health Check Network ]
!43
NETWORK PARTITION
!43
150
MANUAL FAILOVER
operator receives an alarm
vs. operator manually
configures DNS update
SECONDS wait for DNS changes to

propagate
!44
NO CONTROL PLANE INVOLVEMENT
150
REQUIRED FOR FAILOVER TO OCCUR
MANUAL FAILOVER
operator receives an alarm
EDGE LOCATIONS PULL HEALTH RESULTS
vs. operator
DIRECTLY manuallyDISTRIBUTED
FROM GLOBALLY
configures DNS update
SECONDS
HEALTH CHECKER FLEET
wait for DNS changes to
propagate
DONT HAVE TO WAIT FOR API REQUESTS
TO SUCCEED AND THEN PROPAGATE

FAILOVER HAPPENS ENTIRELY WITHIN
THE AMAZON ROUTE 53 DATA PLANE
!44
Region
E-commerce site: example.com

Elastic
Running application stack in multiple Availability Load Balancing
Zones in a single AWS region
Wants a backup in case:
- Own application goes down across multiple

Availability Zones
- Some parts of the world experience

degraded connectivity to this AWS region EC2 Instances EC2 Instances
[ Simple Failover Scenario ]
!46
Region
Primary Secondary
ROUTE
53
Elastic
S3
Health
Load Balancing Check
!47
Region
Primary Secondary
X ROUTE
53
Elastic
S3
Health
Load Balancing Check
X
FAILOVER
HEALTH CHECK FAILS
!48
Static Site Static vs. dynamic content
[ Static Backup Site Options ]
!48
Provides your globally-distributed end users
with faster performance
Tag each destination end-point to the

Amazon EC2 region that its located in
Amazon Route 53 will route end users to the

end-point that provides the lowest latency
[ Latency Based Routing ]
!50
Better performance than running in a single region
Improved reliability relative to running in one region
Easier implementation than traditional DNS solutions Our customers bid on video ad
inventory in real time and our system
Much lower prices than traditional DNS solutions must evaluate the content they're
sponsoring and respond with a
decision in less than 50ms, or they'll
lose the auction. Route 53s Latency
Based Routing lets us easily run
multiple stacks of our whole targeting
platform in each AWS region so we can
meet our customers latency needs.
Jonathan Dodson,
Vice President of Engineering at Affine
[ LBR Benefits ]
!50
Region 1 Region 2
example.com wants faster
page load for customers Elastic Elastic
Load Balancing Load Balancing
Launches application stack in
additional AWS regions
Uses Amazon Route 53

Latency Based Routing
Amazon Route 53 DNS

Failover ensures that end
users are only routed to a
EC2 Instances EC2 Instances EC2 Instances EC2 Instances
region where the application is
healthy
[ Multi-Region Failover ]
!52
Region 1 Region 2
Primary Primary
ROUTE
53
Elastic Health Health Elastic
Load Balancing Check Check Load Balancing
!53
Region 1 Region 2
Primary Primary
ROUTE X
53
Elastic Health Health Elastic
Load Balancing Check Check Load Balancing
X
HEALTH CHECK FAILS AND

TRAFFIC SHIFTS AWAY
!54
Region 1 Region 2
Elastic Elastic
Load Balancing Load Balancing
S3
[ Multi-Region & S3 Failover ]
!55
[ Configuring DNS Failover ]
!56
AWS & InfoSpace
Elastic Load Balancing & Amazon Route 53 for High-Availability
!57
InfoSpace Search
Since 1996, our mission has been to make

it fast and easy for users to find what they
need online.
!57
InfoSpace Search
!58
InfoSpace Search
Search Sites
!58
InfoSpace Search
Search Sites Search API
!58
Types of Users
!59
Types of Users
Search Site
Users
400 million queries per

month
Broad geographical
distribution
!59
Types of Users
Search Site Search API
Users Partners
400 million queries per 150+ partners
month worldwide
Broad geographical Located primarily in
distribution US and EU
2 billion queries/month
!59
Types of Users
Search Site Search API
Click Users
Users Partners
400 million queries per 150+ partners 6.5 billion clicks/month
month worldwide Broad geographical
Broad geographical Located primarily in distribution
distribution US and EU
2 billion queries/month
!59
Global Distribution of Traffic
!60
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
AZ# AZ#
AZ#
AZ# AZ#
AZ#
AZ# AZ#
AZ#
!60
Key Statistics
4.5 billion requests/month
Migrated from 2 data centers to AWS in 5 months
Deployed in 4 regions
Approximately 500 EC2 instances
Approximately 50 load balancers
Approximately 70 Amazon Route 53 zones
!62
AWS Infrastructure Route$53$
Public$Subnet$ Private$Subnet$
NAT$ TSG$ Suppor+ng$

Services$
Search$ Search$
API$ Sites$
Outbound$via$NAT$
Suppor+ng$Services$
!62
Fire and Forget
!63
Fire and Forget
Production System under test
!63
Fire and Forget
!63
Fire and Forget
Asynchronous
!63
Fire and Forget
!63
Fire and Forget
!63
Fire and Forget
!64
Fire and Forget
!64
Fire and Forget
!64
Fire and Forget
LBR
LBR
!64
Fire and Forget
LBR
LBR
!64
Fire and Forget
LBR
!64
Results
Regional failover in 150 seconds consistently
Decreased latency 25% less latent worldwide
Can easily reroute individual partners to different region to avoid routing
problems
Replaced expensive network gear from datacenter
!65
What next?
Expanding to additional regions
Integration of monitoring data with traffic routing
!66
3
Levels
of
Availability
!67
1
Instance
Availability
!68
1 2
Instance Zonal
Availability Availability
!68
1 2 3
Instance Zonal Regional
Availability Availability Availability
!68
Please give us your feedback on this
presentation
CPN104
As a thank you, we will select prize
Thank You
winners daily for completed surveys!

Ha Scalability Elb Rout3e53

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Ha Scalability Elb Rout3e53

Încărcat de

Drepturi de autor:

Formate disponibile

Architecting for Availability & Scalability with

Elastic Load Balancing and Amazon Route 53

Thursday, November 21, 13

[ What is Elastic Load Balancing? ]

EC2 Instances EC2 Instances

EC2 Instances EC2 Instances

[ What is Elastic Load Balancing? ]

Elastic Leastconns used to spread

Elastic Leastconns used to spread

Elastic Leastconns used to spread

EC2 Instance HEALTHY INSTANCES CARRY

[ Health Checks ] 503 ERRORS RETURNED IF

EC2 Instance EC2 Instance EC2 Instance

[ ELB & Auto Scaling ]

EC2 Instance EC2 Instance EC2 Instance

[ ELB & Auto Scaling ]

EC2 Instance EC2 Instance EC2 Instance

[ ELB & Auto Scaling ]

[ ELB & Auto Scaling ]

[ Availability Zone Redundancy ]

DNS ROUND ROBIN USED TO

Availability Zones may

[ Multiple Zone Challenges ]

[ Multiple Zone Challenges ]

EC2 Instances Effectively balances the

[ Cross-Zone Load Balancing ]

Traffic is spread evenly

[ Cross-Zone Load Balancing ]

REDUCES IMPACT OF CLIENTS

[ Cross-Zone Load Balancing ]

Highly available and scalable

Offers tools that provide flexible, high-performance, and

[ What is Amazon Route 53? ]

[ What is Amazon Route 53? ]

[ What is DNS failover? ]

[ How does it work? ]

[ Global Health Check Network ]

SECONDS wait for DNS changes to

[ How does it work? ]

[ How does it work? ]

E-commerce site: example.com

Zones in a single AWS region

Wants a backup in case:

- Own application goes down across multiple

- Some parts of the world experience

[ Simple Failover Scenario ]

EC2 Instances EC2 Instances

[ Simple Failover Scenario ]

HEALTH CHECK FAILS

EC2 Instances EC2 Instances

[ Simple Failover Scenario ]

[ Static Backup Site Options ]

Tag each destination end-point to the

Amazon Route 53 will route end users to the

[ Latency Based Routing ]

Improved reliability relative to running in one region

Uses Amazon Route 53

Amazon Route 53 DNS

EC2 Instances EC2 Instances EC2 Instances EC2 Instances

HEALTH CHECK FAILS AND

EC2 Instances EC2 Instances EC2 Instances EC2 Instances

EC2 Instances EC2 Instances EC2 Instances EC2 Instances

[ Multi-Region & S3 Failover ]

Since 1996, our mission has been to make