The 2017 Insiders Guide to AWS:

55 Best Tips

1 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved
 Introduction PAGE 3

App Development PAGE 4

Automation & Scaling PAGE 6

Cost Optimization PAGE 6

Migration PAGE 8

Monitoring PAGE 8

Operations PAGE 9

Security PAGE 10
 Introduction Going to re:Invent? Check Out These Tips From Jeff Barr!

While on the hunt for AWS scoop, we also got our hands on some
insider tips from AWS Chief Evangelist Jeff Barr. Here is what Jeff
Amazon Web Services (AWS) continues to be the worldwide market leader
recommends for anyone attending re:Invent 2016:
in cloud servicesmore than Microsoft, Google, and IBM combined. In Q3
2016 alone, AWS posted $3.2 billion in revenue, representing a 55%
Tip #1: Wear comfortable shoes. You will be walking 7 to 10 miles
percent increase from Q3 last year.
per day.

Many cloud experts have weighed in on the reasons behind AWS

Tip #2: Plan for travel time between events. The venue is large and
tremendous success. We invited five such experts to give us their opinions
it can take a meaningful amount of time to walk from one location to
on AWS and other cloud computing trends. Andreas Wittig, an AWS cloud
another. You may want to study venue maps ahead of time in order
specialist and author of Amazon Web Services in Action, offers one
to situate yourself.
explanation for their unparalleled growth:

Tip #3: Take this opportunity to make good contacts with the AWS
Being able to automate every part of your infrastructure is the most
partners and the AWS team. Attend sessions, strike up conversa-
important advantage of using AWS. A flexible infrastructure is a game
tions, and make friends.
changer for agile software development. The flexibility of cloud infrastructure
and automating the deployment of applications and infrastructure allows
Tip #4: Balance your time on social media with this once-per-year
you to accelerate the speed of innovation and to increase reliability.
opportunity to make meaningful human connections.

One thing seems certain: 2017 is primed to be the year of the

great migration to the cloud. To help cloud users everywhere prepare for
whats to come, and in honor of AWS re:Invent 2016, we invited AWS users
around the globe to share their best tips for getting the most out of AWS.
The annual AWS Insider Tips competition was judged by a panel of five
industry experts hailing from the top ranks of the cloud computing,
information technology, and security fields. They judged hundreds of
contributions, and hand-picked the winning tips.

Read on to discover the 55 most valuable AWS insights across a range of

subjects, including app development, automation and scaling, cost
optimization, migration, monitoring, operations, and security.

3 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved
 App Development 4 Use - instead of . in bucket names for SSL. If you ever want to use your
bucket over SSL, using a . will cause you to get certificate mismatch
errors. You cant change bucket names once youve created them, so you
would have to copy everything to a new bucket.
Ramazan Varlikli, Founder at Cloudnesil

1
Treat your infrastructure as code. Leverage CloudFormation Tools (Ter-

5
raform, sparkleformation, stack_master) to aid in this. I find it useful to
When using the CLI interactively within your terminal, command completion
destroy/rebuild VPCs/Stacks frequently to help avoid manual changes
is a killer feature you should not miss. When enabled, command
sneaking in. Manual changes get forgotten; code in source control is
completion allows you to use the TAB key to complete commands. This will
forever!
significantly speed up your CLI usage.
Jarrod Pooler, DevOps Engineer at Homesite Insurance
Gloria Perez, Senior Director Strategic Solutions for IT at IPI

2 6
Before trying to secure the world, test your environment with an open
Build your instances using Terraform to fully script the immutable instance
security group so you know its working. Once you confirm connectivity,
deployment/setup. Create shutdown QA/test instances on the weekend
lock it down to specific ingress/egress ports. Many people spend too much
with a script to save money. Always refresh your immutable instances
time chasing issues that are actually just security group lockdowns too
weekly to prevent your CPU time slice being stolen by instances outside
tight to start.
your account.
Frank Scalzo, Director Of Engineering And Support at Avalere Health
Dennis Meinecke, Director Platform Service Operations at Gogo
LLC

3 7
CloudFormation, API Gateway and Lambda are the best tools to quickly
Decide when and what to cache. Determine your own autoscale policy. Un-
set up environments and resources in regions of your choice, so you can
derstand the difference between cooldown and time interval. Account for
create globally distributed, server-less cloud systems.
boot/up processes related on EC2. Use ignore health check if needed.
Adam Konrad, Programmer at Escalation
Set up multiple autoscale policies with different criteria. Balance frequency
and the number of health checks of ELB. Use custom check scripts.

8
Raphael Francis Quisumbing, Chief Technology Officer at Upteam Cor-
If you need to interact with AWS, use the SDK for your language. Dont try
poration
to roll your own. I did this at first as I only needed a simple upload to S3,
but then you add more services and its just an all-around bad idea. The
AWS SDKs are well written, perform authentication automatically, handle
retry logic, and theyre maintained and iterated on by Amazon.
Hema Ganesh, Software Engineer at Capital One

4 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved
 9
Leverage AWS Enterprise Support and the TAMs that are included with it.
The access to the architects and developers through the TAMs will make
the difference between an optimal implementation and one that is just
okay.
Kevin Bodie, Director of Technology Planning and Strategy at Pitney
Bowes
10
Leverage Lambda and DynamoDb in your CloudFormation scripts to max-
imize re-use.
Denis Angleton, St Staff Architect at Qualcomm
11
My operations tip: Disable SSH to your servers! Use automation for any
configuration changes. That way you can guarantee version / patch level /
version and security.
My cost tip: Always have one billing alert. It is very hard to track down
changes or any growth of the environment.
My backup & recovery tip: Check failover and restore operations. Kill/shut
down the server brutallyand verify your environment is up and running.
Use tools like swarm-army for chaos creation. Validate all is working on
the HA/DR siteor AZ. The same goes for restore operationsvalidate
restore files / Application aware / and all others are working and md5 is
the same.
Erez Kirson, Cloud Architect at 2cloud4
12
Without a doubt, the number one tip I repeat to AWS customers is to
invest in AWS certification. Technical certifications are a great way to
distinguish yourself among your peers, stay up to date, andmost im-
portantlyset aside time for your brain to connect technologies and best
practices to YOUR applications specific needs. The AWS certification
program focuses on concepts over trivia, rewards hands-on experience
5 | The 2017 Insiders Guide to AWS: 55 Best Tips
over book learning, and provides the sometimes necessary nudge to deep
dive into an unfamiliar AWS service or identify a new way to leverage a
service thats already a part of your application stack.
Jamey Taylor, CTO at TicketBiscuit, LLC
13
Turning web serving on S3 buckets is a great way to set up static websites.
We publish our CI logs and test reports to S3 buckets that we can then
browse.
Micah Silverman, Developer Evangelist at Stormpath
14
Use enhanced networking to get the most out of network-intensive appli-
cations, such as firewalls, WAFs log collection, monitoring and even high
IO applications using ELB.
Scott Wheeler, Public Cloud Practice Lead at IDS
15
When debugging network issues, use a simple one-line webserver to verify
network port to port connectivity. You can run this on a port to simulate
http traffic. Change the port number to whatever port you want to test on.
This is the command: while true do { echo -e HTTP/1.1 200 OK\r\n; date ;
} | nc -l 8500; done
Dave Hotlosz, DevOps Engineer at DNAnexus
16
When evaluating your needs, start with a small application to gain insights,
and then work to scale bigger.
Tammie Kong-Santos, Senior Director of Engineering at WCIRB
2016 CloudEndure Ltd. All Rights Reserved
 Automation & Scaling
17
To ensure your Lambda functions are ready to go at all times and you
dont get blindsided by spikes in demand, make sure youve got a
second Lambda function that calls the first one every 5 minutes or so.
This will reduce or eliminate the warm-up time as your function is
loaded. If its still not fast enough and youre suffering (particularly with
low network throughput), allocate more memory to the function. This will
increase the resources allocated to the function, although it will increase
the cost per invocation slightly.
Chris Buckley, Solutions Consultant at ITOC
18
You can save up to 70% on all of your development and testing EC2
infrastructure by using scheduled scaling. Autoscaling groups are not
a secret, but very few people use them to align their uptime with their
workday. Most organizations have large pools of resources that go
unused the vast majority of the time while their employees are at home!
See: http://docs.aws.amazon.com/autoscaling/latest/userguide/sched-
ule_time.html
You can put them up at the beginning of the work day so there is no
disruption, and at a MUCH lower cost.
Josh Jordan, Founder at Tenex Developers.
19
Do auto scaling to reduce your IT costs.
Jeremy Lau, Manager at Macquarie
6 | The 2017 Insiders Guide to AWS: 55 Best Tips
Cost Optimization
20
In order to reduce EC2 costs for dev and QA environments, as well as
licensed EC2 instances such at Tableau, you can use Lambda to sched-
ule power off and power on of these instances, e.g. 8:00 a.m. - 6:00
p.m. You can also provide consumers the ability to power them off and
on by placing the same function behind the API gateway and allow
users to invoke them via Slack integration (bot or slash command). Why
keep these instances on when no one is using them? We were able
to cut our EC2 costs by 40% by placing anything non-production and
business apps on a shutdown schedule.
Andrey Budzar, Principal Software Architect at Agero
21
When using S3-IA, and you notice the 128KB minimum billable object
size, it is naive to assume you should use S3 for anything smaller. De-
pending on access patterns, objects as small as 54KB can be cheaper
to store in S3-IA. Its trivial to write a formula that will calculate the best
storage class for each individual object, which can result in massive
cost savings at scale with no reduction in performance, availability, or
durability.
Don MacAskill, CEO & Chief Geek at SmugMug
22
Glacier is about an order of magnitude cheaper than S3, but with much
slower access times (on the order of multiple hours). Rather than delet-
ing old S3 objects, you can configure S3 to automatically expire them to
Glacier. The storage cost will be about 10x less. And if you really need
the data you can slowly restore it from Glacier. (Note: As of Dec. 1,
2016, AWS offers Expedited Retrieval for Glacier.)
Simon Lyon, CTO at Ariel Re
2016 CloudEndure Ltd. All Rights Reserved
23
Looking for a way to save on your EC2 instance development cost, but
dont want to make the commitment involved in reserved instances?
Just build a scheduled availability system with Lambda to manage EC2
instance availability when theyre not being used. Its an easy way to
get the 40%+ savings without being tied to a particular instance type or
25
Always encode your columns in your Redshift table (except the Sort
key). Run `analyze compression` and get the recommended encoding.
It can save Redshift costs up to 60%!
Pavel Kerbel, Data Team Leader at ironSource

26
the upfront cost of a reserved instance.
Ryan Weber, Solutions Architect, Business Intelligence at ISCS Snapshots are great for keeping block level backups, but storing them
for too long can get expensive. Analyze exactly how far back you think

24
Align AZs across linked accounts in consolidated billing to maximize
RI pooling and swapping. Within a consolidated billing group, AWS will
pool RI contracts to try to apply RIs to any eligible instance type/AZ
youll realistically need a snapshot, and only keep that much around.
Automate the deletion based on that date and reap the savings.
Chad Larter, Manager, Technical Operations at Nascar.com

27
combination in any linked account in the group. Many RI optimization
tools will suggest RI swaps/combinations to help put to use wasted
RIs (e.g. trading 2 larges for one X-large, etc.). If the AZs in these
accounts dont line up very well, the automatic pooling or swapping
opportunities will be crippled by poor AZ alignment across accounts,
as RIs are AZ specific.
Switch off your non-production systems outside of business hours and
on the weekend. There are a number of ways to achieve this. You can
use an app such as Cloudcycler or write a Lambda function to achieve
this.
Darren Evans, Cloud Architect at Wolters Kluwer

28
When you make new accounts which you intend to place into a con-
solidated billing group, make it a best practice to have the AZs line up The best way to save money using Redshift is to schedule a start and
with the payer account so the RI pooling can be most efficient. When stop event using AWS Lambda for the nighttime. Our cost was reduced
you create a new account, dont even open the console for it. Get the by 38 percent after we did this.
account name and the email with the account from the confirmation Piyush Patil, Devops at Burson Marsteller
page, and contact your AWS account manager. Tell them the payer
account user this will go to, and have the AWS team do their secret
undocumented procedure and re-jigger the assigned AZs for the new
account to line up with the AZs you are already using. Once AWS
manually assigns your AZs, you can go into the console and proceed
as normal.
David Bernstein, CTO at Cloud Strategy Partners

7 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved

Migration
29
When recovering a Windows instance from a snapshot/image and it
doesnt work, you can use EC2Savior to access the boot disk from an-
other instance in the same zone. Disable the firewall, check that the RDP
service starts at boot and also check the DHCP settings. It saved me
once when a customer configured a static IP address on instance.
Roque Raborg, Cloud Infrastructure Specialist at Cloudware360
30
Use a phased approach for application migrations from traditional data
centers.
1. Do an initial assessment of the application architecture and costs
involved for the feasibility study.
2. Create a Migration CoE within the organization and develop a proof of
concept plan. Implement POC with a pilot application migration strategy.
3. Define different storage options within AWS.
4. Use a forklift migration strategy or a hybrid migration approach.
5. Define and develop automation workflows and patterns.
6. Optimize the AWS environment: use monitoring, utilization parameters
and perform process re-engineering.
Pramod Verma, Sr Architect at IBM
31
Classify your applications as you consider a move to the cloud. This will
help you determine which apps move as they are, which apps need to
be refactored or rebuilt in the cloud, and which apps need to be replat-
formed for the latest OS.
Michael Mahler, Practice Director at Slalom Consulting
8 | The 2017 Insiders Guide to AWS: 55 Best Tips
32
Once youve migrated your applications using CloudEndure, be sure to
enable AWS tools like CloudFront to optimize your costs while speeding
delivery to your end users.
John Reed, Senior Product Manager at vXchnge
Monitoring
33
Monitoring is key: you cant improve if you cannot measure. There are
many expensive tools like Sp`lunk available to help you ingest and
graph the logs. I like to have a native tool available to plug&play
components in the AWS and show correlated logs/alerts. There are lots
of tools in place already but they are scattered everywhere, such as
CloudWatch, CloudWatch Event, CloudWatch Logs, S3, Lambda, etc.
Fei Yao, Sr. Architect at LexisNexis
34
Set up a custom integration in the chat system of your choice (Slack
for us) with the AWS status pages RSS feeds (http://status.aws.am-
azon.com). This way you will automatically get notified on service
outages that affect you and have the ability to check the channel if you
notice anything odd. It also allows you to go back in time and analyze
whether a service was affected at a certain time.
Fabian Meier, Sr. Engineering Mgr Cloud Solutions at Recommind
35
When testing new services, set up a billing alert to avoid going over
budget (and also to avoid forgetting to destroy your test instances).
Lynn Langit, Architect at Aurora Health Care
2016 CloudEndure Ltd. All Rights Reserved
 39
Operations
36
You can encrypt the root volume of your EC2 instance by doing a snap-
shot and copying to an encrypted snapshot. Then create a new, encrypted
volume and replace the original volume. When creating my instances (with
a script) it doesnt affect any production, but you can do it with existing
40
instances as well. This is a good way to have all volumes encrypted, not
just secondary volumes.
Eric Cady, Senior Systems Engineer at Decision Resources Group
37
Use individual AWS accounts to provide account isolation, with the ability
to grant independent administrative groups with different levels of admin-
41
istrative control over AWS resources. This limits visibility of workloads un-
less an administrator of the account enables access to users. It minimizes
the blast radius of a critical event such as security breaches and account
unavailability or suspensions. It supports strong isolation for businesses
that have regulatory or specialized auditing requirements. It also enables
better tracking of spend by line of business or project.
Scott Haglund, Enterprise Architect at Time Warner
42
38
If you enable WAF when working with CloudFront and ELB, do not forget
to lock down ELB to only allow traffic from CloudFront. See goo.gl/h0iFn5
Jake Barwell, Dev at Go MAD Thinking
43
9 | The 2017 Insiders Guide to AWS: 55 Best Tips
Monitor individual EC2 instances by tagging and organizing your apps,
instances, and servers so they roll up into a limited number of mean-
ingful categories. This makes it easier to find, view, and compare only
the apps or servers that matter to you, or quickly see the comparative
health status for categories (such as instance size, region, or AMI) at a
glance.
Shiva Rajagopalan, VP at iFox
Never put a full stop (.) in an S3 bucket name if you intend to use it as
an SSL encrypted endpoint, as it breaks certificate matching. Instead,
stick to hyphens (-) and alphanumeric characters.
Anthony Topper, Software Developer at Topper Studios
Always create multiple billing alerts. These alerts should be used to
match expected spendings to a certain date, and spendings based on
current usage of a service. Misconfigurations may sometimes occur but
its a great way to avoid potential harm.
Anton Zorin, Solutions Architects Team Lead at CloudBerry Lab
Cascading DNS through AWS accounts from a primary domain is
easyjust create your primary domain, and then add sub-domains in
your other accounts. Copy the NS records from the sub-domains and
paste them into a new NS record for the primary domain.
Stewart Barrett, DevOps Manager at BGL Group
CloudFront now supports HTTP/2. It can be configured on old distribu-
tions by simply updating the configuration.
Pasi Niemi, Senior Software Architect at Nitor Creations
2016 CloudEndure Ltd. All Rights Reserved
44
Combine tools that have complementary strengths. Ansible and Cloud-
Formation are a powerful team for orchestration, with Ansible handling Security
instances and software and CloudFormation as the network and re-
source configuration tool.

49
Ryan Brown, Senior Software Engineer at Red Hat
Set up automated security auditing. Its important to keep track of
changes in your infrastructures security settings. One way to do this

45
is to first set up a security auditer role (JSON template), which will give
Use Cloudwatch and Lambda to regularly evaluate and validate resource
anyone assigned that role read-only access to any security-related
tagging. If instances are out of compliance with your tagging policy, alert
settings on your account. You can then use this rather fantastic Python
your teams!
script, which will go over all the items in your account and produce a
Scott Mercer, Director, Data Solutions at Delaware North (DNC)
canonical output showing your configuration. You set up a cron job
somewhere to run this script, and compare its output to the output

46
from the previous run. Any differences will show you exactly what has
Use ElasticSearch in combination with CloudWatch to easily mine log-
been changed in your security configuration. Its useful to set this up
ging data and gain DevOps efficiency bonus points.
and just have it email you the differences of any changes.
Jake Skinner, Solutions Architect at Ontario Systems
Tariq Ali Syed, Software Architect at 8K Miles Software Services

47 50
Use Lambda to handle all EC2 backups, ElastiCache snapshots or other
Create a script that captures firewall rules, network access controls,
backup jobs on a schedule. Potentially tag resources to be included or
internal/external subnets, and operating system hardening and keep
excluded in the jobs. Run snapshots or burn AMIs, or fire off any other
it as your golden environment. AWS CloudFormation captures your
DR-related API calls with assurances that the schedule will be met. Use
security policy and will deploy it reliably. Security best practices can be
SNS for job failure (or success).
reused between many different projects and also are now a part of your
David Rager, Cloud Engineer at NREL
CI pipeline. Enterprises can perform security testing as part of their
release cycles, and automatically discover app gaps that drift from the

48
set security policy.
When using cloud formation, RDS names are random characters. If you
Vipin Chamakkala, Principal at Work-Bench
turn on security groups, you can look at the security group names to
identify servers quickly.
David Hofmann, Business Application Manager at Sony Network En-
tertainment International LLC

10 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved
51
To get better visibility and governance on NACLs, create an approved
set of ports and protocols for each subnet in the VPC. With implementa-
tion, use a combination of Lambda and DynamoDB to allow for InfoSec
to authorize ports during architecture review for the app, which places
the app name and port authorized in the DynamoDB table. Lambda
checks it and creates a NACL for that subnet to allow only those ports
between layers. If multiple apps live in the same subnet, use security
groups to ensure that 2-way trust is formed before users are allowed to
communicate with each other on the approved ports. The audit is done
by logging all approval requests against the DynamoDB and the apps
using it against rules used in the security groups.
Jabez Abraham, Cloud Security Architect at Asurion
55
To achieve end to end security with AWS, you must first think about
creating a safety program. It should be organized from the best prac-
tices already available at Amazon Cloud Adoption Framework (CAF).
These practices are the basis for the transformation of the companys
culture, in a shared responsibility model with AWS. Security is seen as
part of the business, and facilitates communication.
Security as code is easily implemented by AWS CodeDeploy. This ser-
vice automates code deployments in any instance, including Amazon
EC2 and executed on site, using the cloud to protect the cloud itself.
This tool is essential for security, since it eliminates the need for manu-
al operations subject to error. In this way, it is possible to deliver more
quickly and safely, and maintain stable operations. Frequent implemen-

52
tation reduces risks.
Always ensure that appropriate profile policies, security groups and NAT Claudio Correa, Product Manager at Ativas
are in place before going live, to prevent access to *all*.
Dave Ade, Manager at Splunk

53 Everyone gets an IAM account. Never log in to the master.

Keith Son, Director of Infrastructure at iNDEMAND

54
Set 2-factor authentication for login to the admin console. Use a sepa-
rate admin account, with very limited access, to manage/delete security/
audit logs.
Gurinder Johar, CTO at Deutsche Bank

11 | The 2017 Insiders Guide to AWS: 55 Best Tips 2016 CloudEndure Ltd. All Rights Reserved
 About CloudEndure CloudEndure is a recognized AWS Advanced Technology Partner

Migration Technology for

Workload Mobility partner in the AWS
CloudEndure provides Cloud Migration and Cloud
Migration Competency Program
Disaster Recovery for any application, allowing com-
panies to mobilize entire applications with their data to Business Continuity and
and across clouds with near-zero downtime and no data Disaster Recovery partner in the
loss. CloudEndure enables truly consistent, block-level, AWS Storage Competency Program
real-time replication using continuous data protection
(CDP). Founded in 2012, CloudEndures Cloud Work-
load Mobility technology creates an exact copy of the
entire application at an alternative cloud location at
the touch of a button, within minutes, and with the
latest data. CloudEndure supports physical, virtualized Discover the ease of migration to AWS and
or cloud-based applications as the source and Amazon AWS-based disaster recovery with CloudEndure.
Web Services (AWS), Google Cloud Platform (GCP),
Microsofts Azure and OpenStack as target cloud loca-
tions. For more information, visit www.CloudEndure.com. CONTACT US