It was quite the busy day at the AWSome Day Roadshow Nordics 2016 in Copenhagen. From the agenda, you can see it’s a full day of talks about Amazon’s products. Starting out with an introduction, a little bit of history and reasons to come up with AWS as a book reseller, Amazon explained how they had to break their monolithic applilcation into smaller services and later decided to provide this to the public as a Cloud solution.
Four main topics were discussed, wrapping their Services into the following four modules:
- AWS Storage and Content Delivery Services
- AWS Compute and Networking Services
- AWS Managed Services and Database
- AWS Deployment & Management
As we progressed with the day I seem to have outlined a few interesting services in my notes. (The main source of information for this post). I will check on them later, but now I am going to commentn on the one-day experince with the Amazon experts.
I am hoping they will be useful to the current workflow and environment setup in the comapny. I did not reflect on the services I was familiar with already, however, I might drop a line here and there, where relevant.
AWS Storage and Content Delivery Services
Lambda
This service was mentioned throughout the whole day. Not sure if the presenter was trying to promote it or it is just that good. Lambda was the answer to a lot of questions.
You can write a piece of code in Python or JavaScript right in the AWS Console (or upload a zip with Java code) and fire it off when you see fit. Now that it is also available to work within your VPC you can execute specific jobs on instances (terminate, tag etc), move files around, backup databases you name it. That’s of course using the aws CLI (since it’s also written for python). Seems pretty powerful!
CloudTrail
This was mentioned vaguely, but its seems to be the service that logs everything that happening on all accounts. Good to have! It’s also saving all the log data on S3.
S3
It’s cheaper compared to EBS volumes. So if we need to store data, better put it on S3, instead of saving it on a volume within your instance. Especially if that data is to be moved to Glacier later or Redshift.
This could also be any static content used in our appliations.
Has versioning (not by default), but might be of help in some cases.
You could run a whole application from S3. Example was a gallery solution where images when uploaded were processed by Lambda, put into the right bucket and so on. I think you can even have a index.html in a bucket and it will run that. Need to try it out.
Snowball
This i sa pretty cool option where they ship you a “briefcase” with 50TB of storage and you download all your data on it, ship it back to them, and it will be available through your AWS Console at some point 🙂 Isn’t that just neat 🙂 Has an inbuild kinde in it, strong encryption and it’s supposedly very durable if you drop it. Way better than shipping your harddrives with FedEx to the other side of the world, because it will be faster than transferring the data over the internet. Yes, a friend told me his company just did that… 🙂
AWS Compute and Networking Services
Here it was mentioned how it is better to use smaller instances and many of them rather than having the so called “battle cruisers” with 64GB RAM and a gazillion cores. If a smaller instances crashes, in the overall higher number of instances in your pool, it will be a much smaller percentage of failure.
Spot instances were touched upon briefly and would be useful if you are doing some batch jobs and you want to cut the costs a little. You bid for them, you win, and you use them for a while.
A nice feature is to use http://269.254.169.254/ internally on each instance, which apparently has meta-data about the instance itself. Haven’t tried it yet, but seems interesting.
Enabling “enhanced network” is available only through the CLI for the moment. At this point I forgot what enhanced network exactly meant, but I believe it’s always good to have it on. It doesn’t cost anything and it can only give you improvements. I believe it is turned on for all instances except the Windows AMIs.
A problem our company had at the time: high network latency
What I got from the presenter on the topic was:
- We should do our own benchmark
- We need to determine whether this is on application level
- or is it on tcp sockets?
- And in fact: how to benchmark this?
Well, the solution was: Well, switch to HTTP from TCP on our ELB listener. More about this can be found here.
I will not go much into detail about this here, even though this was one of the main reasons to attend this conference. It was an old issue long time ago (today is 31 August 2016 as I am writing this).
AWS Managed Services and Database
So, Amazon has AWS Relational database services: RDS.
This makes me wonder, why are we not using RDS for our databases. there’s SQL Server in there that we can use. It is cheaper I believe and this RDS service is across multiple AZ. This means if one Zone fails, we have 2 zones with a backup (currently Ireland that we use has 3 zones)
Also, RDS is not installed on an EC2 instance, where you are isolated to that instance only and one EBS volume attached to it. If that volume fails on that one zone where you have your instance, it’s gone. Not that it is very likely to happen – but still.
And of course there are auto backups with retention up to 35 days.
ElastiCache: memcached or Redis
The same case with RDS, why don’t we use that instead of EC2 instances with the software installed on it. We can use this service.
We can also reduce the load of a database using ElastiCache
Amazon Redshift was mentioned as good for running heavy queries where we have big data stored. Reducing I/O
AWS Deployment & Management
CloudWatch is a great thing, as soon as you have all the metrics you need. Now that I am writing this I already worked with it for a couple of months. You can create great dashboards and have a good overview of what is happening on the “hardware” that you are running. CPU, Network, Latency etc.
To be able to see memory you would have to add this as a custom metric apparently. That was an obstacle. However, it is possible to add custom metrics. That could be metrics from your application level as well. There was something about Lambda functions as well. They are just everywhere.
Cloud formation: creating Stacks from a configuration file. Or maybe that was OpsWorks. At that time of the conference I was already pretty tired.
But there is something in AWS which helps you build and run full stacks of instances ready to run you applications. I guess that would be very useful if you are testing between different setups and in general maybe more useful in your DEV environment. But of course would make sense to use for PROD if you are a pro 😛
Training and Certificaiton
We were given a bunch of links:
- run.quicklab.com
- aws.amazon.com/free
/ Trainings are good and not free
/certification are very good and expensive
AWS Summit – apparently they are really good. you can also get your hands on AWS with AWS gurus to help you out. Workshops maybe as well. Would love to go to those.
Last words
It was an amazing day full of nerd talk. Loved it. Learned a lot and had a good walk through all services that AWS has. Went there to answer a question, came out with even more. There are many other services that we didn’t have time to talk about and I am sure that they are also very useful in many special cases. This was a basic walk through the main services with which you can’t survive in AWS, with a very good Q&A. We (Liu, my colleague, and I) even stayed after the conference to talk to the presenter. Unfortunately, 5 months later when I am finishing this article, I forgot his name, but I’m sure I can dig it out if I try.
Overall: AWSome day!