In a recent post on the AWS blog, Jeff Barr and Matt Wood, showed the architecture and code they wrote which lists the most interesting AWS related jobs from the Amazon Jobs site.
It serves as a rather good example of how service components such as the ones AWS provides (SNS, SQS, S3 to name a few that are AWS agnostic) a great set of building blocks that can easily help you focus on writing the code you really need to write.
I found the auto scaling policy for spinning up & down machines just to tweet a bit of an over kill at first (and Jeff could have easily added the code on the same instance running the cron), however thinking about it a bit more and considering the various pricing strategies it actually makes a lot sense.
When talking about the cloud, most people talk about running the servers on the cloud. They talk about the fact that they can start or stop virtual servers with a simple API call.
A lot of Cloud Providers do provide you with virtual servers, virtual load balancer and infinite (and/or scalable) storage, but these are all building block of servers. You still need to do the heavy lifting of taking a bunch of servers and making it do the work for you. You need to install the software (not just the one you wrote), mange, handling disaster recovery, backups, logging, etc.
What makes Amazon’s cloud unique on top of the servers and storage they provide is the fact that it provides a set of cloud services that can be used as black boxes for your application/service reducing the code you need to write/maintain as well as the number of servers you need to administer. These services are also available for use outside of Amazon’s cloud but there are benefits of using them within your EC2 instances.
The Amazon non-hardware cloud services offering is split into two categories:
Off the shelf services – These are preconfigured services that Amazon takes most of the hassle off of managing it. Such services are
Amazon ElastiCache – Hosted Memcached which Amazon manages, allows to resize (add or remove servers) as well as patch with the latest software.
Development Building Blocks (Black Boxes) – Web services that provides functionality which you can mix and match to create you service, removing the need for you to handle the load, machines and configuration of these services. Such services are:
Amazon SimpleDB – an Amazon written key/value datastore that is hosted and operated by Amazon. Scalable and simple. It just works
Amazon Simple Notification Service (SNS) – A web service to send notifications to people/machines/service/etc. It allows to send the notification out as an Email or as an HTTP request as well as post it to an SQS queue
What I love the most about Amazon Web Services is the fact that when they do provide a certain Development Building Block such as Simple Email Service (SES), they do so without killing or harming the competition. There is still enough value and features in other Email services such as mailgun, SendGrid and MailChimp for them to co-exist with SES.
Not stepping (too much) on web services developers toes is not something to dismiss and I would love to see the innovation that comes out of Cloud based web services in the future.