Tag Archives: tips

3 Ways to Avoid Google AppEngine Lock-In


Google AppEngine is a Platform-as-a-Service (PaaS) offering that’s been around since April 2008.

It’s very easy to build a web app on it or use it as a backend for a mobile native app helping you focus on the code itself instead of the infrastructure to run it.

AppEngine’s (almost) infinite scalability is gained among other things by using AppEngine in combination with other services such as the DataStore – a schemaless NoSQL docuemnt database built for high performance.

All of these makes AppEngine very compelling and easy to use. However, it comes with a price – vendor lock-in. If you are locked into the various AppEngine platform code and services its going to be very hard to leave.

There are some options such as AppScale – which is an open source implementation of the AppEngine stack but its still limited to running your apps in a certain way – the AppEngine way.

If you are stating a new app or service and are considering AppEngine there are a few tips that can help you migrate off of AppEngine, if you chose to, with relative ease.

1. Choose a Web Platform That Can Run Outside AppEngine Easily

Regardless what language you choose to develop it, AppEngine encourages you to use their Web platform. While in most newer language that HTTP platform is very app agnostic, in some – like Python – its better to use something more robust.

Instead of using WebApp2 you can use Flask to develop on AppEngine. Flask is a very active Web application frameworks with lots of modules and add-ons that do anything from database admin scaffolding to authentication.

If you web platform is platform agnostic you can always run your app outside AppEngine on your own servers (like Google Compute Engine – GCP or Amazon’s Elastic Compute Cloud – EC2)

2. Encapsulate AppEngine or Google only APIs

Try to avoid using AppEngine or Google only APIs. It will be hard to replicate it exactly.

If you have to, make sure to encapsulate the Google API code with your own code so you will only have to update code in very specific places. For example, encapsulate the use of Memcache in AppEngine so you could easily move to your own memcached servers if needed.

3. Avoid Using Google Cloud DataStore

While Google DataStore is a very stable and useful NoSQL schema-less database it’s also AppEngine’s biggest vendor lock in since there is no exact replacement for its API that can be used to replace it.

Instead of using DataStore consider using Google Cloud SQL – Google’s MySQL hosted database. If you like the way data is mapped to objects in Google’s Cloud DataStore you can use various ORM solutions in multiple languages such as SQLAlchemny in Python, Gorm in Go,

Another option would be to host your own database on Google Compute Engine (GCE) like Postgres or MongoDB. You can use the Google Cloud Launcher to have a one click installation of Postgres , MongoDB, Cassandra, etc.

Using a 3rd party common database will allow you to easily migrate off of AppEngine if needed.



SSL Termination for Google Compute Engine (GCE) Load Balancer

I’ve recently been working on moving some apps that I have from Amazon Web Services (AWS) to Google Compute Engine (GCE) to test the service as well as learn the differences.

One of the things that I had to use was SSL termination in the load balancer. AWS’s Elastic Load Balancer (ELB) supports SSL termination on the load balancer side for quite a while now.

Out of the box, GCE’s load balancer does not support SSL termination at the load balancer level, however you can forward TCP port 443 (the ported used by HTTPS) to the instances and have each instance do the SSL termination.

While it will add some extra load on the CPU to decode the encrypted traffic, its a reasonable solution that is relatively easy to deploy via any of the popular web server (Nginx, Apache, etc).



URL Considerations When Using Amazon CloudFront Origin Pull

CloudFront is a great cost effective Content Delivery Network (CDN). When it first started it only supported files located on Amazon’s Simple Storage Service (S3) and on November 2010 Amazon releasedthe “Origin Pull” feature. Origin Pull allows defining a CDN distribution that pull content directing from a preconfigured site (preconfigured hostname) instead of pulling the content from S3.

Cloud Front
by outdoorPDK

The benefits of using the Origin Pull feature includes:

  • No need to sync an S3 bucket with your static resources (CSS, Images, Javascripts)
  • You can serve via the CDN dynamically generated content (like modified images or text fiels) without pre-generating it and putting it inside an S3 bucket.
One of the problems that may occur when intorudcing any caching mechanism is the need to invalidate all or parts of the data. CloudFront provides an invalidation API, however, it has various limitations such as:
  • You need to call it on each object
  • First 1,000 requests are free, each additional one will cost $0.005.
  • It may take up to 15 minutes for the cache to actually clear from all edge locations
There are some techniques to avoid calling the invalidation API but using versioned URLs.

What are versioned URLs?

A versioned URL contain a version part in it, i.e. “http://cdn.example.com/1.0/myimage.jpg”. The version part doesn’t affect the content of the URL, but since the URL to the resource is different, systems using the URL as a key for caching will think of URLs with 2 different version as 2 different resources.
It’s a nice trick to use when you want to quickly invalidate URLs and make a client pull a different/modified version of a resource.

Versioned URLs granularity

You can determine the granularity of the version value to suite your needs. The granularity will allow you to invalidate as little as one file, or every file served via the origin pull in your application.

Common granualirty levels are:
  • A value determined by the build version (i.e. invalidate all static CSS, JS and images one every new build deployed)
  • A value in the configuration, updated automatically or manually to invalidte parts or all of the objects
  • An automatically generated value per file determined by the file content by utilizing a hash function
  • An automatically generated value per file determined by its last modification date

CloudFront will disregard URL query string versioning

Amazon CloudFront (and quite a few other CDN providers) disregard the query string value of a URL (the part after the question mark), whether it is served from an S3 bucket or via an origin pull. This means you will have to rewrite your URLs to contain the version part inside the URL itself. For example:
  • CloudFront will disregard a versioned URL of the following format and consider both URLs the same resource:
    • http://cdn.example.com/css/myfile.css?v123
    • http://cdn.example.com/css/myfile.css?v333
  • CloudFront will consider these 2 URLs 2 different resources:
    • http://cdn.example.com/css/v123/myfile.css
    • http://cdn.example.com/css/v333/myfile.css
You can easily use Apache Rewrite module or Nginx URL rewriting to quickly rewrite the URL http://cdn.example.com/css/v123/myfile.css to http://cdn.example.com/css/myfile.css.
Some common web frameworks put the versioning part in the query string. Be minded about that and change the code appropriately to place the version part somewhere in the URL path (before the question mark).

I would recommend using CloudFront or any other CDN supporting origin pull in any project as it will significantly reduce the loading time of your pages with minimal cost and reduce the load on your servers. It’s a great, quick and easy way to make your site (or even API) work much better.