February 10, 2013
When I started this website, I set it up using a virtual hosting provider. Ten years ago, that was pretty much the only cost-effective option for a small scale personal website. I have spent many of the intervening years designing and building enterprise software and cloud services, and the state of the art has changed.
These days, thanks to cloud services like Amazon’s web services, you don’t need to administer a machine just to serve static content. And, since serving static content is the best way to maintain stability under extreme traffic spikes, I have decided to migrate the site to Amazon’s S3 and CloudFront, and use a static file generation blogging engine to publish these posts.
In the remainder of this post, I am going to describe some of the technical details of how I set up this site without using a traditional web host.
Configuring S3 Web Hosting
Basically, I followed Amazon’s instructions for setting up site hosting. Those instructions are pretty thorough and I won’t repeat them all here, but I do want to point out some important points.
You need to create two S3 buckets. One will hold your data and serve the site, and the other is a placeholder for forwarding web requests to the other bucket. The reason you need two buckets is because you want your site to be available at both the naked domain (example.com) and the www subdomain (www.example.com). You can make either bucket/subdomain the master, but there are some subtly different consequences of this choice if you are using Amazon’s DNS service (Route 53) and you want to use CloudFront.1 In either case, you set need to set up the DNS with an “A” record “alias” that points at the naked domain bucket’s web endpoint, and a “CNAME” record that (for now) points at the subdomain bucket’s web endpoint.
CloudFront is Amazon’s content distribution network It acts as an edge cache, serving requests from the location closest to the requester with a cached copy of the document. Configuring your site to use it is optional, but is should make your site faster and better able to serve extreme load, though at an additional cost.
On the whole, CloudFront is pretty easy to set up. Create a “distribution” with a custom origin2 that is set to the web endpoint for your subdomain bucket, configure it to know that it will be attached to the CNAME for your subdomain, and change the CNAME record in your DNS to point at the new CloudFront distribution, instead of the S3 web endpoint.
Since CloudFront is a cache, you may need to invalidate the distribution when you update your site content if you want the changes to be available immediately.
To publish posts, I am using Jekyll for static site generation.
Getting Jekyll installed correctly was straightforward following the instructions, although I did have to update XCode, the XCode command line tools, and RubyGems before the process completed successfully. Once Jekyll is installed, you need to build up the templates and pages you will need to assemble your site. For more information, here’s a good introduction to Jekyll and a simple example walkthrough.
There is a related project to Jekyll, jekyll-s3, which can publish the generated site directly to an S3 bucket and invalidate a CloudFront distribution.
The deployment workflow is not quite the one step process one would like it to be, but it is pretty straightforward. Create a new file in the
_posts directory (or move one there from a draft folder), run
jekyll to regenerate the site, and then run
jekyll-s3 to post it. At some point, I might script some of these steps together as well as set up a process that would allow me to add material when I’m away from my computer, but the current setup is working pretty well.
Getting this set up was a fairly technical process, but the amount of work was about on par with what I originally went through when I set up the site at a virtual host. Most of the work was re-coding the site to work in the templating language that Jekyll uses, rather than the PHP code that I used before. I find that the new template-driven architecture has made the site far easier to maintain and add to. On the other hand, today, there are hosting providers and blogging services that are easier (and less technical) to set up and that look great and have all kinds of additional features right out of the box. If you want the fine grained control of managing your own site, and you are willing to wade through a few technical details, then this is a great way to build a simple, high-performance site.
If you make the naked domain the master and forward the www subdomain requests to it, this configuration will make using CloudFront difficult, since Route 53 “A” record aliases cannot point at CloudFront distributions (while the CNAME record that the subdomain uses can point at anything). If you want to use a non-Amazon DNS service, there are some that can create non-standard configurations that act like an A record alias. ↩
This is important. If you just select your bucket as the origin from the dropdown, then all of your content shows up fine, but 404 errors are not handled correctly. ↩