Inaccurate Design

Migrating from Jekyll to Hugo and Lambda

Sunday, 23 August 2015

When AWS released a preview of their newest service, Lambda, there was confusion around what the potential use cases for the service were. Essentially the service is designed to facilitate bursts of relatively fast, non-continuous processing - thumbnail generation for images uploaded to an S3 bucket, or in my use case, generation of static blog posts when I upload a new one.

One of the things that’s stopped me blogging more than I currently do (which is a pretty low bar), is it seems as though Jekyll is broken again. That’s probably a bit unfair on Jekyll, but trying to manage Ruby dependencies can be a pain. While there are solutions to manage separate projects (rbenv being my preferred), because I didn’t use it all the time it was just another impediment. Then I saw someone on HN recommend Hugo, a static blog generator written in Go. While I haven’t written anything in Go before, I’ve heard good things about it, particularly related to speed, and the ability to create a single binary without other dependencies.

This makes it a great candidate for static blog generation with Lambda. The basic workflow is this:

  1. I upload a new markdown-formatted post to the input S3 bucket
  2. Lambda is triggered by the upload of a new file
  3. Node.js wrapper and aws-cli pull content from the input bucket into the Lambda environment
  4. Lambda function runs the Hugo executable to generate static content
  5. Node.js wrapper outputs static content to output S3 bucket
  6. Output bucket is served up by CloudFront (optional, you can host out of S3 but you don’t get much control over hosting configuration - in particular, SSL)

basic hugo/lambda workflow

Luckily for me, a bunch of this setup work has already been automated over at the Hugo-lambda project. Unfortunately for me, I was having some dependency issues with kappa, so decided to take the manual way out and set it up by hand, using the install scripts as a guide.

The transition from Jekyll to Hugo is pretty smooth - once I sorted out syntax highlighting (none),themes (slight syntax changes in the templating system), and ugly URLs (S3 won’t serve an index file from a folder except for the root of the bucket, so you have to configure your blogging system to use full URLs ) it was no more complicated than figuring out how the new system worked.

And it works great!

Things I still want to work out:

  • Currently I use a script and aws-cli to invalidate the the index page when I upload a new post. Ideally this can be incorporated in the Lambda task (and invalidate any other updated files like category pages or CSS pages)
  • Markdown posts still need to be uploaded to the input S3 bucket. In my ideal world I can drop the post somewhere (like a folder on Dropbox) and it gets uploaded.
  • Formatting in some code sections got clobbered, and I need to go back through and fix it!