So I’ve had the YSlow plugin for Firebug installed in my browser for well over a year now, and when I’ve had time, I’ve been trying to learn about site optimization. One of the parts that has always given me the most trouble is how to set up and use a content delivery network (CDN) for serving static files. So as I was setting up this blog, I decided to give it another shot. This time I was successful.
So first, a bit about my setup. I have a self-hosted install of WordPress running on a “private server” at Dreamhost.
Step 1: Set up an account with a CDN provider.
There are a number of CDN providers out there. In the past I’ve used CacheFly, and I’ve heard good things about SimpleCDN, but in the end I decided to use Cloudfront from Amazon, I’ll admit, mostly because of the brand name. Setting up a Cloudfront account was pretty easy. Just provide a credit card number and agree to their terms of service and you’re on your way. Cloudfront has no monthly fees. You only pay for what you use. It’s pretty darn cheap, so there’s not much reason not to do this.
Step 2: Create a “distribution” on Cloudfront.
I’m not by any means an expert on this yet, but the way that Amazon keeps track of your files is by putting them into “distributions.” Each distribution has a “bucket” and URL, and can be assigned multiple alias URLs using CNAMEs. When you create your distribution you’ll be asked to name your bucket, and also to select the CNAMEs (i.e. URLs) that you’d like to associate with the bucket. In my case I named my bucket “morphatic” and created four CNAMEs to go along with it: acf0.morphatic.com, acf1.morphatic.com, acf2.morphatic.com, acf3.morphatic.com. Some people prefer to name their buckets by what’s in them, i.e. images.morphatic.com, js.morphatic.com, etc.
Step 3: Setup your custom CNAMEs at your DNS provider.
Okay, so the goal here is that once we’ve uploaded our files to Cloudfront (more on that in a bit), we’d like them to be available via the URLs we’ve set up, i.e. http://acf0.morphatic.com/myfile.gif. To do that you need to redirect traffic from your subdomain to your Cloudfront domain name. First, log into the AWS Management Console for Cloudfront. Make a note of the domain name of the distribution you just created:
Next, if you host at Dreamhost like I do, log into your control panel, and click on Domains > Manage Domains. Find your domain name in the list and click the DNS link underneath it to take you to the place where you can add custom DNS entries. Fill out the form as shown below to register the CDN domain names:
(Incidentally, after I set this up this way I discovered a much easier way to do it via the Goodies section of Dreamhost’s control panel, but given the steps I’m about to describe below, I’ll probably continue to do it this way.)
Step 4: Copy static files from Dreamhost to Cloudfront.
I followed the very helpful advice of Yejun Yang (who wrote the My CDN plugin for WordPress we’ll intall in a bit) on how to compress the static files and then copy them to Cloudfront. Not all of the software necessary to run Yejun’s scripts were installed by default on my Dreamhost PS account so I had to follow a few substeps:
Step 4A: Create your own local bin folder.
SSH into your Dreamhost PS account. Create a directory for local executables:
$ mkdir local $ mkdir local/bin
You’ll want to set up your shell so it will look in your local bin folder to find your programs BEFORE it looks elsewhere. To do that you’ll want to edit the .bash_profile file that’s in your home directory to look something like this:
# ~/.bash_profile: executed by bash(1) for login shells. export PATH=$HOME/local/bin:$PATH umask 002 PS1='[\h]$ '
Then you’ll need to logout and back into your shell to make the new changes take effect.
Step 4B: Install GNU findutils.
Findutils is a set of augmented features that make the built-in Linux ‘find’ command more powerful. You’ll want to download, unzip and compile the software using the following steps:
$ wget http://ftp.gnu.org/pub/gnu/findutils/findutils-4.4.2.tar.gz $ tar -xzf findutils-4.4.2.tar.gz $ rm findutils-4.4.2.tar.gz $ cd findutils-4.4.2 $ ./configure --prefix=$HOME/local/ $ make $ make install
If all has gone well, you should be able to type “which find” and get a response like /home/username/local/bin.
Step 4C: Install YUI Compressor and create a shortcut script.
$ wget http://yuilibrary.com/downloads/yuicompressor/yuicompressor-2.4.2.zip $ unzip yuicompressor-2.4.2.zip $ rm yuicompressor-2.4.2.zip
To make it easier to use, you can create a script called yuicompressor in your local/bin folder that looks like this:
#!/bin/sh java -jar $HOME/yuicompressor-2.4.2/build/yuicompressor-2.4.2.jar $@
Don’t forget to type
chmod +x yuicompressor to make the script executable.
Step 4D: Install s3sync and create a shortcut script.
S3Sync is a Ruby program that can be used to manage your Amazon S3 file repository. It does a lot of handy things, but in this tutorial we’re only going to use it to transfer files from our Dreamhost PS over to S3. Download and unpack it in your home directory:
$ wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz $ tar -xzf s3sync.tar.gz $ rm s3sync.tar.gz
Now the s3cmd.rb file that we’ll be using to transfer files will be in the
$HOME/s3sync folder. We can create a shortcut script in the
local/bin folder as we did for the YUI Compressor as follows:
#!/bin/sh $HOME/s3sync/s3cmd.rb $@
Save this file as s3cmd and again use
chmod +x s3cmd to make the file executable. You’ll also need to provide your AWS (Amazon Web Services) credentials to s3cmd as follows. First, create a hidden directory in your $HOME directory called .s3conf:
$ mkdir ~/.s3conf
Within the .s3conf folder you need to create a file called s3config.yml that looks as follows:
aws_access_key_id: YOURACCESSKEYID aws_secret_access_key: YOURSECRETACCESSKEY
Hopefully it’s obvious that you need to replace the YOURACCESSKEYID part with your actual key that you can get from your account information in the AWS Management Console. You’ll want to keep these keys private since anyone who has them has access to your S3 server and could use your space!
Step 5: Create a script to compress, zip and move all of our static files to our CDN.
Whew! Okay now that all of the necessary tools are in place we can finally write a script to handle moving all of our files. I created a file that I called ‘cloudfront’ in my local/bin folder and used chmod to give it execute privileges. NOTE: I combined the operations that Yejun Yang had in these two scripts. The content of the file looks like this:
Before you run this script, make sure that you create the $HOME/tmp directory. In order to run this script, just cd into the root folder of your WordPress site and type ‘cloudfront’ at the prompt:
$ cd ~/morphatic.com $ cloudfront
It will take a while for this script to process, so sit back and relax.
Step 6: Install and configure the My CDN plugin for WordPress.
If you’ve been able to follow everything thus far, then you probably don’t need me to tell you how to install and configure the My CDN WordPress plugin.
So I just went through this process yesterday, and I’m still working on some of the remaining optimization issues:
- Some of the static files included by plugins and themes don’t get served by the CDN. I’ll have to dig into the code a bit to figure out why some files do and don’t get included.
- There are other types of static content that could potentially be moved to the CDN such as swf files.
- The CDN needs to be synced with the local files whenever new static files are added, e.g. when new images are uploaded to the uploads folder. There may be a way to trigger this within WordPress. Alternatively, it may be desirable to set up a cron job to handle period syncing of files. Of course, a manual sync is also possible.
I’ll close by posting some of the stats with and without using the My CDN plugin.