As you are reading this, Nora and I are returning from Charlottetown, Prince Edward Island, where we attended the 24th annual convention of the Canadian Convention of Southern Baptists. It was the last annual meeting under that name, as messengers approved a name change (to the Canadian National Baptist Convention) and it was also my last official function as Communications Director for the CCSB, a position I held for almost six years.

I came to the CCSB as a 23-year old single fella who thought he knew a lot more than he actually did. Luckily, I was given the opportunity to learn my trade and work on great projects with even better people. I’m particularly proud of our work with e-quip.net, which has expanded to include 15 or so partner sites across North America.

I was also given the freedom to pursue other ventures, most notably my work on SSPAdmin (in 2005) which became SlideShowPro Director in 2006. Since then, Todd Dominey and I have continued to mold Director and SlideShowPro into products that I’m really proud to be a part of. But with continued success came more demands for time in support/development/etc. and toward the end of last year it became clear to me that something had to give. Combined with the monthly travel that my position with the CCSB required, I was burning out pretty quick. So, after months of transitioning to the next Brad, I’m finally a free man and the CCSB is left in good hands. I’ll certainly miss the people at the CCSB, but I won’t miss those 5am wake up calls and four hour flights.

So, what’s next? Well, it’s full-steam ahead working on some really exciting stuff with SlideShowPro and Director (and beyond). Finally being able to devote myself to that full-time is something I’ve been looking forward to for quite some time. Other than that, our personal life will be plenty busy as Nora and I are preparing to move to the US sometime next spring (watch out America, here we come!).

Here’s to tomorrow.

Our initial setup was to pass all PHP requests from nginx back to a pool of PHP FCGIs running on high ports. We use the fair upstream module for nginx that essentially passes the request to the least busy PHP backend. This worked well until things got really busy, when the aforementioned problem would start and my blood pressure would go through the roof.

As always, tailing the log provided a useful wake up call. As the requests flew by, I realized that 95% (maybe more) of the requests were for one of two files: images.php and p.php. These are the two files used by SlideShowPro to communicate with Director. One (images.php) sends back the XML file and the other (p.php) parses all image requests for Director’s on-demand image publishing. Most of the time, these files are simply serving caches back to the client, but because of the sheer number of requests they seemed to be hogging (for lack of a better term) all the PHP backends and not leaving much for the other PHP requests.

The solution was to setup two different pools of PHP FCGIs, one for the images.php and p.php calls and one for everything else. With nginx, this was simple:

upstream main {

    fair;

    server    127.0.0.1:8100;

    # etc...

}

upstream workhorse  {

    fair;

    server   127.0.0.1:8000;

    # etc...

}

So, now we have two separate backends to serve the different requests and we can add more servers to each as need be. Then, we use the nginx location and regex features to split up the requests:

location ~ (images|p)\.php$ {

    fastcgi_pass   workhorse;

}

location ~ .php$ {

    fastcgi_pass   main;

}

Since nginx handles regex location directives in the order they appear in the configuration file, any images.php or p.php file will be captured by the first directive and sent to one of the “workhorse” backends. All other PHP requests are sent to the “main” backends.

The verdict? After days of struggling with uptime (including a few early morning Pingdom text messages), we are now enjoying solid uptime while still churning through approximately 4 million requests a day.

Of particular interest to me was seeing how the new version of SlideShowPro Director fared. Not good. We got an F and it was in the mid 40s at that. Our biggest offense was loading 18 different javascript files and several CSS files as well. Also, we weren’t doing anything to minimize our CSS or JS files. Removing white space, line breaks, etc can save a lot of bandwidth.

Turns out that in the Rails world, there is already a tool to help you do this. It’s called AssetPackager. So, needing that very functionality in CakePHP, I set out to create AssetPackager for CakePHP.

The goal of AssetPackager is to have the best of both worlds. Having separate javascript and CSS files eases development headaches, so we want to maintain the ability to do that. Once in production, we want to combine and compress assets for performance. AssetPackager for CakePHP does this through a Cake console script and a custom helper. This only works with the Cake 1.2.x.x branch, since it relies on the new console.

Installation
Download AssetPackager for CakePHP here. Once you unzip the package, you should find a single “vendors” folder. Place the contents of that folder into either of your CakePHP vendors folders (ROOT/vendors or APP/vendors). That’s it for installation.

Configuration
One important part of any combination/compression strategy is the order in which the files are added. Often times (in both CSS and Javascript) the order that the files are loaded in is extremely important. AssetPackager for CakePHP maintains this order through the use of configuration files.

Throughout this article, I’m going to be using a javascript example but the same technique is used for CSS as well.

First, create a new file in app/webroot/js and name it _asset_config.php. Inside, we will define the different “groups” that we want combined. For example, let’s say I have the following javascript files: main.js, form.js, ajax.js. At runtime, we want all these to be combined to one file and compressed. In app/webroot/js/_asset_config.php we add this:

This defines a group (base) and its members. It also defines the order in which we want the files to be combined. You can add as many groups as you like:

The View
The second step is using the custom helper included with AssetPackager for CakePHP to write our includes into the page. First, make sure your controller has initialized the asset_helper. (I do this in app_controller, since I use it throughout the application).

Now to the view itself. Before, you would have had something like this:

We can now replace that with:

Here’s how it works: If your application is in any DEBUG level other than 0 (thus indicating you are in development mode), the asset_helper will write in the includes one at a time just like you were doing before. However, if you are in DEBUG level 0 (production), the asset_helper looks for a file in your js folder like this: base_1185637913.js. The timestamp ensures that your end users always get the new version, regardless of whether they had the previous version cached.

So, how does that base_1185637913.js file get created?

The Console
The heart of AssetPackager for CakePHP is a Cake console script. First, for a help screen of all the options and commands, enter:

The easiest way to run it is interactively, like so:

Once it has started, it will ask you what assets you would like to combine. Since we are doing just javascript right now, we enter J. It loads your config file, combines all the javascript into one file, then uses JSMin to strip all the whitespace, line breaks, etc out of the file (NOTE: JSMin requires PHP5, if you are on PHP4, AssetPackager for CakePHP will silently skip this step). The finished file is then named using the group name (base in our case) and a timestamp. If multiple versions exist in the js folder, the asset_helper always loads the most recent version. To clean up old versions, just run:

That’s it! Try changing your DEBUG setting to 0 and watch what happens. All your individual includes should now be replaced with one include to your combined, compressed javascript file.

AssetPackager for CakePHP is released under the MIT license, so feel free to modify to your own tastes.

The Why
First, why move in the first place? Moving a website to another host is a pain. Just like a house or apartment, the longer you stay at one host the more embedded you get and the harder it is to move. The SlideShowPro site has been hosted on a shared account at Dreamhost since its inception in early 2005 and includes a variety of components (customer management application, statistical tracking, user forums, etc) all of which use their own database. Moving all that wouldn’t be easy, so there must have been a compelling reason for the move, right?

Dreamhost became the darling of shared web hosts a few years ago and this very blog is hosted there (and will remain there) as a matter of fact. However, over the last year or so, we’ve encountered several setbacks with Dreamhost that included quite a bit of downtime as well as our site getting hacked a few times. In June alone, we had 2 or 3 complete site outages ranging from 3 hours to (in one case) over 12 hours. So, the reason is very simple: Downtime equals money. And if our site goes down for 12 hours, it hurts and at the end of June our sales numbers showed just how much. Turns out that many of the outages were caused by another account on our server getting DOS’d (Denial of Service attack), which Dreamhost remedied by continually changing the IP address of the server which caused DNS lookups to fail.

A non-technical way of putting this: Imagine owning a store. Randomly, a security guard comes by and throws people out of your store, then locks the doors. When will he come back? Well, you have to call the security company, leave a message and they will get back to you. They eventually do, and you find out that the reason your store was closed down was because of what the store next to you was doing. That is very similar to what happens in shared hosting environments.

The Who
We chose Media Temple for a lot of reasons, but mostly because of how impressed we were with their people and the good things we had heard about their Dedicated Virtual system. In a DV setup, the server’s assets are apportioned to you and only you, so you are guaranteed a certain amount of those resources and any other account on the same box will never be able to touch them. You also have full root access, so you can set up the server however you please.

We got the keys to our new server the first week in July and started getting things setup. A big thanks to Chris Lea and the rest of his team at (mt) for providing great support along the way. We even had a little fun trying to track down one of the strangest problems any of us had seen in a while. If it were Dreamhost, something tells me I would still be trying to convince them there was a problem in the first place.

How to minimize downtime
One of the more painful portions of migrating to a new host is the time it takes for the new DNS settings to resolve. A few years ago, this could take 2-3 days and even though it is better now (most of the time less than 24 hours), it still means downtime since some of your users will be visiting the old site while some will be directed to the new one. When everything is built on top of a database (or in our case, databases), that can be a very messy situation.

Once we had the new box up and working with a snapshot of some old data, we were ready to migrate the live data. What we did was take the site down for about 30 minutes just after midnight on Saturday. We used a rewrite rule to make sure any request was routed to a “Be right back” page, ensuring that no purchases, forum posts, or any other data was added or modified. During those 30 minutes we dumped all of our databases to SQL files, transferred them to our new server and used the dump files to populate the new database. Then it was as simple as pointing all of our existing config files to use the new database, that way both sites would be using the same data set and provider. Once that was done, we could change the DNS whenever we wanted. Even if users landed on the old site, it would be using the new database so nothing was stale or lost.

One tip for doing this: Point a domain that you aren’t using for anything to the new server early in the process (we did this as soon as we setup the new server). This will give you an external domain to test with and use to reference the database so that it is not tied to your main domain, which is what makes this work through the DNS switch.

All in all, we couldn’t be happier. The site is snappy and we fell much better knowing that we have full control of our box and will no longer feel the pain of the “bad neighbor” effect of shared hosting. It’s also nice to know that a solid, prompt support team will be ready to help if and when things go south.

Prerequisites
First, you will need to have lighttpd installed, 1.4.12 or above or anything in the newer 1.5.x branch. I’ll be delving into ModMagnet here, so you will need to have lua installed (5.1 or greater) and have lighty compiled –with-lua.

Why ModMagnet?
One of the first things you have to do when moving from Apache to lighty is transition your use of Apache’s fantastic ModRewrite to a suitable replacement in lighty land. The most obvious candidate is lighty’s own rewrite module, but it lacks some of the more advanced features of Apache’s module, notably Apache’s rewrite condition statements.

ModMagnet is a newer module in lighty and aims to give a lot more flexibility to handling, rewriting and forwarding requests. Since we needed to do a lot more than just the standard CakePHP rewrites, ModMagnet fit the bill perfectly.

Let’s look at our lighty configuration. First and foremost, make sure you have ModMagnet in your list of modules to load:

Also, for the purposes of this demo, we have our document root set to the webroot foldler of our Cake application:

Now, we want to tell ModMagnet to attract (get it…it’s a magnet!) any request that comes down the pipe:

This will take any request that is made and pass it directly to the rewrite.lua file, which is placed in our document root.

What a minute…Lua who?
That’s right, ModMagnet uses Lua, a fast, lightweight scripting language that works really well in this environment. You can read more about Lua here.

Writing the Lua Script
Our needs are simple. We want to check if the file requested actually exists on the filesystem. If it does, simply serve up that file without invoking PHP and FastCGI. If the file does not exist, we want to rewrite the request to CakePHP.

That’s it! The lighty.stat() call attempts to stat the file, and if it fails we form the path and query string just like CakePHP likes it. The request then is taken back by lighty and continues to completion. There is a lot you can do here, be sure to check the ModMagnet docs for all the variables available.

The problem with screencasts is that they can be an absolute time-eater. Typically you use a program that records the screen and the input from your microphone at the same time. That’s all well and good, but when you are picky about quality, you end up stopping and starting over at every little “um” and “ah”.

So when I sat down to do my last screencast, I decided to take a different tack and do it in pieces. Here’s how I did it.

First, some prerequisites. You’ll need a Mac with Snapz Pro X, Garageband and Quicktime Pro (or some other program to export the video to its final state, like Squeeze).

Step 1: Capture the video
The frustrating part of trying to do both the audio and video at the same time is that the video is pretty simple, its the audio that you always end up screwing up. With this process, we’ll capture the video on its own first, then use GarageBand to add the voiceover.

So, pop open Snapz Pro X and make sure that you deselect the “Microphone track” box, ensuring you capture just the video. Record your screencast, talking your way through it like you normally would so that your timing will be right when it comes to do the voice over.

When you’re finished, it’s time to export the video. In order to keep the quality pristine through the process, we’ll use the “Apple Animation” codec, which when set to “Best” in the settings menu is lossless. Export the video to somewhere familiar and close out Snapz Pro X.

Step 2: Lay down the audio
GarageBand 3 introduced a nice new feature known as “Movie Scoring”. It allows you to import a video, lay down some audio beneath it and then export it out as a video file. We’ll use this to add the voiceover to our screencast.

Create a new project in GarageBand and select “New Movie Score”. Give it a name and you are presented with a new GB project window. The first track is for the video, so just find the file on your Desktop (or wherever you put it) and drag it on to the top track. Now you need to add a track for your audio, so click Track->New Track from the top menu. You’ll want a “Real Instrument” track for your voiceover.

Now, here’s something that caught me up. GB uses its own setting for what input to use, not really caring about the setting you have in the Sound System Preferences pane. So, go to the GarageBand preferences pane and select the Audio/MIDI header. Make sure your input of choice is selected for the Audio Input.

Now you can start to play the video and record audio at will. Once you have a section the way you want, trim up its ends and start on the next one. GB is really great for audio editing like this, so there are lots of things you can do. Once your happy with it, time to export it.

Step 3: Finishing
One disappointing thing about GB is that it doesn’t allow you to export video using customized settings. So, we’ll export it as is from GB and use Quicktime Pro to export it to its final state.

Go back to GB’s preferences and this time select the Export tab. Set both Audio and Movie settings to their highest values (Higher Quality and Full Quality, respectfully). Now go to Share->Save Podcast to Disk. Again, save it somewhere familiar, you’ll need it in a second.

Now just open that file in QT Pro, or any other video compressor, and export it with its final settings. I use MPEG-4 set to High, Key Frames and Data Rate Automatic. For Audio, I use AAC 22.050 stereo. I’d also suggest using Fast Start - Compressed under “Prepare for Internet Streaming” if you are using a Quicktime based codec.

That’t it, and you couldn’t ask for anything better. You get a better quality product and it actually saves time. Here’s an example of a screencast I did with this technique: Director Preview Video 2.

Love you and Happy Birthday!

On to the picks:

The left side of the bracket (Midwest and West regions) will hold true, with Kansas and Florida arriving in Atlanta. Some big upsets loom in these regions, most notably the Duke-VCU matchup. When the pairings were announced Sunday, VCU’s fan based increased 1000-fold instantly.

The right side of the bracket (East and South regions) will be a Big 12 statement. Fourth seed Texas and third seed Texas A&M will both take out the number one’s in their bracket on the way to Hotlanta. Upset specials? Creighton over Nevada and Texas Tech over Boston College.

In Atlanta, Florida and Kansas will play a tight one, with the Jayhawks coming out on top (like they did in this year’s earlier matchup of the two teams). Texas and Texas A&M will also be a dandy, with Texas pulling through and setting up a rematch of two of the best games of the year: Kansas vs. Texas.

Kansas 88 - Texas 85. Rock Chalk!