Rebuilding Radio NZ - Part 1

This is the first in a series of posts explaining how (and why) we are rebuilding www.radionz.co.nz. I'll be examining the technology behind it and looking at some of the difficult choices we've made along the way.

Warning

This post is about a specific site with its own special functional requirements and traffic loads. radionz.co.nz is a public broadcaster's website that includes news, audio and content related to on-air programmes. Traffic loads are very peaky (and high).

This series of posts should NOT taken as advice for or against any particular system. It deals with our specific pain-points and how we are solving them.

You should do your own research and assessment before choosing any CMS or development framework. A good starting point is Strategic Content Management on A List Apart.

Beginnings

Since October 2005 the site has been running on MySource Matrix (now Squiz Matrix). We started the build of the site around Easter that year, meaning we've used the system for six years - not a bad life for any piece of software. We know it very well.

Matrix was chosen after an exhaustive process where we evaluated dozens of web CMSs and called for a Request for Proposal (we got over 40 responses). The project took a year to complete as we had no existing infrastructure or business processes to support the content we wanted to publish.

We ran the site in-house for 3 months to bed in new publishing processes and iron out any bugs.

The primary reasons we chose Matrix was depth and breadth of functionality and the ability to build sites without needing a programming resource.

The previous version of the site was based on a custom built CMS (PHP), and the requirement to have access to a programmer was a constraint we wanted to avoid for the next version of the site. The only custom work was a Matrix asset to support audio content.

With Matrix we were able to quickly build almost anything we could conceive and have it live quite quickly. We also had the ability to try stuff out, modify it, and then release, all without a code editor in sight.

In five years we grew the site from having a rolling one-week back-catalogue of audio content for some programmes, to having over three years of back-content for most programmes. Traffic increased 10-fold.

In late 2009 we started to experience some pain, and by early 2010 we made the decision to move on from Matrix. This was not a decision made quickly or lightly, and it was based on deeply pragmatic reasons that I'll explain in future posts.

The first major reason was increasing difficulty in managing our content - at the time we had about 5,500 individual programme pages (today it's about 7,000). Moving around the site between pages, and the time required to update content was limiting the amount of content we could publish and causing frustration for editors.

The second was performance. We have a lot of content that is updated frequently - a fast moving news story could easily be updated 10 times in a morning - and the system was not able to cope with our requirement to refresh the caches for these pages for every update (more detail on this in part 3), at least not with the hardware we had at our disposal.

You could say that we outgrew the system. Not because there was necessarily anything wrong with it per se, but because our operation had grown in a direction where the system was no longer a good fit. Pragmatic, as I said.

Today (April 2011) our site is running partly in Matrix and partly in the new system (called ELF). This has presented some challenges both in integration (keeping the experience seamless for visitors), migration of content, and in the training of content editors. I'll cover all this in a later post.

In the next post I'll talk about the birth of ELF.

Showing hash differences in Ruby tests

Today I was writing some tests for a Rails plugin I'm working on. The test had to compare a hash output by a method with an expected hash.

This is reasonably standard stuff - when assertion_equal failed it printed out the expected and provided data. The problem in my case was that the hashes have about 50 elements, and I only want the difference to be shown in the test output.

I knocked up this function to solve the problem, and included it in my test class:

def assert_hashes_equal(expected, actual, message=nil)
full_message = build_message(message, "Hashes were not equal, diff was:\n.\n", expected.diff(actual))
assert_block(full_message) { expected == actual }
end

So now I get this:

Hashes were not equal, diff was:
<{:plays_in_category=>"2", :dynamic_end=>"MFT", :percent_back=>"100w"}>.

Note that Hash.diff is part of Rails.

Here is what the output looks like in practice:

1) Failure:
test: SelectorImporterTest Concert Selector Parse Song 0 XML should Hashes should be equal. (SelectorImporterTest)
[/test/unit/selector_importer_test.rb:500:in `assert_hashes_equal'
/test/unit/selector_importer_test.rb:368:in `__bind_1271217771_635209'
vendor/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/context.rb:253:in `call'
vendor/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/context.rb:253:in `test: SelectorImporterTest Concert Selector Parse Song 0 XML should Hashes should be equal. ']:
Hashes were not equal, diff was:
<{:mood=>"3"}>.


The New Accessibility Wave

There is a new accessibility imperative.

You may have heard of it: Mobile.

The mobile user experience is fundamentally different from browsing the web on a desktop grade computer and a standard web browser.

Touch devices do not have a keyboard. You cannot control + click. You cannot right click. You cannot click and drag items. (Usually). Non-touch platforms have other challenges. Some do not support certain plugins.

This is going to require a reworking of the way we build websites.

Design and functionality will have to be layered onto content and structure based on the consuming platform.

The techniques to do this already exist. One is progressive enhancement, but at the moment this is more about adding advanced features for desktop-grade browsers.

It needs to go further than this. Websites have to work for everyone without discrimination. If they don't work, the user's experience of your site will be poor, or non-existent.

The reason I say 'new' accessibility wave is that we've already had to make websites for technology that interacts with pages differently from desktop web-browsers: screen readers.

Many ignored this old wave completely, deeming that market as irrelevant, insignificant or uneconomic.

I have bad news for you. That wave is being joined by a new one - tens of millions of people using non-traditional browsing technology, and growing every day.

If you've not thought about it, it is time to think about inclusive websites again. Websites will have to change to support not only screen readers, but mobile, TV and who knows what else.

Open technologies are going to be a big part of this - HTML5, CSS and Javascript.

If you didn't catch the first wave, you have a lot of catching up to do.

How to Remember CSS Shorcut Order

I have always had trouble remembering the correct order for CSS shortcuts like this:

margin: 3px 4px 2px 3px;

The order is Top, Right, Bottom, Left.

I have just found two ways to remember this.

The first is using the word trouble:

T R o u B L e

The consonants give the order.

The second (pointed out by a colleague this morning) is using a clockface starting at the top and going clockwise.

12 is at the Top
3 is on the Right side
6 is at the Bottom
9 is on the Left.

Speeding up content delivery on the Radio NZ website

We've made a few changes to the way content is served on radionz.co.nz

1. The first is that all static content is now served from a separate domain (static.radionz.net.nz). We are using Nginx for this task because it is faster and more light-weight than Apache.

One benefit is that the browser can make more connections to the site for downloading page elements in parallel.

The second reason we used a completely different domain (instead of a sub-domain such as static.radionz.co.nz) is that it is cookie-less.

The browser does not send any cookies with requests for this static content. If it was on a sub-domain, then any cookies for the domain would be sent with each request. In the case of our home page, this added about 6k to every page request.

One of the NZ sites I tested while working on this has 65K of cookies sent for static assets on the home page. Someone has to pay for that wasted bandwidth.

2. The expiry date of static content is set one year in the future, and assets are marked as Cache-Control: Public.

This tells any application that the content passes through (including the end-user's browser) that the content can be cached, and that it won't expire for a year.

Intermediate caches (like the ones most ISPs have) are more likely to keep a copy of the assets and pass them on instead coming back to our servers. The browser will also check the content in its local cache and only fetch items that have expired.

The net effect is fewer requests for static assets. The makes for less bandwidth used by the client, and less bandwidth paid for by us.

Initial tests show that browsing between two recently visited pages (or pages that are cached locally) is about 100% faster than before the changes. Pages that are not already cached are about 15% faster due to nginx serving files more efficiently and the gzipping of our HTML content. Previously we did not gzip html content because of the relatively high server load, but nginx seems to be able to do this without any impact.

In fact, the overall server load has been reduced by about 10%.

For those who are interested, browse around the site with the Firebug Net tab open and watch how the content loads. You'll notice two things. The first is that subsequent visits to the page only get the main page and the two analytic scripts. The second is that the page fully renders quickly, even though the browser is still getting content. This was achieved by careful placement of the CSS and JS files in the HTML markup.

The site now gets a YSlow 'A' rating (excluding the CDN test, which is not relevant to us).