Rebuilding Radio NZ - Part 10: Going treeless and Modules

One of the biggest time wasters (for us) in Matrix has been navigating the admin section of the site via a tree. Trees can be useful of many tasks, and for seeing the hierarchy of assets (and URLs). But when the number of items in the tree increases above a certain size it becomes harder to get to the one you want.

I should note that Matrix does have a 'simple edit interface' - a way to directly access the editing screen for a particular page - but we've never used it because the site was (and is) growing too fast to set these up for sections and pages. Work is done by site admins (site-wide) or producers (one programme), so the fit was not quite right.

Take Nine To Noon as an example. This programme has 1200 child pages - one for each day’s episode (see image). To get to the current day’s page you have to open the Site node, scroll down, open the station node (National), scroll down and open the programme node (Nine To Noon), wait for the the children to load and display, then scroll down to the bottom of the list. Whew! You can use key-strokes to do all this, but the loading and scrolling time is still high. Moving between programmes is even more unwieldy.

Another example is the audio folders. Nine To Noon currently has over 7,000 audio items. These were once stored in a single folder. When we got to about 3,000 items we had to move everything into dated sub-folders - year, then month, then day. A script was written (now part of the Matrix core command line utils) to move existing items into dated subfolders.

This structure made it simpler to get to items based on their broadcast date, but much slower to move between assets. It also added unnecessary URL segments to the audio path.

We are adding and editing a lot of content every day. Moving around a tree has a very high operational overhead given the large amount of existing content.

In the early stages of the project Nigel (from AbleTech) and I debated using a tree for ELF. Nigel was dead against it, I was less so provided it could be designed well. I agreed to go treeless for the admin interface and to-date one has not been needed. Nice one Nigel!

The biggest problem with abandoning trees though, is the question of what to replace them with. The major navigation pain-point for us is navigating content by broadcast date.

Most developers' initial answer to the date navigation question looks like this:

There is nothing inherently wrong with these date pickers - they are used in a few places in ELF's admin section. Personally I find them too busy, they all work slightly differently, they require a lot of markup and solid engineering to be accessible, and they make you think. Don’t make me think. The solution we came up with involves a sliding date range selector, powered by jQuery and ajax:

Clicking the left or right arrows takes you forward or back. Valid content links are passed into the widget via an ajax call. The dark red button is today, beige buttons are days with content, and the blank buttons have none.

This widget is used right through the admin section of ELF to move between programme episodes, schedule events, and highlights, and is proving both fast and intuitive.

Modules

As the number of asset type increased (Audio, Episodes, Highlights, Schedules Events) we ended up with some duplicated code in our models. These were often slightly different implementations of scopes, coded at different times by different people, all intended to do the same thing, and all tested in slightly different ways. The answer to this problem was to extract the shared functionality into modules.

The first of these modules covered trash. Every trashable asset has a field ‘trashed’, and the module provided consistent methods for trashing and restoring items, and a scope to skip these items in queries.

The second module was for status - live (public), and under construction (hidden). This wraps the published_at field and control visibility.

The third module contained scopes for selecting items based on broadcast date and time - latest, now, between, and for (a specific date).

Having the code in modules simplifies testing and fixed some bugs caused by the subtle differences in the original many implementations. They key lesson is to look for patterns and common code and extract them into modules.

I put the modules into lib, however DHH published a Gist a week ago suggesting their own folder. And if you are not using ActiveSupport::Concern for this sort of thing, you should be. Here is an explanation.

Rails 3.1

Over the last few weeks I have started work on moving the app to Rails 3.1. The feature that is initially of most interest to us is the asset pipeline. DHH first mentioned this in his 2010 Rails Conf Keynote, and this talk was the inspiration for our CSS Views Gem, coded by Koz of Southgate Labs and released on github.

The asset pipeline is a no-brainer. It allows better organisation of CSS and Javascript, and packaging and compression/minification in production. CSS and Javascript are now first-class citizens in the framework  and can be mixed in with Erb, SASS, CoffeeScript or the language parser of your choice.

Having a fast site is important to Radio NZ - I have spent many hours improving markup and streamlining how content was packaged and served - all to great effect. Over the last few years we've reduced page size by 30% and halved client-side rendering time. Doing this in Matrix has been quite hard due to the way CSS and JS content handling is abstracted. You have to get under the hood and hard-code stuff in the Matrix templates to get the highest benefits.

Rails on the other hand is moving towards being 'fast by default'. A selection of sensible and safe best-practices that will work for most use-cases are baked into the framework and turned on by default.

Next time I'll look in more detail at the editing interfaces in the ELF admin section, and compare them with the work-flow in Matrix.