A Story About Stories

This is a piece I performed for the Northcote Storyteller\’s Club today. It\’s been an odd day, preceded by an even odder night – but that\’s another story.

I hope you enjoy it, whoever you are. The first part is a long and rambling semi-justification for the story that follows. It\’s based on the recent crash-landing of US Airways flight 1549 into the icy Hudson River in New York City.

Northcote Storytellers Gig 2009-01-18

I\’m an avid consumer of the news. I consume the news in the same way some people smoke cigarettes. If I have a moment of boredom, I take a long slow drag on international politics and forget about my own troubles for a while. And that pretty much tells you how relevant the informational component of modern news is to our own lives.

But there\’s something vaguely unsatisfying about it. Something in the cruelty and randomness of reality that makes news stories less like a long slow drink of water and more like a shot of tequila. News is not contemplative. Journalist\’s attempts to draw lessons from it are straightjacketed by the facts. The universality of the tale is quashed because you can\’t legitimately retell it in each culture\’s own context. If George W. Bush says something horrible that highlights a flaw in the modern democratic systems, and Australians journalists reported it as if Kevin Rudd had said that, there would be hell to pay.

So in order to make these global “important” stories work everywhere they have to be amped up for each market and demographic by swirling computer images and aggravated talk show pundits. The importance and universality of these stories is painstakingly carved by news reporters out of a reality that is far more arbitrary, cruel and random than anyone would like to admit. And the end result is that we\’re left feeling a little bit confused and powerless, where a really good yarn might show us how to apply a lesson in our own lives.

So as the bulk of our stories have moved from the exaggeration and customisation of a thousand retellings in a hundred languages to the shoehorning of facts into a compelling narrative that you can maybe sorta kinda draw a lesson from, has anything been lost?

Uh, yes. Look at what happened to the Bible when it went from stories to news.

What I find absolutely fascinating about the Bible is that clearly it went through decades or centuries of oral retellings and written versions, along the way making Christianity one of the world\’s more powerful and interesting and relevant religions, until someone decided there ought to be a canonical version. “That\’s about fantastical enough! Leave in the bit about the fish, take out the bit about turning a little girl\’s ears into legs, and let\’s call it version 1 point oh. Matthew, Mark, Luke and John. The Gospel of Ted is too saucy, The Gospel of Steve tells people to ignore the Romans and concentrate on building really nice gardens, and Joanna\’s gospel has all those tacky rhyming couplets that will be a bitch to translate.”

Incidentally, the way they selected which Gospels to include way more stupid and arbitrary than that: A Greek guy called Irenaeus, tasked around 150 AD with creating a canonical Bible, decided they should select four gospels from the many available, because there are four corners of the earth, four winds, and four beasts of the apocalypse. The first is completely wrong, the second is a useless simplification, and the third won\’t matter until 2011. What an entirely appropriate way to turn an oral tradition into canon, and thus fiction into fact.

And so it was pretty much at that point that the Bible started becoming increasingly boring and irrelevant, and began being enforced by the sword as much as the word. Without being allowed to fundamentally alter in the retelling, it became increasingly difficult for the Bible stories to stay relevant and exciting. Who\’s to say that if the Bible had been allowed to evolve as much in its later years as it did in the earlier ones, we not might have had Jesus riding dirt-bikes and making base-jumping to rescue a box of kittens from a runaway speedboat?

So what I want to start today\’s news down the road, to retell it in a style that gives it a little room to breathe and grow and come to life. And, since most of the news we hear is so far away and disconnected from our own lives anyway, I reckon this makes the news more accurate and useful.

The Hungry Dragon

There once was a hungry dragon. He ate and ate and ate. He ate the earth to fill his belly, and he ate the air to fill his lungs, but no matter what he ate, the dragon was still hungry.

Every morning at 8am the dragon would taxi out of his cave, his tummy rumbling, his mouths yawning, his tail roaring and his wingtips blinking, for he was hideously deformed and everything was in the wrong place. And he would summon the local chieftains.

And the dragon would ask the chieftains to whisper in his ear and tell him where to fly, for the dragon was blind from his many deformities.

This had been happening for many years. When they were young, the chieftains would fly on the dragon for the sheer joy of being up in the air and surveying their beautiful village. The dragon was young and lithe. He would swoop high in the air and low to the ground. The chieftains would describe for him the world that lay below, the peasants waving from their sunlit fields of corn and orchards of fruit, the church steeple that seemed so high from the ground but so small from the sky, and in his mind the dragon could picture it all, and they were happy.

One day, the wealthiest villagers approached the chieftains and asked if they could climb on the dragons back and fly across the land so they could sell their wares to distant villages.

At first, the chieftains were taken aback. “It\’s just for fun”, or “It\’s too dangerous”, they would say to one rich man. “It would take too much food, and would make the dragon tired”, they told another.

But over time, the chieftains grew bored, and they grew greedy, and they got mortgages just before the market dipped, and they agreed to take just one or two of the wealthy villagers on the hungry dragon\’s back in return for some gold.

This went on for many years, and as time went by and the village grew larger, more and more villagers wanted to climb on the dragon\’s back, and the chieftans wanted more and more money.

To convince the dragon to fly ever further, they had to feed more earth into his belly , and more air into his lungs. The dragon grew larger and more deformed, and the chieftains wore silly hats and name tags and stayed in exclusive lounges with free filter coffee and 148 cable channels wherever they went, and the back of his head was painted white and PG-rated films were projected onto it, and some of the wealthiest and most self-important villagers paid ten times as much to sit near the front in a different coloured seat with a cup-holder, and the dragon got fatter and hungrier and fatter and hungrier.

Until one day, much like the previous day and the one before that, at 8am the dragon taxi\’d out of his cave. The chieftains climbed onto his neck and whispered in his ear while the wealthy villagers snuck onto his back and flicked absent-mindedly through a dog-eared copy of yesterday\’s Wall Street Journal.

And once the chieftains had filled his belly with earth, they ordered the dragon up into the sky to fill his lungs with air.

On this day the dragon was hungrier than ever, and as he rose above the earth he smelled something. It was more delicious than earth, and more substantial than air, and without even a single thought he dove straight for it. It was a flock of geese.

The dragon opened his roaring mouths and filled them with delicious geese. He filled them and filled them until they stuck in his throat and he lost consciousness and he fell towards the earth.

At this point, nothing the chieftains could do made any difference. They shouted in his ear and pulled at his wings but the most they could manage to do was turn him away from the earth and towards a wide but cold river.

With a terrifying splash the dragon dove into the icy water. It filled his nostrils and his throat, washing away the flock of geese and decades of earth and his mind regained consciousness in a snap as his body experienced a completely new sensation. And for the very first time in his life, the dragon felt completely at peace, and he bobbed in the river, slowly sinking down and filling himself with the icy water as they wealthy villagers scrambled from his back.

And before he finally drowned, the hungry dragon had one last, long, slow, satisfied thought:

“Oh! That was the problem! I was thirsty!”

Sending HTML Email with Wicket part II: Converting links

In my previous post, I showed how you can use Wicket\’s HTML rendering engine to render HTML emails by faking a request/response cycle.

In this post, I\’ll show you how to use an IVisitor to change image and anchor URLs to be absolute instead of relative. This is absolutely essential in order to make your HTML email work – otherwise all your images can\’t be found, and your links point to your own mail server.

The trick is to use Wicket\’s IVisitor to add a TransformerBehaviour to all the Images and Links that uses a regex to transform the URL after render but before the page is returned.

The code for the IVisitor is below:

private final class RelativeToAbsoluteUrlVisitor implements IVisitor {
        private final String requestPath;
        private Pattern urlPattern;
        private Class<? extends Component> componentClass;

        private RelativeToAbsoluteUrlVisitor(String requestPath, Class<? extends Component> componentClass, String attributeName) {
            this.requestPath = requestPath;
            this.componentClass = componentClass;
            urlPattern = Pattern.compile(attributeName+"="(.*?)"");
        }

        public Object component(Component component) {
            //if this component is of the specified class, update the URL attribute to be absolute instead of relative
            if(componentClass.isInstance(component)) {
                component.add(new AbstractTransformerBehavior() {

                    @Override
                    public CharSequence transform(Component component,
                            CharSequence output) throws Exception {
                        log.warn("Transforming component output: "+output);

                        Matcher m = urlPattern.matcher(output);

                        if(m.find()){
                            String attributeValue = m.group(1);
                            int start = m.start(1);
                            int end = m.end(1);

                            //convert relative to absolute URL
                            String absolutePath = RequestUtils.toAbsolutePath(requestPath, attributeValue);

                            log.warn("Got absolute path \'"+absolutePath+"\' from relative path \'"+attributeValue+"\'");

                            //construct a new string with the absolute URL
                            String strOutput = String.valueOf(output);
                            String finalOutput = strOutput.substring(0, start)+absolutePath+strOutput.substring(end);

                            log.warn("Returning updated component: \'"+finalOutput+"\'");

                            return finalOutput;
                        }

                        return output;
                    }});
            }
            return IVisitor.CONTINUE_TRAVERSAL;
        }
    }

Then we override the onBeforeRender() routine to traverse the component hierarchy and add this behaviour to the appropriate elements. Note that I haven\’t shown how you get the current absolute request URL, as in my system this is proprietary. There\’s plenty of example code floating around on how to do that, anyway.

    protected void onBeforeRender() {
        super.onBeforeRender();

        final String requestPath = MyCustomWebRequestCycle.get().getCurrentUrlAsString();

        IVisitor imageVisitor = new RelativeToAbsoluteUrlVisitor(requestPath, Image.class, "src");
        IVisitor anchorVisitor = new RelativeToAbsoluteUrlVisitor(requestPath, Link.class, "href");

        visitChildren(Image.class, imageVisitor);
        visitChildren(Link.class, anchorVisitor);
    }

So there you have it! All the bits and pieces to create HTML email with Wicket. There\’s one more catch though: You have to generate these emails in the same process as the Wicket Application. Calling Application.get() outside of the main process results in an error. In my system, I get around this by generating the HTML email source every time the user saves my Newsletter bean, which means that when it\’s finally sent (in the background), it just sends the pre-generated HTML. Easy!


	

Render a Wicket page to a string for HTML email

Something that\’s very desirable to do in Apache Wicket is create HTML emails using Wicket\’s brilliant component-oriented markup.

I\’ve been working on this problem on and off for ages — it\’s tricky because of teh way that markup rendering is so deeply tied to the requestcycle, which in turn is deeply dependent on the httpservletrequest — with good reason, too. That\’s where Wicket gets its autoconfiguring magic from!

So in order to use Wicket to create HTML emails, we need to fake the request/response cycle. I wrote this convenient method that renders a bookmarkable page (pageclass + pageparameters) to a string:

protected String renderPage(Class<? extends Page> pageClass, PageParameters pageParameters) {

        //get the servlet context
        WebApplication application = (WebApplication) WebApplication.get();

        ServletContext context = application.getServletContext();

        //fake a request/response cycle
        MockHttpSession servletSession = new MockHttpSession(context);
        servletSession.setTemporary(true);

        MockHttpServletRequest servletRequest = new MockHttpServletRequest(
                application, servletSession, context);
        MockHttpServletResponse servletResponse = new MockHttpServletResponse(
                servletRequest);

        //initialize request and response
        servletRequest.initialize();
        servletResponse.initialize();

        WebRequest webRequest = new WebRequest(servletRequest);

        BufferedWebResponse webResponse = new BufferedWebResponse(servletResponse);
        webResponse.setAjax(true);

        WebRequestCycle requestCycle = new WebRequestCycle(
                application, webRequest, webResponse);

        requestCycle.setRequestTarget(new BookmarkablePageRequestTarget(pageClass, pageParameters));

        try {
            requestCycle.request();

            log.warn("Response after request: "+webResponse.toString());

            if (requestCycle.wasHandled() == false) {
                requestCycle.setRequestTarget(new WebErrorCodeResponseTarget(
                        HttpServletResponse.SC_NOT_FOUND));
            }
            requestCycle.detach();

        } finally {
            requestCycle.getResponse().close();
        }

        return webResponse.toString();
    }

One other thing that\’s desirable to do is change all relative links in the email to absolute URLs — something that Wicket makes super-easy, if you know how. That will be the subject of my next post.

Free SMS service notifications using Google Calendar

Today I had a small revelation.

I was wracking my brains trying to figure out the SMS messaging provider to use to send myself service outage notifications for my clients\’ web sites. Given that I have just a handful of clients so far, it makes no sense to use a provider that requires a minimum monthly or yearly spend.

Ideally of course, I\’d like to spend nothing at all, and in exasperation I finally threw my hands in the air (they\’re detachable) and whined: “Google sends SMS\’s for free – why is it so hard for everyone else?”

(answer: not everyone has billions of dollars)

And then came the revelation: Why not create a command-line tool that uses Google\’s Calendar API to create events 6 minutes in the future that have an SMS notification set for 5 minutes prior to launch? That way, within a minute you get a notification sent to your phone for free within 1 minute. Sweet!

So, here\’s the code (it\’s in Java… sorry)

/**
* Simple command-line notification command that uses Google Calendar ATOM API to create
* a single event 6 minutes in the future with a 5 minute SMS reminder
*
* @author Daniel Walmsley
*
*/

import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Date;
import java.util.List;

import com.google.gdata.client.calendar.CalendarService;
import com.google.gdata.data.DateTime;
import com.google.gdata.data.PlainTextConstruct;
import com.google.gdata.data.calendar.CalendarEntry;
import com.google.gdata.data.calendar.CalendarEventEntry;
import com.google.gdata.data.calendar.CalendarFeed;
import com.google.gdata.data.extensions.Reminder;
import com.google.gdata.data.extensions.When;
import com.google.gdata.data.extensions.Reminder.Method;
import com.google.gdata.util.AuthenticationException;
import com.google.gdata.util.ServiceException;

/**
* This is a test template
*/

public class GCalNotifier {

public static void main(String[] args) {

    /**
     * Command line args: 
     * 
     * username
     * password
     * calendar name (e.g. "Notifications")
     * TimeZone offset (in hours)
     * event start offset (in minutes)
     * event end offset (in minutes)
     * title
     * description
     */

    try {

        // Create a new Calendar service
        CalendarService myService = new CalendarService("GCal Event Notifier");
        myService.setUserCredentials(args[0], args[1]);

        String calendarName = args[2];
        Long tzOffset = new Double(Double.parseDouble(args[3])).longValue() * 60 * 60 * 1000;
        Long startOffset = new Integer(Integer.parseInt(args[4])).longValue() * 60 * 1000;
        Long endOffset = new Integer(Integer.parseInt(args[5])).longValue() * 60 * 1000;
        String title = args[6];
        String description = args[7];

        // Get a list of all entries
        URL metafeedUrl = new URL(
                "http://www.google.com/calendar/feeds/default/allcalendars/full");
        System.out.println("Getting Calendar entries...n");
        CalendarFeed resultFeed = myService.getFeed(metafeedUrl,
                CalendarFeed.class);
        List<calendarentry> entries = resultFeed.getEntries();
        for (int i = 0; i < entries.size(); i++) {
            CalendarEntry entry = entries.get(i);
            String currCalendarName = entry.getTitle().getPlainText();
            System.out.println("t" + currCalendarName);

            if (currCalendarName.equals(calendarName)) {
                sendDowntimeAlert(myService, entry,
                        title, description, startOffset, endOffset, tzOffset);
            }
        }
        System.out.println("nTotal Entries: " + entries.size());

    } catch (AuthenticationException e) {
        e.printStackTrace();
    } catch (MalformedURLException e) {
        e.printStackTrace();
    } catch (ServiceException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

private static void sendDowntimeAlert(CalendarService myService,
        CalendarEntry entry, String title, String description, Long startOffset, Long endOffset, Long tzOffset) throws IOException,
        ServiceException {

    String postUrlString = entry.getLink("alternate", "application/atom+xml").getHref();

    URL postUrl = new URL(postUrlString);//was: "http://www.google.com/calendar/feeds/jo@gmail.com/private/full"

    CalendarEventEntry myEntry = new CalendarEventEntry();

    myEntry.setTitle(new PlainTextConstruct(title));
    myEntry.setContent(new PlainTextConstruct(description));

    Date now = new Date();

    Date startDate = new Date(now.getTime()+startOffset);
    Date endDate = new Date(now.getTime()+endOffset);

    DateTime startTime = new DateTime(startDate.getTime()+tzOffset);

    DateTime endTime = new DateTime(endDate.getTime()+tzOffset);

    When eventTimes = new When();
    eventTimes.setStartTime(startTime);
    eventTimes.setEndTime(endTime);
    myEntry.addTime(eventTimes);

    // Send the request and receive the response:
    CalendarEventEntry insertedEntry = myService.insert(postUrl, myEntry);
    System.err.println("Got response for: "+insertedEntry.getTitle().getPlainText());
    for(When when : insertedEntry.getTimes()) {
        System.err.println("When: "+when.getStartTime()+" to "+when.getEndTime());
    }

    //5 minute reminder
    Reminder reminder = new Reminder();
    reminder.setMinutes(5);
    reminder.setMethod(Method.SMS);
    insertedEntry.getReminder().add(reminder);
    insertedEntry.update();
}

}

Don\’t forget, you\’ll need to download the Google Data APIs and put their JARs in your classpath before this will work!

Personally I use this with Nagios. I always use the same args for the calendar offsets, so I\’ve encapsulated most of my settings (except title and body) in a script.

!/bin/sh

export SCRIPTDIR=/opt/calAlert
export USERNAME=username@gmail.com
export PW=mySecurePassword
export CAL=Notifications
export TZOFFSET=10
export STARTOFFSET=7
export ENDOFFSET=12
export TITLE=$1
export BODY=$2

export CURRDIR=pwd

export CLASSPATH=”${SCRIPTDIR}/calAlert.jar”

assumes GData libs are in “libs” subdirectory of SCRIPTDIR

for jarfile in $(ls “${SCRIPTDIR}/lib”)
do
CLASSPATH=”${CLASSPATH}:${SCRIPTDIR}/lib/${jarfile}”
echo lib/${jarfile}
done

echo “CLASSPATH=${CLASSPATH}”

export CLASSPATH

java GCalNotifier ${USERNAME} ${PW} ${CAL} ${TZOFFSET} ${STARTOFFSET} ${ENDOFFSET} “${TITLE}” “${BODY}”

Training Junk Mail filter using Apple Mail and GMail IMAP Connector

Like most people, I get literally thousands of spam messages a month. I never see them, of course, because they\’re filtered out by Gmail\’s incredible spam filtering system.

This is all well and good if you\’ve got a Gmail account, but what if you want Google-quality filtering for your business or other mail accounts?

Well, thanks to Gmail IMAP support, we now have a massive and incredibly accurate data set for training Apple Mail\’s junk filter. Gmail IMAP lets you browse folders other than your inbox. So once you\’ve added it to your Apple Mail account list, simply browse to the Spam folder, select all, then mark those messages As Junk.

This simple act will train Mail\’s bayesian spam filtering system on everything in your GMail spam folder, no doubt instantly improving its performance somewhat. In my case, the spam folder usually has 3000-4000 messages in it, which is fairly hefty and it would certainly take a lot of clicking to get that kind of data by hand.

Thanks Google!