I've been hacking on node.js for a week now. I won't go into why I think it's awesome, you probably already know (thanks to bloggers like Simon Willison).

My second raw "hello world" speed test went something like this:

# node.js on freenode
spoob: technoweenie; seriously, you should look up how fast nodejs is... :)
technoweenie: yea i was getting about 5k r/s, pretty impressive
spoob: you should be getting around 20k r/s?
technoweenie: really?
technoweenie: oh wait i only ran 5k requests

My first:

# twitter
technoweenie: sample node.js server is *extremely* slow, am i missing something? i'm just trying the example app on nodejs.org  
technoweenie: oh i see, the demo app sets a 2s timeout, haha  
lifo: classic
technoweenie: hey that's a great way to start off a new web framework, simulate rails cgi speeds

sources: 1, 2, 3, 4

With that out of the way, I started hacking on Where's Waldo (taking a detour to bang out a quick test framework along the way). Where's Waldo is a throw-away prototype using Redis for tracking locations of users. Here's my first pass at tests:

// Github: technoweenie/wheres-waldo
// SHA: fa925fe483dac9a02e374971fe392c7e00f1e5d1
// http://github.com/technoweenie/wheres-waldo/blob/fa925fe483dac9a02e374971fe392c7e00f1e5d1/lib/index.js
// this source code has been modified to fit my blog post
function WheresWaldo(redis, prefix) {
  this.track = function(user, location, ttl) {
    this.redis.set(this.prefix + ":" + user, location).wait()
    this.redis.set(this.prefix + ":" + location + ":" + user, user).wait()
  }

  this.locate = function(user) {
    return this.redis.get(prefix + ":" + user).wait()
  }

  this.list = function(location) {
    var locationKey = this.prefix + ":" + location
    var users = this.redis.keys(locationKey + ":*").wait()
    // return all users that aren't blank strings
    return _.reduce(users, [], function(users, user) {
      if(user && user.length > 0)
        users.push(user.substr(locationKey.length+1, user.length))
      return users;
    })
  }
}

// http://github.com/technoweenie/wheres-waldo/blob/fa925fe483dac9a02e374971fe392c7e00f1e5d1/test/waldo_test.js
describe("tracking a user")
  before(function() {
    this.waldo = whereswaldo.create(redis, 'tracking');
    this.waldo.track('bob', 'gym')
  })

  it("tracks a user's location", function() {
    assert.equal('gym', this.waldo.locate('bob'))
  })

  it("lists the user in that location", function() {
    assert.equal('bob', this.waldo.list('gym')[0])
  })

One of the Node.js goals is to never introduce a blocking api. There aren't a lot of libraries yet, but the ones that exist are fully asynchronous. Even a super-fast database like Redis has an async node.js wrapper.

A simple Redis GET command doesn't return a value, it returns a Promise. A Promise is a really basic event emitter with just two events: success and error. Ideally, you'd take this promise, listen for the success and error events, and move on to the next request. When that Redis query comes back, it emits the success event with the result, and any callbacks are run.

If you look at my locate() method, you'll see that I called Promise#wait so that I didn't have to worry about that yet. It's a convenient tactic for node.js newbies, but I would not recommend that you continue to do this. I started off with a familiar synchronous lib that I could test. Once my tests were green, I was free to experiment with these wild new promise objects.

function WheresWaldo(redis, prefix) {
  this.locate = function(user) {
    return this.redis.get(this.prefix + ":" + user)
  }
// ...

// related test
it("tracks a user's location", function() {
  assert.equal('gym', this.waldo.locate('bob').wait())
})

See, promises are easy! To make that locate() method asynchronous, I simply returned the same promise that the Redis client's get method returns. Basically, I moved the wait() call from the library to the test.

function WheresWaldo(redis, prefix) {
  this.list = function(location) {
    var locationKey = this.prefix + ":" + location,
            promise = new process.Promise();
    this.redis.keys(locationKey + ":*") 
      .addCallback(function(keys) {
        var users = _.reduce(keys, [], function(users, key) {
          if(key && key.length > 0)
            users.push(key.substr(locationKey.length+1, key.length))
          return users;
        })
        promise.emitSuccess(users);
      })
      .addErrback(function() {
        promise.emitError();
      })
    return promise;
  }
}

The list() method was a bit more complicated. This time, WheresWaldo creates its own promise object to return. It adds its own callbacks to the promise from the Redis client's keys() method. From that success callback, it filters the keys array as desired, and emits the success event of its promise.

The final gotcha was handling multiple fire-and-forget queries. The track() method sets two Redis values. Personally, I don't have a preference which order they run in. There were three options that I could see:

1. Call wait() on the first one before firing the second one. Synchronous calls are bad!
2. Nest the second call in the success callback of the first call's promise. Nesting is ugly!
3. Fire both queries and let Redis do its job. Good, but I want to return just one Promise.

I went with #3, and wrote a Promise Group class (though I believe this functionality may be on its way to node.js soon?)

The Promise Group takes an array of promise objects, and returns its own promise object that emits success when the group's promises are all finished. This means that I can expect a single promise object from the track() method, and add callbacks as necessary.

That's it for the basics of working with the asynchronous node.js APIs. If you'll notice, I like to work in an iterative, test-driven fashion. I don't feel comfortable writing a lot of code without tests, so it was really helpful for me to start off with horrible synchronous calls and passing tests, and work my way up from there.