Nub Nub: PubSubHubbub protocol implementation
I've been toying around with the idea of adding a PubSubHubbub layer around the GitHub timeline events, so I wrote Nub Nub. Nub Nub is a bare implementation of PubSubHubbub so I can explore how it might be used inside GitHub.
PubSubHubbub (PuSH) is "a simple, open, web-hook-based pubsub protocol," according to the homepage. There's a lot of talk about hub discovery URLs, Atom feeds, multicast publishing, etc. Let's boil it all down to the essentials:
- A feed (or PuSH topic) identifies one or more hub servers.
- A subscriber (a webhook server) subscribes to one or more topics.
- When the feed updates, it pings the hub server.
- The hub server then publishes the data to the subscribers.
As far as GitHub goes, this is still too complex. We're not a big feed aggregation service, or a generic PuSH hub service.
- A GitHub user specifies one or more post-receive URLs that get pinged on every Git push.
- GitHub publishes to these post-receive URLs on every Git push.
It's the same thing, minus all the talk about feed scanning and updates. We already have an internal queue system that can handle this. We also have people asking for an API to manage service hooks and post receive URLs, so why not provide standard API hooks?
As far as Atom/RSS go, there's not a lot in the PuSH spec that really depend on them. The feed scanning portion does, of course. The data is all internal though, so there's no need to implement the Content Notification methods. But there's no sense in parsing our own feeds, so we can push straight JSON. The specs do say that published items need to be Atom or RSS, but I see no reason we can't support JSON. If the subscriber subscribes to a JSON topic, the hub pushes JSON. If the subscriber subscribes to an Atom topic, the hub pushes Atom.
This strategy can be applied to any other web service that sends out notifications. That's why I wrote Nub Nub. It's a pretty bare bones implementation, with no mention of a preferred web server or data store. It just provides a few methods for making or responding to PuSH calls.