I do think there is a lot to be learned from using different interaction styles and true stream-oriented protocols (the kind that don?t care about lost packets), but this FriendFeed example is ridiculous. It took me less than 30 seconds to design a better solution using nothing more than HTTP, and that while sitting in the middle of a conference session. This is exactly what Dare means by: ?If a service doesn?t scale it is more likely due to bad design than to technology choice.?
They are comparing the efficiency of blind polling using HTTP crawls to a coordinated PubSub services setup with XMPP. Spidering an entire site is obviously not going to be efficient if you are only interested in what has changed. One advantage it has is that you don?t need to cooperate with the information provider. However, for the specific example of FriendFeed polling Flickr, cooperation is easy (both companies gain immensely by cooperating) and essential (2.9 million requests per day will get you blocked from any site that doesn?t want cooperation).
The solution, which I mentioned briefly at the talk, is to provide a resource that reflects all of the changes on Flickr during a given time period. Anyone (not just FriendFeed) can perform a GET on that resource to find out what has changed. In fact, one such example is the SUP (Simple Update Protocol) just introduced by FriendFeed. However, I had something more efficient in mind.
Web architects must understand that resources are just consistent mappings from an identifier to some set of views on server-side state. If one view doesn?t suit your needs, then feel free to create a different resource that provides a better view (for any definition of ?better?). These views need not have anything to do with how the information is stored on the server, or even what kind of state it ultimately reflects. It just needs to be understandable (and actionable) by the recipient.
In this case, we want to represent the last-updated state of all Flickr users in a way that minimizes the lag between event and notification (let?s just assume that one minute is ?fast enough? to receive a change notification). The simplest way of doing that is to log state changes by userid in a sequence of journal-style resources named according to the server?s UTC clock minutes. […]
Arquivo Discussoes de Programacao e Tecnologia