So I guess now is the right time to look back and see what I have accomplished in the last 13 weeks. Thankfully, I managed to implement most of the things I wanted to. buddycloud now has an HTTP API that lets its users receive channel feeds, post to a channel, find out who is subscribed to a channel, receive channel metadata, et cetera. Rodrigo Duarte, who was working on a buddycloud media server at the same time, and I even found the time to integrate our servers so that HTTP API clients have access to media without needing an XMPP connection – something that neither of us has planned for before the program started. In short, I think I can be happy with what I have done.

However, there is one feature that didn’t make it – listening for new channel posts in real time. As I have mentioned in my last post, this depends on some functionality in the buddycloud server that is not fully implemented yet. Unfortunately I didn’t find the time to continue working on this as I was on a two-week vacation to the US for the last two weeks, but I hope I can finally get around to finish this up soon after GSoC.

In general, I do not plan to move away from my project once Summer of Code has ended. The buddycloud team is awesome to work with, and the last thing that I want is my code to bitrot and vanish into uselessness. I will continue to maintain and extend the code as long as possible and might help out the buddycloud project in other areas in the future.

Before I end this post, I want to say THANK YOU to the XMPP Foundation for choosing and supporting me for this Summer of Code (especially for sending us copies of XMPP: The Definitive Guide and Getting Things Done – they helped me a lot!), and to Google, without whose sponsoring all of this wouldn’t be possible. You rock!

The other reason is that the next feature I planned to implement – the ability to long-poll the API server for real-time updates to channels – depends on a not-yet-implemented feature in the buddycloud XMPP component. Namely, it should be possible to temporarily subscribe to a channel to get update notifications even if you are not a follower of the channel. I started to implement this, but as Thomas aka Schnouki, who I am drafting this with (hey Thomas, thanks for helping me out!), is not available this week, I cannot currently continue this work sensibly.

Instead, I worked on something that multiple buddycloud core developers bugged me about in the last few weeks. (Here you are, Simon! ) Previously, the API used JSON for several types of API requests such as getting subscriber lists or channel metadata, but the post streams themselves were only delivered as Atom feeds – that is, in XML form. The slightly awkward consequence of this is that you might need to do both XML and JSON processing when using the API. This might not be a problem for several use cases (in JavaScript, the XMLHttpRequest.responseXML attribute makes XML processing pretty easy actually), but often, one would rather like to have a simpler, JSON-only interface.

For such cases, I have sat down this week and implemented the possibility to get a JSON-serialized form of a channel’s posts. It doesn’t include everything the Atom version has – in fact, it’s pretty ad-hoc and downright primitive. It includes pretty much everything that most clients are interested in, though, like the items’ contents, who posted then, and which other post it relates to (if it is a comment).

Atom is still the default, but you can ask for JSON by including Accept: application/json into your request. For instance, this is how the latest two posts to topics@topics.buddycloud.org look when requested this way:

[
  {
    "id": "0a1ef625-8723-47f3-b6fe-d1dc89f242db",
    "author": "tuomas@buddycloud.org",
    "updated": "2012-07-17T20:17:00.818Z",
    "content": "Welcome!",
    "replyTo": "f0ecd09a-284b-41e6-9863-8214253b1aa3"
  },
  {
    "id": "4e60f61d-e87a-4ad8-a8d4-51b04094a56c",
    "author": "fahrertuer@buddycloud.org",
    "updated": "2012-07-17T15:25:53.987Z",
    "content": "coffee@topics.buddycloud.org\nFor all of us, who love their 1,3,7-trimethyl-1H-purine-2,6(3H,7H)-dione coming from the black brew"
  }
]

(I assume he “acct:” prefix in the two older posts’ “author” field comes from an earlier buddycloud server version.) You can also use this JSON format to post new items. For more details and some examples, see the HTTP API documentation. This feature is already deployed at api.buddycloud.org, so you can already play around with it if you like.

Another small thing I did was to slightly tweak the HTTP demo client. It now looks better and shows the viewed channel’s followers. Posting is still not implemented, though; this is for later.

Client:
GET /channels/lounge@topics.buddycloud.org/posts/meta HTTP/1.1
Host: api.buddycloud.org
…

Server:
HTTP/1.1 200 OK
…
{
  "title": "Lounge",
  "description": "Welcome to buddycloud!",
  "access_model": "open",
  "channel_type": "topic",
  "default_affiliation": "publisher"
}

This feature is already deployed. Here is an example link for the gsoc2012@topics.buddycloud.org channel. The next task on my to-do list is to make this metadata editable via POST requests.

Another thing I implemented is the possibility to get the list of users subscribed to a channel node, and to (un)subscribe to/from a node. While this works fine with my regression tests, the buddycloud-server version installed on beta.buddycloud.org seems to be slightly out-of-date and creates somewhat non-standard Pub-Sub <affiliations/> replies (the “jid” attributes are missing). For this reason, requests such as GET /channels/lounge@topics.buddycloud.org/posts/sub return an empty subscriber list currently. I hope this will be sorted out soon.

On the non-coding side, I did some work to document the interfaces I have implemented so far on the buddycloud HTTP API page, using the nice template Simon came up with (I wrote the sections whose titles start with /channels/<name>@domain/…). Hopefully, this does a reasonable job explaining how the API can be used, although some information from my GSoC wiki page is still be missing (e.g., concerning authentication and sessions). I’ll migrate everything valuable to the “official” page during the summer.

That’s all for now. Expect a new post from me soon.

After some fumbling around (and one or two failed attempts – thanks to Simon Tennant for his patience ), I finally managed to deploy the HTTP API server I’m working on to buddycloud.org. This means you can start playing around with it now!

For starters, the API lets you view every buddycloud channel as an Atom feed with a simple HTTP URL. For instance, the posts of the gsoc2012@topic.buddycloud.org channel, on which much GSoC-related discussion takes place, is made accessible as https://api.buddycloud.org/channels/gsoc2012@topics.buddycloud.org/posts. Go ahead and try out the link – the channel is public and thus doesn’t require authentication. This is how it is displayed inside of Firefox on my PC:

If you have an account at beta.buddycloud.org (it’s trivial to set one up if you don’t – you only need to supply a username, password and e-mail address), you can also use the API to post to a channel. For instance, here is how you can greet us with a post to the “lounge” channel by using cURL:

curl -u YOUR_USERNAME@buddycloud.org:YOUR_PASSWORD --data-binary "<entry xmlns='http://www.w3.org/2005/Atom'><content>Hey</content></entry>" https://api.buddycloud.org/channels/deniswtest@topics.buddycloud.org/posts

More details can be found on my project’s wiki page. Have fun!

I took MUCH too long to write this, but here it is: the first post on my Google Summer of Code 2012 project for the XSF.

Before diving into what I have done and will be doing, I’ll introduce myself quickly. My name is Denis Washington, I’m 22 years old and I study Computer Science at the Humboldt University of Berlin (I’m in my fourth semester currently). I have a passion for everything open and collaborative when it comes to software, the web, protocols and standards. I use Ubuntu as my main operating system and love to use my free time to hack code, play indie games and spend time with my wonderful girlfriend. I am also a sucker for programming languages and love to learn a new one, like, every six months or so.

Anyway, here is what I am up to: for buddycloud, I’m developing a small server that allows its clients to access and modify buddycloud channels through a REST-like HTTP API. What this means is that every channel stream and post can be addressed with an HTTP URL, retrieved as an ordinary Atom document, modified, posted to and much more. This is going to open up a great deal of opportunities for interoperability between buddycloud and the web. For instance, here are some use cases copied from my project’s wiki page at buddycloud.org:

• “Post To Channel” button: Joe maintains a cute cat picture gallery site and wants to give his visitors the possibility to share one of these with their kitten-loving buddycloud friends. For this reason, every picture should have a “Post to Channel” button which, when clicked, lets the user choose from a list of his or her subscribed channels and automatically posts a link to the picure to that channel. Using the HTTP API, he can program this easily with only a few XmlHttpRequests and absolutely no knowledge of the underlying XMPP communication.

• External following: Alice is a talented indie game developer who often talks on conferences about good game design and her newest game projects. On one of these conferences she meets Bob, who is very interested in what Alice is doing and would love to follow her newest game development activities and which conferences she is going to speak in. However, Alice writes about these things mostly in a public buddycloud channel, while Bob is Friendica user. However, Alice is able to give Bob an HTTP URI which, when Bob pastes it into Friendica’s Atom connector, allows him to have Alice’s posts seamlessly integrated into his Friendica message stream without having to create a buddycloud account only for this reason.

• Automation: Karen is the system administrator of a small open-source project which mainly communicates through a buddycloud channel. To not have old bug reports forgotten, she has the idea to randomly choose a selection of old unresolved bugs – say, older than six months – from the bug tracker every two weeks and highlight them by posting them to the project’s channel. She is able to quickly automate this process by writing a server cron job which leverages the HTTP API through simple invocations of the ubiquitous cURL command-line utility.

Many more use cases are imaginable. And as buddycloud channels are technically not much more than collections of Atom-based XMPP Publish-Subscribe nodes, the code does not only apply for buddycloud, but could also be useful for to other Pub-Sub based systems. (I am currently concentrating on making it work nicely with buddycloud, though.)

So how far am I? Currently, retrieving and posting to channel streams is completely working. Authenticating requests by passing in a JID and a password (via HTTP Basic Auth) is also implemented. This means that with the soon-to-be-deployed HTTP API server on buddycloud.org, you’ll be able to do requests like this one:

Client:
GET /channels/lounge@topics.buddycloud.org/posts?max=10 HTTP/1.1
Host: api.buddycloud.org
…

Server:
HTTP/1.1 200 OK
…
<feed xmlns="http://www.w3.org/2005/Atom">
  <entry id="…">
    <author>
      <name>denisw@buddycloud.org</name>
       …
    </author>
    <content>This is the newest post!</content>
  </entry>
  …
</feed>

or this one:

Client:
POST /channels/lounge@topics.buddycloud.org/posts
Authorization: Basic …
Content-Type: application/atom+xml
…
<entry xmlns="http://www.w3.org/2005/Atom">
  <content>Hey, my first channel post via HTTP!</content>
</content>

Server:
HTTP/1.1 201 Created
Location: /channels/lounge@topics.buddycloud.org/posts/item?id=…
…

And – my favorite – you’ll be able to follow channels with your feed reader:

Thunderbird supports feeds protected with HTTP Basic, so you can even follow private channels!

For more details on the API looks, see the already-mentioned wiki page. (Not everything described there is implemented yet.)  The code can be found at Github.

Next up on my todo list – other than polishing and cleaning up what I have so far – is implementing access to a channel stream’s metadata (for instance, who is subscribed to it), as well as the possibility to (un)subscribe a channel stream. Stay tuned.