Skip to content

GSoC ends… but my work will continue

Today is almost the last day before the Google Summer of Code hard “pencils down” date (August 20). It was a really great experience to work with the great buddycloud team on creating an HTTP API for them this summer, and I learned a lot about XMPP, HTTP, node.js and several other things in the process. I especially would like to thank my mentor Simon Tennant, who helped me a lot to get started and was always there when I needed him. Thank you!

So I guess now is the right time to look back and see what I have accomplished in the last 13 weeks. Thankfully, I managed to implement most of the things I wanted to. buddycloud now has an HTTP API that lets its users receive channel feeds, post to a channel, find out who is subscribed to a channel, receive channel metadata, et cetera. Rodrigo Duarte, who was working on a buddycloud media server at the same time, and I even found the time to integrate our servers so that HTTP API clients have access to media without needing an XMPP connection – something that neither of us has planned for before the program started. In short, I think I can be happy with what I have done.

However, there is one feature that didn’t make it – listening for new channel posts in real time. As I have mentioned in my last post, this depends on some functionality in the buddycloud server that is not fully implemented yet. Unfortunately I didn’t find the time to continue working on this as I was on a two-week vacation to the US for the last two weeks, but I hope I can finally get around to finish this up soon after GSoC.

In general, I do not plan to move away from my project once Summer of Code has ended. The buddycloud team is awesome to work with, and the last thing that I want is my code to bitrot and vanish into uselessness. I will continue to maintain and extend the code as long as possible and might help out the buddycloud project in other areas in the future.

Before I end this post, I want to say THANK YOU to the XMPP Foundation for choosing and supporting me for this Summer of Code (especially for sending us copies of XMPP: The Definitive Guide and Getting Things Done – they helped me a lot!), and to Google, without whose sponsoring all of this wouldn’t be possible. You rock!

Speak JSON only? The buddycloud HTTP API can talk to you now

My last post about the buddycloud HTTP API was quite a long time ago (almost three weeks!). This has multiple reasons. One of them is that I had to write this semester’s exams in the last three days and thus didn’t have that much time between having to learn for them and going to University. (The semester has ended now, so no more University work from now on – yay!)

The other reason is that the next feature I planned to implement – the ability to long-poll the API server for real-time updates to channels – depends on a not-yet-implemented feature in the buddycloud XMPP component. Namely, it should be possible to temporarily subscribe to a channel to get update notifications even if you are not a follower of the channel. I started to implement this, but as Thomas aka Schnouki, who I am drafting this with (hey Thomas, thanks for helping me out!), is not available this week, I cannot currently continue this work sensibly.

Instead, I worked on something that multiple buddycloud core developers bugged me about in the last few weeks. (Here you are, Simon! :)) Previously, the API used JSON for several types of API requests such as getting subscriber lists or channel metadata, but the post streams themselves were only delivered as Atom feeds – that is, in XML form. The slightly awkward consequence of this is that you might need to do both XML and JSON processing when using the API. This might not be a problem for several use cases (in JavaScript, the XMLHttpRequest.responseXML attribute makes XML processing pretty easy actually), but often, one would rather like to have a simpler, JSON-only interface.

For such cases, I have sat down this week and implemented the possibility to get a JSON-serialized form of a channel’s posts. It doesn’t include everything the Atom version has – in fact, it’s pretty ad-hoc and downright primitive. It includes pretty much everything that most clients are interested in, though, like the items’ contents, who posted then, and which other post it relates to (if it is a comment).

Atom is still the default, but you can ask for JSON by including Accept: application/json into your request. For instance, this is how the latest two posts to topics@topics.buddycloud.org look when requested this way:

[
  {
    "id": "0a1ef625-8723-47f3-b6fe-d1dc89f242db",
    "author": "tuomas@buddycloud.org",
    "updated": "2012-07-17T20:17:00.818Z",
    "content": "Welcome!",
    "replyTo": "f0ecd09a-284b-41e6-9863-8214253b1aa3"
  },
  {
    "id": "4e60f61d-e87a-4ad8-a8d4-51b04094a56c",
    "author": "fahrertuer@buddycloud.org",
    "updated": "2012-07-17T15:25:53.987Z",
    "content": "coffee@topics.buddycloud.org\nFor all of us, who love their 1,3,7-trimethyl-1H-purine-2,6(3H,7H)-dione coming from the black brew"
  }
]

(I assume he “acct:” prefix in the two older posts’ “author” field comes from an earlier buddycloud server version.) You can also use this JSON format to post new items. For more details and some examples, see the HTTP API documentation. This feature is already deployed at api.buddycloud.org, so you can already play around with it if you like.

Another small thing I did was to slightly tweak the HTTP demo client. It now looks better and shows the viewed channel’s followers. Posting is still not implemented, though; this is for later.

Demo client for the buddycloud HTTP API

Hi,

This week, I decided to start dogfooding the HTTP API I am developing for buddycloud. The result is buddycloud-http-client, a minimalist web app that is supposed to offer functionality similar to the official web client (viewing channel posts, posting and commenting, real-time updates etc.), but implemented using HTTP API requests instead of talking to the XMPP server directly over BOSH (and, naturally, much less polished – this is only thought as a demonstration, after all). Currently, the client looks like this:

As you can see, there is not much functionality there yet – you can just view the most recent posts of a particular channel, plus a very modest amount of metadata, and there are no dynamic updates yet – but hey, it’s a start. During the Google Summer of Code timeframe, I am going to extend the client’s functionality to match the features that the HTTP API has to offer (and put features into the API that may be required by the clien).

The code can be found on GitHub. I have also also uploaded it to my small ARM home server box at https://www.denisw.de/buddycloud-http-client/ so that you to can play around with it. By default, it just loads the lounge@topics.buddycloud.org channel and shows its most recent posts; to view another channel, add ?channel=<channel-id> to the URL. For instance, the following link shows the gsoc2012@topics.buddycloud.org channel:

https://www.denisw.de/buddycloud-http-client/?channel=gsoc2012@topics.buddycloud.org

I hope to present a more full-featured version of this client to you in the near future.

buddycloud HTTP API progress

Hello again! Since I have deployed the buddycloud HTTP API server to beta.buddycloud.org, it gained some new tricks I implemented during this and the last week. It is now possible to get some metadata about a host like this:

Client:
GET /channels/lounge@topics.buddycloud.org/posts/meta HTTP/1.1
Host: api.buddycloud.org
…

Server:
HTTP/1.1 200 OK
…
{
  "title": "Lounge",
  "description": "Welcome to buddycloud!",
  "access_model": "open",
  "channel_type": "topic",
  "default_affiliation": "publisher"
}

This feature is already deployed. Here is an example link for the gsoc2012@topics.buddycloud.org channel. The next task on my to-do list is to make this metadata editable via POST requests.

Another thing I implemented is the possibility to get the list of users subscribed to a channel node, and to (un)subscribe to/from a node. While this works fine with my regression tests, the buddycloud-server version installed on beta.buddycloud.org seems to be slightly out-of-date and creates somewhat non-standard Pub-Sub <affiliations/> replies (the “jid” attributes are missing). For this reason, requests such as GET /channels/lounge@topics.buddycloud.org/posts/sub return an empty subscriber list currently. I hope this will be sorted out soon.

On the non-coding side, I did some work to document the interfaces I have implemented so far on the buddycloud HTTP API page, using the nice template Simon came up with (I wrote the sections whose titles start with /channels/<name>@domain/…). Hopefully, this does a reasonable job explaining how the API can be used, although some information from my GSoC wiki page is still be missing (e.g., concerning authentication and sessions). I’ll migrate everything valuable to the “official” page during the summer.

That’s all for now. Expect a new post from me soon.

buddycloud HTTP API deployed – Try it!

Hi,

After some fumbling around (and one or two failed attempts – thanks to Simon Tennant for his patience :)), I finally managed to deploy the HTTP API server I’m working on to buddycloud.org. This means you can start playing around with it now!

For starters, the API lets you view every buddycloud channel as an Atom feed with a simple HTTP URL. For instance, the posts of the gsoc2012@topic.buddycloud.org channel, on which much GSoC-related discussion takes place, is made accessible as https://api.buddycloud.org/channels/gsoc2012@topics.buddycloud.org/posts. Go ahead and try out the link – the channel is public and thus doesn’t require authentication. This is how it is displayed inside of Firefox on my PC:

If you have an account at beta.buddycloud.org (it’s trivial to set one up if you don’t – you only need to supply a username, password and e-mail address), you can also use the API to post to a channel. For instance, here is how you can greet us with a post to the “lounge” channel by using cURL:

curl -u YOUR_USERNAME@buddycloud.org:YOUR_PASSWORD --data-binary "<entry xmlns='http://www.w3.org/2005/Atom'><content>Hey</content></entry>" https://api.buddycloud.org/channels/deniswtest@topics.buddycloud.org/posts

More details can be found on my project’s wiki page. Have fun!

buddycloud and the REST

Hello,

I took MUCH too long to write this, but here it is: the first post on my Google Summer of Code 2012 project for the XSF.

Before diving into what I have done and will be doing, I’ll introduce myself quickly. My name is Denis Washington, I’m 22 years old and I study Computer Science at the Humboldt University of Berlin (I’m in my fourth semester currently). I have a passion for everything open and collaborative when it comes to software, the web, protocols and standards. I use Ubuntu as my main operating system and love to use my free time to hack code, play indie games and spend time with my wonderful girlfriend. I am also a sucker for programming languages and love to learn a new one, like, every six months or so. 🙂

Anyway, here is what I am up to: for buddycloud, I’m developing a small server that allows its clients to access and modify buddycloud channels through a REST-like HTTP API. What this means is that every channel stream and post can be addressed with an HTTP URL, retrieved as an ordinary Atom document, modified, posted to and much more. This is going to open up a great deal of opportunities for interoperability between buddycloud and the web. For instance, here are some use cases copied from my project’s wiki page at buddycloud.org:

  • “Post To Channel” button: Joe maintains a cute cat picture gallery site and wants to give his visitors the possibility to share one of these with their kitten-loving buddycloud friends. For this reason, every picture should have a “Post to Channel” button which, when clicked, lets the user choose from a list of his or her subscribed channels and automatically posts a link to the picure to that channel. Using the HTTP API, he can program this easily with only a few XmlHttpRequests and absolutely no knowledge of the underlying XMPP communication.
  • External following: Alice is a talented indie game developer who often talks on conferences about good game design and her newest game projects. On one of these conferences she meets Bob, who is very interested in what Alice is doing and would love to follow her newest game development activities and which conferences she is going to speak in. However, Alice writes about these things mostly in a public buddycloud channel, while Bob is Friendica user. However, Alice is able to give Bob an HTTP URI which, when Bob pastes it into Friendica’s Atom connector, allows him to have Alice’s posts seamlessly integrated into his Friendica message stream without having to create a buddycloud account only for this reason.
  • Automation: Karen is the system administrator of a small open-source project which mainly communicates through a buddycloud channel. To not have old bug reports forgotten, she has the idea to randomly choose a selection of old unresolved bugs – say, older than six months – from the bug tracker every two weeks and highlight them by posting them to the project’s channel. She is able to quickly automate this process by writing a server cron job which leverages the HTTP API through simple invocations of the ubiquitous cURL command-line utility.

Many more use cases are imaginable. And as buddycloud channels are technically not much more than collections of Atom-based XMPP Publish-Subscribe nodes, the code does not only apply for buddycloud, but could also be useful for to other Pub-Sub based systems. (I am currently concentrating on making it work nicely with buddycloud, though.)

So how far am I? Currently, retrieving and posting to channel streams is completely working. Authenticating requests by passing in a JID and a password (via HTTP Basic Auth) is also implemented. This means that with the soon-to-be-deployed HTTP API server on buddycloud.org, you’ll be able to do requests like this one:

Client:
GET /channels/lounge@topics.buddycloud.org/posts?max=10 HTTP/1.1
Host: api.buddycloud.org
…

Server:
HTTP/1.1 200 OK
…
<feed xmlns="http://www.w3.org/2005/Atom">
  <entry id="…">
    <author>
      <name>denisw@buddycloud.org</name>
       …
    </author>
    <content>This is the newest post!</content>
  </entry>
  …
</feed>

or this one:

Client:
POST /channels/lounge@topics.buddycloud.org/posts
Authorization: Basic …
Content-Type: application/atom+xml
…
<entry xmlns="http://www.w3.org/2005/Atom">
  <content>Hey, my first channel post via HTTP!</content>
</content>

Server:
HTTP/1.1 201 Created
Location: /channels/lounge@topics.buddycloud.org/posts/item?id=…
…

And – my favorite – you’ll be able to follow channels with your feed reader:

Thunderbird supports feeds protected with HTTP Basic, so you can even follow private channels!

For more details on the API looks, see the already-mentioned wiki page. (Not everything described there is implemented yet.)  The code can be found at Github.

Next up on my todo list – other than polishing and cleaning up what I have so far – is implementing access to a channel stream’s metadata (for instance, who is subscribed to it), as well as the possibility to (un)subscribe a channel stream. Stay tuned.

Glade and property binding: Finally continued

Google Summer of Code is long since over, and I still haven’t posted anything new about my project, GObject property binding in GtkBuilder and Glade, since mid-July. (By the way, I passed the final evaluation! Thanks again to my mentor, Juan Pablo Ugarte, and everything else involved!) This has essentially three reasons:

  • Since my last post, I didn’t actually achieve anything feature-wise because I had quite a lot of refactoring to do to get the code into shape. (Actually, I did implement support for transformation functions as I had planned, but had to tear that out again because it is not entirely clear how to implement this on the GTK+ side without introducing annoying bits of new API; see the tracking bug for the GTK+ branch.)
  • Between being busy studying and doing other things, I didn’t spend any time on the code for several weeks.
  • To be honest, I just easily forget such things…

But today, I decided to change this. In fact, I am writing this to tell you that I recently have begun to continue my work in order to resolve all the problems remaining before the code is worthy to enter Glade/GTK+ master. Currently I am concentrating on the Glade side; for those interested in the details, see the message I sent to the glade-devel mailing list today.

I hope that that these branches will soon be ready for prime time. Stay tuned!

Glade and property binding: Create your own bindings now!

Hello inhabitants of the wonderful world of GNOME,

I admit it right away: I lied. Or, rather, didn’t do what I promised and planned. While I wrote in my last post that I would blog about any visible progress of my Google Summer of Code project, GObject property binding in GtkBuilder and Glade, right after it is commited, I didn’t post anything for, gosh, over a whole month. Which is a shame, as there *is* actually quite a lot to see: support for not only viewing, but also creating and deleting property bindings is now fully implemented and working! The UI for this is pretty non-obvious at the moment and could be much improved, but it’s fully functional and will receive a great deal of spit and polish once the mandatory implementation work is done (more on this later).

The right-click menu of every property in the inspector (what, you didn’t know there was a right-click menu? well, I said this isn’t obvious ;)) now contains a new item named Bind to source… which allows you to make the property the target of a new binding.

Choosing this item will present you will a nice little two-pane dialog which allows you to choose the source object and property you would like to bind to. As you can see in the screenshot below, properties whose type don’t match the target property are greyed out and appear after all the compatible ones.

(Yes, these screenshots are made in fallback mode, which is why the dialog is not visually attached to the Glade parent window. For some odd reason, the screenshot tool gives me only shots of my desktop background when I’m in the Shell – probably because the crappy propietary ATI Catalyst drivers which I have to use because my laptop’s GPU fan goes crazy with the open-source ones. *End of off-topic rant*)

Clicking Bind then makes the magic happen. Last but not least, the mentioned right-click menu also offers the option to remove an established property binding you don’t like it anymore, or replacing it if you like another one better.

So, voila, the whole CRUD family is there! You can now actually use the property binding feature in this basic form. (Okay, the resulting app won’t run anywhere as the code in my GTK+ branch is needed so that GtkBuilder is able to read the new <binding> element, but hey.)

So, where to go from here? For this second half of GSoC, my plan is to implement support for transformation functions, that is, being able to specify the name of a function which may arbitrarily transform the value of a binding’s source property before passing it on to the target. This includes further enhancements in my GTK+ branch – possibly with new GtkBuilder API functions for hooking up transformation functions automatically in the manner of gtk_builder_connect_signals() (I sent a message to gtk-devel-list to discuss this) -, some extensions to Glade’s data model and, naturally, exposing the whole thing in the UI. Should be more than doable in six weeks’ time, should it? After this is done, the agenda will be polishing, polishing, polishing: fixing bugs, gracefully handling edge cases and, lastly, making the UI a bit prettier and more discoverable.

Well, this is it for now. Sorry for taking so long for blogging about all of this, but I had a lot on my plate in the last weeks with my semester coming to an end and course work getting more and more time consuming. But now my lectures are over, next Monday is my last exam, and then I’ll have all the time of the world to bring you plenty of Glade property goodness! Stay tuned…

Glade and property binding: Finally something to see!

Hello fellow GNOMErs,

Since shortly before Google Summer of Code started, I haven’t blogged the least bit about my Google Summer of Code project (GObject property binding support for GtkBuilder and Glade; see my last post or here for what this about). The reason for this is easy: there simply wasn’t anything to show! Not that I didn’t get any coding work done in the first two weeks, I naturally did, and I am even slightly ahead of schedule despite having to study in parallel as my semester does not end until July. (Yay me.)

However, that work was of the boring type: implement support for a new <binding> tag in GTK+’s GtkBuilder class to codify property bindings in GtkBuilder files, writing a small GObject class named “GladeBinding” to represent property bindings in Glade, integrate it into Glade’s existing data model, extend Glade’s GtkBuilder parser to create GladeBinding objects from <binding> tags, and make Glade write such objects to <binding> tags again when saving. Sounds exciting, doesn’t it?

So after all of that, I basically was at the point where you can open GtkBuilder files with <binding>‘s and save them again without losing the binding definitions in the process. So far so good. However, the read binding definitions didn’t actually do anything whatsoever within Glade. Kind of boring. So this week, I devoted my time to make property bindings in Glade ALIVE!

First of all, this means that property bindings read from a GtkBuilder file are now actually created for real. That is, you can change the value of a binding’s source property and the target property is updated to the same value instantly.

The property inspector is now also aware of property bindings. If a property is the target of a binding, its name is now bold and italic, and its value editing contols are disabled (the property’s value depends on the source property, right?). As cherry on top, the tooltip of a bound property shows which property it is bound to.

Lovely, isn’t it? Finally something to see!

All of this work can be found in my “gbinding” branch at git.gnome.org. If you feel like compiling it and giving it a spin (which I would absolutely be happy about!): you can test the new code by taking a GtkBuilder file of your choice (or quickly creating a new one), opening it with a text file, and adding a tag like the following to the object you would like to be the binding target:

<binding to="target-property" from="source-property" source="source-object"/>

with target-property, source-property and source-object replaced with sensible values.

Well, that’s it for now. Next on my TODO list, apart from fixing bugs (which I have likely introduced) and memory leaks (which I have *most likely* introduced), is to make it possible to actually create and edit property bindings from within Glade. This will require me to go back to some boring work again for while (integrate with the undo/redo system and things), but if there is be anything new on the UI side, you’ll hear from me as soon as possible. I promise!

Dear Planet GNOME readers…

… please welcome your new kid on the blog: his name is Denis Washington, and he is a 21-year old Computer Science student living in Berlin, Germany. He is a passionate Linux user since the days of Fedora Core 1 and has made several small contributions to GNOME in the past few years, including participation in the Google Highly Open Participation Contest 2007/2008 (now called Google Code-in). He has been accepted to this year’s GNOME Summer of Code and will try as well as he can to keep you updated about his progress in the next months with his blog. Please be nice to him!

That guy would be me, yes.

So now it’s finally time for me to say: Hello, Planet GNOME! First of all, let me tell you how much of a honor it is to be aggregated here. I’ve been reading the Planet for years and have learned a lot about GNOME, it’s community and the development of Free Software (thanks to everyone who has ever posted here!). Being part of it now feels just awesome!

I would also like to thank the GNOME project for accepting me to Google Summer of Code and thus giving me the possibility to not only post here, but also work full-time on making GNOME rock even harder – and even be paid a impressive amount of cash (for a student) in the process. Special thanks go to Juan Pablo Ugarte, who agreed to be my mentor for the summer (I hope he won’t regret it!). Also, great thanks to Google for even making it possible for the GNOME project to give me the possibility to… erm… just thanks! 🙂

Phew. Now with all the acknowledigments of my limitless gratitude out of the way, let me introduce you to what I am going to work on during the summer. My project is titled GObject property binding support in GtkBuilder and Glade, and this is exactly what I’m going to deliver: the possibility to create bindings between widget properties right within Glade – our beloved GTK+ user interface editor – and save them along with the interface to GtkBuilder files.

Now before you follow the above link to my extremely lengthy Summer of Code application, let you give you a quick overview of the project.

What is this about?

Since 2.26, GLib has a GObject property binding API (named GBinding) which implements a simple form of what the cool kids call reactive programming. Basically, it lets you define that whenever a specified property of a particular GObject (the source) changes its value, a certain other GObject instance property (the target) should be updated accordingly – either by setting it to the same value, or by passing that value to a transformer function which computes the target’s new value. This is a nice thing to have, as this can save a lot of potentially error-prone boilerplate code (writing and connecting signal handlers which call setters on property changes) in favor of a single call to g_object_bind_property() for things like “This widget should be sensitive if that checkbox is enabled” or “The page displayed by this notebook  depends on that radio group’s currently selected item” (among other things).

Now this is nice and all, but it would be even better if you could define bindings between your Glade UI’s widgets graphically and save them as part of GtkBuilder files, without having to write a single line of additional code, wouldn’t it? Well, this is exactly what I’d like to implement this summer. To be exact, you’ll be able to:

  • Right-click a property of a widget in Glade’s property inspector and choose “Bind to Source Property…” (or something similar) to make create a property binding
  • Specify source-to-target transformation functions for property bindings by name which will be automatically connected like gtk_builder_connect_signals() already does for signals
  • See the effect of  the defined property bindings in preview mode and directly in the Glade workspace
  •  Free your code from tedious manual-property-binding signal handlers and calls
  • Thank me for my awesome work 😉

I hope I’ll be able to deliver all of this, including a lot of polish and extensive testing,  in the next 12 weeks; considering that my semester ends only a few weeks after GSoC starts, it won’t be easy, but I’ll certainly do my best.

In any case, I will keep you updated about my progress during this summer. (This is, of course, the reason why I created this blog, and why I was able to sneak into Planet GNOME.) So expect to hear from me again soon!

Best regards,
Denis Washington