Facebook documentation requires login, stopper for generating code? :-/

Tonight I’ve ran into a real obstacle with generating code from the Facebook online documentation. Please see my post on the Facebook Developer Forum here:

http://forum.developers.facebook.net/viewtopic.php?pid=323044#p323044

If this isn’t resolved, it’ll be a real stopper for generating code from the documentation… :-/

Taking a stab at generating strongly typed Facebook Graph C# classes for .Net/Asp.Net

During the last few weeks I’ve finally continued working on a Facebook application project I started probably over a year ago.

First of all I upgraded the application to use the new open source Facebook C# SDK, this seems to be the one used and I really like it even though Facebook has another “Official” one that seems pretty... Small.

The Facebook C# SDK utilises the .Net 4.0 dynamic type to create a very light-weight and flexible frame work, which I actually think is very good. You can read more about the thoughs on this approach here and here. The SDK has methods for sending Get, Post, Delete and FQL Query request to the Facebook APIs, as well as authenticating to get the access tokens needed. the Get method also comes in a generic version allowing you to choose whether you want to create strongly typed classes or not and directly deserealize the API results to these types (using Json.Net).

However, working with strongly typed objects are kinda nice ;-) and with my recent learnings about code generation in Visual Studio using T4 text templates I started playing with the thought of generating these classes directly from Facebooks online documentation of the Graph API objects. I found that this was not a new idea but I decided to give it a go, in part cause what I would learn from it, but also because, as already stated in above linked blog posts, the Facebook APIs are really big and extensive, and managing classes and methods for all this manually would be a nightmare indeed – and boring. While writing T4 templates is quite fun :-).

Before reading further you should probably be acquainted with the following:
http://developers.facebook.com/docs/reference/api/
http://developers.facebook.com/docs/authentication/
http://json.codeplex.com/

Have a look at the source
I have checked in the workings described in this blog post as a fork to the Facebook C# SDK Contrib project, you can have a look at the described check-in here while reading this post (my stuff is located in the Facebook.Entities folder, which is a new and separate assembly). Please note that the classes that are generated from the GraphEntities.tt file are not included in the checked in code.

Finding the patterns
After inspecting the documentation pages I found usable patterns in the markup and with the help of the HtmlAgilityPack I started to write my text template where I parse the documentation pages to produce one class for each object that is listed in the left side menu. I call these classes “First level classes”. Inside these classes I generate properties for all of the Properties and Connections.

Determining the types of properties and generating helper classes
This is what the generated ID and From properties looks like in the Message class:

///<summary>
///The unique ID for this message. Permissions: read_mailbox.
///Returns JSON string.
///</summary>
[JsonProperty("id")]
public string ID { get; set; }

///<summary>
///The sender of this message. Permissions: read_mailbox.
///Returns a JSON object that contains the name, email and Facebook id (if available) of the sender.
///</summary>
[JsonProperty("from")]
public From From { get; set; }

The first row in each summary XML comment is the text contents from the Description and Permissions columns from the documentation page. The second row is the text content from the Returns column and this is what I’ve used to determine what type the property should have. “JSON string” is simple, it results in a string property. When it comes to the From property you can see that it is of type “From”. This is also a generated class that is created from the returns information, I call these classes “Helper classes”:

///<summary>
///A JSON object that contains the name, email and Facebook id (if available) of the sender
///</summary>
[JsonObject(MemberSerialization.OptIn)]    
public partial class From
{
    [JsonProperty("name")]
    public string Name { get; set; }

    [JsonProperty("email")]
    public string Email { get; set; }

    [JsonProperty("id")]
    public string ID { get; set; }

}

This is possible thanks to the fact that the words “name”, “email” and “id” are marked up as <code> html elements in Facebooks documentation. The types for these properties are a bit harder to determine in a certain way. For now I’m associating different names to specific types and if the property has a “non listed” name I’m falling back on string. It is a concern that this is a bit fragile.

A very common combination of properties in these HelperClasses turned out to be two string properties named “id” and “name”. So I decided to handcode a class for that and use that whenever that combination was found instead of generating yet another helper class. Thats why the from property in the Album class is generated like this:

///<summary>
///The profile that created this album. Permissions: Publicly available.
///Returns a JSON object containing the id and name fields.
///</summary>
[JsonProperty("from")]
public IdName From { get; set; }

Connection properties
The connection properties are properties that requires another data fetch from the Facebook API and the Json is always formatted as an object with a “data” property containing an array of objects and a “paging” property containing url strings for “previous” and “next”. For this I created the generic DataContainer class and this is what the generated Friends property looks like in the User class:

private DataContainer<IdName> _friends;
///<summary>
///The user's friends. Permissions: Available to everyone on Facebook.
///An array of JSON objects containing friend id and name fields.
///This Connection Property requires UserAccessToken to be set on instance before it is accessed.
///To force refresh from Facebook servers after first access (by instance), set RefreshFriends to true before accessing again.
///</summary>
[JsonProperty("friends")]
public DataContainer<IdName> Friends
{ 
    get
    {
        this.EnsureUserAccessToken();
        if (_friends == null || this.RefreshFriends)
        {
            _friends = new FacebookClient().Get<DataContainer<IdName>>(this.ID + "/friends/?access_token=" + this.UserAccessToken + "") ?? new DataContainer<IdName>().InitEmpty();
            this.RefreshFriends = false;
        }
        return _friends;
    }
}
public bool RefreshFriends { get; set; }

This introduces some new requirements. First of all, the Facebook.Entities dll has a dependency towards the Facebook.dll from the Facebook C# SDK, so that data for the connection properties can be fetched using the FacebookClient class. To fetch data you need to include an access_token in the request, so all first level classes will be generated to inherit from the GraphEntityBase class. This class defines a property called UserAccessToken that needs to be set before you access a Connection Property that requires the access_token to make the call. The method EnsureUserAccessToken is called from each property before making requests and will throw an exception if the UserAccessToken is not set, to avoid unnecessary calls to the Facebook API.

This is how you would get the current users friends in your application:

User currentUser = new FacebookClient().Get<User>("me/?access_token=ACCESSTOKEN");
currentUser.UserAccessToken = "ACCESSTOKEN";
DataContainer<IdName> friends = currentUser.Friends;

As you can see in the code for the property it uses lazy initalization and caches the friends list in a private member and returns that if you access the Friends property repeatedly on the same instance. This is also to avoid unnecessary calls to the Facebok API. If you want to force fresh data from the Facebook API on a subsequent access to the property you must set the generated RefreshFriends property to true before accessing the Friends property again.

When it comes to caching and getting fresh data Facebook advices you to make use of the Real-time updates API and that is something I’ve been working on including support for in this framework, and I will cover the progress on that in a later blog post. (really, you’ll wanna check this out, I’m working on a solution where you’ll be able to configure what real time updates you want to subscribe to in web.config and then the generated classes and some other code will take care of the rest ;-)

Picture connection properties
There is one exception when it comes to the Connection properties and that is the connection properties named “picture”. Due to that, this is what is generated for a Picture property when it is publicly available. Just assign any of these string properties to the ImageUrl of an Image control and the picture will be displayed:

///<summary>
///The user's profile picture. Permissions: Publicly available.
///Returns a HTTP 302 with the URL of the user's profile picture (use ?type=small | normal | large to request a different photo).
///</summary>
public string Picture { get {  return Utils.GetGraphBaseUrl() + this.ID + "/picture?"; } }

///<summary>
///The user's profile picture. Size small. Permissions: Publicly available.
///Returns a HTTP 302 with the URL of the user's profile picture (use ?type=small | normal | large to request a different photo).
///</summary>
public string PictureSmall { get { return Utils.GetGraphBaseUrl() + this.ID + "/picture?&type=small"; } }

///<summary>
///The user's profile picture. Size normal. Permissions: Publicly available.
///Returns a HTTP 302 with the URL of the user's profile picture (use ?type=small | normal | large to request a different photo).
///</summary>
public string PictureNormal { get { return Utils.GetGraphBaseUrl() + this.ID + "/picture?&type=normal"; } }

///<summary>
///The user's profile picture. Size large. Permissions: Publicly available.
///Returns a HTTP 302 with the URL of the user's profile picture (use ?type=small | normal | large to request a different photo).
///</summary>
public string PictureLarge { get { return Utils.GetGraphBaseUrl() + this.ID + "/picture?&type=large"; } }

When the type can’t be determined
If the type for a property in a “first level class” can not be determined from the Returns information I’m falling back on dynamic. This is what the generated Actions property looks like in the Post class:

///<summary>
///A list of available actions on the post (including commenting, liking, and an optional app-specified action). Permissions: read_stream.
///Returns a list of JSON objects containing the 'name' and 'link'.
///Type for property actions could not be auto-generated from Facebook online documentation. Use this dynamic implementation or check for, or hand-code, property named Actions in partial class definition.
///</summary>
[JsonProperty("actions")]
public dynamic ActionsDynamic { get; set; }

This allows me to work with the Actions data in my strongly typed class, but for this particular property I will have to check out the Facebook documentation to see what type it is and what properties it has. The reason that the type for this specific property can’t be determined is an irregularity in the facebook documentation where the words “name” and “link” in the returns column are NOT marked up with <code> html elements. I have found a few of these irregularities and documented them in notes.txt. I will probably post these documentation corrections/change suggestions in the Facebook Developer Forum.

Before commenting the “Dynamic” added to the property name, let’s discuss the Facebook.Entities assembly; I think the approach must be that the Facebook.Entities assembly actually should be released as a downloadable C# Class Library project and not as a compiled dll. It could be part of the Facebook C# SDK or it could be stand alone. Either way you can use it only in projects where you determine that strongly typed classes are of certain interest. If so, you should include it as a project in your Facebook application solution and be able to extend the generated classes. This would allow you to hand code the properties that can’t be strongly typed generated, and that’s why I leave the name “Actions” available to implement. (The hand coded implementation could also be included in the project, in that scenario I would probably introduce three partial class definitions; one generated, one project maintained and one that by default is empty but meant for you to use to include any type of custom stuff you might want there. The last set of files would never be updated in the project and should not be replaced when you download a new version of the library ).

However, JSON.net introduces a problem in this case, because you can’t reference the same property name in two different JsonProperty attributes within the same class. And since I must be able to de/serialize the generated classes I must use the facebook api names in these attributes. Because of this I’ve made this suggestion in the JSON.net project.

Going forward
Although there certainly are some problems with generating these classes I think these numbers looks pretty good:

/*
The GraphEntities.tt of Facebook.Entities can currently generate
- 19 first level classes
    with a total of 156 properties, 3 were generated as dynamic.
    with a total of 88 connection properties, 9 were generated as dynamic.
- 38 helper classes.
- 2 helper enums.
*/

The code I’ve produced so far certainly hasn’t been very tested but I will use this in the Facebook application I’ll be releasing within a couple of weeks. I’ll keep you posted on how it goes. If you’re interested in contributing to this work I’d be happy to get in contact with anyone that has some experience in unit testing that could help me out on how to make this code testable and write tests for it.

As I mentioned I’ve been working on including support for the Real-time Updates API, and a first version of this is already checked in here. I will cover this in a later blog post.

Please let me know your reactions, thoughts and ideas on this.