Open Kosmaczewski

Playing with HTTP libraries

March 26, 2008

It’s fun to find out how to tackle the same task in different programming languages; in this case, it’s all about doing HTTP requests over a network: fortunately, there are networking libraries in virtually all major programming languages. In my current project, I’m generating wrappers easing the access to the core of the project itself, a RESTful API. This way, developers interested in using the API can just take a wrapper, include it in their projects, and start coding right away. No need to know this (relatively low-level) stuff; just use the API. The wrappers themselves are auto-generated from the API definition itself, but that’s another story ;)

Below there is a sample of the different ways I’ve found to do a network access to a remote server, using HTTP Basic Authentication and a couple of headers, in PHP, Ruby, Python, JavaScript, and even Objective-C! I’m even generating ActionScript 3.0 code, but I’m not a Flash coder :) So I’ll post the wrappers that work best at the moment, and in the future I’ll include other examples, particularly for .NET, C++ and Java.

In all the cases below, there is a “request” function or method that takes an HTTP verb (GET, POST, PUT, DELETE, etc), a URL (without the slash “/” at the beginning) and some parameter data, in the form of a dictionary. The function wraps the underlying libraries of each programming language, offering a simpler interface, and allowing for HTTP Basic Authentication (for HTTP Digest Authentication it would be much, much more complex!). There are synchronous (useful for server or command-line applications) and asynchronous versions (for GUI systems). Off to the code!

In PHP (synchronous):

$conf = array(
  "server" => "localhost",
  "username" => "",
  "password" => ""
);

The curl library has a distinctive C smell :) The good thing is that after doing this wrapper, doing the C++ one will be fairly straightforward!

In Ruby (synchronous):

require 'net/http'
require 'uri'
require 'json'

$conf = {
    "server" => "localhost",
    "username" => "",
    "password" => ""
}

def request(verb, url, parameters = nil)
  Net::HTTP.start($conf["server"]) do |http|
    headers = {
        "Accept" => "application/javascript",
        "User-Agent" => "Identify yourself!"
    }
    path = "/" + url
    req = nil
    case verb.upcase
      when "GET":
        req = Net::HTTP::Get.new(path, headers)
      when "POST":
        req = Net::HTTP::Post.new(path, headers)
      when "PUT":
        req = Net::HTTP::Put.new(path, headers)
      when "DELETE":
        req = Net::HTTP::Delete.new(path, headers)
      else
        raise Exception.new("Invalid HTTP verb")
    end
    req.basic_auth $conf["username"], $conf["password"]
    req.set_form_data(parameters) if not parameters == nil
    response = http.request(req)
    if response.code == "200" && verb.upcase == "GET"
        JSON.parse(response.body)
    else
        print response.code + " " + response.message + " " + response.body
    end
  end
end

The Ruby library is the only one I found so far that features the four HTTP verbs in the interface!

In Python (synchronous):

I much prefer the Ruby way, if you ask me. I don’t know, it’s more elegant. I can’t get used to the indenting!

In JavaScript, using Prototype (asynchronous):

Again in JavaScript, but this time using jQuery (asynchronous):

};

The last JavaScript one, but this time using bare bones XMLHttpRequest (asynchronous):

To use these JavaScript versions, just pass a couple of functions as parameters, and you’re done; they will be called in case of success or error, asynchronously.

Finally, the Cocoa / Objective-C wrapper (both synchronous and asynchronous, header and implementation files):

/*
Command-line example that uses this library:
#import <Foundation/Foundation.h>

int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool;
    pool = [[NSAutoreleasePool alloc] init];
    
    AKResourceSubClass* resource;
    resource = [[AKResourceSubClass alloc] init];
    [resource get];
    NSDictionary* plist = [resource getPropertyList];
    // Do something with the results...

[pool drain];
    return 0;
}
*/

#import <Foundation/Foundation.h>

@interface AKResource : NSObject {
    NSMutableData* receivedData;
    
    NSString* mimeType;
    NSString* server;
    NSString* username;
    NSString* password;
    
    BOOL async;
}

-(id)init;
-(void)setServer:(NSString*)serverName;
-(void)setUsername:(NSString*)user
    andPassword:(NSString*)pwd;
-(void)connection:(NSURLConnection *)connection 
    didReceiveResponse:(NSURLResponse *)response;
-(void)connection:(NSURLConnection *)connection 
    didReceiveData:(NSData *)data;
-(void)connection:(NSURLConnection *)connection
    didFailWithError:(NSError *)error;
-(void)connectionDidFinishLoading:(NSURLConnection *)connection;
-(void)sendRequestTo:(NSString*)urlString 
    usingVerb:(NSString*)verb 
        withParameters:(NSDictionary*)parameters;
-(void)setAsynchronous:(BOOL)asynchronously;
-(NSDictionary*)getPropertyList;
-(NSString*)getResponseText;

@end

Now the implementation file:

#import "AKResource.h"

@implementation AKResource

-(id)init
{
    self = [super init];

if(self)
    {
        receivedData = [[NSMutableData alloc] init];

mimeType = @"application/plist";
        server = @"localhost";
        username = @"";
        password = @"";
        
        async = NO;
    }

return self;
}

-(void)setServer:(NSString*)serverName
{
    server = serverName;
}

-(void)setUsername:(NSString*)user andPassword:(NSString*)pwd
{
    username = user;
    password = pwd;
}

-(void)sendRequestTo:(NSString*)resource 
    usingVerb:(NSString*)verb 
    withParameters:(NSDictionary*)parameters
{
    NSURL* url = [NSURL URLWithString:[NSString 
        stringWithFormat:@"http://%@/%@", server, resource]];

NSMutableDictionary* headers = [[NSMutableDictionary alloc] init];
    [headers setValue:@"text/html; charset=utf-8" 
        forKey:@"Content-Type"];
    [headers setValue:mimeType forKey:@"Accept"];
    [headers setValue:@"no-cache" forKey:@"Cache-Control"];
    [headers setValue:@"no-cache" forKey:@"Pragma"];

NSMutableURLRequest* request;
    request = [NSMutableURLRequest requestWithURL:url
                cachePolicy:NSURLRequestUseProtocolCachePolicy
                timeoutInterval:60.0];
    [request setHTTPMethod:verb];
    [request setAllHTTPHeaderFields:headers];

if (parameters)
    {
        NSMutableString* params = [[NSMutableString alloc] init];
        for (id key in parameters)
        {
            [params appendFormat:@"%@=%@&", 
                [key stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding], 
                [[parameters objectForKey:key] 
                    stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
        }
        [params deleteCharactersInRange:NSMakeRange([params length] - 1, 1)];
    
        NSData* body = [params dataUsingEncoding:NSUTF8StringEncoding];
        [request setHTTPBody:body];
    }
    if (async)
    {
        NSURLConnection* connection;
        connection = [[NSURLConnection alloc]  initWithRequest:request
                                                delegate:self
                                                startImmediately:YES];

if (!connection)
        {
            NSLog(@"Could not open connection to resource");
        }
    }
    else
    {
        NSURLResponse* response = [[NSURLResponse alloc] init];
        NSError* error = [[NSError alloc] init];
        NSData* data = [NSURLConnection 
            sendSynchronousRequest:request 
            returningResponse:&response 
            error:&error];
        [receivedData setData:data];
    }
}

-(void)connection:(NSURLConnection *)connection
        didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge
{
    NSURLCredential *newCredential;
    newCredential = [NSURLCredential credentialWithUser:username
                        password:password
                        persistence:NSURLCredentialPersistenceNone];
    [[challenge sender] useCredential:newCredential 
        forAuthenticationChallenge:challenge];
}

-(void)connection:(NSURLConnection *)connection 
    didReceiveResponse:(NSURLResponse *)response
{
    NSHTTPURLResponse* httpResponse;
    httpResponse = (NSHTTPURLResponse*)response;
    int statusCode = [httpResponse statusCode];
    if (statusCode != 200)
    {
        NSMutableDictionary* info;
        info = [NSMutableDictionary dictionaryWithObject:[response URL] 
            forKey:NSErrorFailingURLStringKey];
        [info setObject:@"An error code different than 200 was received!" 
            forKey:NSLocalizedDescriptionKey];
        NSError* error = [NSError errorWithDomain:@"API Wrapper" 
            code:statusCode 
            userInfo:info];
        NSDictionary* errorData = [NSDictionary 
            dictionaryWithObject:error 
            forKey:@"error"];

[[NSNotificationCenter defaultCenter]
            postNotificationName:@"ConnectionDidFailWithStatusCodeNotOK"
            object:self
            userInfo:errorData];
    }
    [receivedData setLength:0];
}

-(void)connection:(NSURLConnection *)connection 
    didReceiveData:(NSData *)data
{
    [receivedData appendData:data];
}

-(void)connection:(NSURLConnection *)connection 
    didFailWithError:(NSError *)error
{
    [connection release];

NSDictionary* errorData = [NSDictionary 
                                dictionaryWithObject:error 
                                forKey:@"error"];
 
    [[NSNotificationCenter defaultCenter]
        postNotificationName:@"ConnectionDidFailWithError"
        object:self
        userInfo:errorData];

NSLog(@"Connection failed! Error - %@ %@",
          [error localizedDescription],
          [[error userInfo] objectForKey:NSErrorFailingURLStringKey]);
}

-(void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    [[NSNotificationCenter defaultCenter]
        postNotificationName:@"ConnectionDidFinishLoading"
        object:self];
    [connection release];
}

-(void)setAsynchronous:(BOOL)asynchronously
{
    async = asynchronously;
}

-(NSDictionary*)getPropertyList
{
    NSString* errorStr = nil;
    NSDictionary* propertyList;
    NSPropertyListFormat format;

propertyList = [NSPropertyListSerialization 
                    propertyListFromData:receivedData
                    mutabilityOption: NSPropertyListImmutable
                    format: &format
                    errorDescription: &errorStr];
    return propertyList;
}

-(NSString*)getResponseText
{
    return [[NSString alloc] 
            initWithData:receivedData 
            encoding:NSUTF8StringEncoding];
}

@end

Rather verbose this last one huh? Objective-C has a distinctive characteristic, inherited from Smalltalk (I suppose): methods with named parameters, like “stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding” (decidedly, one of the longest method calls ever :) The good thing is that the code reads like a book. This is why I like this language, as well as Ruby: you can read code in these languages very easily.

Another nice aspect of this Objective-C wrapper is that if you have an API that can return “Property Lists” (like the one I’ve been working on lately), you can send object graphs serialized in XML that are completely Cocoa-compatible. That’s what the getPropertyList method does: client code can deal with standard NSDictionary instances, no need to do anything else!

Of course these are not the only ways to do this, but so far these wrappers have helped me a lot. Don’t hesitate to leave your comments below! And most important, have fun! As always, use this code at your own risk.

Similar Posts:

Tagged Code · How to?

36 Comments

#1. Yoan
03.27.2008

httplib has AIDS in it. http://code.google.com/p/httplib2/
#2. Chase
06.27.2008

Could you post some sample code on implementing your Obj C sample codes? I am new to Obj C, and am frustrated with not being able to make a simple HTTP Get request… and be able to read what the returned XML. Thank you!
#3. Adrian
06.27.2008

Hi Chase,
In the comment section of the ObjC wrapper above you can find a quick sample of how to use the class:

#import
int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool;
pool = [[NSAutoreleasePool alloc] init];

AKResourceSubClass* resource;
resource = [[AKResourceSubClass alloc] init];
[resource get];
NSDictionary* plist = [resource getPropertyList];
// Do something with the results…

[pool drain];
return 0;
}

Take into account that to read generic XML you should use the [resource getResponseText] message. This class works best (and is easier to use) if your XML follows the plist (property list) specs from Apple…
#4. Adrian
06.27.2008

I forgot to add that you should subclass the AKResource class, and the “get:” method should be implemented there, calling the sendRequestTo:usingVerb:withParameters: mother class method, with the required parameters, of course.
#5. Chase
06.27.2008

Adrian, wow, that was fast man! I know C# and AS3 really well, but I am not wrapping my brain around Obj C very fast. I tried for 2 hours last night to get your code to get a readable response, but it was a no go. Could you post some a simple implementation that subclasses AKResource and sends (via get) a few vars to a php file and shows the response?
#6. BINU
07.24.2008

Hi, Could you please add complete code for Objective-C , I need to post some data to website and get the result, If you addd these code it will be useful for me…I am a beginner
#7. dbfreq
07.25.2008

I can’t get the Obj-C version to work using the sample code you provided for the command line tool. It says that ‘get’ is an unrecognized selector, which seems to makes sense, since the ‘get’ method is neither declared nor defined. What should this method be? Also, the command line sample code declares AKResourceSubClass *resource, but the name of the class is actually AKResource.

I’m trying to use a wrapper like this. I want one class to construct the string to be sent, invoke this wrapper to send the string and then post the response back to the sending method so it can parse the response and update the UI. NSLogs show that the class responsible for updating the UI is logging the string that is supposed to represent the response before the HTTP Post actually receives a response. How do I make my string construction class wait for a response from the HTTP wrapper before logging to the console?
#8. sanjay
07.26.2008

i am facing a problem that how will we chek internet connection is connected or not on objective-c
#9. Simon
08.17.2008

Adrian, nice demonstration code, I have a query though.
I had a challenge working with POST verbs. Shouldn’t a POST verb be multipart/form-data in place of text/html?
I ended up building two cases into your code, one for POST verbs and left the other for the remainder. I suspect I’ve just not got the hang of the REST programming, but if you could shed some light on what the receiver needs to do when you initiate a POST using your code without a form based context, It would be appreciated.

FYI I built receivers using ruby and php, and both fell over when using your code with a POST verb, as they couldn’t see any parameters being passed (were expecting a form).

Cheers
Simon
#10. Adrian
08.17.2008

Hi Simon, thanks for your comments. Indeed this code is quite old now :) I’ve used it in a project, sending image data in HTTP POST requests and I had to tweak some stuff of course. I will publish that code as soon as I can!
#11. iDeveloper
10.16.2008

Hi Adrian. How can I send a SOAP request in a similar manner??
#12. Adrian
10.17.2008

Sorry, no idea about SOAP (yuck!) but you might want to check this out: http://developer.apple.com/samplecode/CocoaSOAP/index.html
#13. iDeveloper
10.17.2008

Hi Adrian.

I managed to change your code and send in the SOAP request instead of parameters as the request body. Thanks :)
#14. Adrian
10.17.2008

Cool! Indeed, after all, SOAP is based on HTTP… ;)
Good work and glad that my code could help you.
#15. Simon
10.18.2008

iDeveloper, would you consider sharing your learnings regarding modifying the code to support SOAP requests? I’d be interested to see your approach.

Adrian, any chance of seeing that update to your code regarding the POST case you mentioned a couple of months back in response to my earlier query? I have to admit my hack of your original code still leaves me wondering if I’m handling it correctly.

Regards Simon
#16. iDeveloper
10.18.2008

Hi Simon
I am using the Objective-C wrapper. The only change I did is instead of passing parameter values as body of the http request, I am sending in NSData object which reads the SOAP request from a file and setting that as the body.
#17. Adrian
10.18.2008

Hey guys :) I love when my blog becomes a place to chat. I have added some code to the ObjC wrapper above, and also corrected some memory leaks here and there ;) (thanks Instruments!). The new version includes file uploading, which might prove interesting to many others. I’ll post that today, hang tight!
#18. Adrian
10.18.2008

I’ve just uploaded a new version of the Objective-C REST client, together with a small desktop Cocoa app showing how to use it, here:
http://kosmaczewski.net/projects/objective-c-rest-client/
Some new features:
- delegate protocol replacing the NSNotification structure used before;
- removed that pesky “get” method that was puzzling everyone :)
- added the upload code, which has been tested used in production apps already;
- removed memory leaks (there might be a few more, I hope not!);
- added @property statements for some fields;
- the sample project shows how a controller class uses the wrapper to send requests and receive data later, the whole thing is pretty simple to use I think.
Enjoy!
#19. iDeveloper
10.21.2008

Hey Adrian

Any idea on how we can stream audio over the internet using Objective-C? The documentation’s pretty confusing! :(
#20. Adrian
10.21.2008

You read my thoughts or what??? :))) I’m actually looking for information about that right now :) since UDP is the required underlying transport for that (correct me if I’m wrong), using the NSURL* classes won’t do it (HTTP works over TCP/IP), and then I think that the libraries are a bit *below* from what we expect :))) I think the solution is down there, bsd sockets and so. We’ll post more as soon as I have some more info.
#21. iDeveloper
10.24.2008

Hi Adrian… any progress towards the audio streaming?? here’s a good link http://idevkit.com/forums/general-sdk/225-streaming-audio-class-4.html though it has some glitches….
#22. iDeveloper
11.03.2008

Hi Adrian
Was tryin to use the wrapper class.if we need to make multiple async calls,is it possible to know the request for which the data was returned?
#23. Adrian
11.03.2008

well, you could send and receive an identifier with each request (the server must echo), or you could use several wrapper instances, with a different delegate each time. I prefer this last option, if you want my opinion.
#24. iDeveloper
11.04.2008

Thanks. Ended up using different wrappers for each request!
#25. iDeveloper
11.10.2008

Hi Adrian. Am getting a strange problem when sending a request to a URL. The code has the URL set in correct, but the server just receives half the URL. Any reasons why that would happen?!?!? I’m baffled!
#26. Simon
11.16.2008

Adrian,
Once again, thanks for posting the code and updates. Migrating to a delegate method at the moment. I can see the advantages. I’m still somewhat hung up on your sendRequestTo method. I’m going to go out on a limb and say I don’t think it’s all encompassing (yet), that is, works for all verbs, in particular POST with form data. If I’m wrong please let me know and I’ll study it again to try and figure it out.

I do see in your example (in the comment history), the uploadData method so I see a form data based method, (though not with a set of parameters) It is a POST example that seems to be almost what you need for generic form POST requests.

In your sendRequestTo, you set
[headers setValue:@"text/html; charset=utf-8" forKey:@"Content-Type"];

That’s great for GET, but for POST form data won’t work.

Also adding parameters via:
for (id key in parameters)
{
[params appendFormat:@"%@=%@&", [key stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding], [[parameters objectForKey:key] stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
}

is also great for GET, but not for POST (unless you want url paramters in addition to form parameters).

I have (pulled from some other code elsewhere) some form parameter construction code of the following nature:
NSString *stringBoundary = [NSString stringWithString:@"0xKhTmLbOuNdArY"];
if (parameters)
{
NSMutableData *postBody = [NSMutableData data];
[postBody appendData:[[NSString stringWithFormat:@"--%@",stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
for (id key in parameters)
{
[postBody appendData:[[NSString stringWithFormat:@"\r\nContent-Disposition: form-data; name=\"%@\"\r\n\r\n",
key] dataUsingEncoding:NSUTF8StringEncoding]];
[postBody appendData:[[NSString stringWithFormat:@"%@",
[parameters objectForKey:key]] dataUsingEncoding:NSUTF8StringEncoding]];
[postBody appendData:[[NSString stringWithFormat:@"\r\n--%@",
stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
}
//Add the final components of the string.
[postBody appendData:[[NSString stringWithFormat:@"--\r\n",
stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
[request setHTTPBody:postBody];

Yes, this is a bit rough, but it seems to work OK.

Am I missing something here? Do you have the capability to post a form, or have you only developed an example where you can use a POST verb, but effectively only with URL parameters, similar to a GET verb?

Cheers
Simon
#27. Adrian
11.16.2008

Hi Simon, thanks to you for the work you’ve done in your comment! Indeed this version is my first attempt to make a REST wrapper for Objective-C code, you might want to check out this one:
http://kosmaczewski.net/2008/10/18/rest-and-objective-c-again/
It’s a more polished version, and it’s being used not only in my own projects but also in a friend of mine’s :) And as far as I can tell, POST requests work fine, but of course I might not be doing it properly ;) Thanks for the “heads up” and I’ll fix the code as soon as I can to make it more HTTP-compliant.
#28. Simon
11.16.2008

Adrian, Just had a scan through. Looks good. Will try it out. Am wondering if the method used for POSTing form data via URL encoding ends up putting a restriction on how much data you are able to submit as I recall there being a limit on the length of a URL (though I don’t know what it is). Perhaps this is a REST interaction characteristic that I don’t yet understand. Again thanks
Cheers
Simon
#29. Adrian
01.19.2009

@Simon: please check http://kosmaczewski.net/2009/01/19/objective-c-rest-client-update/ as I’ve fixed the formatting of data sent via POST… it turns out that the method I was using was wrong, and CoreFoundation has a better way to handle URL formatting.
#30. iPhone SDK, Cocoa & RESTful Web Services, Memory Leak « Ambient Ideas
02.01.2009

[...] A Blog about 5 languages calling web services, including Objective-C [...]
#31. Peter
06.14.2009

If you guys need to debug, run REST or any HTTP calls there is an Eclipse plugin where you can visually construct your REST call: http://http4e.roussev.org/

Of course this is not an Objective-C client, yet it is very powerful tool.
#32. Matthew Gravatt
07.22.2009

Thank you, Adrian. Extremely useful code and thread.
#33. sandrar
09.11.2009

Hi! I was surfing and found your blog post… nice! I love your blog. :) Cheers! Sandra. R.
#34. Adrian
09.11.2009

Thanks Sandra! :)
#35. Jeffrey Leo Oloresisimo
02.22.2010

Hi, i really like this wrapper class you created but i have a question…. how could i do it in a secure HTTP? im trying to implement a secure HTTP RESTful web service but when I use this project to test connection between my server it does not work.

I would really appreciate the help….
#36. Rajendra Bhole
03.13.2010

I was using some HTTP REST-ful services from above code. But i faced serious problem in that, the above code will work fine with Wi-Fi connectivity but it crashes using GPRS connection after web service called.There is any extra coding should i want to be written for GPRS?

Playing with HTTP libraries

March 26, 2008

36 Comments

Commenting

Living

Searching

Tweeting

Reading

Bookmarking

Writing

Tagging

Remembering