Introducing URLMock

Testing networking code is hard. The web service you’re connecting to can fail in myriad ways, from basic networking errors to malformed responses. All but the simplest APIs return a wide variety of data that your app has to handle correctly. Sometimes APIs can be so complex that it’s hard to verify that you’re calling things in the right order, much less sending appropriate parameters and handling responses correctly.

There aren’t a lot of good solutions out there. The best seems to be to write a special test server that responds to API requests with canned responses so you can test your code. Because writing the app and the test server is high on your list of things to do. Realistically, you’ll probably do what most developers do: forgo thorough testing and hope for the best. Let’s all agree that this isn’t the best strategy.

A better strategy would be to use a native library to transparently intercept network requests inside your app and respond with data or errors for testing. You could then verify that you sent the correct requests and handled all the various responses and error conditions correctly. Since the library is native, you could use it for either manual or automated testing environments. If only that library existed…

URLMock is an open-source Cocoa framework for mocking and stubbing URL requests and responses, brought to you by the fine folks at Two Toasters. It works seamlessly with apps that use the NSURL loading system, e.g., NSURLConnection or AFNetworking. You don’t even have to change your client code. Just tell URLMock how to respond to a URL request, and it takes care of the rest.

In this post, we’re going to show you how you can use URLMock in your networking code’s automated tests. We’re going to focus on HTTP, but URLMock can be used with other protocols and for manual testing as well. Check out the readme for more details.

Core Concepts

URLMock revolves around three basic types of objects:

  • Mock requests describe the URL requests that URLMock should intercept. Mock HTTP requests are represented by UMKMockHTTPRequest objects, which intercept requests that match a given URL, HTTP method, and request body. Parameters can be checked as well.
  • Mock responders respond to intercepted URL requests. UMKMockHTTPResponder instances can respond with data or an NSError.
  • Finally, UMKMockURLProtocol is the primary interface to the URLMock system. After registering mock requests with it, it intercepts your app’s URL requests and responds to them as you specified. It also does the bookkeeping so that you can check if any mock requests were unserviced or any unexpected requests were received.

Unit Testing with URLMock

We’re going to write tests for a simple OpenWeatherMap API client. The full example project is available on GitHub.

Also, URLMock includes a lot of test helpers to make testing easier. We’ll be using them liberally throughout the example, so take a look at the header to see what the different UMK* functions do.

The code we’ll be testing is ‑[TWTOpenWeatherMapAPIClient fetchTemperatureForLatitude:​longitude:​success:​failure:], which fetches the current temperature for the specified location.

- (NSOperation *)fetchTemperatureForLatitude:(NSNumber *)latitude
                                   longitude:(NSNumber *)longitude
                                     success:(void (^)(NSNumber *))successBlock
                                     failure:(void (^)(NSError *))failureBlock
{
    NSParameterAssert(latitude && ABS(latitude.doubleValue) <= 90.0);
    NSParameterAssert(longitude && ABS(longitude.doubleValue) <= 180.0);

    return [self.operationManager GET:@"weather"
                           parameters:@{ @"lat" : latitude, @"lon" : longitude }
                              success:^(AFHTTPRequestOperation *operation, id response) {
                                  if (successBlock) {
                                      successBlock([response valueForKeyPath:@"main.temp"]);
                                  }    
                              } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
                                  if (failureBlock) {
                                      failureBlock(error);
                                  }
                              }];
}

Keen-eyed readers may notice some potential bugs in this implementation. Let’s see how it’s tested in TWTOpenWeatherMapAPIClientTests.m:

- (void)testFetchTemperatureForLatitudeLongitude
{
    __block NSNumber *temperature = nil;
    [self.APIClient fetchTemperatureForLatitude:@(35.99)
                                      longitude:@(-78.9)
                                        success:^(NSNumber *kelvins) {
                                            temperature = kelvins;
                                        }
                                        failure:nil];

    // Assert that temperature != nil before 2.0s elapse
    UMKAssertTrueBeforeTimeout(2.0, temperature != nil, @"temperature isn't set in time");
}

This is a pretty bad test. It checks that an object is returned, but doesn’t check if the API client gets the right value out of the API response. It also doesn’t test error conditions, like what happens when the API request fails or the response is malformed. Let’s see if URLMock can help.

Enabling URLMock

Before updating our tests, we need to tell URLMock to intercept URL loading requests. We do this using +[UMKMockURLProtocol enable]. We should also enable verification, which allows us to check if we received any unexpected requests or didn’t receive any expected ones. We can do these both in our XCTestCase’s +setUp method:

+ (void)setUp
{
    [super setUp];
    [UMKMockURLProtocol enable];
    [UMKMockURLProtocol setVerificationEnabled:YES];
}

When we’re done testing, we should undo what we did in +setUp so we don’t inadvertently interfere with our other testing code. The ideal place for this is in +tearDown.

+ (void)tearDown
{
    [UMKMockURLProtocol setVerificationEnabled:NO];
    [UMKMockURLProtocol disable];
    [super tearDown];
}

Finally, before we run our individual tests, we want URLMock to be in a clean state. We can accomplish this by invoking +[UMKMockURLProtocol reset] in our ‑setUp method.

- (void)setUp
{
    [super setUp];
    [UMKMockURLProtocol reset];
    …
}

Now that we’re set up, let’s actually write some tests. We’ll write one that tests success with good data, one that tests failure due to a network error, and one that tests failure due to a malformed JSON response.

A Better Test for Success

To ensure that ‑fetchTemperatureForLatitude:​longitude:​success:​failure: succeeds with good data, we first need to test that it accesses the correct API endpoint with the correct parameters:

- (void)testFetchTemperatureForLatitudeLongitudeCorrectData
{
    NSNumber *latitude = @12.34;
    NSNumber *longitude = @-45.67;
    NSNumber *temperature = @289.82;

    NSURL *temperatureURL = [self temperatureURLWithLatitude:latitude longitude:longitude];

Here we get the URL from a helper method in our test class. It’s important to note that we don’t ask the API client for the URL to use. If we were in the habit of trusting the API client author to always do things correctly, we wouldn’t be writing unit tests.

Now we need to create a mock request to match the URL request that the API client’s method will access. The request should be an HTTP GET and have no HTTP body. We can create a mock request like so:

    UMKMockHTTPRequest *mockRequest = [UMKMockHTTPRequest mockHTTPGetRequestWithURL:temperatureURL];

Next, let’s tell URLMock how to respond when it sees this request. HTTP servers typically use a status code of 200 to indicate success, so our response should do that too. Also, the body of the response should be JSON that looks like { "main": { "temp": «Temperature» } }. We can mock this up really easily:

    UMKMockHTTPResponder *mockResponder = [UMKMockHTTPResponder mockHTTPResponderWithStatusCode:200];
    [mockResponder setBodyWithJSONObject:@{ @"main" : @{ @"temp" : temperature } }];
    mockRequest.responder = mockResponder;

That last line is really important. It tells the mock request which responder to use when it’s serviced by URLMock. Speaking of which, we need to register our mock request with UMKMockURLProtocol.

    [UMKMockURLProtocol expectMockRequest:mockRequest];

We’re done registering our mock request and responder. While that wasn’t hard, it took more code than I’d like. URLMock includes some helper methods to make this easier. Using one of those, we can reduce those previous lines to a single method invocation:

    [UMKMockURLProtocol expectMockHTTPGetRequestWithURL:temperatureURL
                                     responseStatusCode:200
                                           responseJSON:@{ @"main" : @{ @"temp" : temperature } }];

That’s much better. So, we’ve prepped URLMock to expect an HTTP GET request for the endpoint URL and respond with the JSON body above. Now we need to actually invoke the API client method and verify that it behaves correctly.

    __block BOOL succeeded = NO;
    __block BOOL failed = NO;
    __block NSNumber *kelvins = nil;
    [self.APIClient fetchTemperatureForLatitude:latitude
                                      longitude:longitude
                                        success:^(NSNumber *temperatureInKelvins) {
                                            succeeded = YES;
                                            kelvins = temperatureInKelvins;
                                        }
                                        failure:^(NSError *error) {
                                            failed = YES;
                                        }];

    UMKAssertTrueBeforeTimeout(1.0, succeeded, @"success block is not called");
    UMKAssertTrueBeforeTimeout(1.0, !failed, @"failure block is called");
    UMKAssertTrueBeforeTimeout(1.0, [kelvins isEqualToNumber:temperature], @"incorrect temperature");

This code fetches the temperature and saves away the response. We make sure that the success block is called, the failure block isn’t, and that the temperature passed to the success block is the one we sent using our mock responder.

As a final step, we should make sure that no additional URL requests get made. To do this, we use +[UMKMockURLProtocol verifyWithError:]:

    NSError *verificationError = nil;
    XCTAssertTrue([UMKMockURLProtocol verifyWithError:&verificationError], @"verification failed");
}

And that’s it. If you run this test, it should pass. Next, let’s test some error conditions.

Responding with Errors

Responding to requests with an error object is trivial:

- (void)testFetchTemperatureForLatitudeLongitudeError
{
    …
    NSURL *temperatureURL = [self temperatureURLWithLatitude:latitude longitude:longitude];
    [UMKMockURLProtocol expectMockHTTPGetRequestWithURL:temperatureURL responseError:[self randomError]];
    …
}

This code should be pretty self-explanatory. We generate a random error using a test case helper method and tell URLMock to respond with that. To verify that the API client handles it properly, we do the following:

    __block BOOL succeeded = NO;
    __block BOOL failed = NO;
    [self.APIClient fetchTemperatureForLatitude:latitude
                                      longitude:longitude
                                        success:^(NSNumber *temperature) {
                                            succeeded = YES;
                                        }
                                        failure:^(NSError *error) {
                                            failed = YES;
                                        }];

    UMKAssertTrueBeforeTimeout(1.0, !succeeded, @"success block is called");
    UMKAssertTrueBeforeTimeout(1.0, failed, @"failure block is not called");

    NSError *verificationError = nil;
    XCTAssertTrue([UMKMockURLProtocol verifyWithError:&verificationError], @"verification failed");

This is a lot like the success case, but we make sure that the failure block is called and that the success case isn’t. And that’s it. If we run this test, it should pass too.

For our final test, let’s respond to the request with malformed data.

Malformed Data, Malformed Method

Our test for malformed data is a blend between the previous two. We register a mock response that responds with some random JSON.

- (void)testFetchTemperatureForLatitudeLongitudeMalformedData
{
    …
    NSURL *temperatureURL = [self temperatureURLWithLatitude:latitude longitude:longitude];
    [UMKMockURLProtocol expectMockHTTPGetRequestWithURL:temperatureURL
                                     responseStatusCode:200
                                           responseJSON:UMKRandomJSONObject(3, 3)];
    …

We then test that it fails as before:

    __block BOOL succeeded = NO;
    __block BOOL failed = NO;
    [self.APIClient fetchTemperatureForLatitude:latitude
                                      longitude:longitude
                                        success:^(NSNumber *temperature) {
                                            succeeded = YES;
                                        }
                                        failure:^(NSError *error) {
                                            failed = YES;
                                        }];

    UMKAssertTrueBeforeTimeout(1.0, !succeeded, @"success block is called");
    UMKAssertTrueBeforeTimeout(1.0, failed, @"failure block is not called");
}

If you run this test, there’s a high likelihood the API client method will crash. The API client code contains the line [response valueForKeyPath:@"main.temp"], but it doesn’t validate that response is a dictionary that contains the required keys. If it’s not, the method could crash in ‑valueForKeyPath:. Fixing this bug and getting the test to pass is left as an exercise for the reader.

Conclusion

That’s just a taste of what you can do with URLMock. We’ve used it on a few client projects over the last few months and uncovered some pretty significant bugs in our error handling code. We’ve got a major release planned soon that will hopefully improve it and make it even more flexible. Until then, install it, play with it, and consider using it on your next project!

Tweet about this on Twitter