Faking the API

If you're building a web application and you want to follow the single-page app design pattern - because, let's face it, this is the new hotness in web design - you will probably end up with an API whose primary responsibility is to grab data from the database, possibly manipulate parts of it, and return it in a consumable format. If you're doing this alone or your app is small, then that pattern will work for you pretty well. But if you're doing a big project as part of a larger team, especially in a split stack - i.e. frontend and backend developers work on different parts of the code - then allow me to suggest a better way. I honestly thought at first that the concept of a fake API was . . . bizarre. But having worked on a web application that used one for close to two years now, I'm hooked, to the point that even for a solo project where I'm writing all the parts of code, I think I would probably use one. There are just too many advantages to pass up.

But before I get into what makes a fake API useful, there may be some who are still wondering, "Yeah, but what is it?" A fake API, at least for Manta, is a set of server routes that mirrors a real API, but returns fake, static data. There are a lot of different ways you could set this up, but we use JSON files. So for example, if we have a production API route at /api/v1/foo/bar, then our frontend node server will have an express-defined route with the same path that does nothing except return a json file. We even have a generic helper to make this a simple one liner:

router.get('/api/v1/foo/bar', common.load('foo/bar'));  

This helper returns a function that matches the typical express middleware signature, so that when the route is invoked, the express middleware reads in the right file and returns it as JSON. Here's a super over-simplified version of that helper:

common.load = function(filePath) {  
  // This is the function that actually gets invoked
  // when the route is hit.
  return function(req, res, next) {
    // root being the directory where the JSON files live
    var contents = require(root + filePath);

It actually does a lot more than this, as it's functionality has expanded over time, but this should be sufficient to grasp how our fake API works.

There's at least one downside to this particular pattern, namely that reducing redundancy can be challenging, but there are some ways around this. In fact that's part of the "expanded functionality" our generic helper handles. Amongst other things, it recursively interpolates certain patterns, so that common parts of our JSON can be abstracted into other files. But even without this enhancement, there are plenty of other reasons to prefer using a fake API to a real one.

Parallel Development

Having a fake API in our frontend infrastructure means that our team is able to work in parallel with the API team, increasing our . . . WARNING: Agile software development buzzword ahead . . . velocity. We are not tethered to the API and dependent on the completion of a service or route to test (or even write in some cases) code that we're working on. If we know that we're going to need data about bananas, we'll agree up front with the API team on where the route to collect this data will live, and then we'll create a matching fake route that returns fake data with virtually no development overhead - just one line to set up the route and then whatever JSON we need. Then we can start right away building out whatever logic we need in our frontend server and constructing the HTML and Angular code to finish the frontend piece; meanwhile the API developers are composing a service that talks to the database and returns real information about bananas. Of course, we have to communicate closely with the API team to confirm the data contract, so that we know that the data in our fake API (the data we're expecting) matches what we actually get back. But after some practice, this becomes fairly trivial. You would expect to have to have these kinds of discussions even if you weren't using a fake API.

Decoupled Design

Additionally, faking the API allows us to focus on UI/UX design, rather than data format. If, after trying to bridge the gap between real API and frontend, we discover that we need the data in a different format, we decide collectively (frontend and API teams both) whether those changes belong on the frontend or in the API. Typically, we ask ourselves, "Would another consumer of this API, other than our own frontend, expect to receive this data in this format or that format?" And that helps us decide where any logic to map data into a different format belongs. But I digress. The point is that we are not making decisions about the design of our frontend components based on the structure of the data coming to us. We can build it right, independent of actual data, and then cover the gaps later.

Easy contract definition and testing

Because we're creating actual, physical JSON files, it's pretty easy for us to define the contract for an API. We can just grab the Github link to the JSON file for a route and send it to an API developer as a way of saying, "Here's what we're expecting. Let us know if that doesn't work." There are artifacts of our expectations. Additionally, this makes it straightforward (I say this, but we haven't actually done this yet) to add contract tests to assure that either the real or fake API doesn't change in an unexpected way. We have fake data served by a route and they have real data served by a route. Simply comparing the schemas of the two responses will tell us where there are gaps.

It's in git

Another benefit of the actual, physical nature of fake JSON is that it's part of our source control. We have a history of every change made to our fake data by virtue of having it in git. We can easily revert a change or see when an inconsistency was introduced. Of course, you can do this with a database too, but you have to do it explicitly. Audit tables don't write themselves; and the results can be difficult to consume.

No database setup

And in general, there is a lot less database to worry about. In our case, none. Our frontend application literally does not have a database. I'm in danger of forgetting how to interact with them. If we hire a new developer, as soon as they clone our application, they have everything they need to get started. They don't have to worry about installing, managing permissions for, starting, restarting, stopping . . . etc. etc. etc. . . . any database. There are no worries about whether they learned MySQL and here we are using PostgreSQL. Everyone (I think) knows JSON now. It's the universal language of developers. On an increasingly less significant note, flat JSON files take up considerably less space than a database too.

No database cleanup

I have only one time worked on a project where the tests generated actual database entries which therefore had to be cleaned up as part of the test teardown. I know this is a popular pattern, but it's one I don't understand. Our fake routes don't ever write anything, so the data is always the same. We rarely have any test teardown, and if you're a developer who's spent any time at all writing tests, you know that the less friction you have with writing tests the better. If you have to write a giant tedious teardown function for a test, you're more likely to say, "Forget it" than if you have 0 test teardown to write.

Predictable test data

And because our fake data never changes, it's completely predictable. If our fake banana route says the color of the banana is "yellow," it's always going to be yellow. I don't have to worry about whether someone else happened to hit the update route and set it to green and forgot to change it back. This is especially great for tools that delete things. We can test that a thing was deleted, but as soon as you refresh the page, it's there again in it's same pristine state.

No data corruption

And similarly, we don't have to worry about accidentally writing to the wrong table or database. It's probably somewhat rare in well-controlled environments for test data to make it into a production table, but let's not pretend it's impossible. But we don't have to worry about our fake data being written to production because a) there's no database and no writing and b) our fake routes don't even get set up in production so they couldn't do any writing there even if they did use a database.

Dynamic route stubbing

Finally, perhaps my favorite advantage of a fake API is that you can set up dynamic routes to serve different things. Meaning, at run time you can alter the response of a particular route, because the route is fake and the data it returns is fake. We have a few additional API endpoints in our fake API that accept a path, an http method, response data, and some other things and create new express route handlers that get inserted into the current list of handlers. Why would you ever need this? As previously mentioned, one of the cons of the JSON file fake API approach is the redundancy of putting similar data into multiple files. We often have moments where we need company data, but we need the company industry code to be different, or we need member data, but specifically for someone whose last name is long, and it can be extremely repetitive to create new JSON files every time you need a single field to change. So we have a few development tools that let you modify the response data of an existing route, so that if you want to log in as "Bob the Builder," but with no member image, you can just unset the member image field dynamically without affecting other developers.

Great, how do I get started?

I don't intend to delve into this too deeply in this post, as 1) that's not the point of this post, and 2) this post is long, but a tl;dr version is:

  1. Setup multiple environments (development and production at the least) that have their own configuration.
  2. Include the host for your API in the environment configuration. The production one will point to the real API, but in dev, have it point back to your own (local) server. We add our dev routes (all mapped at /api) to our middleware stack when we are in the dev environment and then send API responses to the same host (localhost:8000 in our case). A side benefit of this is that it's possible to do a lot of frontend work even if your internet goes out. You're never making any actual network requests.
  3. Use some config manager when making API requests. We use nconf, so when we request something from the API, the url gets mapped to something like nconf.get('API_HOST') + '/api/v1/foo/bar'.

That's about it. If you now start your server in the right environment - e.g. in node: NODE_ENV=development node server - all API requests will be mapped to your local server instead, where you can stub whatever routes you need.

Tell Me More!

Perhaps I shall in another post. Our fake API actually does a number of cool things, and as I mentioned, we have some great development tools for working with them. There's plenty that would be worth sharing. But for now, I think I've exhausted my thoughts on the topic.

comments powered by Disqus