What is API load testing?
What’s an API, anyway?
API stands for Application Programming Interface. In simple terms, an API is a set of rules for how to communicate with an application.
These rules are necessary so that the application server can understand requests and process them accordingly. When a user accesses a web app through a browser, his or her clicks on the page and other interactions actually send requests to the server. APIs allow you to bypass those clicks in the UI and just get the information directly by sending requests.
How do you know what format you can use to formulate your requests in? That’s where API documentation comes in. Let's have a look at an example.
Poké API is a collection of data about Pokémon, and their site includes a link to the Poké API documentation:
GET <https://pokeapi.co/api/v2/pokemon/>{id or name}
This is how you should formulate a request in order to retrieve information from their database. In this example:
GET is the method (or ”verb”) that describes what kind of request you’re making
https://pokeapi.co/api/v2 is the base URL of the API
/pokemon is the endpoint we’re using and describes where the data we want is kept
{id or name} is extra information we want to send along with our request
There are a few ways to actually consume (use) this API. In this case, let’s say we want to do a search in the database for details on the pokemon Vaporeon. Using the format of the request above, we can use this request:
GET <https://pokeapi.co/api/v2/pokemon/vaporeon
You can test a simple GET request like this by typing https://pokeapi.co/api/v2/pokemon/vaporeon in your browser’s URL bar and hitting enter. You’ll likely see a LOT more information about Vaporeon than you ever wanted to know. To see a formatted version of all this data, use Poké API’s simple interface and type in /pokemon/vaporeon:
This returns the same data, but formatted better.
You can also make the same requests using cURL via the command line or Postman for a friendlier interface.
Because the request is in a format that the application server understands, the application server replies and responds with the resources for that page. So the API defines a language and syntax that clients (such as web browsers) need to use in order to interact with an application.
There are several types of APIs, including SOAP, REST, JSON-RPC, and XML-RPC. Each one has a different standardized format for communicating with the web application. RESTful APIs (REST stands for REpresentational State Transfer) are particularly popular because of their simplicity. The Poké API is a RESTful or REST API. However, the principles discussed in this book will apply to load testing any API.
How to do API load testing
APIs can also present a more efficient way to load test your web application.
Load testing is determining the traffic that you can expect from production and applying it systematically to your application servers in order to determine how the application behaves. It's a way to check whether your application is robust enough to handle the traffic you want it to handle. Load testing is one type of performance testing, which is sometimes referred to as non-functional testing.
In API load testing, instead of your browser sending messages like GET <https://pokeapi.co/api/v2/pokemon/vaporeon>, you can use testing tools that will do that for you at scale. There are commercial and open source tools that you can use to create test scripts that you can use to tell those tools which messages to send and when. With API load testing, you can simulate multiple concurrent users sending requests to your server at the same time.
There are several steps to executing an API load test. Feel free to jump around to the respective sections for each step if you would prefer.
pagePlanning for API load testingpageScripting an API load testpageExecuting an API load testpageAnalyzing results and reportingpageAPI load testing and Continuous IntegrationIt's also important to be clear up-front about the advantages and disadvantages of API load testing so that you know whether it's the type of testing you need.
Disadvantages of API load testing
API load testing does not simulate real users interacting with elements of your webpage.
In contrast to a customer opening up a browser and filling out forms on your site, an API load testing script will consist only of the underlying requests to the server that are made by clicking on those on-screen elements. Exactly how the customer triggers those requests, and what buttons they’ve pressed, is irrelevant. It’s all about the raw requests.
It doesn’t give you an idea of how user-friendly your application is.
API load testing scripts don’t give you feedback on how long your pictures took to render on your users’ browsers or whether that “SUBMIT” button is in an obvious spot.
It doesn’t measure front-end performance or how quickly pages render in different browsers.
While all the resources that the server returns can be downloaded by your API load testing tool, there is no browser on which to run them.
API load testing doesn’t run client-side scripts.
It isn’t what you’re looking for if your application is a single-page web app that relies heavily on JavaScript or AJAX to dynamically populate and update the page. Your load testing tool will download the scripts, but not execute them.
This includes scripts like those required to trigger Google Analytics, which prompts many an engineer to question the results of an API load test due to the inability to see the traffic come in on Google Analytics. API load testing won’t help you with that. Running browser-level load tests using tools like Flood Element, Selenium or Tricentis Tosca may be more useful.
Advantages of API load testing
Now that we’ve talked about what API load testing isn’t good at, let’s talk about what it IS good at.
It allows you to load test specific servers rather than the whole stack.
API load testing allows you to tailor your load testing by applying load only on particular servers. This is particularly useful in more complicated applications that involve several components and would require substantial effort to reproduce in a test environment. Using APIs, you can test only the functions you want to test.
It is well-supported.
API load testing has been around for decades. There are lots of robust tools you can choose from, both commercial and open-source, and many of these tools have large communities and extensive documentation around how to script the most common cases. It’s a far cry from the browser-level testing space, which is relatively new and sparsely populated by comparison.
It’s not as resource-intensive as browser-level testing.
You can simulate more users with API load testing than with browser-level load testing. Since most interactions on the browser-level get translated to requests anyway, generating load this way is incredibly efficient, allowing you to hit your server with requests while bypassing the UI layer and the resource overhead that that entails.
For example, at Flood we’ve baselined some of the tools that we support to see how many users we can run on an AWS m5.xlarge instance:
Selenium: 5 users Flood Element: 20-40 users JMeter: 1000 users Gatling: 1000 users
You’ll see that the browser-level test tools, Selenium and Flood Element, can run significantly fewer users on the same sized node as can the protocol-level tools, JMeter and Gatling. This isn’t due to an inefficiency in the tools but rather to the fact that browser-level tools inherently need more CPU and memory to start separate instances of browsers per user and render pages graphically. You can run more users per node (or machine) by carrying out API load testing with protocol-level tools.
It’s cheaper.
The efficiency in resource utilisation translates directly into cost savings because every node that you don’t need to execute your tests on is one fewer node that you’ll have to pay to provision (whether on premises or in the cloud).
For this reason, API load testing is one of the most cost-efficient ways you can get started with load testing, allowing you to scale up your load relatively cheaply while getting immediate results.
Good API load testing will generate load that is a good representation of load you could expect to see in production, giving you an accurate estimate of how your application architecture will behave and highlighting any performance bottlenecks that can be fixed prior to go-live.
The rest of the book
In this book, we’ll go over everything you need to know to plan, script, execute and analyse your API load tests and even how to add load testing into your Continuous Integration framework.
Last updated
