Architecture Design for APIs
Building Efficient, Scalable, and Secure Interfaces for Modern Applications
Designing APIs requires more than the simple act of developing endpoints:
APIs are complex pieces of software that require robustness and stability in order to satisfy all the demands of one or more client applications that require the transformation and retrieval of secure data stored in a database.
In this post, we’ll cover the architectural pieces involved in the design and development of APIs, and you’ll see how there are many more considerations beyond the simple development of endpoints hooked up to a database.
This guide does not intend to create a simple, ready-to-go API. If the intention of the reader is such, there is a much more agile approach to architecture design, which skips multiple steps at the expense of creating a less efficient and scalable system. Our approach here focuses more on creating a solid foundation based on good design principles in order to incrementally build upon our original decisions.
1. Define the Purpose and Requirements
Before diving into the code, we’ll showcase how requirements planning takes place based on a simple practical example of an API intended to store, retrieve and transform data related to books for a bookstore.
Practical Example
If you were building an API for a bookstore, we’d begin by defining the actions that take place. Actions are defined by behaviors that the web app might need.
Some actions may include:
Allowing users to browse books
Adding new books to our collection
Updating specific book details
Deleting or hiding books
Managing user accounts for the people that can either access or modify the books collection
2. Choose the Right Architectural Style
There is no law of nature that states how API architecture design must be done. However, with the advent of the Internet in the 1990s and 2000s a conceptual framework/protocol with guidelines was established and continues to be the most prevalent way to design APIs in the modern day. This framework is known as the REST framework, which defines a specific way to design our endpoints.
REST
REST focuses on two primary principles:
Stateless: Each request from a client contains all the information needed to process the request - this means that a request does not contain state on the server about it. It is the responsibility of the request to provide the required information for the action that it wants to achieve, including unique identity, permissions, authentication, authorization, and the required payload data needed to create/update/delete data on the server.
Resources: Everything is a resource, identified by URIs. A URI is nothing more than a unique identity to a specific entity instance. For example, each of the Harry Potter books would have its own unique URI.
Example
With that in mind, a REST API pushes atomicity and simplicity in its design.
GET calls retrieve data, either for an entire entity (books) or a specific entity instance (book item, based on a URI)
POST calls are used to create new information
PUT calls are used to modify existing information, given a URI
DELETE calls are used to destroy existing information, giving a URI
GET /books
POST /books
PUT /books/:id
DELETE /books/:id3. Design the API Endpoints
RESTful Endpoints
Given how REST APIs work, we’ll define our book actions based on them:
Resources: Identify the books entity
CRUD Operations: Create, Read, Update, Delete
Example:
GET /books
GET /books/:id
POST /books
PUT /books/:id
DELETE /books/:id4. Authentication and Authorization
If we were to write our endpoints now and publish the server, our API would be complete and available to use. However, it would be completely exposed for anyone to interact with it. While initially they may not have the correct inputs required to manipulate the server data in the correct way, APIs are often designed with error messages in mind for bad inputs. This means that with some maneuvering, time and reverse engineering, a bad actor could utilize our server/API to alter sensitive information stored in our database.
To prevent this, all secure APIs implement authentication and authorization mechanisms to protect them from requests that are not from desired parties.
Authentication refers to the ability to access protected endpoints/actions to retrieve or manipulate data on the server - it is simply the process of identifying whether you are a good user or bad user
Authorization refers to the ability of having the permissions to go through with the retrieval or manipulation of data that you desire. At times, you (the agent with its keys) may have authentication access but may not have the permissions required to manipulate certain objects. For example, authenticated user 1 would not have permissions to alter data of authenticated user 2. In this case, each user’s permissions are limited to their own user/instance.
Practical Example
JSON Web Tokens are a standard that specify how you can implement stateless authentication for an API
They utilize request headers, passing secret information back and forth between a user and an API, and private-public key cryptography in order to verify whether a client is authenticated
In this implementation method, every request must include a valid token
Example
The example below is for an API request received. It contains, as input, a JSON web token (JWT), which is just a long-string of random characters
The JWT is used against a secret stored on the server to decrypt a secret object - and if the object is successfully decrypted, it means that we authenticate a user. Otherwise, we return an error that the user is not authenticated.
const jwt = require('jsonwebtoken');
function authenticate(req, res, next) {
const token = req.header('Authorization').replace('Bearer ', '');
if (!token) return res.status(401).send('Access denied');
try {
const decoded = jwt.verify(token, 'your_jwt_secret');
req.user = decoded;
next();
} catch (ex) {
res.status(400).send('Invalid token');
}
}5. Error Handling
Proper error handling ensures your API can gracefully handle unexpected situations and provide useful feedback to the client.
Node, for example, provides middleware to allow you to centralize and maintain error handling as a singular module inside your API
You can also create more targeted error handling, but the global middleware provides a mechanism to ensure that our application never crashes, even in cases where there are unexpected errors in our app, often caused by bugs
Example:
app.use((err, req, res, next) => {
console.error(err.stack);
res.status(500).send({ error: 'Something went wrong!' });
});6. Versioning
Since APIs evolve over time, versioning is an implementation strategy to allow gradual upgrades and changes to your APIs
Changing stable endpoints to new code can be problematic, especially in scenarios where you don’t have test-driven development strategies in place. When major upgrades come along which can significantly disrupt an already functioning system, they present new risk to a business and its technology.
A method of mitigating this is by writing API implementations based on a givern version, primarily oriented for major versions. This allows you to gradually upgrade specific endpoints independently of one-another, allowing certain endpoints to be on newer versions and others on previous versions
Practical Example
Use versioning in your URLs:
GET /v1/books
GET /v2/books
7. Rate Limiting
Rate limiting is more an advanced concept that you will encounter as your application begins to scale in its usage
Rate limiting is primarily a topic of discussion when you have implemented public APIs for 3rd parties to be able to interact with their underlying data without the use of the UI and through a programmatic approach
Rate limiting prevents these 3rd parties from abusing the system and ensuring that they respectfully use the system in a manner that is within the limits of your organization and the requirements or plans that users fall under
Example
Use a middleware to limit the number of requests from a single IP
const rateLimit = require('express-rate-limit');
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
message: 'Too many requests from this IP, please try again later.'
});
app.use('/api/', limiter);8. Documentation
Finally, documentation is something that is typically ignored or pushed off to the side, especially when teams are frantically moving in agile cycles between one sprint and the next. However, strong documentation can significantly improve the quality of your code, the availability of others to contribute to it, as well as provide opportunities for automated testing and pipelines in deployment.
The two most common tools in API documentation are Swagger or Postman.
Swagger Example:
swagger: '2.0'
info:
version: 1.0.0
title: Bookstore API
paths:
/books:
get:
summary: Get all books
responses:
200:
description: A list of books
schema:
type: array
items:
$ref: '#/definitions/Book'
definitions:
Book:
type: object
properties:
id:
type: string
title:
type: string
author:
type: string