From API Security in Action by Neil Madden
This article covers the definition of an API and what it means for an API to be secure.
Application Programming Interfaces (APIs) are everywhere. Open your smartphone or tablet and look at the apps you have installed. Almost without exception those apps are talking to one or more remote APIs to download fresh content and messages, poll for notifications, upload your new content, and perform actions on your behalf.
Load your favorite web page with the developer tools open in your browser, and you likely see dozens of API calls happening in the background to render a page which is heavily customized to you as an individual (whether you like it or not). On the server, those API calls may themselves be implemented by many microservices, communicating with each other via internal APIs.
Increasingly, even the everyday items in your home are talking to APIs in the cloud—from smart speakers like Amazon Echo or Google Home, to fridges, electricity meters, and lightbulbs. The Internet of Things (IoT) is rapidly becoming a reality in both consumer and industrial settings, powered by ever-growing numbers of APIs in the cloud and on the devices themselves.
Although the spread of APIs is driving ever more sophisticated applications that enhance and amplify our own abilities, they also bring increased risks. As we become more dependent on APIs for critical tasks in work and play, we become more vulnerable if they’re attacked. The more APIs are used, the greater their potential to be attacked. The property that makes APIs attractive for developers, ease of use, also makes them an easy target for malicious actors.
This article is about how to secure your APIs against these threats allowing you to confidently expose them to the world.
Taking your driving test
You finish work at 5pm as usual, but today is special. Rather than going home to tend to your carnivorous plant collection and then flopping in front of the TV with a pre-peeled avocado and tasting platter of organic pea shoots, you have somewhere else to be. Today you’re taking your driving test.
You rush out of your office and across the park to catch a bus to the test center. As you stumble past the queue of people at the hot dog stand, you see your old friend Alice walking her pet alpaca, Horatio.
“Hi Alice!” you bellow jovially, “How’s the miniature recreation of eighteenth century Paris coming along?”
“Good!” she replies. “You should come and see it soon.”
She makes the universally recognized hand-gesture for “call me” and you both hurry on your separate ways.
You arrive at the test center a little hot and bothered from the crowded bus journey. If only you could drive, you think to yourself! After a short wait, the examiner comes out and introduces himself. He asks to see your learner’s permit and studies the old photo of you with the ill-advised haircut you thought was pretty cool at the time. After a few seconds of quizzical stares, he eventually accepts that it’s you, and you can finally begin the test.
Oh dear, it doesn’t go well. Better luck next time. I think that may be the first time a driving test was failed due to a UFO sighting, but if you’d hit the brake rather than the gas… You decide it’ll be quicker to take the train home from the test center.
At the station you buy a standard class ticket back to your suburban neighborhood, but feeling a little devil-may-care, you decide to sneak into the first-class carriage. Unfortunately, an attendant blocks your way and demands to see your ticket. Meekly you scurry back into standard class and slump into your seat with your headphones on.
When you arrive home, you see the light flashing on your answerphone. Huh, you’d forgotten you even had an answerphone. It’s Alice, inviting you to the hot new club that recently opened in town. You could do with a night out to cheer you up, and you decide to go.
The doorwoman takes one look at you.
“Not tonight,” she says with an air of sniffy finality.
At that moment, a famous celebrity walks up and is ushered straight inside. Dejected and rejected you head home.
What you need is a vacation. You book yourself a two-week stay in a fancy hotel. While you’re away, you give your neighbor Bob the key to your tropical greenhouse; he can feed your carnivorous plant collection. Unknown to you, Bob throws a huge party in your back garden and invites half the town. Thankfully, due to a miscalculation they run out of drinks before any real damage is done (except to Bob’s reputation) and the party disperses. Your prized whisky selection remains safely locked away inside the liquor cabinet in the basement.
When you return from your trip, you review the footage from your comprehensive (some might say over the top) camera surveillance system. You cross Bob off the Christmas card list and make a mental note to ask someone else to look after the plants next time.
The next time you see Bob you confront him about the party. He denies it at first, but when you point out the cameras, he admits everything. He buys you a lovely new Venus fly trap to say sorry. It may not immediately seem like it, but almost every element in that story relates to an aspect of API security.
Unless you develop software for self-driving cars, it’s unlikely that your API has taken its driving test, but like the protagonist, your API needs to interact with people, devices, and services. As with Alice, sometimes there’s a long-standing trust relationship based on a history of previous interactions, although in other cases a more formal proof of identity is required, like showing a driving license. The examiner trusts the license because it’s issued by a trusted body, and you match the photo on the license.
Beyond identifying your users, an API also needs to be able to decide what level of access they should have. This can be based on who they are, like the celebrity getting into the club, a limited-time token like a train ticket, or a long-term key like the key to the greenhouse that you lent your neighbor. Each approach has different trade-offs. A key can be lost or stolen and then used by anybody. On the other hand, you can have different keys for different locks (or different operations) allowing only a small amount of authority to be given to somebody else. Bob could get into the greenhouse and garden but not into your house and whisky collection.
Finally, the video cameras show the advantage of having good audit logs which allow you to find out who did what when things go wrong, and if necessary, prove who was responsible in a way they can’t easily deny.
Definition An audit log records details of significant actions taken on a system, with which you can later work out who did what and when. Audit logs are crucial evidence when investigating potential security breaches.
You can hopefully now see a few of the mechanisms which are involved in securing an API, but before we dive into the details let’s review what an API is and what it means for it to be secure.
What is an API?
Traditionally, an API was provided by a software library which could be linked into an application either statically or dynamically at runtime, allowing reuse of procedures and functions for specific problems, such as OpenGL for 3D graphics, or libraries for TCP/IP networking. Such APIs are still common, but a growing number of APIs are now made available over the internet as RESTful web services.
Broadly speaking, an API is a boundary between one part of a software system and another. It defines a set of operations that one part of the system provides for other parts of the system (or other systems) to make use of. For example, a photography archive might provide an API to list albums of photos, to view individual photos, add comments, and so on. An online image gallery could then use that API to display interesting photos, although a word processor application could use the same API to allow embedding images into a document. As shown in figure 1, an API handles requests from one or more clients on behalf of users. A client may be a web or mobile application with a user interface (UI), or it may be another API with no explicit UI. The API itself may talk to other APIs to get its work done.
Figure 1 An API handles requests from clients on behalf of users. Clients may be web browsers, mobile apps, devices in the Internet of Things, or other APIs. The API services requests according to its internal logic and then at some point returns a response to the client. The implementation of the API may require talking to other “backend” APIs, provided by databases or processing systems.
A user UI also provides a boundary to a software system and restricts the operations that can be performed. What distinguishes an API from a UI is that an API is explicitly designed to be easy to interact with by other software, although a UI is designed to be easy for a user to interact with directly. Although a UI might present information in a rich form to make the information pleasing to read and easy to interact with, an API typically presents instead a highly regular and stripped-back view of the raw data in a form which is easy for a program to parse and manipulate.
This article focuses on APIs exposed over HTTP using a loosely RESTful approach, as this is the predominant style of API at the time of writing. Although the APIs in this article try to follow REST design principles, you’ll sometimes deviate from those principles to demonstrate how to secure other styles of API design. Much of the advice applies to other styles too, and the general principles even apply when designing a library.
API security in context
API Security lies at the intersection of several security disciplines, as shown in figure 2. The most important of these are the following three areas:
- Information security (or InfoSec) is concerned with the protection of information over its full life-cycle from creation, storage, transmission, back-up, and eventual destruction.
- Network security deals with both the protection of data flowing over a network, and prevention of unauthorized access to the network itself.
- Application security ensures that software systems are designed and built to withstand attacks and misuse.
Figure 2 API security lies at the intersection of three security areas: network security, application security and information security.
Each of these three topics has filled many books individually, but we won’t cover each of them in full depth. As figure 2 illustrates, you don’t need to learn every aspect of these topics to know how to build secure APIs. We’ll instead pick the most critical areas from each and blend them to give you a thorough understanding of how they apply to securing an API.
From information security it’s important to know how to:
- Define your security goals and identify threats
- Protect your APIs using access control techniques
- Secure information using applied cryptography
Definition Cryptography is the science of protecting information of two or more people who can communicate without their messages being read or tampered with by anybody else. It can also be used to protect information written to disk in which case it may be the same person reading it at a later time.
From network security it’s important to understand:
- The basic infrastructure used to protect an API on the internet, including firewalls, load-balancers, and reverse proxies, and roles they play in protecting your API
- Use of secure communication protocols such as HTTPS to protect data transmitted to or from your API
Definition HTTPS is the name for HTTP running over a secure connection. Although normal HTTP requests and responses are visible to anybody watching the network traffic, HTTPS messages are hidden and protected by Transport Layer Security (TLS, also known as SSL).
Finally, from application security you need to know:
- Secure coding techniques
- How to test software for security vulnerabilities
- How to store and manage system and user credentials used to access your API and others
A typical API deployment
An API is implemented by application code running on a server. It is rare to directly expose such a server to the internet, or even to an internal intranet. Instead, requests to the API typically passes through one or more additional network services before they reach your API servers, as shown in figure 3. Each request passes through one or more firewalls, which inspect network traffic at a relatively low level and ensure that any unexpected traffic’s blocked. For example, if your APIs are serving requests on port 80 (for HTTP) and 443 (for HTTPS), then the firewall is configured to block any requests for any other ports. A load balancer routes traffic to appropriate services and ensures that one server isn’t overloaded with lots of requests while others sit idle. Finally, a reverse proxy is typically placed in front of the application servers to perform computationally expensive operations like handling TLS/SSL encryption and validating credentials on requests.
Figure 3 Requests to your API servers typically pass through several other services first. A firewall works at the TCP/IP level and only allows traffic in or out of the network that matches expected flows. A load balancer routes requests to appropriate internal services based on the request and on its knowledge of how much work each server is currently doing. A reverse proxy can take care of expensive tasks on behalf of the API server, such as terminating HTTPS connections or validating authentication credentials.
Beyond these basic elements, you may encounter a number of more specialist services:
- An API gateway is a specialized reverse proxy that can make different APIs appear as if they were a single API. It’s often used within a microservices architecture to simplify the API presented to clients. API gateways can often also take care of some of the aspects of API security discussed in this article.
- A web application firewall (WAF) inspects traffic at a higher level than a traditional firewall and can detect and block many common attacks against HTTP web services.
- An intrusion detection system (IDS) or intrusion prevention system (IPS) monitors traffic within your internal networks. When it detects suspicious patterns of activity it can either raise an alert or actively attempt to block the suspicious traffic.
Definition In a microservices architecture, an application is deployed as a collection of loosely-coupled services rather than a single large application, or monolith. Each microservice exposes an API that other services talk to.
In practice, there’s often some overlap between these services. For example, many load balancers are also capable of performing tasks of a reverse proxy, such as terminating TLS connections, while many reverse proxies can also function as an API gateway. Some more specialized services can even handle many of the security mechanisms and it is becoming common to let a gateway or reverse proxy handle at least some of these tasks. These components have limits to what they can do, and poor security practices in your APIs can undermine even the most sophisticated gateway. Understanding the basic security mechanisms being used by these products helps you assess whether a product’s suitable for your particular application, and exactly what its strengths and limitations are.