|From Spring Microservices in Action by John Carnell
This article discusses client-side resiliency patterns intended to help make your app more robust and resistant to failure.
What are client-side resiliency patterns?
Client resiliency software patterns are focused on protecting a remote resource’s (another microservice call or database lookup) client from crashing when the remote resource is failing because it’s throwing errors or performing poorly. The goal of these patterns is to allow the client to “fail fast”, not consume valuable resources like database connections and thread pools, and prevent the problem of the remote service from spreading “upstream” to the client’s consumers.
Four client resiliency patterns:
- Client-side load balancing
- Circuit breakers
Figure 1 demonstrates how these patterns sit between the microservice service consumer and the microservice.
Figure 1 The four client resiliency patterns act as a protective buffer between a service consumer and the service
These patterns are implemented in the client calling the remote resource. The implementation of these patterns logically sits between the client consuming the remote resources and the resource itself.
Client-side load balancing
Client-side load balancing involves having the client look up all of a service’s individual instances from a service discovery agent (like Netflix Eureka) and then caching the physical location of those service instances. Whenever a service consumer needs to call that service instance, the client-side load balancer returns a location from the pool of service locations it’s maintaining.
Because the client-side load-balancer sits between the service client and the service consumer, the load balancer can detect if a service instance is throwing errors or behaving poorly. If the client-side load balancer detects that there is a problem, it can remove that service instance from the pool of available service locations and prevent any future service calls from hitting that service instance.
This is exactly the behavior that Netflix’s Ribbon libraries provide “out of the box” with no extra configuration.
The circuit breaker pattern is a client resiliency pattern which is modeled after an electrical circuit breaker. In an electrical system, a circuit breaker detects if too much current is flowing through the wire. If the circuit breaker detects a problem, it breaks the connection with the rest of the electrical system and keeps the downstream components from the being “fried.”
With a software circuit breaker, when a remote service is called, the circuit breaker monitors the call. If a call takes too long, the circuit breaker intercedes and “kills” the call. In addition, the circuit breaker monitors all calls to a remote resource, and, if enough calls fail, the circuit break implementation will pop, “failing fast” and preventing future calls to the failing remote resource.
With the fallback pattern, when a remote service call fails, rather than generating an exception, the service consumer executes an alternative code path and tries to carry out an action through another means. This usually involves looking for data from another data source or queueing the user’s request for future processing. The user’s call isn’t shown an exception indicating a problem, but they may be notified that their request will be fulfilled later.
For instance, suppose you have an e-commerce site that monitors your user’s behavior and tries to give them recommendations of what else they could buy. Typically, you might call a microservice to run an analysis of the user’s past behavior and return a list of recommendations tailored to that specific user. If the preference service fails, your fallback might be to retrieve a more general list of preferences based off all user purchases. This data might come from a completely different service and data source.
The bulkhead pattern is based on a concept from shipbuilding. With a bulkhead design, a ship is a divided into completely-segregated and watertight compartments called bulkheads. The bulkhead keeps the water confined to the area of the ship where a puncture has occurred and prevents the entire ship from filling with water and sinking, because the ship is divided into watertight compartments (bulkheads), even if the hull is punctured.
The same concept can be applied to a service that interacts with multiple remote resources. By using the bulkhead pattern, we can break the calls to remote resources into their own thread pools and reduce the risk that a problem with one slow remote resource call takes down the entire application. The thread pools act as the bulkheads for your service. Each remote resource is segregated and assigned to the thread pool. If one service responds slowly, the thread pool for that type of service call becomes saturated and stops processing requests. Service calls to other services won’t become saturated because they’re assigned to other thread pools.
That’s all for this article.