Netflix Hystrix Circuit Breaker

Netflix Hystrix Circuit Breaker

In one of my previous blogs, I have already discussed the Circuit breaker pattern and its usage, Today, we will see how can we implement it in our application using Spring Cloud Netflix Hystrix.

In this document, I’ll walk you through the process of applying circuit breakers to potentially-failing method calls using the Netflix Hystrix fault tolerance library.

Hystrix is watching methods for failing calls to related services. If there is such a failure, it will open the circuit and forward the call to a fallback method.

To understand it in a better way, I’ll take the same problem statement that I have already discussed in my previous blog i.e. E-commerce Portal. 

I am assuming you must be aware of Spring boot framework as this implementation is completely based on it.

As per problem statement, we need two applications i.e. Product Service & Price Service but in this tutorial, I’ll talk about just Product Service as Price Service is simple web application exposing a single API and you can clone it from GitHub.

I’ll talk about Product Service in which we will integrate Circuit breaker pattern and Cache Service implementation and internally, it will invoke Price Service to get the Price Detail.

Optional: You can download the below project from GitHub or you can try it by creating a new one. I would recommend you to try it from scratch.

Product Service:

Create Spring boot application and add below dependency to your pom.xml file.

<!-- Optional. Application health monitor check -->

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-actuator</artifactId>

</dependency>

<!-- Compulsory for circuit breaker implementation -->

<dependency>

<groupId>org.springframework.cloud</groupId>

<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>

</dependency>

<!-- Optional. Dashboard in case you want to see the circuit state UI -->

<dependency>

<groupId>org.springframework.cloud</groupId>

<artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>

</dependency>

To enable Circuit Breaker in spring boot application, add @EnableCircuitBreaker annotation on product-service entry-point class. 

Now use @HystrixCommand annotation on themethod we want to apply timeout and fallback method.

I have annotated @HystrixCommand(fallbackMethod = “getProductDetailFromCache”) on getProduct(String Id) so that if it doesn’t receive the response within the certain time limit or request get failed while calling price service API then fallback() should get called over here and fetch the price value from Cache Service. 

Make sure, the fallback method should be defined in the same class and should have the same signature.

We can customize the @HystrixCommand default behavior by configuring properties using @HystrixProperty annotations. Will check in depth in another blog. 

Add RestController class and other required class as per source code present in GitHub.

Add server.port to the application.properties file.

Start both applications and try to fetch the product Detail. In our case, our product-service app URL is:

http://localhost:7001/products/1

Now try to fetch product which does not exist

You will observe that over here circuit breaker is still in the closed state even after the exception. 

Why? The reason is that we have added  ignoreExceptions = { ProductNotFoundException.class } to @HystrixCommand so that it should not trip circuit if the product is not present.

Now, Stop the price-service application and hit the product URL again:

Check the log:

Congratulations! Our application is working as expected.

Monitoring Circuit Breakers using Hystrix Dashboard

Hystrix comes with a decent dashboard where we can monitor the status of Hystrix commands

To enable it, Add Hystrix dashboard dependency to the pom.xml file 

<!-- Optional. Application health monitor check -->

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-actuator</artifactId>

</dependency>

<!-- Optional. Dashboard in case you want to see the circuit state UI -->

<dependency>

<groupId>org.springframework.cloud</groupId>

<artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>

</dependency>

Add @EnableHystrixDashboard annotation on the entry point class

Add management.endpoints.web.exposure.include=hystrix.stream to application.properties file.



Start the product-service application and then go to http://localhost:port/hystrix to view the dashboard. 

In Hystrix Dashboard home page enter http://localhost:7001/actuator/hystrix.stream as stream URL and give Product Service as Title and click on Monitor Stream button. 

Now, we can see that the Circuit status along with how many calls succeed and how many failures occurred, etc. 

Spring-boot microservices can be downloaded from GITHUB

Happy Coding...!!!

Circuit Breaker and Microservices Architecture

Circuit Breaker and Microservices Architecture 

In this article, I’ll talk about the Circuit Breaker pattern which is widely used in Microservices but before jumping to Circuit breaker design pattern or microservices. Let’s first understand the requirement so that you should know where we can use it in our application and Of Couse, How 😊

Let’s understand it with the real scenario as It’s already June month and we know there are so many online summer sales is going to start where they will offer you a various type of deals on their products.

And as we know most of the online portal has upgraded their application arch from Monolithic design to Microservices like Amazon, Flipkart, Snapdeal and so on and with that their each service will be running as an independent micro-services in the cloud like Authentication Service using which a user can be logged in to the application, Product Detail Service, Billing Service, Price Service, Card Services, etc.

Now just imagine the scenario where millions of users are doing online shopping on some online portal and because of load or some other technical issue, their price services goes down for some time. So, what will happen over here if a customer can’t order anything?

From a business point of view: it will give the bad impact of the sales as well as it will also decrease sales. Right?

From a technical point of view: All the request going to Price Service will fail eventually and failure in one part of the system might lead to cascading failures and all the future request will be blocked until the timeout expires and there are high chances that these blocked requests might hold critical system resources such as memory, threads, database connections and so on. Consequently, these resources could become exhausted, causing the failure of other possibly unrelated parts of the system that need to use the same resources.

How to solve it?
One simple approach is to return the cached price if the price service goes down and the original price once the price system available.

How can we achieve the above solution?
We can achieve this approach easily by implementing a circuit breaker pattern that allows a fallback when one of the applications goes down.

The concept of Circuit breaker is very similar to automatically operated electrical panel switches of our home which goes down after the fault detected (either an electrical storm or power surge) and can be reset (either manually or automatically) to resume normal operation after the fault is involved.

The Circuit breaker pattern helps to prevent such a catastrophic cascading failure across multiple systems. It allows us to build a fault-tolerant and resilient system that can survive gracefully when key services are either unavailable or have high latency.

The concept of circuit breaker is straightforward and has three distinct states:

1. Closed: When everything is normal, the circuit breaker remains in the closed state and all calls pass through to the services according to its lifecycle until it encounters a failure and when the number of failures exceeds a predetermined threshold the breaker trips and the circuit breaker will change the state to the Open state.
2. Open: When the breaker in its open state then the request will not go the service but will acts as a backup plan like throwing error directly or in our case, get the response from the Cache service.
3. Half-Open: A limited number of requests from the application are allowed to pass through and invoke the operation. If these requests are successful, it's assumed that the fault that was previously causing the failure has been fixed and the circuit breaker switches to the Closed state (the failure counter is reset). If any request fails, the circuit breaker assumes that the fault is still present so it reverts back to the Open state and restarts the timeout timer to give the system a further period of time to recover from the failure.

Advantages:

1. Helps to add logic for a fault tolerant system like try to get the data from some other source or from the cache.
2. Handle downtime and slowness of services gracefully
3. It will switch back to the main service once the service is again available automatically.

In my next blog, I'll talk about how can we implement circuit pattern using Spring Boot.