Showing posts with label Spring. Show all posts
Showing posts with label Spring. Show all posts

Tuesday, May 5, 2020

Creating first Jenkins pipeline: tutorial


Jenkins uses a feature called Jenkins Pipeline which is a collection of jobs that brings the software from version control into the hands of the end-users by using automation tools. They represent multiple Jenkins jobs as one whole workflow in the form of a pipeline.

In this blog, I am going to share my knowledge on how can we write multiple Jenkins jobs as a pipeline and it uses two different syntaxes i.e. Declarative and Scripted pipeline and in our examples, we're going to use the Scripted Pipeline which is following a more imperative programming model built with Groovy.


Prerequisite:
  • Code on bitbucket/GitHub
  • Jenkins Installation
  • Download required plugins to run pipelines like Pipeline, SonarQube Scanner, Check Style, Junit, Git Integration, Maven Integration.
  • Sonar up and running. 
Let’s start creating pipeline will do below tasks:
  • Clone Project from Jenkins
  • Build and run Junit test cases
  • Run Sonar
  • Run Checkstyle
  • Package it as a jar file


 
Configuration Steps: 
  • Let's create new Jenkins jobs. Goto Jenkins -> New Item
  • Add name under 'Enter an item name', Select pipeline as the type, and click Ok button.

  • I am skipping the description and others tab here and directly jumping to the Pipeline tab as I already discussed it in my previous blog and we can run pipeline without worrying about it.
  • Add below script and check Use Groovy Sandbox and Save it.

node {
// clone the project from Github
    stage('Clone'){
    git 'https://github.com/abdulwaheed18/demo.git'
}
//Build the project
stage('Build'){
  sh "mvn clean install"
}
// Run Sonar for Code Coverage
       // Ignore this stage if sonar instance is not present
stage('Sonar') {
sh "mvn sonar:sonar"
}
// Run code check
stage("Checkstyle") {
        sh "mvn checkstyle:checkstyle"
         
        step([$class: 'CheckStylePublisher',
          canRunOnFailed: true,
          defaultEncoding: '',
          healthy: '100',
          pattern: '**/target/checkstyle-result.xml',
          unHealthy: '90',
          useStableBuildAsReference: true
        ])
    }
    //package the application
     stage('Package') {
         junit '**/target/surefire-reports/TEST-*.xml'
         archiveArtifacts 'target/*.jar'
   }
}



  • To configure Sonarqube URL, Goto Jenkins -> Manage Jenkins -> Configure System and set Server URL and save it. 
  • You can see your newly created pipeline on the Jenkins dashboard

  • Click on Jenkins-pipeline-demo and then on the right side, click on Build now to build the project, to start the Jenkins pipelines.

  • Once your job is completed, you will see below screen 

  • As the final job was packing as the jar. you can see a blue downward arrow button clicking on which will download your application as a JAR file.
  • you can check the logs by clicking on the blue circle button on the left side or you can hover over a stage cell and click the Logs button.

  • To Check Sonar report, goto Sonar Server URL that you configured it. It will show you total code coverage, unused import, and bad code.

  • We had also added the Checkstyle stage to the pipeline so to check the report. Click on the Checkstyle Warning present below build now link.

  • Here we see 12 High Priority Warning browsable by clicking it. The Details tab gives you more insight into each class error. 
Conclusion :
We are able to set up a simple Jenkins pipeline to show code pull, build, to run sonar, and other code analysis tools, and as always the source code used in this project can be found over on Github.


Friday, November 8, 2019

Streaming Spring boot logs to ELK stack

In my previous blog, we have done ELK installation on windows 10 and we have even tried to push messages from input console to Elastic Search and finally viewed on Kibana Server.

I will write a separate blog on why do we need ELK?

In this blog, I’ll show you how can we push spring boot application log directly to Elastic search using Logstash which we can analyze on Kibana and If you don’t know how to install ELK on windows 10 then you can refer my previous blog and start Elastic Search and Kibana server.

Prerequisite


  • Elastic Search and Kibana running on your machine
  • Basic knowledge of Spring boot application


If you don’t want to start your application from scratch then you can download one spring boot application from my GitHub repository as well.

I am assuming that the Elastic Search and Kibana server are running on your machine and you have a fair idea of how to start the Logstash server and what is Logstash conf file.

So, to push spring boot logs continuously to Elastic Server, We have to open one TCP port in Logstash server and for that we have to create one Logstash config file (say elklogstash.conf) under ${LOGSTASH_HOME}/conf directory mentioning on which port TCP port should be listening under input filter and where to push the data once we received under Output filter.

For simplicity, I am skipping the filter tag as it is optional.

elklogstash.conf




Now start the Logstash server bypassing newly created conf file.
   bin\logstash -f .\config\elklogstash.conf



Cool! Now Logstash server is also up and running and if you observe the log, you will realize that it is also listening on port 4560 as mentioned in the conf file. Configure the newly created index (elkbootlogs) on Kibana as we have done during the ELK setup.

Now let's do some changes to spring boot application so that it can push all the logs to 4056 TCP port.

For this tutorial, I am using spring-logger project from my Github repository.

Add below dependency to the pom.xml file. We need Logstash encoder to encode messages.

<!-- Added for logstash Encoder-->
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>6.2</version>

</dependency>

Open logback-spring.xml file which is under the resource folder and create new appender (say elk). The task of this appender is to push logs to the destination TCP socket and under this appender, compulsory use LogstashEncoder.

<appender name="elk" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>localhost:4560</destination>
    <!-- encoder is required -->
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />

</appender>

Add new appender to root level

<!-- LOGGING everything at INFO level -->
<root level="info">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
<appender-ref ref="elk" />
</root>

Save all files and start your application. So, we are done with all the setup. Its time to check whether all the changes are done properly or not.

Open Kibana on your browser (http://localhost:5601) and select your index under the Discover tab. You will see all logs are populating on Kibana as well.



Congratulations! Our configuration is working absolutely fine and it is pushing logs to Elastic Search. 

You can download the source code from here, ELK code chnages are under elkstack branch.






Sunday, November 3, 2019

How to create Docker Image and push java app in a Docker Engine

In this blog, I am going to share my knowledge on the creation of a docker image and how can we run in a Docker Engine.

Prerequisite

  • Basic Knowledge of Docker
  • Docker must be running on your machine.
  • Good to aware of Spring boot application.
I already have one spring boot application in my IntelliJ which expose one endpoint /users/{id}. We will see how can we push and run this application in a docker container. 



We need to create one file named Dockerfile to add docker instruction (Check above image).


Now go to Terminal and check whether the docker is running or not on your machine.


Run docker build to create an image and push it to the container using the command.
docker build -f Dockerfile -t docker-spring-ehcache .

The above command will execute all the operations that we have mentioned in our Dockerfile like pulling OpenJDK 8 from the docker hub if not exist.

Let's see if our image got pushed to docker containers or not by listing all docker images.
docker images


Great! Our image is present in the docker container. Let's run it.
docker run -p 8070:8085 docker-spring-ehcache

Over here, we are telling the Docker to start the application and map docker container port 8085 to our local port 8070. 

Note: Make sure your application has started at 8085 in docker container else it won't be able to map it. Spring boot by default start application on port 8080 so please specify server.port-8050 in application.properties file.



Now go to your browser and hit the endpoint and see if we are getting the response from localhost:8070 or not.



Happy Coding!!!

Sunday, September 8, 2019

Spring Boot Tutorial

Prerequisite: Basic knowledge of Spring boot application

I am working on a series of implementing frameworks with Spring boot application but not getting enough time to blog it and post it here or on my LinkedIn profile.

So, I have started uploading my work on my GitHub repository from where it can be downloaded easily. I tried my best to add short notes for each annotation/configuration/properties in README and even I have uploaded a few screenshots to understand in a more better way.

Try it out and Please do let me know in case of any confusion.
  1. Spring Boot Actuator
  2. Spring Boot Ehcache
  3. Spring Boot Swagger
  4. Spring Boot JPA

Will keep uploading with others framework as well.

Feedback is also most welcome.

Thank you.
Happy Learning!

Thursday, August 29, 2019

MicroServices: Spring cloud ribbon with Discovery Server

In this article, I am going to share my knowledge on Spring Cloud Ribbon and how can we use Ribbon with RestTemplate as well as with Feign Client. We will also see how Enabling discovery Sever will improve the scalability of Microservice.

Before jumping into Spring Cloud, I am assuming you must be having knowledge of Eureka Server, Feign Client, and Client-Side load balancer. If not then read my below blog before jumping to Spring Cloud ribbon. Also, I am going to use my existing code to implement Ribbon.

URLs:
In my previous blog, I have already talked about the Eureka Server and how other applications are taking advantages of Eureka Server to fetch the host/port of client application.

We have seen that three microservices application are up and running i.e.
  • Discovery Server
  • Product Service
  • Price Service
Where Product and Price service will register themselves to Discovery Server and Product-service will always communicate to Discovery Server to get the exact location of Price-service and then only it will talk to the price-service application.

Imagine there is a high load on the price-service application and to handle it, we have started two more instances of price-service.

  • How will you make sure your product-service should talk to all three instances of price-service and divides the load equally to each server?
  • How will you manage the heartbeat of the application so that product-service should not hit INACTIVE instance of price-service which just got shut down because of some internal reason?
  • How will you get to know how many instances are up and running of price-service?
For all the question, there is only answer which is Netflix Cloud Ribbon. It's a Spring cloud library which primarily provides client-side load balancing algorithms.

let's implement it and see how it can solve our problem.

Add below dependency to product-server pom.xml file as it is the one which is going to consume price-service API.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-ribbon</artifactId>
</dependency>


Case 1: Ribbon + Eureka Server + RestTemplate

As our application is already acting as Eureka Client and using RestTemplate to fetch price record. Let's just add @LoadBalanced annotation to RestTemplate to enables Ribbon functionality.

It will allow the product-service application to use price-service as the address of price-service application and will discover the host/port of all instances of price-service from discovery-server.



Note:

@EnableCircuitBreaker <-> Used to enable Netflix Hystrix
@EnableHystrixDashboard <-> Used to check circuit state on Dashboard.
@EnableFeignClients <-> To enable feign client (Needed for case 2 scenario)

Start discovery-server, product-service and start two instances of price-service. You can do it easily by just overriding port number under Eclipse Run As Configuration.



Let's confirm it, whether all the instances are up and running or not by hitting discovery-server URL (http://localhost:8761/)



Now hit product-service URL multiple times from the browser and then go and check both the price-service logs. you will observe that few requests are coming to one instance and others on second one.



Great! Netflix Cloud Ribbon is successfully implemented and working absolutely fine.

If you start another price-service instance and hit the product-service URL again then you will find the request logs in the third instance too without doing any modification/configuration to any files.


Case 2: Ribbon + Eureka Server + Feign Client

If you are not aware of the feign client then you can read my blog here.

I'll just talk about Ribbon Integration with existing FeignClient application assuming you are already aware of Feign Client and implemented it.

In product-service application, I have already exposed another Endpoint (http://localhost:7001/products/feign/1) which consume price-service API using Feign Client.

To enable Netflix Ribbon, Add @RibbonClient annotation to the feighClient interface and pass your consuming service name (price-service).



Now restart your application and hit new endpoint. you will observe that the requests are distributed to all the instances of price-service.

Case 3: Ribbon + (RestTemaple/FeignClient) + NO Eureka Server


Can we use Netflix ribbon without integrating Eureka Server, The answer is YES but it would not be a good design. So, I would not recommend it.

When your application is not integrated with Eureka Server, in that case you have to list down all the address manually to properties file.

Remove @RibbonClients annotation.
Add below entry to your application.properties file under 
product-service.

#Enable this property if you are not using Eureka Server
price-service.ribbon.listOfServers=http://localhost:8002,http://localhost:7002

Imagine there are 1000 instances of price-services are up and running so we have to add all the instances manually. It could be a nightmare if we have to do it. :)

That's all for Netflix Ribbon now and do let me know if you have any confusion/query or you think I am not right somewhere. Please feel free to comment. Thank you!

As usual, you can download the spring-boot-microservices from GITHUB.

Sunday, August 18, 2019

Declarative REST Client: Feign


We already know that how microservices communicate with each other using RestTemplate. In this blog, we will see that how this can happen using Feign Client as well. 

What is a Feign Client?

Feign is an abstraction over REST based call. It makes writing web service clients easier. It Is as easy as creating interface and then annotate it. It has pluggable annotation support including Feign annotations and JAX-RS annotations. It also supports pluggable encoders and decoders and has supports for Spring MVC annotation.

Using Feign, microservices can easily communicate with each other and developers don't have to bother about REST internal details and can only concentrate on business logic.
Implementation.

I am going to use my previous applications to demonstrate the working of Feign Client. So before starting feign-client application. Make sure all the below three applications are already up and running. 

Feign Client Implementation

  • Create a spring boot application and add Feign starter dependency to our pom.xml file.

  • Add @EnableFeignClients annotation. With this annotation, we enable component scanning for interfaces that declare they are Feign clients. 

  • Declare a Feign client using the @FeignClient annotation.

  • Either name or url must be passed under @FeignClient annotation. If name attribute present then it will fetch the service location from service registry and then hit product-service and in case of url, it will directly hit the application to get the data from product-service. In case of both present then it will always first check for the name attribute. 
  • Use Spring Web annotations to declare the APIs that we want to reach out to.
Note: If your application is not registered with service registry then directly use url attribute to get the data but I would recommend to use name attribute and to use name attribute, make sure your application is connecting to Eureka server to get the service location. 

Read my blog to understand on how to add support for Eureka Client to your application.

Now start your application and you should be able to fetch product-service from feign-client. 


You can download the source code from here. 

Custom Configuration

Feign Client also support for Custom configuration changes like telling Feign to use OkHttpClient instead of the default one in order to support HTTP/2. 

Let’s go deep down to check how can we customize Feign Client.

There are two ways to configure it i.e. using properties file or override them using a @Configuration class, which we then need to add to the FeignClient annotation. 

Properties file:



Using @Configuration





Logging:

By default, a logger gets created for each Feign Client and to enable it, we have to declare it in the application.properties file using the package name of the client interface.

logging.level.com.com.waheedtechblog.feignclient=DEBUG

There are four logging levels to choose from:

NONE – no logging, which is the default
BASIC – log only the request method, URL, and response status
HEADERS – log the basic information together with request and response headers
FULL – log the body, headers, and metadata for both request and response

Handling Errors with Feign

By default, Feign’s default error handler, ErrorDecoder.default, always throws a FeignException.

To customize the Exception thrown, we can override it by writing our own Custom Error class and implements ErrorDecoder and declare this bean under @Configuration as we did earlier.



Spring-boot microservices can be downloaded from GITHUB

Happy Coding...!!!

Microservices: Service Registry and Discovery


In my previous blog, I have already talked about Netflix Hystrix Circuit Breaker Microservices and explained it by creating two applications i.e. Product Service and Price Service.
In this post, I’ll use both applications to explain what is the Service Registry and Discovery Server.

What is Service Registry and Discovery and why do we need it in the first place?

In our monolithic application, mostly service invoke one another through language methods or procedure calls and even in traditional distributed system deployment, services run at fixed, well-known locations (hosts and ports) and so can easily call one another using HTTP/REST. Over here the network locations of service instances are relatively static.

With the microservice architecture system, this is a much more difficult problem as service instances have dynamically assigned network locations. Service instances change dynamically because of auto-scaling, failure or upgrade. So, you can see that it’s not possible to hard code the locations (hosts and ports) of service instances in our application.

Apart from that, most of the time we have to deploy the same instance on multiple hosts (E.g. During Big billions day sale where 1000’s of instances is running to support high demand) which will be again a tedious task for the load balancer to register and deregister itself for all these services.
 

What is Service Registry and Discovery?

Service Registry is the server instance where all the service provider register itself when it starts up and deregisters itself when it leaves the system. The service instance’s registration is typically refreshed periodically using a heartbeat mechanism.

Whenever a client has to communicate with other service instances, it has to first query to Service registry to get the network location and then uses a load-balancing algorithm to select one of the available service instances and invoke a request. 

The above instance discovery pattern is called a Client-Side discovery pattern and this can be easily implemented using Netflix Eureka as a Service Registry. It provides a REST API for managing service‑instance registration and for querying available instances.

Netflix also provides Netflix Ribbon which is an IPC client that works with Eureka to load balance requests across the available service instances.


Implementation

As we already have two microservices product-service and price-service and we know product-service is internally invoking price-service to get the price detail of the products. We will do minor changes with these two applications to support service registry.

Discovery Server Implementation

  • Create a spring boot application and Eureka Server starter dependency to your pom.xml file 

  • Add @EnableEurekaServer annotation to Enable Eureka Server.
  • By default, Each Eureka Server is also a Eureka Client which needs at least one service URL to locate service registry. We have to disable this feature as we are creating single node Eureka Server. 
  • Add below properties to application.properties file.

  • Now start the discovery-server application and access http://localhost:8761 which will display the UI similar to below screenshot. 

  • As of now, no application is registered with the Eureka Server. 

2. Registering product-service and price-service to Eureka Clients

  • Let’s make out product-service and price-service as Eureka Client which should register to Eureka Server itself on starts up.
  • Add Eureka Client dependency on both pom.xml files.

  • Configure eureka.client.service-url.defaultZone property in application.properties to automatically register with the Eureka Server. 

  • Now start product-service and price service application and hit http://localhost:8761. We will see product-service and price-service is registered with SERVICE ID as PRODUCT-SERVICE and PRICE-SERVICE respectively. We can also notice the status as UP(1) which means the services are up and running and only one instance of product and price-service are running. 

3. Update hard code URL with application name in product-service and restart the application. 


4. Now open browser and hit product URL and you will get the below response.


Congratulation!!! Our application is perfectly working with Discovery Service. 

Spring-boot microservices can be downloaded from GITHUB

Happy Coding...!!!

How TOPT Works: Generating OTPs Without Internet Connection

Introduction Have you ever wondered how authentication apps like RSA Authenticator generate One-Time Passwords (OTPs) without requiring an i...