ELK (ElasticSearch Logstash and Kibana ) Installation on Windows 10

In this blog, I’ll show you how can we install ELK on our windows 10 machine - that is ElasticSearch, Logstash, and Kibana.

These three different products are most commonly used together for log analysis. Using ELK stack, we can achieve centralized logging which helps in identifying the problems. 

ELK is heavily used in microservices architecture where your docker images are running on 1000's of POD and you can't go to each pod to trace the logs.

Logstash:

It is the data collection pipeline tool. It is the first component of ELK Stack which collects data inputs and feeds it to the Elasticsearch. It collects various types of data from different sources, all at once and makes it available immediately for further use.

Elasticsearch:

It is a NoSQL database which is based on Lucene search engine and is built with RESTful APIs. It is a highly flexible and distributed search and analytics engine. Also, it provides simple deployment, maximum reliability, and easy management through horizontal scalability. It provides advanced queries to perform detailed analysis and stores all the data centrally for quick search of the documents.

Kibana:

Kibana is a data visualization tool. It is used for visualizing the Elasticsearch documents and helps the developers to have an immediate insight into it. Kibana dashboard provides various interactive diagrams, geospatial data, timelines, and graphs to visualize the complex queries done using Elasticsearch. Using Kibana you can create and save custom graphs according to your specific needs.

Download

You can download all the zip from here

• https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-windows-x86_64.zip
• https://artifacts.elastic.co/downloads/logstash/logstash-7.4.2.zip
• https://artifacts.elastic.co/downloads/kibana/kibana-7.4.2-windows-x86_64.zip

Unzip all the three folders to get their folder files.

Install ElasticSearch

• Open ElasticSearch folder and go to its bin folder.
• Run ElasticSearch to start the ElasticSearch server.

• Once started, go to browser and type localhost:9200

Install Kibana

• Open Kibana folder and go to its bin folder.
• Run kibana.bat file to start the Kibana server.

• Once started, go to browser and type localhost:5601

Install Logstash

• Logstash is a pipeline that pushes data to elasticSearch. So, before starting the Logstash. We have to create one config file.
• Logstash config file will be having three parts i.e. input, filter (Optional) & output.
• Create logstash.conf file under ${logstash}/conf folder. it will simply take the input from the console and push it to ElasticSearch.

• Run below command to start Logstash server and it will wait for the input to push it to elasticsearch.

bin\logstash -f .\config\logstash.conf

• Once started, go to browser and type localhost:9600

• To push the data, I have copied the log file of one project to the console.

• Once done, go to Kibana portal, Management-> Index Patterns -> Create Index pattern. You will observe that logstashdemo which we have set in logstash.conf file is already present here. Now define Kibana index by setting the same name and click on next step button.

• Add @timestamp to set Default time and click on Create Index Pattern.

• Index has been created. Now, to view the data, go to discover tab and click on message (All the logs will be pushed under message index)

Congratulation! We are done with the ELK Setup on Windows 10

How to create Docker Image and push java app in a Docker Engine

In this blog, I am going to share my knowledge on the creation of a docker image and how can we run in a Docker Engine.

Prerequisite

• Basic Knowledge of Docker
• Docker must be running on your machine.
• Good to aware of Spring boot application.

I already have one spring boot application in my IntelliJ which expose one endpoint /users/{id}. We will see how can we push and run this application in a docker container. 

We need to create one file named Dockerfile to add docker instruction (Check above image).

Now go to Terminal and check whether the docker is running or not on your machine.

Run docker build to create an image and push it to the container using the command.

docker build -f Dockerfile -t docker-spring-ehcache .

The above command will execute all the operations that we have mentioned in our Dockerfile like pulling OpenJDK 8 from the docker hub if not exist.

Let's see if our image got pushed to docker containers or not by listing all docker images.

docker images

Great! Our image is present in the docker container. Let's run it.

docker run -p 8070:8085 docker-spring-ehcache

Over here, we are telling the Docker to start the application and map docker container port 8085 to our local port 8070. 

Note: Make sure your application has started at 8085 in docker container else it won't be able to map it. Spring boot by default start application on port 8080 so please specify server.port-8050 in application.properties file.

Now go to your browser and hit the endpoint and see if we are getting the response from localhost:8070 or not.

Happy Coding!!!

Sonar Integration with Maven

In my previous blog, we have already seen how to setup SonarQube server on Windows 10. We have also seen that how can we generate sonar report using sonar-scanner. In this blog, I’ll show you how to generate sonar report by configuring sonar dependency to maven project. 

Steps to setup sonar in Maven

• We have to configure pluginManagement and Profile for Sonar in pom.xml file
• Add below pluginManagement dependency to your pom.xml 

<pluginManagement>

<plugins>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-compiler-plugin</artifactId>

<version>3.8.1</version>

</plugin>

<plugin>

<groupId>org.sonarsource.scanner.maven</groupId>

<artifactId>sonar-maven-plugin</artifactId>

<version>3.6.0.1398</version>

</plugin>

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.8.4</version>

</plugin>

</plugins>

</pluginManagement>

• Add Profile to pom.xml file

<profiles>

<profile>

<id>coverage</id>

<activation>

<activeByDefault>true</activeByDefault>

</activation>

<build>

<plugins>

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<executions>

<execution>

<id>prepare-agent</id>

<goals>

<goal>prepare-agent</goal>

</goals>

</execution>

<execution>

<id>report</id>

<goals>

<goal>report</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

</profile>

</profiles>

• Build the project, execute all the tests and analyze the project with SonarQube Scanner for Maven:

                 mvn clean verify sonar:sonar

• Once done, Check your SonarQube which will generate the code analysis report for your current project. My project name was api-gateway so it generated with the name api-gateway.