Wednesday, April 22, 2020

Continuous Integration with Jenkins and Spring Boot App


Jenkins can be used for multiple purposes like whenever any developer commits any code changes to SCM, Jenkins triggers job which can Checkout the code, build it, run JUnit test case, run tools like sonar or checkmarx and if everything works properly then deploy it to some instance.

In this tutorial, I’ll share my knowledge on how can we automate our test process by introducing CI like Jenkins. We will configure Jenkins such that it should trigger it whenever any code commits by any developer to SCM, pull out the code from GIT, run maven to build and test the code.

Prerequisite:
  • Code upload on GIT (or use this link)
  • Jenkins up and running. Refer here for installation.
  Once your Jenkins is up and running then you will see below screen


  Configuration Steps:
·       Install the required plugin.
o   Make sure all the required plugins like GIT, Maven are already present
o   Goto Manage Jenkins -> manage plugins -> Installed tab


o   In case, if you don’t find the required plugin then search under the Available tab. E.g. to install the Maven Integration plugin.
o   Select plugin and click on Download now and install after restart.

o   It will take some time to download and install the plugin as per your network bandwidth.



·    
     Set Java and Maven Path
o   by default, Jenkins pick up the JAVA and Maven path running on Ubuntu instance but if you have any specific java or maven directory then you can configure those as well.
o   Goto Manage Jenkins -> Global tool configuration

·   
      Generate SSH keygen for Jenkins user
o   If you want to access a private Git repo, for example at GitHub, you need to generate an ssh key-pair. Create a SSH key with the following command.

o   The public-key must be uploaded to the service you are using, e.g., GitHub.


·       Setting up Jenkins job
o   The build of a project is handled via jobs in Jenkins. Click New Item. Afterward, enter a name for the job and select the Freestyle Project and press OK.

o   Add description of the job
o   Check Discard old builds. It will help you in cleaning the stale log. You can choose the log rotation based on the number of builds or days. Always enable and choose this option to make sure you are not running out of memory of Jenkins log.

o   Under Source code Management choose Git as we are going to checkout the code from GIT.SET git url (https://github.com/abdulwaheed18/demo.git) and no need of credential as my repo is public and set branch as master.

o   Under Build triggers, choose poll SCM so that it should run this job whenever someone commit the code to the master branch.
o   Schedule value in * which defines the time.


o   Which means pull the SCM every day of every month and every minute of every hour.
o   Select top-level maven targets under Build and set Goals as clean package.

o   Apply and save the configuration.

·       Run the build
o   Job is created as jenkins-demo.

o   Click on jenkins-demo and then Build now to run the newly created job.

·       Congratulation! Your first job is created successfully and the blue circle under Build history means there was no issue during job execution and it ran successfully.






Tuesday, April 21, 2020

Jenkins Installation on Ubuntu

Jenkins is an open source Continuous Integration server capable of orchestrating a chain of actions that help to achieve the Continuous Integration process (and not only) in an automated fashion.

Jenkins is free and is entirely written in Java. Jenkins is a widely used application around the world that has around 300k installations and growing day by day.

It is a server-based application and requires a web server like Apache Tomcat. The reason Jenkins became so popular is that of its monitoring of repeated tasks which arise during the development of a project. For example, if your team is developing a project, Jenkins will continuously test your project builds and show you the errors in early stages of your development.

By using Jenkins, software companies can accelerate their software development process, as Jenkins can automate build and test at a rapid rate. Jenkins supports the complete development lifecycle of software from building, testing, documenting the software, deploying and other stages of a software development lifecycle.

Prerequisite

  • Make sure you are logged in as a user with sudo privileges
  • Java 8 or later version up and running
    • sudo apt-get update
    • sudo apt-get install default-jdk
    • sudo java –version
  • Maven is up & running
    • Sudo apt-get install maven
    • Sudo mvn --version


Installation steps:
  • Add repository key to System
    •  wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
  • Append the Debian package repository address to the server’s sources.list:
    •  sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
  • sudo apt-get update
  • sudo apt-get install Jenkins
  • Start the Jenkins
    • sudo service jenkins start


By default, Jenkins will start on port 8080, find the IP address of your ubuntu using ifconfig command and try hitting http://ip:8080 from your browser to view the Jenkins dashboard.



Above screen is asking for Jenkins initial password which can be retrieved from initialAdminPassword file.

sudo cat /var/lib/jenkins/secrets/initialAdminPassword



From the next screen, click on install suggested plugin which will immediately begin the installation process:




Once done with the installation, you will be asked to set up the first administrative user. You can skip this step and continue as admin using the initial password we used above or we can create the Jenkins user.


Update all the fields and click on save and continue button. The next screen will be instance Configuration page that will ask to confirm the preferred URL for your Jenkins instance. Click Save and Finish.





Click Start using Jenkins to visit the main Jenkins dashboard:

















Congratulation! your Jenkins is up and running on your Ubuntu instance.

Wednesday, January 8, 2020

Setting up Lombok with SpringToolSuite and Intellij Idea

Lombok is a java library that you can plug into your editor which will automatically generate code in .class file instead of in source file.
E.g:  getters, setters toString, equals, hashcode, builder, loggers, and many others.

In this tutorial, I’ll talk about configuring it in two of the most popular IDEs- IntelliJ IDEA and Spring Tool Suite.

Check my Github repo to learn project Lombok using java source code.

Note: Step for installing the plugin for Eclipse and Spring tool Suite (STS) are the same.

Steps to configure Lombok in STS


  • Download Lombok jar from the Lombok site.
  • Double click on Lombok jar which will open below Installer wizard, Choose IDEs in which you want to install. If your not IDE is not listed then you can browse using Specify location tab.

  • Once selected, click on Install/update button and you are done.



Steps to configure Lombok in IntelliJ IDEA

  • Open IntelliJ Idea and click on File-> Settings…
  • Click on Plugin option and then search for Lombok
  • Click on the Install button on the plugin page



  • Once done with the installation, click on the restart IDE button.

Friday, November 8, 2019

Streaming Spring boot logs to ELK stack

In my previous blog, we have done ELK installation on windows 10 and we have even tried to push messages from input console to Elastic Search and finally viewed on Kibana Server.

I will write a separate blog on why do we need ELK?

In this blog, I’ll show you how can we push spring boot application log directly to Elastic search using Logstash which we can analyze on Kibana and If you don’t know how to install ELK on windows 10 then you can refer my previous blog and start Elastic Search and Kibana server.

Prerequisite


  • Elastic Search and Kibana running on your machine
  • Basic knowledge of Spring boot application


If you don’t want to start your application from scratch then you can download one spring boot application from my GitHub repository as well.

I am assuming that the Elastic Search and Kibana server are running on your machine and you have a fair idea of how to start the Logstash server and what is Logstash conf file.

So, to push spring boot logs continuously to Elastic Server, We have to open one TCP port in Logstash server and for that we have to create one Logstash config file (say elklogstash.conf) under ${LOGSTASH_HOME}/conf directory mentioning on which port TCP port should be listening under input filter and where to push the data once we received under Output filter.

For simplicity, I am skipping the filter tag as it is optional.

elklogstash.conf




Now start the Logstash server bypassing newly created conf file.
   bin\logstash -f .\config\elklogstash.conf



Cool! Now Logstash server is also up and running and if you observe the log, you will realize that it is also listening on port 4560 as mentioned in the conf file. Configure the newly created index (elkbootlogs) on Kibana as we have done during the ELK setup.

Now let's do some changes to spring boot application so that it can push all the logs to 4056 TCP port.

For this tutorial, I am using spring-logger project from my Github repository.

Add below dependency to the pom.xml file. We need Logstash encoder to encode messages.

<!-- Added for logstash Encoder-->
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>6.2</version>

</dependency>

Open logback-spring.xml file which is under the resource folder and create new appender (say elk). The task of this appender is to push logs to the destination TCP socket and under this appender, compulsory use LogstashEncoder.

<appender name="elk" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>localhost:4560</destination>
    <!-- encoder is required -->
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />

</appender>

Add new appender to root level

<!-- LOGGING everything at INFO level -->
<root level="info">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
<appender-ref ref="elk" />
</root>

Save all files and start your application. So, we are done with all the setup. Its time to check whether all the changes are done properly or not.

Open Kibana on your browser (http://localhost:5601) and select your index under the Discover tab. You will see all logs are populating on Kibana as well.



Congratulations! Our configuration is working absolutely fine and it is pushing logs to Elastic Search. 

You can download the source code from here, ELK code chnages are under elkstack branch.






How TOPT Works: Generating OTPs Without Internet Connection

Introduction Have you ever wondered how authentication apps like RSA Authenticator generate One-Time Passwords (OTPs) without requiring an i...