Wednesday, September 26, 2018

Introduction to RESTful API Modeling Language (RAML)



Introduction to RESTful API Modeling Language (RAML)

1. Overview

RAML is a YAML based language which is built on YAML1.2 and JSON for describing RESTful APIs. It provides all the information necessary to describe Restful API.
It focuses on cleanly describing resources, methods, parameters, responses, media types, and other HTTP constructs that form the basis for modern APIs that obey many, though perhaps not all, Restful constraints.

2. Requirement

Before jumping to RAML. Let’s assume we have to create one web application which is exposing CRUD Operations and couple of query parameters to access USER resources:
· POST /api/v1/users
· GET /api/v1/users
· GET /api/v1/users?username={username}
· GET /api/v1/users/{userId}
· PUT /api/v1/users/userId{}
· DELETE /api/v1/users/{userId}
All the API’s are secured via Basic Authentication and all the communication will be done over HTTPS and all the request response will be in JSON format.

3. Implementation


3.1 Adding Root level details

To start with RAML, first create a file with extension .raml and at rool level, you need to define the setting starting with RAML version.
1. #%RAML 1.0
2. title: REST Services API using Data Types
3. version: v1
4. protocols: [ HTTPS ]
5. baseUri: http://hostname/api/{version}
6. mediaType: application/json

#1 defines the RAML version
#2 Title of the application
#3 API versions
#4 HTTP or HTTPS channel
#5 URL of the application where ‘versions’ refers to a property which will be replaced with #3 version.
#6 Media type for the request/response.

3.2 Add Security

Security should also be added at the root level and as I mentioned earlier all the API’s are secured via Basic Authentication.

  1. securitySchemes:
  2. basicAuth:
  3. description: Basic Authnetication to authenticate API
  4. type: Basic Authentication
  5. describedBy:
  6. headers:
  7. Authorization:
  8. description: Used to send the Base64-encoded "username:password" credentials
  9. type: string
  10. example: Authorization NTA5NjUsInN1YiI6IkJhcmNsYXlzX1BheW1lbnRfU2Vydmlj
  11. responses:
  12. 401:
  13. description: |
  14. Unauthorized. Username/password is invalid


3.3 Add Data type

Once you are done with security then starts adding all your data type. In my case I will define it for users.
There are multiples ways to define your type:
1. You can create a new file users .raml and include the path over here (Will discuss at the end of this document).
2. Define the types after root setting using expanded syntax
3. Or you can do it via shortcut.

  1. types:
  2. Users:
  3. type: object
  4. properties:
  5. id:
  6. required: true
  7. type: integer
  8. username:
  9. required: true
  10. type: string
  11. roles:
  12. required: false
  13. type: string

Now we will see how we can define the user using shorthand.
1. types:
2. Users:
3. properties:
4. id: integer
5. username: string
6. roles?: string

? -> Adding to any property declares that the field is not mandatory.

3.4 Define Resource, method, URL parameters and query parameters


To define a resources
/users:
To add the method
/users:
get:
post:

To add the URL parameters
/users:
get:
post:
/{userId}:
get:
put:
delete:

To add the Query Parameters:
/users:
get:
description: List all users with/without filters
queryParameters:
name?: string
roles?: string
post:
/{userId}:
get:
put:
delete:

Over here, we have seen how we can declare the resources with VERB and how we can define path as well as query parameters.

3.5 Request and Response Body

In above example we have defined the users resources with POST method but have not defined the payload. Let’s see how we can add payload, response and Status Code to an API.
/users:
get:
description: List all users with/without filters
queryParameters:
name?: string
roles?: string
post:
description: create a new user
body:
application/json:
type: Users
example: { "id" : 101, "name" : "Abdul Waheed”, “roles”:”Admin” }
responses:
201:
body:
application/json:
type: Users
example: { "id" : 101, "name" : "Abdul Waheed”, “roles”:”Admin” }
500:
   body:
     application/json:
        type: Error
       example: { "message" : "Internal Server Error, "code" : 500 }

/{userId}:
get:
put:
delete:


In this example, We are performing POST operation where we are passing Users object in the form of JSON and in response we are getting STATUS code as 201 and again response type in the form of JSON.


4. Usage of Includes in RAML
In the above example, we just took one resouces but let’s assume you have to create RAML for your application where you have many resouces and each resouces have multiple consuming API.
Handling such requirement makes our .raml file more verbose and repetetive.
Using !include, we can externalize our duplicate lengthy code.
Eg: we can put the data type for a Users object in the file types/Users.raml and the type for an Error object in types/Error.raml. Then our types section would look like this:
types:
Users: !include types/Users.raml
 Error: !include types/Error.raml


5. Completing the API
After externalizing all of the data types and examples to their files, we can refactor our API using the include facility
  1. #%RAML 1.0
  2. title: REST Services API using Data Types
  3. version: v1
  4. protocols: [ HTTPS ]
  5. baseUri: http://hostname/api/{version}
  6. mediaType: application/json
  7. securitySchemes:
  8. basicAuth:
  9. description: Basic Authnetication to authenticate API
  10. type: Basic Authentication
  11. describedBy:
  12. headers:
  13. Authorization:
  14. description: Used to send the Base64-encoded "username:password" credentials
  15. type: string
  16. example: Authorization NTA5NjUsInN1YiI6IkJhcmNsYXlzX1BheW1lbnRfU2Vydmlj
  17. responses:
  18. 401:
  19. description: |
  20. Unauthorized. Username/password is invalid
  21. types:
  22. Users:
  23. type: object
  24. properties:
  25. id:
  26. required: true
  27. type: integer
  28. username:
  29. required: true
  30. type: string
  31. roles:
  32. required: false
    type: string
  33. /users:
  34. get:
  35. description: List all users with/without filters
  36. queryParameters:
  37. name?: string
  38. roles?: string
  39. response:
  40. 200:
  41. body:
  42. application:json
  43. type: Users[]
  44. example: [
  45. {“id”:1,”username”:”Abdul”}
  46. {“id”:2, “username”:”Waheed”}]
  47. post:
  48. description: create a new user
  49. body:
  50. application/json:
  51. type: Users
  52. example: { "id" : 101, "name" : "Abdul Waheed”, “roles”:”Admin” }
  53. responses:
  54. 201:
  55. body:
  56. application/json:
  57. type: Users
  58. example: { "id" : 101, "name" : "Abdul Waheed”, “roles”:”Admin” }
  59. 500:
  60.    body:
  61.      application/json:
  62.         type: Error
  63.        example: { "message" : "Internal Server Error, "code" : 500 }
  64. /{userId}:
  65. get:
  66. put:
  67. delete:

In my next blog, I ll talk about different RAML tools and the difference between SWAGGER (recently renamed to OAS) Vs RAML and which one to prefer.

Friday, June 1, 2018

Private method in Java 9



As we know till Java 7, we are not allowed to add any concrete function to the Interface, All the function should be abstract and must be implemented in Child class which is implementing the interface. i.e. an interface can only have
  • Constant variable
  • abstract method

With Java 8, we can add static and default method as well in an Interface. Check my blog on Java 8 for more details. So, an Interface now can have
  • Constant Variable
  • Abstract Method
  • Default Method
  • Static Method

and with Java 9, It become more powerful and now we can add private method and private static method.




but why do we need private function in an Interface. Let’s understand this with an example.




In above example, we can observe that all the default function has same code to create database connection (duplicate code) and fetching the data and database details is also exposed to outside the world.


So over here, Private method will come to rescue. Check below example where the database code reside in one private method which can be easily accessible to all default function in that interface and it is also hidden to the outside world.

These private methods will improve code re-usability inside interfaces and will provide choice to expose only our intended methods implementations to users. These methods are only accessible within that interface only and cannot be accessed or inherited from an interface to another interface or class.

Rules For using Private Methods in Interfaces

  • Private interface method can be static or instance and In both cases, the private method is not inherited by sub-interfaces or implementations.
  • Private interface method cannot be abstract. it will give compiler error.
  • Private method can be used only inside interface and other static and non-static interface methods.
  • Private non-static methods cannot be used inside private static methods.
  • We should use private modifier to define these methods and no lesser accessibility than private modifier.
Java 9 code can be downloaded from GITHUB

Wednesday, September 20, 2017

Access User Profile API via Google OAuth 2.0 Playground ?

The OAuth Playground is an application/tool by Google for learning how OAuth works. It presents you with a three-step process for selecting the services you want to authorize, generating an access token, and making API requests.
In OAuth terminologies, Google OAuth playground will act as a client Application which does contain client id, Client secret and OAuth Endpoints required to access Service provider.
It also supports custom endpoints as well i.e. using Google OAuth playground you can connect to another service provider as well apart from Google like Salesforce.
Resource Owner: You
Client Application: Google OAuth 2.0 Playground

Service Provider: Google
In this blog, I’ll only focus on Google API and will try to retrieve user profile via playground.





Step 2: You will see a list of scope using which you can access particular resources. As our aim is to fetch user profile so will scroll down and select https://www.googleapis.com/auth/userinfo.profile from Google People API v1 scope.



Step 3:  Click on Authorize APIs button and If you are not logged into google then it will ask for your user credentials and after successful authentication, it will show the authorization consent page. Click on Accept button.


Step 4: Now we have the Authorization Code but if you check Request/Response section then you will notice that playground application hits AUTHORIZATION ENDPOINTS

Authorization request:


We hit this request to the authorization endpoint to obtain an authorization code

https://accounts.google.com/o/oauth2/v2/auth?redirect_uri=https%3A%2F%2Fdevelopers.google.com%2Foauthplayground&prompt=consent&response_type=code&client_id=407408718192.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile&access_type=offline

where
response_type  ->  it always should be code, specifies that request is sent to the authorization endpoint to obtain an authorization code. ( Required)
client_id -> The Id issued to this application by Authorization server at the time of registration. (Required)
redirect_uri -> URI registered on the authorization server as a RedirectURI or callback URL. 
Scope -> Access scope of the request

and after successful authentication and authorization, Playground is getting the Authorization Code on its registered redirect URL.

Authorization response:


It contains the Auth code in exchange of which we will get the access token.

/oauthplayground/?code=4%2F0pGBATfaL6OoBr4qotf_W6FrCnTiCR7Bidd7BgaWenU

where 

code -> Authorization code (Required)

Step 5: To get the access token, Click on Exchange authorization code for tokens button and you will get the access and refresh token.



Check Request/Response section, Playground hits TOKEN ENDPOINTS to get the access token.

Token request:

We hit token endpoint to obtain an access token by exchanging the Auth code.

code=4%2F4by6pl_Bc22NnjpiCMvqrW4R2SL4oI7OjioWBHqWrD0&redirect_uri=https%3A%2F%2Fdevelopers.google.com%2Foauthplayground&client_id=407408718192.apps.googleusercontent.com&client_secret=************&scope=&grant_type=authorization_code

where
client_id -> The Id issued to this application by Authorization server at the time of registration. (Required)
client_secret -> The key secret issued to this application by Authorization server at the time of registration. (Required)
Grant_type -> Must be set to authorization_code. (Required)
Code -> The Authorization code received by the Authorization server.
(Required)

redirect_uri -> URI registered on the authorization server as a RedirectURI or callback URL. (Required)

Token Response:

The token response contains an access token along with other information in a JSON format.
{
  "access_token": "ya29.GlvMBFEQ8efg2id-lwfWJJIZl0_7rlCRyCKVWvv3v_cXIUlnYXVn6D04nUEIh9AFB65pdZcfms5TaDz692-2hmadFg0o6R1X7hdYUKuFwA9v2NvXqiLwOIFpUJlV",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "1/KVEh2UxFjMt5zynnbm9qajpnkDsREQTA3kldrAvcqKQ",
  "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6IjhiNmE3ZDhhM2I0NTQ4YWU1MjBmZDJkMTY2ZWEzN2U2ZGRjY2JkOWYifQ.eyJhenAiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJhdWQiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJzdWIiOiIxMDUyOTA1MTEwNTU1NTA4NjY0ODciLCJhdF9oYXNoIjoiODVIUHJYSnZiWTNmN0FTOEg2VjBYZyIsImlzcyI6Imh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbSIsImlhdCI6MTUwNTkwNzc3MCwiZXhwIjoxNTA1OTExMzcwLCJuYW1lIjoiYWJkdWwgd2FoZWVkIiwicGljdHVyZSI6Imh0dHBzOi8vbGgzLmdvb2dsZXVzZXJjb250ZW50LmNvbS8tcDRVaEFJTUNkWkUvQUFBQUFBQUFBQUkvQUFBQUFBQUFCQ0UvTy1YUnEyUFVhYW8vczk2LWMvcGhvdG8uanBnIiwiZ2l2ZW5fbmFtZSI6ImFiZHVsIiwiZmFtaWx5X25hbWUiOiJ3YWhlZWQiLCJsb2NhbGUiOiJlbi1HQiJ9.sS2Du_uR0Y-Sr2BlxBkbYSbzNfXDLy_Baj1ZbX21x-tWUHTe5R9v9ZR8S7gwfmVjAPAjHm-ivQg4aOBzQs8U9YmCei2tg1vLFTg51KsTWXK6u4WEbgHQQaPOORAfeKrZtKaUjvQgTKy007Dqdv_nzT6dt9b-vezTDLHX-fV9lx-k_-ApD9BKdrBjsqx8tAJ1cC_vg4NY_M--6ztYSf8xKG_aAF296Mq_aUakVGSl2pA5n3k0SMXyJlkIaWCilz2jQjBt_Hi_zeKdezCIU4jHTr8lstyz_SV9V2nueWV7n82K2RzS3uo-PtMDUWceNRr0r6dhWh_JxXivCqH2twh9pw"
}

Where
access_token -> token using which you can access the user details from resource server.
token_type -> define the nature of token
expire_in -> number of seconds after which the access token get expired.

refresh_token -> using refresh token, we can get new access token once the above access token is no longer valid.

Step 6: Select V2 UserInfo from list possible operations to get the user profile.


Step 7: Click on Send the request button and it will access the user profile by passing access token as authorization bearer in the headers and display it in JSON format.


Conclusion: Google playground is an amazing tool to learn OAuth flow with Google as well as with custom application. If you are planning to build your Authorization server then you can use it as a testing tool as Google playground is a standard OAuth Client web application.


If you need an idea on how can we get it done the same thing via JAVA web application then you must check my blog over here. I have done something similar where my Service provider is Salesforce instead of Google.

Friday, September 15, 2017

Registering an Application with Facebook

This guide walks you through the steps of registering an application to integrate with Facebook.
Register a new application

From http://developer.facebook.com, click on "My Apps" at the top of the page to go to the application dashboard. The dashboard shows a list of applications that the developer has already created or you can create a new one by clicking on Add a new App. 

A dialog prompts you to name your application.

Enter Display Name, Contact Email and Choose a category from drop down list and click on Create App ID. After you click, Facebook performs a Captcha check to verify that you’re not setting up applications through an automated process.

Once you’ve satisfied the verification process, your application is created. The next page you see is your application’s application page.

Click on the Settings button and it will open you a setting page of your application.

Now we can configure various details about our application. The choices you make here depend on what kind of application you plan to build and what you want your application to do. For IAM, we just need to add App Domains and Site URL. Click on + Add Platform and select Platform as Website and enter your Site URL. Site URL is shibboleth Idp URL and App Domains is the domain name of the URL.

and click on Save Changes. 

The main thing to note from the application settings page is the App ID and App Secret near the top. These values are your application’s credentials to Facebook. We need these credentials to connect to facebook via Shibboleth Idp.

and the last step, go to App Review → drag button to the left to make your application public. You will see message as "your app is currently live and available to the public"


Monday, September 4, 2017

JWT: Symmetric and Asymmetric key Implementation

Prerequisite: Understanding of JWT or read here to understand what is JSON Web token.

As we already know that JWT is special because it is digitally signed and we can verify the authenticity of JWT using signature.
Today, we will discuss on how we can actually sign this JWT using Symmetric and Asymmetric key.

Symmetric key: Symmetric key uses the same key for the signature generation as well as at the time of token verification. So, extra precaution is required during the exchange of the secret key between sender and receiver.Use symmetric key if there is one sender and one receiver, the exchanging of the key will be easy. 
Eg: One web application talking to the backend services.


Asymmetric key: It uses a key pair. The key pair consists of a public key and a private key. JSON data will be signed using the private key and can be verified using the public key.
Use Asymmetric key if you have one sender and multiple receivers as you cannot share the same key to all the end parties.
Eg: Centralized application.



There are various open source libraries using which we can create and verify the access token like Stormpath and Auth0 libraries.


Click here to download the source code from GITHUB.

Friday, August 11, 2017

Synchronization in JAVA


You should be aware of Synchronization if you are working on an application where multiple threads are accessing the same resources or else it will show you erroneous or unforeseen results.
Imagine a scenario where you are doing the multiple transactions in your bank. I believe you don’t want one thread is updating your balance and in parallel, the other thread is reading your balance otherwise you will end up in error or unexpected results.

Example 1:
Public class HDFCBank {
            Protected long balanace =1000;
            Public void deposit( int amount) {
                        this.balance = this.balance + amount;
            }
}

Assume if two threads, A and B are executing the deposit method on the same instance of the HDFCBank Class then there will be no way to guess when the operating system is switching between these two threads.

Let’s assume current balance is 1000 
1.      A & B thread reads balance as 1000.
2.      A will execute deposit(1000) and will add 1000 so total balance should be 2000
3.      B will also execute deposit (500) but over here B’s balance is 1000 (read balance before step 2) so it will add 1000 +500 = 1500

You can observe that the total balance should be 2500 but we end up with only 1500 which is not we have expected. The above code in the deposit () contains a critical section and when multiple threads execute this critical section, race conditions occur. The situation where two threads compete for the same resource, where the sequence in which the resource is accessed is significant, is called race conditions and the code section that leads to race conditions is called a critical section.

The above problem can be easily solved by ensuring that the deposit() will be used by only one thread at a time. The process by which this is achieved is called synchronization and Java provides different constructs for synchronization and locking e.g. volatile keyword, atomic variable, ReentrantLock, ReentrantReadWriteLock & Synchronized.

Today, I am only going to discuss about synchronization in details. The synchronization keyword in java creates a block of code referred to as the critical section. It is built around an internal entity known as the lock or monitor. Every object has a lock associated with it and by convention, a thread that needs consistent access to an object's fields has to acquire the object's lock before accessing them, and then release the lock when it's done with them and till then all other threads attempting to enter the locked monitor will be suspended until the first thread exits the monitor.

Let’s remodify the above example

Example 2:
public class HDFCBank {
            Protected long balanace =1000;
            public void synchronization deposit( int amount) {
                        this.balance = this.balance + amount;
            }
}

Now, what will happen if threads A & B again executing the deposit() method?
·         A & B thread will try to get the lock and let’s say A got the lock. In such case thread B will wait till the time A is not releasing the lock. A->balance = 1000;
·         Will add 1000 to balance now total balance is 2000 and exit from deposit ()
·         B thread will get the lock and NOW it will read balance value which will be 2000
·         Add 500 to balance which will be 2500 as expected.

We can use synchronized keyword at the functional level or at block level but not with any variable and also it can be synchronized on the instance (object) or on the class object.

Synchronized Instance method

The above Example 2 is an example of instance method where you need to add synchronized keyword in the method declaration. This tells Java that the method is synchronized on the instance thus only one thread can execute at a time for one object. One thread one instance.
Assume HDFCBank has two instances hdfcBank1 & hdfcBank2 and each instance has its own two threads, T1 & T2 with hdfcBank1 and T3 and T4 with hdfcBank2.

Now, if we execute T1 and T2 at the same time then there will be no interference and T2 will wait for T1 to complete the task. Similarly, with T3 & T4 but if we execute all 4 threads then there can be interference between t1 and t3 or t2 and t4 because t1 acquires another lock and t3 acquires another lock.



Synchronized static method:
Similar to Example 2 but you have to add the static declaration at the method level.
Example 3:
            public void static synchronization deposit( int amount) {
                        this.balance = this.balance + amount;
            }
Over here, the deposit methods are synchronized on the class object of the class the synchronized static method belongs and we know only one class object exists in the Java VM per class therefore only one thread can execute inside a static synchronized method in the same class.
The problem that we saw in Synchronized Instance method can be easily solved by Synchronized static method.


Synchronized at block level:
There can be a scenario where you don’t want to synchronize the whole function but only the critical section or part of a method to be synchronized to an object or you want to lock an object for any shared resources. In all such situation, you can go with synchronized block.
Example 4:
            public void deposit(int amount) {
                        System.out.println(“Depositing amount: ” + amount);
                        synchronized(this) {
                                    this.balance = this.balance + amount;
                        }
                        System.out.println(“Total balance: ” + this.balance );
            }
In Example 4, you must have observed that the synchronized block construct takes an object in parentheses (this) which is the instance the deposit method is called on. Over here, the execution is very similar to synchronized Instance methods i.e. only one thread can execute per instance.

Synchronized Blocks in Static Methods

Similar to Synchronized static method but over here you can pass the class object in the block.
Example 5:
public void static deposit(int amount) {
                        System.out.println(“Depositing amount: ” + amount);
                        synchronized(HDFCBank.class) {
                                    this.balance = this.balance + amount;
                        }
                        System.out.println(“Total balace: ” + this.balance );
            }

Points to remember while implementing synchronized keyword
·         Use synchronized wherever necessary as it can slow and degrade performance and prefer block synchronization over method synchronization as it will only lock critical section of the code, not the whole method.
·         Double check the object that you are using as monitor object in synchronized block as the uninitialized object will throw NULL POINTER EXCEPTION.
·         Thread acquires an object level lock when it enters into an instance synchronized method and a class level lock when it enters into the static synchronized method and the lock is released even if thread leaves synchronized method after completion or due to any Error or Exception.
·         Don’t use String object as a lock in java synchronized block as string is an immutable object and internally literal string gets stored in String pool. so, by any chance, if any other part of the code or any third-party library used same String as their lock then they both will be locked on the same object despite being completely unrelated which could result in unexpected behavior and bad performance.


Happy Coding…!!! 

How TOPT Works: Generating OTPs Without Internet Connection

Introduction Have you ever wondered how authentication apps like RSA Authenticator generate One-Time Passwords (OTPs) without requiring an i...