Elasticsearch Tutorial : A Complete guide for the beginners

Thursday, December 26, 2019

Elasticsearch Tutorial : A Complete guide for the beginners

Elasticsearch is a highly scalable open-source full-text search and analytics engine. It allows you to store, search and analyse big volume of data quickly and in near real time. It is generally used as the underlying engine/technology that powers applications that have complex  search features and requirements. Elasticsearch provides a distributed system on top of Lucene Standard Analyser for indexing and automatic type guessing and utilizes a JSON based REST API to refer to Lucene features.

1. Overview of Elasticsearch

Developed By -  Elasticsearch developed by Elastic NV, 8 Feb 2010.

Features - 

  1. Data Storage
  2. Flexibility, Data types, Full Text Search, 
  3. Unstructured, Document Store 
  4. Field and Document level API
  5. Cluster Indices, Data snapshots, Rollup Indices
  6. Elasticsearch SQL & Role based access control e.t.c 

Latest Version - 7.5 released on Nov 2019 


Before going to discuss about more into Elasticsearch we need to know what is SQL and NOSQL, because I felt it's more important.

SQL - SQL database are scale vertically, means we need to increase the capacity of single server (CPU, RAM) to scale your database.

NOSQL - No SQL database are scale horizontally means we can add more servers to power up your database.

3. Relational vs Non Relational Database

Into more convenient way, here you can see that in below diagram is the example of scale out and scale up. In scale up you can see we need primary key, we need secondary and foreign key. Also we have a joining concept like left outer joining, right outer joining, full joining e.t.c. If the data size getting increase if it's a single server we need to increase our ram and CPU capacity. 

Relational vs. nonrelational databases
Traditional SQL NoSQL
Primary Secondary
Scale up
Scale out
In scale out database we don't have any concept of joining. Instead of increasing the power of CPU, RAM we can add more servers to power up your database.

4. Scale up vs Scale out Database

Well this the real time example, thought to bring it up here. Though you might be thinking why I am discussing all those things this blog is all about Elasticsearch right ?

Well Elasticsearch is a no sql and scale out database. Before discussing about Elasticsearch first we need to know few basic things.

SQL vs. NoSQL wedding cake

5. What is Elasticsearch ?

Well, Elasticsearch is an open source, Restful, distributed search and analytics engine build on Apache Lucene. Since its release in 2020, Elasticsearch has quickly become the most popular search engine and its commonly used for log analytics, full text search, security intelligence, business analytics and operational intelligence use case. 

6. How does Elasticsearch works ?

Raw data flow into Elasticsearch from a variety of resources including logs, system metrics,
and web applications. Data ingestion is the process by which this raw data is parsed, normalised enriched before it is indexed in Elasticsearch. Once indexed in Elasticsearch, user can run complex queries against their data and use aggregations to retrieve complex
summaries of their data. From Kibana, users can create powerful visualisations of their data, shared dashboards and manage the Elastic Stack. 

7. Real Time Example  - Case 1 ?

In the below diagram let me explain what is happening. This the basic architecture for web application while you are dealing with huge number of data. So here in the frontend part is nothing but web browser, you can see that when user want to search anything from the browser if it is huge number of data sits into your database, then its very difficult to get the proper data and give immediate result to user. For that reason Elasticsearch came into the picture.

8. Real Time Example  - Case 2 ?

In the second scenario, if you have a huge number of data and you are going to deal with it, and bring to your frontend, that case you can bring Elasticsearch into the picture. Not only that if your data is there into Elasticsearch you can visualised your data via Kibana as  pie chart, bar chart, table e.t.c.

There are only 2 real time example I have explained, there are n no of reason & n number of challenges we will faced while we are dealing with big data. Based on that we need to decide when and where we need Elasticsearch and Kibana into the picture.

9. Popular Company's are using ?

There are popular company are using Elasticsearch, Kibana, Logstash, filebeat, there are - 

  1. CISCO
  2. SAP
  3. IBM
  10. REDHAT
  11. ADOBE
  13. BOSCH
  14. HIKE
  15. EBAY
  16. HTC
  18. AMAZON
So right now 1 questions comes in mind is it free or not ?

Yes it is free on certain limits and certain services and open sources. 

More details please watch below videos - 

10. Installation of Elasticsearch ?

For installation of Elasticsearch please visit the below link - 

More details please watch below video - 

11. Indexing the Bulk data from Mongo DB to Elasticsearch ?

Software Link -
Download Node JS - https://nodejs.org/en/download/ for MAC/Windows/Linux/UbuntuDownload Code Editor - Visual Studio - https://visualstudio.microsoft.com/

Project Link - 

git clone https://atique1224@bitbucket.org/atique1224

More details and code explanation's please watch below video

11. Elasticsearch with Node JS, Elasticsearch Aggregation, Elasticsearch GET, POST, PUT, DELETE Method's ?

Software Link -
Download Node JS - https://nodejs.org/en/download/ for MAC/Windows/Linux/UbuntuDownload Code Editor - Visual Studio - https://visualstudio.microsoft.com/

Project Link - 

git clone https://atique1224@bitbucket.org/atique1224/

More details & practical example please watch below video