Setup/configure a three node Elasticsearch cluster on CentOS 7

Elasticsearch is a widely using Search Engine and it’s other use cases are log analytics, full-text search, security intelligence, business analytics etc. It’s open source, you can set it up as a cluster on your own servers. In this article, we will discuss about the basics of Elasticsearch and it’s use cases. How to setup a three node Elasticsearch cluster on CentOS servers.

Little bit history

Shay Banon is the founder of Elasticsearch. The first version of Elasticsearch was released on 2010 February. Here I am adding few words from Wiki…

While thinking about the third version of Compass he realized that it would be necessary to rewrite big parts of Compass to "create a scalable search solution". So he created "a solution built from the ground up to be distributed" and used a common interface, JSON over HTTP, suitable for programming languages other than Java as well.[6] Shay Banon released the first version of Elasticsearch in February 2010.

Since its release in 2010, Elasticsearch has quickly become the most popular search engine.

What is Elasticsearch?

Elasticsearch is an open-source, RESTful, distributed search and analytics engine built on Apache Lucene. We can use Elasticsearch in many areas to improve the performance of your infra. Apart from Search Engine, It’s a good option in analytics area. It’s a core component in RELK stack. To analyse the logs and metrics you can use the Elasticsearch cluster as the data store.

We are not discussing these thing in detail in this article. In this blog article, I will explain the steps to setup / configure a three node Elasticsearch cluster in CentOS.

Prerequisites

1, Three CentOS servers for setting up the Elasticsearch cluster. Elasticsearch cluster should have a minimum of 3 master-eligible nodes. 

2, If possible attach a separate disk for data storage.

3, Memory: Use a minimum 2 GB, the more heap available to Elasticsearch, the more memory it can use for its internal caches, but the less memory it leaves available for the operating system to use for the filesystem cache. Refer this official documentation: Setting the heap size

4, Don’t expose the Elasticsearch process to Public. Make sure you have a private network for inter node communication. For a cluster setup, nodes need to communicate each other.

5, Enable port 9200 and 9300 on all nodes for other nodes in the cluster.

6, Java: Install Java on all the servers.

That’s it. You’re all set to start setting up the three node Elasticsearch cluster.

Steps to setup three node Elasticsearch cluster on CentOS 7

Step 1: Install Java

As I mentioned in prerequisites, Elasticsearch needs Java, so we need to install Java first. To install Java on CentOS, please execute the following command:

yum install java-1.8.0-openjdk

Execute “java -version” and make sure the Java is installed correctly.

Step 2: Create a Data Directory for Elasticsearch

It’s better to attach a separate disk for Elasticsearch Data. If you have enough space on your primary disk, you can go ahead with that one. Just create a new directory and set relevant permissions to that directory.

mkdir /var/lib/elasticsearch/data
chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/data
chmod -R 775 /var/lib/elasticsearch/data

Step 3: Download the Elasticsearch RPM

curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.2.rpm

You can download the latest version from here >> Download Elasticsearch << In this page you can see all the packages, RPM, DEB etc…

Step 4: Install using RPM

rpm -i elasticsearch-6.7.2.rpm

Step 5: Start / Enable service

systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Installation part is done. Once you installed it on all three servers, you can start editing the configuration to setup the cluster using these three nodes.

The Elasticsearch configuration file is located here: /etc/elasticsearch/elasticsearch.yml

Before making changes in the configuration make sure that the port 9200 and 9300 are open between the nodes in the cluster. Add firewall rules accordingly. Try telnet / nc and make sure that the connections are okay between nodes..

Step 6: Set minimum memory for JVM

By default, the minimum memory set for JVM is 2gb, if your server has small memory size, change this value in /etc/elasticsearch/jvm.options

-Xms2g
-Xmx2g

Change the value to a minimum based on the memory available on your servers. Examples, -Xms512m or -Xms1g etc

Step 7: Set Data Directory

We already create a Directory for saving Elasticsearch Data, set that in configuration file.

path.data: /var/lib/elasticsearch/data

Step 8: Configure Elasticsearch cluster

As I mentioned, we have to make changes on this configuration file /etc/elasticsearch/elasticsearch.yml You have to make the following changes in configuration file to setup a cluster.

8.1: Stop Elasticsearch, if it’s running.

systemctl stop elasticsearch.service

8.2: On all nodes, setup a cluster name:

cluster.name: es-crybit

Open the configuration file on all the three servers and set the same name as cluster name.

8.3: Set node name for all nodes

node.name: es1

8.4: Bind an IP for Elasticsearch

By default, the Elasticsearch process listen on 0.0.0.0 we need to assign the private IP.

network.host 10.10.10.10

8.5: Set discovery by specifying all Nodes IP addresses (Add it on all nodes)

discovery.zen.ping.unicast.hosts: ["10.22.28.112", "10.22.28.113", "10.22.28.114"]

8.6: Specify the number of Master eligible nodes (Add it on all nodes)

discovery.zen.minimum_master_nodes: 2

8.7: Define Data & Master nodes

node.master: true
node.data: true

This you can add based on your requirement. I added it on all nodes.

8.8: Start Elasticsearch

systemctl start elasticsearch.service

That’s it your cluster is ready. Now you need to check the cluster health and make sure that the cluster is ready for Production use.

Run the following curl call and make sure that the cluster status is Green:

curl http://10.22.28.112:9200/_cluster/health?pretty

Yes, your cluster is ready to use now. I will create a separate article on basic commands (API calls) of Elasticsearch later.

Post navigation

Arunlal Ashok

Operations Engineer at Endurance International Group. Linux lover. Like to play on Linux console. I started this blog to share and discuss Linux thoughts.

Always happy for an open discussion! Write to arun (@) crybit (dot) com. Check about me for more details. About this blog and our strong members, check The team CryBit.com
We like to travel. Our travelogues are published at Trip Mentor

Leave a Reply

Your email address will not be published. Required fields are marked *